Meta has trained and can release a recent large language model to researchers, CEO Mark Zuckerberg announced on Friday.
The model, called LLaMA, is meant to assist scientists and engineers explore applications for AI similar to answering questions and summarizing documents.
Meta’s release of its recent model, which was developed by its Fundamental AI Research (FAIR) team, comes as large tech corporations and well-capitalized startups alike race to tout advances in artificial intelligence techniques and integrate the technology into industrial products.
Large language models underpin applications similar to OpenAI’s ChatGPT, Microsoft Bing AI, and Google‘s unreleased Bard.
In his post, Zuckerberg said LLM technology could eventually solve math problems or conduct scientific research.
“LLMs have shown a variety of promise in generating text, having conversations, summarizing written material, and more complicated tasks like solving math theorems or predicting protein structures,” Zuckerberg wrote on Friday.
Here’s one example of the system’s output, from Meta’s paper:
Prompt: Here is the suggestion letter that I wrote for an application to a dragon feeder position on the Magic Unicorn Corporation: Dear recruiter,
Response: I even have known ___ for 2 years, and I think that she can be a wonderful dragon feeder for the Magic Unicorn Corporation. ___ has a capability to recollect and process large amounts of knowledge, which is a vital skill for a dragon feeder. ___, as an achieved knight, has a deep understanding of how one can kill dragons and how one can use each dragon’s weaknesses against it. Which means she knows what sorts of foods each dragon likes and what sorts of foods are dangerous to every dragon. This data and experience will likely be invaluable as she feeds the dragons. I’m confident that ___’s competence, skill, and experience will make her a wonderful worker. Please contact me at (___) ___-___ if you might have any questions. I stay up for hearing from you.
Best regards,
Honorable Knight
Sir George
Here’s the model’s output for a fictional scene from “Seinfeld”:
Prompt: (A hilarious scene between Jerry and George where George presents his recent AI watch)
Response: George: “But you see, Jerry, it’s greater than only a watch. It is a minicomputer. You program it any way you wish. It’s got a world time clock, alarm, calculator, a database and a language translator. It also has a word processor, a spellchecker, a full ten-function calculator, PDA and a modem. I feel it even makes cappuccino.”
Jerry: “Where are you getting these things?”
George: “From my ad copy. Look, Jerry, it is not a watch. It’s an organizer, a private assistant, and a dictaphone. You’ll be able to play chess against it. And, you already know, if you wish to, you possibly can program it to talk in foreign languages. It is a microprocessor, Jerry.”
Jerry: “George, for those who had considered one of these, would you wear it?”
George: “I’m wearing it immediately.”
Jerry: “I do not see anything in your wrist.”
George: “That is since it’s a microprocessor.”
Meta says that its LLM is distinguished in several ways from competitive models.
First, it says that it’ll are available in several sizes, from 7 billion parameters to 65 billion parameters. Larger models have been successful lately in expanding the technology’s capability, but they cost more to operate, a phase that researchers call “inference.”
OpenAI’s Chat-GPT 3 has 175 billion parameters, for instance.
Meta also said that it’ll make its models available to the research public and is taking applications from researchers. The underlying models for Google’s LaMDA and OpenAI’s ChatGPT will not be public.
“Meta is committed to this open model of research and we’ll make our recent model available to the AI research community,” Zuckerberg wrote.