On Thursday morning, Meta released its latest artificial intelligence model, Llama 3, touting it as the most powerful to be made open source so that anyone can use it. The same afternoon, Yann LeCun, Meta’s chief AI scientist, said an even more powerful successor to Llama is in the works. He suggested it could potentially outshine the world’s best closed AI models, including OpenAI’s GPT-4 and Google’s Gemini.
Meta released two versions of Llama 3 today, one with 8 billion parameters—an industry term that roughly conveys a model’s power—and another with 70 billion parameters. LeCun said that bigger models are in the works and that the most powerful, with more than 400 billion parameters, is currently in training.
“It takes a lot of time to fine-tune, but a bunch of variations of these models are going to come out in the next few months,” LeCun said at Imagination in Action, a conference at MIT focused on generative AI. It’s unclear when the model with more than 400 billion parameters might be released.
Meta on Tuesday launched a new assistant called Meta AI based on Llama 3 and says its testing shows that the models are better than previous open source models with similar numbers of parameters. Although most of the biggest AI developers like Google and OpenAI keep their technology closed, LeCun predicted that open source AI models will advance more rapidly. In theory, he said, they’ll push AI toward human-level intelligence more quickly.
LeCun argued that looking at the wider software industry shows that open source technology ultimately wins out. The open approach has become predominant in software infrastructure, he said, because it allows improvements to be shared more rapidly and code to be scrutinized more comprehensively. “AI is better when more people look at the code,” he said. “Infrastructure needs to be open source—it just progresses faster.”
Meta’s open source AI strategy has helped accelerate the current frenzy around generative AI. When the company released its Llama 2 model in July 2023 it provided many startups, researchers, and entrepreneurs access to much more powerful AI models to download to experiment with and build upon.
OpenAI’s GPT-4 and Google’s Gemini are available only through paid APIs, although Google made a smaller model called Gemma available to download in February. A handful of other companies have opted to join Meta in releasing powerful open source AI models. Just a couple of weeks ago, a new model from startup Databricks was the most powerful open variant available. Llama 3 appears to be more capable, if benchmarking scores reported by Meta are correct.
LeCun also acknowledged on Tuesday that Meta’s strategy involves giving away AI models that cost Meta dearly in research and computing resources to develop. Mark Zuckerberg, Meta’s founder and CEO, told CNBC in January that Meta is spending billions of dollars in 2024 to acquire the GPUs needed to train machine learning algorithms. That same month he declared that Meta’s ultimate goal—like that professed by OpenAI and Google—is to develope so-called artificial general intelligence, machines that can do anything and everything that a human intellect can.
Zuckerberg took to Instagram today to explain that Meta would incorporate the new Meta AI assistant, powered by Llama 3, into products that include Whatsapp, Instagram, Facebook, and Messenger.
Meta said in its blog post announcing Llama 3 that it had focused heavily on improving the training data used to develop the model. It was fed seven times as much data as its predecessor, Llama 2, the company said. Some AI experts noted that figures released by Meta also showed that creating Llama 3 required huge amounts of energy to power the servers required.
The growing capabilities of open source AI models have spurred some experts to worry that they could make it easier to develop cyber, chemical, or biological weapons—or even become hostile toward humans. Meta has released tools that it says can help ensure Llama does not output potentially harmful utterances.
Others in the field of AI say that Meta’s Llama models are not as open as they could be. The company’s open source license on the models places some restrictions on what researchers and developers can build.
“It’s great to see more and more models openly releasing their weights,” said Luca Soldaini, senior applied research scientist at Allen Institute for AI, a nonprofit lab, n a statement after Llama 3’s release. “But the open community needs access to all other parts of the AI pipeline—its data, training, logs, code, and evaluations. This is what will ultimately accelerate our collective understanding of these models.”
Stella Biderman, an AI researcher involved with EleutherAI, a nonprofit open source AI project, says Meta’s license for Llama 2 limited the experiments that AI researchers can run with it, and adds that the Llama 3 license looks even more restrictive. “Meta releases weights but is famously restrictive about what you can do with them,” Biderman says.
One part of the model’s license says that companies with “greater than 700 million monthly active users” must seek a special license from Meta—a clause apparently designed to prevent the project from helping the company’s closest rivals.
Even so, Llama 3 seems likely to spark a new burst of AI experimentation. Clement Delange, CEO of HuggingFace, a repository for open AI models, including Llama 3, says developers created more than 30,000 variants of Llama 2. “I’m sure we’ll see a flurry of new models based on Llama 3 as well,” he says. “Awesome community move by Meta.”
Source : Wired
