Meta’s Open Source Llama Upsets the AI Horse Race
In May an anonymous memo apparently written by a Google researcher concerned about the company’s future leaked online. It argued that, while executives squabbled about the competitive threat of text-generation technology from OpenAI, open source software was “quietly eating our lunch.”
As proof, the memo cited Llama, a large language model made by Meta that was initially available only to researchers by invitation but within days leaked on 4Chan, and quickly became popular with programmers who adapted and built on the project. Within weeks of its release, variants called Alpaca and Vicuna were nearly as good as ChatGPT but agile enough to customize on a laptop computer. “The impact on the community cannot be overstated,” the leaked Google memo said. “Suddenly anyone is able to experiment.”
Last week, Meta released the second version of its unexpectedly popular model, Llama 2. This time, it is open source and free for commercial use from the start. The new version was made using 40 percent more data than the original, and a chatbot built with the model is capable of generating results on par with OpenAI’s ChatGPT, Meta claims.
Just like ChatGPT, Google’s Bard, and other generative AI models released recently, Llama 2 likely cost millions to create. But only Meta’s system is available for free to developers, startups, and others interested in creating custom variations of the model. By supplying a cheaper option, Meta’s Llama 2 makes it easier for small companies or lone coders to create new products and services, potentially accelerating the current AI boom.
Meta isn’t offering up Llama 2 alone. It has support with some major partners that are already making the model available to their customers, including AI startups Hugging Face, Databricks, and OctoML.
Microsoft, which has invested $10 billion in OpenAI, will nonetheless also offer Llama 2 downloads to developers for use in the cloud or on Windows. At a conference for Microsoft customers last week, CEO Satya Nadella talked excitedly about developers being able to use Meta’s open source AI alongside the proprietary offerings of OpenAI. Amazon’s cloud division, AWS, also offers access to Llama 2.
Ahmad Al-Dahle, Meta’s vice president for generative AI, declines to say what role the leak of the first Llama model played in the company’s new strategy for Llama 2.
“If you look back at Meta’s history, we've been a huge proponent of open source,” he says, pointing to the example of PyTorch, a popular tool for developers working with machine learning. “One of the major motivations for building a community around this was that we saw there was demand beyond researchers to work on these models and improve them.” Al-Dahle says work is already underway on the development of Llama 3, but he would not specify how it will be different.
Though Llama 2 lends credibility to Meta as a leader in open source AI, not all aspects of the release can be characterized as open. The training data used to create the model is described in release materials only as “publicly available online sources,” and the company won’t offer further details about what went into the model’s creation.
Source: WIRED
