Meta shook up the race to build more powerful artificial intelligence last July by releasing Llama 2, an AI model similar to the one behind ChatGPT, for anyone to download and use. In November, a little-known startup from Beijing, 01.AI, released its own open source model that outperforms Llama 2 and scores near the top of many leaderboards used to compare the power of AI models.
Within a few days of its release 01.AI’s model, Yi-34B, rocketed to the top spot on a ranking maintained by startup Hugging Face, which compares the abilities of AI language models across various standard benchmarks for automated intelligence. A few months on, modified versions of 01.AI’s model consistently score among the top models available to developers and companies on the Hugging Face list and other leaderboards. On Monday, the startup launched a “multimodal” AI model called Yi-VL-34B that can process images and discuss their contents.
OpenAI, Google, and most other AI companies tightly control their technology, but 01.AI is giving away its AI models in hopes of inspiring a loyal developer base that helps it hatch some killer AI apps. 01.AI, founded in June of last year, has raised $200 million in investment from Chinese ecommerce giant Alibaba and others and is valued at over $1 billion, according to Pitchbook.
The startup’s founder and CEO is Kai-Fu Lee, a prominent investor who did pioneering artificial intelligence research before founding Microsoft’s Beijing lab and then leading Google’s Chinese business until 2009, a year before the company largely pulled out of the country. He says the creation of Yi-34B is the culmination of his life’s work trying to build more intelligent machines.
“This has been the vision of my whole career,” Lee says over Zoom from a handsomely decorated apartment in Beijing. “It’s been too long that we’ve had to learn computers’ language—we really need systems that can understand our language, which is speech and text.” In Chinese 01.AI is known as 零一万五, Ling-Yi Wan-Wu in Chinese, which means “zero-one, everything” and alludes to a passage from the Taoist text Tao Te Ching.
01.AI is one of China’s leading contenders in the AI race that was started by OpenAI and ChatGPT and has so far been dominated by US companies. Lee says his company aims to lead the next phase of this revolution by building some of the first “killer apps” built on the capabilities of language models, which earn 01.AI healthy revenues. “The apps that won the mobile era are ones that are mobile-first, like Uber, WeChat, Instagram, TikTok,” Lee says. “The next-gen productivity tools shouldn’t look like Office anymore—Word, Excel, PowerPoint—that’s the wrong way to go.”
01.AI’s engineers are experimenting with different “AI-first” apps, Lee says, for office productivity, creativity, and social media. He says the plan is for them to become successful around the globe, in a similar way to how Chinese-backed social network TikTok and online retailer Temu are top apps with US consumers.
None of 01.AI’s apps have launched, but the startup’s open source language model has already won admirers in the West. “For many things, it’s the best model we have, even compared to 70-billion-parameter ones,” which might be expected to be twice as capable, says Jerermy Howard, an AI expert who recently founded Answer AI, another new venture that will do both AI research and AI app development.
Lee has had a notable career in AI. After emigrating from Taiwan to the United States and attending high school in Oak Ridge, Tennessee, he studied computer science at Columbia and Carnegie Mellon universities, receiving his PhD for a thesis involving the development of a speech recognition system that was cutting edge for the time.
Lee joined Apple as a research scientist in 1990, moved to Silicon Graphics in 1996, then returned to China in 1998 to help establish Microsoft Research Asia—a now legendary Beijing lab that helped train countless prominent Chinese engineers and executives. In 2005, he became president of Google’s search business in China, leaving in 2009 to start his own investment firm, Sinovation Ventures, in China’s now thriving tech industry.
As the rise of the smartphone in China drove rapid growth in tech, Sinovation backed a number of successful Chinese AI startups, including Megvii, an image recognition firm, and TuSimple, a company working on autonomous trucking. Lee became a champion of the Chinese AI industry, traveling to the US to encourage Chinese grad students to consider returning home to build AI projects, and in 2018 publishing AI Superpowers, in which he argued that Chinese AI labs and companies would soon rival those in the US thanks to the country’s abundance of talent, data, and users. But Lee also frequently advocated for collaboration between the US and China.
The publication of AI Superpowers coincided with a growing realization in the West that Lee appeared to be correct that China’s tech industry was on track to rival—and perhaps even eclipse—that of the United States. Policymakers and pundits in Washington began talking about China’s goal of challenging US hegemony across the world, and talking up the risks that might bring.
That has posed challenges for anyone trying to build bridges between China and the US. In 2019, Sinovation Ventures shut down its office in Silicon Valley, citing the growing challenges involved in doing deals with US firms. In October of that year, the US government took direct action against China’s AI industry when it imposed sanctions on Megvii over government use of the company’s face recognition technology.
With the release of 01.AI’s open source Yi-34B AI model, Lee is back to building bridges. A few months after Yi-34B was released, modified versions began appearing from developers in the West and exceeding its performance on the Hugging Face model leaderboard. Some US and European countries are building their AI strategies on the Chinese model, which is proficient in both Mandarin and English.
“It’s a really good model that a lot of people are building on,” said Clément Delangue, CEO of HuggingFace, at a briefing in November shortly after 01.AI’s model was released.
Delange said that open source language models are improving rapidly and can be better than OpenAI’s market-leading GPT-4 for some specialized tasks. But he noted that many of the best open source models have come from outside the US, saying that 01.AI could be positioned to benefit from innovations that spring up around its model. “US companies have become a little bit less open and transparent,” he said at the briefing. “But there’s this interesting dynamic with AI where the more a company releases open source, the more the ecosystem develops, and so the stronger they become at building AI.”
Meta’s Llama 2 is a rare example of a top open source model from a US company and is the social media giant’s challenge to OpenAI, Microsoft, Google, and other major tech rivals investing heavily in generative AI. Meta chose to release its AI language model under a license that allows commercial reuse, with some caveats.
Yi-34B and Llama 2 appear to have more in common than just being leading open source AI models. Not long after the Chinese model was released, some developers noticed that 01.AI’s code had previously included mentions of Meta’s model that were later removed. Richard Lin, 01.AI’s head of open source, later said that the company would revert the changes, and the company has credited Llama 2 for part of the architecture for Yi-34B. Like all leading language models, 01.AI’s is based on the “transformer” architecture first developed by Google researchers in 2017, and the Chinese company derived that component from Llama 2. Anita Huang, a spokeswoman for 01.AI, says a legal expert consulted by the company said that Yi-34B is not subject to Llama 2’s license. Meta did not respond to a request for comment.
Whatever the extent to which Yi-34B borrows from Llama 2, the Chinese model functions very differently because of the data it has been fed. “Yi shares Llama’s architecture but its training is completely different—and significantly better,” says Eric Hartford, an AI researcher at Abacus.AI who follows open source AI projects. “They are completely different.”
The connection with Meta’s Llama 2 is an example of how despite Lee’s confidence in China’s AI expertise it is currently following America’s lead in generative AI. Jeffrey Ding, an assistant professor at George Washington University who studies China’s AI scene, says that although Chinese researchers have released dozens of large language models, the industry as a whole still lags behind the US.
“Western companies gained a significant advantage in large language model development because they could leverage public releases to test out issues, get user feedback, and build interest around new models,” he says. Ding and others have argued that Chinese AI companies face stronger regulatory and economic headwinds than their US counterparts.
Speaking at the World Economic Forum in Davos last week, Lee argued—perhaps hoping the message would travel back home—that the open approach would be crucial for any country to take full advantage of AI.
“One of the issues with one or a few companies having all the most power and dominating the models is that it creates tremendous inequality, and not just with people who are less wealthy and less wealthy countries, but also professor researchers, students, entrepreneurs, hobbyists,” Lee said. “If there were not open source, what would they do to learn; because they might be the next creator, inventor, or developer of applications.”
If he’s right, 01.AI’s technology—and applications built on top of it—will put Chinese technology at the heart of the next phase of the tech industry.
Source : Wired
