Note: The article was first published on TechNode China written by Evan Huang and translated by Zinan Zhang.
In the dynamic AI era, venture capital is increasingly attuned to the transformative potential of this technology. As generative AI advances in creating text, images, and videos, a plethora of opportunities and challenges are emerging. This article explores the pivotal role of the Scaling Law, the emergence of super apps, and the promising future of AI-driven innovations. Highlighting insights from industry leaders, it underscores the potential for AI to revolutionize various sectors and entrepreneurial ventures, providing valuable directions for future venture capital investments.
The utility of the Scaling Law
The training and inference stages of large models demand substantial computational resources. The Scaling Law suggests that significant advancements in intelligence are achieved through consistent investment in vast amounts of data and powerful computing, provided the algorithmic architecture remains stable.
OpenAI, a strong proponent of the Scaling Law, has showcased the potential of generative AI across various fields by leveraging transformer architecture, extensive training data, and considerable computational resources.
Recently, Kevin Scott, Microsoft CTO, mentioned in an interview with Pat Grady and Bill Coughran of Sequoia Capital that they have yet to observe diminishing returns from scaling. He announced that the next generation of OpenAI models would soon be available, offering cheaper, more powerful solutions capable of tackling more complex problems. “This is the story with each generation of models as we scale up,” he remarked.
On May 18, Yang Zhilin, founder of Moonshot AI, discussed the computational aspects of the Scaling Law. He noted that initial improvements in model performance are driven by enhanced computational power and efficiency. However, further advancements require increased computational investment and ensuring that this investment effectively translates into intelligence. “This involves two issues: sustaining computational investment and maximizing the intelligence output of each computation unit,” he explained.
In an interview with TechNode, Wu Yunsheng, vice-president of Tencent Cloud, shared his perspective. “Currently, there are different viewpoints, including realistic and idealistic views. Some believe the Scaling Law has reached a plateau, where continued investment yields diminishing returns. Others argue it is still in a phase of rapid development.” He emphasized that the Scaling Law remains significant, citing rapid progress in multimodal research over the past year. “In this field, various capabilities improve significantly with added data or computing power. We will continue to explore and observe its development and changes across different scenarios and technologies,” he added.
The super app is on the way
As of March 28, 2024, there are 117 large models registered with the Cyberspace Administration of China, including Baidu’s ERNIE Bot, Alibaba’s Tongyi Qianwen, and the open-source ChatGLM. The rapid development of AI large models is becoming a key driver of innovation and breakthroughs in super applications.
As these large model technologies mature and improve, they are gradually permeating various industries, sparking a range of entrepreneurial opportunities. From healthcare to fintech, from smart manufacturing to cultural creativity, the application potential of AI is limitless.
Zhou Zhifeng, Managing Partner of Qiming Venture Partners, pointed out at the World Artificial Intelligence Conference in Shanghai that compared to the timeline of application deployment during the internet wave, he predicts that the explosion of applications in the current AI wave will occur significantly earlier. Currently, generative AI is gaining substantial user favor in three “C fields” — Copilot, Creativity, and Companionship — showing a development trajectory similar to internet applications and transitioning from efficiency-enhancing applications to those aimed at providing enjoyment. He noted that the internet reduced the marginal cost of information distribution to almost zero, while the core of generative AI is to reduce the marginal cost of digital content creation to nearly zero, indicating that AI technology is bound to release enormous value.
When discussing the future of AI-driven super apps, Zhang Fan, COO of Zhipu AI, expressed optimism, arguing that although creating super apps is not easy, the AI era will see many unimaginable applications emerge. This process requires advancements in computing power, networks, hardware levels, and user habits, following the principle of gradual development from small-scale applications. Zhang emphasizes that by embracing and utilizing existing AI technologies to gradually transform current applications and products, the future will undoubtedly usher in super apps in the AI era.
Regarding the challenges of implementing generative AI applications, Zhou Zhifeng believes that reducing the cost of model usage necessary for the widespread adoption of generative AI, improving the effectiveness of large models, and enhancing user retention rates of generative AI applications are crucial. Since the growth period from zero to one for generative AI application companies is longer than in other fields, they need to overcome both TPF (Technology-Product Fit) and PMF (Product-Market Fit) challenges simultaneously. Therefore, the founding team needs greater patience, determination, and understanding of the technology, the product, and the world.
Embodied intelligence, infinite imagination
There were 45 intelligent robots, including 25 humanoid robots, showcased at WAIC this year. A video of a humanoid robot walking on the Great Wall was repeatedly played at the event. The humanoid robot L2 in the video has successfully conquered the steep slopes of the famous structure, achieving steady walking on it.
At the recent Huawei Developer Conference 2024, Zhang Ping’an, Executive Director and CEO of Huawei Cloud, unveiled the Pangu Model 5.0. During the introduction of the Pangu model for embodied AI, he showcased the broad potential of the KUAVO humanoid robot, equipped with the Pangu model, in both industrial and household scenarios, attracting widespread attention.
Chen Jianyu, an assistant professor at Tsinghua University and founder of the humanoid robot company Robot Era, believes that humanoid robots will be the ultimate form of general-purpose robots. This is not only because the pure humanoid form with two legs and two arms is more compatible with existing environments, but also because it’s easier to transfer training data from the human world. Technically, an end-to-end integration of the brain and cerebellum will be a crucial research direction in the future. Using human language as the interface between the brain and cerebellum is limited, and it is better to borrow from the end-to-end joint training process of autonomous driving, where physical layer data is directly fed back to the text and image models, significantly enhancing overall model performance.
Last week, Tencent, in collaboration with Shanghai Jiao Tong University, released the Top Ten Trends of Large Models 2024: Entering the Era of ‘Machine External Brain’ report, which pointed out that the combination of robot technology and large models provides a “body” for the machine’s external brain. In the future, humanoid robots will not only be able to perform physical tasks but also interact with humans more naturally and intuitively, endowing physical products with intelligent “brains”.
The report states that the development of humanoid robots relies on two major technical pillars: motion control and task training. The application of large models has greatly improved the robots’ learning efficiency and ability to execute complex tasks. The integration of these technologies not only drives technological innovation in humanoid robots but also opens possibilities for their widespread deployment in practical applications. This also heralds a future of human-machine symbiosis, where humanoid robots will play increasingly important roles in various industries, from household services to high-risk industrial operations, showcasing their efficiency and safety. Through continuous technological innovation and application expansion, humanoid robots will play a key role in improving the quality of life and work efficiency, further integrating into human daily life as indispensable assistants and the ultimate carriers of artificial intelligence.
Conclusion
In conclusion, the era of AI is not just a technological revolution but a transformative force that is redefining the landscape of innovation and investment. As we look to the future, the challenges of implementing generative AI applications remain significant. The need to reduce costs, improve effectiveness, and enhance user retention rates is crucial for the widespread adoption of these technologies. However, the potential rewards are immense, offering a glimpse into a world where AI is not just a tool but an integral part of our daily lives, from household services to high-risk industrial operations.
In summary, the dynamic AI era presents a wealth of opportunities for venture capital and entrepreneurial ventures. As we continue to explore and invest in AI-driven innovations, the future holds huge promise for transforming industries, enhancing human-machine interactions, and ultimately, improving the quality of life for all.