Limited AGI: The Hidden Constraints of Intelligence at Scale

In his recent blog post The Intelligence Age, a few days ago, Sam Altman has expressed confidence in the power of neural networks and their potential to achieve artificial general intelligence (AGI—some strong form of AI reaching a median human level of intelligence and efficiency for general tasks) given enough compute. He sums it up in 15 words: “deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.” With sufficient computational power and resources, he claims, humanity should reach superintelligence within “a few thousand days (!)”

AI will, according to Altman, keep improving with scale, and this progress could lead to remarkable advances for human life, including AI assistants performing increasingly complex tasks, improving healthcare, and accelerating scientific discoveries. Of course, achieving AGI will require us to address major challenges along the way, particularly in terms of energy resources and their management to avoid inequality and conflict over AI’s use. Once all challenges are overcome, one would hope to see a future where technology unlocks limitless possibilities—fixing climate change, colonizing space, and achieving scientific breakthroughs that are unimaginable today. While this sounds compelling, one must keep in mind how the concept of AGI and its application remains vague and problematic. Intelligence, much like compute, is inherently diverse and comes with a set of constraints, biases, and hidden costs.

Historically, breakthroughs in computing and AI have been tied to specific tasks, even when they seemed more general. For example, even something as powerful as a Turing machine, capable of computing anything theoretically, still has its practical limitations. Different physical substrates, like GPUs or specialized chips, allow for faster or more efficient computation in specific tasks, such as neural networks or large language models (LLMs). These substrates demonstrate that each form of AI is bound by its physical architecture, making some tasks easier or faster to compute.

‍

Beyond computing, this concept can be better understood in its extension to biological systems. For instance, the human brain is highly specialized for certain types of processing, like pattern recognition and language comprehension, but it is not well-suited for tasks that require high-speed arithmetic or complex simulations, in which computers excel. Reversely, biological neurons, in spite of operating much slower than their digital counterparts, achieve remarkable feats in energy efficiency and adaptability through parallel processing and evolutionary optimization. Perhaps quantum computers make for an even stronger example: while they promise enormous speedups for specific tasks like factoring large numbers or simulating molecular interactions, the idea of them being universally faster than classical computers is absolutely false. Additionally, they will also require specialized algorithms to fully leverage their potential, which may require another few decades to develop.

These examples highlight how both technological and biological forms of intelligence are fundamentally shaped by their physical substrates, each excelling in certain areas while remaining constrained in others. Whether it’s a neural network trained on GPUs or a biological brain evolved over millions of years, the underlying architecture plays a key role in determining which tasks can be efficiently solved and which remain computationally expensive or intractable.is bound by its physical architecture, making some tasks easier or faster to compute.

As we look toward the potential realization of an AGI, whatever this may formally mean—gesturing vaguely at some virtual omniscient robot overlord doing my taxes—it’s important to recognize that it will likely still be achieved in a “narrow” sense—constrained by these computational limits. Additionally, AGI, even when realized, will not represent the most efficient or intelligent form of computation; it is expected to reach only a median human level of efficiency and intelligence. While it might display general properties, it will always operate within the bounds of the physical and computational layers imposed on it. Each layer, as in the OSI picture of networking, will add further constraints, limiting the scope of the AI’s capabilities. Ultimately, the quest for AGI is not about breaking free from these constraints but finding the path of least resistance to the most efficient form of intelligent computation within these limits.

While I see Altman’s optimism about scaling deep learning as valid, one should realize that the implementation of AGI will still be shaped by physical and computational constraints. The future of AI will likely reflect these limits, functioning in a highly efficient but bounded framework. There is more to it. As Stanford computer scientist Fei-Fei Li advocates for it, embodiment, “Large World Models” and “Spatial Intelligence” are probably crucial for the next steps in human technology and may remain unresolved by a soft AGI as envisioned by Altman. Perhaps the field of artificial life too may offer tools for a more balanced and diverse approach to AGI, by incorporating the critical concepts of open-endedness, polycomputing, hybrid and unconventional substrates, precariousness, mutually beneficial interactions between many organisms and their environments, as well as the self-sustaining sets of processes defining life itself. This holistic view could enrich our understanding of intelligence, extending beyond the purely computational and human-based to include the richness of embodied and emergent intelligence as it could be.

Originally posted on September 26, 2024