The race for AGI is on
With the release of ChatGPT late last year, Sam Altman, the co-founder and CEO at OpenAI, recklessly fired the starting gun in what has since become a global race for Artificial General Intelligence (AGI). Soon after that, Anthropic, a splinter group of OpenAI researchers who left due to concerns over misalignment, launched Claude, the first credible rival to ChatGPT. These events awoke the leadership at Google, who quickly launched Bard before pivoting the entire company to focus on AI by merging the once independent Deepmind with Google Brain, with the stated goal of beating OpenAI to the finish line1.
Meanwhile, the pace of the race is accelerating as the capabilities of AI models are improving at a mind-boggling double exponential rate due to continuing advances in both the hardware and software on which AI systems are built. As models improve, new capabilities, known as emergent properties, will continue to arise unpredictably2. These trends will only intensify as more and more mindshare and capital shift into AI, adding more players to the race. Human-level AI is on the horizon, now estimated within the next three years — decades sooner than many experts predicted less than one year ago3. The first application of this tool will be towards the development of the ultimate goal for all players, Super-Intelligent AGI. Some researchers predict that AGI may follow within as little as days or weeks of attaining Human-level AI, due its ability to recursively self-improve, as AGI is used to create better AGI4.
Like most of us, I was unaware of how powerful AI had become and how quickly it was improving. ChatGPT served as a personal wake-up call for me too. As a life-long AI skeptic who has dedicated the last five years to scaling Web3, I had a lot of catching up to do. While firmly grounded in the liberal arts, I am a self-taught programmer and consensus researcher. So I began replicating that journey, now into the field of AI, learning everything on the subject from all angles. Despite the numerous potential benefits of AI, the more I understand, the more deeply concerned I become.
But alignment remains unsolved
The critical problem with the ongoing race towards super-intelligence is that we still don’t know how to align AI systems like ChatGPT properly5. While their capabilities are improving with astonishing speed, our ability to safely control them has not kept pace.
The Alignment Problem: The task of ensuring that artificial intelligence systems, particularly powerful ones, always act in ways that are beneficial to human beings and consistent with human values.
The existential consequences of failing to solve the alignment problem before a company like OpenAI achieves AGI include6:
Extinction Threat: misaligned super-intelligence may destroy the human race intentionally (an engineered virus) or accidentally (a paperclip maximizer).
Enslavement Threat: the all-powerful nature of AGI suggests that the first group to unlock and control it could very well use it to control the rest of humanity.
Replacement Threat: even human-level AGI will cause massive job replacement for most of humanity and a meaningless future dominated by existential nihilism.
If the race for AGI continues in its current form, we will witness some combination of the above scenarios play out soon. These risks are why so many influential figures from the AI research community have become outspoken about the dangers of misaligned AI7. As a result, many well-meaning solutions have been proposed, centering on the idea of slowing down, pausing, or refereeing the race, through some form of regulation, so that we may have more time to solve the alignment problem. While these proposals have helped raise public awareness on the the risks of AI, they have yet to make a meaningful impact on the pace of the race.
Calls for self-regulation, such as the six-month pause on developing new models, have been largely ignored by the key players, as they are fundamentally at odds with the Moloch dynamics of the race itself.8 Increasingly louder calls for government regulation often compare AI to weapons of mass destruction, and argue for a similar regulatory framework, under the assumption that large models are the only path to AGI. A better analogy would be the printing press, another well-understood and easy to replicate technology which served as means of mass disruption for both good and bad actors.
In a similar fashion, open-source models are already proving impossible to control and regulate9, while becoming smaller and more capable by the day10. The next breakthrough in AGI could very well come from a lone programmer, as foreshadowed by the BabyAGI experiment11. Try as we might, we cannot put the AI genie back into the bottle — the race will go on. This is not to say that we don’t need any regulation, but that it should be appropriate to the unique nature of the technology12. Regulation should also serve the interests of all humanity, not just a handful of companies13.
Unpacking the alignment problem
Alignment is seen as the hardest open problem in AI, for which there is still no commonly accepted solution. Many factors influence the ultimate behavior of AI systems, including the content of the data it is trained on, the objective function it tries to maximize for, how its outputs are scored, and even how a user interacts with it through prompting.
But even if we can solve the technical means of alignment, the more significant question looms of what human ends these systems should be aligned towards. In other words, beneficial for which human beings and consistent with what human values. More importantly, who gets to make these decisions, ultimately control the AI, and reap the resulting rewards? It follows that AI alignment is as much a social, political, and economic problem as it is a technical one. These aspects may explain why so many alignment researchers, steeped in the clean language of mathematics, have struggled with the messy business of values central to alignment.
Today, practitioners almost exclusively apply a brute-force, top-down solution to the alignment problem. Take ChatGPT as an example14. First, it is pre-trained on the entire Internet, including the good and bad parts. The resulting model will then mimic the lowest common denominator of human behavior. The model is “aligned” using Reinforcement Learning with Human Feedback (RLHF) to filter out the bad parts. This process consists of deciding on a set of supposed group values for what is right and wrong and then paying humans, primarily overseas contractors with a different value system, to rank the model’s outputs according to these values15.
We need Humaic Intelligence (HI)
To solve the alignment problem, we need a new paradigm for building AI that is fundamentally different from systems like ChatGPT. We need to start from the bottom-up, rather than from the top down. We need AI that is built by the global community, not by a single company that is only open in name. To do this, we will need a regulatory framework that ensures AI is fully-open source by law, rather than one which protects the incumbent players. We also need to rethink the purpose of AI, such that it is human-centric, not machine-centric, allowing it to become a tool for empowerment, not replacement. This goes far beyond treating AI as a simple co-pilot, but as an extension of the human user, tailored to each individual. Finally, we need AI that is not only safe in the narrow technical sense but ultimately beneficial for all humans, not just for companies like OpenAI, who control and monetize it.
We need Humaic Intelligence.
humaic [hue-may-ick]
adjective
A portmanteau of human and AI, which describes any system that combines the best qualities of each, in a symbiotic fashion.
Humaic Intelligence (HI) is a human-centric approach to building human-aligned AI systems which empower, extend, and enhance individual human beings. Many humaic systems have already been proposed and built. Peter Thiel’s Palantir emerged from early attempts at humaic AI to fight online fraud at Paypal16. Elon Musk started Neuralink to build humaic hardware. PersonalAI and InflectionAI are building humaic chatbots that can already be used today. MercuryOS is a humaic operating system. On the other hand, ChatGPT, alongside nearly all popular LLMs made today, are not humaic. Instead, they represent a stepping stone towards AGI, which lies at the opposite end of the spectrum from HI.
HI is based on humaism, a moral philosophy for how we should design, employ, and control artificial intelligence. Humaism revolves around three core principles:
Humanistic: it is rooted within the humanities, not science or engineering. It treats AI as a means for improving the human condition, not as an end in itself.
Human-Centric: it begins by aligning AI to individual humans, not the entire society. Group alignment may then emerge naturally through social discourse.
Human-Right: it treats access to AI as a basic human right, not as a product to be bought or sold. We need Universal HI, not Universal Basic Income (UBI).
Humaism is a reaction to the current practice of pursuing AI research for its own sake, as exemplified by the mission statement of OpenAI to create AGI. It provides an alternative path that resolves the existential threats to human existence, agency, and relevance presented by AGI. Humaism is inspired by Renaissance Humanism, humanistic philosophy, and the humanities. It prioritizes all human beings’ continued value, worth, and dignity. Humaism seeks to place AI firmly under human control. It sees individually tailored HI as a tool for human empowerment and self-actualization. Humaic AIs become our trusted companions and guardian angels.
The vision of humaism may be summarized within the humaic thesis.
THE HUMAIC THESIS argues that the best way to protect humanity from the existential threat of AGI is to give everyone an individually-aligned HI guardian.
The extinction threats are solved through bottom-up alignment of HI.
The enslavement threat disappears once we have universal access to HI.
The replacement threat falls away if we treat HI as a vital companion.
The Humaic Stack
Before creating Humaic Intelligence, we must throw away the book and forget how systems like ChatGPT are built. We must start over from first principles, using humaism as a guide. This path leads us to the seven-layered humaic stack — a technical blueprint for designing open-source HI systems that can compete with AGI at scale. What follows serves as an early sketch, with a more in-depth explanation to follow in subsequent posts.
Synthetic Data: instead of brute-force training on the entire Internet, we need to train HI on smaller, high-quality data sets personalized to each user. This could involve leveraging retrievers for factual data while only training on smaller synthetic data sets as needed for tasks like reading, writing and arithmetic.
Small Models: instead of large cloud models, HI must be small enough to run on personal devices, maintaining privacy while keeping the user in control. Models like Llama, Falcon, and Orca are on the right path, while techniques like LoRA and quantization are making it possible to run them on consumer hardware.
Individual Alignment: instead of aligning to a single ideal group, we need to align each HI to the unique preferences of its individual user. This could be done through limited fine-tuning using personal data to generate an individual constitution with Reinforcement Learning from AI Feedback (RLAIF) or DPO.
Cognitive Architecture: instead of simple assistants that can only answer questions, we need to craft HI as semi-autonomous intelligent agents capable of reasoning, planning, self-reflection, using tools, and remembering our interactions. This requires new purpose-built agentic frameworks for developers.
Universal Interface: instead of just a textual chatbot, we need to re-imagine HI as our interface to all software, by allowing it to generate UIs on-demand while treating traditional apps as APIs, perhaps even as the Operating System itself, whether that be through standard text, voice, video, or even in extended reality.
Open Collaboration: instead of just interacting with their users, HIs should be able to interact directly with each other, while acting to facilitate human communication. Web3 could enable this in global, permissionless, and trustless manner; by providing the infrastructure for identity, payments and collaboration.
Collective Intelligence: instead of monolithic, singular AGI for its own sake, we need a collective super-intelligence powered by the people, which reflects the diversity of human culture and values. This intelligence would be implicitly aligned through bottom-up social consensus, facilitated by our AI companions.
Humaic Consensus
Both the humaic stack and the humaic thesis are predicated on the central idea of humaic consensus, or the idea that humans, when paired with their AI companions, will be able to work together more effectively at scale, as AI will allow us to more easily unlock the emergent collective super-intelligence of the group. The power of humaic consensus lies in its ability to resolve many of the challenges inherent to collective decision making, by increasing participation, improving understanding, and helping to keep the discussion objective. Practically speaking, humaic AIs could leverage existing forms of online collaboration, such as forums and messaging apps, to help us surface, discuss, and resolve open problems much faster.
The logical extension of humaic consensus is Open Collective Intelligence (OCI) — the antithesis to AGI. OCI is based on the idea that collectively aligned super-intelligence could arise organically from the bottom-up, unlocking the collective intelligence of humanity in a manner that is both safe and beneficial to all. In its most open and secure form, OCI would be powered by Web3. This requires giving every human access to a personal AI and a self-sovereign identity, while connecting them through a blockchain-based Decentralized Autonomous Organization (DAO).
Unlike Sam Altman’s dystopian Worldcoin project, this is not about using a blockchain for tracking humans through their biometric identity to issue payments for an AGI-inspired Universal Basic Income (UBI). Instead, OCI removes the need for UBI since it allows everyone to maintain their economic relevance while allowing personal identity to remain private and permissionless.
Announcing Humaic Labs
Today I am pleased to announce the formation of Humaic Labs, the first organization dedicated to advancing the field of Humaic Intelligence. The mission of Humaic Labs is to protect humanity from the existential threat of AGI posed by companies like OpenAI, Anthropic, and Google. Our vision is to realize the humaic thesis by empowering every person on the planet with a humaic companion. Our values are the principles of humaism. Humaic Labs will lead efforts to re-orient the field of AI towards HI.
As the first step, we will soon release an alpha version of Sapial.js, the first developer framework purpose-built for HI. Sapial is derived from homo-sapien, meaning human-like. It is also an acronym for Semi-Autonomous Personal Intelligent Assistants from Large language models. Humaic Labs will maintain Sapial under a permissive open-source license while also building the Humaic Hub, an online platform for sharing, hosting, and connecting sapials.
Humaic Labs is funded by a research grant from the Continuum Collective, an incubator for open-source moonshots that have the potential to empower billions of people around the world. Humaic Labs is already working with key players in the space to build HI, including Hugging Face, for collaborating on open-source models, and the Subspace Network, to realize OCI at global scale.
The Time is Now
If current trends continue, the race for AGI will be won before we solve the alignment problem, and all humanity will suffer the consequences. Proposals to slow down or regulate this race have fallen short, leaving us with few options. The time is now to pursue a fundamentally new approach centered on humaic intelligence. HI offers a human-centric alternative to AGI, empowering each person with an AI companion tailored to their unique needs and values. By giving everyone access to safe and beneficial AI guardians, we can unlock humanity’s potential, through humaic consensus and OCI.
Realizing this vision requires swift and decisive action. We must work together, as researchers, developers, and founders to quickly build an open-source technology stack for humaic intelligence. We need new models, frameworks, interfaces, and infrastructure designed from the ground up according to the principles of humaism. If we act collectively with wisdom, care, and determination, then humaic intelligence offers hope for aligning AI to humanity's diverse values while unlocking the best in each and every one of us. The time for humaic intelligence is now.
Jeremiah Wagstaff is the founder and CEO at Humaic Labs. He holds a B.S. and M.S. in Cultural Geography from Texas A&M University and is a former US Army Infantry Officer. He is also the co-founder and chief architect of the Subspace Network and CEO of the Continuum Collective.
References
These articles provide excellent overviews of the history, motivations, and latest state of OpenAI, Anthropic, and Google Deepmind.
This technical blog post from AssemblyAI provides a great overview of emergence, though the original idea comes from the research paper “Emergent Properties of Large Language Models”.
Before the release of ChatGPT in November 2022, half of all AI experts predicted human-level AGI would be achieved in the 2060’s. As of August 2023, the median predictions of experts for full “embodied” AGI, hover between 2029 and 2032. Purely digital AGI, lacking robotics, would certainly come sooner. Researchers have already noted that GPT-4 exhibits “Sparks of AGI”. Many actively working to build the next generation of model predict that human-level intelligence may occur with the next two to three years, including Demis Hassabis of Google Deepmind, Mustafa Suleyman of InflectionAI, and Elon Musk of xAI.
See the cautionary tale of the Alpha’s and their behind-the-scenes development of AGI, as described within the introduction to Max Tegmark’s Life 3.0.
See Human Compatible by Stuart Russel and The Alignment Problem by Brian Christian for background on the challenges of controlling and aligning AI. Note that while Reinforcement Learning has helped us to steer LLMs like ChatGPT, this technique is not widely considered to be a solution to the alignment problem.
Based on Nick Bostrom’s definition of existential risk from Superintelligence, though the threat assessment is my own synthesis of the existing debate amongst thinkers like Max Tegmark, Eliezer Yudkowsky, Jeremy Howard, and Martin Ford.
Witness the open letter for a six-month pause on the development of models larger than GPT-4 and the statement on the extinction level risks of AI recently singed by hundreds of leaders in the field. The most outspoken individuals include Eliezer Yudkowsky, founder and lead researcher at the Machine Intelligence Research Institute (MIRI) and Max Tegmark, president of the Future of Life Institute. Even Professor Geoffrey Hinton, the so-called godfather of AI, now believes the existential threats from AGI are real, as does his colleague and co-recipient of the Turing Award for their work on neural networks, Yoshua Bengio, who has re-oriented his research towards AI safety.
A Moloch dynamic refers to a situation where individuals or groups act in their own self-interest, leading to an outcome that is detrimental to all parties involved — in this case, all of humanity. For a more in-depth explanation, see the AI Alignment Forum or the post “AI, Moloch, and the Race to the Bottom”.
To truly understand the speed and power of open-source AI, look no further than the leaked internal document from Google “We have no moat and neither does OpenAI”
Following the release of GPT-4, Sam Altman claimed the era of giant models was already over, stating they would actually become both smaller and powerful over time as their inner workings became more clear. The popularity of distillation techniques (compressing large models into smaller ones) as well as training on higher quality (often synthetic) datasets, have so far supported this claim, while demonstrating that those who build LLMs are still practicing something more akin to alchemy than science.
BabyAGI was developed by a lone programmer, with help from GPT-4, which wrote the short paper “Task Driven Autonomous Agents” and wrote the first version of the code. Similar agentic systems, like AutoGPT, are now becoming a standard pattern in AI development frameworks like Langchain and Transformers.
See AI Safety in the age of disenlightenment for an more expansive treaties on the role of open-source in AI and the dangers of centralization that the current approach to regulation will lead to.
Call for self-regulation by leading AI companies look suspiciously like early attempts at regulatory capture, allowing the incumbents to write the rules in their favor such that they ultimately control this incredibly powerful technology.
See Anjre Karapathy’s excellent talk on The State of GPT, describing how ChatGPT was trained and aligned.
While this article rightly focuses on the exploitative and traumatic aspects of how ChatGPT was aligned, it also hints at the subtle nuances of aligning AI with cross-cultural human values.
See Zero to One for background.