To reliably prevent extinction from AI, we need to prevent godlike AI from being developed, at least for a very extended period of time until we know how to proceed safely. Here’s what it would take to do that.
Leading AI companies’ executives and researchers should be called to appear before parliaments around the world and testify. Under oath, they should state the capabilities they expect their AI to have, the risks it may pose, their plan to mitigate risks, and the limitations of their plans.
These executives and researchers have repeatedly stated their concerns about risk from the technology they are developing, saying it could threaten the “continued existence of humanity”. These concerns should be elaborated on and explained truthfully to the public and policymakers.
Governments around the world should place a moratorium on training runs using more than 10^23 FLOP in compute (approximately the amount of compute used for the original GPT-3 175B).
This moratorium should apply globally, however governments can already begin to implement these bans nationally. Any government can already require companies operating in its jurisdiction to show they are not running training runs above the threshold: if they are, they should be asked to terminate the run, or not be allowed to operate in the country.
AIs from training runs above the size of GPT-3 may be capable of becoming existentially dangerous, either by themselves or with architecture on top of them that makes them fully autonomous and helps them self-improve. Until researchers can reliably prevent these dangers, systems above this training run size should be built. The amount of computation used in a training run is the most straightforward predictor of an AI’s abilities and dangerousness, and computing resources are easy to track and monitor. The cap on training run size should be set conservatively, to account for algorithmic progress.
Projects involving AIs interacting with themselves, autonomously planning and executing tasks, or AIs interacting with other AIs should be banned. This includes connecting AIs to the internet and making them capable of interacting with copies of themselves, self-prompting and self-prompt selection, training on AI generated data, autonomous loops, and any other activities that could lead to fully autonomous or situationally-aware AIs.
AI companies alone would not be responsible for ensuring this is upheld. Other actors in the supply chain, such as compute providers and websites hosting code repositories, should also be held accountable to preventing the use and development of these capabilities.
Making AIs increasingly autonomous is the most straightforward path towards AIs being able to escape human control. It is what moves AI risk from a regime of manageable accidents to a regime for which we have almost no countermeasures.
All AIs trained with more compute than 10^23 FLOP should be removed from public and private circulation, including at the organizations that developed them. AI models’ weights should be destroyed.
Even if developers limit access to AIs above the threshold, a single hack or accident could lead to their exfiltration. With enough scaffolding and development on top of these AIs, rogue individuals or nation states could then still build fully autonomous AIs.
To conduct deep learning training runs below GPT-3 levels, organizations should be licensed by the government. Licenses should only be provided to organizations that comply with security standards. Governments should monitor compliance with licensing requirements continuously.
Systems trained with less computing resources than GPT-3 are unlikely to be on the development path of existentially dangerous, fully autonomous systems. However, with research progress, they will increasingly near the capabilities of current powerful systems, and they still constitute a threat in case they proliferate. Given this, their development and production should be monitored and regulated. Fast iteration loops on their improvements should be constrained.
Governments should assess each proposed training run. They should determine approval based on factors like how capable the AI being built is expected to be, what the AI is expected to be used for, the competence of the involved research team and laboratory, the effectiveness and limitations of safety techniques, etc. Data centers should only provide compute to approved projects.
Governments should know when high-risk projects are happening and be able to block projects that seem unmanageably risky.
The primary resource that enables AI research is compute. AI computing resources, in particular data center-grade GPUs used for machine learning, should accordingly be tracked and secured.
Governments should keep track of data center-grade computing resources’ location and whether they’re being used for AI development. Individual data center-grade GPUs should have hardware-level monitoring systems capable of detecting whether an AI training run above a certain size is being run on them, and they should be equipped with remote shut down capabilities. (For more, see Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring.)
Compute is the physical resource that powers AI development, akin to uranium for nuclear fission. Current trends in hardware and software improvement suggest in 6 years, it may be possible to train a system as powerful as GPT-4 on a personal laptop running for a few weeks. If compute access is not restricted soon, this could mean the proliferation of millions of hard to trace models that can be modified to recursively self improve and destroy humanity.
Governments should coordinate to create an institution for researching controlling AI systems. Because this research is inherently dual-use, the institution must be exceptionally secure – more so than existing nuclear weapon and biosafety-level 4 projects. It must also have international oversight to ensure it remains committed to safety only. Outputs of this research project should only be made public for commercialization once they have been demonstrated to be safe.
Algorithmic improvements may eventually make it possible to run very powerful AIs on consumer-grade GPUs. This and other possibilities make our proposals insufficient for preventing godlike AI forever. Therefore, whether or not humanity wants to build godlike AI, we should continue researching how to control less powerful AI.