While there is high uncertainty about the timing of future AI developments, there are plausible scenarios in which AI systems could reach or even vastly surpass human-level proficiency across most relevant cognitive capabilities in the next few years. The international community is currently unprepared to handle the potential global challenges that could arise from the development of such highly capable AI systems. These challenges include: how to ensure that advanced AI systems translate into widespread prosperity and benefits, how to mitigate global-scale public-safety risks from AI, and how to ensure legitimate and effective decision-making on AI development affecting all of humanity. Proactive efforts to design and test necessary building blocks for future international agreements on AI can help to ensure that the international community is equipped to successfully navigate even the most challenging AI development scenarios.