The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma

“This book is about confronting failure. […] If technology damages human lives, or produces societies filled with harm, or renders them ungovernable because we empower a chaotic long tail of bad (or unintentionally dangerous) actors—if, in the aggregate, technology is damaging—then it can be said to have failed in another, deeper sense, failing to live up to its promise. Failure in this sense isn’t intrinsic to technology; it is about the context within which it operates, the governance structures it is subject to, the networks of power and uses to which it is put.”

Suleyman explores failure within the context of technological progress. He unpacks the idea that failure extends beyond mere technical malfunction, emphasizing the broader societal implications and ethical considerations surrounding technology’s impact. Through this analysis, Suleyman prompts readers to critically evaluate the role of technology in shaping human society.

“Technology has a clear, inevitable trajectory: mass diffusion in great roiling waves. This is true from the earliest flint and bone tools to the latest AI models. As science produces new discoveries, people apply these insights to make cheaper food, better goods, and more efficient transport.”

This quote illustrates Suleyman’s perspective on the historical trajectory of technology. By drawing parallels between ancient tools and modern AI, he emphasizes the persistent pattern of technological advancement throughout human history. He establishes the motif of the wave early in the book and employs it throughout the rest of his arguments, using it to convey both the force and the pattern of technological diffusion.

“Understanding technology is, in part, about trying to understand its unintended consequences, to predict not just positive spillovers but ‘revenge effects.’ Quite simply, any technology is capable of going wrong, often in ways that directly contradict its original purpose.”

Suleyman delves into the complex interplay between technology and unintended consequences. By introducing the concept of “revenge effects,” he highlights the inherent unpredictability of technological innovation and the potential for adverse outcomes. The term suggests a sort of backlash or retaliation by technology against its users or the environment. This phrase could even be interpreted as a form of personification, hinting at the increasingly autonomous nature of technology in the coming wave.

“Technology is an eternally dangling carrot, constantly promising more, better, easier, cheaper. Our appetite for invention is insatiable.”

Here, Suleyman employs imagery and metaphor, likening technology to a “dangling carrot” that perpetually entices human innovation. By describing humans as insatiable for innovation, he characterizes technological progress as both a natural and an unending phenomenon.

“It’s often said that there are more potential configurations of a Go board than there are atoms in the known universe; one million trillion trillion trillion trillion more configurations in fact!”

Here, Suleyman uses statistics and comparison to effectively convey the immense scale of possibilities inherent in the game Go. This underscores the difficulty of AlphaGo’s challenge. The exclamation mark at the end of the sentence adds a touch of emphasis, perhaps indicating Suleyman’s own awe at the magnitude of the numbers involved.

“All of this is just the start. We are only beginning to scratch at the profound impact large language models are about to have. If DQN and AlphaGo were the early signs of something lapping at the shore, ChatGPT and LLMs are the first signs of the wave beginning to crash around us.”

Suleyman employs the recurring motif of waves to illustrate the transformative impact of large language models on society. The comparison of DQN and AlphaGo to “something lapping at the shore” contrasts with the impending impact of ChatGPT and LLMs, described as “the wave beginning to crash around us.” This metaphorical language emphasizes the exponential growth and significance of LLMs, suggesting a paradigm shift in technological capabilities. Suleyman’s choice of metaphor evokes a sense of urgency and inevitability in the face of rapid change.

“I think of this as ‘artificial capable intelligence’ (ACI), the point at which AI can achieve complex goals and tasks with minimal oversight. AI and AGI are both parts of the everyday discussion, but we need a concept encapsulating a middle layer in which the Modern Turing Test is achieved but before systems display runaway ‘superintelligence.’ ACI is shorthand for this point.”

Suleyman introduces the concept of “artificial capable intelligence” as a middle ground between artificial intelligence and artificial general intelligence. By defining ACI as the point where AI can achieve complex tasks with minimal oversight but before reaching “runaway superintelligence,” Suleyman provides a nuanced framework for understanding AI development. The term “Modern Turing Test” serves as a benchmark for assessing AI capabilities, indicating a stage where AI systems exhibit human-like intelligence. Through this conceptualization, Suleyman navigates the complex discourse surrounding AI ethics and existential risks, offering a vocabulary to discuss AI progress and its societal implications.

“The bio-revolution is coevolving with advances in AI, and indeed many of the phenomena discussed in this chapter will rely on AI for their realization. Think, then, of two waves crashing together, not a wave but a superwave.”

Here, Suleyman uses the motif of waves crashing together to depict the synergistic relationship between these two domains of AI and biology. By likening AI and synthetic biology to interchangeable components of a single wave, Suleyman emphasizes their intertwined development and mutually reinforcing relationship.

“At its core, the coming wave is a story of the proliferation of power. If the last wave reduced the costs of broadcasting information, this one reduces the costs of acting on it, giving rise to technologies that go from sequencing to synthesis, reading to writing, editing to creating, imitating conversations to leading them. In this, it is qualitatively different from every previous wave, despite all the big claims made about the transformative power of the internet.”

Suleyman characterizes the coming wave of technology as a story of power proliferation, contrasting it with previous waves of innovation. The analogy of reducing costs from “sequencing to synthesis, reading to writing” highlights the transformative nature of this wave, emphasizing its broad impact on various domains.

“AI creates asymmetric risks beyond those of a bad batch of food, a plane accident, or a faulty product. Its risks extend to entire societies, making it not so much a blunt tool as a lever with global consequences. Just as globalized and highly connected markets transmit contagion in a financial crisis, so with technology. Network scale makes containing damage, if or when it comes, almost impossible.”

In this quote, Suleyman explores the asymmetric risks posed by AI, likening it to a lever with global consequences, thereby emphasizing the way in which it confers outsized power to individuals. This draws attention to the challenges of containing AI-related damage, emphasizing the need for proactive risk management strategies. By framing AI as a tool with unprecedented societal implications, Suleyman underscores the urgency of addressing its risks and implications for global governance.

“AlphaGo was quickly labeled China’s Sputnik moment for AI. The Americans and the West, just as they had done in the early days of the internet, were threatening to steal a march on an epoch-making technology. Here was the clearest possible reminder that China, beaten at a national pastime, could once again find itself far behind the frontier.”

Suleyman employs a historical reference—“China's Sputnik moment for AI”—to underscore the geopolitical implications of technological advancements. The comparison to the Space Race era highlights the competitive dynamics between nations in the AI domain. By characterizing AlphaGo’s victory as a wake-up call for China, Suleyman emphasizes the strategic significance of AI development in shaping national competitiveness and global power dynamics. The reference to being “far behind the frontier” evokes a sense of urgency and competition, suggesting the stakes involved in technological innovation. Overall, this quote highlights the geopolitical dimensions of AI advancement and its implications for national strategy and competitiveness.

“It’s here, it’s happening. It is a point so obvious it doesn’t often get mentioned: there is no central authority controlling what technologies get developed, who does it, and for what purpose; technology is an orchestra with no conductor.”

Suleyman employs imagery and metaphor to convey the inevitability and complexity of the technological arms race. The metaphor of technology as an orchestra without a conductor emphasizes the decentralized nature of technological development, highlighting the absence of centralized coordination. This metaphor underscores the lack of control over technological progress, suggesting that it has far-reaching consequences beyond human intervention.

“Whether noble and high-minded or bitter and zero-sum, when you work on technology, it’s often this aspect, even more than the needs of states or the imperatives of distant shareholders, animating progress. Find a successful scientist or technologist and somewhere in there you will see someone driven by raw ego, spurred on by emotive impulses that might sound base or even unethical but are nonetheless an under-recognized part of why we get the technologies we do.”

In this quote, Suleyman delves into the psychological motivations driving technological progress, emphasizing the role of ego and emotive impulses. Through introspective language, he humanizes scientists and technologists, portraying them as driven by personal ambition and emotional instincts. Suleyman’s characterization adds depth to the narrative, revealing the complex interplay of individual motivations and societal progress. By acknowledging the influence of ego and emotion in technological innovation, Suleyman provides insight into the underlying dynamics shaping technological development.

“Too fast in its development, too global, too protean and enticing for any simple model of containment, strategically critical, relied upon by billions, modern technology itself is a prime actor, a monumental force nation-states struggle to manage. AI, synthetic biology, and the rest are being introduced to dysfunctional societies already rocked back and forth on technological waves of immense power. This is not a world ready for the coming wave. This is a world buckling under the existing strain.”

Suleyman employs evocative language and imagery to depict the overwhelming challenges posed by the rapid development of modern technology. The use of descriptors such as “too fast,” “too global,” and “too protean” underscores the scale and complexity of the technological landscape. Through the metaphor of a world buckling under existing strain, Suleyman conveys the sense of impending crisis and societal disarray. Furthermore, his portrayal of technology as a “monumental force” highlights its transformative power and the inadequacy of traditional containment measures.

“Power is […] the mechanical or electrical energy that underwrites civilization. The bedrock and central principle of the state. Power in one form or another shapes everything. And it, too, is about to be transformed. Technology is ultimately political because technology is a form of power. And perhaps the single overriding characteristic of the coming wave is that it will democratize access to power.”

Suleyman explores the transformative potential of technology. By defining power as “the mechanical or electrical energy that underwrites civilization,” he establishes a broad conceptual framework for understanding its significance. The assertion that technology is ultimately political underscores its role as a tool for shaping social structures and governance systems. Suleyman’s analysis illuminates the democratizing effects of the coming wave, suggesting that it will redefine existing power structures and empower diverse actors.

“It doesn’t take a wild imagination, however, to envisage how such research could go wrong. Deliberately engineering or evolving viruses like this was, some felt, including myself, a bit like playing with the nuclear trigger. Gain-of-function research is, suffice to say, controversial.”

Through a cautionary tone and rhetorical questions, Suleyman highlights the ethical dilemmas associated with gain-of-function research in synthetic biology. The comparison to nuclear weapons evokes imagery of immense power and immense potential consequences, emphasizing the gravity of the situation. Suleyman’s use of personal reflection, including his own reservations about such research, adds credibility to his argument and invites readers to consider the moral implications. This quote underscores the ethical complexities inherent in scientific research and the need for careful deliberation.

“There is already a pronounced and accelerating ‘superstar’ effect, where leading players take ever more outsized shares of the pie. […] The top 10 percent of global firms take 80 percent of the total profits. Expect the coming wave to feed into this picture, producing ever-richer and more successful superstars—whether regions, business sectors, companies, or research groups. I think we’ll see a group of private corporations grow beyond the size and reach of many nation-states.”

Suleyman explores the socioeconomic implications of the coming wave, focusing on the concentration of power and wealth among a select few. The metaphor of a “superstar effect” vividly conveys the disproportionate success enjoyed by leading players in the global economy. By highlighting the widening gap between top performers and the rest, Suleyman underscores the potential for inequality to intensify in the wake of technological advancements. This quote offers a reflection on the distribution of wealth and power in society, raising questions about equity and fairness in the digital age.

“Humanity is too multifarious, too impulsive to be boxed in like this. In the past, the tools available to totalitarian governments simply weren’t equal to the task. […] The coming wave presents the disturbing possibility that this may no longer be true. Instead, it could initiate an injection of centralized power and control that will morph state functions into repressive distortions of their original purpose. Rocket fuel for authoritarians and for great power competition alike.”

Suleyman employs imagery to convey the ominous implications of centralized power in the face of technological advancement. The metaphor of humanity being boxed in underscores the potential for loss of freedom and autonomy in a highly controlled society. Comparing the coming wave to rocket fuel, Suleyman highlights the transformative nature of the coming wave, suggesting that it could catalyze authoritarian regimes and geopolitical tensions.

“Something more like the pre-nation-state world emerges in this scenario, neo-medieval, smaller, more local, and constitutionally diverse, a complex, unstable patchwork of polities. Only this time with hugely powerful technology. When northern Italy was a patchwork of small city-states, it gave us the Renaissance, yet was also a field of constant internecine war and feuding. Renaissance is great; unceasing war with tomorrow’s military technology, not so much.”

Through historical analogy and metaphor, Suleyman paints a picture of the potential societal implications of technological advancement. The comparison to the pre-nation-state world evokes imagery of complexity and instability, highlighting the potential for conflict and innovation in a decentralized society. By drawing parallels to historical events, Suleyman underscores the cyclical nature of human progress and the unpredictable consequences of technological advancement.

“Once fast-evolving, self-assembling automatons or new biological agents are released, out in the wild, there’s no rewinding the clock. After a certain point, even curiosity and tinkering might be dangerous.”

Suleyman employs cautionary language to underscore the irreversible nature of technological innovation. The phrase “no rewinding the clock” evokes a sense of urgency and risk, highlighting the potential dangers of unchecked technological progress. By emphasizing the need for caution and vigilance, Suleyman warns against complacency in the face of technological advancement.

“If zombielike states will sleepwalk into catastrophe, their openness and growing chaos a petri dish for uncontained technology, authoritarian states are already gladly charging into just this techno-dystopia, setting the stage, technologically if not morally, for massive invasions of privacy and curtailments of liberty. And on the continuum between the two there is also a chance of the worst of all worlds: scattered but repressive surveillance and control apparatuses that still don’t add up to a watertight system. Catastrophe and dystopia.”

Through vivid imagery and parallel structure, Suleyman highlights the contrasting trajectories of totalitarian and authoritarian states in response to technological advancement. The dichotomy between “zombielike states” and “authoritarian states” underscores the divergent paths of governance, from passivity to aggression. By juxtaposing these scenarios, Suleyman underscores the complexity of the geopolitical landscape and the potential for catastrophic outcomes. The use of parallel structure adds rhetorical impact, emphasizing the urgent need for action.

“The central problem for humanity in the twenty-first century is how we can nurture sufficient legitimate political power and wisdom, adequate technical mastery, and robust norms to constrain technologies to ensure they continue to do far more good than harm. How, in other words, we can contain the seemingly uncontainable.”

Suleyman uses declarative language to underscore the central dilemma of technological containment. The juxtaposition of “legitimate political power and wisdom” with “technical mastery” highlights the multifaceted nature of the challenge. By framing containment as a balancing act between political governance and technical expertise, Suleyman underscores the complexity of the task at hand.

“A useful comparison here is climate change. It too deals with risks that are often diffuse, uncertain, temporally distant, happening elsewhere, lacking the salience, adrenaline, and immediacy of an ambush on the savanna—the kind of risk we are well primed to respond to. Psychologically, none of this feels present. Our prehistoric brains are generally hopeless at dealing with amorphous threats like these.”

Suleyman uses analogy to contextualize the challenge of addressing diffuse and uncertain risks. The comparison to climate change highlights the psychological barriers to addressing long-term, systemic threats. By comparing these challenges, Suleyman underscores the need for a nuanced and proactive approach to risk management, one that relies on careful analysis rather than emotion. This quote uses imagery—“an ambush on the savanna”—to highlight the vast difference between the current threat and the kinds of dangers human brains have evolved to deal with.