We are having the wrong nightmares about artificial intelligence. At their extremes, current narratives portray an AI utopia where superintelligence comes to either save or kill us all. To be clear, many of the risks of powerful AI in those discussions deserve attention. But artificial intelligence may wreak social havoc way before, and regardless of whether AI displaces humans from most  jobs or Terminator comes to destroy us. Our research group contends that dominant narratives suffer from three core errors.

Error 1: A systemic historical bias in how technologists close to a particular technology and commentators involved in early discussion evaluate its impacts in its initial days, focusing on individual-level changes and benchmarks rather than complex societal interactions.

Error 2: Conceptualizations of  the new technology in metaphors and analogies derived from the past, or as a direct replacement for something else on a one-to-one basis, rather than on its own terms.

Error 3: Not incorporating  lessons from sociology like the impacts from scale, scarcity vs. abundance, load-bearing frictions, and societal mechanisms – to name a few.

Consider cars and horses. When automobiles came about, most technologists in the automobile industry’s early days thought of them as replacements for horses, with early automobiles earning the name “horseless carriages.” Proponents focused on speed as a revealing, all-important benchmark. Early defenders of the automobile cited less manure as a social benefit. But while speed may be a helpful indicator of a car’s capabilities, and while a single car may reduce horse manure on the streets of Manhattan, the effects at scale are  different. Replacing a few thousand horses with cars can provide cleaner streets in the city, but mass car ownership by millions allows people to move out of the city and into the suburbs wholesale, with accompanying issues like long commutes, isolation, unhealthy lifestyles and pollution. Ten thousand cars allow faster travel to their relatively rare owners, but ten million cars sit in traffic bumper to bumper. Hundreds of millions of cars bring about resource wars, geopolitical shifts, and climate change.

And just like cars are not horses, manuscripts are not books, physical mail is not electronic mail, octopodes’ eyes are not human eyes, electricity is not the steam engine, Generative AI is not a human on a chip slowly getting smarter. We contend that the specific ways in which generative AI differs from human intelligence is highly consequential. However, we do believe this is a very powerful technology that is already starting to destabilize our world in myriad ways. Large Language Models are speaking machines that output emotive and affective language that is nearly – and sometimes completely – indistinguishable from humans. Unsurprisingly, humans are anthropomorphizing these statistical systems, and in some cases, are developing a sense of computer-person-relation with them, exhibiting delusional thinking, or experiencing psychotic breaks. Crucially, Generative AI will break, and in some cases, already is breaking, many mechanisms of proof: effort, authenticity, accuracy, sincerity, and even humanity.

Error 4: Current debate on AI alignment misses the big one – The Looming AI Business Model – Engagement is Coming

It has become clear that at least some of the companies will bring over the engagement model of social media to chatbots, monetizing ads, shopping recommendations, affiliate links, and sponsored answers. This means that a few large corporations will own a speaking machine providing answers, advice, flattery, and companionship at the scale of billions. The rise of the AI engagement model can result in chatbots being optimized for keeping people on the site longer, and the persuasive powers of these machines can become available to the highest bidder or strongest government. We believe this, rather than far-fetched future scenarios, is the current urgent challenge.