In her book Prophecy, Carissa Véliz argues that predictions about the future, particularly in the area of AI, are rarely neutral: they are often ways of nudging us towards a desired course of action. People do things to us with predictions. When the big names in tech start telling us that a swarm of bots could take over the internet within a year, or that there is a 10% chance that AI will destroy humanity within a decade, it’s worth asking: “Why say this now?”
Do their pronouncements and their timing relate to imminent stock market launches? Is this all a way of slowing government intervention by showing that these companies are run by responsible adults, and are not children playing with fire?
Or are these genuine warnings from the very people who have rushed ahead with this technology and who are now suddenly recognising what they have created: machines that could, if things go well and they behave themselves, save us from pandemics, climate catastrophe, and more, or, if badly, wipe us off the planet on a whim?
In the 17th century, the theologian-philosopher and mathematician Blaise Pascal put forward a famous argument about why agnostics ought to gamble on the existence of God, now known as Pascal’s Wager. If you bet on God’s existence by believing, and he does actually exist, then you get eternal bliss, a huge payout. If you gamble on him not existing and are wrong and he does, you risk eternal damnation, the worst outcome imaginable – think Hieronymus Bosch. If God doesn’t exist and you mistakenly believe that he does, you’ll probably miss out on some earthly delights and live a life based on falsehood, but that is nothing compared with an eternity of being tortured in Hell.
If God doesn’t exist, and you guess correctly that he doesn’t, then you will live a full human life without illusion, but that is not as big a win as an unending experience of Heaven (unless everlasting life turns out to be the equivalent of being trapped in a cocktail party you can never leave, something Bernard Williams once suggested).
Pascal argued that a rational gambler, someone who wants to maximise possible winnings while avoiding the risk of devastating losses, should believe in God. You win the jackpot if you bet correctly, and whatever happens you eliminate the risk of infinite punishment. If you find you can’t bring yourself to believe in God, Pascal recommended going through the motions: attend church, say prayers, and so on. Soon you will have not just the external behaviour, but also the belief that could save your soul.
No one knows how the great AI experiment is going to turn out. It could place us in a kind of heaven, completely annihilate us, or deliver something in between. Should we then bet on it and continue to divert our resources towards building data centres, which further pollute the planet, because the potential benefits are so great? How, in a world where reliable predictions are impossible, do we balance possible pay-offs against immense risks?
The real problem with the tech bros’ wager, one that we are all subject to, is that unlike with Pascal’s, gambling on AI still carries the risk of the worst case scenario. The bet doesn’t protect you from Hell and might in fact hasten it.
Suggested Reading
AI is designing viruses – what could possibly go wrong?
If you believe in God you (allegedly) insure yourself against the worst imaginable outcome. But if you bet on AI saving the world it could still destroy it, or else damage it so badly that no one would want to carry on living here.
My money is on the doomsday scenario. The genie is already out of the bottle, and no retrospective legislation is going to force that lid back on.
But I still hope the gamble pays off and our future is full of bliss. If not, let’s hope annihilation is painless.
