Simulation and AI

 Here are some thoughts on connections between on the one hand the simulation hypothesis of Nick Bostrom and others, and on the other hand the risk to humanity allegedly posed by AI. The author is indebted to the responses of others in an online discussion elsewhere for the development of these thoughts.

(The Wikipedia article linked below gives a quick introduction to the simulation hypothesis, with extensive references.)

1. The feasibility of simulation

The simulation hypothesis relies on a civilization developing very powerful computers before it is extinguished.

A civilization with the level of computing power needed to run whole-society simulations would have more than enough to run AI that was considerably more powerful than we have now or can foresee in any detail. And if sophisticated AI were feasible, members of the civilization would develop it. They are after all, by the hypothesis, the sort of beings who would seek to use their computing power in interesting ways.

It is alleged by some that even our AI will lead to our destruction. But if that were inevitable, no civilization would get as far as developing whole-society simulations, because they would have had to go through a stage of powerful AI and survived.

So firm believers in the simulation hypthesis should not think destruction by AI inevitable, and firm believers in inevitable destruction by AI at our current level or modestly more powerful than that level should not believe the simulation hypothesis. Pessimism has its logical limits.

2. Destruction within a simulation

The foregoing argument leaves open the possibility of our being in a simulation and still facing destruction by AI (or in other ways).

We, suspecting that we might be in a simulation, might care about that prospect. And our care might be legitimate, even while thinking about ourselves as snippets of computer code, because we would be snippets of code that were highly sophisticated, that had histories and prospects, and that interacted.

So how might we be destroyed within a simulation?

2.1 Destruction of the simulators

It is possible that the simulators' sophisticated computer system will eventually lead to destruction of the simulators' civilization, and that the computers then left in charge will have no interest in continuing the simulation. And this might even be inevitable. Perhaps the critical level of AI power is up there somewhere, but far above the level we can currently conceive in any detail.

2.2 Destruction by AI within the simulation

Perhaps the simulation, which would include the development of AI by the characters within it, will include its takeover by AI. The characters might stop reproducing and die out. Or the AI might set about achieving some anti-human objective such as devoting all resources to the manufacture of paperclips.

Alas, the simulators might be very happy to see how things might played out in such directions. Like the RAND Corporation, they might be keen to find out what could go wrong for them and how to avert danger.

2.3 Loss of interest by the simulators

If a simulation starts, it will be because the simulators expect it to be interesting. And we can easily suppose that if we are in a simulation, they find us interesting. They have after all, so far as we can tell, let it run over what comes across to us as thousands of years of recorded history, with a chain of communication from one generation to the next, and maybe actually simulated (rather than prefabricated) prehistory. But what if they lose interest because they can see where it is all going?

Our best hope here is that seeing how anything as sophisticated as a society will progress is challenging in the extreme. A society is a chaotic system, highly susceptible to the butterfly effect. There has been plenty of mathematical work on this. Here are two particularly relevant examples.

Lorenz, "The predictability of a flow which possesses many scales of motion", gives the argument that, even in the world of classical physics, there are diminishing returns to greater precision in knowledge of the initial conditions. The diminution in returns sets a hard limit to predictability unless you know the initial conditions with absolute perfection.  

Zhang et al., "A New Approach to Estimating the Limit of Predictability", argues that quantum effects in solar radiation (which would presumably be included in a simulation of the Earth) mean we are stuck with such a limit. We cannot reliably forecast beyond a certain point how things will go, however good our computers may get.

We may conclude that if simulators want to see how a simulation will progress, their most efficient option, and perhaps their only option, will be to let it run. This gives us hope that we will be allowed to keep calm and carry on.

References

Lorenz, Edward N. "The predictability of a flow which possesses many scales of motion", Tellus, volume 21, number 3, 1969, pages 289-307. https://doi.org/10.3402/tellusa.v21i3.10086

Wikipedia: Simulation Hypothesis https://en.wikipedia.org/wiki/Simulation_hypothesis

Zhang, Wei, Zoltan Toth, Feifan Zhou, Jie Feng, Malaquias Peña and Ben Kirtman. "A New Approach to Estimating the Limit of Predictability". Advances in Atmospheric Sciences, volume 43, 2026, pages 2228-2243. https://doi.org/10.1007/s00376-026-5621-8