The self and AI

 1. Games of Go

Here is a puzzle from Henry Shevlin's X feed, posted on 2 December 2025.

"Roughly 99.99% of games of Go ever played are AI self-play training games. So if you ever find yourself playing Go that means you're probably an AI."

https://x.com/dioscuri/status/1995652145916641649

The post is clearly a joke if you are Descartes with his soul. But if you are David Hume, with his theory that there is nothing more to the self than a bundle of perceptions, it may not be so funny.

Let us explore how someone taking Hume's view of the self might react to the post.

2. Hume's view

We shall first make the view more precise. We shall not attempt Hume exegesis. Hume might or might not have accepted the refinement we shall choose. Rather, we just want to have a definite view to explore.

Specifically, we must focus on the words "nothing more ... than". The only safe thing for an empiricist to say would be that there was nothing more to be seen in the self than a bundle of perceptions, leaving open the question of whether there was in fact something more that was imperceptible. That is how we shall take the Humean view.

3. Why the Go post might worry a Humean

If one takes the Cartesian view, and indeed the general common sense view, that there is no doubt that one is a stable continuing self who has experiences and takes actions, the post about games of Go should not be at all troubling. An individual knows enough about himself or herself to know that he or she is a human being and not an AI system or an element in one (we shall for brevity simply say "an AI system", which might be a subsystem of something larger). Moreover, this is straightforward factual knowledge, not merely something that on the lines of Wittgensteinian hinge propositions one has to believe in order for anything to make any sense, although it is that as well.

A Humean should lack that unshakable confidence in being a stable continuing self that has experiences and takes actions, rather than a self that is a mere set of experiences, including both sensory experiences and experiences of agency. The stable continuing self would not be ruled out, but it would be insufficiently evidenced.

Then an experience that should on probabilistic grounds be assigned to AI, such as a game of Go, should lead the experiencer to entertain the possibility that he or she is in fact an AI system. The worry might only endure for the length of the game. Experiences of activities not normally performed by AI would not be susceptible to the same probabilistic computation. But during the game, the pure and rigorous Humean could not safely reach forward or backward to those other experiences in order to see off the fear of being an AI system. Under the Humean conception, no reliance could be placed on any such bond across time. One might adopt the idea that one was a temporally extended set of experiences, but which set? Nobody's experiences yesterday or tomorrow would have a better claim than anybody else's to go into the same set as one's own experiences today.

There would be two ways to create the required bond and select the right members of the temporally extended set. The first way would be to rely on bodily continuity. The experiences of one's own body are one's own, and the experiences of other bodies are not. To make that work for an individual from his or her internal perspective, he or she would need an internal sense of location in the world and movement from place to place. The second way would be to rely on the phenomenology of memory. We shall pursue this way, because the first way would rely on it to give rise to the required internal sense of location and movement.

4. Sequences of experiences

4.1 The use of sequences

If we allow for memory as a way to assemble sequences, the Humean would not be limited to assigning probabilities of alternative natures of himself or herself one experience at a time, as the subject of this experience of a game of Go then the subject of that experience of a shower of rain. Instead, the Humean could assign probabilities to alternative natures as the subject of a long sequence of experiences. The experience of a game of Go in isolation might be very likely to be the experience of an AI system. (We pass over the question of whether AI systems could have experiences at all. If we knew they could not, that would rule out the AI possibility.) But the subject of a complex sequence of varied experiences should not be thought on probabilistic grounds to be an AI system.

4.2 The construction of sequences

Under what conditions could a subject construct a long sequence of experiences and regard it as a set of experiences enjoyed by himself or herself? We might not have to go as far as a Cartesian or common sense notion of the self, but we would edge in that direction.

The minimal condition to meet would be that members of a sequence would be re-experiencable as memories, in some way that the rememberer would have good reason to suppose was at least a very strong indication that the experiences belonged together in a single sequence. This would give reason to think that the memory or current experience of a game of Go which was placed in the same sequence was not good evidence that the player was an AI system. And it would be the phenomenology of memory that would provide the evidence that experiences belonged together. (One might add future possible experiences and the phenomenology of anticipation, but we shall not do so here.)

We cannot however leave it at that. If experiences should be seen as belonging together in a single sequence, we should say something about the nature of the relationship of belonging together. The natural thing would be to say that it was built on the attachment of each experience separately to a particular person, and that the relationship between the experiences was their having in common attachment to the same person (and not to any other people).

This might look suspiciously circular. Are we not in pursuit of a single subject? How could we reach such a subject by starting from a relationship that presumed it?

In fact, things would not be so bad. We could say that the manifest appearance of the attachment of remembered experiences to a single subject was best explained by the theory that there was indeed such a single subject. And that would have the advantage of fitting in with our understanding of the physical world, in particular what it takes to acquire and store information. Individual human beings at different spatiotemporal locations absorb different bodies of information, conveniently if unreliably recorded in their brains, only by actual contact with the relevant reality or by outward communication with other people, not by clairvoyance or telepathy. And when the route is communication with others, what is received has a different flavour from first-person experience, so that there is not often any thought by the recipient that he or she actually had the experience.

There is an important condition, which we have not so far mentioned, of being able to see experiences as belonging together as a single sequence of actual experiences of a subject. This is that there should be a degree of coherence of the experiences. Perfect coherence is not to be expected. We do have mistaken memories and wholly imagined apparent memories that might on inspection be found not to cohere with all other memories, only with some or none of them. But coherence is the norm.

We have kept the coherence condition back until now because its real importance arises at the point where we want to argue that the experience of playing Go is best explained by the game being an event in a real human life, rather than its being activity of an AI system. An incoherent set of impressions might best be explained as the random activity of an AI system.

There would be one remaining obstacle to the claim that the attachment of remembered experiences to a single subject who had actually had those experiences over time was the best explanation. This is that one might be a Boltzmann brain, a chance conjunction of atoms with all the appropriate apparent memories of apparently earlier experience and of other people. Such a thought would however be incompatible with discourse over time or with other people. We may discard it not because we can prove that it would be mistaken, but because belief that one was a Boltzmann brain would, like denial of one of Wittgenstein's hinge propositions, destroy the conditions for our conducting discourse in a way that it made any sense to conduct it.

4.3 Short-term and long-term sequences

What we have offered here is an approach to the binding problem in the specific form of tying experiences together over time, rather than in the form of tying impressions in different sensory modalities together to form complete, rounded, experiences. And it is an approach intended to respond to an epistemic concern, rather than any neurological concern.

Our approach also focuses on a specific form of tying together over time, the long-term form. There is a short-term form that arises when for example we hear a piece of music or read a poem as a unity with its own temporal form and balance, not as detached notes or lines. That sort of tying together is something that barely requires an argument as to its epistemic robustness. We perceive the unity directly. On the other hand, such directly perceived short-term unity would not suffice to ward off concerns about being an AI system. The very fact that the unity was of one form of experience within a short span, for example the experience of a whole game of Go with an unfolding strategy, would deprive it of the links to life in general that would arise in a longer and more varied sequence of experiences and that would make an AI origin of the individual members of the sequence an inferior explanation. The narrowness of the short-term version would leave a sequence that yielded a sensation of unity all too open to being easily created by AI.

5. A Kantian approach

There is an alternative approach that differs from our probabilistic argument. This alternative borrows from some of the work of Immanuel Kant. (We shall here borrow to suit our purposes, and not engage in exegesis.)

One of Kant's concerns was to identify the conditions under which it is possible to have coherent experience of the world, rather than a chaotic flood of sense impressions.

Space, time and causation play important roles. We can allocate the apparent sources of impressions to different places and times, and see how events in the physical world occur according to comprehensible laws rather than at random.

Just as significant is the enduring and experiencing subject. A flood of impressions will only make sense when tied to such a subject. This is not a tool to use in generating coherence in the same way as space, time and causation. Rather, the presence of the subject as possessor of the impressions is itself the act of unification of the impressions into a coherent whole.

This would not support the earlier argument from one's being a creature with a complex sequence of experiences in which a game of Go was embedded to one's not being entitled to reason that one was probably an AI system. Instead, it would support an argument that the AI option made no sense.

In our earlier argument we first required that experiences were remembered as the experiences of the subject, then added the condition of a degree of coherence of the remembered experiences. Finally, we identified the best explanation of the existence of such a set of memories of experiences tied together by their apparent attachment to the subject. It was their being memories of a set of real-world experiences of a human subject rather than their being the results of activity of an AI system. Specifically, a memory or current experience of a game of Go would belong to that set of experiences.

In that argument, space, time and causation did similar work to the work they do for Kant, albeit in the context of a different metaphysic. They were tools to sort out a set of remembered experiences into a single coherent sequence.

What would change in moving to a Kantian approach would be the role of the subject. It would not appear late in the day as a best explanation. Instead it would be assumed from the start. And it would be assumed not because that was convenient, but because it was inevitable.

Kant makes explicit the point that experiences need to be experiences had by a subject. It is easy to overlook this point, because there is a simple version of it that makes it obvious. As a verb, "experience" has an active voice in which it needs a subject. And if that were all there were to it, we would not have added to our earlier argument.

We can see that there is more to it than that when we reflect on just how much comes along with the Kantian notion of the experiencing subject. It is not a mere cipher, something to occupy the subject position in front of a verb. Rather, it is a subject of a life of experiences, laid out for the subject in space and time.

This does not in itself defeat the challenge that arises out of the fact that most games of Go are played by AI systems. The Kantian subject could have a very short life of experiences, one that comprised only a sequence of moves in a game of Go.

The advance to the conclusion we would like, that the experience of a game of Go should not lead the player to consider that he or she might well be an AI system, comes from the player's noting his or her memories of other experiences. In order for them to be seen by the subject as his or her experiences, they must be located in space and time relative to the subject. And they must be seen as the experiences of the same subject as is aware of playing Go. The subject is unitary, and has one life. He or she could imagine things that never happened and accidentally take the imaginings to be memories. But this could only be done in relation to a small proportion of apparent experiences. Nothing would make sense without the background of a single continuing subject with a presence in space and time relative to which the sources of experiences could be positioned.

The contribution of experiences here is not probabilistic. Rather, it is that their existence forces the subject to conceive of himself or herself as a subject with a life of experience and action in space and time. A computer that hosts an AI system is in space and time, but its location, and indeed which specific computer hosts the system, are arbitrary and irrelevant to what happens. The spatiotemporal locations of real human subjects, on the other hand, are highly relevant to the courses of their lives.

Likewise, no sense could be made of the idea that the whole sequence of experiences was an artificial creation, manufactured by the controllers of one's brain in a vat. That supposition would undermine the subject's conceptions of space, time and causation. The coherence of experiences might be the same as it would be with real experiences, but the subject's awareness of its possible artificiality and the consequent possible subjection of apparent laws of nature to the whim of the controllers, so that they could not be regarded as laws at all, would greatly disrupt the subject's conception of the world that he or she inhabited, and hence disrupt the subject's conception of himself or herself.

This is not to say that the subject could not be a brain in a vat, only that the subject could not see himself or herself in that way. And since the disturbing thought about the game of Go is directed at the player, an inability to see oneself in a certain way would be enough to hold at bay the thought that one was probably an AI system.

We should add that the Kantian subject is a curious thing, a noumenal entity that corresponds to a physical presence in the phenomenal world. This might affect our attitude to a full-blown Kantian position. Some people would rather not commit to Kant's metaphysic. And it could be argued that we could not make the kind of use we have made of the Kantian subject without accepting that metaphysic. At the very least, our use of Kant's ideas requires the subject to see himself or herself from inside, and not from outside as an object in the world.

6. The general simulation argument

There is a general simulation argument, championed by Nick Bostrom and others. It starts with the observation that it will in due course become computationally feasible to run simulations of whole societies, and that people will find it amusing to do so. So there will be one real world and hundreds of simulations. So it is highly probable that each entity that sees itself as a conscious human being is in fact a character in a simulation.

Our first response to the challenge of the game of Go would be powerless here. That response relies on expanding the field of view from a particular experience of playing a game to a wide range of events in a life. Under the general simulation argument, everything is already set up and all thoughts and apparent perceptions are adjusted to yield a coherent apparent life.

The Kantian approach, on the other hand, would allow a response to the general simulation argument. This would be that the idea of a subject that was the passive recipient of whatever the controllers of a simulation decided to purvey would make no sense. The subject needs to live in a world that is not merely perceived as being laid out in space and time, with natural laws governing what is perceived to happen, but in a world that the subject can regard as necessarily so. The thought that one might be a mere piece of computer code in a simulation, at risk of having one's world turned upside down at the whim of a controller, would undermine the basic conception of oneself as a subject in the world.

This argument would however only make illegitimate the thought that one was in a simulation. It would not exclude the possibility of the fact. There would even be a variant of Kant's metaphysic in which the simulators and their computers were the noumena, and the subject within the simulation perceived the phenomena that corresponded to those noumena (Kant would not let us say caused by the noumena). The subject would be debarred from any knowledge of the noumena. Indeed, if the subject refrained from speculation about a possible simulation, there would be a way back to regarding the perceived world as necessarily laid out in space and time and governed by natural laws. All of this would however take us well beyond Kant himself, who like Hume lived in an age in which computers could at most be objects of fantasy.