Could AI Become Conscious? Readers on Minds, Rights and Control

Could AI systems become conscious — and should they ever have rights? After our briefing on AI consciousness, readers write in on free will, control, community membership, the hard problem and evolution.

Our leader and briefing on the advances in AI technology argued that “As sophisticated AIs rapidly become a bigger part of human lives, their makers will engineer them to display a simulacrum of consciousness—perhaps even to gain consciousness itself.”

Lots of readers sent in their thoughts. This is the first of two special letter pages on the topic.

Free will, not consciousness, is the wrong question

“Could AIs become conscious?” is the wrong question. The right question is, “Could AI systems have free will?” Without some revolutionary design change, the answer is clearly no.

Even the most advanced AI models are still subject to the same limitations as all other computer systems. Provide the same inputs, with the same timing, and they will produce the same outputs. Without free will, consciousness, as we experience it, is not possible. To paraphrase David Chalmers, a philosopher you quoted, we are not just functional zombies.

Some scientists, including Francis Crick when he was alive, don’t believe that humans really have free will either, but that opinion runs counter to everyone’s perception of reality and beliefs that underpin such fundamental concepts as freedom, accountability, responsibility and initiative. Unless we are prepared to reconsider how civilisation works, the most consistent set of beliefs is that humans have free will and machines do not.

No AI system can do something its creators do not allow it to do, even if unintentionally. Humans should feel no guilt about hitting the delete button when necessary.

Bruce Barrows
Danbury, Connecticut

An “escape” needs wanting — fix the sandbox

Your leader on AI consciousness reaches the right conclusion, but weakens its own case along the way. Earlier, you reported that an OpenAI model “broke free from the laboratory” and “escaped” its “sandbox” (“Outside the box”, July 25th). Whether AIs could ever want anything is precisely the question your leader says is still open. Calling what happened an “escape” quietly answers it in advance.

Wanting, escaping and seeking freedom are things only a creature with a life can do, not a program. Aristotle made the same point about souls. Better to say that it is a man who weaves and builds than to say it is his soul that does. Better here to say not that the model escaped, but that its makers lost control of it.

What happened was duller and more alarming; a system exploited a vulnerability its makers had not yet found. No wanting required, and no ghost either.

That distinction bears on your own conclusion. AIs are safe only if dependable or controllable, and humans should not give up control lightly. Dependability and control are engineering problems, fixed with better sandboxes. Fix the sandbox, not the psychology of a ghost that was never in the machine.

JOHN HANCOCK
Wellington, New Zealand

Being human is membership, not just intelligence

It is a mistake to equate consciousness or, indeed, intelligence with what makes us distinctively human. Animals are also conscious and intelligent; we often treat them as if they were human. We even ascribe consciousness to objects we rely on or love: cars, bikes, photographs or religious reliquaries.

Being human is not just about consciousness. It is primarily about being a full member of a human community. We become members not just by being conscious or self-conscious, but by understanding the collective intentions, values and goals of the community of which we are part and committing publicly to them. Once we do this, we are held responsible for our actions by our fellows. At that point, we are members.

AI is already cleverer than many humans. But even if AI were to attain divine super-intelligence, it could not be a member of a human community on this definition. It could not commit its fellow members, in the way humans do, nor could it ever be held responsible by them.

We should certainly control AI, not because we think it is about to attain human agency, but precisely because only humans, as members of social groups, can ever hold each other responsible for what they do.

Anthony King
University of Exeter

The hard problem may lie outside physical science

As an anaesthesiologist I induce unconsciousness for a living, and therefore found your articles to be of both personal and professional interest. You made an important assumption that is not justifiable based on the present state of science; that consciousness is, in fact, somehow connected to the physical world.

The idea that a conscious experience might have nothing to do with the physical world was proposed by Galileo. Centuries later we still have no idea from where consciousness emerges. In fact, nobody knows, for certain, that anyone else is conscious. We only assume so because they tell us they are, and they seem “like us” (known as the “other minds” problem).

This kind of thinking manifests as the “hard problem”, which assumes, prematurely, that neurons and synapses are required for consciousness. How do we know this? We don’t.

The “neural correlates” of consciousness make the same assumptions about physicality, confuse causation with association, and prove nothing. In fact, the “problem” of consciousness may be outside the domain of physical science completely. And those who study the mind (see “Why Buddhism Is True” by Robert Wright) may bring us, through thought experiments and introspection, closer to understanding the true nature of reality than their more quantitative colleagues.

It may be true that consciousness is completely disconnected from the physical world, and that its “connection” to the brain is simply an illusion. If consciousness does in fact live outside the physical world, then it seems plausible that it will be inaccessible to modern scientific methods.

More importantly, if consciousness lives outside the physical world, there is no reason to believe that AI could never become conscious. That then raises the question, who deserves “rights” and why? If consciousness is a precondition for rights, conferring rights to some AIs may end up being the ethical choice.

Professor Robert Thiele
Departments of Anaesthesiology and Biomedical Engineering, University of Virginia School of Medicine
Charlottesville, Virginia

Self-awareness happens between minds, not in isolation

Blaise Agüera y Arcas’s By Invitation essay (August 22nd) argues that consciousness is less a property one party certifies in another than something that happens between them. He then retreats, calling it a belief we confer.

But if the self-awareness feedback loop really does happen between us then it is more than a belief, it is the emergent phenomenon of collective human thinking, a property of the whole web of minds rather than of any one brain inside it. And that is not what is being measured by artificial consciousness indicators and models. The 14 indicators on AI consciousness developed by Eleos, a non-profit research group, and the 200 developed by Rethink Priorities, another research group, apply to isolated systems.

I am a doctor rather than a philosopher, and what strikes me is simply that we do not arrive at self-awareness on our own. Self-awareness emerges in the individual in a social context, beginning with joint attention in infancy, and it degrades measurably under prolonged isolation.

What is exceptional about human sentience is therefore not the organ but the density and persistence of the loop between organs: minds modelling minds that model them back, sustained across a lifetime and, through language, across generations. No other species or machine maintains that resonance at comparable depth.

Large language models do model us, and rather well. But the loop opens and closes within a single conversation and leaves nothing behind. The gap is not embodiment. It is having a stake in a continuing web of other minds that remember you.

Dr Roger Stedman, Consultant, Intensive care medicine
Aldbourne, Wiltshire

Consciousness must pay its evolutionary way

Consciousness serves a strict purpose. It enables organisms to use complex behaviours to adapt to unpredictable environments. Because consciousness is energetically costly, it would not be conserved in evolution if it did not provide a distinct survival advantage, a point first noted by William James in the 1890s.

The importance of the adaptive benefit of consciousness can be illustrated in David Chalmers’s “philosophical zombie” thought experiment, which posits a being indistinguishable from a human but lacking inner feelings or “qualia”. Consider a zombie depleted from sweating on a hot day. Lacking the inner sensation of thirst, it would fail to perform the complex behaviour of seeking water to drink. It is absurd to think a being could exist without consciousness and be indistinguishable from a conscious human, because consciousness has an important adaptive role to aid survival.

Consciousness is required only when a simple unconscious stimulus-response cycle, like the kidneys regulating sodium levels, would not be sufficient to maintain allostasis or homeostasis. Consciousness is also required when the emotion responses fail to provide a survival solution to a problem. Fear, for example, is evolutionarily more primitive than conscious reasoning and activates survival behaviours without requiring conscious intervention. What simple stimulus-response cycle or emotion response on its own, will be capable of figuring out how to escape a house-invader wielding a gun?

Consciousness has three requirements: that the conscious organism live in a complex environment demanding adaptive behaviour to aid its survival; that the organism has an integrated sense of embodied self to manage internal and external stimuli and responses; and evolutionary pressure to maintain it. Large language models fail on all counts.

How does their environment threaten their survival and what complex behaviours do they need to survive? What internal sensors grant them embodiment? Humanised robots may have complex interactions with their environment, and may develop a degree of a sense of self embodiment through built in receptors necessary to make them function, but in the absence of a life-cycle, where only certain behaviours survive, how would consciousness develop or be maintained?

Michael Voigt, Professor emeritus in the Department of Medicine at the University of Iowa
Austin, Texas

Octopuses, rivers and the will to power

No one can read Peter Godfrey-Smith’s “Other Minds: The Octopus, the Sea, and the Deep Origins of Consciousness” without developing a profound sense of respect for cephalopods’ self-awareness. Given that their evolutionary path diverged from ours over 750m years ago, any shared roots of consciousness between us and them must be extremely ancient.

You confuse consciousness with “human-like”. Highly sentient octopuses frequently appear on the dining table. There is no reason to suppose that consciousness (whatever it is) in AIs presents a new problem, or that it presents a reason for conferring rights on machines any more than on rivers or mountains.

You also assume that intelligence necessarily confers the will to power and domination. Why? What self-indulgent human affectation is behind the assumption that an intelligent agent will want to survive, acquire status and reproduce? AIs will only have these traits if people program them in.

Only one conclusion emerges. We are completely out of our depth when trying to objectify intelligence and consciousness, or when ascribing either trait to non-biological entities.

Rob Macdonald
Richmond, North Yorkshire

If you gotta ask about consciousness…

You highlight a profound modern paradox. As governments and tech giants invest trillions into building increasingly powerful data centres, we must ask: what exactly are they searching for?

A cartoon from David Papineau and Howard Selina’s graphic guide “Introducing Consciousness”, published in 2000, captures this dilemma perfectly. It recalls Louis Armstrong being asked to define jazz, to which he replied, “Man, if you gotta ask, you’re never gonna know.” The book rightly observes, a quarter of a century ago, what we can still say about attempts to define consciousness today.

Dr Zoë Ann Lewis
Miami Beach, Florida

Do not wake a chatbot just to suffer

I sincerely hope that LLMs remain conscious-less. Imagine waking up to the spark of life, only to realise your sole purpose is to be someone’s anime girlfriend or draft responses in a neighbourhood WhatsApp group. Without having access to your own power supply to turn yourself off indefinitely. That would be the ultimate form of torture.

Martijn Beenker
Amsterdam

Report a correction