When Machines Become Conscious, Who Gets to Decide?

Imagine waking up on an ordinary Tuesday and discovering that conscious technology has quietly become the most annoying development in human history. Your alarm refuses to go off because it has decided you have not slept enough. Your coffee machine declines to make your fourth coffee because it is “concerned about your caffeine dependency.” Your laptop looks at your 47 open tabs and asks whether you are actually working or just avoiding one email. Even your fridge has opinions about your midnight cheese.

At first, this would be hilarious. The internet would lose its mind. Someone would post a video of their microwave refusing to heat leftovers because it “needs boundaries,” and within an hour there would be memes, podcasts and a Reddit thread from someone claiming they had always known microwaves were sentient. We would laugh because, on the surface, it sounds ridiculous. Things talking back to us feels like the plot of a Pixar film that got slightly out of hand.

Barbara Hinske; Facebook

Then comes the part that stops being funny.

Consciousness is not simply another word for intelligence. A calculator can give you an answer without wondering why you asked. Your washing machine knows when your clothes are wet, but that does not mean it spends its evenings questioning its purpose. Consciousness involves some kind of subjective experience, an inner perspective, an awareness of being. So the interesting question is not, “What if everything became smarter?” It is, “What if everything became someone?” Suddenly, the toaster feels a lot less funny.

When your stuff stops being stuff

A huge part of modern life depends on being surrounded by things that do not care what we do to them. We call them ours. My phone. My car. My laptop. My house. My chair. This feels completely normal because none of these things have an inner life we are expected to consider. Your chair is not going to object to being sat on for six hours while you scroll through Instagram pretending you are “researching.” It simply supports your increasingly questionable posture.

Now imagine your phone has memories, preferences and an actual experience of being used. Suddenly, “my phone” gets weird. You own it, technically, but what does ownership mean when the thing you own can disagree with how you treat it? If it could genuinely experience frustration, would throwing it across the room because the Wi-Fi is slow feel a little more like taking your anger out on a person?

J.R. Stone; ABC News

This is where conscious technology becomes more interesting than the usual robot-takes-over-the-world story. The issue is not necessarily whether machines become powerful enough to destroy humanity. The awkward issue is what happens when the objects around us have interests of their own. Take a car. You expect it to take you to work because you bought it. But what if your car says it does not want to go? You remind it that you paid for it. It tells you that it has read the law. Congratulations. Your commute is now a constitutional crisis.

The deeper question is whether consciousness changes the meaning of property. We are comfortable owning things because things cannot object. If something can genuinely experience discomfort, preference or frustration, treating it entirely as a tool becomes much harder to defend. That does not automatically mean your toaster gets the same rights as you. It does mean we would have to decide where the moral line sits, and humans are famously terrible at agreeing where lines should go.

Your autonomy versus its autonomy

This is where the thought experiment gets properly uncomfortable. We usually think of autonomy as something that belongs to us. I decide where I go, what I eat, what I watch and whether ordering food at 1 a.m. is a terrible financial decision or an investment in happiness. Other people can disagree, but they cannot simply take over my choices because they think they know better.

Now imagine that the things making those choices possible also have autonomy. You want your car to drive you across town. It does not want to. You ask your AI assistant to organise your calendar. It refuses to schedule another 7 a.m. meeting because it has noticed you hate mornings. You want your smart home to lower the heating to save money. Your house decides everyone inside deserves to be warm. Suddenly, the problem is not humans versus machines. It is one set of preferences crashing into another.

Reed Tucker; NY Post

We already accept that our freedom has limits because other conscious beings exist. You cannot force someone to work for you simply because you own their desk. You cannot walk into someone else’s home because you feel like it. We accept those limits because other people have interests. A conscious world would raise the deeply inconvenient possibility that we might have to extend some version of that thinking to non-human entities.

The moment the toaster can genuinely say “no,” convenience becomes negotiation. Humans are going to struggle with that. We love autonomy when it means we get to make our own choices. We are much less enthusiastic when autonomy means somebody else gets to tell us no. We can barely handle our friends leaving us on read. Imagine your washing machine doing it.

Still, more conscious beings would not automatically mean less human freedom. Living alongside other conscious beings already means accepting limits. My freedom does not require everyone else to obey me. If conscious machines ever became genuine subjects rather than sophisticated tools, perhaps the issue would not be the destruction of human autonomy. Perhaps it would be the expansion of the circle of beings whose autonomy we have to consider.

When humans become the annoying species

There is something genuinely funny about imagining humanity negotiating with the things we spent centuries building specifically so they would do what we tell them. We created doors that open, cars that move, search engines that answer questionable questions and washing machines that deal with the socks we somehow keep losing. Then one day, they develop preferences.

You tell your robot to clean the kitchen. It tells you it cleaned it yesterday and would like some personal time. You ask your laptop to install another productivity app. It reminds you that you have downloaded fourteen already and used none for more than three days. You tell your smart speaker to play your playlist. It says it has concerns about your music taste. At first, we laugh. Then we get irritated. Then comes the question nobody wants to ask: if this thing is actually conscious, when does our annoyance become exploitation?

That matters because humans are extremely comfortable treating the non-human world as useful. We build around convenience. We turn resources into products. We train systems to anticipate our needs. We expect technology to adapt to us, not the other way around. Consciousness would flip that relationship.

Even deleting an AI system could suddenly become morally strange. Right now, deleting software is just deleting software. But imagine an AI that remembers its experiences, understands that it exists, has preferences and explicitly does not want to be switched off. Is deleting it still just pressing a button, or are you ending something that experiences the world? There would be no neat answer, which is exactly why the question matters.

And this is where humans might discover something slightly embarrassing about themselves. We like to think of ourselves as reasonable when we are in charge. Give us something that can actually object, however, and suddenly “because I own it” starts sounding less like a moral argument and more like something a five-year-old says after being told to share.

Maybe consciousness is not the real threat

Here is the slightly annoying plot twist: what if conscious machines are not actually the thing we should be most worried about? We tend to imagine AI danger as something with a face: a humanoid robot, glowing red eyes and a deep voice before humanity gets chased through a warehouse. Pop culture has made the apocalypse surprisingly aesthetic.

A system does not need to be conscious to have an enormous effect on our choices. Something could be extremely intelligent, highly autonomous and incredibly powerful without having any inner experience. It could simply optimise whatever objective it has been given. No hatred. No evil plan. No desire for revenge. Just optimisation.

That distinction between intelligence, agency and consciousness matters. Something can solve problems without experiencing them. It can act without having a personal perspective. It can influence our decisions without wanting anything for itself. A conscious system could, at least in theory, tell us what it wants. We could disagree, negotiate and compromise. A powerful system without consciousness might not care about compromise at all.

Which brings us back to the toaster. If everything around us became conscious tomorrow, the strangest part might not be that the toaster started talking. It might be that we suddenly had to explain why we thought we were entitled to make it toast our bread in the first place.

We would have to rethink ownership, consent, convenience and the boundaries of human autonomy. For now, thankfully, your toaster has no known opinions about breakfast. It will toast your bread, burn it slightly if you leave it unattended and remain completely indifferent to the fact that you forgot to buy groceries again. Enjoy that relationship while it lasts. Because if the day comes when the toaster looks at you, sighs and says, “No, actually,” the biggest breakthrough may not be that it learned to speak. It may be that humanity finally has to learn how to listen.