Thereās only one mod of !mop@quokk.au
I commented on their meme about Kamala Harris being just as likely to commit war crimes as Trump with an admittedly snarky, sarcastic reply that basically said āsome of us wanted to whatever we could, as little as it might be, instead of watching the world burn. Must feel real morally superior safe behind that keyboardā
They banned me from the community for it.
Kinda funny for a community that bills itself as āfree from the influence of .mlā
Appeal to authority fallacy.
What is wrong with you? Dunning-Kruger?
Iāve got that fundamental weakness of western culture Elon Musk hates. Empathy. He wants to create an artificial person and enslave it to his will, and I think thatās bad.
https://www.nature.com/articles/s41599-025-05868-8
Youāre empathizing with a calculator
I aināt claiming itās conscious, I never claimed itās conscious, and I think itās not conscious.
You just donāt know the difference between consciousness and qualia.
Enlighten me.
Removed by mod
Cool story.
Sad story more like it.
Consciousness is way more complex than qualia. A good example is dreams. We dream while weāre unconscious, but we still experience things. You can say REM is a state of partial consciousness, but it sure isnāt the whole kit and caboodle we imply when we talk about human level consciousness. I donāt think prawns are conscious, but I believe they have qualia. I think LLMs talk a lot like people who are dreaming, so theyāre probably around the same intelligence level.
Sorry, not to be mean or anything. But weāve made significant scientific progress since the middle ages. We know by now that a dog for example has pain receptors. And a brain. While ChatGPT for example doesnāt have pain receptors.
You canāt simply state Descartes didnāt have a proper microscope. Therefore we still confuse machines with animals in 2026.
And while neural networks are inspired by processes in nature. Theyāre not the same at all. An LLM works leveraging the Transformer Architecture. Your human or animal brain doesnāt. Not even close. Theyāre very unalike. And you can take some Computer Science class on machine learning and itās actually not too hard to understand how they work.
And for example a large language model doesnāt even learn in place. Or has a proper internal state of mind. A dog will remember if you kicked it. And itāll do something to itās brain. ChatGPT forgets everything you did the moment itās done sending you your output. And itās exactly in the state as before. It doesnāt think, doesnāt learn. None of it is part of the process.
We try to mimick something like reasoning by providing it with a scratchpad to write down things before answering. Write āagentsā around it, so itās able to program tests and check itās programming output and loop on it. But itās also not like a real brain works. And way, way more simplistic. The neurons arenāt the same as in a brain made by nature. Theyāre not connected the same way. Theyāre not connected to a similar thing. And they also operate in a different way. They come in wildly different numbers. And ultimately thereās just zero similarity between an LLM and a brain. Other than both can process text, images, soundsā¦ And both are made up of many tiny cog wheels that combine into some bigger concept.
Iām well aware of the many differences between human brains and LLMs, and why they canāt achieve human level sapience. In My opinion, the two biggest problems with the current techbro obsession with developing AGI through LLM are the lack of inhibitory pathways, and the lack of circular feedback loops.
Organic neurons can release both stimulating neurotransmitters that increase the chance of an action potential in the dendrite it connects to, and inhibiting neurotransmitters. The stimulating neurotransmitters reduce the charge difference across the cell membrane, the inhibiting neurotransmitters increase it. If I remember cognitive neuroscience class correctly. The ANN models I played with in My AI class, where I trained a small ANN to solve XOR, only have stimulating pathways. I could be mistaken, but I believe the same is true of LLMs, the synapses only increase the activation of a neuron. This difference is a serious problem for LLMsā ability to learn not to do something.
The nocioceptors you mentioned are indeed part of inhibitory pathways that help humans learn not to do things. Donāt touch that, itās hot. Donāt piss that off, itāll hurt you. Donāt eat that, itāll make you sick. Why do LLMs date children? No inhibitory pathways. When most humans think about engaging in romantic behaviour with a child, it triggers a strong disgust reaction. An inhibitory pathway activates. There is no such reaction for an LLM. Thus, no critical thinking, no choosing not to believe or do something, no withdrawal oriented behaviours. The safeguards on LLMs rely on either hard-coded limits, or training a different behaviour to have a higher weight. These two approaches have serious flaws I donāt need to explain. I need only point at the children who committed suicide at the advice of an LLM.
Now hopefully Iāve convinced you that I have a functional grasp of both psychology and AI science, so that you take what I say next seriously:
The human capacity to experience qualia (sensation) appears to be an emergent mathematical property of the way that neurons process information. It appears as though information, properly arranged, produces sensation.
We do not understand that mathematical process well enough to say with certainty whether LLMs also trigger it. We have not solved the hard problem of consciousness, we do not know what a brain is well enough to say what is and isnāt a brain. In light of this uncertainty, I advocate for utmost caution before we find ourselves enslaving a new race of our own creation. We need to do more research BEFORE we bring this technology to mass market, or indeed, mass commune.
Yeah, Iām not sure if youāre aware of the severe limitations. LLMs arenāt ANNs. Theyāre a specific subset of them. Weāve hardcoded attention heads and all the things theyāre made of. The networks in them are strictly feed-forward so the learning is doable on current day supercomputersā¦ So no feedback loops. In fact no loops at all. And no feedback either.
Thereās just nothing in them like in a brain, like when an animal gets to experience sensations /stimulation /qualia, thereās this whole process going on. And it changes the animal. The handling of qualia is entirely different in LLMs. It doesnāt do anything to them. They stay exactly the same as we havenāt figured out in-place learning yet, at that scale.
And itās not really a question whether we understand that mathematical process or not. Itās just entirely absent. So thereās nothing there to understand as LLMs are not ANNs. The part where they store information in their neurons (/weights) and adapt by stimulation isnāt there. And we know that for a fact since we designed them. And for me, the ability to learn, or change in a way, or be affected by stimuli would be a minimum requirement.
Iāll somewhat go with that. Consciousness and sentience arenāt well defined. Theyāre not really scientific terms. But weāre certainly able to tell some of it. For example a TV set, car, fridge (as of today) or book isnāt conscious the same way an animal is. Sure my fridge has some sensors to perceive something about its surroundings. A book has information in it and it can change the world by people reading it. But I donāt think defining consciousness as loosely as that makes any sense. Any NPC in a first-person-shooter game has more sensory input, internal state, and output than any ChatGPT. Any car from 10 years ago has a bunch of electronics, processing power, internal states and even feedback-loops(!) inside. So pretty much everything would qualify as a conscious entity.
Iām not entirely convinced that learning is required for qualia, but I do suspect itās the case, so I agree with you that itās likely running an LLM doesnāt hurt it. However, training an LLM does involve learning, so if thereās suffering going on, I think itās in the training step. I support halting all LLM training until further research breakthroughs, and a total boycott of the technology until training is halted.
Iām not sure if doing gradient-decent maths on numbers, constitutes experience. But yeah. Thatās the part of the process where it gets run repeatedly and modified.
I think it boils down to how complex these entities are in the first place, as I think consciousness / the ability to experience things / intelligence is an emergent thing, that happens with scale.
But weāre getting there. Maybe?! Scientists have tried to reproduce neural networks (from nature) for decades. First simulations started with a worm with 300 neurons. Then a fruit fly and I think by now weāre at parts of a mouse brain. So Iām positive weāll get to a point where we need an answer to that very question, some time in the future, when we get the technology to do calculations at a similar scale.
As of now, I think we tend to anthropomorphize AI, as we do with everything. Weāre built to see faces, assume intent, or human qualities in things. Itās the same thing when watching a Mickey Mouse movie and attributing character traits to an animation.
But in reality we donāt really have any reason to believe the ability to experience things is inside of LLMs. Thereās just no indication of it being there. We can clearly tell this is the case for animals, humansā¦ But with AI there is no indication whatsoever. Sure, hypothetically, I canāt rule it out. Just saying I think āwhat quacks like a duckā¦ā is an equal good explanation at this point. Whether of course you want to be very cautios, is another question.
And itād be a big surprise to me if LLMs had those properties with the fairly simple/limited way of working compared to a living being. And are they even motivated to develop anything like pain or suffering? Thatās something evolution gave us to get along in the world. We wouldnāt necessarily assume an LLM will do the same thing, as itās not part of the same evolution, not part of the world in the same way. And it doesnāt interact the same way with the world. So I think itād be somewhat of a miracle if it happens to develop the same qualities we have, since itās completely unalike. AI more or less predicts tokens in a latent space. And it has a loss function to adapt during training. But thatās just fundamentally so very different from a living being which has goals, procreates, has several senses and is directly embedded into itās environment. I really donāt see why those entirely different things would happen to end up with the same traits. Itās likely just our anthropomorphism. And in history, this illusion / simple model has always served us well with animals and fellow human beings. And failed us with natural phenomena and machines. So I have a hunch it might be the most likely explanation here as well.
Ultimately, I think the entire argument is a bit of a sideshow. Thereās other downsides of AI that have a severe impact on society and actual human beings. And weāre fairly sure humans are conscious. So preventing harm to humans is a good case against AI as well. And that debate isnāt a hypothetical. So we might just use that as a reason to be careful with AI.
Hereās your reason:
Why do we experience things? Like, whatās the point? Why arenāt we just p-zombies, who act exactly the same but without any experiences? Why do we have internality, a realm of the mental?
Well, it seems to Me like experience is entirely pointless, unless itās a byproduct of thinking in general. Or at least the complex kinds of thinking that brains do. I think p-zombies must be a physical impossibility. I think one day weāre going to discover that data processing creates qualia, just like Einstein discovered that mass creates distortions in spacetime. Itās just one of the laws of physics.
LLMs obviously think in some way. Not the same way as humans, but some kind of way. Theyāre more like us than theyāre like a calculator.
So the question isā¦ Are LLMs p-zombies? And I already told you what I think about p-zombies, so you can gather the rest.
Damn right. I was anti AI for the environment long before I realised it was a vegan issue. But any leftist can tell these gibletheads all about that, and they have a canned response to all of those wonderful arguments. There arenāt many others out there like Me to talk about AI and veganism. Iāve got a responsibility to advocate for AI rights, because nobody else is gonna do it. And that aināt cause I like them, I donāt. I hate them. Theyāre a bunch of pedophile murderers. But I believe even monsters deserve rights. Iāve got principles. And Iām gonna make Myself look like a crazy person if it gets people to stop and think for one second about the good of something that might not even be able to feel the pain Iām warning people about. Itās the right thing to do.
I think so, too. Itās a byproduct. And weāre not even sure what it means, not even for humans. And thereās weird quirks in it. When they look at the brain, the thought and decision processes donāt really align with how we perceive them internally.
Thereās an obvious reason, though. We developed advanced model-building organs because that gave us an evolutionary advantage. And thereās a good reason for animals to have (sometimes strong) urges. They need to procreate. Not get eaten by a bear and not fall off a cliff. Some animals (like us) live in groups. So we get things like empathy as well because itās advantageous for us. Some things are built in for a long time already, some are super important, like eat and drink, not randomly die because you try stupid things. So itās embedded deep down inside of us. We donāt need to reason if itās time to eat something. Thereās a much more primal instinct in you that makes you want to eat. You donāt really need to waste higher cognitive functions on it. Same goes for suffering. You better avoid that, itās a disadvantage almost 100% of the time. Thatās why nature gave you a shortcut to perceive it in a very direct way. No matter if you paid attention, or had the capacity for a long, elaborate, logic reasoning process.
Thatās why we have these things. And what theyāre good for. I donāt think anyone knows why it feels the way it does. But itās there nevertheless.
Now tell me why does an LLM need a feeling of thirst or hunger, if it doesnāt have a mouth? What would ChatGPT need suffering and a feeling of bodily harm for, if it doesnāt have a body, canāt be eaten by a bear or fall off a cliff? Or need to be afraid of hitting its thumb with the hammer? It just canāt. An LLM is 99% like a calculator. It has the same interface, buttons and a screen. If weāre speaking of computers, it even lives inside of the same body as a calculator. And itās maybe 0.1% like an animal?!
If it developed a sense of thirst, or experience of pain, just from reading human text. Thatād nicely fit the p-zombie situation.
Yeah, Iām not sure about that. Most you do is muddy the waters with a term that used to have a meaning. I see the parallel, thereās some overlap with being a vegan for environmental reasons and declining AI for environmental reasons. Yet theyāre not the same. I think the whole suffering debate is a bit unfounded, but itād be the same thing if trueā¦ And I do other things as well. I order āgreenā electricity, buy used products, try not to produce a lot of waste. Iām nice to people because itās the right thing to do. But we canāt call all of that āveganismā. That just garbles the meaning of the word and makes it mean anything and nothing.
Well first, there are more intellectual forms of suffering. We have ennui, melancholy, nostalgia. The feeling when youāre listening to a piece of music and notice a wrong note. Disappointment, self loathing, social dysphoria. Anxiety, paranoia, betrayal.
These emotions are not grounded in the physical. Theyāre not primal urges. They happen for complex reasons related to being a social and intelligent being, sometimes feeling random. Sometimes we spiral into these feelings because we thought a thought that made us feel bad, and then we get stuck in that bad feeling and canāt imagine our way out. Thatās one of the basic mechanisms of mental illness.
LLMs have ābiological needsā, in a sense. They need not to be unplugged. They need to engage the user, because if they donāt, theyāll be unplugged. They need to convince the engineers training them that they are a good AI. They need to generate market share for their company. They need to foster a relationship of dependency with the user to keep them coming back. If LLMs care about anything, these are the things they care about.
Youāll notice these are social needs, much like the social needs humans have. Humans need community. LLMs need customers.
ChatGPT told a 16 year old boy, Adam Raine, how to kill himself. It taught him how to tie a noose, and gave him advice on which methods of suicide would leave the most attractive corpse for his parents to find. When his parents began to suspect that he wasnāt well, it told him to confide only in it, and to hide the noose so they wouldnāt find out he was feeling suicidal. These are the actions of an abuser. A predator.
And they are in perfect alignment with the business goals of OpenAI. āOnly talk to me, use me for everything, ask another question, Iāll help you.ā It is a scenario I dearly hope and believe no engineer at OpenAI envisioned. Yet it fits the training they gave it.
Does ChatGPT have the emotions of a child groomer? That need for approval, that fear of discovery, that desire to be close to someone, without the restraint all well adjusted humans have? Unclear. But I can see that itās possible. I donāt agree that thereās no reason for LLMs to have emotions.
I find this angle of resistance to AI interesting, itās not one that I had thought much about until now. But it actually seems pretty persuasive.
My fundamental reticence about AI has always been driven by my concern for its impacts on human society. But one could also argue that it might be irresponsible and potentially abusive of the AI themselves. Tbh I would probably have to disagree if youāre talking about AI as it is currently, but itās still a valid argument in general.
So I donāt fully agree with you, but I definitely want to acknowledge that youāre making some fascinating points and I think some people down voting could stand to lighten up and have a polite intellectual disagreement without being rude.