Thats the case for the “simple” neural network behind it but what if we give this neural network a whole persona/character. It feels as if the neuronal network would be similiar to the bare neuronal mass in our brain but the arrangement of it gives us our personality which forms itself through our experiences and genetics. It feels as if we can skip experiences and genetics with these networks and are able to arrange them through words/tokens which makes them behave even more like a real human. The question is if we even are that much different or if we just have invented a more efficient way of forging the power of a brain to our likings and if that is the case its worrying.

Here is the text.

chinese room 2

So there’s this guy, right? He sits in a room by himself, with a computer and a keyboard full of Chinese characters. He doesn’t know Chinese, though, in fact he doesn’t even realise that Chinese is a language. He just thinks it’s a bunch of odd symbols. Anyway, the computer prints out a paragraph of Chinese, and he thinks, whoa, cool shapes. And then a message is displayed on the computer monitor: which character comes next?

This guy has no idea how the hell he’s meant to know that, so he just presses a random character on the keyboard. And then the computer goes BZZZT, wrong! The correct character was THIS one, and it flashes a character on the screen. And the guy thinks, augh, dammit! I hope I get it right next time. And sure enough, computer prints out another paragraph of Chinese, and then it asks the guy, what comes next?

He guesses again, and he gets it wrong again, and he goes augh again, and this carries on for a while. But eventually, he presses the button and it goes DING! You got it right this time! And he is so happy, you have no idea. This is the best day of his life. He is going to do everything in his power to make that machine go DING again. So he starts paying attention. He looks at the paragraph of Chinese printed out by the machine, and cross-compares it against all the other paragraphs he’s gotten. And, recall, this guy doesn’t even know that this is a language, it’s just a sequence of weird symbols to him. But it’s a sequence that forms patterns. He notices that if a particular symbol is displayed, then the next symbol is more likely to be this one. He notices some symbols are more common in general. Bit by bit, he starts to draw statistical inferences about the symbols, he analyses the printouts every way he can, he writes extensive notes to himself on how to recognise the patterns.

Over time, his guesses begin to get more and more accurate. He hears those lovely DING sounds that indicate his prediction was correct more and more often, and he manages to use that to condition his instincts better and better, picking up on cues consciously and subconsciously to get better and better at pressing the right button on the keyboard. Eventually, his accuracy is like 70% or something – pretty damn good for a guy who doesn’t even know Chinese is a language.

One day, something odd happens.

He gets a printout, the machine asks what character comes next, and he presses a button on the keyboard and-- silence. No sound at all. Instead, the machine prints out the exact same sequence again, but with one small change. The character he input on the keyboard has been added to the end of the sequence.

Which character comes next?

This weirds the guy out, but he thinks, well. This is clearly a test of my prediction abilities. So I’m not going to treat this printout any differently to any other printout made by the machine – shit, I’ll pretend that last printout I got? Never even happened. I’m just going to keep acting like this is a normal day on the job, and I’m going to predict the next symbol in this sequence as if it was one of the thousands of printouts I’ve seen before. And that’s what he does! He presses what symbol comes next, and then another printout comes out with that symbol added to the end, and then he presses what he thinks will be the next symbol in that sequence. And then, eventually, he thinks, “hm. I don’t think there’s any symbol after this one. I think this is the end of the sequence.” And so he presses the “END” button on his keyboard, and sits back, satisfied.

Unbeknownst to him, the sequence of characters he input wasn’t just some meaningless string of symbols. See, the printouts he was getting, they were all always grammatically correct Chinese. And that first printout he’d gotten that day in particular? It was a question: “How do I open a door.” The string of characters he had just input, what he had determined to be the most likely string of symbols to come next, formed a comprehensible response that read, “You turn the handle and push”.

One day you decide to visit this guy’s office. You’ve heard he’s learning Chinese, and for whatever reason you decide to test his progress. So you ask him, “Hey, which character means dog?”

He looks at you like you’ve got two heads. You may as well have asked him which of his shoes means “dog”, or which of the hairs on the back of his arm. There’s no connection in his mind at all between language and his little symbol prediction game, indeed, he thinks of it as an advanced form of mathematics rather than anything to do with linguistics. He hadn’t even conceived of the idea that what he was doing could be considered a kind of communication any more than algebra is. He says to you, “Buddy, they’re just funny symbols. No need to get all philosophical about it.”

Suddenly, another printout comes out of the machine. He stares at it, puzzles over it, but you can tell he doesn’t know what it says. You do, though. You’re fluent in the language. You can see that it says the words, “Do you actually speak Chinese, or are you just a guy in a room doing statistics and shit?”

The guy leans over to you, and says confidently, “I know it looks like a jumble of completely random characters. But it’s actually a very sophisticated mathematical sequence,” and then he presses a button on the keyboard. And another, and another, and another, and slowly but surely he composes a sequence of characters that, unbeknownst to him, reads “Yes, I know Chinese fluently! If I didn’t I would not be able to speak with you.”

That is how ChatGPT works.

In the notes of the post, someone links to this page which is a fairly thorough overview of the Chinese Room argument and the various questions and answers that have been proposed around it: https://plato.stanford.edu/entries/chinese-room/

Don’t tell me this story is written by the same guy

Yes? this is common knowledge.