Most arguments that AI can't really reason/think/invent essentially reduce to defining these terms as things only humans can do. Even if you had an LLM-based AGI that passes the Turing test 100% of the time, cures cancer, unites quantum physics with relativity, and so on, many of the people who say that ChatGPT can't reason will keep saying the same thing about the AGI.
I don't think there's anything wrong with people trying to see what, if anything, differentiates ChatGPT from humans. Curing cancer etc. is useful, as is ChatGPT, regardless of how it achieves these results. But how it achieves them is important to many people, including myself. If it's no different from humans, then we need to treat it like a human---well no, strike that, we need to treat it _well_ and protect it and give it rights and so on. If it's a fancy calculator, then we don't.
I don't think there's anything wrong with it either. It's an important debate. I just think the arguments usually become very circular and repetitive. If there's nothing an AI could ever do to convince you that it's thinking or reasoning, then really you should be explicit and say "I don't believe an AI can produce human thought or human reasoning" or "an AI is not a human" and nobody will disagree with you on those points.
> and nobody will disagree with you on those points
But that's the point, they do. Even on HN there are many comments saying that humans are just fancy autocomplete, i.e. there's no fundamental difference between humans and LLMs.
tines says>"Even on HN there are many comments saying that humans are just fancy autocomplete, i.e. there's no fundamental difference between humans and LLMs."<
LLMs'may prove a useful analogy as to how parts of human intelligence operate, an analogy that, at the very least, should be thoroughly researched.
"there's no fundamental difference between humans and LLMs."
I think that's a straw man. No one disagrees that humans and LLMs produce cognition differently. One uses a wet, squishy brain. The other uses silicon chips. There's no disagreement here.
My point is that's not a debate anyone is having. No one claims that ChatGPT is human! The claim is merely that ChatGPT is engaging in (non-human) forms of reasoning, abstraction, creativity, and so on, with varying levels of ability.
There's a separate debate on whether the brain produces human thoughts in a similar way to ChatGPT's non-human thought. The question here is whether brains are essentially biological LLMs, and whether GPT's current limitations relative to humans could be overcome simply by scaling up the number of GPT's parameters to match or exceed the number of neurons in the human brain. But whether or not that turns out to be the case, it would not mean that AIs are the same as humans, or use exactly the same processes to think and reason, and no one is claiming that.
The word "thought" means something. When you use it to describe ChatGPT, you have in fact argued "there's no fundamental difference between humans and LLMs."
That presupposes that the only thought that exists or can exist is human thought. You can define it that way if you like, but it’s not the only definition.
I'm not saying the only thought that exists is human thought. (I believe animals can think).
I'm saying using a word invented to describe animals behavior, "thought" to describe a large language model has no discernible meaning other than you think it works like an animal brain.
If you think it's an open question whether it works like an animal, you should find a better word than "thought".
A CPU "runs". A disk "seeks". An OS stores data in "memory". Re-purposing terms to describe new concepts is routine in the evolution of language, and (non-human, non-biological) "thought" is a perfectly apt way to describe what we can observe in the output of massive LLMs like GPT.
A CPU "runs" but nobody debates whether a CPU has legs and leaps around.
A OS stores "memories" but nobody argues whether the OS fondly recalls the day their son was born.
When people are debating how the LLM works and you call what the LLM does "thought" it's perfectly reasonable to conclude you believe the human brain works essentially the same as an LLM.
I just asked ChatGPT 4.0 and here's my question and the response, in the interest of full disclosure I edited out the part that I thought wasn't relevant so as not to bore:
"...if I call your processes "thoughts" is that the correct term? If not what term should I use?"
"If you want to describe the processes of an AI language model like me, it's more accurate to use terms such as "information processing," "pattern recognition," or "data-driven predictions." While these terms don't fully capture the complexity of the underlying algorithms, they are more appropriate than "thoughts," which implies consciousness and self-awareness that AI models like me do not possess.
Using the term "thoughts" might be misleading, as it could imply that the AI has a subjective experience or a sense of understanding, which is not the case. Instead, I rely on advanced algorithms to generate responses based on the patterns and associations learned from the data during my training."
So ChatGPT doesn't state you used the correct term.
I genuinely wonder if you think ChatGPT is consciouss and self-aware and you used a word that implied that intentionally, or if you just like how the word "thought" sounds and are indifferent to what people think you are implying.
I don’t think it’s conscious. Though at some point in the future, it will likely be hard to say that with total certainty.
Perhaps you’re right that the term thought has too much baggage. I’m just saying that if you look at it at naively, it’s engaging in forms of abstractions, reasoning, world modeling, invention, and so on that seem a lot like “thought”. If a human told you they were doing those things, you’d say they were thinking, right?
I agree it’s not thought in exactly the way that we are used to using the word, but I think it can be classified as a type of thought.
