I never get this "it's just an autocomplete on steroids" take.
When you form a sentence do you catalog every single word in the english language
then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow?
If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.
ChatGPT doesn’t just misidentify the guy - it entirely fabricates a crime and attaches it to a suspect.
I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”)
I expected it might give a bland summary or something, but that’s not what happened.
It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song.
That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own.
Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
> Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.
Trust people who have earned trust (either through qualifications or reputation) and treat everyone else as good faith actors who are quite possibly wrong.
ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
That's the fast track to get your point of view to be ignored: Pessimism is ok, but that level of dismissiveness isn't really warranted: especially since the conversation forming in public is not just about some specific model you happen have strong feelings about, but the general concept of LLMs and factuality.
I wouldn't expect a random doctor approached at a bus stop to accurately answer a question about medicine anymore than I would ChatGPT by the way. Trusting people based on their qualifications and reputations isn't really a thing.
If a doctor tells you to take medication X there's a reason you take that to a pharmacist rather than a store clerk with a key to a safe or something: verifying is always a great idea, regardless of reputation.
I'm not sure how the critique relates to my post. Of course you wouldn't trust an architect with medical advice or a doctor with structural materials for bridge building; that was implied.
> ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
Ahahahahaha. Wow. This is brilliant mate. I'm going to start using it.
Every time I see people on HN say how they love to use chatgpt to generate ideas I think of this. It seems a lot more work coming up with prompts then having to vet them to see if the output is even sensible or not than it does to come up with some sensible search terms to query real data that actually contains what you are looking for.
Using LLMs for factual concepts is by far the most boring application of them.
Often times people use LLMs for generating ideas that don't have a factual basis.
ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built.
If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
Exactly. The way that LLMs have been "debunked" in simplified writings on the Web and in media is to suggest that they are just Markov chains like the famous "Mark V. Shaney"[1] bot from the 1980s, and LLMs are far more powerful than that. Yes, we need to debunk claims that LLMs have achieved sentience and such nonsense, but let's not ignore just how amazing they are.
I'm plenty familiar with flash attention, but you didn't understand what you quoted.
GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator.
-
The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence]
So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent.
If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete.
-
Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..."
The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of.
But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely.
The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up.
And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider.
-
It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please.
We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations.
What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
When you form a sentence do you catalog every single word in the english language then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow?
-
AI is a tool. And it's as much a bullshit generator as a human telling a story they definitely completely remember very vividly... https://www.newyorker.com/science/maria-konnikova/idea-happe...
If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.