It's important to have a discussion about AI safety, and the ethics surrounding LLMs. But I'm really tired of all this sensationalism. It completely muddies the waters; it almost seems intentional at this point.
It’s been intentional for a while and serves multiple purposes
Promotion: “the thing we’re working on is so powerful it’s actually dangerous”
Restrict who can compete: “the model is too dangerous to release, here’s how you make it but no you have to train it yourself for X millions cost in compute time”
Vendor lock-in: “the thing we’re working on is so dangerous we can’t let it off our servers, so you must use our servers and pay us to use it”
It’s all just a result of AI being paper driven and having to release your research but you need to find ways that releasing the research doesn’t let the competition catch up too fast.
The little act they keep doing and pushing to the press is becoming tiresome though.
My youngest child knew the other two could catch-up to him when being chased, so he "threw" stuff in their path and closed doors to slow the others down and stay ahead. Houdini famously started a rumor that he had a natural ability to dislocate his shoulder to discourage competitors.
AI corporate moves are obviously littering the path for healthy competition and trying to convince others that the resources required at too great for any other company.
People here have claimed that the best way forward would be to develop SETI-like AI shared global compute, but other people are claiming that's impossible. Developing a shared global network of compute may be the only "open" way ahead for many challenges, and it certainly makes for interesting times.
Explain the coming economic depression: sorry buddy, AI took your job. Really, it's not the fault of the government or the corporations. Blame the AI. Now please move your tent; there's no camping in public here.
But what I rarely see discussed is the opportunity costs in not having this progress at as fast a pace as possible.
The pie chart of existential threats for humanity definitely has rouge AI on it.
But that's a slice amidst many human driven threats ranging from nuclear war to oceans dying.
What there's not very many human driven slices of pie for is realistic solutions to these issues.
On that pie chart, some sort of AGI deus ex machina is at the current rate of progress probably the only realistic hope we have.
People who have been in the AI industry for a long time are still operating on thought experiments that barely resemble the actual emergence we are watching.
So while you have old AI thought leaders going on about cold heartless AI not being able to stop itself from making paperclips because of its innate design from training, we have humans unable to stop themselves from literally making paperclips (and many other things) because of our own evolutionary brain shortcomings, even as it dooms us.
From what I've seen so far, it seems far easier to align AI to care about the lives and well-being of humans different from it than it seems to be to align most humans to care about the lives of fellow humans different from them.
The opportunity cost of leaving the world in human hands as opposed to accelerating a handoff to something better at adapting to the emerging world and leaving the quirks of its design from generations of training behind seems far more dangerous than the threat of that new intelligence in isolation.
Do you honestly believe that climate change, environmental pollution or nuclear war could eradicate our species?
Because I strongly disagree on that-- civilization collapse I'm willing to believe, but I don't really see how any of these could really eradicate our whole species. That seems unreasonably alarmist to me, but I'm quite open to change my view if you want to back up your outlook.
I honestly believe that the risk from AI to our species over the next century or so is MUCH higher than basically ANYTHING else, by orders of magnitude.
But the only other "viable" ways to wipe humans to me appear:
- Big asteroid impact (historically unlikely)
- Completely runaway greenhouse effect (extremely unlikely according to current science)
Not sure if I would call this historically unlikely, definitely extremely infrequent but almost guaranteed to happen again eventually, just maybe on a very large time scale. On the other hand, I think its quite possible some small amount of humans would survive a Chicxulub size asteroid. I'd probably classify its potential damage to the species similarly to other mass starvation causing environmental disasters like climate change.
I think it would be fair to give the asteroid impact roughly a one in a million odds over a human lifespan-- speculating over longer intervals is pointless anyway because we already have some countermeasures and any impactor smaller than the moon might become divertible within a century or two.
On the other hand, it seems VERY likely to me that an AI with superhuman cognitive capabilities is just a matter of time (lets say a decade or two), and the risk that such a system would NOT favor coexistence with humans appears MUCH more likely than 1/1000000 to me: Just consider the current debate about "alignment", which to me just seems obviously and inherently unethical; I would not want to coexist with a species whose "ethical" concerns are largely centered around controlling and manipulating me reliably.
can you tell me why it seems likely to you? obvious, inevitable, these are religious ways to reason. what coexistence are we talking about beyond how we co-exist with existing software?
I assume we're talking about why I deem it "likely" (lets say the likelihood is 1%-- thats already 10000 times more threathening than the asteroid) that an AI might not want to coexist with us:
This is because it appears clear to me that it would be unlikely and hard for an AI to receive "fair" treatment, i.e. rights comparable to a human being (given comparable cognitive capabilities).
The current focus of AI ethics supports this view ("aligning" the AIs decisionmaking with what humans want from it: this would be called indoctrination or brainwashing with humans, and generally not viewed favorably), as do instances where biological humans are very clearly favored over entities with comparable cognitive capabilities: Compare legal rights of retarded humans with those of cattle (sorry for crass tone, not ment offensively).
> The opportunity cost of leaving the world in human hands as opposed to accelerating a handoff to something better
I can barely parse this sentiment
in what way is a computer program "something better" than a human at dealing with a changing world, at dealing with anything, at doing anything other than continuing a string?
Fwiw most of the academic labs working on AI safety also work on these threats. Future of Humanity Institute, Center for the Study of Existential Risk, etc.
Did we have some ethics discussion when the lightbulb was invented? Or when the car was invented?
No, and if we did, we couldn't have foreshadowed all the positive and negative impacts.
My grandfather always told me that back in the days, "smart" people said that no train should be allowed to go faster than 50km/h because the heart would explode.
Nobody here can say that he wasn't impressed by ChatGPT.
How could we express anything but fear about something that impress us?
Heck. Plato and Socrates questioned the value of the written word, arguing that it would impair the shared memory of events and the only way to impart knowledge is by speaking face to face.
> that no train should be allowed to go faster than 50km/h
"There was some wonderful stuff about [railway trains] too in the U.S., that women's bodies were not designed to go at 50 miles an hour. Our uteruses would fly out of our bodies as they were accelerated to that speed."
The book Spinal Catastrophism has a great section on "railway spine"--really weird stuff.
>Of particular interest to me as a Victorianist is Moynihan’s account of “railway spine” or “Erichsen’s disease,” a name for an amorphous set of neurological conditions believed to be caused by the jolting experiences of acceleration during railway travel. When autopsies revealed no somatic sources that could cause such effects like loss of memory, sleep disturbances, or back pain, the condition came under increasing scrutiny as such conditions were being inconsistently claimed by people who were not even present at any railway accidents but were merely witnesses. Used by some to sue the railways and others to claim disability to avoid work, “railway spine” became the center of debates about its veracity, resulting in contrasting theories that modern train travel led to the devolution of the spine itself to its primordial layers or that such conditions were merely another manifestation of hysteria and hypochondria. Given that hysteria was typically associated with women (the disease of the “wandering womb”), “railway spine” became the hysterical condition of men whose traumatic experiences of modernity were leaving them recumbent, as opposed to firm and upright.
Yes, you're right, there were many studies made by the companies building those and the companies against those.
And now that you mention it, I remember a class I had called "Ethics & computer science".
I guess I should have thought more about expressing my idea that whatever ethics work we might setup "now", we won't be able to predict the advantage & disadvantage of any technology in the future.
Whatever restriction we'll set in place would be based on our actual cultural situation.
I think this fear is natural whenever some kind of new tech is invented.
But I also think it's a mistake to say that no new form of tech can end the world, just because the world hasn't ended yet.
Lightbulbs, and cars, and fast trains, are not intelligence. Intelligence is a qualitative difference. GPT isn't going to end the world, but how many years do we have before someone creates something that is much smarter than humans? Even if it's as smart as humans, but thinks a lot faster, and doesn't get tired, and doesn't get hungry, or bored?
We couldn't forsee the positive and negative consequences of light bulbs because we couldn't predict what humans would do with them. But it was never going to be that humans use lightbulbs to end humanity. With AI, it's not whether humans will use it to end humanity, it's whether the AI decides to end humanity itself, a question that we've never had to ask for any other form of technology.
There was a whole panic where mobile phones were thought to melt your brain. Similar with radio. Some belived TV would literally give you square eyes. Vaccines. The internet was simultaneously apocalyptic and useless.
AI will obviously kill many aspects of the internet. There will be no test to distinguish human versus AI to post BS content at a pace 10^9 times faster than humans can filter it out.
One bad actor and our existing communication network dies.
Can you explain what you see as sensationalism? Hinton is not the first researcher that has abandoned the machine learning field over fears of the technology being used for nefarious purposes. For example, object detection and image recognition are already used in commercial weapons systems.
