It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future.
It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it.
There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.
Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility:
The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption.
> Trying to impute the motives of ones interlocutor is dumb and boring.
I know right? You should see the response to my point that nobody has been convinced to fly a plane into a building by an LLM. “Dumb and boring” hits the nail on the head.
> Seeing how confidently and obtusely people dismiss the risks of AI
If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that.
Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry risk.
That's a perfectly sensible position. The optimist position isn't even internally consistent. See Andreessen on Sam Harris's podcast: AI will produce consumer utopia and drive prices down. Also, there are no downside risks because AI will be legally neutered from doing much of anything.
Is it legally neutered or is it transformative? The skeptical case doesn't rely on answering this question: to the extent it's effective, powerful, and able to do good things in the world, it will also be effective, powerful, and able to do bad things in the world. The AI skeptics don't need to know which outcome the future holds.
> The AI skeptics don't need to know which outcome the future holds.
But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.
Brilliant proof of exactly my point. When it comes to discussing any possible outcome other than utopia, suddenly the power of these tools drops to zero! Remarkable :)
You responded to my previous comment that called utopians as cultish as the fanfic folks.
When it comes up discussing any possible outcome that isn’t the opinion of [cult x], the only reason that the other person disagrees is because they are in [cult y]
What? I never proposed that dichotomy and don't believe in it. You did and you're tripping over it a bit. You can just discard that model and engage with the substance of the topic, you know!
The substance of this discussion is that a language model has never convinced anyone to crash a plane into a building and your position is that pointing out that fact is to ignore the material reality of the possibility of some theoretically advanced technology (which could maybe be adjacent to language models) at some point convincing a hypothetical person to crash a plane into a building.
That is to say, the substance here is an emotional doomsday reaction to a benign statement of undisputed fact — a language model has never convinced a person to crash a plane into a building.
Watching a cell divide into two cells: “Why isn’t anyone talking about the possibility that this is Hitler??”
Why aren't you mentioning that an LLM has already convinced someone to commit suicide?
Why are you being deliberately obtuse about the unknowable-ness of the exponential curve the enthusiasts on one hand salivate over, and on the other confidently assert optimism just as you're doing?
It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it.
There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.