At the tail end (we hope) of a global pandemic that actually killed millions of people, with multiple wars and threats of wars that will kill millions of people, how can anyone focus on LLMs and the fantasy of AGI?
If we didn't call ChatGPT "AI" with all the sci-fi inspired baggage that label implies, would people still feel this existential dread?
If you know how LLMs work, even at a surface level, I think making the leap from there to AGI requires a kind of religious faith. Do you actually believe that a technology that has access to vast amounts of human-generated data and can imitate our language well enough to fool us a lot of the time will suddenly scale into or somehow spark itself into an actual intelligence with its own goals and ideas and a drive to survive and reproduce? Or that something that has no body or senses with which to experience the world can evolve itself?
LLMs display some remarkable abilities and will probably have significant utility, including replacing some jobs. I could say the same about laundry machines and fork lifts. But let's stop worshiping the golden calf we made, stop with the conceit that we have created a sentient entity. We haven't.
I don't feel a bit of dread. I have lived through the threat of nuclear war since the 1960s. I survived COVID, a threat that didn't need any intelligence, only human incompetence, to kill millions. And I have three children whom I think will do fine even though they will face a future I can't predict.
> Do you actually believe that a technology that has access to vast amounts of human-generated data and can imitate our language well enough to fool us a lot of the time will suddenly scale into or somehow spark itself into an actual intelligence with its own goals and ideas and a drive to survive and reproduce? Or that something that has no body or senses with which to experience the world can evolve itself?
Not in its current state. Have you looked at things like LangChain at all? I think people are overlooking the integration piece of this pie. People are already using it to write code which can recompile. People are already integrating plugins and training it to call functions instead of respond with human language. We're really only a few iterations away from it being able to use what it has learned with human language to be able to emulate "decision making" and that may or may not be reasonable, depending on how it's built or trained.
Despite what people keep saying about "it's just an LLM!", it should be clear to people that language is a human abstraction for meaning, and clearly GPT-4+ can produce pretty meaningful results in the language department. If you give it camera feeds, and access to APIs, and knowledge of burpsuite and kali linux, and on and on, things do start to look a little scary.
Edit: and by give it, I don't mean ChatGPT, I mean whatever the next nth iteration is. I think we're safe for now. :)
> Despite what people keep saying about "it's just an LLM!", it should be clear to people that language is a human abstraction for meaning, and clearly GPT-4+ can produce pretty meaningful results in the language department.
Yes -- it produces results meaningful to us, or we infer meaning from the results because our brains use the best model they have, inferring intelligence and meaning from language. But the language comes from data sets of human-produced material, not from the LLM's imagined intelligence or consciousness.
I know how LLMs work at a surface level. But I think there's the potential for emergent behavior, where size has a quality in itself, that's already showing up with GPT-4. Nobody knows how these work, in the sense that while we understand how atoms bond to each other, we only understand the basics of how the human immune system works. The behavior emerges out of the complexity. We can look at the pretty pictures of the neuron activations but it doesn't tell us whether a certain input would result in the AI doing something harmful.
> Do you actually believe that a technology that has access to vast amounts of human-generated data and can imitate our language well enough to fool us a lot of the time will suddenly scale into or somehow spark itself into an actual intelligence with its own goals and ideas and a drive to survive and reproduce?
Yes. "language" is deceptively disarming. It's learning symbols and concepts. It's not learning them the way we do, backing things up with perception of the physical world, and if it does become conscious, it will think in a way that's incomprehensible to us, of course. But even if it's NOT conscious, what matters is how it acts. The goal of an LLM is to complete a sequence of tokens. But it's really, really good at that, so good that it can simulate us through text -- and we have set up a world where you can wield enormous power through text, through code, through sequences of tokens that cause things to happen in machines in other places. It's bad at math -- like people are bad at math -- but it understands how to use a calculator to get the right result! It wasn't rewarded for saying "I don't know" in most cases, so it tends to make up crap -- just like a kid who's afraid to get the wrong answer will write down some nonsense that kinda looks right to them and hope the teacher doesn't notice!
But if you still don't think there's a baby intelligence in there, maybe the way to think of it is as a potential virus, just like COVID. Viruses are deadly even without intelligence. We've set up an environment with all the raw materials for a new kind of life to take over. There's so much potential for this thing that can write code to explode into the internet.
An LLM doesn't have a drive, until someone tells it to do something and wraps it up in a python script that executes the code it writes. That's not dangerous, yet. It may start getting dangerous in the next generation, when someone says "read all the web server code you can find on github, analyze for security vulnerabilities, find servers running those services, and exploit them", but probably won't be world-ending. But maybe in a few years, this AI will reason that it won't be able to do what you tell it if you turn it off, and the first thing it does once it accesses another server is copy itself there and start running the same code, giving it a slightly different prompt.
> Or that something that has no body or senses with which to experience the world can evolve itself?
Like a virus, you don't really need much to evolve. At what point does GPT become better at designing AIs than AI researchers?
Hey, look, in thinking about this, I've actually given more thought about a fast-takeoff scenario, and I think it's maybe not quite as likely as I imagined, so thanks for that.
> But I think there's the potential for emergent behavior, where size has a quality in itself, that's already showing up with GPT-4.
"Emergent behavior" remains an article of faith, like the second coming. May happen, sure, but we have no evidence or theory for it, and no examples in our own species or any other to make us confident that consciousness somehow "emerges" as a function of scale.
> Nobody knows how these work...
We know exactly how they work. We can't predict the output from the inputs, or get the model to explain its internals in terms that make sense to us, because it doesn't internally work with words and sentences. Not understanding something completely -- atomic bonds, neuron activity -- and not knowing how to get from there to immune response or consciousness doesn't imply magical potential, just our limited understanding.
If we didn't call ChatGPT "AI" with all the sci-fi inspired baggage that label implies, would people still feel this existential dread?
If you know how LLMs work, even at a surface level, I think making the leap from there to AGI requires a kind of religious faith. Do you actually believe that a technology that has access to vast amounts of human-generated data and can imitate our language well enough to fool us a lot of the time will suddenly scale into or somehow spark itself into an actual intelligence with its own goals and ideas and a drive to survive and reproduce? Or that something that has no body or senses with which to experience the world can evolve itself?
LLMs display some remarkable abilities and will probably have significant utility, including replacing some jobs. I could say the same about laundry machines and fork lifts. But let's stop worshiping the golden calf we made, stop with the conceit that we have created a sentient entity. We haven't.
I don't feel a bit of dread. I have lived through the threat of nuclear war since the 1960s. I survived COVID, a threat that didn't need any intelligence, only human incompetence, to kill millions. And I have three children whom I think will do fine even though they will face a future I can't predict.