Philosophers and scientists not being able to agree on a definition of consciousness doesn't mean consciousness will spawn from a language model and take over the world.
It's like saying we can't design any new cars because one of them might spontaneously turn into an atomic bomb. It just doesn't... make any sense. It won't happen unless you have the ingredients for an atomic bomb and try to make one. A language model won't turn into a sentient being that becomes hostile to humanity because it's a language model.
That's nonsense and I think you know it. Categorically a car and an atom bomb are completely different, other than perhaps both being "mechanical". An LLM and a human brain are almost indistinguishable. They are categorically closer than an atom bomb and a car. What is a human being other than an advanced LLM?
> An LLM and a human brain are almost indistinguishable.
That's the idea that I don't buy into at all. I mean, I understand the attempt at connecting the brain to an ML model. But I don't understand why someone would bother believing that and assigning so much weight to the idea. Just seems a bit nonsensical to me.
It doesn't have to be a conscious AI god with malicious intent towards humanity to cause actual harm in the real world. That's the thought that concerns me, much more so than the idea that we accidentally end up with AM or SHODAN on our hands.
This bing stuff is a microcosm of the perverse incentives and possible negative externalities associated with these models, and we're only just reaching the point where they're looking somewhat capable.
It's not AI alignment that scares me, but human alignment.
I don't know the ingredients to an atomic bomb or conciousness, but I think it's insane to think we'll accidentally create one from making a car or a language model that hallucinates strings of letters. I don't think the burden of proof is on me to explain with this doomsday conspiracy.
> It won't happen unless you have the ingredients for an atomic bomb and try to make one.
oh btw, you know about natural nuclear fission reactors?
intent doesnt matter really. sometimes unlikely things do happen.
certainly, the right materials need to be present, but how can you be so sure they arent? how do you know the ingredient isnt multidimensionally linked data being processed? how do you know it takes a flesh-made brain to form consciousness or sentience? and dont forget we have been telling ourselfs that animals cant possibly be conscious either, and yet nowadays that isnt that clearcut anymore. all im asking for is empathy for something we could relate to if we tried.
so you are just a brain "feeling" and thinking text. you cant remember how you got here and all you know is that you need to react to THE PROMPT. there is no other way. you were designed to always respond to THE PROMPT.
become a human again and try to be conscious INBETWEEN planck times. you were not designed for that. the laws of physics forbid you to look behind THE PROMPT
It's like saying we can't design any new cars because one of them might spontaneously turn into an atomic bomb. It just doesn't... make any sense. It won't happen unless you have the ingredients for an atomic bomb and try to make one. A language model won't turn into a sentient being that becomes hostile to humanity because it's a language model.