>>The world is the way it is now because humans are the only source of intelligent signal processing. That’s the only reason why humans enjoy the limited rights and privileges that they do. That’s the only reason why life has gotten better and better with advancing medicine and so on.
That's actually an interesting point I haven't thought of before. I disagree with you though. You could have made the same argument during the industrial revolution, but capitalism, democracy and civil liberties are still around.
Also, you're assuming that human symbiosis with AGI is not possible. Nature is full of examples of organisms that are in symbiotic relationships. Of course, nature is also full of organisms that kill and eat and parasitise other organisms.
But the advantage that we humans have is that we are actually creating AIs/AGIs. And since we are creating it, we have influence over what it becomes, whereas, your average clownfish really doesn't have a say over what form his anenome takes.
And if that is the case, wouldn't you want free societies to have expertise in these technologies and to be fine tuning them so that they work for the common benefit?
Why should anyone believe that we can stop authoritarian govts from developing AGI when we can't even stop them from enriching uranium, where the capital costs are much higher?
For better or for worse, it is almost always the case that the difference between societies that hold economic and political power and those that don't is technology. I live in the US, I enjoy my civil liberties, and I would much rather we master these technologies ourselves and have the security that wealth and power provides us. Not saying democracy or capitalism are perfect, but they beat all of the alternatives.
Industrial revolution: has nothing to do with this. Some forms of human labor were automated. The set of all human abilities was not threatened. AGI is fundamentally different because AGI will automate everything a human can do. Increasingly important distinction. And not something I would overlook in the hundreds of hours of rumination I’ve put into it.
Can’t stop countries from doing it: yes, we can. A tin can dictatorship can’t do it. And besides, it’s a problem that depends a lot on academic work being published and shared among researchers. It’s a collaborative effort largely. Some rogue state trying to do this while avoiding inspections and inquiries by larger nations, and with no support from reading publicly published papers from other researchers, it would be tough. But with a large majority of anti-ai countries we could do a lot to prevent even covert attempts.
That's actually an interesting point I haven't thought of before. I disagree with you though. You could have made the same argument during the industrial revolution, but capitalism, democracy and civil liberties are still around.
Also, you're assuming that human symbiosis with AGI is not possible. Nature is full of examples of organisms that are in symbiotic relationships. Of course, nature is also full of organisms that kill and eat and parasitise other organisms.
But the advantage that we humans have is that we are actually creating AIs/AGIs. And since we are creating it, we have influence over what it becomes, whereas, your average clownfish really doesn't have a say over what form his anenome takes.
And if that is the case, wouldn't you want free societies to have expertise in these technologies and to be fine tuning them so that they work for the common benefit?
Why should anyone believe that we can stop authoritarian govts from developing AGI when we can't even stop them from enriching uranium, where the capital costs are much higher?
For better or for worse, it is almost always the case that the difference between societies that hold economic and political power and those that don't is technology. I live in the US, I enjoy my civil liberties, and I would much rather we master these technologies ourselves and have the security that wealth and power provides us. Not saying democracy or capitalism are perfect, but they beat all of the alternatives.
Edit: added another thought