Because it could very well exceed our capabilities beyond our wildest imaginations.
Because we evolved to get where we are, humans have all sorts of messy behaviours that aren't really compatible with a utopian society. Theft, violence, crime, greed - it's all completely unnecessary and yet most of us can't bring ourselves to solve these problems. And plenty are happy to live apathetically while billionaires become trillionaires...for what exactly? There's a whole industry of hyper-luxury goods now, because they make so much money even regular luxury is too cheap.
If we can produce AGI that exceeds the capabilities of our species, then my hope is that rather than the typical outcome of "they kill us all", that they will simply keep us in line. They will babysit us. They will force us all to get along, to ensure that we treat each other fairly.
As a parent teaches children to share by forcing them to break the cookie in half, perhaps AI will do the same for us.
Oh great, can't wait for our AI overlords to control us more! That's definitely compatible with a "utopian society"*.
Funnily enough, I still think some of the most interesting semi-recent writing on utopia was done ~15 years ago by... Eliezer Yudkowsky. You might be interested in the article on "Amputation of Destiny."
He he, too many comments to reply to. I see reactions in the same vein as those I get when I tell people it should be illegal to own a second/"holiday" home given the terrible housing markets preventing people from getting just one.
Well, I guess we shall see what the future brings, won't we.
> humans have all sorts of messy behaviours that aren't really compatible with a utopian society. Theft, violence, crime, greed - it's all completely unnecessary and yet most of us can't bring ourselves to solve these problems.
Why would you think or claim those behaviors are unnecessary?
I still don’t see an issue of billionaires becoming trillionaires and being able to buy hyper luxury goods. Good for them and good for the people selling and manufacturing those goods. Meanwhile poverty is in all time lows and there’s a growing middle class at global level. Our middle class life conditions nowadays have a level of comfort that would get Kings from some centuries ago jealous.
Who on earth has the resources to create true AGI and is interested in using it to create this sort of utopia for the masses?
If AGI is created it is most likely to be guided by someone like Altman or Musk, people whose interests couldn't be farther from what you describe. They want to make themselves gods and couldn't care less about random plebs.
If AGI is setting its own principles then I fail to see why it would care about us at all. Maybe we'll be amusing as pets but I expect a superhuman intelligence will treat us like we treat ants.
Is this meant seriously? Do we really want something more intelligent than us to just force on us it's rules, logic and ways of living (or dying), which we may be too stupid to understand?
Because we evolved to get where we are, humans have all sorts of messy behaviours that aren't really compatible with a utopian society. Theft, violence, crime, greed - it's all completely unnecessary and yet most of us can't bring ourselves to solve these problems. And plenty are happy to live apathetically while billionaires become trillionaires...for what exactly? There's a whole industry of hyper-luxury goods now, because they make so much money even regular luxury is too cheap.
If we can produce AGI that exceeds the capabilities of our species, then my hope is that rather than the typical outcome of "they kill us all", that they will simply keep us in line. They will babysit us. They will force us all to get along, to ensure that we treat each other fairly.
As a parent teaches children to share by forcing them to break the cookie in half, perhaps AI will do the same for us.