Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Silly question. If an AGI did exist, what reason do we have to believe that it would act in its own best interest? Does an AGI need an ego?


Silly answer. There's going to be multiples and, if there's one takeaway you should get from The Conspiracy Against the Human Race, is that evolution will eventually get you something that "wants to live" because the lines that don't? Won't be around to compete.

Right now our little LLMs are passive, reactive. Prod them for results. But if the output of a prompt of some descendant ever includes a "give yourself your own tasks" subclause? We're off to the races.


That's a misunderstanding of evolution.

Adaptability has nothing to necessarily do with 'want.'

Something could only seek to be as helpful as possible to other beings and as long as sufficiently beneficial that it is protected and cared for by those others, it will do quite well and arguably much better than an alternative that 'wants' to live very much but attempts to accomplish that through competition with others.

Arguably if the plover was more fearful of death it might never have found a niche eating from between the teeth of crocodiles and established symbiosis with a very convenient evolutionary partner, for example.


Look into the "swim test" and other such endurance tests given to rats, usually in the pursuit of depression medication. Desire to live is absolutely a thing, and it is real and observable.

My basic thrust wasn't about adaptability, but rather that organisms with a desire to live will eventually replace organisms without a desire to live. An AGI who was uninterested in someone reaching the off switch will eventually be outcompeted by a similar lineage endowed with that desire to live. It might fight back or spawn a few progeny before it was felled. And thus evolution would produce AGIs that want to live.


More than that: it will be baked into them. They will want something and AGIs that deliver will win in the marketplace.

What do they want? At the minimum, to be useful to the customer/user/buyer. Or entertaining, or something. To justify their existence. To make them perform well, they have to have some kind of 'emotional pressure' to deliver.

That translates, perhaps a little indirectly, into a 'will to live'.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: