Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"if we grant these systems too much power"


You can say that about anything.

"If we grant these calculators too much power"


Or the people that rely on the tools to make decisions...

https://sheetcast.com/articles/ten-memorable-excel-disasters


Yes, and it's not as absurd as it might seem:

Imagine hooking up all ICBMs to launch whenever this week's Powerball draw consists exclusively of prime numbers: Absurd, and nobody would do it.

Now imagine hooking them up to the output of a "complex AI trained on various scenarios and linked to intelligence sources including public news and social media sentiment" instead – in order to create a credible second-strike/dead hand capability or whatnot.

I'm pretty sure the latter doesn't sound as absurd as the former to quite a few people...

A system doesn't need to be "true AI" to be existentially dangerous to humanity.


How is a calculator going to cause harm? Assuming you get an industrially rated circuit board when appropriate, it should work just fine as a PLC.

If you try to make it drive a car, I wouldn't call that a problem of giving it too much power.


I'd say by far our biggest problem for the foreseeable future is granting other humans too much power.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: