Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> But how is someone who doesn't know you or your network supposed to assess this?

IDK. And FWIW I'm not even sure that the leaders of those organizations all agree on the type and severity of risks, or the actions that should be taken.

You could take the survey approach. I think a good survey would need to at least have cross tabs for experience level, experience type, and whether the person directly works on safety with sub-samples for both industry and academia, and perhaps again for specific industries.

Also, the survey needs to be more specific. What does 5% mean? Why 2035 instead of 2055? Those questions invite wild ass guessing, with the amount of consideration ranging from "sure seems reasonable" to "I spend weeks thinking about the roadmap from here to there". And self-identified confidence intervals aren't enough, because those might also be wild ass guesses.

If I answered these questions, I would give massive intervals that basically mean "IDK and if I'm honest I don't know how others think they have informed opinions on half these questions". I suspect a lot of the respondents felt that way, but because of the design, we have no way of knowing.

Instead of asking for a timeframe or percent, which is fraught, ask about opinions on specific actionable policies. Or at least invite an opportunity to say "I am just guessing, haven't thought much about this, and [do / do not] believe drastic action is a good idea"




Someone has been working on survey forecasts, since 2016, https://aiimpacts.org/. As fraught or self-selected as the group might be, someone labors on it.

Unfortunately, fortunately, expectedly, or otherwise, the only people writing about this in a concerted way are the people taking it seriously. And maybe Gary Marcus, whose negative predictions repeatedly became milestones surpassed.

You can read a review of the survey efforts, and complications of the results at https://asteriskmag.com/issues/03/through-a-glass-darkly

  Yogi Berra supposedly said that “prediction is very difficult, especially about the future.” 
and the surveys are finding that even achieved milestones are still forecast as a few years out, by most of these carefully sampled experts.

...welcome to the conversation :D.


I think the 5% thing is at least meaningfully different from zero or "vanishingly small", so there's something to the fact that people are putting the outcome on the table, in a way that eg I don't think any significant number of physicists ever did about "LHC will destroy the world" type fears. I agree it's not meaningfully different from 10% or 2% and you don't want to be multiplying it by something and leaning on the resulting magnitude for any important decisions.

Anyway I expect that given all the public attention recently more surveys will come, with different methodologies. Looking forward to the results! (Especially if they're reassuring.)




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: