Remember that an AI doesn't have to grab power or be sentient or be an agent of any kind. AI could lead to the destruction or domination of humanity, even if AI is simply a passive tool completely obedient to its operators.
Suppose GPT5 or 6 is multimodal and extremely intelligent. Any work that could be done remotely at a computer could be done by the superhuman intelligence of GPT 6. This is everything from creating music and movies, virtual YouTubers and live streamers, to customer support agents, software developers and designers, most legal advice, and many more categories besides. Providing all of these jobs at extremely low cost will make Open AI exceedingly rich - what if they want to get richer? How about robotics? The AI can help them not only design and iterate and improve on robots and robot hardware, but the AI can also operate the robots for them. Now they can not only do the kind of jobs that can be done on the computer. They can do the jobs that require physical manipulation as well and that category of jobs extends to things like security, military, police.
At this point there are at least two possibilities. First is that the AI is effectively controlled by OpenAI in which case the decision makers that open AI would have effective and lasting control over all of humanity. No plucky band of rebels could possibly overthrow the AI powered tyranny - That's the strength of relentless super intelligence. The second possibility is that open AI doesn't have it well controlled - that could mean bugs or unexpected behavior or it could mean the model expressing some kind of agency of its own - think of the things that the Bing AI was saying before Microsoft got it mostly under control. If the AI isn't under control, it may choose to eliminate humanity simply because we might be in its way or we might be an impediment to its plans.
Suppose GPT5 or 6 is multimodal and extremely intelligent. Any work that could be done remotely at a computer could be done by the superhuman intelligence of GPT 6. This is everything from creating music and movies, virtual YouTubers and live streamers, to customer support agents, software developers and designers, most legal advice, and many more categories besides. Providing all of these jobs at extremely low cost will make Open AI exceedingly rich - what if they want to get richer? How about robotics? The AI can help them not only design and iterate and improve on robots and robot hardware, but the AI can also operate the robots for them. Now they can not only do the kind of jobs that can be done on the computer. They can do the jobs that require physical manipulation as well and that category of jobs extends to things like security, military, police.
At this point there are at least two possibilities. First is that the AI is effectively controlled by OpenAI in which case the decision makers that open AI would have effective and lasting control over all of humanity. No plucky band of rebels could possibly overthrow the AI powered tyranny - That's the strength of relentless super intelligence. The second possibility is that open AI doesn't have it well controlled - that could mean bugs or unexpected behavior or it could mean the model expressing some kind of agency of its own - think of the things that the Bing AI was saying before Microsoft got it mostly under control. If the AI isn't under control, it may choose to eliminate humanity simply because we might be in its way or we might be an impediment to its plans.