My thought as well. Nuclear weapons are also horrifying.
And with LLMs, it's difficult to prevent the proliferation to bad actors.
It seems like we're racing towards a world of fakery where nothing can be believed, even when wielded by good actors. I really hope LLMs can actually add value at a significant level.
> It seems like we're racing towards a world of fakery where nothing can be believed
Spend a couple minutes on social media and it is clear we are already there. The fakes are getting better, and even real videos are routinely called out as fake.
The best that I can hope for is that we all gain a healthy dose of skepticism and appreciate that everything we see could be fake. I don't love the idea of having to distrust everything I see, but at this point it seems like the least bad option.
But I worry that what we will experience will actually be somewhat worse. A sufficiently large number of people, even knowing about AI fakery, will still uncritically believe what they read and see.
Maybe I am being too cynical this morning. But it is hard to look at the state of our society today and not feel a little bleak.
That is assuming that ASI doesn't centralize power itself. I mean, if you are a non-human part of the animal kingdom you'd probably say that humans have centralized power around themselves.
Are you claiming a lack of gun / AI control is democratizing? That's not working for (the lack of) gun control in the US at the moment though.
Compare also with capitalism; unchecked capitalism on paper causes healthy competition, but in practice it means concentration of power (monopolies) at the expense of individuals (e.g. our accumulated expressions on the internet being used for training materials).
>Are you claiming a lack of gun / AI control is democratizing?
This is obviously the case. It results in a greater distribution of power.
>That's not working for (the lack of) gun control in the US at the moment though.
In the US, one political party is pro gun-control and the other is against. The party with the guns gets to break into the capitol, and the party without the guns gets to watch. I expect the local problem of AI safety, like gun safety will also be self-solving in this manner.
Eventually, Gun control will not work anywhere, regardless of regulation. The last time I checked, you don't need a drone license. And what are the new weapons of war? Not guns. The technology will increase in acessibility until the regulation is impossible to enforce.
The idea that you can control the use of technology by limiting it to some ordained group of is very brittle. It is better to rely on a balance of powers. The only way to secure civilization in the long run is to make the defensive technology stronger than the offensive technology.
>> Are you claiming a lack of gun / AI control is democratizing?
> This is obviously the case. It results in a greater distribution of power.
That's the theory. In practice, it doesn't work.
Most people don't spend a lot of time looking for ways to acquire and/or retain wealth and power. But absent regulation, we'll gradually lose out to those driven folks who do. Perhaps they do so because they want to serve humanity and they imagine that their gifts make them the logical choice to run things. Or perhaps they just want to dominate things.
And the rest of us have every right to insist on guardrails, so those driven folks can't take us over the cliff. Certainly those folks can make huge contributions to society. But they can also fuck up spectacularly — because talent in one field isn't necessarily transferable to another. (Recall that Michael Jordan was one of the greatest basketball players of all time. But he wasn't even close to being the GOAT ... as a baseball player.)
Sure, maybe through some combination of genetics, rearing, and/or just plain hard work, you've managed to acquire "a very particular set of skills" (to coin a phrase ...) for making money, or for persuading people to do what you want. That doesn't mean you necessarily know WTF you're talking about when it comes to the myriad details of running the most-complex "organism" ever seen on the planet, namely human society.
And in any case, the rest of us are entitled to refuse to roll the dice on either the wisdom or the benevolence of the driven folks.
And equally rebutted by Eddie Izzards "Well, I think the gun helps".