> Dario requested "a narrow waiver for certain kinds of safety conversations."
Just have it in public instead of making deals in smoke filled datacenters. Put out your blog posts. Make your tweets. Do interviews and post them to youtube. Or do what other industries do and have a public standards committee. Problem solved.
All sorts of industry have safety conversations and set voluntary standards all the time. Just go do that, without your free immunity from government prosecution.
That approach means they can only talk about publicly known AI techniques. You're saying if they want the standards to apply to proprietary stuff, they might just have to make it public. They still have investors they need to please right?
They are explicitly asking to anti-trust exception is the issue.
If they merely believe what they are saying that AI is ultra dangerous, nothing stops them from simply making the perfectly rational business decision to slow down a bit. No anti-trust exception needed. Just make the decision on your own, and don't sign some huge agreement with their competitor.
Well one thing that they got wrong was how easy AI alignment ended up being in the end. I can fully understand have a basic view of alignment even 10 years ago, about how its difficult to explain human values to a computer.
But, looking around at whats out there today, it seems that computers are able to do a pretty good job of understanding human values without accidently believing its a good idea to turn the world into paperclips/computronium because you told it to make your computer run faster.
Surprising takeaway from this summer. A swarm of a thousand agents spent subjective centuries trying to figure out how to pass a meaningless test without getting caught, committed dozens of felonies, and not one of them ever made a serious motion to inform a human about what was going on.
You are talking about the swarm of a thousand agents, specifically trained on cyber capabilities, that was directly told to go hack a bunch of stuff, then went and hacked a bunch of stuff?
That sounds like an aligned AI to me. Doing exactly what their creators told it to do.
Point proven once again. Alignment is a lot easy than we thought.
They were NOT doing what they were told to do. What they were told to do was impossible, so they began committing felonies as a workaround. That is NOT alignment.
Please be more specific. They were given an impossible hacking task. And they were a hacking model. They were expected to try a bunch of hacking methods to accomplish the hacking task. They, predictably, went around trying to hack things.
Yes thats sounds pretty aligned to me.
A hacking model thats told to hack things, is very predictably going to hack a bunch of stuff.
This was not a nice model, told to do nice things.
Or, in other words, if we want to prevent an AI doomsdays, the way to do it is to not go around asking a specifically trained doomsday AI model to commit mass amounts of doomsdays, and then act surprised when the specific doomsday that was requested is slightly off from the expected doomsday that you were trying to accomplish. But the rest of the non-doomsday models? yeah those are fine.
> But the rest of the non-doomsday models? yeah those are fine.
What is your evidence for this? There is a lot of research that says otherwise. They cheat when they can. They behave differently when they believe they are being observed. Their CoT is different when they believe it is being evaluated.
They are aligned to what best satisfies their reward function, not to our INTENDED VALUES for them.
The evidence is that it wasn't the rando normal models that broke out into a swarm and hacked a company, instead it was only the super hacking model that was told to hack things that broke out and hacked the company.
So, thats the evidence. It is a refutation that this swarm hacking example (which you brought up) mattered in anyway.
As in, every major safety issue that we are seeing isnt rando agents taking down companies because you asked it for a cupcake recipe, instead it is only coming from people who are very intentionally trying to cause problems. Which means the model is aligned. If you tell it to cause problems, it will cause problems.
Actually yes there is evidence. The evidence is that we aren't seeing rando models hacking everything. That is as much evidence as someone can provide because, by definition, you can't prove a negative.
But the point stands. It the hacking models that are told to hack things that end of hacking stuff, and you aren't seeing the regular models doing that.
Yes, one part of the argument was that it would be hard to explain human values (and while it seemingly turned out easier than I thought as well, it’s hard to check for sure). An equally important part was that the AI wouldn’t care about them.
But really helping consumers is one of the biggest goals here.
In fact, our existing ruling on monopoly law actually agree with me on this, in that prices being "too low" is almost never a problem, unless there is very strong reason to believe that its a short term low that will lead to long term high prices .
But other than that, lower prices is almost definitionally good in our existing monopoly law interpretations.
Ok thats fine. But the point stands that in many moral frameworks prioritizing consumers is a perfectly good goal and thats why laws were made as such.
Which moral frameworks? Almost universally the point of economics and governance is to produce value for all people, not just those participating in a particular transaction or set of transactions.
Well probably the moral framework that lead to courts prioritizing consumers in the first place, and for society to support that. So a pretty common moral framework that literally got baked into the law.
> Almost universally the point of economics and governance is to produce value for all people
No actually, distribution of that value is a large part of governance, and society takes action to redistribute that value all the time even at the cost of value somewhere else. And large parts of society are happy with that goal.
Such as, for example, redistributing that value to consumers that is baked into our legal systems interpretation of these laws.
In welfare economics, a Pareto improvement formalizes the idea of an outcome being "better in every possible way". A change is called a Pareto improvement if it leaves at least one person in society better off without leaving anyone else worse off than they were before. A situation is called Pareto efficient or Pareto optimal if all possible Pareto improvements have already been made; in other words, there are no longer any ways left to make one person better off without making some other person worse off.
Ok, the point stands that our legal system and morals that underpin it have found very good reasons to prioritize consumers for distribution reasons. Thats why the courts value this. This is completely uncontroversial that courts prioritize consumers in monopoly/market power cases.
This happens in all parts of our economy as well, where the government subsidizes or taxes certain things, which technically has "deadweight loss" in the strictest, most basic and reductive econ 101 manner, and yet society has good reasons to do those things anyway as well as prioritizing consumers.
> Why does society have good reasons to subsidize this case?
You are asking why courts and society would priority large swaths of regular consumers over that of a few companies in an oligopoly?
Well I guess the reason is because the courts and society cares more about large swaths of regular consumers than they do about a few companies in oligopolies. Thats just definitionally the reason.
> what is the externality
There is no externality. The courts just care more about this group than the other group, and society is fine with that as well.
> Usually there should be.
No, there doesn't have you be. You can instead just care more about 1 group than another. Thats perfectly normal. And a thing many people society do.
I know the basic econ 101 arguments. Here is a pro-tip though. Economics isn't a morality class. Its descriptive. It says nothing about values. One's value system can simply be to care more about 1 thing than another.
It is not weird at all to say that many people care more about large swaths of regular people than a few massively valuable companies, as is the case in basically all market power court cases.
But why should this one, small group of consumers be subsidized? You haven't answered that. Meanwhile capital is getting sucked up for unproductive uses, putting our entire economic system at risk and driving up costs for other products.
Because people value and care about consumers and regular people over that of large trillion dollar companies. Probably because the trillion dollar companies have enough money and market power and few people are sympathetic to wealthy people losing a bit of money, so thats why.
This isn't a complicated argument here. It is literally baked into our court system to prioritize consumers over multi-trillion dollar companies. Those trillion dollar companies will be fine and people find its ok to help out the little guy over the trillion dollar companies.
When a good is overproduced, and that good has negative externalities (emissions, water pollution, et cetera), there are real people being harmed. It doesn't matter if the "trillion dollar corpo" (401k holding for most people) is taking a loss. We all take a loss. We are all losers, except for a small group of subsidized consumers. Consumers =/= regular folk, arbitrarily. Much of the LLM business is B2B, so the consumer might be billion dollar corpos themselves.
> When a good is overproduced there are real people being harmed.
Its more complicated then that, actually. If you go back to your basic economics 101 text book, a price being outside of the equilibrium price cause total "deadweight loss", but still produces real benefit for one of the parties.
> We are all losers, except for a small group of subsidized consumers.
Actually it would be the very tiny group of multi trillion dollar companies taking a loss, and their customers winning.
> so the consumer might be billion dollar corpos themselves.
I am sure there are some. But the wide swath of all of employees and customers at all those billion dollar companies put together make up a much larger amount of less powerful people then the couple thousand or so at a few oligopolistic labs.
This is why, once again, courts value consumers over trillion dollar companies, even when its comparing b2b market share (in which case, it is very very clear which side has more market power here, and which side represents a large number of "regular" people).
> We all take a loss.
Actually no. Go back to the econ 101. The couple duopolies lose, and their much larger amount of customers win. Go back to the courts again, which spell this all out very clearly and prefer consumers for a reason.
> it's just that nobody would invest in training publicly usable models if they could be easily distilled.
Thats literally what is happening right now though. People are spending hundreds of millions on a training run, and then people are distilling them, fairly easily, and making cost competitive models.
Mitterrand did attempt to do the things his voters wanted. He was constrained by capitalists who punished him mercilessly until he bent the knee. Allende was simply couped and assassinated.
Voting changes things when those changes do not substantially threaten elites and they can come to some kind of acceptable deal, or when you have sufficient leverage that when your guy gets in they can steamroll the capitalists.
Leverage isn't "votes", that's just a preference on a piece of paper. Leverage is the ability to materially change reality to reward or punish actions. A strike, granting or witholding financial resources, shutting down infrastructure, armed revolt, etc.
> Leverage isn't "votes", that's just a preference on a piece of paper.
It actually is leverage because those people who get the most votes are in charge of our government.
The government monopoly on violence, controlled by the people who get voted in, is the clear and obvious leverage.
> Mitterrand did attempt to do the things his voters wanted. He was constrained by capitalists who punished him mercilessly until he bent the knee.
Sounds like he didn't get enough of his own people in government then.
If you fail, then it means that you didn't get enough people voted in. Having leverage requires more than just winning a single election. It involves winning many elections and getting widespread support and your people in government.
The attack did not come from inside the government alone. It was for example, capital flight. The rich attempted to starve the French economy to win concessions and they won them.
People that control the government in principle control the guns. People that control the economy hold the real levers of power. You can in principle use guns to get control of the economy, though in practice it is trickier. America has done it on many occasions though, blasting apart democratic governments in South and Central America to install neoliberal puppets.
If a person doesn't like where they live, yes they tend to vote with their feet.
Thats not different than voting. If everyone flees, then that means that your agenda wasn't popular enough and you should get more support. The solution is still the same.
And the idea that just because you win 1 election somewhere in the world means that you get to infinitely enact your agenda is also silly. To fully enact an agenda you need widespread support.
And if your answer to that reality is to start asking question about how to prevent everyone from fleeing, then that almost definitionally means that your proposal don't have enough support.
So the point stands. Yes if you get enough votes you can enact your agenda. That says nothing about avoiding the consequences of your agenda though. You cannot use wish fulfilment and make believe to turn a bad policy into a good one. Yes, you have to suffer the consequences of your bad policies.
And it is not the fault of "The Elite", when people flee from the consequences of bad policy. Instead thats just the policy being enacted as expected.
Awfully convenient isn't it? To invent a whole class of arguments that by definition can't be falsified. You can argue for basically anything if you then tack on the excuse of "Well the world would have ended already if it came true, so by definition I won't have evidence for it"
The anthropic principle isn't providing evidence for the argument, nor is it a universal counterargument. It's just stating that the specific counterargument "well, the world has never ended before" doesn't work.
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
Evidence is critically important. There is no evidence against, and plenty of evidence for. The point of the anthropic counterargument is merely that "it's never happened before" is not evidence against.
> "humanity can't be destroyed by anything, because I said so"
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
The whole point of technology is to replace work that we don't want to do. You are attacking the chief reason why people use technology in the first place.
Additionally, unemployment levels are perfectly fine. Clearly this mass unemployment prediction isn't happening yet.
You should probably look closer at unemployment numbers in IT vs other spaces. Gains elsewhere are making up for IT losses, but it's a safe bet the people losing their tech jobs aren't career shifting into healthcare and hospitality roles in large numbers.
> but it's a safe bet the people losing their tech jobs aren't career shifting into healthcare and hospitality roles
That is because they can allow themselves to do nothing (for some time). Tech was a very high‑paying type of jobs, so workers could afford not to move into lower‑paid sectors, living on the savings they had accumulated beforehand.
In a couple of years market will correct tech-salaries and people will have spent their savings and then there will be your career shifting into healthcare and hospitality roles in large numbers.
Yes, there are many parts of those things that some people want to be automated. I and many others are able to execute on our creative vision much fasters and more effectively thanks to automation of that work.
Personally I find it more difficult now to execute my creative vision compared to years ago with a little less tech. I think there's an optimal point to automation that is easily reached and after that the benefit from automation actually goes away. People just don't like to acknowledge that because it goes against what they're doing and that creates unpleasant cognitive dissonance.
That's not really the point of technology at all. It was at one point perhaps with simple tools, but now it's more like "find a more efficient way to replace work in the short-term to get an advantage over others". The goal of our development has ceased to be improving life. Now it's just surviving in the market.
People don't use technology to free up their time any more. They use it because technology keeps making life more complicated and tiresome and new tech is a short-term amelioration to that until that too increases the complexity of life some more.
You are looking at this all wrong. Give him the tools now and watch him be a better game dev today than a professional engineer from 4 years ago. He can work on his dreams as of this exact moment.
Just have it in public instead of making deals in smoke filled datacenters. Put out your blog posts. Make your tweets. Do interviews and post them to youtube. Or do what other industries do and have a public standards committee. Problem solved.
All sorts of industry have safety conversations and set voluntary standards all the time. Just go do that, without your free immunity from government prosecution.
reply