If we could come up with a system to classify the probabilities across a large number of candidate words (or components thereof) then this could actually be good at producing text, one element at a time. We could call these elements 'tokens' and picking the right one could be called something like 'decoding'. Crazy idea but hear me out...
On a more serious note, it will be fascinating to see how this different spin on modelling inference will create new paradigms or slot into existing ones.
What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can and should happen downstream of that.
I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability.
A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety.
After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake.
The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.
Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring.
Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible.
If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
> If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
If we instead imagined autonomous robots going and breaking stuff in the adjacent business' warehouse during a test run, I have to imagine a very different response. That this is happening in an abstract digital world (for now) is part of the problem.
> Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
The point that I'm trying to make is that there's an opportunity to reduce this ambiguity. Reducing the ambiguity and aligning incentives is much more likely to prevent bad stuff happening than reacting to a fait accompli model with latent humanicidal tendencies.
All crime is a result of people choosing to commit a crime. This is why crime has a pure causal relationship with integrity of character. If Dario and Sama don't choose crime and it just spontaneously happens, then it's not a crime.
I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?
Let's say that we are "pacing the frontier", and an approved model goes and causes billions of damage, or worse, loss of life. The auditor signed off on the model, so who is to blame? Who put their akin in the game? METR did. The public did. But it seems like the lab has crafted a system in which they remain blameless.
The calculus of risk is fundamentally different when the adverse consequences are obvious and tangible.
I don't see how some federal agency or external auditor will do anything to align incentives. It just seems like a way to socialize the risk while holding onto the benefits to me.
Again, you're presenting ideas that I just don't understand where they're coming from. Why would the lab be blameless in such a situation? You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems, so who would give them immunity and why?
Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense.
We don’t get to roll cars downhill and get away with destroying others’ property, right?
The perceived difficulty is not what is at play here. People and employers are not comfortable with the idea of long lived credentials to begin with -- and even less in the 'hands' of an AI agent.
The complexity in these protocols is mostly essential in nature (to the extent that you're not willing to totally reinvent the protocol, like AAuth).
If you want to go back to a simpler editing experience, https://plnkr.co/edit/?preview might just do the trick for you. Hasn't changed in literally years. Nowhere near the polish or flair you see from legends like Chris Coyier though.
My oh shit moment was when tool calling was emerging as a capability. That was the moment I realized that LLMs would be the glue connecting a million different use-cases in a million ways we wouldn't even be able to imagine.
Lot's of comments are nit-picking on how this demo misses the mark on specific technicalities of domains they're familiar with.
But that's totally missing the point: this is a demo of what seems like a totally new interaction model that could not previously have existed. It's a small glimpse at a possible future where those nit-picky details get better.
Remember when image models produced grotesque hands? Now look at GPT Image 2 and Nano Banana Pro; those original nits are mostly a thing of the past.
Take a look at this not for its immediate flaws but for the new doors it opens and the expansion of our collective imagination.
Is it really AI slop if someone leverages AI to improve / transform their novel experiences and ideas into a rendition that they prefer?
I'm not suggesting whether or not the article is AI assisted. I'm wondering if the ease of calling someone's work "AI slop" is a step along the slippery slope towards trivializing this sort of drive-by hostility that can be toxic in a community.
You are right about the toxicity, I will edit my comment.
There's a difference between leveraging AI to proofread or improve parts of their writing and this - I feel like AI was overused here; gave the whole article that distinctive smell and significantly reduced its information density.
Given that the fix appears to be to look for own properties, the attack was likely to reference prototype level module properties or the gift-that-keeps-giving the that is __proto__.
I see this type of vulnerability all the time. Seen it in Java, Lua, JavaScript, Python and so on.
I think deserialization that relying on blacklists of properties is a dangerous game.
I think rolling your own object deserialization in a library that isn’t fully dedicated to deserialization is about as dangerous as writing your own encryption code.
not `__proto__` but likely `constructor`, if you access `({}).constructor` you'd get the Object constructor, then if you access `.constructor` on that you'd get the Function constructor
the one problem I haven't understood is how it manages to perform a second call afterwards, as only being able to call Function constructor doesn't really amount to much (still a serious problem though)
On a more serious note, it will be fascinating to see how this different spin on modelling inference will create new paradigms or slot into existing ones.
reply