Jeff Dean literally featured it in a tweet announcing the model. Personally it feels absurd to believe they've put absolutely no thought into optimizing this type of SVG output given the disproportionate amount of attention devoted to a specific test for 1 yr+.
I wouldn't really even call it "cheating" since it has improved models' ability to generate artistic SVG imagery more broadly but the days of this being an effective way to evaluate a model's "interdisciplinary" visual reasoning abilities have long since passed, IMO.
It's become yet another example in the ever growing list of benchmaxxed targets whose original purpose was defeated by teaching to the test.
I mean if you want to make your own benchmark, simply don't make it public and don't do it often. If your salamander on skis or whatever gets better with time it likely has nothing to do with being benchmaxxed.
Revenue should not be confused with profit. The large AI companies must easily be spending more on compute than they're making from a $20-200/mo subscription. In the best case it might break even for the AI companies. There is no way that they're actually earning a profit from these subscriptions at this time.
It's where the revenue is, but it isn't going to be where the profit is. Developers will easily use absurdly large amounts of compute, costing the AI provider a lot more than they receive in revenue.
Given Israel's successful precision targeting of various senior Hezb members in recent months, I wonder if the pagers were initially used as such, but as suspicion mounted, and chances of an overhaul increased, they decided to hit the kill switch while they still could.
Although as as per an WSJ article: "The affected pagers were from a new shipment that the group received in recent days"
