Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I also wouldn't be surprised if labs were starting to mix in a few pelican SVGs into their training data.


Even "accidentally" it makes sense that "SVGs of pelicans riding bikes" are now included into datasets used for training as it has spread as a wildfire on the internet, making it less useful as a simple benchmark.

This is why I keep all my benchmarks private and don't share anything about them publicly, as soon as you write about them anywhere publicly they'll stop being useful in some months.


> This is why I keep all my benchmarks private

This is also why, if I were an artist or anyone commercially relying on creative output of any kind, I wouldn't be posting anything on the internet anymore, ever. The minute you make anything public, the engines will clone it to death and turn it into a commodity.


That makes it so much harder to show art to people and market yourself though.

I considered experimenting with web DRM for art sites/portfolios, on the assumption that scrappers won't bother with the analog loophole (and dedicated art-style cloners would hopefully be disappointed by the quality), but gave up because of limited compatible devices for the strongest DRM levels, and HDCP being broken on those levels anyway. If the DRM technique caught on it would take attackers, at most, a few bucks and hours once to bypass it, and I don't think users would truly understand that upfront.


Somewhat defeats the purpose of being an artist, doesn't it?


Defeating the purpose of creating almost anything, really.

AI is definitely breaking the whole "labor for money" architecture of our world.


Worth noting, in case people weren't aware, but bunch of people have different motivations for being artists, not everyone wants everything they do to be as widely shared as possible, some are happy playing/drawing/whatever for a small group of people, and not even making money off it.


Eeeehhhh.

Maybe the thing to do is provide public, physical exhibits of your art in search of patronage.


I'll believe they are doing that when one of the models draws me an SVG that actually looks like a pelican.


Someone needs to craft a beautifully bike donned by a pelican, throw in some seo, and see how long it takes a model to replicate it.

Simon probably wouldn't be happy about killing his multi-year evaluation metric though...


I would be delighted.

My pelican on a bicycle benchmark is a long con. The goal is to finally get a good SVG of a pelican riding a bicycle, and if I can trick AI labs into investing significant effort in cheating on my benchmark then fine, that gets me my pelican!


SVG benchmarking is a thing since GPT-4, so probably all major labs are overfitting on some dataset ov svg images for sure




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: