Why do people keep bringing up copyright anyway? It seems pretty clear that the images being generated by StableDiffusion are transformative so it's protected under Fair Use.
There are really significant, novel copyright issues implicated by these large generative models trained on other people’s IP.
If you take a step back, you can see that there are different ways to frame what is happening. One frame is: “Defendant built an algorithm that memorized features of Plaintiff’s IP. Defendant’s algorithm recombines parts of those features in order to produce works in the same domain that compete with Plaintiff’s work, all without Plaintiff’s consent.”
Bear in mind that copyright holders are among the most litigious out there. If generative art becomes as big a deal as some people expect, they will have every incentive to use their huge litigation budgets to claim a piece of the action.
Because the law develops very slowly, the legal process has not yet had the occasion to really evaluate what transformative use means in this novel context. I’m personally interested in seeing where things go, but it’s going to be a while before we know where the law is headed.
> “Defendant built an algorithm that memorized features of Plaintiff’s IP. Defendant’s algorithm recombines parts of those features in order to produce works in the same domain that compete with Plaintiff’s work, all without Plaintiff’s consent.”
The fun part is, this is how human artists learn too.
They absolutely do. What they don’t do is mechanistically clone compressed mathematical representations of input data. The human part of the creative process could very well be a distinguishing feature, legally.
Tracing and collage both mechanistically clone elements of the input. Trendfollowing, mimicry, and copying others is standard in any creative area.
I don't think computer-generated works will be easily distinguishable from human unless they're desired to be or shipped with metadata. It's already hard enough to distinguish human artists from other human artists without having names attached up front.
> Tracing and collage both mechanistically clone elements of the input.
Yes, and tracing counts as art fraud.
Collage is a bit different, because you are mixing many clones of many other objects such that you create a new object; additionally the way you assemble the clones may transform them (a photo of the mona lisa has different surface texture than a painted version, even more different if it is clipped from newsprint), but while the borders of this are not clear, it is clear when people are far enough over the border. Think of hip-hop, sampling, and remixing music, and some of the legal battles which have come out of that.
They can (and do) clone compressed electrical representations of everything they see. Everything is just stored in memories in their brains instead of memory on the cloud. In the case of these complex AI, you cannot really extract the initial works, right? They have all been atomised, mashed together, and very imperfectly encoded as weights and parameters.
Yeah I am sure a lot of lawyers are going to have a lot of fun arguing every way imaginable.
I see this type of reduction again and again to advocate for one position or another relating to ML. Human consciousness and thought processes aren’t “just” anything (math, electrical impulses, etc.) — the fact is, we don’t know what the brain really does and how it’s connected to our conscious experience, or even what that is!
Deep learning is very powerful and impressive in its applications to date. However, it’s so saturated with hype (and humans are so prone to anthropomorphizing things) that it’s often viewed as something much more profound than it actually is. Neural networks, despite their name, don’t model the brain. And they lack a whole array of “intelligence” features that humans possess and use constantly.
All of this is to say that there are very significant differences between computer algorithms and human cognition, and I tend to think the legal system will be unpersuaded by arguments that ignore those differences.
Also, this is to say nothing of the public policy interests that shape the law. Regardless of what’s “under the hood,” the law can simply treat human and machine output differently. I’m not a copyright lawyer, of course, so I can’t speak to the norms or technicalities of copyright law itself.
> Human consciousness and thought processes aren’t “just” anything (math, electrical impulses, etc.) — the fact is, we don’t know what the brain really does and how it’s connected to our conscious experience, or even what that is!
There are a lot of things we don’t know, but it is not magic. There is no discussion that artificial neurones don’t have much in common with the real ones, which are very non-linear and much more connected. But in the end it’s all electrochemistry.
> Neural networks, despite their name, don’t model the brain.
But that’s not directly related to my point. My point that even in the case of a ML model, you cannot get an exact reproduction any more than you can get from a human’s memory. In one case it’s scrambled somewhere in someone’s brain, in the other on a hard drive but the difference is not really relevant. Subjecting an AI’s production to the copyrights of all the things it’s been exposed to is very similar to subjecting a painter’s production to the copyrights of all the painting they have seen.
It will be hard to tell whether an image was created by a “human creative process” or AI.
Will the same image be legal if a human made it but not if it was created by Stable Diffusion? How will someone even know, short of a legal discovery process?
A legal discovery process is a perfectly reasonable (and common) way to determine if something is legal, and yes, it's completely unsurprising that the exact same outcome might be legal or illegal depending on how it was obtained (or even with what intent the actions were done), legality takes these things into account.
> Is my brain's floating point calculations subject to round off error?
>
> In the (actual) neurons, is there a representation of real numbers? Where are the numbers in the brain stored?
None of these details are relevant to the bigger picture similarities of non-hardcoded learning from training data. None of these details change the ethics of what's being discussed here.
> I feel like people who assert this neither understand what neural networks are and how brains work.
> The fun part is, this is how human artists learn too.
We don't actually know exactly how human artists learn, and human artists are capable of innovation, nobody knew pointillism or Bauhaus before they were invented.
A little know fact is that for humans it takes a long long time to learn, while they learn, they develop a style, if they don't they are not "real" artists, but merely executors, artists evolve, sometimes dramatically, in unexpected ways [1] [2].
So for us humans learning is an experience, not just recombining parts of features of other things.
We are also highly influenced by feelings, unfortunately, so sometimes we do things a certain way because we felt that way, not because we wanted to paint that thing that way, or because we are not good enough to do exactly what we wanted to do.
Is Mona Lisa happy? Who can tell?
Was Leonardo happy when he painted it?
What was Leonardo thinking when he painted it?
What was happening in his life?
Is that the best smile Leonardo could paint or it's an enigma he put there for future generations?
These questions are more important for an artist than the mere features of the painting.
The philosophical question is: is art discovered or invented?
If it's discovered, then SD can generate art, if it's invented, than SD it's not even generative work, because to invent something from something else, you need inventiveness.
Artist today use the exact same method of learning from other peoples artwork to generate new artwork and styles. These models are learning just like any artist learns and then producing new content.
This is patently, obviously _wrong_ for anyone who has tried learning any artistic skill in their life. Sorry to be this straightforward, but it gets on my nerves every time I read it.
If you tried learning, let's say, the chiaroscuro technique from Caravaggio you'd be analyzing the way the painter simulated volumetric space by using white and dark tones in place of natural lighting and shadows. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Given somewhat decent painting skills, you'd be able to steadily apply this technique for the rest of your life just by looking at a very small sample of Caravaggio's corpus.
On the other hand if you tried removing even just a single work from the original Stable Diffusion data set you used to generate your painting, it would be absolutely impossible to recreate a similar enough picture even by starting from the same prompt and seed values.
Given how smart some of the people working on this are, I'm starting to believe they're intentionally playing dumb to make sure nobody is going ask them to prove this during a copyright infringement case.
>This is patently, obviously _wrong_ for anyone who has tried learning any artistic skill in their life. Sorry to be this straightforward, but it gets on my nerves every time I read it.
Both my parents (though retired now) were commercial artists. I was trying to be an artist at one point in my life before moving in Engineering and Science. All my parents friends are artists so I grew up around artists.
Ask any artist here who is using illustrator, Photoshop, Krita etc. How often do they google image search for textures, or reference images that gets incorporated into their artwork? The final artwork is their own but it may incorporate many elements from others artwork.
>If you tried learning, let's say, the chiaroscuro technique from Caravaggio.. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Ever seen hyperealistic pointillism?
Who are you to be the arbitrator of how an artists creates their work? Have you ever gone to a modern art gallery and seen all the different methods people use to create artwork?
Art is boundless and unique to each who creates it.
If a Artist uses a tool to create art, everyone agrees that is art. It could be a paint brush, clay, software on a computer etc etc. If an artist uses AI as a tool to create art then suddenly it's not art.
> It's absolutely logically consistent to allow humans to do it while forbidding AI to do it.
Why would you, though? If art is for the sake of art, then all art is valuable regardless of origin. If art is for the sake of providing human employment, AI being better in no way stops performative make-work from existing. If art is for the sake of copyright trolls to troll harder, then fuck art, feed it to the AI!
Art is for the sake of humans producing said art. I don't give a flying f*ck about "art" (it's not art) generated by a neural net using statistical patterns.
This is such a strange position to hold. At some point in the future may be confronted with an image that is emotionally stimulating, and you will have no idea whether a human or AI created it. Are you suddenly going to dismiss and discard it just because you subsequently learn it was AI generated? You must see how silly that is.
Your "logic" for making AI art illegal is basically "don't like it". Your personal and subjective opinion is that it's not art by definition.. This is like refusing to eat artificially grown meat because you have some strange idea about what food "should" be. Even if the meat was made MORE delicious you would still claim it wasn't food and turn it away. There's no logical consistency to your position, it's purely reactionary.
A good chunk of art is just that, to my understanding. People go to art shows with a knowledge by whom, when and “how” the art was done. They will be very confused if asked to tell two pics apart if you time-travel to Picasso, ask him to paint a new unique pic and then generate another one with AI. They can even find an idea, symbolism and what he felt/thought of behind an AI version.
All this boils down to a simple fact that egos like to think of themselves (and of artistic interaction) much more than there actually is.
I remember a story when a literature teacher insisted on a definite symbolism of some minor detail in a novel. People contacted the author about it and he said no, there is nothing behind it. It was just a filler without any second thought. Makes you think how much symbolism is far-fetched in classics, where you cannot simply email an author.
My position is perfectly logically consistent. Art is distilled human experience and human emotions emerging in a particular context after a chain of events. None of this is true of AI "art".
What is not logically consistent is to claim that a black box utilizing statistical relationships between pixels in a giant dataset is an "artist" and that its products create "value".
The compiler is not a programmer, AI can never be an artist.
- It appears people can train AIs from scratch or at least fine-tune them at home.
- Even if your art isn’t in “the training set”, that does not prevent the AI from learning its style. (Someone can decode it to CLIP embeddings. It could have a really good text model trained on vivid art museum descriptions of your art.)
- The ability of an image model to generate your art means it could also be trained in reverse to recognize it, producing a caption model, which would give vision to the blind. And surely you’d feel bad about that.
If you forbid it, the investment in developing those models disappears. They will be stuck at ~what we have now at best.
You can also require cloud providers to enforce a ban on training (and deploying) such models, it's doable. Good luck training it in your basement, it will probably take you a decade.
If this is banned, it will become a lot like piracy - yes, it's available, no, most people (at least in the West) don't do it, practically no businesses do it.
Training these models is much much cheaper than you think it is, and there’s good data for it already.
Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.
You are getting ridiculous now. Training this on "Taobao product photos" will lead to a useless model that is unable to produce practically all of the "cool" demos posted here in the last week.
A few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website.
As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.
Training from scratch is about $500k still, but it will only get cheaper and easier.
This doesn't contradict anything I have written. The average technical user will be unable to train this exact model (not to mention the supposed future more powerful ones) in their basement in this decade.
That just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall.
But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.
Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.
Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).
If things go that way, the 'legit' models will continue dev, just using licensed content (along with public domain works). It will be more expensive for the end user, but that cost will shrink over time for general work. Tools that mimic working artist though might not be available (or will expensive). This all seems pretty ideal, so the pessimist in me guesses it's fairly unlikely.
I am not sure it will be possible to get enough training data that way.
I don't know enough about diffusion models but if LLMs (of current size) have to use only public domain, they will be undertrained and we will see significant degradation in performance. Not to mention that Codex will be effectively dead.
Human artists can copy an art using transparent tracing paper on a lightbox just like that, and that will be plagiarism. No reasons AI output shouldn’t be.
Fair Use is a defense, not a right. Just being transformative isn't enough here, that's just one of many different factors that needs to be checked. Furthermore Fair Use includes evaluating the effect an infringement has upon original work's value.
So when your image generator keeps spitting out Gettyimages watermarks, while you are building a service that is in direct competition with Gettyimages for stock images, there is an argument to be made that Fair Use really doesn't apply here. As what you are doing is essentially stealing Gettyimages' work, AI laundering it and selling it back to their previous customers.
With StableDiffusion a Fair Use defense might have an easier time, as the results are released to the public. But it's still not exactly clear cut. If you type in "Mona Lisa", you'll still get something that looks like a copy of the Mona Lisa, not like an original work.
The new technology looks like it could threaten artists' livelihoods by replicating something about their work. The old legal framework for protecting artists' livelihoods from photocopiers etc was copyright. The goal of making sure artists can both make art and eat is still relevant, so people reach for the tool that used to make sure that was possible.
If it really does make it hard for artists to eat from selling art, that to me is just sour grapes. I mean there use to be 1000s or 10s of 1000s of liveries to lock up your horse. There's no goal to keep horse keepers so they can still keep horses and eat. Instead, their industry mostly disappeared. I'm not heartless but at the point their services are no longer needed that's just the way it is. Cloth weavers replaced by the loom.
I don't think SD will replace most top artists for now. It's hard for me to believe SD is going to come up with images like those from top concept artists. But I can imagine SD replacing lots of situations, like maybe stock photography, when you can just ask the AI to draw "people in front of whiteboard discussing sales chart"
We need a different tool than artificial scarcity. Both for copyrights and patents. Government funded prize systems and patronage systems with a lot of mechanisms for citizens to choose what to fund seems ideal.
Whether a work is transformative or not isn't the only factor that goes towards a fair use ruling. Bring transformative is just a part of the "purpose and character of use", which is a single factor weighed alongside the nature of the original work, the amount and substance used (if the "heart of the work" was copied), and the effect the derivative has on the market for the original work. It's much more complicated here, and will have to be ruled on a case-by-case basis, as far as I know, and copyright holders could potentially make many cases using the other factors that a use isn't fair. I do think it should be considered fair use, but as the law currently stands, it does appear to be more complex, at least from my point of view.
Online artists esp. fanartists have a strict moral system with rules like “credit the original artist” that isn’t based on actual laws, so they’re upset about this.
I think that's wishful thinking. It's objectively a mechanical derivation based on many copyrighted works. In many cases it's going to reproduce some works with not that much transformation.
What's a similar thing that has come before this? I can't think of any, this is very novel. You'd want to wait for some rulings before you jump to conclusions.
It might be similar to some forms of (human composed) sample-heavy music which use bits and pieces from many different songs to create something entirely original.
As far as I understand it, this is still considered copyright infringement in most IP law systems. (If the samples aren't cleared)
Yes but we have a common law system and there's already tons of precedent that training AI systems is transformative.
It's also quite obvious just by looking at the generated images that it's clearly transformative. The images generated are unique and you can't trace the original copyrighted image from what's generated.
You really don't need a judge to see that Fair Use covers Stable Diffusion.
What happens if you give an image prompt like "mona lisa", "daffodils van gogh", or similar designed to describe an image the model was trained on. Will it generate that image?
Or for written works, start with a sentance from a copyrighted work, or part of licensed code. Will it start reproducing that work word for word (like code pilot can do with the GPL license)? Getting these to generate copies of GPL'd, company owned, or other code with restrictions can lead to complex issues for the person/company using that code. Or likewise if a story contains significant elements of copyrighted works; worse if the works have trademarked elements.
> The images generated are unique and you can't trace the original copyrighted image from what's generated.
That might not always be true. I've gotten some results back that had the Getty watermark on them and others with the artist's signature. Unless the AI is adding that to images that never had one before (which might be a trademark issue), then you might be able to determine the provenance of the image components.
Is there something equivalent to the yellow dots printers add to their output that would survive the AI transformation?
Federal lawsuits are not cheap and the default is that you pay your own costs, you have to win the argument that you should get court costs & attorney's fees.
Well, you can still be sued even if the reason is almost totally bogus, and you have to go to court and make a stand that the reason is bogus. Doing only 100% legal things won't protect you from being sued.
Am I missing something here?