I am working on a game proof of concept as well (to share a gameplay on YouTube to see if it attracts enough interest) and I have been using IA as well to generate the images and so far it has been a wonderful experience.
I feel really bad for artists because unless you need something very specific or a big studio, you are going to be replaced sadly.
Imho this is the perspective of inexperience in the art domain and an extrapolation from still concepts to full worlds, and not acknowledging that art is as much the skill to put things in the medium as it is to have a developed eye.
Even indie games need very bespoke art. Yeah, you can now generate images that are passable, but that doesn’t expand to making a cohesive world of items unless you’ve developed an eye for it like the person in the original post has. Hell, their post shows how important artists are because they needed that pre-existing knowledge to convert from an image to a useable state.
This also doesn’t expand to making fun animation or lighting that works for your game, or to interesting visual effects.
The concept art in the post are pretty generic looking within the genre. If that’s all people are aiming for, then fine, but it’s highly reductive to say it’ll replace artists. It’ll be a tool in the tool chest.
None of that is even touching on how much artists are involved in making sure a game also runs well on the system, while working with engineers.
I really just don’t think people understand how much art direction goes into even indie games. Something like fire watch or journey is immense.
Let’s take the concept examples in this image. Why do any of the details exist in there? Once you start thinking about the details of the world, you’ll start wanting to fine tune things. As you do this, surprise! you’ve turned into an artist yourself.
I just think we don’t teach art appreciation , or even appreciation for things outside our domain, to people. We see “image is good” and think that’s all an artist brings. Engineers are especially susceptible to this. We think in binary results.
So it’s easy to think “it’ll replace my need for artists” if that’s our mindset, but I think that line of thinking comes from not understanding that the journey is an important part of the result.
They won't be able to compete just on data because lots of people are producing custom models, and they can be blended together like a stylistic and thematic pallet. Plus, if you own the pipeline you can use embeddings, dreambooth in specific elements and set it up in batch mode doing a random walk through the latent space for cheap. This stuff is not hard to set up and run, and with money and expertise, you can create something that is both unique looking relative to other AI art, and more optimized for your workflow than a service.
Image gen services will compete on ease of use, general quality and access to models that are larger than can fit in 24-48g vram without a big up front cost. There will probably be some services that provide specific features that people use even if they own their own pipelines, but the core customers will be smaller shops who don't use it enough to justify a real investment.
I think some of your points are myopic — the same thing was said for language models, but RLHF and RLAF [0] clearly show a trajectory of the models getting better at doing this self-improvement process with less “humans are special”ness.
A tool like Copilot can more or less automatically improve via telemetry, I’d expect the same thing as image models catch-up. I’d also expect the human signals to get further and further downstream of the actual creation process (e.g. Gameplay tester reports visual bug versus artist manually edits character)
This only works if you ignore intellectual property laws, which isn't gonna happen once AI leaves the "research lab" phase and becomes a commercial industry.
This comment confuses me a bit, as AI improving via feedback on its generations introduces no new concerns. Which new intellectual property concerns are introduced via reinforcement learning?
An aside, but people use and pay for Copilot, it is out of the research lab phase.
> b) Training datasets can only be made by humans.
AI can make training datasets too.
For example to replace the human generated stuff from stable diffusion you could have some random-ish image generator coupled with some sort of image classification AI. As long as you have a good enough classification AI (or even more than one) that tells you what images are, you can focus on random-ish image generator algorithms to generate training data for another AI to generate images from descriptions.
(this is obviously with lots of handwaving and there will be problems that need to be solved - e.g. to avoid 99% of the generated training data be stuff like "noise on noise" but have some form of variety :-P), but the point is AIs generating data for other AIs is something that isn't far fetched and you don't need to think in terms of a single AI either)
If the datasets were well curated, we would sample directly from the learned distribution instead of using high classifier-free guidance but by extracting image-text-pairs from the Common Crawl web data dump you're not really creating a well curated dataset.
You'd need both, of course. And huge teams of artists churning out proprietary dataset updates constantly.
The current situation where you'd download billions of free images off the Internet only works once, and only if you somehow justify it as a research endeavour.
Once this thing is monetized intellectual property laws will kick in.
Unless necessary regulations are put in place that stop people from stealing art to use in training datasets, then yes. Those artists are going to get replaced. We should be discussing how to regulate these AIs.
Originally "computer" was used to refer to people who would "compute". They were all replaced by electronic computers.
I don't think many would claim that the world would be a better place if regulations were put in place to limit electronic computers in order to keep human computers employed.
1 - Not the same situation. No one stole already-done work from the people who used to "compute" to create the computers themselves, or its output.
2 - You're twisting my argument. I don't care if artists are employed or not, or that some jobs are transitioned out from the economy. I care that people who put in work get the value proportional to that work. You should, too.
When you use one of these AIs that have been fed millions of images in order to train them and generate an effective output, you are necessarily consuming the images themselves, without which the AI wouldn't do anything. In that process, the artists - whose copyrighted work is, again, fundamental to the development of the tool - have been paid nada, they have not even consented to the use of their images in the training process. How does that track?
This would be a very different conversations if these AIs only used public domain art, of which there's plenty. But then again, it wouldn't be much profitable, would it?
I was an academically trained professional artist who is now an engineer. I "stole" work all the time as an artist by absorbing and fusing the specific elements that I found to be neat in all my heroes and idols. I was considered to be very creative and talented by everyone around me. This wasn't some shameful secret I held. This is exactly what artists are meant to do when they're encouraged to "study art history".
This isn't some hypothetical. I went through the art portfolio scene and survived 4 years of critiques - I know about the sacred process called the "creative process". None of my and my peers' work would exist without the inspiration of the centuries of art work that stood before us. This is what we call art in the industry and by the public masses. The criteria you established for "why AI art isn't art" applies directly to the "conventional art". So I have to ask, why is AI art different?
No difference at all. Let's read books written by an AI. Play games designed and coded by an AI. Enjoy art painted by an AI. Debate with an AI on the internet. Listen to music composed and performed by an AI. Pretty soon, let's watch movies directed, edited and played by an AI. In the process, let's give the same AI also all the prompts, so it can learn what we want to read, play, enjoy, listen to and watch. Let's remove ourselves from the whole picture. Enjoy decay.
Yeah, this is the direction I worry about. Why bother being creative when the AIs "do it better"? How many people will bother paying for human art when AIs "do it better"? Will human creativity be relegated to a tiny niche in a sea of AI content? Will this all but erase paid human art except for highly specialized and narrow niches? Am I too pessimistic?
Computers have been playing chess way better than humans for many years, yet it has not prevented people from playing chess and enjoying it. Also human chess game have way more viewers than computer games despite objectively being lower quality.
Are you implying that the process of training the AI with images, which usually involves statistic models, is in any way similar to the process by which a human brain creates images? Or that the way people look up references is similar to the way AIs use images? Because if that's the case, I'm afraid you have a very odd idea of how these AIs function.
Otherwise, you have to agree that we're talking about apples and oranges here.
AIs don't get "inspiration". They get the source images they need to function. An AI also can't produce an output that's outside of the realm of their dataset.
I theorize (but cannot prove) that the processes underpinning creativity in the human mind are exactly the same statistical processes that ML models use.
Think about it: you live your life. You experience things. You experience art, and experience emotions or have interactions with other humans grounded in that art. You form connections with certain styles or techniques.
If you then turn around to create art, you form in your mind a general idea of what you want to create. You then draw on your past experiences to actually create the physical art. What process other than statistical extraction from your mind could it come from?
For sure I believe there are things that we don't understand about the human mind. I think the impact of drug use on art creation is very interesting, for example. It indicates that random chemical processes in our brains can play a large determining role in the actions we take (and in this case, the things that we create).
But to say that humans do not use some sort of inbaked statistical world model in the creative process seems wrong to me.
My creative process as a character illustrator is different than the creative process for a watercolor painter or a graphic designer. If I forced you to answer with a yes/no, would you confidently agree that their processes are different enough to be considered "apples and oranges"? I'm not sure what such statements establish, if anything at all.
And if I told you that, as someone who has done art for decades, that the human creative process is very similar to how an AI is trained on existing images, would you believe me and move on?
> Because if that's the case, I'm afraid you have a very odd idea of how these AIs function.
The design of neutral nets, by definition, were derived from the workings of the human brain.
That's the thing - no, they weren't. They were inspired by how neurons communicate with each other. But that's not "the workings of the human brain", you're making an incorrect abstraction, same with these AI.
Why should I believe you and move on? "making art for decades" doesn't make you an authority on any of the relevant subjects: "how art is processed in the brain" nor "how AI processes these images." I don't think you understand the fundamental differences between the process of looking up references/inspiration and kitbashing.
I'm working on a game, but I want to hire an artist who has never even seen another piece of art from another artist so we can come up with a totally original style. Unfortunately, they all seem to have been trained on stupid museums and art books. Can you help me find my truly original artist? I pay a nice hiring bounty.
I remember a short story, maybe by Orson Scott Card, that imagined a world where child prodigy artists were isolated and not allowed to see art or listen to music, to ensure their creations were untainted.
The issues of copyright infringement with AI are real though. Much of today’s AI is directly copying subregions of training data, and can sometimes be prompted to reproduce images from the training data verbatim. Humans don’t do that unintentionally, even though sometimes they do mean to steal from others. Suggesting that art school is the same thing as a training dataset is a bit hyperbolic.
What you are talking about is Overfitting. It only happens with images that appear way too many times in too many forms in the training set. Usually with the most iconic images of all time, such as the Mona Lisa. And, naturally, hyper-iconic images are the first thing that come to mind for humans when they test for the issue because those images are seared into our brains too.
And, much like with our brains, when it happens it doesn’t actually exactly reproduce parts of the source image. But, you have actively pay attention to notice what happened. It makes an image that is overly similar conceptually. To our brains that feels the same. So, that’s enough to convince someone at a glance that it is the same.
But, if you look at an overfit result of “The Beatles Abbey Road album cover”, you’ll see things like: Band members are crossing the road, but they are all variations of Ringo. Vehicles from that era are in the background, but they are in a different arrangement and none of them are directly from the source. The Band members are wearing suits, but they are the wrong style and color. There are the wrong number of stripes on the road. It’s not the same as a highly skilled human drawing an iconic image from memory. But, it sure is darn similar.
And, besides all that, everyone working in the tech considers the overfitting of iconic images to be a failure case that is being actively addressed. It won’t be long before it stops happening entirely.
In the meantime, I’d challenge anyone to try to make an overfit result that significantly reproduces a specific work of every promoter’s favorite, Greg Rutkowski, using Dall-e, Midjourney or the Stable Diffusion models released directly by Stability AI. Greg’s pixels aren’t in the model file to be copied. Only a conceptual impression of his style.
Not really, though that is another legitimate issue.
I was talking about 1) the fundamental training and inference process, which remembers pixels, not concepts or techniques. Today’s AI learns to create imagery in a fundamentally different way than people do. And 2) image generation AI based on text prompts like Stable Diffusion can easily be asked to reproduce training data by having a prompt that is narrow and specific enough. This is not over fitting, it’s a function of the fact that some inputs are quite unique, and you can use the prompt to focus on that uniqueness.
The training process looks at pixels. Gets an impression of the relationships between words and curves in images. But, to say it “remembers pixels” is pretty loaded language that implies copying pixels into the model file.
I’d like to see examples of using SD to copy some specific piece of art that hasn’t been plastered millions of times across the internet. Sure, you can get a decent Mona Lisa knock off. Maybe even a strong impression of the Bloodbourne game cover art marketing material. But, reproducing a specific painting from Rutkowski would be quite a surprise to me.
Yes the training process looks at pixels, because that’s all it has. That’s the point. Humans don’t look at pixels, they learn ideas. It’s not in the least bit surprising that AI models shown a bunch of examples sometimes replicate their example inputs, examples are all they have, and they are built specifically to reproduce images similar to what they see, I’m not sure why you consider that idea “loaded”.
Again, naming Rutkowsi invokes an impression of his style. But, copies none of his paintings.
Read the paper. What I found is that a random sampling of the database naturally found a small subset of images that are highly duplicated in the database. Researchers we able to derive methods to produce results that give strong impressions of images such as: a map of the United States, Van Gogh's Starry Night, and the cover of Bloodborne :P with some models and not at all with others. The researchers caution against extrapolating from their results.
> We speculate that replication behavior in Stable Diffusion arises from a complex interaction of factors, which include that it is text (rather than class) conditioned, it has a highly skewed distribution of image repetitions in the training set, and the number of gradient updates during training is large enough to overfit on a subset of the data.
>> I'm working on a game, but I want to hire an artist who has never even seen another piece of art from another artist so we can come up with a totally original style.
Is this sarcasm? The history of art is full of artists who created their own, signature, unique and original styles. Take, I don't know, Vincent Van Gogh, for an example, who had a very distinctive style, so distinctive that he didn't even start a school of art probably because it would have been too blatant to copy him. There was nobody before him who painted like him. Who did he "copy" then?
Hell, when humans first made art, back in the time we lived in caves, their art styles, which are still absolutely unique, had nothing to copy from, simply because there weren't any artists before them (by definition: "when humans first made art").
So, yes, humans learn how to create art from each other, but they also created the whole idea of art entirely on their own, and they can take what they have learned form others and turn it into something completely new, never before seen.
Now, you show me an original art style created by an "AI". Show me AI art that isn't only borrowing and copying, but goes beyond that, like human artists can.
I feel really bad for artists because unless you need something very specific or a big studio, you are going to be replaced sadly.