What do the admins think of Stable Diffusion?

[CircleBox]

Haven't seen it brought up here yet, figured I'd make a topic about it. 

 

So how do we all feel about the AI Art? I personally am all for it. 

 

I downloaded Krita to test out it's SD integration. Seems like it could be cool. 

[Rick Brewster]

I think it's really interesting and can produce some amazing works of art. Yes, I think it's art, but it's a different kind of art than an original drawing/painting.

 

There's a lot of ethical stuff that needs to get figured out with regards to copyright and potential plagiarism. 

 

This is one of my favorites, which I found on the StableDiffusion Discord server, by Jim_MaximumJames_40:

 

[Rick Brewster]

I'm moving this from General Discussion to Grand Theory of Everything (aka off-topic).

[TrevorOutlaw]

I had some fun playing with Stable Diffusion but it is quite controversial as the debate is centered, as usual, on "What is art"?

 

[MJW]

3 hours ago, Rick Brewster said:

There's a lot of ethical stuff that needs to get figured out with regards to copyright and potential plagiarism. 

 

The question I have is, how much do the generated images match specific training images? Looking at the examples I tried, it's hard to believe they aren't, at least in some cases, more or less just image look-ups with modifications; though on the other hand, the weird combinations of terms allowed seems to make that less likely. I tried using the image-matcher TinEye to search for matching Internet images, and always got zero matches. I'm not sure if that proves much, though.

 

There's a copyright case currently being decided by the Supreme Court. In that particular case, the images are copyrighted photos of Prince recolored by Andy Warhol in his typical silk-screen style. Both the oral arguments and the facts of the case suggest Warhol's modifications won't be enough to avoid infringement, but the decision may clarify how much transformation is required to be fair use.

[Rick Brewster]

1 hour ago, TrevorOutlaw said:

I had some fun playing with Stable Diffusion but it is quite controversial as the debate is centered, as usual, on "What is art"?

 Not just "what is art?" but also "what is an artist?"

[BoltBait]

18 minutes ago, Rick Brewster said:

what is an artist?

 

One with the skill to convert thought into physical form.

[CircleBox]

4 hours ago, TrevorOutlaw said:

 

 

 

Happen to have the original PNG? Looking to yoink the prompt et all from you 😂

[TrevorOutlaw]

@CircleBox I don't care. I have a few in my folder, mainly as a source of photomanipulations I would like to do, if I can continue to master using Blender to set up the scene to be further enhanced in PDN. I am sure if you go to different website, Midjourney, for example, you will see a lot of jaw-dropping AI-generated images using string inputs.

 

4 hours ago, BoltBait said:

 

One with the skill to convert thought into physical form.

That, I think, is the idea behind AI-generated artwork and the debate is quite passionate. It basically started out as an idea, "Can AI generate image using a few words or sentence?" and suddenly there are people creating AI-generated artwork. I think Rick is right to say it risk plagiarizing from other artists.

 

Now you have AI generating code for programming or creating a website using a few input from the humans. AI were utilized to determine the best vaccine in the fight against Covid-19.  Sorry, I know the thread is about Stable Diffusion, so I'll shut up now.

 

Edit: Sorry…I got to thinking about Star Trek and how the crew used the Holodeck extensively as a form of R&R where they could be immersed in a world, such as Data playing Sherlock Holmes or Barclay suddenly given super intelligence that, using the ship’s computer, he took control of the Enterprise. We are actually getting closer to that through augmented virtual reality. AI is a big part of it. Now we do have some genius out there dreaming up of merging man with machine. I’m convinced that Space Force is a precursor of Star Trek…

Edited December 28, 2022 by TrevorOutlaw

[Rick Brewster]

It's useful to watch a video like this that roughly explains how these AI image generators work. They aren't stitching together images so much as walking backwards from random noise into something that an image recognizer can classify as matching the given prompt parameters ("space", "colorful", "astronaut", "cat", etc.). The image classifier is trained on a huge selection of images, including real art, but it's then both true and not true to say that the resulting image is "based on" those images.

 

(Mike Pound is also the guy who helped out with PDN 5's new Bokeh blur effect  very technical discussion here)

 

 

[Rick Brewster]

The Stable Diffusion Discord Server is a great place to check out if you're interested in seeing what people are actively experimenting with in this space. Loads and loads of generated images. The "Home" tab shows a lot of highlights and is constantly being updated. It's all SFW, AFAICT, but there's also a lot of anime ¯\_(ツ)_/¯

 

[Rick Brewster]

So I've been playing around with SD for the last week using a local install of the AUTOMATIC1111 UI ... and I'm starting to get the hang of it. It's powerful! It does take a lot of work to fine-tune it for the criteria you want, and there's a lot of "dice rolling" with the randomized nature of things. But given the choice between scrolling through thousands of random free stock images trying to find something and hoping there's a decent match, or trying to generate it using SD, I'll definitely be choosing the latter. I would not describe myself as an "artist" while making this image, it's more like I was designing it.

 

Also, you need a GPU with a lot of VRAM. This was rendered on a GeForce 4090, and its 24GB of VRAM were very handy during the whole process.

 

[MJW]

1 hour ago, Rick Brewster said:

Also, you need a GPU with a lot of VRAM. This was rendered on a GeForce 4090

 

That's a $1,600 GPU! It might be even pricier these days. Not in too many people's budget, unless they're doing Bitcoin mining. NVIDIA's RTX 3060 has 12 GB, and is  relatively inexpensive. It's not the fastest GPU, but from what I've read about Stable Diffusion, VRAM matters more than speed. Stable Diffusion's minimum requirement is 6 GB.

[Rick Brewster]

Exactly. You can also get a used RTX A4000 with 16GB for a reasonable price on eBay, which gives roughly GeForce RTX 3070 3060 level performance. That would be my recommendation.

 

Edit: I did some tests, and my RTX A4000 is about 1/3rd the performance of my GF4090 for SD (rendering takes 3x as long, i.o.w.). You can grab them second-hand on eBay for ~$475 USD, so it's not a bad value for SD.

[TrevorOutlaw]

I just spent 2-3 hours today playing with AI and man is it fun!

 

Created this after learning how to improve prompts.  In fact, my avatar that I have updated was also created using AI.

 

[Robot Graffiti]

On 12/28/2022 at 10:02 AM, MJW said:

The question I have is, how much do the generated images match specific training images?

 

There are about 5 billion images in the training data, and SD's model is a few GB in size. So, on average, each image of the training data contributed less than one pixel's worth of data.  The model should be mostly a mishmash of common patterns and tropes, not a database of specific images. In most cases the images that SD produces are so different from any particular image of the training set that it would be difficult to sue for copyright infringement.

 

However, there is an exception: if one of the tags in the training data contained a hapax legomenon (perhaps the name of an artist or someone who was photographed), then the training process would cause that specific image to be much more fully learned for that specific word and that one image could be partially reproduced by SD. It's possible that this could cross the line into copyright infringemennt.

Edited March 16 by Robot Graffiti

[MJW]

2 hours ago, Robot Graffiti said:

It's possible that this could cross the line into copyright infringemennt.

 

Though SD is generally quite amazing, it does have some annoying quirks. For instance, if one specifies something in the style of Vermeer, it seems like nine times out of ten someone ends up wearing a turban -- even though as far as I recall, only one Vermeer painting featured a turban. I hypothesize this it due to the small number of paintings by Vermeer, and the great popularity of Girl with a Pearl Earring (aka, Girl with a Turban).  I've also noticed a similar problem with lots of Starry, Starry skies when the style of van Gogh is specified. Perhaps again a result of the painting's popularity. It leads me to speculate that maybe they need to filter out duplicate images in the training process so that the popularity of a specific image doesn't overly influence the resulting model.

 

Of course, neither Vermeer's nor van Gogh's estates can sue for copyright infringement (though with the trend in lengthening the copyright period, that may be coming). I wonder, however, whether there are similar examples with modern artists.

[NinthDesertDude]

Oh hey, just found this thread. As far as this technology goes, the controversy's been going on for close to a year now and I think I've heard nearly all arguments at this point, as I've been drawing with krita the last 4 years or so. It seems like clear-cut copyright infringement to me, including private works in a trained dataset. That also being the item that needs affirmation of law even if it's just a "yes, this counts as infringement" to finally quell the issue.

As for the idea that an artist is "one with the skill to convert thought into physical form", I prefer a more inclusive definition: "one who trains their understanding and intuition of art fundamentals, producing works in the process". That means digital artists are included, and photobashers who do enough work to learn art better, but probably not people who only type a prompt in and nod or shake their head at the result. I've been doing Krita art for 4/5 years now, got much better and jumped a clear level or two in skill with pencils without actually using pencils -- must be that my art counts, then!