• Describe the problem of multiple subjects • Explanation of cut-and-paste method • Example render
Problem of multiple subjects
A common problem with Stable Diffusion is that it is difficult to create renders with multiple subjects, especially if they are of different species, gender, or are just visually different for any reason. Problems can include creating renders with the desired: • number of subjects • species/gender/visual character for each subject • placement/pose of each subject
These are problems when you are just creating an individual render with one prompt. There are additional tools like ControlNet which can be a solution to this problem, and those will be covered in later tutorials.
The cut-and-paste method
I had to name this something, so this is just what I am calling it. One way to solve this problem is what I’m calling the cut-and-paste method.
• Make 1 base render with a background and optionally with 1 subject. • Make a separate render for each additional subject, with simple backgrounds that are easy to remove. I usually use blue, but here I used green. • In GIMP, edit the additional subject renders to remove their backgrounds. • In GIMP, combine the additional renders into the base render. • Run img2img on the final edited image with a low denoise (usually 0.05 or 0.1)
Pros:
• Having each subject be a different render gives you a lot of control over the physical characteristics of each subject. • You won’t have to deal with a prompt tuning process that needs to simultaneously get the every single subject AND the background correct all at the same time. • This process is probably easier than learning how to set up ControlNet or Latent Couple.
Cons:
• It’s difficult to edit subjects together when they are physically touching or interacting in any way. Even simple things like a hand on a shoulder is difficult to get right in this method. • Some amount of editing is guaranteed to be required for this process. • When you copy-paste a subject into a scene, it can sometimes be easy to tell that they were copy-pasted in. Some careful work can be needed to more believably combine subjects
I should point out that I don’t really use this method anymore. I used to use it a lot, but now I always use ControlNet or Latent Couple for multi-subject renders. However, I think cut-and-paste is still good to cover as a tutorial since it can be a much easier process for someone to learn.
Example: initial renders
In this example render, I need a scene with multiple characters that look different. I’m using foxes for all my subjects in this tutorial series, but I can make them different types of foxes, and also make them wear different colors. I decided to go for an RGB red/green/blue theme for the clothing, which means they have to be placed in an exact left-to-right order. Enforcing this order would be difficult without using cut-and-paste or some other solution.
Images 1A, 1B, 1C are the initial renders; where 1A is the base image with the green fox in the middle and 1B/1C are the supplementary renders of the red and blue foxes with green backgrounds. I usually use blue backgrounds but in this case one of the foxes is wearing blue, so that would have been a bad idea.
The supplementary renders have “simple green background” as part of the prompt. You may have to give this a prompt strength of 1.1 or 1.2 to ensure the background is simple enough. It may put some objects in the background, but that is fine as long as the outer outline of the subject is only touching background pixels.
For the base render I used “t-pose” in the prompt so that the subject’s arms would be outstretched. I wanted to combine the subjects such that the middle subject had their arms around the others. The end result does not pull this off as well as I would like, but that is what I was going for.
Example: GIMP editing
Attached is a video of my GIMP editing of these renders. Here I will go over some details on how I personally do this kind of edit.
• I use a separate white layer and a black layer to visually confirm that all of the background is removed. In the video you can see me cycling between black/white layers which together can reveal which parts of the background are still there. It’s also useful for revealing parts of the subject that were accidentally removed. • In GIMP, I use the Fuzzy Select tool (shortcut “U”) for most of the background removal. I usually stay between a Threshold value of 10-20 for most of it. • I create a copy layer of the image before I start removing anything. I use one layer to do the Fuzzy Select part, and the deletion is done on the other layer. This is good for situations where most of the background is removed, but there are maybe some wispy bits left behind. (Around 2:05 in the video is a good example of this) • If there are objects/text in the background, I’ll just delete those with other select tools like Rectangle (R) or Free Select (F). This is fine since they are not touching the subject so I don’t have to do this super carefully.
Additional notes about this specific render:
• I actually messed up on the gray fox in this edit. Some of the edges around the subject were still pretty green and I had to go back and fix them. Starting around 4:55 you can see where I did this. Ideally I would have noticed and taken care of this before copying over. The way I fixed this here was using the Free Select and the “Hue-Chroma” tool under “Colors” menu, and turning down Chroma to desaturate and make it more greyscale. This ONLY works because the fox is already grey, so desaturate works here. • This edit had to be quick because I was recording it for a video. If I were putting in more effort, I would have added hands on shoulders to the left and right subject. Instead, I made sure the middle subject’s hands were completely obscured or removed, so that it was more that her hands were on their backs. • I removed a bit of the right subject’s tail that was obscuring the middle subject. Adding small touches like this to show a subject obstructing another can help sell that the pasted subject belongs in the scene.
Example: img2img step
Running img2img on the edited scene is important because it really helps with the combining of the subjects and make the scene more believable in really subtle ways. It can also really help smoothing out the edges of the pasted-in subjects.
Images 2A and 2B are the edited image and img2img render. Comparing these images, you can see parts where the img2img improved things. Image 2C shows a couple examples where this is improvement is very noticeable.
Just like in my previous tutorial on GIMP, I am using a low denoise value. In this case it is 0.1. I tried 0.05 first, but that ended up not doing enough to correct things. My general rule is that denoise should be as low as possible, but just high enough to correct editing issues.
The prompt in this step is different than the initial renders, since we don’t want to give the prompt too many details specific to the base subject. With such a low denoise, it doesn’t matter that much for the prompt to be super precise. Usually what I do is leave in any details about the background, and be less specific about the subjects.
