Sunsets with SDXL
Using seeds, we can unlimited versions of sunsets by an artist.
AI models work by starting with random noise and then iteratively de-noising towards your prompt. The seed determines the random noise.
On one that changing a number gives us another similar image. On the other, the as you look up and down the columns, there are usually big changes in the image as you change the seed.
Variation: Updating Prompt
The first thing you might think is updating the prompt while using the same seed. Changing "Sunset by Van Gogh" to "Sunset in winter by Van Gogh".
DDIM Inversion
Inversion: Given an image, find the seed that generated it.
cloneofsimo implemented DDIM inversions for SDXL.
Inversion Guidance
In the previous example the "guidance scale" (how strong the prompt is) was 3.5, which is quite low compared to a normal SDXL generation (7.5)... As you can see below, a strong guidance can result in overfitting to the prompt, since the noise we are starting from has already been shaped by the inversion process.
Inversion Steps
Since the latents produced by inversion have a lot of "structure" in place, it seems possible that less than 50 steps would be needed to generate your variation
The following grid lets you look at the affect of inference steps in creating your final image