[Stable Diffusion Inpaint & ControlNet inpaint] Correct timestep inpaint #3749
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint.py
Outdated
Show resolved
Hide resolved
|
After investigating this closer, the timesteps for the two inpainting pipelines was indeed incorrect. Before this PR, we had the following logic: The timesteps: [999, 699, 399, 1] were diffused. Before every diffusion step the area to be diffused was always filled with noise of strength of the prev timestep. This means that the timestep 699 tried to diffuse the original image + noise strength from step 999 instead of 699. Instead we need to add noise from the previous step after every step so that it works correctly. Great catch @https://github.com/ernestchu and @https://github.com/jackloomen . Instead of doing Will make a patch release for this PR. |
patrickvonplaten
changed the title
…int (#3749) * Correct timestep inpaint * make style * Fix * Apply suggestions from code review * make style
…int (huggingface#3749) * Correct timestep inpaint * make style * Fix * Apply suggestions from code review * make style
…int (huggingface#3749) * Correct timestep inpaint * make style * Fix * Apply suggestions from code review * make style
Investigating it seems like taking the next timestep to generate the noise works better. Investigating after comment here:
and very old PR here:
Conclusion is that the timesteps for adding noise for SD inpainting indeed need to be changed. This was incorrectly implemented in #3533