@@ -21,7 +21,7 @@ The abstract of the paper is the following:
2121## Tips
2222
2323- Stable Diffusion XL works especially well with images between 768 and 1024.
24- - Stable Diffusion XL output image can be improved by making use of a refiner as shown below
24+ - Stable Diffusion XL output image can be improved by making use of a refiner as shown below.
2525
2626### Available checkpoints:
2727
@@ -40,7 +40,7 @@ pip install safetensors
4040pip install invisible-watermark >=2.0
4141```
4242
43- ### * Text-to-Image*
43+ ### Text-to-Image
4444
4545You can use SDXL as follows for *text-to-image*:
4646
@@ -71,6 +71,7 @@ pipe = StableDiffusionXLPipeline.from_pretrained(
7171)
7272pipe.to(" cuda" )
7373
74+ use_refiner = True
7475refiner = StableDiffusionXLImg2ImgPipeline.from_pretrained(
7576 " stabilityai/stable-diffusion-xl-refiner-0.9" , torch_dtype = torch.float16, use_safetensors = True , variant = " fp16"
7677)
@@ -82,7 +83,29 @@ image = pipe(prompt=prompt, output_type="latent" if use_refiner else "pil").imag
8283image = refiner(prompt = prompt, image = image[None , :]).images[0 ]
8384```
8485
85- ### Loading single file checkpoitns / original file format
86+ ### Image-to-image
87+
88+ ``` py
89+ import torch
90+ from diffusers import StableDiffusionXLImg2ImgPipeline
91+ from diffusers.utils import load_image
92+
93+ pipe = StableDiffusionXLImg2ImgPipeline.from_pretrained(
94+ " stabilityai/stable-diffusion-xl-refiner-0.9" , torch_dtype = torch.float16
95+ )
96+ pipe = pipe.to(" cuda" )
97+ url = " https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/aa_xl/000000009.png"
98+
99+ init_image = load_image(url).convert(" RGB" )
100+ prompt = " a photo of an astronaut riding a horse on mars"
101+ image = pipe(prompt, image = init_image).images[0 ]
102+ ```
103+
104+ | Original Image | Refined Image |
105+ | ---| ---|
106+ | ![ ] ( https://huggingface.co/datasets/diffusers/docs-images/resolve/main/sd_xl/init_image.png ) | ![ ] ( https://huggingface.co/datasets/diffusers/docs-images/resolve/main/sd_xl/refined_image.png ) |
107+
108+ ### Loading single file checkpoints / original file format
86109
87110By making use of [ ` ~diffusers.loaders.FromSingleFileMixin.from_single_file ` ] you can also load the
88111original file format into ` diffusers ` :
@@ -127,7 +150,7 @@ You can speed up inference by making use of `torch.compile`. This should give yo
127150+ refiner.unet = torch.compile(refiner.unet, mode="reduce-overhead", fullgraph=True)
128151```
129152
130- ### Running with ` torch ` < 2.0
153+ ### Running with ` torch ` \ < 2.0
131154
132155** Note** that if you want to run Stable Diffusion XL with ` torch ` < 2.0, please make sure to enable xformers
133156attention:
0 commit comments