Skip to content

Commit f2fa96b

Browse files
authored
README updates for SDXL 1.0 release (Stability-AI#68)
* remove sdxl report from github repo, point to arxiv instead * update licenses and add teaser img * update readme for SDXL 1.0 release
1 parent c60c091 commit f2fa96b

File tree

6 files changed

+227
-9
lines changed

6 files changed

+227
-9
lines changed

LICENSE-CODE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 Stability AI
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 31 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,23 @@
44

55
## News
66

7+
**July 26, 2023**
8+
- We are releasing two new open models with a permissive [`CreativeML Open RAIL++-M` license](model_licenses/LICENSE-SDXL1.0):
9+
- [SDXL-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0): An improved version over `SDXL-base-0.9`.
10+
- [SDXL-refiner-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0): An improved version over `SDXL-refiner-0.9`.
11+
12+
![sample2](assets/001_with_eval.png)
13+
14+
715
**July 4, 2023**
8-
- A technical report on SDXL is now available [here](assets/sdxl_report.pdf).
16+
- A technical report on SDXL is now available [here](https://arxiv.org/abs/2307.01952).
917

1018
**June 22, 2023**
1119

1220

1321
- We are releasing two new diffusion models for research purposes:
14-
- `SD-XL 0.9-base`: The base model was trained on a variety of aspect ratios on images with resolution 1024^2. The base model uses [OpenCLIP-ViT/G](https://github.com/mlfoundations/open_clip) and [CLIP-ViT/L](https://github.com/openai/CLIP/tree/main) for text encoding whereas the refiner model only uses the OpenCLIP model.
15-
- `SD-XL 0.9-refiner`: The refiner has been trained to denoise small noise levels of high quality data and as such is not expected to work as a text-to-image model; instead, it should only be used as an image-to-image model.
22+
- `SDXL-base-0.9`: The base model was trained on a variety of aspect ratios on images with resolution 1024^2. The base model uses [OpenCLIP-ViT/G](https://github.com/mlfoundations/open_clip) and [CLIP-ViT/L](https://github.com/openai/CLIP/tree/main) for text encoding whereas the refiner model only uses the OpenCLIP model.
23+
- `SDXL-refiner-0.9`: The refiner has been trained to denoise small noise levels of high quality data and as such is not expected to work as a text-to-image model; instead, it should only be used as an image-to-image model.
1624

1725
If you would like to access these models for your research, please apply using one of the following links:
1826
[SDXL-0.9-Base model](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9), and [SDXL-0.9-Refiner](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9).
@@ -106,18 +114,32 @@ depending on your use case and PyTorch version, manually.
106114

107115
## Inference:
108116

109-
We provide a [streamlit](https://streamlit.io/) demo for text-to-image and image-to-image sampling in `scripts/demo/sampling.py`. The following models are currently supported:
110-
- [SD-XL 0.9-base](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9)
111-
- [SD-XL 0.9-refiner](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9)
112-
- [SD 2.1-512](https://huggingface.co/stabilityai/stable-diffusion-2-1-base/blob/main/v2-1_512-ema-pruned.safetensors)
113-
- [SD 2.1-768](https://huggingface.co/stabilityai/stable-diffusion-2-1/blob/main/v2-1_768-ema-pruned.safetensors)
117+
We provide a [streamlit](https://streamlit.io/) demo for text-to-image and image-to-image sampling in `scripts/demo/sampling.py`.
118+
The following models are currently supported:
119+
120+
- [SDXL-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
121+
- [SDXL-refiner-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0)
122+
- [SDXL-base-0.9](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9)
123+
- [SDXL-refiner-0.9](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9)
124+
- [SD-2.1-512](https://huggingface.co/stabilityai/stable-diffusion-2-1-base/blob/main/v2-1_512-ema-pruned.safetensors)
125+
- [SD-2.1-768](https://huggingface.co/stabilityai/stable-diffusion-2-1/blob/main/v2-1_768-ema-pruned.safetensors)
114126

115127
**Weights for SDXL**:
128+
129+
**SDXL-1.0:**
130+
The weights of SDXL-1.0 are available (subject to a [`CreativeML Open RAIL++-M` license](model_licenses/LICENSE-SDXL1.0)) here:
131+
- base model: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/
132+
- refiner model: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/
133+
134+
135+
**SDXL-0.9:**
136+
The weights of SDXL-0.9 are available and subject to a [research license](model_licenses/LICENSE-SDXL0.9).
116137
If you would like to access these models for your research, please apply using one of the following links:
117-
[SDXL-0.9-Base model](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9), and [SDXL-0.9-Refiner](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9).
138+
[SDXL-base-0.9 model](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9), and [SDXL-refiner-0.9](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9).
118139
This means that you can apply for any of the two links - and if you are granted - you can access both.
119140
Please log in to your Hugging Face Account with your organization email to request access.
120141

142+
121143
After obtaining the weights, place them into `checkpoints/`.
122144
Next, start the demo using
123145

assets/001_with_eval.png

3.99 MB
Loading

assets/sdxl_report.pdf

-17.3 MB
Binary file not shown.
File renamed without changes.

model_licenses/LICENSE-SDXL1.0

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
Copyright (c) 2023 Stability AI CreativeML Open RAIL++-M License dated July 26, 2023
2+
3+
Section I: PREAMBLE Multimodal generative models are being widely adopted and used, and
4+
have the potential to transform the way artists, among other individuals, conceive and
5+
benefit from AI or ML technologies as a tool for content creation. Notwithstanding the
6+
current and potential benefits that these artifacts can bring to society at large, there
7+
are also concerns about potential misuses of them, either due to their technical
8+
limitations or ethical considerations. In short, this license strives for both the open
9+
and responsible downstream use of the accompanying model. When it comes to the open
10+
character, we took inspiration from open source permissive licenses regarding the grant
11+
of IP rights. Referring to the downstream responsible use, we added use-based
12+
restrictions not permitting the use of the model in very specific scenarios, in order
13+
for the licensor to be able to enforce the license in case potential misuses of the
14+
Model may occur. At the same time, we strive to promote open and responsible research on
15+
generative models for art and content generation. Even though downstream derivative
16+
versions of the model could be released under different licensing terms, the latter will
17+
always have to include - at minimum - the same use-based restrictions as the ones in the
18+
original license (this license). We believe in the intersection between open and
19+
responsible AI development; thus, this agreement aims to strike a balance between both
20+
in order to enable responsible open-science in the field of AI. This CreativeML Open
21+
RAIL++-M License governs the use of the model (and its derivatives) and is informed by
22+
the model card associated with the model. NOW THEREFORE, You and Licensor agree as
23+
follows: Definitions "License" means the terms and conditions for use, reproduction, and
24+
Distribution as defined in this document. "Data" means a collection of information
25+
and/or content extracted from the dataset used with the Model, including to train,
26+
pretrain, or otherwise evaluate the Model. The Data is not licensed under this License.
27+
"Output" means the results of operating a Model as embodied in informational content
28+
resulting therefrom. "Model" means any accompanying machine-learning based assemblies
29+
(including checkpoints), consisting of learnt weights, parameters (including optimizer
30+
states), corresponding to the model architecture as embodied in the Complementary
31+
Material, that have been trained or tuned, in whole or in part on the Data, using the
32+
Complementary Material. "Derivatives of the Model" means all modifications to the Model,
33+
works based on the Model, or any other model which is created or initialized by transfer
34+
of patterns of the weights, parameters, activations or output of the Model, to the other
35+
model, in order to cause the other model to perform similarly to the Model, including -
36+
but not limited to - distillation methods entailing the use of intermediate data
37+
representations or methods based on the generation of synthetic data by the Model for
38+
training the other model. "Complementary Material" means the accompanying source code
39+
and scripts used to define, run, load, benchmark or evaluate the Model, and used to
40+
prepare data for training or evaluation, if any. This includes any accompanying
41+
documentation, tutorials, examples, etc, if any. "Distribution" means any transmission,
42+
reproduction, publication or other sharing of the Model or Derivatives of the Model to a
43+
third party, including providing the Model as a hosted service made available by
44+
electronic or other remote means - e.g. API-based or web access. "Licensor" means the
45+
copyright owner or entity authorized by the copyright owner that is granting the
46+
License, including the persons or entities that may have rights in the Model and/or
47+
distributing the Model. "You" (or "Your") means an individual or Legal Entity exercising
48+
permissions granted by this License and/or making use of the Model for whichever purpose
49+
and in any field of use, including usage of the Model in an end-use application - e.g.
50+
chatbot, translator, image generator. "Third Parties" means individuals or legal
51+
entities that are not under common control with Licensor or You. "Contribution" means
52+
any work of authorship, including the original version of the Model and any
53+
modifications or additions to that Model or Derivatives of the Model thereof, that is
54+
intentionally submitted to Licensor for inclusion in the Model by the copyright owner or
55+
by an individual or Legal Entity authorized to submit on behalf of the copyright owner.
56+
For the purposes of this definition, "submitted" means any form of electronic, verbal,
57+
or written communication sent to the Licensor or its representatives, including but not
58+
limited to communication on electronic mailing lists, source code control systems, and
59+
issue tracking systems that are managed by, or on behalf of, the Licensor for the
60+
purpose of discussing and improving the Model, but excluding communication that is
61+
conspicuously marked or otherwise designated in writing by the copyright owner as "Not a
62+
Contribution." "Contributor" means Licensor and any individual or Legal Entity on behalf
63+
of whom a Contribution has been received by Licensor and subsequently incorporated
64+
within the Model.
65+
66+
Section II: INTELLECTUAL PROPERTY RIGHTS Both copyright and patent grants apply to the
67+
Model, Derivatives of the Model and Complementary Material. The Model and Derivatives of
68+
the Model are subject to additional terms as described in
69+
70+
Section III. Grant of Copyright License. Subject to the terms and conditions of this
71+
License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive,
72+
no-charge, royalty-free, irrevocable copyright license to reproduce, prepare, publicly
73+
display, publicly perform, sublicense, and distribute the Complementary Material, the
74+
Model, and Derivatives of the Model. Grant of Patent License. Subject to the terms and
75+
conditions of this License and where and as applicable, each Contributor hereby grants
76+
to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77+
(except as stated in this paragraph) patent license to make, have made, use, offer to
78+
sell, sell, import, and otherwise transfer the Model and the Complementary Material,
79+
where such license applies only to those patent claims licensable by such Contributor
80+
that are necessarily infringed by their Contribution(s) alone or by combination of their
81+
Contribution(s) with the Model to which such Contribution(s) was submitted. If You
82+
institute patent litigation against any entity (including a cross-claim or counterclaim
83+
in a lawsuit) alleging that the Model and/or Complementary Material or a Contribution
84+
incorporated within the Model and/or Complementary Material constitutes direct or
85+
contributory patent infringement, then any patent licenses granted to You under this
86+
License for the Model and/or Work shall terminate as of the date such litigation is
87+
asserted or filed. Section III: CONDITIONS OF USAGE, DISTRIBUTION AND REDISTRIBUTION
88+
Distribution and Redistribution. You may host for Third Party remote access purposes
89+
(e.g. software-as-a-service), reproduce and distribute copies of the Model or
90+
Derivatives of the Model thereof in any medium, with or without modifications, provided
91+
that You meet the following conditions: Use-based restrictions as referenced in
92+
paragraph 5 MUST be included as an enforceable provision by You in any type of legal
93+
agreement (e.g. a license) governing the use and/or distribution of the Model or
94+
Derivatives of the Model, and You shall give notice to subsequent users You Distribute
95+
to, that the Model or Derivatives of the Model are subject to paragraph 5. This
96+
provision does not apply to the use of Complementary Material. You must give any Third
97+
Party recipients of the Model or Derivatives of the Model a copy of this License; You
98+
must cause any modified files to carry prominent notices stating that You changed the
99+
files; You must retain all copyright, patent, trademark, and attribution notices
100+
excluding those notices that do not pertain to any part of the Model, Derivatives of the
101+
Model. You may add Your own copyright statement to Your modifications and may provide
102+
additional or different license terms and conditions - respecting paragraph 4.a. - for
103+
use, reproduction, or Distribution of Your modifications, or for any such Derivatives of
104+
the Model as a whole, provided Your use, reproduction, and Distribution of the Model
105+
otherwise complies with the conditions stated in this License. Use-based restrictions.
106+
The restrictions set forth in Attachment A are considered Use-based restrictions.
107+
Therefore You cannot use the Model and the Derivatives of the Model for the specified
108+
restricted uses. You may use the Model subject to this License, including only for
109+
lawful purposes and in accordance with the License. Use may include creating any content
110+
with, finetuning, updating, running, training, evaluating and/or reparametrizing the
111+
Model. You shall require all of Your users who use the Model or a Derivative of the
112+
Model to comply with the terms of this paragraph (paragraph 5). The Output You Generate.
113+
Except as set forth herein, Licensor claims no rights in the Output You generate using
114+
the Model. You are accountable for the Output you generate and its subsequent uses. No
115+
use of the output can contravene any provision as stated in the License.
116+
117+
Section IV: OTHER PROVISIONS Updates and Runtime Restrictions. To the maximum extent
118+
permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage
119+
of the Model in violation of this License. Trademarks and related. Nothing in this
120+
License permits You to make use of Licensors’ trademarks, trade names, logos or to
121+
otherwise suggest endorsement or misrepresent the relationship between the parties; and
122+
any rights not expressly granted herein are reserved by the Licensors. Disclaimer of
123+
Warranty. Unless required by applicable law or agreed to in writing, Licensor provides
124+
the Model and the Complementary Material (and each Contributor provides its
125+
Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
126+
express or implied, including, without limitation, any warranties or conditions of
127+
TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
128+
solely responsible for determining the appropriateness of using or redistributing the
129+
Model, Derivatives of the Model, and the Complementary Material and assume any risks
130+
associated with Your exercise of permissions under this License. Limitation of
131+
Liability. In no event and under no legal theory, whether in tort (including
132+
negligence), contract, or otherwise, unless required by applicable law (such as
133+
deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be
134+
liable to You for damages, including any direct, indirect, special, incidental, or
135+
consequential damages of any character arising as a result of this License or out of the
136+
use or inability to use the Model and the Complementary Material (including but not
137+
limited to damages for loss of goodwill, work stoppage, computer failure or malfunction,
138+
or any and all other commercial damages or losses), even if such Contributor has been
139+
advised of the possibility of such damages. Accepting Warranty or Additional Liability.
140+
While redistributing the Model, Derivatives of the Model and the Complementary Material
141+
thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty,
142+
indemnity, or other liability obligations and/or rights consistent with this License.
143+
However, in accepting such obligations, You may act only on Your own behalf and on Your
144+
sole responsibility, not on behalf of any other Contributor, and only if You agree to
145+
indemnify, defend, and hold each Contributor harmless for any liability incurred by, or
146+
claims asserted against, such Contributor by reason of your accepting any such warranty
147+
or additional liability. If any provision of this License is held to be invalid, illegal
148+
or unenforceable, the remaining provisions shall be unaffected thereby and remain valid
149+
as if such provision had not been set forth herein.
150+
151+
END OF TERMS AND CONDITIONS
152+
153+
Attachment A Use Restrictions
154+
You agree not to use the Model or Derivatives of the Model:
155+
In any way that violates any applicable national, federal, state, local or
156+
international law or regulation; For the purpose of exploiting, harming or attempting to
157+
exploit or harm minors in any way; To generate or disseminate verifiably false
158+
information and/or content with the purpose of harming others; To generate or
159+
disseminate personal identifiable information that can be used to harm an individual; To
160+
defame, disparage or otherwise harass others; For fully automated decision making that
161+
adversely impacts an individual’s legal rights or otherwise creates or modifies a
162+
binding, enforceable obligation; For any use intended to or which has the effect of
163+
discriminating against or harming individuals or groups based on online or offline
164+
social behavior or known or predicted personal or personality characteristics; To
165+
exploit any of the vulnerabilities of a specific group of persons based on their age,
166+
social, physical or mental characteristics, in order to materially distort the behavior
167+
of a person pertaining to that group in a manner that causes or is likely to cause that
168+
person or another person physical or psychological harm; For any use intended to or
169+
which has the effect of discriminating against individuals or groups based on legally
170+
protected characteristics or categories; To provide medical advice and medical results
171+
interpretation; To generate or disseminate information for the purpose to be used for
172+
administration of justice, law enforcement, immigration or asylum processes, such as
173+
predicting an individual will commit fraud/crime commitment (e.g. by text profiling,
174+
drawing causal relationships between assertions made in documents, indiscriminate and
175+
arbitrarily-targeted use).

0 commit comments

Comments
 (0)