Skip to content

[AudioLDM 2] Pipeline fixes #4738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Aug 25, 2023
Merged

[AudioLDM 2] Pipeline fixes #4738

merged 17 commits into from
Aug 25, 2023

Conversation

sanchit-gandhi
Copy link
Contributor

@sanchit-gandhi sanchit-gandhi commented Aug 23, 2023

What does this PR do?

Pipeline fixes:

  1. Use return_dict=False in the UNet to allow for torch.compile
  2. Use the ImagePipelineOutput when we return latents (not the AudioPipelineOutput)

Test fixes:

  1. Fast tests: don't override the dtype test (all CLAP params now respect the default dtype)
  2. Slow tests: Use Hub checkpoints (not local paths!)

@sanchit-gandhi sanchit-gandhi changed the title [AudioLDM 2] Pipeline + doc fixes [AudioLDM 2] Pipeline fixes Aug 23, 2023
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Aug 23, 2023

The documentation is not available anymore as the PR was closed or merged.

@@ -469,29 +469,6 @@ def test_save_load_optional_components(self):
# increase tolerance from 1e-4 -> 2e-4 to account for large composite model
super().test_save_load_optional_components(expected_max_difference=2e-4)

def test_to_dtype(self):
Copy link
Contributor Author

@sanchit-gandhi sanchit-gandhi Aug 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that huggingface/transformers#25682 is merged, overriding this test won't be necessary - CLAP will always have the default dtype of float32

@sanchit-gandhi sanchit-gandhi marked this pull request as ready for review August 23, 2023 12:19
@sanchit-gandhi
Copy link
Contributor Author

Ready for review @sayakpaul!

@sanchit-gandhi sanchit-gandhi merged commit 29a11c2 into huggingface:main Aug 25, 2023
@sanchit-gandhi sanchit-gandhi deleted the audioldm2-fix branch August 25, 2023 10:38
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
* fix docs

* fix unet docs

* use image output for latents

* fix hub checkpoints

* fix pipeline example

* update example

* return_dict = False

* revert image pipeline output

* revert doc changes

* remove dtype test

* make style

* remove docstring updates

* remove unet docstring update

* Empty commit to re-trigger CI

* fix cpu offload

* fix dtype test

* add offload test
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
* fix docs

* fix unet docs

* use image output for latents

* fix hub checkpoints

* fix pipeline example

* update example

* return_dict = False

* revert image pipeline output

* revert doc changes

* remove dtype test

* make style

* remove docstring updates

* remove unet docstring update

* Empty commit to re-trigger CI

* fix cpu offload

* fix dtype test

* add offload test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants