fix dummy import for AudioLDM2

[patil-suraj]

What does this PR do?

fix dummy import for AudioLDM2

[sayakpaul]

Not sure it wasn't caught in the CI :(

[patil-suraj]

I'm getting import errors locally when my transformers version is old, so it should return dummy objects, but the dummy objects are wrong. Also even though it's not caught in CI, why do we import the dummy AudioLDMPipeline instead of the dummy AudioLDM2Pipeline ?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patil-suraj]

Colab to reproduce: https://colab.research.google.com/drive/1BFDZWqHWj1eaQeSZX44SU4kyJXO7g4P3?usp=sharing
This breaks any imports from diffusers if the required transformer version is not met.

[sayakpaul]

I don't immediately see any problem with make fix-copies. Maybe it slipped through the PR reviews.

[patil-suraj]

I've attached colab to reproduce the issue.

[sayakpaul]

This breaks any imports from diffusers if the required transformer version is not met.

But from the looks of it, the error shouldn't be dependent on the transformers version, no?

[patil-suraj]

You can try the colab to reproduce the issue.

[patil-suraj]

• Here the AudioLDM2Pipeline2 is imported in main init https://github.com/huggingface/diffusers/blob/main/src/diffusers/__init__.py#L136
• If optional deps are not available we import dummy pipes/models, but here we don't import the correct dummy pipe https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/audioldm2/__init__.py#L12
• So AudioLDM2Pipeline2 is never defined if the deps are not met for that pipeline, but we still import it in main init
• Hence it breaks other imports as well.

This should not happen as we don't have hard dep on transformers version, imports should work for transformers<"4.27.0"

[sayakpaul]

A bit confused here.

I was able to do:

!pip install git+https://github.com/huggingface/diffusers.git

And then:

from diffusers import T2IAdapter

(no transformers installation)

[sayakpaul]

ImportError: cannot import name 'AudioLDM2Pipeline' from 'diffusers.pipelines.audioldm2' (/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/audioldm2/__init__.py)

Tells me that AudioLDM2Pipeline is not present in https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/__init__.py. Even though it's here:

diffusers/src/diffusers/pipelines/__init__.py

Line 49 in 4909b1e

from .audioldm2 import AudioLDM2Pipeline, AudioLDM2ProjectionModel, AudioLDM2UNet2DConditionModel

, it won't be able to execute because of

diffusers/src/diffusers/pipelines/audioldm2/__init__.py

Line 15 in 4909b1e

AudioLDMPipeline,

.

But even with that https://github.com/huggingface/diffusers/blob/main/tests/others/test_dependencies.py should have caught it.

[patrickvonplaten]

Great thanks! This is a tough issue to spot and here I don't expect our import tester to catch it.

[patrickvonplaten]

@sayakpaul you need to install transformers==4.26 in order to see the bug

[sayakpaul]

The CLAP model isn't available when transformers version is less than 4.27.0 so it should raise a dependency error.
But I guess it's not giving the right error message because of this:

diffusers/src/diffusers/pipelines/audioldm2/__init__.py

Line 15 in 4909b1e

AudioLDMPipeline,

.

Is my understanding correct?

[sanchit-gandhi]

Good catch - thanks @patil-suraj for the fix