AttributeError: type object 'CosmosTransformer3DModel' has no attribute 'from_single_file'

[mingyi456]

Describe the bug

I would like to run the Cosmos-Predict2-14B-Text2Image model, but it is too large to fit in 24GB of VRAM normally, so I tried to load a Q8_0 GGUF quantization. I copied some code from the HiDreamImageTransformer2DModel page and tried to adapt it, but I get the following error:

AttributeError: type object 'CosmosTransformer3DModel' has no attribute 'from_single_file'

Is there supposed to be another way to load a 8 bit quantization? From what I have seen, Q8_0 typically produces results that are much closer to full precision compared to FP8.

Reproduction

transformer = CosmosTransformer3DModel.from_single_file(
    rf"{model_14b_id}\cosmos-predict2-14b-text2image-Q8_0.gguf",
    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
    torch_dtype=torch.bfloat16
)
pipe_14b = Cosmos2TextToImagePipeline.from_pretrained(
    model_14b_id,
    torch_dtype=torch.bfloat16,
    transformer = transformer
)

Logs

    transformer = CosmosTransformer3DModel.from_single_file(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: type object 'CosmosTransformer3DModel' has no attribute 'from_single_file'

System Info

• 🤗 Diffusers version: 0.35.0.dev0
• Platform: Windows-10-10.0.26100-SP0
• Running on Google Colab?: No
• Python version: 3.11.9
• PyTorch version (GPU?): 2.7.1+cu128 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.33.1
• Transformers version: 4.53.0
• Accelerate version: 1.8.1
• PEFT version: 0.15.2
• Bitsandbytes version: 0.46.1
• Safetensors version: 0.5.3
• xFormers version: not installed
• Accelerator: NVIDIA GeForce RTX 4090, 24564 MiB
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Who can help?

@DN6

[a-r-r-o-w]

If you use the main branch, you should be able to use from_single_file directly. I'm not sure about GGUF compatibility yet of Cosmos (cc @DN6).

Since you mention you made custom modifications follow HiDream, you probably just need to derive the FromOriginalModelMixin in your CosmosTransformer3DModel implementation:

diffusers/src/diffusers/models/transformers/transformer_cosmos.py

Line 381 in 425a715

class CosmosTransformer3DModel(ModelMixin, ConfigMixin, FromOriginalModelMixin):

[mingyi456]

After installing the main branch of diffusers via pip install git+https://github.com/huggingface/diffusers, I get the following error with my code:

Traceback (most recent call last):
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_http.py", line 409, in hf_raise_for_status
    response.raise_for_status()
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\requests\models.py", line 1026, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/transformer/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\diffusers\configuration_utils.py", line 390, in load_config
    config_file = hf_hub_download(
                  ^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 1008, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 1071, in _hf_hub_download_to_cache_dir
    (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) = _get_metadata_or_catch_error(
                                                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 1533, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
               ^^^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 1450, in get_hf_file_metadata
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 286, in _request_wrapper
    response = _request_wrapper(
               ^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\file_download.py", line 310, in _request_wrapper
    hf_raise_for_status(response)
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_http.py", line 420, in hf_raise_for_status
    raise _format(EntryNotFoundError, message, response) from e
huggingface_hub.errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-686a0463-7c7db661382b4960584104e7;fc417385-36b0-4219-b953-82b348f935cd)

Entry Not Found for url: https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/transformer/config.json.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2\app 14b-q8_0.py", line 35, in <module>
    transformer = CosmosTransformer3DModel.from_single_file(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\diffusers\loaders\single_file_model.py", line 364, in from_single_file
    diffusers_model_config = cls.load_config(
                             ^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "F:\AI setups\CosmosPredict2\Cosmos-Predict2-venv\Lib\site-packages\diffusers\configuration_utils.py", line 417, in load_config
    raise EnvironmentError(
OSError: stable-diffusion-v1-5/stable-diffusion-v1-5 does not appear to have a file named config.json.

Not sure why it is attempting to download a completely unrelated repo? Or is it something wrong with my code?

By the way, the variable model_14b_id is set to r"..\Cosmos-Predict2-2B-Text2Image", because I dumped the GGUF file into the folder containing the uncompressed 2B model. That should not be the cause of the issue right?