[ldm3d] first PR

[estelleafl]

This is the first PR of the LDM3D pipeline
For reference: LDM3D: Latent Diffusion Model for 3D: https://arxiv.org/abs/2305.10853

Example of use:

from diffusers import StableDiffusionLDM3DPipeline
pipe_ldm3d = StableDiffusionLDM3DPipeline.from_pretrained("Intel/ldm3d")
pipe_ldm3d.to("cuda")
prompt ="A picture of some lemons on a table"
name = "lemons"
rgb_image, depth_image = pipe_ldm3d(prompt).images
rgb_image[0].save(name+"_ldm3d_rgb.jpg")
depth_image[0].save(name+"_ldm3d_depth.png")

[patrickvonplaten]

Hey @estelleafl,

Is there some reference to the original code base?

[estelleafl]

Hi @patrickvonplaten

I am from the Intel organization and one of the authors of the paper. We are not planning to have any other opensource release.

[abhiwand]

@patrickvonplaten and @patil-suraj we look forward to your valuable feedback and suggestions.

[dheera]

@estelleafl I was wondering if there might be an issue with the model URL.

Repository Not Found for url: https://huggingface.co/api/models/Intel/ldm3d. 

I'm also not seeing anything at https://huggingface.co/Intel/ldm3d

Thanks for publishing this!

[patrickvonplaten]

Very cool!

[sayakpaul]

@patrickvonplaten could you give this final look? For me, it looks good to go 🚢

[sayakpaul]

There is one issue on the safety checker side which needs to be rejigged. @estelleafl is currently taking care of it as discussed over Slack.

[estelleafl]

this should be done in this commit 3a93f8a

[patrickvonplaten]

Also good to go for me once this: https://github.com/huggingface/diffusers/pull/3668/files#r1229159929 is treated :-)