Edge blur and line artefacts

[stephanedebove]

Description

Hi,

thanks for the great software!

I’m running into strong artefacts in my animations, both in the preview window and in the exported files. Specifically, I have some kind of blur around some edges and some white lines appearing on some objects. I haven’t seen anything in the doc about this, except that increasing the --ssaa to 2 can help (but it doesn’t really in my case).
Have I missed something? I don’t see these artefacts in most of the example videos I see online.

I’m using the auto-generated depth map if that can be the problem.

[Tremeschin]

Hey there!

Could you send a video example with the artifacts you're mentioning? (drag and drop here in a github comment)

I might be making wrong assumptions in this comment, a video will help further discussion! Source image/depthmap are also appreciated. Click to expand any of these:

blur around some edges and some white lines appearing on some objects

If you're referring to the "stretchy" edges by blur, sadly there's not much we can do here, the information simply doesn't exist in the source image. There's some effort in finding a way to improve this point on issue #82, just a matter of time!

Nevertheless, I have some perceptual quality improvements on the upcoming v0.9 version in the math and animations, you can get a pre-release with pip install depthflow==0.9.0.dev1, full release with executables etc might take a bit

Not sure of the white lines, maybe they are thin foreground objects blending with the background? Would need the example video to tell properly (related to next point)

I’m using the auto-generated depth map if that can be the problem.

The automatic ones the webui/realtime window is the right way to go, shouldn't be the issue here!

However, raw data from estimators is kinda bad, as they are too precise. My code does some post-processing to fatten the edges for smoother transition between high contrast layers, but it can add a minor "aura" on complex objects 🤔

except that increasing the --ssaa to 2 can help

That helps with jagged/aliased edges, it's the easiest to implement but most expensive form of anti-aliasing (you might know fxaa, taa, etc. from games). It mostly helps here with the stability of ray intersections when they switch layers as the algorithm is discrete (see "Parallax Occlusion Mapping" section on this link).

Can go as high as ssaa=4 in v0.9 using 16x more GPU, each pixel being the average of the 16 nearby data lol

But yea, it won't solve any of the stretch or "aura" thing, if anything it makes the final image closer to the idealistic output (coupled with quality=100, which is kinda overkill), using 2 is fine for almost all cases in a final render

Let me know if this helps!, I can help you further with a example video 🙂

[stephanedebove]

Thanks for the reply !

Source image is :

Command is simply:

depthflow input -i "img.jpg" main -o "output.mp4"

and this gives (blur is clearly visible, for the lines I’m mentioning you will probably want to be in fullscreen, they follow the megaphone contours for instance):

output.mp4

[Tremeschin]

Oh no, I remember this one a while back testing on a friend's computer..

Do you happen to be on an AMD GPU on Windows? If not, what is your combination of GPU and OS?

This is a difference in how each platform's driver compiles the shader code and/or executes it, or maybe bad data formats. I've had many issues with macOS in the past and some AMD on Linux too, this one is quite serious yea, I thought it was isolated to his setup

I'm quite sure I couldn't reproduce it on my RX 570 on W11, and my friend was on a 6700XT. Mind sharing your model too so we can find a pattern?

Will see if he has free time so I can remote code and test on his machine, but at first it's unknown to me why this is happening, if you're not on AMD then it'll get complex haha

[stephanedebove]

I have an NVIDIA Gefore RTX 4060 Laptop GPU and Windows 11.

If there’s no easy fix, is there a way to run Depthflow online through an API, that i would call from my local computer?

[Tremeschin]

Uh, problem is, it really should work in your system, I've no reports of it failing this way on nvidia laptops hybrid or not

Also, is it using the NVIDIA GPU in the terminal logs? something like:

│DepthFlow├┤5'15.609├┤INFO   │ ▸ (Module  1 • DepthScene  ) Initializing scene 'DepthScene' with backend WindowBackend.GLFW
│DepthFlow├┤5'15.787├┤INFO   │ ▸ (Module  1 • DepthScene  ) OpenGL Renderer: NVIDIA GeForce RTX 3060/PCIe/SSE2

Maybe you have an integrated GPU and it's using it instead, causing pixel ownership issues? It can be swapped on windows/nvidia control panel, I never used a hybrid system if that's the case to tell, need to search 😅

[stephanedebove]

Ha you’re right! It’s using the integrated GPU by default:

│DepthFlow├┤2'09.755├┤INFO   │ ▸ (Module  1 • DepthScene  ) Initializing scene 'DepthScene' with backend WindowBackend.GLFW
│DepthFlow├┤2'10.189├┤INFO   │ ▸ (Module  1 • DepthScene  ) OpenGL Renderer: AMD Radeon 780M Graphics

For others running into this problem, you can fix it with:

• Windows key → NVIDIA Control Panel.
• Go to Manage 3D Settings → Global Settings tab.
• Under Preferred graphics processor, select High-performance NVIDIA processor.
• Click Apply.

Now works flawlessly. Thanks!

[Tremeschin]

Oh, awesome!, thanks for the steps, will surely improve the documentation on that one for the next release!

Sneaky AMD GPUs hehe, but that's still an issue to solve for windows systems 🙂