In Compatibility mode (XR) after a period of time running a game XR Tracking stops and objects flicker back and forth

[DanielSnd]

Tested versions

• Using a custom version/fork of 4.4.1stable with a few extra commits on top and some extra custom modules.

System information

Quest 3 Horizon OS v77

Issue description

In Compatibility mode (XR) after a period of time running a game XR Tracking stops and objects flicker back and forth.

When that happens the game gets into a super broken state with what I believe to be the view from both eyes flickering non-stop from one view to the other, no errors in the debugger or anything. In that state can't really do anything, the XR hand doesn't following my hand, the view doesn't follow the headset.

Video of the problem happening:
https://github.com/user-attachments/assets/c1d2a0e1-feef-4e52-bf44-f03654e3874a

Steps to reproduce

In compatibility mode I run into the issue when playing the game for long enough, but it could be related to something in my project as well, perhaps shaders, perhaps amount of models, unclear. Not sure if it's reproducible by any project.

Minimal reproduction project (MRP)

I have been running into it pretty consistently in my project, but it's a big project and it only happens after playing the game for a certain amount of time so I don't have a small MRP. I brought it up on the latest XR Meeting and it seems other people have run into the issue as well.