Enable Near Mode for K4W

[dxli]

Hi,

I need to set my kinect for windows(K4W) in near mode.

with microsoft kinect SDK, it's done by the following line,

m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE );

Any help is appreciated on how to get the similar done in libfreenect.

Thanks,

dxli

[nneonneo]

There's no support for near mode at the moment. I expect the fix is probably a simple register write, but since I don't have a K4W unit, I can't test this out.

Any K4W-endowed hardware hackers willing to find out?

[dxli]

Do we have a tool to figure out the transmission through USB?

For example, collect registers set in cases of far-mode and near-mode, then find the difference.

Right now, I can keep everything else the same, and toggle the near-mode on/off by a single line from Windows Kinect SDK 1.0

[nneonneo]

You could use a tool like UsbLyzer to dump the communication between the Kinect and your computer. Turn off all the streams to avoid a flood of data, then toggle the near/far mode a few times. In theory, you will be able to capture a relatively small set of USB packets; we can examine those and find the packets corresponding to the near mode functionality.

[dxli]

hi nneonneo,

I have tested the UsbLyzer. I have difficulty in turning streams off. Still I keep the flag settings in regular intervals,

while(true){
OutputDebugString(L"change flags\n");
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_SUPPRESS_NO_FRAME_DATA);
Sleep(250);
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE);
Sleep(250);
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_DISTINCT_OVERFLOW_DEPTH_VALUES);
Sleep(500);
}

The data file is lzma compressed:

https://github.com/dxli/libfreenect/blob/master/b.ulz.lzma

[nneonneo]

Thanks! That's very helpful.

It looks like there are two registers involved. I'm not sure which is which, but here's the sequence:

t=0.35 write 0x0007 to reg 0x0015 (wiki claims this is an apparent no-op)
t=0.45 write 0x0000 to reg 0x02EF (new register, could be near mode enabler)
t=0.70 write 0x001E to reg 0x0015 (default value for this register)
t=0.80 write 0x0190 to reg 0x02EF

t=1.55 write 0x0007 to reg 0x0015
t=1.65 write 0x0000 to reg 0x02EF
t=1.90 write 0x001E to reg 0x0015
t=2.00 write 0x0190 to reg 0x02EF

So, setting the flag seems to take 100ms, and otherwise the looping flag changes are very obvious. I would suggest trying to set reg 0x02EF to 0x0190 in libfreenect and seeing if that will trigger near mode (you may want to write reg 0x0015 first, just in case?)

[nneonneo]

The fact that 0x190 = 400 lends credence to the notion that 0x2EF is the near-mode enable, because 400mm is the closest distance that the Kinect sensor will see in near mode, and also the difference between the near-mode minimum distance and the normal-mode minimum distance (800mm).

[dxli]

Hi nneonneo,

Now, I got near-mode with libfreenect. however, it's still very puzzling at this point, because I can not get far mode back unless I try the kinect with windows SDK.

I can set the register 0x2ef to 0x320 (800), or simply skip setting it, while still get the kinect camera in near mode.

my code to tweak the 0x2ef is here:

dxli@5cb5051

thanks!

[nneonneo]

Try setting 0x2EF to 0x0000, and also try setting 0x0015 to 0x0007 before setting 0x02EF.

[dxli]

Hi,

I have tested all the combinations here, with one hang with setting reg
0x15 to 7 and reg 0x2ef to 0, although it doesn't always hang.

I'm not able to get far mode back yet.

thanks,

dxli
On Mar 8, 2012 3:09 PM, "Robert Xiao" <
reply@reply.github.com>
wrote:

Try setting 0x2EF to 0x0000, and also try setting 0x0015 to 0x0007 before
setting 0x02EF.

---

Reply to this email directly or view it on GitHub:
#274 (comment)

[nneonneo]

Make sure you stop the depth image before setting any of these
registers. Also try waiting 100ms between setting 0x015 and 0x02EF, as
the K4W library seems to do that.

Robert

在 2012-3-8，15:57，Dongxu Li
reply@reply.github.com
写道：

Hi,

I have tested all the combinations here, with one hang with setting reg
0x15 to 7 and reg 0x2ef to 0, although it doesn't always hang.

I'm not able to get far mode back yet.

thanks,

dxli
On Mar 8, 2012 3:09 PM, "Robert Xiao" <
reply@reply.github.com>
wrote:

Try setting 0x2EF to 0x0000, and also try setting 0x0015 to 0x0007 before
setting 0x02EF.

---

Reply to this email directly or view it on GitHub:
#274 (comment)

---

Reply to this email directly or view it on GitHub:
#274 (comment)

[dxli]

Hi Robert,

I have tested setting registers early, but still couldn't get far mode back. Here are my changes:

https://github.com/dxli/libfreenect/tree/nearMode

If you want to me to test other ideas, you may send me pull requests or patches.

Thanks a lot already!

dongxu

[nneonneo]

Actually, are you certain it is in near mode? It's actually possible that I got this backwards: 7 to 0x15 and 0 to 0x2EF might be near mode enable; 0x1E to 0x15 and 400 to 0x2EF might be near mode disable. I'm basing this on the fact that the enable and disable are 250ms apart in both the code you sent and the UsbLyzer output...

Can you please double-check to make sure the K4W unit is really being set to a near mode? The easiest way to find out if the toggle does anything would be to make a function which toggles the value, then run something like

{
  freenect_stop_depth(dev);
  freenect_set_near_mode(dev, 1);
  freenect_start_depth(dev);
  for(int i=0; i<50; i++) { freenect_process_events(ctx); } // process events normally for 50 frames

  freenect_stop_depth(dev);
  freenect_set_near_mode(dev, 0);
  freenect_start_depth(dev);
  for(int i=0; i<50; i++) { freenect_process_events(ctx); } // process events normally for 50 frames
}

in, e.g. glview. Then, you should be able to see the depth data changing if the settings are working.

[dxli]

Hi Robert,

First, I'm 100% sure that the camera is in near mode, because I can reboot to windows and get far mode, and the difference between 400mm and 800mm is beyond any doubt.

I run out of ideas here, after tried all suggestions here. If there are new ideas to test, please send me a pull request or patch.

Thanks,

dxli

[nneonneo]

I feel like we missed something. Just a few more things to try off the top of my head:

1. Are you using DEPTH_REGISTERED or DEPTH_11BIT in libfreenect?
2. Could you post another UsbLyzer log with the depth data intact, switching between just near and far modes? Ideally, use different delays (e.g. 1200ms, 1900ms) so that it's clear which is which in the output.
3. The K4W library actually changes the registers (apparently) without stopping the depth stream in-between. I suppose you could try that, though it definitely seems weird.

[renewagner]

It turns out that what the above patch did was to enable and then immediately disable near mode again. Even in default mode the Kinect will sense objects as close as 500mm in front of the sensor. You really want to check whether you get depth info when the object is closer than 500mm.

The sequence to enable near mode is:
write 0x0007 to reg 0x0015
write 0x0000 to reg 0x02EF

To disable it:
write 0x001E to reg 0x0015
write 0x0190 to reg 0x02EF

I've implemented this in my libfreenect fork:

renewagner@34f4c1a

Cheers,

Rene