The official code of "Analyzing the Synthetic-to-Real Domain Gap in 3D Hand Pose Estimation. CVPR 2025."

Paper Link

bpy is used to call different blender API.

Here are steps to install bpy:

1. Create conda environment according to blender python module version (usually python 3.10.0, please refer to the cp version shown in bpy whl below)

   conda create --name yourEnv python=3.10.0
   

2. Download bpy-3.6.0-cp310-cp310-manylinux_2_28_x86_64.whl from Links_for_bpy

3. Install bpy

   pip install bpy-3.6.0-cp310-cp310-manylinux_2_28_x86_64.whl
   

blender is the engine to run the rendering script.

Here are steps to install blender:

1. Download blender Linux from https://www.blender.org/download/lts/3-6/ (Recommend to choose the 3.6 version rather than 4.2)

2. export PATH:

   export PATH=/home/zhuoran/blender-3.6.X-linux-x64:$PATH
   

   Replace "X" with the version you downloaded.

• Pytorch3D installation.
• bpycv installation

1. Download the NIMBLE models from NIMBLE assets provided by NIMBLE.

2. Create an assets folder under ./my_NIMBLE_model and place the downloaded models under /my_NIMBLE_model/assets/.

3. Download the MANO_RIGHT.pkl from MANO in the Models & Code section. Place the MANO_RIGHT.pkl under /my_NIMBLE_model/assets/.

We have uploaded our HDRI scenes for rendering in Hugging Face. Download the images and set the hdri_bg_path in config_syn_data.json.

• render_syn_data.py: image rendering script
• my_NIMBLE_model/main.py: NIMBLE mesh generation script
• my_NIMBLE_model/view_samples.ipynb: image and annotation viewing script

cd my_NIMBLE_model

python main.py

• Linux:

  /home/zhuoran/blender-3.6.X-linux-x64/blender --background --python render_syn_data.py -- 100 1 0
  

  Replace "X" with the version you downloaded.

• MAC (Require installing blender application on MAC first):

  /Applications/Blender.app/Contents/MacOS/blender --background --python render_syn_data.py -- 100 1 0
  

You can specify your configurations in config_syn_data.json.

Use view_samples.ipynb to visualize image and annotation:

We have uploaded the synthetic hand data to Hugging Face. To use syn_frei, syn_dex, and syn_occ, you need to first download their annotations from FreiHAND, DexYCB, and preprocessed DexYCB, and Obman. The dataloaders can be found in the synthetic_data folder.

If you find our paper useful, please consider citing our paper.

@InProceedings{Zhao_2025_CVPR,
    author = {Zhao, Zhuoran and Yang, Linlin and Sun, Pengzhan and Hui, Pan and Yao, Angela},
    title = {Analyzing the Synthetic-to-Real Domain Gap in 3D Hand Pose Estimation},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month = {June},
    year = {2025},
    pages = {12255-12265}
}

Special thanks to these great projects: NIMBLE, blender-cli-rendering, bpycv, S2HAND.