[Question] Finger Joint Motion Amplitude Mismatch in Third-Party Hand Retargeting

[wudinggui-qwe]

I'm encountering an issue where the finger joint motion amplitude is significantly reduced when retargeting hand movements from DexPilot to a third-party robotic hand using the rohand_left_dexpilot.yml configuration. The wrist position and rotation are accurately mapped, but all finger joints exhibit a consistent reduction in range of motion despite the scaling_factor being set to 1.0 in the configuration file.

keywords: openxr, vision pro, dex-retargeting, pinkik

Initially, there was an nv != nq issue when running the simulation, which was later resolved by changing the joint type from continuous to revolute. This modification might also be related to the current problem.

Expected Behavior:
Finger joint movements should reflect the full range of motion from the real-world input, with no attenuation.

Actual Behavior:
Simulated finger joint movements show a reduced range of motion compared to the real-world input. For example, while the physical hand may achieve 90° of flexion, the hand in the simulation environment remains in a curled/flexed state.

The following ranges represent the variation in teleoperation data for specific joints:
'lefthand_if_proximal_link': (-2.1261518,-1.46026433),
'lefthand_if_distal_link': (-1.22051189, -0.88547993),
'lefthand_lf_proximal_link': ( -2.1261518, -1.46026433),
'lefthand_lf_distal_link': (-1.22051189, -0.88547993),
'lefthand_mf_proximal_link': (-2.1261518,-1.46026433),
'lefthand_mf_distal_link': (-1.22051189, -0.88547993),
'lefthand_rf_proximal_link': (-2.1261518, -1.46026433),
'lefthand_rf_distal_link': (-1.22051189, -0.88547993),
'lefthand_th_root_link': ( 0.80475027, 1.59655143),
'lefthand_th_proximal_link': (-2.1261518, -1.46026433),
'lefthand_th_distal_link': (-1.22051189, -0.88547993)

Request for Insights:
Has anyone else experienced similar amplitude mismatches with third-party hand models in DexPilot retargeting? Any suggestions for debugging or resolving this issue would be greatly appreciated.

rohand_dex_retargeting_utils.txt is the corresponding retargeting/mapping utility class file.
rohand_left_dexpilot.txt is the rohand_left_dexpilot.yml configuration.

rohand_dex_retargeting_utils.txt
rohand_left_dexpilot.txt

[RandomOakForest]

Thank you for posting this. Which versions of Isaac Sim and Isaac Lab are you using? Could you share more details about how you are implementing this and the assets you are working with?

[wudinggui-qwe]

Thanks for your interest in our project. We're currently using Isaac Sim version 4.5 and Isaac Lab version 2.1 in Ubuntu 22.04. As for the implementation, we've integrated the robot's dexterous hand model into Isaac Sim by loading its URDF file.
Here's the URDF file for the dexterous hand we're utilizing

rohand_left.txt

rohand_left.txt is the corresponding urdf file.

[RandomOakForest]

Thank you for following up. I will move this post to our Discussions section for the team to reply.