Description
Application: Pick & Place Workflow With Isaac Sim & MTC
I am developing a Franka Pick & Place Workflow Simulation application with Isaac Sim 4.5.0. My Isaac Sim app is integrated with MoveIt2, and I am able to plan/execute arm movement sometimes thanks to MTC. I am able to view the collision objects in RViz accurately (Figure 1 & 2). The objective is to pickup a cube and place it in a target destination.
Figure 1: Planning Scene in Isaac Sim
Figure 2: Planning Scene with collision objects displayed accurately in RViz
The motion_planner is mostly able to compute successful solutions without collisions enabled. In such cases, the arm attempts to grasp the cube located on the table surface. However, the movement is not perfect and grasping action is incomplete.
In a few cases , the motion planning task execution works E2E with collisions enabled as well. However the grasping still requires improvement. I have faced frequent issues that rendered the motion planner to fail (Figure 3, 4 & 5). I have tried making necessary adjustments to grasp-transformation offsets, collision object properties, gripper velocity etc. to circumvent some of these issues. As a matter of fact, they happen to persist as well with modifications around object source and destination translation co-ordinates in Isaac Sim.
It would be really helpful if anyone can suggest any feedback or improvements needed. I am new to MoveIt and hence definitely could use some guidance to solve these problems in order execute motion trajectories and grasping successfully.
Figure 3: Task Planning failure for generating grasp pose
Figure 4: Task Planning failure for approaching the object
Figure 5: Task Planning failure for approaching the object (another attempt with minor modifications)