A real-time robot tracking system that follows a person using skeletal detection and depth sensing, built by a team of engineering students at ENISo.

🎓 Project by: Mohamed Ali Jomaa Ghouil, Yasmine Saad, Yahya Ben Turkia
🧑‍🏫 Supervised by: Dr. Lamine Houssein — Engineer in Mechatronics, PhD in Robotics, Assistant Professor at ENISo

Design and implement a robot capable of:

• Detecting a person using real-time pose estimation
• Reacting to a hand gesture to start/stop following
• Maintaining a safe distance (~2 meters) using depth camera
• Sending movement commands via Modbus TCP

(not available yet)

• 📷 Intel RealSense D435i — Color + Depth Camera
• 🧍‍♂️ MediaPipe Holistic — Real-time skeleton & gesture tracking
• 🧠 Python 3.11 with:
  • opencv-python (video processing)
  • numpy
  • pyrealsense2
  • mediapipe
  • pyModbusTCP
• 🧰 Ubuntu/Linux + VS Code
• ⚙️ Modbus TCP for robot control

• Detects shoulders → calculates chest center
• Uses depth data to determine distance
• Calculates:
  • Vx (linear speed) → forward/backward
  • Wz (angular speed) → left/right rotation
• Gesture Recognition:
  • Raise right hand (with open hand gesture) → toggle following
  • Robot stops if target lost > 10 seconds

• Press f — Manually toggle follow mode
• Press c — Activate gesture mode (to start/stop with hand)
• Press u — Deactivate gesture mode
• Press q — Quit the program

🎥 **Demo ** – Robot following a person and Wheel speed change when turning(screen recording)
▶️ Watch on Google Drive

├── person\_following.py                    # Final Python code
├── requirements.txt                      # Dependencies list
├── Person\_Following\_Robot\_Documentation.pdf # Full technical doc (in frensh)(english version will be available soon)
├── person\_following\_robot\_architecture.png # System diagram
├── /videos                                # Demo videos (optional)
└── /screenshots                           # Output images

• Ubuntu 20.04+ with Python 3.11
• Intel RealSense SDK installed
• A working Modbus-compatible robot (or run in simulation)

git clone https://github.com/YOUR_USERNAME/person-following-robot.git
cd person-following-robot

python3 -m venv open3d_env
source open3d_env/bin/activate

pip install -r requirements.txt

python3 -m pymodbus.server --host 127.0.0.1 --port 1502

python3 person_following.py

📧 [email protected] 🔗 LinkedIn – Mohamed Ali Jomaa Ghouil

National Engineering School of Sousse (ENISo) Mechatronics Department – Semester 2 Project – May 2025