Experimental Study on the Imitation of the Human Head-and-Eye Pose Using the 3-DOF Agile Eye Parallel Robot with ROS and Mediapipe Framework

Download

• Paper (IEEE Xplore)

Abstract

This paper proposes a method to mimic human face and eye movements by combining computer vision techniques and neural network concepts. A 3-DOF spherical parallel robot (Agile Eye) is used for head movement imitation, with a 2-DOF mechanism attached to its end-effector for eye movement. The study uses the Mediapipe library to extract face mesh data and obtain the roll, yaw, and pitch angles. A linear regression model was trained to calculate face angles, and experimental tests conducted on a ROS platform demonstrated the effectiveness of the proposed methods in tracking and imitating human head and eye movements with high accuracy.

Citation

Radmehr, Amir, Milad Asgari, and Mehdi Tale Masouleh. 2021. “Experimental Study on the Imitation of the Human Head-and-Eye Pose Using the 3-DOF Agile Eye Parallel Robot with ROS and Mediapipe Framework.” In 2021 9th RSI International Conference on Robotics and Mechatronics (ICRoM), 579–584. IEEE. https://doi.org/10.1109/ICRoM54204.2021.9663445.

@inproceedings{radmehr2021agile,
  author = {Radmehr, Amir and Asgari, Milad and Masouleh, Mehdi Tale},
  title = {Experimental Study on the Imitation of the Human Head-and-Eye Pose Using the 3-DOF Agile Eye Parallel Robot with ROS and Mediapipe Framework},
  booktitle = {2021 9th RSI International Conference on Robotics and Mechatronics (ICRoM)},
  pages = {579--584},
  year = {2021},
  doi = {10.1109/ICRoM54204.2021.9663445}
}