As a midterm project in Intro to Robotics & Mechatronics at Tufts, I built a smart teleprompter which tracks the speakers face. I completed this project by myself over the course of one week. The teleprompter uses the Arduino-based WIO Terminal, the LEGO Spike Prime, and a Raspberry Pi 4.
I quickly built the teleprompter structure out of LEGO, and got to work on the software. The teleprompter incorporated several tricky problems, including communication between microcontrollers, facial tracking, and PID control. To accomplish tracking of the user's face, I used OpenCV with Haar Cascades. Using this method, I was able to detect faces and locate their center.
With facial tracking from the Raspberry Pi working, I built software to control the LEGO Spike Prime microcontroller from the RPi. Using serial communication, I wrote code to send Python commands over a USB cable, encoded as binary. These commands then ran on the Python prompt on the Spike Prime. This allowed me to control the LEGO motor from the Raspberry Pi.
Next, I wrote a PID controller to move the LEGO DC motor to the appropriate location to track the user's face. After tuning the controller, the teleprompter was able to follow a user. To add text to the display, I wrote Arduino code to display text on the screen of the WIO terminal and scroll as the user reads.
Overall, this project ended up working well and helped develop software skills that I've used heavily since.