My son wanted me to make him a Harry Potter-like ‘talking portrait’ of myself for lab day. I had a few ideas on this and kicked it off at first with just motion detection using a camera and pi. However, I wanted to play specific videos depending on if the person was going into, or coming out of, a room, and the time of day. This would enable the ‘friendly reminder’ features my son wanted such as:
- Good night
- Go back to bed
- Good morning
- Clean your room
- Don’t forget to brush your teeth
- … and just a friendly hello here and there
I had been looking into APIs for detecting body parts (for exercise apps), and also detecting distances on images (for various B2V applications). Fritz AI checked the box on both of these, so I took this opportunity to get more familiar with the Fritz AI Human Pose Estimation product specifically. The Human Pose Estimation detects 17 key points of the body, including eyes, which is what I was interested in. The API was easy to use and worked pretty much as expected.
I used the recognition of eyes (presence of both) to determine if the person was entering or exiting the room. This, along with time of day, allowed for the customized videos to play. I used a video editing tool to make my picture more ‘portrait like’ and to add ‘creepy’ background music to the video since I knew my son would appreciate that 🙂 After creating a handful of videos to align with the list above, I added my portrait to the main view and when a person is detected using the front camera, the appropriate video was played looking like a natural segue from portrait. It was around bed time when I hung the new portrait on the wall. My son was totally geeked with a bunch of requests for ‘can you make it do this too …’
Lab Day Tech Used
- iPhone
- XCode – Swift
- FRITZ AI Human Pose Estimation
- Video Editor for sound and visual (poster) effect
- Construction paper
- Tape