Getting Started with MediaPipe on reTerminal
MediaPipe is a an open-source framework from Google for building multimodal (eg. video, audio, any time series data), cross platform (i.e Android, iOS, web, edge devices) applied ML pipelines. It is performance optimized with end-to-end ondevice inference in mind. Mediapipe is currently under active development and includes multiple demos, that can be run out-of-the box after installing Mediapipe on reTerminal.
ML solutions in MediaPipe
The following is list of solutions currently tested on reTerminal:
| Model | Result | Comments |
|---|---|---|
| Face Detection |
| Model complexity: 0 71.4 FPS 14 ms. per inference Model complexity: 1 21.2 FPS 47 ms. per inference |
| Face Mesh |
| 20 FPS, 50 ms. per inference with tracking 16.1 FPS 60 ms. without tracking |
| Pose |
| Model complexity: 1 11.8 FPS 85 ms. per inference Model complexity: 2 6.1 FPS 163 ms. per inference Model complexity: 3 -- FPS -- ms. per inference |
| Hand landmarks |
| Model complexity: 0 8.9 FPS 112 ms. per inference Model complexity: 1 4.4 FPS 226 ms. per inference |
Installation
Currently Python bindings are tested with both 32bit and 64bit Raspberry Pi OS images for reTerminal. For best performance it is recommended to use 64bit version.
Python bindings for 32bit version
sudo apt install ffmpeg python3-opencv
pip3 install mediapipe-rpi4
Python bindings for 64bit version
Pre-built wheels for Python 3.7 64bit OS were not available at the moment of writing of this article, so we compiled and shared them ourselves.
sudo apt install ffmpeg python3-opencv
wget https://files.seeedstudio.com/ml/mediapipe/mediapipe-0.8-cp37-cp37m-linux_aarch64.whl
pip3 install mediapipe-0.8-cp37-cp37m-linux_aarch64.whl
After installation is complete, try importing mediapipe package:
pi@raspberrypi:~/reterminal $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mediapipe
>>> mediapipe.__path__
['/home/pi/.local/lib/python3.7/site-packages/mediapipe']
>>>
Links to samples
You can find Sample applications in Seeed Python Machine Learning repository, inside examples/mediapipe folder.
F.A.Q
Q1: My company's policy doesn't allow us to use 3rd party binaries. How to compile MediaPipe from source?
You can compile MediaPipe for 32bit OS by following instructions here and for 64-bit version by following instruction here.
Resources
[Web Page] MediaPipe Official Webpage
[Web Page] MediaPipe Official Documentation