Getting Started with TensorFlow Lite on Seeed Studio XIAO nRF52840 Sense
This wiki will demonstrate how you can use TensorFlow Lite on Seeed Studio XIAO nRF52840 Sense and detect gestures such as punching and flexing using the on-board accelerometer. Here the data training will be done on the device itself.
For embedded AI applications, we highly recommond using the "Seeed nrf52 mbed-enabled Boards Library".
Software setup
Make sure to first follow "Getting Started with Seeed Studio XIAO nRF52840 (Sense)" wiki for the initial hardware and software setup.
Now let's move on to the rest of the software setup.
- Step 1. Download Seeed_Arduino_LSM6DS3 library as a zip file
- Step 2. Download tflite-micro-arduino-examples library as a zip file
- Step 3. Open Arduino IDE, navigate to
Sketch > Include Library > Add .ZIP Library...
and open both the downloaded zip files one after the other
- Step 4. Navigate to
File > Examples > Seeeed Arduino LSM6DS3 > IMU_Capture
to open IMU_Capture.ino
- Step 5. Upload the codes and open the Serial Monitor
Note: Once you upload the codes, it will not be executed automatically until you click Serial Monitor on the upper right corner of the Arduino window.
Train data
Punch action
While the serial monitor is open, start training data for the punch action.
- Step 1. Grab the Seeed Studio XIAO nRF52840 Sense on your palm and start punching forward and you will see some data printed out on serial monitor
Step 2. Repeat the punching action for about 10 times and you will see new data generated after each punch
Step 3. Copy and paste the complete serial monitor output to a text file and save the text file as punch.csv
Note: Please copy the first line as well which contains aX,aY,aZ,gX,gY,gZ
Flex action
- Step 1. Reconnect USB cable to Seeed Studio XIAO nRF52840 Sense, open serial monitor, grab the Seeed Studio XIAO nRF52840 Sense on your palm and start flexing inwards and you will see some data printed out on serial monitor
Step 2. Repeat the flexing action for about 10 times and you will see new data generated after each flex
Step 3. Copy and paste the complete serial monitor output to a text file and save the text file as flex.csv
Note: Please copy the first line as well which contains aX,aY,aZ,gX,gY,gZ
Generate TensorFlow Lite model file
Now we will generate a TensorFlow Lite model file (model.h), by using the punch.csv and flex.csv files that we created before.
- Step 1. Open this Python notebook which will help generate the model.h file that we need
- Step 2. Navigate to files tab on the left navigation panel, drag and drop punch.csv and flex.csv files inside
- Step 3. Inside Setup Python Environment section, change the code from pip install tensorflow==2.0.0-rc1 to pip install tensorflow
- Step 4. Navigate to
Runtime > Run all
to run all the code cells
- Step 5. Click Run anyway for the error message that pops up
- Step 6. Once all the code cells are executed, you will find a new model.h file generated under the files tab that saw before
Note: If you cannot see the above model.h file, refresh the page.
- Step 7. Right click on the file and click Download to download the file to your PC
Inference
Now we will use the downloaded TensorFlow Lite model file (model.h) to recognize the punch and flex actions from Seeed Studio XIAO nRF52840 Sense.
- Step 1. Navigate to the library path of Seeed_Arduino_LSM6DS3 Library (normally under Documents > Arduino > libraries > Seeed_Arduino_LSM6DS3), and visit examples > IMU_Classifier
Step 2. Replace the model.h file with the one that we downloaded before
Step 3. Double click IMU_Classifier.ino and upload the codes to the Seeed Studio XIAO nRF52840 Sense.
Punch action
Open serial monitor and perform a punching action. You will see that it will give a result close to 1 next to punch
Flex action
Perform a flexing action. You will see that it will give a result close to 1 next to flex
Resources
- [Webpage] TensorFlow Lite Documentation
Tech support
Please do not hesitate to submit the issue into our forum.