Version 0.2.0-7
Finding the fun, while teaching machine learning with microcontrollers to the general population.
QR Code is for this link https://hpssjellis.github.io/tinyMLjs/public/acceleration/a00-best-acceleration.html To the main acceleration TinyMLjs webpage
TinyMLjs by Jeremy Ellis. My Github Profile at https://github.com/hpssjellis
- When I heard of the ICTP Workshop on Widening Access to TinyML Network by Establishing Best Practices in Education, I was eager to connect with like-minded educators who could collaborate on creating an engaging method for teaching machine learning on microcontrollers to a wider audience.
- By "fun," I mean an approach that is open-source, powerful, fosters a passion for learning, enables building proof of concepts, affordable, fast, user-friendly, operates on the client-side, ensures security, covers the entire process, supports future edge devices, is hardware and internet/cloud independent.
- I am Jeremy Ellis, known online as @rocksetta, jerteach, or hpssjellis. As an unconventional learner, I am self-taught in machine learning but probably have no chance of a PhD. My strength lies in simplifying technology.
- Around 2017 I made a machine learning curriculum based on TensorflowJS, but deprecated it when I found out about EdgeImpulse.com
- My Robotics course is called Maker100 based on the Arduino PortentaH7 with LoRa vision Shield and the corresponding PortentaH7 library is called the Portenta Pro Community Solutions with over 100 of my examples relevant to my course.
- With 48 years of computer programming experience, 35 years of teaching high school coding, 30 years of obsession with coding neural networks, and 8 years of teaching robotics and TensorflowJS on microcontrollers (the last 3 using EdgeImpulse.com), the only constant in my journey has been the deprecation of my work.
- The only consistent thing about teaching coding for that many years is the amount of times all my work has been deprecated! It doesn't matter if the cloud platform has been sold: (Cloud9 to AWS) or the software has been updated: (Too many to mention), or the IDE has changed (Arduino IDE 1.8.19 to 2.10), the board has changed (too many to mention) the software has changed (Python 2 to Python 3), and each deprecation destroys any relevant lesson plans or videos.
- One of the methods that has been reasonably stable is Javascript, mainly because it's script tag can be versioned
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]"> </script> - However, converting TensorflowJS machine learning to microcontrollers has been a challenge. Although I managed to accomplish it years ago here (though it may be deprecated now), it may no longer be necessary. If data from any hardware can be saved as a CSV file, it can later be loaded onto any cloud platform.
- At the start of June 2023 I had successfully implemented desktop webSerial/polyfill (also works on Android Pixel Phones) and achieved functionality for saving microcontroller data to CSV files. I also experimented with using coPilot, chatGPT, and BingChat.
- In summary, I accomplished in a much shorter timeframe what I initially believed would take a year. On a webpage, we can now load raw CSV data or data directly from a connected microcontroller using the
println()command to a desktop or laptop computer. We can clean the data if necessary, convert it to a tensor, train a machine learning model, load more data, clean and classify it, and finally send the classification results back to the microcontroller (e.g., turning on an LED, etc). - All this functionality is encapsulated in approximately 1000 lines of code on a single webpage, allowing for quick testing of machine learning viability with any sensor data from any microcontroller.
- Since CSV files can be saved, you can now load that data onto your favorite cloud platform, such as EdgeImpulse.com, for microcontroller programming.
Let's have a look at sections of the webpage:
Click Choose Files to select CSV files. Currently, the file name is important, and there are no column headings—just raw, cleaned data.
Information here about number of samples, and sensors. Click Convert Data to Tensor then Train Model. View console ctrl-shift-i for any issues
Here we can save the model or upload a previously saved model. Note: Labels are not loaded with the model. This is a work in progress.
This section focuses on tuning a vision model.
Here, we can tune a sound model.
This section demonstrates the model for acceleration or any other sensor combination.
Here is where we connect a microcontroller Connect via Serial Port Then Clear and send Start
If needed Clean the data and check the label name and Keep and/or Save CSV checking the file name.
When using more than two labels, you can return to the model training part of the webpage to train your model.
Now it is time to test your model. Load more data, clean if needed and click Classify Data
Note that the code to be loaded onto the Nano33BleSense (Rev1) is displayed in the textarea for easy copying.
Tutorial playlist Video here
Direct link https://youtu.be/3f4led32SL8
The github is at: https://github.com/hpssjellis/tinyMLjs
The index webpage is at https://hpssjellis.github.io/tinyMLjs/public/index.html
While this presentation represents a starting point, it demonstrates that powerful, proof of concept, end-to-end machine learning on edge devices does not have to rely on the cloud or specific hardware. It can be done in the field or in a classroom without internet access.
By Jeremy Ellis @rocksetta
Github Profile at https://github.com/hpssjellis