Learn how to Handle an M5Stack Core2 for AWS – C/C++
15 min read
The primary article mentioned M5Stack administration primarily based on the Micropython language. Now, we have to dive a lot deeper right into a rabbit gap. Let’s attempt to use C and C++ solely. An important benefit of utilizing C is the potential of full, low-lever management of all controller facets. An important drawback of utilizing C is the need of full, low-lever management of all controller facets. Effectively… with nice energy comes nice duty.
FreeRTOS
AWS FreeRTOS is a real-time working system devoted to AWS cloud and resource-constrained gadgets.
There’s lots of code to write down this time, so we’ll use an instance straight from AWS. There is no such thing as a must burn any firmware with the burning device; nevertheless, we nonetheless must cross the USB port to the WSL atmosphere utilizing usbip, as we’ve performed within the „Micropython” part of the primary chapter.
You may obtain the code we’re going to make use of from https://github.com/m5stack/Core2-for-AWS-IoT-EduKit.git . The one subdirectory we want is Blinky-Hey-World, however the repository is de facto small, so utilizing a sparse checkout is pointless, and you’ll merely clone your entire repo.
Open VSCode and set up a plugin referred to as PlatformIO. There’s a bug in PlatformIO, so you’ll be able to’t see any information out of your WSL atmosphere utilizing the PlatformIO browser (Windows WSL: I can’t open any files(projects) in the PIO open browser. · Issue #2316 · platformio/platformio-home · GitHub). To repair it, shut VSCode, edit ~/.platformio/packages/contrib-piohome/principal.*.min.js file in Home windows, change „”: „/” with „/”: „/”, and open VSCode once more.
To confirm the connection between PlatformIO and your controller, open PlatformIO from the very left menu after which decide „Units” from the principle left menu. You need to see /dev/ttyUSB0 within the middle a part of the display. Please keep in mind to cross the USB machine to WSL utilizing usbip and to permit all customers to make use of the port with chmod.
If every little thing appears to be like good up to now, you’ll be able to open the Blinky-Hey-World listing (not your entire cloned repository) as a challenge from the PlatformIO dwelling display. Now you’ll be able to observe the important components of the official instruction offered beneath.
You must have AWS CLI v2 put in in your machine. If you happen to don’t, you’ll be able to set up it utilizing the official guide: Installing or updating the latest version of the AWS CLI – AWS Command Line Interface (amazon.com)
Now guarantee you will have a legitimate token, and you’ll work together together with your AWS account utilizing CLI (I suggest itemizing some sources because the verification, e.g., aws s3 ls).
We are going to use the built-in script to create a Factor in AWS IoT. Simply open a terminal utilizing PlatformIO (commonplace bash terminal gained’t work, so you must open it from Miscellaneous -> New Terminal from the principle PlatformIO menu in VSC), be sure to’re in Blinky-Hey-World listing, and run pio run -e core2foraws-device_reg -t register factor. The script will create the Factor and obtain the required certificates/key information. You are able to do it manually in case you don’t belief such scripts; nevertheless, this one is created by the AWS crew, so I imagine it’s reliable.
Within the AWS IoT console, go to Handle -> All gadgets -> Issues and see the brand new Factor created by the script. The Factor identify is autogenerated. In my case, it’s 0123FAA32AD40D8501.
OK, the following step is to permit the machine to hook up with the Web. There’s one other script that can assist you with this job. Name pio run ‐‐atmosphere core2foraws ‐‐goal menuconfig. You’ll see a easy menu. Navigate to AWS IoT EduKit Configuration and arrange WiFi SSID abd WiFi Password. Remember that your community’s SSID and password might be saved as plaintext in just a few information in your code now.
Let’s construct the applying. Simply name pio run ‐‐atmosphere core2foraws from the PlatformIO terminal after which pio run ‐‐atmosphere core2foraws ‐‐goal add ‐‐goal monitor to run it in your machine and monitor logs.
Now you should utilize the MQTT check shopper from the AWS IoT console to ship something to <<factor identify>>/blink matter. In my case, it’s 0123FAA32AD40D8501/blink. The message payload doesn’t matter for this instance. Simply ship one thing to start out blinking and ship something once more to cease it.
As you’ll be able to see, now we have performed lots simply to speak between AWS Cloud and the controller. It was a lot easier with Micropython and much more with UiFlow. Nevertheless, C is way more highly effective, and what’s most vital right here, we are able to lengthen it with libraries.
TensorFlow Lite for Microcontrollers
TensorFlow is an end-to-end open-source platform for machine studying. TensorFlow Lite is a library for deploying fashions on cell, microcontrollers, and different edge gadgets.
TensorFlow Lite for Microcontrollers is only a light-weight model of TensorFlow Lite designed to run machine studying fashions on microcontrollers and different gadgets with just a few kilobytes of reminiscence. The core runtime suits in 16 KB on an Arm Cortex M3 and may run many primary fashions. It doesn’t require working system assist, any commonplace C or C++ libraries, or dynamic reminiscence allocation.
TensorFlow Lite is just not designed to work on ESP32 processors, so the one one accessible for M5Stack is TensorFlow Lite for Microcontrollers. It has some limitations – it helps only a restricted subset of TensorFlow operations and gadgets, it requires guide reminiscence administration in Low-level C++ API, and it doesn’t assist on-device coaching. Due to this fact, to construct a „studying on the edge” answer, you want a extra highly effective IoT Edge machine, e.g. Raspberry Pi.
However you’ll be able to nonetheless run ML fashions on the M5Stack controller.
Now, let’s attempt to modify our Blinky-Hey-World so as to add the TensorFlow Lite for the Microcontrollers library.
TensorFlow Lite for Microcontrollers in FreeRTOS
The primary situation to resolve is the place to get the TensorFlow supply code from. In the principle TensorFlow repository, you’ll find info that it’s moved to a standalone one (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro) even when many of the documentation and examples nonetheless level there. The standalone repository used to comprise a makefile for ESP32, however it appears to be deleted when transferring examples to one more repository (https://github.com/tensorflow/tflite-micro/commit/66cfa623cbe1c1ae3fcc8a4903e9fed1a345548a). Right this moment, the most effective supply appears to be this repository: https://github.com/espressif/tflite-micro-esp-examples/tree/master/components.
We’ll want tfline-lib, however it doesn’t work with out esp-nn, so you need to copy each to the parts listing in your Blinky-Hey-World challenge.
Let’s modify our code, ranging from together with tensorflow headers originally of the principle.c file.
#embrace "tensorflow/lite/micro/all_ops_resolver.h" #embrace "tensorflow/lite/micro/micro_error_reporter.h" #embrace "tensorflow/lite/micro/micro_interpreter.h" #embrace "tensorflow/lite/schema/schema_generated.h"
Now we are able to attempt to use it. For instance, simply earlier than the void app_main() operate, let’s declare TF error reporter and use it within the operate.
tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main() Core2ForAWS_Init(); Core2ForAWS_Display_SetBrightness(80); ui_init(); TF_LITE_REPORT_ERROR(error_reporter, "Hey TensorFlow" "That is only a check message/n" ); initialise_wifi(); xTaskCreatePinnedToCore(&aws_iot_task, "aws_iot_task", 4096 * 2, NULL, 5, NULL, 1); xTaskCreatePinnedToCore(&blink_task, "blink_task", 4096 * 1, NULL, 2, &xBlink, 1); }
Clearly, it’s not an precise utilization of TensorFlow, however it proves the library is linked and can be utilized no matter you want.
In the principle listing, you should additionally add new libraries tflite-lib and esp-nn to the required parts in CMakeLists.txt
set(COMPONENT_REQUIRES "nvs_flash" "esp-aws-iot" "esp-cryptoauthlib" "core2forAWS" "tflite-lib" "esp-nn")
It appears to be like good, however it gained’t work but. Throughout compilation utilizing pio run –atmosphere core2foraws, you’ll discover out that your entire Blinky-Hey-World is made in pure C, and TensorFlow Lite for Microcontrollers library requires C++. The best technique to convert it’s as follows:
- Rename principal.c to principal.cc
- Change principal.c to principal.cc within the first line of principal/CMakeList.txt
- Create extern „C” part for your entire principal file code aside from tensorflow imports.
It ought to look by some means like that:
#embrace "tensorflow/lite/micro/all_ops_resolver.h" #embrace "tensorflow/lite/micro/micro_error_reporter.h" #embrace "tensorflow/lite/micro/micro_interpreter.h" #embrace "tensorflow/lite/schema/schema_generated.h" extern "C" ######unique principal.c content material goes right here###### tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main() #the principle operate code from the itemizing above
- In principal.cc, delete TaskHandle_t xBlink; declaration as a result of it’s already declared in one other file
- In platform.ini, in [env:core2foraws] part add build_flags = -fpermissive to vary permissive compilation errors into warnings
Now you’ll be able to construct the challenge once more. When operating it with the goal –monitor, you’ll see the „Hey TensorFlow” message in logs, which implies the TensorFlow library is included and dealing appropriately.
Now, you are able to do no matter you need with an out-of-the-box machine studying library and AWS integration.
Arduino
As you’ll be able to see, C is way more highly effective however requires way more work. Let’s attempt to join the identical blocks (tensorflow, AWS IoT, and M5Stack library) however utilizing a extra user-friendly atmosphere.
Arduino is an open-source digital prototyping platform enabling customers to create interactive digital objects. Let’s attempt to mix the official M5Stack Core 2 for AWS with the Arduino IDE guide (https://docs.m5stack.com/en/quick_start/core2_for_aws/arduino) with TensorFlow Lite for Microcontrollers (https://github.com/tanakamasayuki/Arduino_TensorFlowLite_ESP32).
Hey world!
Firstly, set up Arduino IDE from the official web page https://www.arduino.cc/en/software. I assume you have already got the CP210x driver put in, and the USB mode chosen in your machine.
Open the IDE, go to File -> Preferences, and add the boards’ administration URL: https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/arduino/package_m5stack_index.json.
Then open the Boards supervisor from the left menu and set up M5Stack-Core2. Now join the controller to the pc and select it from the highest drop-down menu.
To make use of the M5Stack-specific library within the code, you must open Sketch -> Embrace Libraries -> Library catalog and set up M2Core2.
Now you’ll be able to write the straightforward „Hey World!” code and run it with the inexperienced arrow within the IDE prime menu.
#embrace <M5Core2.h>
void setup()
M5.start();
M5.Liquid crystal display.print("Hey World");
void loop()
Typically, Arduino can not reset the controller through an RTS pin, so you must reboot it manually after writing a brand new code to it.
Up to now, so good.
TensorFlow and AWS integration
The TensorFlow-official, Arduino_TensorFlowLite_ESP32 library is just not designed for use with M5Stack. Let’s adapt it. Clone the library and duplicate the Hey World instance to a different listing. You may open it from Arduino IDE now. It’s a completely working instance of the utilization of the TensorFlow mannequin. Let’s adapt it to make use of the M5Core2 library. To hello_world.ino you must add #embrace <M5Core2.h> originally of the file and likewise M5.start(); originally of void setup() operate. You can too add M5.Axp.SetLed(true); after this line to activate the small inexperienced led and make sure the machine is operating.
Now, begin the applying. You may see TensorFlow output within the Serial Monitor tab. Simply change the baud fee to 115200 to make it human-readable.
Can we combine it with AWS IoT integration? Sure, we are able to.
We are going to use the PubSubClient library by Nick O’Leary, so open the library catalog in Arduino IDE and set up it, after which let’s hook up with AWS IoT and MQTT.
Utilizing Arduino IDE, create a brand new file secrets and techniques.h. We’d like just a few declarations there:
#outline AWS_IOT_PUBLISH_TOPIC " m5stack/pub" #outline AWS_IOT_SUBSCRIBE_TOPIC " m5stack/sub" #outline WIFI_SSID "ThisIsMyWiFiSSID" #outline WIFI_PASSWORD "Do not use really easy passwords!" int8_t TIME_ZONE = 2; #outline MQTT_HOST "xxxx.iot.eu-west-1.amazonaws.com" #outline THINGNAME "UiFlow_test" static const char* ca_cert = R"KEY( -----BEGIN CERTIFICATE----- … -----END CERTIFICATE----- )KEY"; static const char* client_cert = R"KEY( -----BEGIN CERTIFICATE----- … -----END CERTIFICATE----- )KEY"; static const char* privkey = R"KEY( -----BEGIN RSA PRIVATE KEY----- … -----END RSA PRIVATE KEY----- )KEY";
AWS_IOT_PUBLISH_TOPIC and AWS_IOT_SUBSCRIBE_TOPIC are our check subjects we’re going to make use of on this instance. WIFI_SSID and WIFI_PASSWORD are our WiFi credentials. TIME_ZONE is the time zone offset. MQTT_HOST is the general public AWS IoT endpoint (the identical as within the first UiFlow instance). THINGNAME is the identify of Factor in AWS (I’ve used the identical as within the UiFlow instance). client_cert and privkey, you must copy from the secrets and techniques generated when creating Factor for the UiFlow instance. ca_cert is the general public key of AWS certificates authority, so you’ll be able to get hold of it from the Factor creation wizard (certificates step) or from https://good.sca1a.amazontrust.com/).
Now it’s time to adapt the principle hello_world.ino file.
We must always add new imports (together with our secret.h file).
#embrace <WiFiClientSecure.h> #embrace <PubSubClient.h> #embrace "secrets and techniques.h" #embrace <time.h>
Then we want just a few new fields.
WiFiClientSecure web; PubSubClient shopper(web); time_t now; time_t nowish = 1510592825;
The sector nowish is just a few timestamp previously.
Within the setup() operate, we have to open a WiFi reference to our native community and the Web, arrange the time to verify certificates, set up the certificates, arrange the MQTT shopper, and open the AWS IoT connection.
delay(3000);
WiFi.mode(WIFI_STA);
WiFi.start(WIFI_SSID, WIFI_PASSWORD);
WiFi.waitForConnectResult();
whereas (WiFi.standing() != WL_CONNECTED)
Serial.print(".");
delay(1000);
M5.Liquid crystal display.println(String("Making an attempt to hook up with SSID: ") + String(WIFI_SSID));
M5.Liquid crystal display.println(WiFi.localIP());
M5.Liquid crystal display.print("Setting time utilizing SNTP");
configTime(TIME_ZONE * 3600, 0 * 3600, "pool.ntp.org", "time.nist.gov");
now = time(nullptr);
whereas (now < nowish)
delay(500);
Serial.print(".");
now = time(nullptr);
M5.Liquid crystal display.println("performed!");
struct tm timeinfo;
gmtime_r(&now, &timeinfo);
M5.Liquid crystal display.print("Present time: ");
M5.Liquid crystal display.print(asctime(&timeinfo));
web.setCACert(ca_cert);
web.setCertificate(client_cert);
web.setPrivateKey(privkey);
shopper.setServer(MQTT_HOST, 8883);
shopper.setCallback(messageReceived);
M5.Liquid crystal display.println("Connecting to AWS IOT");
whereas (!shopper.join(THINGNAME))
Serial.print(".");
delay(1000);
if (!shopper.related())
M5.Liquid crystal display.println("AWS IoT Timeout!");
return;
shopper.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);
M5.Liquid crystal display.println("AWS IoT Linked!");
That is a whole code wanted to arrange the applying, however I suggest splitting it into a number of smaller and extra readable capabilities. As you’ll be able to see, I exploit Serial output for debugging.
To obtain messages, we want a brand new operate (the identify matches the declaration in shopper.setCallback(messageReceived);)
void messageReceived(char *matter, byte *payload, unsigned int size)
M5.Liquid crystal display.print("Acquired [");
M5.Lcd.print(topic);
M5.Lcd.print("]: ");
for (int i = 0; i < size; i++)
M5.Liquid crystal display.print((char)payload[i]);
M5.Liquid crystal display.println();
The very last thing to do is to loop the shopper with your entire software. To try this, simply add a one-liner to the loop() operate:
shopper.loop();
You want one other one-liner to ship one thing to AWS, however I’ve added two extra to make it seen on the controller’s show.
M5.Liquid crystal display.println("Sending message");
shopper.publish(AWS_IOT_PUBLISH_TOPIC, ""message": "Hey from M5Stack"");
M5.Liquid crystal display.println("Despatched");
The communication works each methods. You may subscribe to m5stack/pub utilizing the MQTT Take a look at shopper within the AWS console to learn messages from the controller, and you’ll publish to m5stack/sub to ship messages to the controller.
As you’ll be able to see, utilizing Arduino is simpler than utilizing FreeRTOS, however sadly, it’s somewhat bit babyish. Now we’ll attempt to keep away from all IDE’s and use pure console solely.
Espressif IoT Improvement Framework
Mainly, there are 3 ways to burn software program to the controller from a Linux console – Arduino, esptool.py, and ESP-IDF. While you create a brand new challenge utilizing PlatformIO, you’ll be able to decide Arduino or ESP-IDF. Now, let’s attempt to take away the IDE from the equation and use a pure bash.
Initially, you must set up just a few conditions after which obtain and set up the library.
sudo apt set up git wget flex bison gperf python3 python3-venv cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0 mkdir -p ~/esp cd ~/esp git clone --recursive <a href="https://github.com/espressif/esp-idf.git">https://github.com/espressif/esp-idf.git</a> cd ~/esp/esp-idf ./set up.sh esp32 ./export.sh
Please observe you must run set up and export (final two instructions) everytime you open a brand new WSL console. With the library, you even have some examples downloaded. Run one in all them to verify, does every little thing work.
cd examples/get-started/hello_world/ idf.py set-target esp32 set ESPPORT=/dev/ttyUSB0 idf.py construct flash monitor
You need to see the output like this one.
Hey world! That is an esp32 chip with 2 CPU core(s), WiFi/BT/BLE, silicon revision 300, 2MB exterior flash Minimal free heap dimension: 295868 bytes Restarting in 10 seconds... Restarting in 9 seconds... Restarting in 8 seconds... Restarting in 7 seconds... Restarting in 6 seconds... Restarting in 5 seconds... Restarting in 4 seconds... Restarting in 3 seconds... Restarting in 2 seconds... Restarting in 1 seconds... Restarting in 0 seconds... Restarting now.
To cease the serial port monitor, press CRTL + ]. Remember that the applying remains to be operating on the controller. You must energy off the machine by the {hardware} button on the facet to cease it.
If you wish to use TensorFlow Lite for Microcontrollers with ESP-IDF, you must create a brand new challenge and add a correct library. You need to use the command idf.py create-project <<project_name>> to create a challenge. My challenge identify is hello_tf. The script creates a pure C challenge; we have to rename hello_tf.c file to hello_tf.cc. Then, we are able to copy tflite-micro and esp-nn libraries from FreeRTOS instance and place them within the parts listing. The principal/CMakeList.txt content material must be like that.
set(COMPONENT_SRCS "hello_tf.cc") set(COMPONENT_REQUIRES "tflite-lib" "esp-nn") register_component()
As you’ll be able to see, the default parts sources definition is modified, and new libraries are added.
Now, let’s see the principle hello_tf.cc file content material.
#embrace "tensorflow/lite/micro/all_ops_resolver.h" #embrace "tensorflow/lite/micro/micro_error_reporter.h" #embrace "tensorflow/lite/micro/micro_interpreter.h" #embrace "tensorflow/lite/schema/schema_generated.h" extern "C" tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main(void) TF_LITE_REPORT_ERROR(error_reporter, "Hey from TensorFlown");
As you’ll be able to see, we had to make use of extern „C” block once more as a result of, by default, ESP-IDF runs the void app_main() operate from C, not C++ context.
To run the applying run idf.py construct flash monitor.
In the identical manner, you’ll be able to add different libraries wanted, however with out PlatformIO, dependency administration is difficult, particularly for the core2forAWS library with a number of dependencies. Alternatively, you should utilize https://github.com/m5stack/M5Stack-IDF as a library with M5Stack dependencies to manage the I/O gadgets of the controller.
Abstract
As I wrote originally of this text, with C++, you are able to do way more; nevertheless, you’re pressured to handle your entire machine by your self. Sure, you should utilize AWS integration, M5Stack I/O interfaces, and TensorFlow (TensorFlow Lite for Microcontrollers model solely) library collectively, however it requires lots of code. Can we do something to hitch the benefits of utilizing Micropython and C collectively? Let’s attempt to do it within the final chapter.
