What keeps me awake at night? Wondering why our microcontrollers (like STM32 Blue Pill above) have become so powerful… that we can’t write powerful programs to exploit them

Hosting Embedded Rust apps on Apache Mynewt with STM32 Blue Pill

Today’s microcontrollers (like the STM32 Blue Pill) pack so many features in a tiny package… yet few embedded programmers are capable of exploiting the full potential of modern microcontrollers. Many of us (my IoT students included) seem to be stuck in the 1980s — painstakingly writing C programs for small computers.

It’s time to drop our legacy programming practices and adopt smarter, safer ways to exploit these microcontrollers… starting with Apache Mynewt and Rust.

Mynewt is a modern realtime operating system that runs on many microcontroller platforms, even on devices with little RAM and ROM like Blue Pill. It has an excellent Sensor Framework for creating IoT devices.

Sensor Network with Blue Pill, ESP8266, nRF24L01 and Mynewt

Stretching Mynewt on Blue Pill to the limit, I have created sophisticated sensor networks with CoAP encoding (JSON and CBOR), transmitting and receiving simultaneously on both ESP8266 and nRF24L01 modules.

Sadly, Mynewt only supports applications developed in C, consistent with its frugal origins. But C programming is a rare skill today (and C frustrates my IoT students too). What about other languages? MicroPython and JavaScript seem too heavy for a lightweight Mynewt device… perhaps Rust?

Mynewt and Rust: The Perfect Match

In my previous experiments with Embedded Rust (including this one), I know that Rust flies light and fast just like C. Rust has great support in coding tools like Visual Studio Code, which makes it easier to code for newbies.

Rust advocates “Safe Coding”. Rust disallows bad code that may cause our device to crash. Rust could be the perfect match for Mynewt!

In this article we’ll mix Mynewt with Rust and learn…

1️⃣ Why is Rust better than C for embedded programming

2️⃣ How to call C functions from Rust and vice versa

3️⃣ What’s Unsafe Coding and how to make coding safer

4️⃣ How Rust compiles programs

5️⃣ How to inject Rust code into the Mynewt build

6️⃣ How to install and configure the Rust and Mynewt code

7️⃣ And finally we’ll see an actual Rust application running on Mynewt with Blue Pill

Here’s the Rust application code that we’ll be running: A sensor application that polls Blue Pill’s internal temperature sensor every 10 seconds and displays the result…

Walkthrough of the Rust application code

The complete Rust and Mynewt code may be found here…

💎 Sections marked with a diamond are meant for advanced developers. If you’re new to embedded programming, you may skip these sections


Embedded Programming in Rust vs C

Why is Rust better than C for Embedded Programming? Here’s what I think…

Calling the Mynewt Sensor Listener API: Rust Code (left) vs C Code (right)

Rust is Strict and Stubborn

Rust syntax is highly similar to C. If you’re looking at Rust code for the first time, you may think that Rust is an untyped language (like JavaScript) that doesn’t care about the precise types of each variable…

Why would Rust allow let rc = ... without forcing us to declare rc as an int like in C? Does it mean that rc can change its type from int to char*?

Nope, that’s because Rust has Type Inference — the Rust compiler analyses our source code and deduces that rc must be an int (which is named i32 in Rust). Why? Because sensor_set_poll_rate_ms() returns an i32 (32-bit integer). So Rust enforces variable types as strongly as C, just smarter.

Rust is actually stricter than C — All Rust variables are assumed to be constant, unchanging, immutable (“stubborn”) unless we declare the variable as mut (mutable). Check out the two examples of mut in the Rust code above.

C is the opposite (“loose and relaxed”). All C variables are mutable unless you declare the variable as const.

Check “The Rust Programming Language” for details.

Rust is Lean and Lightweight

According to this actual build log of our combined Rust and Mynewt Sensor application, the entire compiled executable fits into 56 KB of ROM. The compiled Rust application and Rust libraries occupy under 4 KB of ROM. (Most of the ROM space was taken up by the ESP8266 and nRF24L01 drivers written in C.)

The lightweight nature of Rust makes it an excellent replacement for embedded C programming on devices with constrained resources. Like Blue Pill!

Rust Plays Nice with C

Rust was designed for low-level systems programming, like for coding an operating system or web browser (Mozilla Firefox). Just like C, Rust is used for bare-metal programming, so it makes sense for Rust and C functions to be interoperable — We may call C functions from Rust, and Rust functions from C. Even pass the same structs from Rust to C and back!

We’ll cover the Rust and C interoperability in a while.

Rust is Smarter than C

Visual Studio Code: Rust vs C coding

Check out this comparison of Rust vs C coding in Visual Studio Code. Why is Rust better than C for coding embedded applications?

Remember that Rust supports Type Inference. The Rust compiler keeps deducing all the types of the variables as we type. So Rust provides more helpful code completion and error highlighting than C. Rust generates documentation for our functions too.

This is great for preventing programming errors in Rust, especially for beginners. Often in embedded programs we make undocumented assumptions and take shortcuts to make the programs run on tiny devices. With Type Inference, Rust can warn new embedded developers about these traps.

Rust is Safer than C

Helping us to write safer, crash-proof programs is a key feature of Rust, that’s why we see unsafe keywords in the Rust code above. We’ll cover unsafe in a while…

But…

Rust is evolving rapidly, especially for embedded platforms like Blue Pill. A year ago I wrote two tutorials on Rust embedded programming…

…And they have already become obsolete.

Be very patient if things don’t work quite right in Rust. I may not have the answer for some deep issues, but I’m confident that Rust maintainers will have the answer for us someday.


Mixing Mynewt and Rust

Built in C, Mynewt is a modern, well-designed operating system for microcontrollers. Rust is a modern, smarter, safer programming language for embedded systems. How shall we enjoy the best of both worlds — Mynewt and Rust? I propose to…

Layering Rust above Mynewt

1️⃣ Leave the Mynewt operating system untouched, in C. Same goes for Mynewt drivers and libraries, since they were coded by experienced C programmers.

2️⃣ Expose the popular Mynewt APIs through safe Rust interop libraries

3️⃣ Allow Rust applications to be hosted on Mynewt with these exposed APIs

So new embedded developers (and my IoT students) can start coding Mynewt applications in Rust, the smarter, safer way. No more crashing pointers. Yay!

To host a Rust application on Mynewt, we need to be sure that C and Rust can really call each other seamlessly…

Rust and C interoperability

1️⃣ Rust must allow importing of Mynewt’s C API. So Rust functions must be able to call C functions:
Rust ⟶ C

2️⃣ Rust must allow callbacks from Mynewt’s C API. So Rust must allow its functions to be called from C functions:
Rust ⟵ C

3️⃣ Rust must allow structs to be passed from Rust to C and back:
Rust ⟵📦⟶ C

Fortunately, Rust fulfils all the criteria above. For example, to import the following Mynewt Sensor API functions into our Rust application…

int sensor_set_poll_rate_ms(const char * devname, uint32_t poll_rate)

struct sensor* sensor_mgr_find_next_bydevname(const char * devname, struct sensor* prev_cursor)

int sensor_register_listener(struct sensor * sensor, struct sensor_listener * listener)

…We just compile this Rust code with our Rust application, then we can call the Mynewt functions as though they were Rust functions!

💎 How did we derive the Rust import declarations above from the Mynewt C declarations? This is explained in the section “Calling C from Rust and back” below.


Is This Code Unsafe?

Question: What’s wrong with this C function that calls the Mynewt API sensor_register_listener()?

Answer: listener is a variable created on the stack frame for start_sensor_listener(). When passed to sensor_register_listener(), the listener is appended directly to the global list of sensor listeners, without copying.

When start_sensor_listener() returns, the stack frame is reused by other functions and the listener may contain garbage. When the sensor is polled 10 seconds later, Mynewt fetches the garbled listener to call the listener function. Which results in a catastrophic device failure. One IoT Device Down!

Bewildering variable choices for the C programmer

This is one key reason why my IoT students find C programming so hard… Every time we define a variable in C, we need to pick wisely whether it should be a 1️⃣ Static Variable, 2️⃣ Stack Variable or 3️⃣ Heap Variable.

Choosing the wrong one will have dire consequences… that’s why Rust calls this “Unsafe” code.


Unsafe Coding in Rust

Rust doesn’t prevent us from writing unsafe code though — it requires us to flag the code as unsafe using the unsafe keyword. Otherwise the Rust compiler politely terminates and refuses to generate any unsafe executables, for our safety.

In the Rust code above we are required to ring the alarm bells and flag as unsafe the call to sensor_register_listener() because…

1️⃣ We are passing the address of the listener to a function sensor_register_listener(). Which may cause problems if the listener was allocated incorrectly (like on the stack). &mut listener is equivalent to &listener in C, just that it also declares to Rust that the listener contents may change (mutable).

2️⃣ sensor_register_listener() is a C function defined by Mynewt, that we have imported into Rust. All imported functions must be flagged as unsafe because, as we know, C functions are capable of doing really weird things.

But we don’t really want unsafe alarm bells to ring in our heads every time we call the Mynewt API. Is there a safer way to silence the alarms?


Safer Coding in Rust

The safer way to call unsafe Mynewt APIs in Rust is to create wrapper functions to check the parameters and the return values.

Check this out… we can now register a sensor listener without flagging as unsafe! And the listener was allocated on the stack!

This is the perfect kind of API that my IoT students should be calling, without fear of crashing their devices. What is the magic that makes this happen?

Here’s how we created register_listener() as the safe version of the sensor_register_listener() API in Mynewt… register_listener() makes a local static copy of the listener, and passes the local copy to the unsafe sensor_register_listener(). So even if the listener was created on the stack, we are actually passing a static copy to the Mynewt API. Which won’t crash the device.

We have created a simple Rust wrapper that reuses the same types used by the Mynewt API. (That’s why register_listener() returns an integer as the result code.) But Rust APIs generally use a different convention to return results — the Result type. We’ll cover this in the next article.

💎 This implementation looks simplistic… what if we need more sensor listeners? But in reality our device RAM is highly constrained and we should plan in advance how many sensor listeners we really need (and set it as a #define). Then we create an array of listeners that’s allocated by register_listener().

What about allocating listeners on the heap? That could be a good solution but I’m not keen on using the heap on constrained devices (because we never know when we’ll run out of heap space). The alloc heap allocator needs to be implemented for our Rust environment.


The Typical Rust Build

Files involved in the Rust build

To understand how we merged the Rust build with Mynewt, let’s look at the typical Rust build process.

The following files are present in a standard Rust build. You can find them in your stm32bluepill-mynewt-sensor folder too…

1️⃣ Cargo.toml: Just like building a typical Rust application, in Mynewt we run the command cargo build to build our embedded Rust application.

cargo performs the build according to the settings file Cargo.toml. This file must be present at the root of the workspace folder.

We’ll check the contents of Cargo.toml in a while.

2️⃣ .cargo/config: Our Rust build is for an embedded platform: STM32 Blue Pill. The target platform is specified in this file.

We’ll check the contents of .cargo/config in a while.

3️⃣ src: This folder contains the Rust source code. lib.rs is the main Rust module that pulls in the other Rust files for the build (via the mod declaration).

4️⃣ target/thumbv7m-none-eabi/debug: The cargo build command compiles the src files and generates in this debug folder the Rust executable or library (according to Cargo.toml).

Compiled Rust code in libmylib.rlib

In our build, cargo generates a library archive file libmylib.rlib. If we peek at the contents of the archive file …
arm-none-eabi-ar t target/thumbv7m-none-eabi/debug/libmylib.rlib
…we’ll see that it contains *.o object files, the compiled Rust code.

But the files in this debug folder exclude the external Rust libraries called by our application (like cty and cstr_core).

To include the external Rust libraries for the Mynewt build, we need to look one level deeper: debug/deps

This folder contains the compiled *.rlib archive files for our Rust application (libmylib-*.rlib) as well as external Rust libraries.

As we’ll see later, all the *.rlib files in debug/deps will be injected into the Mynewt build.

Typical Rust Build

5️⃣ Rust Core Library libcore: There’s one important Rust library that’s missing — the Rust Core Library libcore. This library contains the fundamental code needed to implement the core Rust functions: data structures, math, panicking, text formatting, …

libcore is automatically included during the cargo build if we use cargo to generate the executable. But here we are using Mynewt to generate the executable, so we need to add libcore ourselves.

Where is libcore located? It’s actually located together with the Rust compiler (not with the cargo installation). Our build script runs this command to get the location…

On my Mac, libcore is located at /Users/Luppy/.rustup/toolchains/nightly-2019–05–22-x86_64-apple-darwin/lib/rustlib/thumbv7m-none-eabi/lib/libcore-e6b0ad9835323d10.rlib

Note that libcore is specific to the target platform (Blue Pill is thumbv7m-none-eabi). We’ll link the libcore library in the Mynewt build.

💎 The disassembled code for our Rust application is available here. The disassembled code for libcore is available here.


Rust Build Settings

Let’s look at the simple Rust build settings for our project (which was inspired by this sample)…

Here we indicate to cargo that we’re generating a library [lib] (instead of an executable [bin]).

To keep the ROM size small, we have included only a few small libraries.

This Cargo.toml produces the compiled Rust library libmylib.rlib that we’ll link with Mynewt.

Here we select the target platform for the Rust compiler.

STM32 Blue Pill runs on an Arm Cortex-M3 processor (which is based on the ARMv7-M architecture), so we have chosen the target thumbv7m-none-eabi

cargo build will then generate compiled code in libmylib.rlib that will run on our Blue Pill.

As we have seen, we haven’t changed anything in the Rust cargo build process. We used the standard cargo build command to generate a Rust library libmylib.rlib for our Rust application.

The integration of Rust with Mynewt is actually done by our custom build script that injects the compiled Rust files into the Mynewt build. Before going into the integration details, let’s learn about the Mynewt build…


Typical Mynewt Build

The Typical Mynewt Build

The normal Mynewt build command newt build compiles some C and assembly files from these folders…

Libraries generated by newt build

1️⃣ apps/my_sensor_app: Our custom C application for Mynewt

2️⃣ libs: Custom drivers (e.g. ESP8266) and libraries (e.g. Sensor Network) for Mynewt

3️⃣ repos: Mynewt OS source code

newt build compiles each source module (e.g. my_sensor_app, esp8266) into a separate *.a library (e.g. apps_my_sensor_app.a, libs_esp8266.a), located at bin/targets/bluepill_my_sensor

newt build links the *.a libraries together to create the Blue Pill executable image /bin/targets/bluepill_my_sensor/app/apps/my_sensor_app/my_sensor_app.elf

This is the image that get flashed into the Blue Pill ROM.


Merging the Rust build with Mynewt

To host a Rust application on Mynewt, we just need to inject three pieces of compiled code into the Mynewt build…

Overwriting libs_rust_app.a by the compiled Rust application and libraries

1️⃣ Compiled Rust application: libmylib.rlib

2️⃣ Compiled external Rust libraries: target/thumbv7m-none-eabi/debug/deps/*.rlib. This folder also contains libmylib.rlib

3️⃣ Rust Core Library libcore: ~/.rustup/toolchains/nightly-*/lib/rustlib/thumbv7m-none-eabi/lib/libcore-*.rlib

To do that, we have a super build script scripts/build-app.sh that runs the Rust cargo build command and injects the above files at these Mynewt build locations…

1️⃣ Rust application and external Rust libraries: Copy and overwrite bin/targets/bluepill_my_sensor/app/libs/rust_app/libs_rust_app.a

2️⃣ Rust libcore: Copy and overwrite bin/targets/bluepill_my_sensor/app/libs/rust_libcore/libs_rust_libcore.a

rust_app and rust_libcore are empty stub libraries

What are rust_app and rust_libcore? These are custom stub libraries that we added to Mynewt at libs/rust_app and libs/rust_libcore. No meaningful code inside, just stubs.

But because we instructed Mynewt to include them as part of the Mynewt build, Mynewt compiles them and generates the tiny libraries libs_rust_app.a and libs_rust_libcore.a

Which gives us the perfect opportunity to substitute libs_rust_app.a and libs_rust_libcore.a with our compiled Rust code.

Our super build script overwrites libs_rust_app.a with the compiled Rust application and libraries, and overwrites libs_rust_libcore.a with the Rust libcore library. So when newt build links all the *.a libraries, the Rust code gets injected into my_sensor_app.elf, which will be flashed into the Blue Pill ROM. Sneaky!

But how do we replace libs_rust_app.a by multiple *.rlib libraries: libmylib.rlib + libcty.rlib + libcstr_core.rlib + ...?

Our super build script extracts the *.o object files from every *.rlib file and archives them into a single rustlib.a library.

That’s how we create the Mynewt ROM for Blue Pill that includes the Rust application and libraries!

This video explains the combined Rust and Mynewt build…

Building the combined Rust and Mynewt code

💎 The complete Rust and Mynewt build log is available here. The disassembled code for our Mynewt ROM image is available here.


Installation and Configuration

If you have the STM32 Blue Pill or Super Blue Pill and you wish to install and configure the Rust and Mynewt demo code, follow the instructions here. The steps are quite lengthy and you may run into hiccups, so I don’t recommend this if you’re new to embedded development.

Make sure that you’re using the rust branch of the code, not the master branch (that was used in previous articles)…

https://github.com/lupyuen/stm32bluepill-mynewt-sensor/tree/rust

Verify that the README.md says…

Note: This is the rust branch that contains a Rust application hosted on Mynewt

Conclusion

Running our Rust application on Mynewt

Here’s a video of our Rust application, hosted on Mynewt, running on an actual Blue Pill. It works as expected: Reading the Blue Pill’s internal temperature sensor every 10 seconds and displaying the results— Yay!

With some simple build integration, we have proven that it’s indeed possible to host Rust applications on Mynewt (and probably other embedded operating systems too). And it requires very little RAM and ROM to support embedded Rust.

Perhaps the bigger challenge is to design a Rust API for Mynewt that’s safe and easy to use.

In the meantime, what we have created is probably sufficient for new embedded developers (and my students) to create IoT devices in Rust and Mynewt without fear of memory corruption. Saving the world from unsafe embedded C programming… one microstep at a time!

Check out the latest article on Visual Embedded Rust…


Further Reading for Embedded Rust

1️⃣ The Embedded Rust Book was highly informative for embedded Rust programming on Arm Cortex M3

2️⃣ Writing an OS in Rusthas plenty of tips for coding operating systems in Rust

3️⃣ freertos.rs (Rust wrapper for FreeRTOS) was the original inspiration for creating a safe Rust wrapper around an embedded operating system


💎 The following Advanced Topic sections are for advanced developers. If you’re new to embedded programming, you may stop here.


💎 Advanced Topic: Calling C from Rust and back

In this section we go into detail to see how C functions and structs are imported into Rust, and how to export Rust functions to C.

Import C Functions into Rust: Rust ⟶ C

For our demo Rust application, we shall be calling the Mynewt Sensor API to configure the onboard temperature sensor to be polled every 10 seconds. After polling the sensor, Mynewt should call a Listener Function (defined in Rust) to display the result.

To import the Mynewt Sensor API functions into our Rust application, we should include this code…

How did we derive the Rust import declarations above? Let’s look at the C declarations for the three Mynewt functions (right column) and see how they map to the Rust import declaration (left column)…

  • #[link(name = “hw_sensor”)] declares to Rust that the C functions to be imported will be located in the compiled C library archive hw_sensor.a (which will be generated by the Mynewt build)
  • i32 in Rust is equivalent to int in C (signed 32-bit integer)
  • u32 in Rust is equivalent to uint32_t in C (unsigned 32-bit integer)
  • u8 in Rust is equivalent to uint8_t in C (unsigned byte)
  • *const u8 in Rust is equivalent to const uint8_t * in C. We use this to pass devname because const uint8_t * has the same size as const char *
  • SensorListener is a struct we imported from Mynewt to pass the listener info from Rust to C (to be explained in a while)
  • *mut SensorListener in Rust is equivalent to struct SensorListener * in C
  • Why *mut SensorListener and not *const SensorListener? Because SensorListener will be stored and updated by Mynewt.
  • We define SensorPtr as follows…
  • SensorPtr represents a pointer to a Mynewt sensor struct. Since we are just passing the pointer through from Rust to Mynewt, without touching its contents, we declare SensorPtr as a void * type, which is *const CVoid in Rust.
  • CVoid is our Rust equivalent of void in C

Import C Structs into Rust: Rust ⟵📦⟶ C

The Rust code above should be included into our Rust application so that the SensorListener struct may be passed through sensor_register_listener(). To understand how the Rust import declaration was derived, let’s compare the Rust import declaration with the Mynewt declaration in C…

  • #[repr(C)] tells Rust that this struct is used by both Rust and C functions. The struct memory layout shall be fixed according to the C convention.
  • #[repr(C, packed)] is used for C structs that are packed
  • SensorType, SensorArg, SensorDataFunc are defined in the Sensor Types Definition code above
  • SensorDataFunc is a pointer to a callback function in Rust that Mynewt will call from a C function. This is explained below.
  • sl_next is defined as u32 (32-bit unsigned integer) because we should initialise the field to 0 in our Rust code. sl_next is updated by Mynewt to store the pointer to the next SensorListener in the linked list

Export Rust Functions to C (Callback): Rust ⟵ C

We have seen how C functions and structs may be imported into Rust. Now let’s see how Rust functions may be called from C.

Recall that sensor_register_listener() registers a Listener Function (defined in Rust) that will be called by Mynewt when the temperature sensor has been polled. sensor_register_listener() accepts a SensorListener struct (named LISTENER here) as a parameter. The SensorListener struct includes sl_func, the pointer to the Listener Function.

Here we pass the Rust function read_temperature() as the Listener Function.

read_temperature() is declared as extern so that it may be called from C as a callback function (instead of calling by function name). Note that read_temperature() has the same function signature as sl_func (which has type SensorDataFunc)…

Export Rust Functions to C (Call By Name): Rust ⟵ C

Previously we have seen how C may call a Rust function via callback (i.e. the Rust function is passed to C as a pointer). What if our C function wishes to call a Rust function by its function name?

Normally the Rust compiler converts Rust function names into a “mangled” format that includes the function signature. So the read_temperature() function would be compiled and renamed as: _ZN5mylib13listen_sensor16read_temperature17h318673dbfba97d4bE()

But if we are calling read_temperature() from C, this becomes a problem because the function name has changed. To prevent the mangling of the Rust function name, use the #[no_mangle] directive like this:

So in the above code, the function main() will not be renamed by the Rust compiler. The pub extern “C” directive tells the Rust compiler to export the Rust function main() to C functions.

From a C function, we may now call the function by its Rust name, main(). In fact that’s how Mynewt starts our Rust application.

This interoperability of Rust and C is known as the Foreign Function Interface. Read the details here.


💎 Advanced Topic: Enhancements

Although we have done a lot in this article, we haven’t provided a comprehensive and maintainable environment in Mynewt for hosting all kinds of Embedded Rust applications.

Here are my suggestions for improving the integration of Rust with Mynewt. If you’re keen to work on the integration of Rust with Mynewt, lemme know!

  1. Rust folders should follow the Mynewt folder structure:
    Move Rust source files from src to apps/my_sensor_app/src
    Generate compiled Rust object files in bin instead of target
  2. Auto-generate the Rust code for importing Mynewt C functions and structs. bindgen is a tool that we could use to generate the Rust import declarations by feeding in the Mynewt header files. Already done here.
  3. Create safe Rust wrappers for the complete Mynewt API. We may need lots of thinking to create a safe Rust API for Mynewt that’s easy to use and error-proof. Already done here.
  4. Rust wrappers for Mynewt should return Result type instead of integers (thanks to Marcel Hellwig). Already done here.
  5. CoAP (JSON and CBOR) API for Rust, similar to the network-agnostic Sensor Network API. Already done here.
  6. Allow Mynewt drivers and libraries to be created with Rust
  7. Support other microcontroller platforms besides Blue Pill
  8. Support text formatting and heap allocator in Rust. While keeping RAM and ROM usage low.
  9. Keep monitoring for RAM and ROM bloat as we implement enhancements. For example, the ROM size dropped from 73 KB to 55 KB after removing the unwrap() / panic() error checking. Check the memory map before and after. (Learn more about memory maps)