📝 7 Apr 2024
My mentee Rushabh Gala and I are anxiously awaiting the results of the Google Summer of Code (GSoC) Project Selection. While waiting, we explain the current steps for running barebones Rust Apps on Apache NuttX RTOS (and the challenges we faced)…
How we compile Rust Apps for NuttX
Running NuttX and Rust Apps on QEMU RISC-V Emulator
Console Input and Output for Rust on NuttX
Software vs Hardware Floating-Point and why it’s a problem
Linking Issues with the Rust Panic Handler
Standard vs Embedded Rust and why it matters
Why we’re doing all this for Google Summer of Code
Thanks to PINE64, the sponsor of Ox64 BL808 RISC-V SBCs for our GSoC Project Testing!
Below is the “Hello Rust” Demo App that’s bundled with Apache NuttX RTOS: hello_rust_main.rs
// main() function not needed
#![no_main]
// Use Rust Core Library (instead of Rust Standard Library)
#![no_std]
// Import printf() from C into Rust
extern "C" {
pub fn printf(
format: *const u8, // Equivalent to `const char *`
... // Optional Arguments
) -> i32; // Returns `int`
} // TODO: Standardise `i32` as `c_int`(We’ll explain [no_std] in a while)
The code above imports the printf() function from C into Rust.
This is how we call it in Rust: hello_rust_main.rs
// Main Function exported by Rust to C.
// Don't mangle the Function Name.
#[no_mangle]
pub extern "C" fn hello_rust_main(
_argc: i32, // Equivalent to `int argc`
_argv: *const *const u8 // Equivalent to `char **argv`
) -> i32 { // Returns `int`
// Calling a C Function might have Unsafe consequences
unsafe {
printf( // Call printf() with...
b"Hello, Rust!!\n\0" // Byte String terminated by null
as *const u8 // Cast as `const char *`
);
}
// Exit with status 0
0
}Rust expects us to provide a Panic Handler. We write a simple one: hello_rust_main.rs
// Import the Panic Info for our Panic Handler
use core::panic::PanicInfo;
// Handle a Rust Panic. Needed for [no_std]
#[panic_handler]
fn panic(
_panic: &PanicInfo<'_> // Receives the Panic Info and Stack Trace
) -> ! { // Never returns
// TODO: Print the Panic Info and Stack Trace
// For now, we loop forever
loop {}
}Which doesn’t do much right now. We’ll create a proper Panic Handler during GSoC.
How to compile our Rust App?
Follow these steps to build Apache NuttX RTOS for QEMU RISC-V (32-bit), bundled with our “Hello Rust” Demo App…
Install the Build Prerequisites, skip the RISC-V Toolchain…
Download the RISC-V Toolchain for riscv64-unknown-elf…
Download and configure NuttX…
mkdir nuttx
cd nuttx
git clone https://github.com/apache/nuttx nuttx
git clone https://github.com/apache/nuttx-apps apps
cd nuttx
tools/configure.sh rv-virt:nsh
make menuconfig
In menuconfig, browse to “Device Drivers > System Logging”
Disable this option…
Prepend Timestamp to Syslog Message
Browse to “Build Setup > Debug Options”
Select the following options…
Enable Debug Features
Enable Error Output
Enable Warnings Output
Enable Informational Debug Output
Enable Debug Assertions
Enable Debug Assertions Show Expression
Scheduler Debug Features
Scheduler Error Output
Scheduler Warnings Output
Scheduler Informational Output
Browse to “Application Configuration > Examples”
Select “Hello Rust Example”
Select it Twice so that “<M>” changes to “<*>”
Save and exit menuconfig.
Build the NuttX Project and dump the RISC-V Disassembly to nuttx.S (for easier troubleshooting)…
## Add the Rust Target for RISC-V 32-bit (Soft-Float)
rustup target add riscv32i-unknown-none-elf
## Build the NuttX Project
make
## Dump the NuttX Disassembly to `nuttx.S`
riscv64-unknown-elf-objdump \
-t -S --demangle --line-numbers --wide \
nuttx \
>nuttx.S \
2>&1
This produces the NuttX ELF Image nuttx that we may boot on QEMU RISC-V Emulator. (Next Section)
If the GCC Linker fails with “Can’t link soft-float modules with double-float modules”
$ make
LD: nuttx
riscv64-unknown-elf-ld: nuttx/nuttx/staging/libapps.a
(hello_rust_main.rs...nuttx.apps.examples.hello_rust_1.o):
can't link soft-float modules with double-float modules
riscv64-unknown-elf-ld: failed to merge target specific data of file
nuttx/staging/libapps.a
(hello_rust_main.rs...nuttx.apps.examples.hello_rust_1.o)
Then we patch the ELF Header like this, and it should link correctly…
xxd -c 1 ../apps/examples/hello_rust/*hello_rust_1.o \
| sed 's/00000024: 00/00000024: 04/' \
| xxd -r -c 1 - /tmp/hello_rust_1.o
cp /tmp/hello_rust_1.o ../apps/examples/hello_rust/*hello_rust_1.o
make
(We’ll come back to this)
We’re ready to boot NuttX on QEMU Emulator and run our Rust App!
Download and install QEMU Emulator…
## For macOS:
brew install qemu
## For Debian and Ubuntu:
sudo apt install qemu-system-riscv32
Start the QEMU RISC-V Emulator (32-bit) with the NuttX ELF Image nuttx from the previous section…
qemu-system-riscv32 \
-semihosting \
-M virt,aclint=on \
-cpu rv32 \
-bios none \
-kernel nuttx \
-nographic
NuttX is now running in the QEMU Emulator! (Pic above)
NuttShell (NSH) NuttX-12.4.0-RC0
nsh>
Enter “hello_rust” to run our Rust Demo App (which will print something)
nsh> hello_rust
Hello, Rust!!
Enter “help” to see the available commands…
nsh> help
help usage: help [-v] [<cmd>]
. cp exit mkdir rmdir umount
[ cmp expr mkrd set unset
? dirname false mount sleep uptime
alias dd fdinfo mv source usleep
unalias df free pidof test xd
basename dmesg help printf time
break echo hexdump ps true
cat env kill pwd truncate
cd exec ls rm uname
Builtin Apps:
hello hello_rust nsh ostest sh
To Exit QEMU: Press Ctrl-A then x
What about QEMU for 64-bit RISC-V?
Sorry Rust Apps won’t build correctly on NuttX for 64-bit RISC-V…
We’ll fix this in GSoC and test it on Ox64 BL808 SBC.
We’ve done Console Output. How about Console Input?
This is how we read Console Input in Rust: hello_rust_main.rs
// main() function not needed. Use Rust Core Library.
#![no_main]
#![no_std]
// Import the Types for C Interop
use core::ffi::{ c_char, c_int, c_void };
// Import the Functions from C into Rust
extern "C" {
pub fn printf(format: *const u8, ...) -> i32;
pub fn puts(s: *const c_char) -> c_int;
pub fn fgets(buf: *mut c_char, n: c_int, stream: *mut c_void) -> *mut c_char;
pub fn lib_get_stream(fd: c_int) -> *mut c_void;
}The code above imports the fgets() function from C into Rust.
Calling fgets() is a little more complicated: hello_rust_main.rs
// Main Function exported by Rust to C
#[no_mangle]
pub extern "C" fn hello_rust_main(_argc: i32, _argv: *const *const u8) -> i32 {
// Receive some text from Standard Input and print it
unsafe {
// Standard Input comes from https://github.com/apache/nuttx/blob/master/include/stdio.h#L64-L68
let stdin: *mut c_void = // Equivalent to `void *`
lib_get_stream(0); // Init to Stream 0 (stdin)
// Input Buffer with 256 chars (including terminating null)
let mut buf: [c_char; 256] = // Input Buffer is Mutable (will change)
[0; 256]; // Init with nulls
// Read a line from Standard Input
if !fgets(
&mut buf[0], // Input Buffer
buf.len() as i32, // Buffer Size
stdin // Standard Input
).is_null() { // Catch the Input Error
// Print the line
printf(b"You entered...\n\0" as *const u8);
puts(&buf[0]);
}
}
// Exit with status 0
0
}
// Omitted: Panic HandlerThis gets a bit dangerous… The Input Buffer might Overflow if we’re not careful with the Parameters!
// Read a line from Standard Input
fgets(
&mut buf[0], // Input Buffer
buf.len() as i32, // Buffer Size
stdin // Standard Input
);Which makes us ponder about Memory Safety: “Hmmm the fgets() buffer size… Does it include the terminating null?” (Yep it does!)
What about Rust? Does it safely handle Console Input?
Reading the Standard Input in Rust looks simpler and safer…
// Allocate an Input Buffer from Heap Memory
let mut buffer = String::new();
// Read a line from Standard Input
io::stdin().read_line(&mut buffer)?;But this won’t work on NuttX because…
Rust Standard Input io::stdin() isn’t supported on Embedded Platforms
Dynamic Strings and Heap Memory won’t work on Embedded Platforms either
Bummer. How to do I/O safely on NuttX?
During GSoC we shall…
Create Rust Wrappers that will safely call NuttX POSIX Functions: open(), close(), ioctl(), …
Which will support Simple Strings via malloc()
Rustix Project tried to provide Comprehensive Rust Wrappers for NuttX POSIX. Sadly the project has stalled. We’ll implement simpler, lighter wrappers instead.
What happens when we compile our Rust App?
Watch how NuttX builds Rust Apps by calling rustc. (Instead of the usual cargo build)
## Build the NuttX Project with Tracing Enabled
$ make --trace
## Compile `hello_rust_main.rs` to `hello_rust.o`
## for Rust Target: RISC-V 32-bit (Soft-Float)
rustc \
--edition 2021 \
--emit obj \
-g \
--target riscv32i-unknown-none-elf \
-C panic=abort \
-O \
hello_rust_main.rs \
-o hello_rust_main.rs...apps.examples.hello_rust.o
## Copy `hello_rust.o` to `hello_rust_1.o` (Why?)
cp \
hello_rust_main.rs...apps.examples.hello_rust.o \
hello_rust_main.rs...apps.examples.hello_rust_1.o
## Omitted: Bundle `hello_rust_1.o`
## into library `staging/libapps.a`
## Link `staging/libapps.a` into `nuttx`
riscv64-unknown-elf-ld \
--entry=__start \
-melf32lriscv \
--gc-sections \
-nostdlib \
--cref \
-Map=nuttx/nuttx.map \
-Tboards/risc-v/qemu-rv/rv-virt/scripts/ld.script.tmp \
-L staging \
-L arch/risc-v/src/board \
-o nuttx \
qemu_rv_head.o \
--start-group \
-lsched \
-ldrivers \
-lboards \
-lc \
-lmm \
-larch \
-lm \
-lapps \
-lfs \
-lbinfmt \
-lboard riscv64-unknown-elf-toolchain-10.2.0-2020.12.8-x86_64-apple-darwin/bin/../lib/gcc/riscv64-unknown-elf/10.2.0/rv32imafdc/ilp32d/libgcc.a \
--end-group
(Rust Build with rustc is defined here)
(Why NuttX calls rustc instead of cargo build)
Here are the Rust Binaries produced by the NuttX Build (which will be linked into the NuttX Firmware)…
$ ls -l ../apps/examples/hello_rust
total 112
-rw-r--r-- 1 650 Jul 20 2023 Kconfig
-rw-r--r-- 1 1071 Jul 20 2023 Make.defs
-rw-r--r-- 1 141 Mar 17 09:44 Make.dep
-rw-r--r-- 1 1492 Mar 16 20:41 Makefile
-rw-r--r-- 1 3982 Mar 17 00:06 hello_rust_main.rs
-rw-r--r-- 1 13168 Mar 17 09:44 hello_rust_main.rs...apps.examples.hello_rust.o
-rw-r--r-- 1 18240 Mar 17 09:54 hello_rust_main.rs...apps.examples.hello_rust_1.o
Now that we understand the build, let’s talk about the hiccups…
What’s this error? “Can’t link soft-float modules with double-float modules”
$ make
LD: nuttx
riscv64-unknown-elf-ld: nuttx/nuttx/staging/libapps.a
(hello_rust_main.rs...nuttx.apps.examples.hello_rust_1.o):
can't link soft-float modules with double-float modules
riscv64-unknown-elf-ld: failed to merge target specific data of file
nuttx/staging/libapps.a
(hello_rust_main.rs...nuttx.apps.examples.hello_rust_1.o)
GCC Linker failed to link the Compiled Rust Binary (hello_rust_1.o) into our NuttX Firmware because…
Rust Binary hello_rust_1.o was compiled with…
Software Floating-Point (“soft-float”)
But NuttX Firmware was compiled with…
Double Precision Hardware Floating-Point (“double-float”)
The two are incompatible. And the GCC Linking fails.
How to fix this?
For now we Patch the ELF Header of our Rust Object File. And NuttX Firmware will link correctly…
## Patch ELF Header from Soft-Float to Double-Float
xxd -c 1 ../apps/examples/hello_rust/*hello_rust_1.o \
| sed 's/00000024: 00/00000024: 04/' \
| xxd -r -c 1 - /tmp/hello_rust_1.o
cp /tmp/hello_rust_1.o ../apps/examples/hello_rust/*hello_rust_1.o
make
## NuttX links OK. Ignore these warnings: (why?)
## riscv64-unknown-elf-ld: warning: nuttx/staging/libapps.a(hello_rust_main.rs...nuttx.apps.examples.hello_rust_1.o):
## mis-matched ISA version 2.1 for 'i' extension, the output version is 2.0
What exactly are we patching in the ELF Header?
Inside the ELF Header of an Object File: There’s a Flag (at Offset 0x24) that says whether it was compiled for…
Software Floating-Point: Flags = 0, or…
Double-Precision Hardware Floating-Point: Flags = 4
We modified the Flag in the ELF Header so that it says Double-Float…
## Before Patching: ELF Header says Software Floating-Point
$ riscv64-unknown-elf-readelf -h -A ../apps/examples/hello_rust/*hello_rust_1.o
Flags: 0x0
## After Patching: ELF Header says Double-Precision Hardware Floating-Point
$ riscv64-unknown-elf-readelf -h -A ../apps/examples/hello_rust/*hello_rust_1.o
Flags: 0x4, double-float ABI
And it links correctly!
(We had a similar issue with Zig Compiler)
But why Soft-Float instead of Double-Float? (Mmmm ice cream float)
Yeah patching the ELF Header is a Bad Hack! Here’s the complete analysis and proper solution…
What’s this core::panicking::panic? Why is it undefined?
$ make
riscv64-unknown-elf-ld:
nuttx/staging/libapps.a(hello_rust_main.rs...apps.examples.hello_rust_1.o):
in function `no symbol':
apps/examples/hello_rust/hello_rust_main.rs:90:
undefined reference to `core::panicking::panic'
Suppose we’re reading Console Input in our Rust App: hello_rust_main.rs
// Input Buffer with 256 chars (including terminating null)
let mut buf: [c_char; 256] = // Input Buffer is Mutable (will change)
[0; 256]; // Init with nulls
// Read a line from Standard Input
fgets(
&mut buf[0], // Buffer
buf.len() as i32 - 1, // Size (cast to Signed Integer)
stdin // Standard Input
);“buf.len() - 1” might Panic and Halt. (Why?)
To implement the panic, Rust Compiler inserts a call to the Core Function core::panicking::panic.
(Which comes from the Rust Core Library)
And the Panic Function is missing somehow?
Rushabh has implemented a fix for the Undefined Panic Function…
But when we add Another Point of Panic: We see the Undefined Panic Error again (sigh)…
What’s causing this Undefined Panic Function?
According to this discussion, the Rust Core Library is compiled with Link-Time Optimisation (LTO). (Including the Panic Function)
But we’re linking it into our NuttX Firmware with GCC Linker, with LTO Disabled (by default). Which causes the Missing Panic Function.
How is this different from typical Rust Builds?
Normally we run cargo build to compile our Embedded Rust Apps. And it handles LTO correctly.
But NuttX calls rustc to compile Rust Apps, then calls GCC Linker to link into our NuttX Firmware. Which doesn’t seem to support LTO.
We’ll sort this out in GSoC!
(Why NuttX calls rustc instead of cargo build)
(Which means we can’t import Rust Crates from crates.io!)
What is [no_std]? Will Rust call C Standard Library, like for malloc()?
Earlier we saw [no_std] inside our Rust App.
There are 2 “flavours” of Rust, depending on the Rust Libraries that we use:
Rust Standard Library: This is used by most Rust Apps on Desktops and Servers.
Supports Heap Memory and the Rust Equivalent of POSIX Calls.
Rust Core Library [no_std]: Barebones Rust Library that runs on Bare Metal, used by Rust Embedded Apps.
Calls minimal functions in C Standard Library. Doesn’t support Heap Memory and POSIX.
The malloc() that we mentioned: It’s called by the Rust Standard Library. (Like this)
What about Rust Drivers for NuttX Kernel?
For Kernel Dev (like Linux): We’ll use the Rust Core Library. Which doesn’t support Heap Memory and doesn’t need malloc().
But most Kernel Drivers will need Kernel Heap!
That’s why Linux Kernel supports the alloc Rust Library / Crate for Heap Memory. To implement Rust alloc, Linux Kernel calls krealloc() to allocate Kernel Heap. (Like this)
For NuttX Kernel: We’ll implement Rust alloc by calling kmm_malloc().
Anything else we need for Rust in NuttX Kernel?
Since we’re calling Rust Core Library in NuttX Kernel, we won’t touch any POSIX Application Interfaces. Thus if we need to support the Kernel Equivalent of Errno (and other Global State), we’ll have to create the Rust Library ourselves.
(See the Rust Library for Linux Kernel)
(GSoC Project Report will discuss a Simple LED Driver in Rust for NuttX Kernel)
Why are we doing all this?
Yeah it’s tough work but it needs to be done because…
— Some folks are urging us to explore Memory-Safe Programming in Rust
— NuttX Devs among us might already be coding Rust Apps and Rust Drivers for NuttX? (We know of one Corporate User of NuttX that’s super keen on Rust)
— Hence we’re helpfully drafting the Standards and Guidelines for folks already coding Rust in NuttX
Learning Rust looks kinda hard. Any other way to write Memory-Safe Apps?
If we’re familiar with Python: Check out the Nim Programming Language.
Zig Programming Language is safer than C and easier to learn. But not quite Memory-Safe like Rust.
AI Tools might be helpful for coding the difficult bits of Rust: ChatGPT, GitHub Copilot, Google Gemini, …
(We’ll validate this during GSoC)
Giving in to our AI Overlords already?
But Rust Devs are familiar with smarty tools. Borrow Checker and Cargo Clippy are already so clever, they might as well be AI!
And Rust Compiler is almost Sentient, always commanding us Humans: “Please do this to fix the build, you poopy nincompoop!”
(My Biggest Wish: Someone please create a Higher-Level Dialect of Rust that will use bits of AI to compile into the present Low-Level Rust. Which might simplify Generics, Lifetimes, Box, Rc, Arc, RefCell, Fn*, dyn, async, …)
Will there be Resistance to Rust Drivers inside NuttX Kernel?
Ouch we’re trapped between a Rock and… Another Rusty Rock!
— NuttX Devs are concerned about the extra complexity that Rust Drivers add to the Kernel Build
— Rust Community is probably thinking we’re not doing enough to promote Memory-Safe Coding in NuttX Kernel
For now we walk the Middle Way…
— Lay the Groundwork for Future Integration of Rust Drivers into NuttX Kernel
— Observe the Rust Development in Linux Kernel and Zephyr OS. Then adapt the Best Practices for NuttX Kernel.
Coming This Summer: Plenty to be done for Rust Apps on Apache NuttX RTOS!
Compiling Rust Apps for NuttX works mostly OK, but could be improved
Rust Apps run OK on QEMU 32-bit RISC-V Emulator, though somewhat broken on 64-bit RISC-V (like Ox64 BL808 SBC)
Software vs Hardware Floating-Point becomes a problem for 32-bit and 64-bit RISC-V Platforms, we should remove the awful patchy hack
Link-Time Optimisation causes Linking Issues with the Rust Panic Handler
Console Input / Output works fine for Rust on NuttX. We’ll explore safer ways to call NuttX POSIX Functions.
All this will happen during Google Summer of Code!
Check out the next article…
Many Thanks to my GitHub Sponsors (and the awesome NuttX Community) for supporting my work! This article wouldn’t have been possible without your support.
Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…
lupyuen.github.io/src/rust3.md
What’s this core::panicking::panic? Why is it undefined?
$ make
riscv64-unknown-elf-ld:
nuttx/staging/libapps.a(hello_rust_main.rs...apps.examples.hello_rust_1.o):
in function `no symbol':
apps/examples/hello_rust/hello_rust_main.rs:90:
undefined reference to `core::panicking::panic'
Earlier we spoke about the Undefined Panic Function…
Which Rushabh has fixed with this patch…
But watch what happens when we add Another Point of Panic…
Below is our Test Code that has Two Potential Panics: hello_rust_main.rs
Buffer Length might panic. (Why?)
// Input Buffer with 256 chars (including terminating null)
let mut buf: [c_char; 256] = // Input Buffer is Mutable (will change)
[0; 256]; // Init with nulls
// Read a line from Standard Input
fgets(
&mut buf[0], // Buffer
// This might Panic! (Why?)
buf.len() as i32 - 1, // Unsigned Size cast to Signed Integer
stdin // Standard Input
);Divide by Zero will also panic
// Buffer might begin with null
// Which causes Divide by Zero
let i = 1 / buf[0];What happens when we compile this?
If we omit RUSTFLAGS=-O: We see Two Undefined Panic Functions…
apps/examples/hello_rust/hello_rust_main.rs:84
buf.len() as i32 - 1, // Might Panic (Why?)
a0: 00000097 auipc ra,0x0
a0: R_RISCV_CALL_PLT core::panicking::panic
apps/examples/hello_rust/hello_rust_main.rs:90
let i = 1 / buf[0]; // Might Divide by Zero
108: 00000097 auipc ra,0x0
108: R_RISCV_CALL_PLT core::panicking::panicAfter we add RUSTFLAGS=-O: We still see One Undefined Panic Function for the divide-by-zero…
apps/examples/hello_rust/hello_rust_main.rs:90
let i = 1 / buf[0]; // Might Divide by Zero
d0: 00000097 auipc ra,0x0
d0: R_RISCV_CALL_PLT core::panicking::panicWhich leads to the Undefined Panic Error again (sigh)…
$ make
riscv64-unknown-elf-ld:
nuttx/staging/libapps.a(hello_rust_main.rs...apps.examples.hello_rust_1.o):
in function `no symbol':
apps/examples/hello_rust/hello_rust_main.rs:90:
undefined reference to `core::panicking::panic'
What’s causing this Undefined Panic Function?
According to this discussion, the Rust Core Library is compiled with Link-Time Optimisation (LTO). (Including the Panic Function)
But we’re linking it into our NuttX Firmware with GCC Linker, with LTO Disabled (by default). Which causes the Missing Panic Function.
How is this different from typical Rust Builds?
Normally we run cargo build to compile our Embedded Rust Apps. And it handles LTO correctly.
But NuttX calls rustc to compile Rust Apps, then calls GCC Linker to link into our NuttX Firmware. Which doesn’t seem to support LTO.
We’ll sort this out in GSoC!
(Why NuttX calls rustc instead of cargo build)
(Which means we can’t import Rust Crates from crates.io!)
We tested Rust Apps on QEMU for 32-bit RISC-V. What about 64-bit RISC-V?
Sorry Rust Apps won’t build correctly on NuttX for 64-bit RISC-V…
$ tools/configure.sh rv-virt:nsh64
$ make menuconfig
## TODO: Enable "Hello Rust Example"
$ make
RUSTC: hello_rust_main.rs error: Error loading target specification:
Could not find specification for target "riscv64i-unknown-none-elf".
Run `rustc --print target-list` for a list of built-in targets
make[2]: *** [nuttx/apps/Application.mk:275: hello_rust_main.rs...nuttx.apps.examples.hello_rust.o] Error 1
make[1]: *** [Makefile:51: nuttx/apps/examples/hello_rust_all] Error 2
make: *** [tools/LibTargets.mk:232: nuttx/apps/libapps.a] Error 2
Which says that riscv64i-unknown-none-elf isn’t a valid Rust Target.
(Should be riscv64gc-unknown-none-elf instead)
We’ll fix this in GSoC and test it on Ox64 BL808 SBC.
TODO: Test on QEMU Arm32 and Arm64