Build an IoT App with Zig and LoRaWAN

📝 15 Jun 2022

Pine64 PineDio Stack BL604 RISC-V Board (left) talking LoRaWAN on Zig to RAKwireless WisGate LoRaWAN Gateway (right)

Pine64 PineDio Stack BL604 RISC-V Board (left) talking LoRaWAN on Zig to RAKwireless WisGate LoRaWAN Gateway (right)

In our last article we learnt to run barebones Zig on a Microcontroller (RISC-V BL602) with a Real-Time Operating System (Apache NuttX RTOS)…

But can we do something way more sophisticated with Zig?

Yes we can! Today we shall run a complex IoT Application with Zig and LoRaWAN

Which is the typical firmware we would run on IoT Sensors.

Will this run on any device?

We’ll do this on Pine64’s PineDio Stack BL604 RISC-V Board.

But the steps should be similar for BL602, ESP32-C3, Arm Cortex-M and other 32-bit microcontrollers supported by Zig.

Why are we doing this?

I always dreaded maintaining and extending complex IoT Apps in C. (Like this one)

Will Zig make this a little less painful? Let’s find out!

This is the Zig source code that we’ll study today…

Pine64 PineCone BL602 Board (right) connected to Semtech SX1262 LoRa Transceiver (left). This works too!

Pine64 PineCone BL602 Board (right) connected to Semtech SX1262 LoRa Transceiver (left). This works too!

§1 LoRaWAN Network Stack

What’s a LoRaWAN Network Stack?

To talk to a LoRaWAN Wireless Network, our IoT Gadget needs 3 things…

Together, the LoRa Driver and LoRaWAN Driver make up the LoRaWAN Network Stack.

Which LoRaWAN Stack will we use?

We’ll use Semtech’s Reference Implementation of the LoRaWAN Stack…

That we’ve ported to PineDio Stack BL604 with Apache NuttX RTOS

The same LoRaWAN Stack is available on many other platforms, including Zephyr OS and Arduino.

(My good friend JF is porting the LoRaWAN Stack to Linux)

But the LoRaWAN Stack is in C! Will it work with Zig?

Yep no worries, Zig will happily import the LoRaWAN Stack from C without any wrappers or modifications.

And we’ll call the LoRaWAN Stack as though it were a Zig Library.

So we’re not rewriting the LoRaWAN Stack in Zig?

Rewriting the LoRaWAN Stack in Zig (or another language) sounds risky because the LoRaWAN Stack is still under Active Development. It can change at any moment!

We’ll stick with the C Implementation of the LoRaWAN Stack so that our Zig IoT App will enjoy the latest LoRaWAN updates and features.

(More about this)

Why is our Zig IoT App so complex anyway?

That’s because…

Great way to test if Zig can really handle Complex Embedded Apps!

Import LoRaWAN Library

(Source)

§2 Import LoRaWAN Library

Let’s dive into our Zig IoT App. We import the Zig Standard Library at the top of our app: lorawan_test.zig

/// Import the Zig Standard Library
const std = @import("std");

Then we call @cImport to import the C Macros and C Header Files

/// Import the LoRaWAN Library from C
const c = @cImport({
  // Define C Macros for NuttX on RISC-V, equivalent to...
  // #define __NuttX__
  // #define NDEBUG
  // #define ARCH_RISCV

  @cDefine("__NuttX__",  "");
  @cDefine("NDEBUG",     "");
  @cDefine("ARCH_RISCV", "");

The code above defines the C Macros that will be called by the C Header Files coming up.

Next comes a workaround for a C Macro Error that appears on Zig with Apache NuttX RTOS…

  // Workaround for "Unable to translate macro: undefined identifier `LL`"
  @cDefine("LL", "");
  @cDefine("__int_c_join(a, b)", "a");  //  Bypass zig/lib/include/stdint.h

(More about this)

We import the C Header Files for Apache NuttX RTOS…

  // Import the NuttX Header Files from C, equivalent to...
  // #include <arch/types.h>
  // #include <../../nuttx/include/limits.h>
  // #include <stdio.h>

  @cInclude("arch/types.h");
  @cInclude("../../nuttx/include/limits.h");
  @cInclude("stdio.h");

(More about the includes)

Followed by the C Header Files for our LoRaWAN Library

  // Import LoRaWAN Header Files from C, based on
  // https://github.com/Lora-net/LoRaMac-node/blob/master/src/apps/LoRaMac/fuota-test-01/B-L072Z-LRWAN1/main.c#L24-L40
  @cInclude("firmwareVersion.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/githubVersion.h");
  @cInclude("../libs/liblorawan/src/boards/utilities.h");
  @cInclude("../libs/liblorawan/src/mac/region/RegionCommon.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/Commissioning.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandler/LmHandler.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandler/packages/LmhpCompliance.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandler/packages/LmhpClockSync.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandler/packages/LmhpRemoteMcastSetup.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandler/packages/LmhpFragmentation.h");
  @cInclude("../libs/liblorawan/src/apps/LoRaMac/common/LmHandlerMsgDisplay.h");
});

(Based on this C code)

The LoRaWAN Library is ready to be called by our Zig App!

This is how we reference the LoRaWAN Library to define our LoRaWAN Region

/// LoRaWAN Region
const ACTIVE_REGION = c.LORAMAC_REGION_AS923;

(Source)

Why the “c.” in c.LORAMAC_REGION_AS923?

Remember that we imported the LoRaWAN Library under the Namespace “c

/// Import the LoRaWAN Library under Namespace "c"
const c = @cImport({ ... });

(Source)

Hence we use “c.something” to refer to the Constants and Functions defined in the LoRaWAN Library.

Why did we define the C Macros like __NuttX__?

These C Macros are needed by the NuttX Header Files.

Without the macros, the NuttX Header Files won’t be imported correctly into Zig. (See this)

Why did we import “arch/types.h”?

This fixes a problem with the NuttX Types. (See this)

Let’s head over to the Main Function…

Zig App calls LoRaWAN Library imported from C

(Source)

§3 Main Function

This is the Main Function for our Zig App: lorawan_test.zig

/// Main Function that will be called by NuttX.
/// We call the LoRaWAN Library to join a 
/// LoRaWAN Network and send a Data Packet.
pub export fn lorawan_test_main(
  _argc: c_int, 
  _argv: [*]const [*]const u8
) c_int {
  _ = _argc;
  _ = _argv;

  // Init the Timer Struct at startup
  TxTimer = std.mem.zeroes(c.TimerEvent_t);

(We init TxTimer here because of this)

We begin by computing the randomised interval between transmissions of LoRaWAN Data Packets…

  // Compute the interval between transmissions based on Duty Cycle
  TxPeriodicity = @intCast(u32,  // Cast to u32 because randr() can be negative
    APP_TX_DUTYCYCLE +
    c.randr(
      -APP_TX_DUTYCYCLE_RND,
      APP_TX_DUTYCYCLE_RND
    )
  );

(We’ll talk about @intCast in a while)

Our app sends LoRaWAN Data Packets every 40 seconds (roughly). (See this)

Next we show the App Version

  // Show the Firmware and GitHub Versions
  const appVersion = c.Version_t {
    .Value = c.FIRMWARE_VERSION,
  };
  const gitHubVersion = c.Version_t {
    .Value = c.GITHUB_VERSION,
  };
  c.DisplayAppInfo("Zig LoRaWAN Test", &appVersion, &gitHubVersion);

Then we initialise the LoRaWAN Library

  // Init LoRaWAN
  if (LmHandlerInit(&LmHandlerCallbacks, &LmHandlerParams)
    != c.LORAMAC_HANDLER_SUCCESS) {
    std.log.err("LoRaMac wasn't properly initialized", .{});

    // Fatal error, endless loop.
    while (true) {}
  }

(LmHandlerCallbacks and LmHandlerParams are defined here)

(We’ll explain “.{}” in a while)

We set the Max Tolerated Receive Error

  // Set system maximum tolerated rx error in milliseconds
  _ = c.LmHandlerSetSystemMaxRxError(20);

And we load some packages for LoRaWAN Compliance

  // The LoRa-Alliance Compliance protocol package should always be initialized and activated.
  _ = c.LmHandlerPackageRegister(c.PACKAGE_ID_COMPLIANCE,         &LmhpComplianceParams);
  _ = c.LmHandlerPackageRegister(c.PACKAGE_ID_CLOCK_SYNC,         null);
  _ = c.LmHandlerPackageRegister(c.PACKAGE_ID_REMOTE_MCAST_SETUP, null);
  _ = c.LmHandlerPackageRegister(c.PACKAGE_ID_FRAGMENTATION,      &FragmentationParams);

(LmhpComplianceParams and FragmentationParams are defined here)

Everything is hunky dory! We can now transmit a LoRaWAN Request to join the LoRaWAN Network

  // Init the Clock Sync and File Transfer status
  IsClockSynched     = false;
  IsFileTransferDone = false;

  // Join the LoRaWAN Network
  c.LmHandlerJoin();

(LoRaWAN Keys and EUIs are defined here)

We start the Transmit Timer that will send a LoRaWAN Data Packet at periodic intervals (right after we join the LoRaWAN Network)…

  // Set the Transmit Timer
  StartTxProcess(LmHandlerTxEvents_t.LORAMAC_HANDLER_TX_ON_TIMER);

Finally we loop forever handling LoRaWAN Events

  // Handle LoRaWAN Events
  handle_event_queue();  //  Never returns
  return 0;
}

(handle_event_queue is explained in the Appendix)

That’s all for the Main Function of our Zig App!

Our LoRaWAN Zig App

Wait… Our Zig Code looks familiar?

Yep our Zig Code is largely identical to the C Code in the Demo App for the LoRaWAN Stack…

Converting C Code to Zig looks rather straightforward. In a while we’ll talk about the tricky parts we encountered during the conversion.

Why did we call LmHandlerInit instead of c.LmHandlerInit?

That’s one of the tricky parts of our C-to-Zig conversion, as explained here…

Demo App for the LoRaWAN Stack

(Source)

§4 Convert Integer Type

Earlier we saw this computation of the randomised interval between transmissions of LoRaWAN Data Packets: lorawan_test.zig

// In Zig: Compute the interval between transmissions based on Duty Cycle.
// TxPeriodicity is an unsigned integer (32-bit).
// We cast to u32 because randr() can be negative.
TxPeriodicity = @intCast(u32,
  APP_TX_DUTYCYCLE +
  c.randr(
    -APP_TX_DUTYCYCLE_RND,
    APP_TX_DUTYCYCLE_RND
  )
);

(Roughly 40 seconds)

Let’s find out why @intCast is needed.

In the Original C Code we compute the interval without any Explicit Type Conversion

// In C: Compute the interval between transmissions based on Duty Cycle.
// TxPeriodicity is an unsigned integer (32-bit).
// Remember that randr() can be negative.
TxPeriodicity = 
  APP_TX_DUTYCYCLE + 
  randr( 
    -APP_TX_DUTYCYCLE_RND, 
    APP_TX_DUTYCYCLE_RND 
  );

(Source)

What happens if we compile this in Zig?

Zig Compiler shows this error…

unsigned 32-bit int cannot represent 
all possible signed 32-bit values

What does it mean?

Well TxPeriodicity is an Unsigned Integer

/// Random interval between transmissions
var TxPeriodicity: u32 = 0;

But randr() returns a Signed Integer

/// Computes a random number between min and max
int32_t randr(int32_t min, int32_t max);

Mixing Signed and Unsigned Integers is a Bad Sign (pun intended)…

randr() could potentially cause TxPeriodicity to underflow!

How does @intCast fix this?

When we write this with @intCast

TxPeriodicity = @intCast(u32,
  APP_TX_DUTYCYCLE +
  c.randr(
    -APP_TX_DUTYCYCLE_RND,
    APP_TX_DUTYCYCLE_RND
  )
);

We’re telling the Zig Compiler to convert the Signed Result to an Unsigned Integer.

(More about @intCast)

What happens if there’s an underflow?

The Signed-to-Unsigned Conversion fails and we’ll see a Runtime Error

!ZIG PANIC!
attempt to cast negative value to unsigned integer
Stack Trace:
0x23016dba

Great to have Zig watching our backs… When we do risky things! 👍

(How we implemented a Custom Panic Handler)

§5 Transmit Data Packet

Back to our Zig App: This is how we transmit a Data Packet to the LoRaWAN Network: lorawan_test.zig

/// Prepare the payload of a Data Packet 
/// and transmit it
fn PrepareTxFrame() void {

  // If we haven't joined the LoRaWAN Network...
  if (c.LmHandlerIsBusy()) {
    // Try again later
    return;
  }

LoRaWAN won’t let us transmit data unless we’ve joined the LoRaWAN Network. So we check this first.

Next we prepare the message to be sent (“Hi NuttX”)

  // Message to be sent to LoRaWAN
  const msg: []const u8 = "Hi NuttX\x00";  // 9 bytes including null
  debug("PrepareTxFrame: Transmit to LoRaWAN ({} bytes): {s}", .{ 
    msg.len, msg 
  });

(We’ll talk about debug in a while)

That’s 9 bytes, including the Terminating Null.

Why so smol?

The first LoRaWAN message needs to be 11 bytes or smaller, subsequent messages can be up to 53 bytes.

This depends on the LoRaWAN Data Rate and the LoRaWAN Region. (See this)

Then we copy the message into the LoRaWAN Buffer

  // Copy message into LoRaWAN buffer
  std.mem.copy(
    u8,              // Type
    &AppDataBuffer,  // Destination
    msg              // Source
  );

(std.mem.copy is documented here)

(AppDataBuffer is defined here)

We compose the LoRaWAN Transmit Request

  // Compose the transmit request
  var appData = c.LmHandlerAppData_t {
    .Buffer     = &AppDataBuffer,
    .BufferSize = msg.len,
    .Port       = 1,
  };

Remember that the Max Message Size depends on the LoRaWAN Data Rate and the LoRaWAN Region?

This is how we validate the Message Size to make sure that our message isn’t too large…

  // Validate the message size and check if it can be transmitted
  var txInfo: c.LoRaMacTxInfo_t = undefined;
  const status = c.LoRaMacQueryTxPossible(
    appData.BufferSize,  // Message Size
    &txInfo              // Unused
  );
  assert(status == c.LORAMAC_STATUS_OK);

Finally we transmit the message to the LoRaWAN Network…

  // Transmit the message
  const sendStatus = c.LmHandlerSend(
    &appData,                      // Transmit Request
    LmHandlerParams.IsTxConfirmed  // False (No acknowledge required)
  );
  assert(sendStatus == c.LORAMAC_HANDLER_SUCCESS);
  debug("PrepareTxFrame: Transmit OK", .{});
}

And that’s how PrepareTxFrame transmits a Data Packet over LoRaWAN.

How is PrepareTxFrame called?

After we have joined the LoRaWAN Network, our LoRaWAN Event Loop calls UplinkProcess

/// LoRaWAN Event Loop that dequeues Events from 
/// the Event Queue and processes the Events
fn handle_event_queue() void {

  // Loop forever handling Events from the Event Queue
  while (true) {
    // Omitted: Handle the next Event from the Event Queue
    ...

    // If we have joined the network, do the uplink
    if (!c.LmHandlerIsBusy()) {
      UplinkProcess();
    }

(Source)

UplinkProcess then calls PrepareTxFrame to transmit a Data Packet, when the Transmit Timer has expired.

(UplinkProcess is defined here)

(handle_event_queue is explained in the Appendix)

ChirpStack LoRaWAN Gateway receives Data Packet from our Zig App

ChirpStack LoRaWAN Gateway receives Data Packet from our Zig App

§6 Logging

Earlier we saw this code for printing a Debug Message

// Message to be sent
const msg: []const u8 = "Hi NuttX\x00";  // 9 bytes including null

// Print the message
debug("Transmit to LoRaWAN ({} bytes): {s}", .{ 
  msg.len, msg 
});

(Source)

The code above prints this Formatted Message to the console…

Transmit to LoRaWAN (9 bytes): Hi NuttX

The Format Specifiers {} and {s} embedded in the Format String are explained here…

What’s .{ ... }?

.{ ... } creates an Anonymous Struct with a variable number of arguments that will be passed to the debug function for formatting.

And if we have no arguments?

Then we do this…

// Print the message without formatting
debug("Transmit to LoRaWAN", .{});

We discuss the implementation of Zig Logging in the Appendix…

§7 Compile Zig App

Now that we understand the code, we’re ready to compile our LoRaWAN Zig App!

First we download the latest version of Zig Compiler (0.10.0 or later), extract it and add to PATH…

Then we download and compile Apache NuttX RTOS for PineDio Stack BL604…

Before compiling NuttX, configure the LoRaWAN App Key, Device EUI and Join EUI in the LoRaWAN Library…

After building NuttX, we download and compile our LoRaWAN Zig App

##  Download our LoRaWAN Zig App for NuttX
git clone --recursive https://github.com/lupyuen/zig-bl602-nuttx
cd zig-bl602-nuttx

##  TODO: Edit lorawan_test.zig and set the LoRaWAN Region...
##  const ACTIVE_REGION = c.LORAMAC_REGION_AS923;

##  Compile the Zig App for BL602
##  (RV32IMACF with Hardware Floating-Point)
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
zig build-obj \
  --verbose-cimport \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -isystem "$HOME/nuttx/nuttx/include" \
  -I "$HOME/nuttx/apps/examples/lorawan_test" \
  lorawan_test.zig

(See the Compile Log)

Note that target and mcpu are specific to BL602…

How did we get the Compiler Options -isystem and -I?

Remember that we’ll link our Compiled Zig App with Apache NuttX RTOS.

Hence the Zig Compiler Options must be the same as the GCC Options used to compile NuttX.

(See the GCC Options for NuttX)

Next comes a quirk specific to BL602: We must patch the ELF Header from Software Floating-Point ABI to Hardware Floating-Point ABI…

##  Patch the ELF Header of `lorawan_test.o` 
##  from Soft-Float ABI to Hard-Float ABI
xxd -c 1 lorawan_test.o \
  | sed 's/00000024: 01/00000024: 03/' \
  | xxd -r -c 1 - lorawan_test2.o
cp lorawan_test2.o lorawan_test.o

(More about this)

Finally we inject our Compiled Zig App into the NuttX Project Directory and link it into the NuttX Firmware

##  Copy the compiled app to NuttX and overwrite `lorawan_test.o`
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
cp lorawan_test.o $HOME/nuttx/apps/examples/lorawan_test/*lorawan_test.o

##  Build NuttX to link the Zig Object from `lorawan_test.o`
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
cd $HOME/nuttx/nuttx
make

##  For WSL: Copy the NuttX Firmware to c:\blflash for flashing
mkdir /mnt/c/blflash
cp nuttx.bin /mnt/c/blflash

We’re ready to run our Zig App!

Running our LoRaWAN Zig App

§8 Run Zig App

Follow these steps to flash and boot NuttX (with our Zig App inside) on PineDio Stack…

In the NuttX Shell, enter this command to start our Zig App…

lorawan_test

Our Zig App starts and transmits a LoRaWAN Request to join the LoRaWAN Network (by controlling the LoRa Transceiver over SPI)…

Application name   : Zig LoRaWAN Test
###### =========== MLME-Request ============ ######
######               MLME_JOIN               ######
###### ===================================== ######

(See the complete log)

5 seconds later, our app receives the Join Accept Response from our ChirpStack LoRaWAN Gateway (by handling the GPIO Interrupt triggered by the LoRa Transceiver)…

###### =========== MLME-Confirm ============ ######
STATUS      : OK
###### ===========   JOINED     ============ ######
OTAA
DevAddr     :  00D803AB
DATA RATE   : DR_2

(Source)

We have successfully joined the LoRaWAN Network!

Every 40 seconds, our app transmits a Data Packet (“Hi NuttX”) to the LoRaWAN Network…

PrepareTxFrame: Transmit to LoRaWAN (9 bytes): Hi NuttX
###### =========== MCPS-Confirm ============ ######
STATUS      : OK
###### =====   UPLINK FRAME        1   ===== ######
CLASS       : A
TX PORT     : 1
TX DATA     : UNCONFIRMED
48 69 20 4E 75 74 74 58 00
DATA RATE   : DR_3
U/L FREQ    : 923200000
TX POWER    : 0
CHANNEL MASK: 0003

(Source)

The Data Packet appears in our LoRaWAN Gateway (ChirpStack), like in the pic below.

Yep our LoRaWAN Zig App has successfully transmitted a Data Packet to the LoRaWAN Network! 🎉

ChirpStack LoRaWAN Gateway receives Data Packet from our Zig App

Can we test our app without a LoRaWAN Gateway?

Our app will work fine with The Things Network, the worldwide free-to-use LoRaWAN Network.

Check the Network Coverage here…

And set the LoRaWAN Parameters like so…

To get the DevEUI and AppKey from The Things Network…

(I don’t think NWK_KEY is used)

The Things Network receives Data Packet from our LoRaWAN App

The Things Network receives Data Packet from our LoRaWAN App

§9 Safety Checks

Our IoT App is now in Zig instead of C. Do we gain anything with Zig?

We claimed earlier that Zig is watching our backs (in case we do something risky)…

Let’s dig for more evidence that Zig really tries to protect our programs…

This C Code (from the original LoRaWAN Demo) copies an array, byte by byte…

static int8_t FragDecoderWrite(uint32_t addr, uint8_t *data, uint32_t size) {
  for (uint32_t i = 0; i < size; i++ ) {
    UnfragmentedData[addr + i] = data[i];
  }

(Source)

Our Zig Compiler has a fascinating feature: It can translate C programs into Zig!

When we feed the above C Code into Zig’s Auto-Translator, it produces this functionally-equivalent Zig Code

pub fn FragDecoderWrite(addr: u32, data: [*c]u8, size: u32) callconv(.C) i8 {
  var i: u32 = 0;
  while (i < size) : (i +%= 1) {
    UnfragmentedData[addr +% i] = data[i];
  }

(Source)

Hmmm something looks different?

Yep the Array Indexing in C…

//  Array Indexing in C...
UnfragmentedData[addr + i]

Gets translated to this in Zig…

//  Array Indexing in Zig...
UnfragmentedData[addr +% i]

+” in C becomes “+%” in Zig!

What’s “+%” in Zig?

That’s the Zig Operator for Wraparound Addition.

Which means that the result wraps back to 0 (and beyond) if the addition overflows the integer.

Exactly how we expect C to work right?

Yep the Zig Compiler has faithfully translated the Wraparound Addition from C to Zig.

But this isn’t what we intended, since we don’t expect the addition to overflow.

That’s why in our final converted Zig code, we revert “+%” back to “+

export fn FragDecoderWrite(addr: u32, data: [*c]u8, size: u32) i8 {
  var i: u32 = 0;
  while (i < size) : (i += 1) {
    //  We changed `+%` back to `+`
    UnfragmentedData[addr + i] = data[i];
  }

(Source)

But what happens if the addition overflows?

We’ll see a Runtime Error…

panic: integer overflow

(Source)

Which is probably a good thing, to ensure that our values are sensible.

What if our Array Index goes out of bounds?

We’ll get another Runtime Error…

panic: index out of bounds

(Source)

We handle Runtime Errors in our Custom Panic Handler, as explained here…

So Zig watches for underflow / overflow / out-of-bounds errors at runtime. Anything else?

Here’s the list of Safety Checks done by Zig at runtime…

Thus indeed, Zig tries very hard to catch all kinds of problems at runtime.

And that’s super helpful for a complex app like ours.

Can we turn off the Safety Checks?

If we prefer to live a little recklessly (momentarily), this is how we disable the Safety Checks

PineDio Stack BL604

§10 Zig Outcomes

Once again… Why are we doing this in Zig?

Let’s recap: We have a complex chunk of firmware that needs to run on an IoT gadget (PineDio Stack)…

We wished we could rewrite the LoRaWAN Stack in a modern, memory-safe language… But we can’t. (Because LoRaWAN changes)

But we can do this partially in Zig right?

Yes it seems the best we can do today is to…

And Zig Compiler will do the Zig-to-C plumbing for us. (As we’ve seen)

Zig Compiler calls Clang to import the C Header Files. But NuttX compiles with GCC. Won’t we have problems with code compatibility?

We have validated Zig Compiler’s Clang as a drop-in replacement for GCC…

Hence we’re confident that Zig will interoperate correctly with the LoRaWAN Stack and Apache NuttX RTOS.

(Well for BL602 NuttX at least)

Were there problems with Zig-to-C interoperability?

We hit some minor interoperability issues and we found workarounds…

No showstoppers, so our Zig App is good to go!

Is Zig effective in managing the complexity of our firmware?

I think it is! Zig has plenty of Safety Checks to help ensure that we’re doing the right thing…

Now I feel confident that I can safely extend our Zig App to do more meaningful IoT things…

We’ll extend our Zig App the modular way thanks to @import

Extending our Zig App with CBOR, The Things Network, Prometheus and Grafana

Is there anything else that might benefit from Zig?

LVGL Touchscreen Apps might be easier to maintain when we code them in Zig.

(Since LVGL looks as complicated as LoRaWAN)

Someday I’ll try LVGL on Zig… And we might possibly combine it with LoRaWAN in a single Zig App! Check the updates here…

LVGL Touchscreen Apps might benefit from Zig

LVGL Touchscreen Apps might benefit from Zig

§11 What’s Next

I hope this article has inspired you to create IoT apps in Zig!

In the coming weeks I shall flesh out our Zig App, so that it works like a real IoT Sensor Device.

(With Temperature Sensor, CBOR Encoding, The Things Network, …)

For the next article we’ll take a quick detour and explore Zig on PinePhone

And then back to Zig on Apache NuttX RTOS

Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/iot.md

§12 Notes

  1. This article is the expanded version of this Twitter Thread

  2. This article was inspired by a question from my GitHub Sponsor: “Can we run Zig on BL602 with Apache NuttX RTOS?”

  3. These articles were super helpful for Zig-to-C Interoperability

    “Working with C”

    “Compile a C/C++ Project with Zig”

    “Extend a C/C++ Project with Zig”

    “Maintain it With Zig”

  4. Can we use Zig Async Functions to simplify our Zig IoT App?

    Interesting idea, let’s explore that! (See this)

  5. I’m now using Zig Type Reflection to document the internals of the LoRaWAN Library…

    “Zig Type Reflection for LoRaWAN Library”

    The LoRaWAN Library is a popular library that runs on many platforms, would be great if Zig can create helpful docs for the complicated multithreaded library.

Handle LoRaWAN Events with NimBLE Porting Layer

§13 Appendix: Handle LoRaWAN Events

Let’s look at the Event Loop that handles the LoRa and LoRaWAN Events in our app.

Our Event Loop looks different from the Original LoRaWAN Demo App?

Yep the Original LoRaWAN Demo App handles LoRaWAN Events in a Busy-Wait Loop. (See this)

But since our Zig App runs on a Real-Time Operating System (RTOS), we can use the Multithreading Features (Timers and Event Queues) provided by the RTOS.

So we’re directly calling the Timers and Event Queues from Apache NuttX RTOS?

Not quite. We’re calling the Timers and Event Queues provided by NimBLE Porting Layer.

NimBLE Porting Layer is a Portable Multitasking Library that works on multiple operating systems: FreeRTOS, Linux, Mynewt, NuttX, RIOT.

By calling NimBLE Porting Layer, our modded LoRaWAN Stack will run on all of these operating systems (hopefully).

(More about NimBLE Porting Layer)

Alright let’s see the code!

Our Event Loop forever reads LoRa and LoRaWAN Events from an Event Queue and handles them.

The Event Queue is created in our LoRa SX1262 Library as explained here…

The Main Function of our LoRaWAN App calls this function to run the Event Loop: lorawan_test.zig

/// LoRaWAN Event Loop that dequeues Events from the Event Queue and processes the Events
fn handle_event_queue() void {

  // Loop forever handling Events from the Event Queue
  while (true) {

    // Get the next Event from the Event Queue
    var ev: [*c]c.ble_npl_event = c.ble_npl_eventq_get(
      &event_queue,           //  Event Queue
      c.BLE_NPL_TIME_FOREVER  //  No Timeout (Wait forever for event)
    );

This code runs in the Foreground Thread of our app.

Here we loop forever, waiting for Events from the Event Queue.

When we receive an Event, we remove the Event from the Event Queue…

    // If no Event due to timeout, wait for next Event.
    // Should never happen since we wait forever for an Event.
    if (ev == null) { debug("handle_event_queue: timeout", .{}); continue; }
    debug("handle_event_queue: ev=0x{x}", .{ @ptrToInt(ev) });

    // Remove the Event from the Event Queue
    c.ble_npl_eventq_remove(&event_queue, ev);

We call the Event Handler Function that was registered with the Event…

    // Trigger the Event Handler Function
    c.ble_npl_event_run(ev);

The rest of the Event Loop handles LoRaWAN Events

    // Process the LoRaMac events
    c.LmHandlerProcess();

LmHandlerProcess handles Join Network Events in the LoRaMAC Layer of our LoRaWAN Library.

If we have joined the LoRaWAN Network, we transmit data to the network…

    // If we have joined the network, do the uplink
    if (!c.LmHandlerIsBusy()) {
      UplinkProcess();
    }

(UplinkProcess calls PrepareTxFrame to transmit a Data Packet, which we have seen earlier)

The last part of the Event Loop will handle Low Power Mode in future…

    // TODO: CRITICAL_SECTION_BEGIN();
    if (IsMacProcessPending == 1) {
      // Clear flag and prevent MCU to go into low power mode
      IsMacProcessPending = 0;
    } else {
      // The MCU wakes up through events
      // TODO: BoardLowPowerHandler();
    }
    // TODO: CRITICAL_SECTION_END();
  }
}

And we loop back perpetually, waiting for Events and handling them.

That’s how we handle LoRa and LoRaWAN Events with NimBLE Porting Layer!

§14 Appendix: Logging

We have implemented Zig Debug Logging std.log.debug that’s described here…

Here’s how we call std.log.debug to print a log message…

//  Create a short alias named `debug`
const debug  = std.log.debug;

//  Message with 8 bytes
const msg: []const u8 = "Hi NuttX";

//  Print the message
debug("Transmit to LoRaWAN ({} bytes): {s}", .{ 
  msg.len, msg 
});

// Prints: Transmit to LoRaWAN (8 bytes): Hi NuttX

.{ ... } creates an Anonymous Struct with a variable number of arguments that will be passed to std.log.debug for formatting.

Below is our implementation of std.log.debug

/// Called by Zig for `std.log.debug`, `std.log.info`, `std.log.err`, ...
/// https://gist.github.com/leecannon/d6f5d7e5af5881c466161270347ce84d
pub fn log(
  comptime _message_level: std.log.Level,
  comptime _scope: @Type(.EnumLiteral),
  comptime format: []const u8,
  args: anytype,
) void {
  _ = _message_level;
  _ = _scope;

  // Format the message
  var buf: [100]u8 = undefined;  // Limit to 100 chars
  var slice = std.fmt.bufPrint(&buf, format, args)
    catch { _ = puts("*** log error: buf too small"); return; };
    
  // Terminate the formatted message with a null
  var buf2: [buf.len + 1 : 0]u8 = undefined;
  std.mem.copy(
    u8, 
    buf2[0..slice.len], 
    slice[0..slice.len]
  );
  buf2[slice.len] = 0;

  // Print the formatted message
  _ = puts(&buf2);
}

(Source)

This implementation calls puts(), which is supported by Apache NuttX RTOS since it’s POSIX-Compliant.

§15 Appendix: Panic Handler

Some debug features don’t seem to be working? Like unreachable, std.debug.assert and std.debug.panic?

That’s because for Embedded Platforms (like Apache NuttX RTOS) we need to implement our own Panic Handler

With our own Panic Handler, this Assertion Failure…

//  Create a short alias named `assert`
const assert = std.debug.assert;

//  Assertion Failure
assert(TxPeriodicity != 0);

Will show this Stack Trace…

!ZIG PANIC!
reached unreachable code
Stack Trace:
0x23016394
0x23016ce0

How do we read the Stack Trace?

We need to generate the RISC-V Disassembly for our firmware. (Like this)

According to our RISC-V Disassembly, the first address 23016394 doesn’t look interesting, because it’s inside the assert function…

/home/user/zig-linux-x86_64-0.10.0-dev.2351+b64a1d5ab/lib/std/debug.zig:259
pub fn assert(ok: bool) void {
2301637c:	00b51c63          	bne	a0,a1,23016394 <std.debug.assert+0x2c>
23016380:	a009                j	23016382 <std.debug.assert+0x1a>
23016382:	2307e537          	lui	a0,0x2307e
23016386:	f9850513          	addi	a0,a0,-104 # 2307df98 <__unnamed_4>
2301638a:	4581                li	a1,0
2301638c:	00000097          	auipc	ra,0x0
23016390:	f3c080e7          	jalr	-196(ra) # 230162c8 <panic>
    if (!ok) unreachable; // assertion failure
23016394:	a009                j	23016396 <std.debug.assert+0x2e>

But the second address 23016ce0 reveals the assertion that failed…

/home/user/nuttx/zig-bl602-nuttx/lorawan_test.zig:95
    assert(TxPeriodicity != 0);
23016ccc:	42013537          	lui	a0,0x42013
23016cd0:	fbc52503          	lw	a0,-68(a0) # 42012fbc <TxPeriodicity>
23016cd4:	00a03533          	snez	a0,a0
23016cd8:	fffff097          	auipc	ra,0xfffff
23016cdc:	690080e7          	jalr	1680(ra) # 23016368 <std.debug.assert>
/home/user/nuttx/zig-bl602-nuttx/lorawan_test.zig:100
    TxTimer = std.mem.zeroes(c.TimerEvent_t);
23016ce0:	42016537          	lui	a0,0x42016

This is our implementation of the Zig Panic Handler

/// Called by Zig when it hits a Panic. We print the Panic Message, Stack Trace and halt. See 
/// https://andrewkelley.me/post/zig-stack-traces-kernel-panic-bare-bones-os.html
/// https://github.com/ziglang/zig/blob/master/lib/std/builtin.zig#L763-L847
pub fn panic(
  message: []const u8, 
  _stack_trace: ?*std.builtin.StackTrace
) noreturn {
  // Print the Panic Message
  _ = _stack_trace;
  _ = puts("\n!ZIG PANIC!");
  _ = puts(@ptrCast([*c]const u8, message));

  // Print the Stack Trace
  _ = puts("Stack Trace:");
  var it = std.debug.StackIterator.init(@returnAddress(), null);
  while (it.next()) |return_address| {
    _ = printf("%p\n", return_address);
  }

  // Halt
  while(true) {}
}

(Source)

How do we tell Zig Compiler to use this Panic Handler?

We just need to define this panic function in the Root Zig Source File (like lorawan_test.zig), and the Zig Runtime will call it when there’s a panic.

§16 Appendix: Zig Compiler as Drop-In Replacement for GCC

Apache NuttX RTOS calls GCC to compile the BL602 firmware. Will Zig Compiler work as the Drop-In Replacement for GCC for compiling NuttX Modules?

Let’s test it on the LoRa SX1262 Library for Apache NuttX RTOS.

Here’s how NuttX compiles the LoRa SX1262 Library with GCC…

##  LoRa SX1262 Source Directory
cd $HOME/nuttx/nuttx/libs/libsx1262

##  Compile radio.c with GCC
riscv64-unknown-elf-gcc \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -march=rv32imafc \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/radio.c \
  -o  src/radio.o

##  Compile sx126x.c with GCC
riscv64-unknown-elf-gcc \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -march=rv32imafc \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/sx126x.c \
  -o  src/sx126x.o

##  Compile sx126x-nuttx.c with GCC
riscv64-unknown-elf-gcc \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -march=rv32imafc \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/sx126x-nuttx.c \
  -o  src/sx126x-nuttx.o

(As observed with “make –trace” when building NuttX)

We switch GCC to “zig cc” by making these changes…

After making the changes, we run this to compile the LoRa SX1262 Library with “zig cc” and link it with the NuttX Firmware…

##  LoRa SX1262 Source Directory
cd $HOME/nuttx/nuttx/libs/libsx1262

##  Compile radio.c with zig cc
zig cc \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/radio.c \
  -o  src/radio.o

##  Compile sx126x.c with zig cc
zig cc \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/sx126x.c \
  -o  src/sx126x.o

##  Compile sx126x-nuttx.c with zig cc
zig cc \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/sx126x-nuttx.c \
  -o  src/sx126x-nuttx.o

##  Link Zig Object Files with NuttX after compiling with `zig cc`
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
cd $HOME/nuttx/nuttx
make

Zig Compiler shows these errors…

In file included from src/sx126x-nuttx.c:3:
In file included from nuttx/include/debug.h:39:
In file included from nuttx/include/sys/uio.h:45:
nuttx/include/sys/types.h:119:9: error: unknown type name '_size_t'
typedef _size_t      size_t;
        ^
nuttx/include/sys/types.h:120:9: error: unknown type name '_ssize_t'
typedef _ssize_t     ssize_t;
        ^
nuttx/include/sys/types.h:121:9: error: unknown type name '_size_t'
typedef _size_t      rsize_t;
        ^
nuttx/include/sys/types.h:174:9: error: unknown type name '_wchar_t'
typedef _wchar_t     wchar_t;
        ^
In file included from src/sx126x-nuttx.c:4:
In file included from nuttx/include/stdio.h:34:
nuttx/include/nuttx/fs/fs.h:238:20: error: use of undeclared identifier 'NAME_MAX'
  char      parent[NAME_MAX + 1];
                   ^

Which we fix this by including the right header files

#if defined(__NuttX__) && defined(__clang__)  //  Workaround for NuttX with zig cc
#include <arch/types.h>
#include "../../nuttx/include/limits.h"
#endif  //  defined(__NuttX__) && defined(__clang__)

Into these source files…

(See the changes)

Also we insert this code to tell us (at runtime) whether it was compiled with Zig Compiler or GCC…

void SX126xIoInit( void ) {
#ifdef __clang__
  //  For zig cc
  puts("SX126xIoInit: Compiled with zig cc");
#else
#warning Compiled with gcc
  //  For gcc
  puts("SX126xIoInit: Compiled with gcc");
#endif  //  __clang__

(Source)

We run the LoRaWAN Test App (compiled with GCC) that calls the LoRa SX1262 Library (compiled with “zig cc”)…

nsh> lorawan_test
SX126xIoInit: Compiled with zig cc
...
###### =========== MLME-Confirm ============ ######
STATUS      : OK
###### ===========   JOINED     ============ ######
OTAA
DevAddr     :  000E268C
DATA RATE   : DR_2
...
###### =========== MCPS-Confirm ============ ######
STATUS      : OK
###### =====   UPLINK FRAME        1   ===== ######
CLASS       : A
TX PORT     : 1
TX DATA     : UNCONFIRMED
48 69 20 4E 75 74 74 58 00
DATA RATE   : DR_3
U/L FREQ    : 923400000
TX POWER    : 0
CHANNEL MASK: 0003

(See the complete log)

This shows that the LoRa SX1262 Library compiled with “zig cc” works perfectly fine with NuttX!

Zig Compiler calls Clang to compile C code. But NuttX compiles with GCC. Won’t we have problems with code compatibility?

Apparently no problemo! The experiment above shows that “zig cc” (with Clang) is compatible with GCC (at least for BL602 NuttX).

(Just make sure that we pass the same Compiler Options to both compilers)

§17 Appendix: LoRaWAN Library for NuttX

In the previous section we took 3 source files (from LoRa SX1262 Library), compiled them with “zig cc” and linked them with Apache NuttX RTOS.

But will this work for larger NuttX Libraries?

Let’s attempt to compile the huge (and complicated) LoRaWAN Library with “zig cc”.

NuttX compiles the LoRaWAN Library like this…

##  LoRaWAN Source Directory
cd $HOME/nuttx/nuttx/libs/liblorawan

##  Compile mac/LoRaMac.c with GCC
riscv64-unknown-elf-gcc \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -march=rv32imafc \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/mac/LoRaMac.c \
  -o  src/mac/LoRaMac.o

We switch to the Zig Compiler…

##  LoRaWAN Source Directory
cd $HOME/nuttx/nuttx/libs/liblorawan

##  Compile mac/LoRaMac.c with zig cc
zig cc \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe   src/mac/LoRaMac.c \
  -o  src/mac/LoRaMac.o

##  Link Zig Object Files with NuttX after compiling with `zig cc`
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
cd $HOME/nuttx/nuttx
make

We include the right header files into LoRaMac.c

#if defined(__NuttX__) && defined(__clang__)  //  Workaround for NuttX with zig cc
#include <arch/types.h>
#include "../../nuttx/include/limits.h"
#endif  //  defined(__NuttX__) && defined(__clang__)

(See the changes)

The modified LoRaMac.c compiles without errors with “zig cc”.

Unfortunately we haven’t completed this experiment, because we have a long list of source files in the LoRaWAN Library to compile with “zig cc”.

Instead of rewriting the NuttX Makefile to call “zig cc”, we should probably compile with “build.zig” instead…

§18 Appendix: LoRaWAN App for NuttX

Thus far we have tested “zig cc” as the drop-in replacement for GCC in 2 NuttX Modules…

Let’s do one last test: We compile the LoRaWAN Test App (in C) with “zig cc”.

NuttX compiles the LoRaWAN App lorawan_test_main.c like this…

##  App Source Directory
cd $HOME/nuttx/apps/examples/lorawan_test/lorawan_test_main.c

##  Compile lorawan_test_main.c with GCC
riscv64-unknown-elf-gcc \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -march=rv32imafc \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe \
  -I "$HOME/nuttx/apps/graphics/lvgl" \
  -I "$HOME/nuttx/apps/graphics/lvgl/lvgl" \
  -I "$HOME/nuttx/apps/include" \
  -Dmain=lorawan_test_main  lorawan_test_main.c \
  -o  lorawan_test_main.c.home.user.nuttx.apps.examples.lorawan_test.o

We switch GCC to “zig cc”…

##  App Source Directory
cd $HOME/nuttx/apps/examples/lorawan_test

##  Compile lorawan_test_main.c with zig cc
zig cc \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -c \
  -fno-common \
  -Wall \
  -Wstrict-prototypes \
  -Wshadow \
  -Wundef \
  -Os \
  -fno-strict-aliasing \
  -fomit-frame-pointer \
  -fstack-protector-all \
  -ffunction-sections \
  -fdata-sections \
  -g \
  -mabi=ilp32f \
  -mno-relax \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -pipe \
  -I "$HOME/nuttx/apps/graphics/lvgl" \
  -I "$HOME/nuttx/apps/graphics/lvgl/lvgl" \
  -I "$HOME/nuttx/apps/include" \
  -Dmain=lorawan_test_main  lorawan_test_main.c \
  -o  *lorawan_test.o

##  Link Zig Object Files with NuttX after compiling with `zig cc`
##  TODO: Change "$HOME/nuttx" to your NuttX Project Directory
cd $HOME/nuttx/nuttx
make

As usual we include the right header files into lorawan_test_main.c

#if defined(__NuttX__) && defined(__clang__)  //  Workaround for NuttX with zig cc
#include <arch/types.h>
#include "../../nuttx/include/limits.h"
#endif  //  defined(__NuttX__) && defined(__clang__)

(See the changes)

When compiled with “zig cc”, the LoRaWAN App runs OK on NuttX yay!

nsh> lorawan_test
lorawan_test_main: Compiled with zig cc
...
###### =========== MLME-Confirm ============ ######
STATUS      : OK
###### ===========   JOINED     ============ ######
OTAA
DevAddr     :  00DC5ED5
DATA RATE   : DR_2
...
###### =========== MCPS-Confirm ============ ######
STATUS      : OK
###### =====   UPLINK FRAME        1   ===== ######
CLASS       : A
TX PORT     : 1
TX DATA     : UNCONFIRMED
48 69 20 4E 75 74 74 58 00
DATA RATE   : DR_3
U/L FREQ    : 923400000
TX POWER    : 0
CHANNEL MASK: 0003

(See the complete log)

§19 Appendix: Auto-Translate LoRaWAN App to Zig

The Zig Compiler can auto-translate C code to Zig. (See this)

Here’s how we auto-translate our LoRaWAN App lorawan_test_main.c from C to Zig…

Like this…

##  App Source Directory
cd $HOME/nuttx/apps/examples/lorawan_test

##  Auto-translate lorawan_test_main.c from C to Zig
zig translate-c \
  -target riscv32-freestanding-none \
  -mcpu=baseline_rv32-d \
  -cflags \
    -fno-common \
    -Wall \
    -Wstrict-prototypes \
    -Wshadow \
    -Wundef \
    -Os \
    -fno-strict-aliasing \
    -fomit-frame-pointer \
    -fstack-protector-all \
    -ffunction-sections \
    -fdata-sections \
    -g \
    -mabi=ilp32f \
    -mno-relax \
  -- \
  -isystem "$HOME/nuttx/nuttx/include" \
  -D__NuttX__ \
  -DNDEBUG \
  -DARCH_RISCV  \
  -I "$HOME/nuttx/apps/graphics/lvgl" \
  -I "$HOME/nuttx/apps/graphics/lvgl/lvgl" \
  -I "$HOME/nuttx/apps/include" \
  -Dmain=lorawan_test_main  \
  lorawan_test_main.c \
  >lorawan_test_main.zig

Here’s the original C code: lorawan_test_main.c

And the auto-translation from C to Zig: translated/lorawan_test_main.zig

Here’s a snippet from the original C code…

int main(int argc, FAR char *argv[]) {
#ifdef __clang__
    puts("lorawan_test_main: Compiled with zig cc");
#else
    puts("lorawan_test_main: Compiled with gcc");
#endif  //  __clang__

    //  If we are using Entropy Pool and the BL602 ADC is available,
    //  add the Internal Temperature Sensor data to the Entropy Pool
    init_entropy_pool();

    //  Compute the interval between transmissions based on Duty Cycle
    TxPeriodicity = APP_TX_DUTYCYCLE + randr( -APP_TX_DUTYCYCLE_RND, APP_TX_DUTYCYCLE_RND );

    const Version_t appVersion    = { .Value = FIRMWARE_VERSION };
    const Version_t gitHubVersion = { .Value = GITHUB_VERSION };
    DisplayAppInfo( "lorawan_test", 
                    &appVersion,
                    &gitHubVersion );

    //  Init LoRaWAN
    if ( LmHandlerInit( &LmHandlerCallbacks, &LmHandlerParams ) != LORAMAC_HANDLER_SUCCESS )
    {
        printf( "LoRaMac wasn't properly initialized\n" );
        //  Fatal error, endless loop.
        while ( 1 ) {}
    }

    // Set system maximum tolerated rx error in milliseconds
    LmHandlerSetSystemMaxRxError( 20 );

    // The LoRa-Alliance Compliance protocol package should always be initialized and activated.
    LmHandlerPackageRegister( PACKAGE_ID_COMPLIANCE, &LmhpComplianceParams );
    LmHandlerPackageRegister( PACKAGE_ID_CLOCK_SYNC, NULL );
    LmHandlerPackageRegister( PACKAGE_ID_REMOTE_MCAST_SETUP, NULL );
    LmHandlerPackageRegister( PACKAGE_ID_FRAGMENTATION, &FragmentationParams );

    IsClockSynched     = false;
    IsFileTransferDone = false;

    //  Join the LoRaWAN Network
    LmHandlerJoin( );

    //  Set the Transmit Timer
    StartTxProcess( LORAMAC_HANDLER_TX_ON_TIMER );

    //  Handle LoRaWAN Events
    handle_event_queue(NULL);  //  Never returns

    return 0;
}

(Source)

And the auto-translated Zig code…

pub export fn lorawan_test_main(arg_argc: c_int, arg_argv: [*c][*c]u8) c_int {
    var argc = arg_argc;
    _ = argc;
    var argv = arg_argv;
    _ = argv;
    _ = puts("lorawan_test_main: Compiled with zig cc");
    init_entropy_pool();
    TxPeriodicity = @bitCast(u32, @as(c_int, 40000) + randr(-@as(c_int, 5000), @as(c_int, 5000)));
    const appVersion: Version_t = Version_t{
        .Value = @bitCast(u32, @as(c_int, 16908288)),
    };
    const gitHubVersion: Version_t = Version_t{
        .Value = @bitCast(u32, @as(c_int, 83886080)),
    };
    DisplayAppInfo("lorawan_test", &appVersion, &gitHubVersion);
    if (LmHandlerInit(&LmHandlerCallbacks, &LmHandlerParams) != LORAMAC_HANDLER_SUCCESS) {
        _ = printf("LoRaMac wasn't properly initialized\n");
        while (true) {}
    }
    _ = LmHandlerSetSystemMaxRxError(@bitCast(u32, @as(c_int, 20)));
    _ = LmHandlerPackageRegister(@bitCast(u8, @truncate(i8, @as(c_int, 0))), @ptrCast(?*anyopaque, &LmhpComplianceParams));
    _ = LmHandlerPackageRegister(@bitCast(u8, @truncate(i8, @as(c_int, 1))), @intToPtr(?*anyopaque, @as(c_int, 0)));
    _ = LmHandlerPackageRegister(@bitCast(u8, @truncate(i8, @as(c_int, 2))), @intToPtr(?*anyopaque, @as(c_int, 0)));
    _ = LmHandlerPackageRegister(@bitCast(u8, @truncate(i8, @as(c_int, 3))), @ptrCast(?*anyopaque, &FragmentationParams));
    IsClockSynched = @as(c_int, 0) != 0;
    IsFileTransferDone = @as(c_int, 0) != 0;
    LmHandlerJoin();
    StartTxProcess(@bitCast(c_uint, LORAMAC_HANDLER_TX_ON_TIMER));
    handle_event_queue(@intToPtr(?*anyopaque, @as(c_int, 0)));
    return 0;
}

(Source)

Wow the code looks super verbose?

Yeah but the Auto-Translated Zig Code is a valuable reference!

We referred to the auto-translated code when we created the LoRaWAN Zig App for this article.

(Especially the tricky parts for Type Conversion and C Pointers)

We’ll see the auto-translated code in the upcoming sections…

§20 Appendix: Opaque Type Error

(Note: We observed this issue with Zig Compiler version 0.10.0, it might have been fixed in later versions of the compiler)

When we reference LmHandlerCallbacks in our LoRaWAN Zig App lorawan_test.zig

_ = &LmHandlerCallbacks;

Zig Compiler will show this Opaque Type Error

zig-cache/.../cimport.zig:1353:5: 
error: opaque types have unknown size and 
therefore cannot be directly embedded in unions
    Fields: struct_sInfoFields,
    ^
zig-cache/.../cimport.zig:1563:5: 
note: while checking this field
    PingSlot: PingSlotInfo_t,
    ^
zig-cache/.../cimport.zig:1579:5: 
note: while checking this field
    PingSlotInfo: MlmeReqPingSlotInfo_t,
    ^
zig-cache/.../cimport.zig:1585:5: 
note: while checking this field
    Req: union_uMlmeParam,
    ^
zig-cache/.../cimport.zig:2277:5: 
note: while checking this field
    OnMacMlmeRequest: ?fn (LoRaMacStatus_t, [*c]MlmeReq_t, TimerTime_t) callconv(.C) void,
    ^

Opaque Type Error is explained here…

Let’s trace through our Opaque Type Error, guided by the Auto-Translated Zig Code that we discussed earlier.

We start at the bottom with OnMacMlmeRequest

export fn OnMacMlmeRequest(
  status: c.LoRaMacStatus_t,
  mlmeReq: [*c]c.MlmeReq_t, 
  nextTxIn: c.TimerTime_t
) void {
  c.DisplayMacMlmeRequestUpdate(status, mlmeReq, nextTxIn);
}

(Source)

Our function OnMacMlmeRequest has a parameter of type MlmeReq_t, auto-imported by Zig Compiler as…

pub const MlmeReq_t = struct_sMlmeReq;

pub const struct_sMlmeReq = extern struct {
  Type: Mlme_t,
  Req: union_uMlmeParam,
  ReqReturn: RequestReturnParam_t,
};

(Source)

Which contains another auto-imported type union_uMlmeParam

pub const union_uMlmeParam = extern union {
  Join: MlmeReqJoin_t,
  TxCw: MlmeReqTxCw_t,
  PingSlotInfo: MlmeReqPingSlotInfo_t,
  DeriveMcKEKey: MlmeReqDeriveMcKEKey_t,
  DeriveMcSessionKeyPair: MlmeReqDeriveMcSessionKeyPair_t,
};

(Source)

Which contains an MlmeReqPingSlotInfo_t

pub const MlmeReqPingSlotInfo_t = struct_sMlmeReqPingSlotInfo;

pub const struct_sMlmeReqPingSlotInfo = extern struct {
  PingSlot: PingSlotInfo_t,
};

(Source)

Which contains a PingSlotInfo_t

pub const PingSlotInfo_t = union_uPingSlotInfo;

pub const union_uPingSlotInfo = extern union {
  Value: u8,
  Fields: struct_sInfoFields,
};

(Source)

Which contains a struct_sInfoFields

pub const struct_sInfoFields = 
  opaque {};

(Source)

But struct_sInfoFields is an Opaque Type… Its fields are not known by the Zig Compiler!

Why is that?

If we refer to the original C code…

typedef union uPingSlotInfo
{
  /*!
   * Parameter for byte access
   */
  uint8_t Value;
  /*!
   * Structure containing the parameters for the PingSlotInfoReq
   */
  struct sInfoFields
  {
    /*!
     * Periodicity = 0: ping slot every second
     * Periodicity = 7: ping slot every 128 seconds
     */
    uint8_t Periodicity     : 3;
    /*!
     * RFU
     */
    uint8_t RFU             : 5;
  }Fields;
}PingSlotInfo_t;

(Source)

We see that sInfoFields contains Bit Fields, that the Zig Compiler is unable to translate.

Let’s fix this error in the next section…

§21 Appendix: Fix Opaque Type

Earlier we saw that this fails to compile in our LoRaWAN Zig App lorawan_test.zig

_ = &LmHandlerCallbacks;

That’s because LmHandlerCallbacks references the auto-imported type MlmeReq_t, which contains Bit Fields and can’t be translated by the Zig Compiler.

Let’s convert MlmeReq_t to an Opaque Type, since we won’t access the fields anyway…

/// We use an Opaque Type to represent MLME Request, because it contains Bit Fields that can't be converted by Zig
const MlmeReq_t = opaque {};

(Source)

We convert the LmHandlerCallbacks Struct to use our Opaque Type MlmeReq_t

/// Handler Callbacks. Adapted from 
/// https://github.com/lupyuen/zig-bl602-nuttx/blob/main/translated/lorawan_test_main.zig#L2818-L2833
pub const LmHandlerCallbacks_t = extern struct {
  GetBatteryLevel: ?fn () callconv(.C) u8,
  GetTemperature: ?fn () callconv(.C) f32,
  GetRandomSeed: ?fn () callconv(.C) u32,
  OnMacProcess: ?fn () callconv(.C) void,
  OnNvmDataChange: ?fn (c.LmHandlerNvmContextStates_t, u16) callconv(.C) void,
  OnNetworkParametersChange: ?fn ([*c]c.CommissioningParams_t) callconv(.C) void,
  OnMacMcpsRequest: ?fn (c.LoRaMacStatus_t, [*c]c.McpsReq_t, c.TimerTime_t) callconv(.C) void,

  /// Changed `[*c]c.MlmeReq_t` to `*MlmeReq_t`
  OnMacMlmeRequest: ?fn (c.LoRaMacStatus_t, *MlmeReq_t, c.TimerTime_t) callconv(.C) void,

  OnJoinRequest: ?fn ([*c]c.LmHandlerJoinParams_t) callconv(.C) void,
  OnTxData: ?fn ([*c]c.LmHandlerTxParams_t) callconv(.C) void,
  OnRxData: ?fn ([*c]c.LmHandlerAppData_t, [*c]c.LmHandlerRxParams_t) callconv(.C) void,
  OnClassChange: ?fn (c.DeviceClass_t) callconv(.C) void,
  OnBeaconStatusChange: ?fn ([*c]c.LoRaMacHandlerBeaconParams_t) callconv(.C) void,
  OnSysTimeUpdate: ?fn (bool, i32) callconv(.C) void,
};

(Source)

We change all auto-imported MlmeReq_t references from…

[*c]c.MlmeReq_t

(C Pointer to MlmeReq_t)

To our Opaque Type…

*MlmeReq_t

(Zig Pointer to MlmeReq_t)

We also change all auto-imported LmHandlerCallbacks_t references from…

[*c]c.LmHandlerCallbacks_t

(C Pointer to LmHandlerCallbacks_t)

To our converted LmHandlerCallbacks_t

*LmHandlerCallbacks_t

(Zig Pointer to LmHandlerCallbacks_t)

Which means we need to import the affected LoRaWAN Functions ourselves…

/// Changed `[*c]c.MlmeReq_t` to `*MlmeReq_t`. Adapted from
/// https://github.com/lupyuen/zig-bl602-nuttx/blob/main/translated/lorawan_test_main.zig#L2905
extern fn DisplayMacMlmeRequestUpdate(
  status:   c.LoRaMacStatus_t, 
  mlmeReq:  *MlmeReq_t, 
  nextTxIn: c.TimerTime_t
) void;

/// Changed `[*c]c.LmHandlerCallbacks_t` to `*LmHandlerCallbacks_t`. Adapted from
/// https://github.com/lupyuen/zig-bl602-nuttx/blob/main/translated/lorawan_test_main.zig#L2835
extern fn LmHandlerInit(
  callbacks:     *LmHandlerCallbacks_t, 
  handlerParams: [*c]c.LmHandlerParams_t
) c.LmHandlerErrorStatus_t;

(Source)

After fixing the Opaque Type, Zig Compiler successfully compiles our LoRaWAN Test App lorawan_test.zig.

§22 Appendix: Macro Error

(Note: We observed this issue with Zig Compiler version 0.10.0, it might have been fixed in later versions of the compiler)

While compiling our LoRaWAN Test App lorawan_test.zig, we see this Macro Error

zig-cache/o/23409ceec9a6e6769c416fde1695882f/cimport.zig:2904:32: 
error: unable to translate macro: undefined identifier `LL`
pub const __INT64_C_SUFFIX__ = @compileError("unable to translate macro: undefined identifier `LL`"); 
// (no file):178:9

According to the Zig Docs, this means that the Zig Compiler failed to translate a C Macro…

So we define LL ourselves…

/// Import the LoRaWAN Library from C
const c = @cImport({
  // Workaround for "Unable to translate macro: undefined identifier `LL`"
  @cDefine("LL", "");

(Source)

LL is the “long long” suffix for C Constants, which is probably not needed when we import C Types and Functions into Zig.

Then Zig Compiler emits this error…

zig-cache/o/83fc6cf7a78f5781f258f156f891554b/cimport.zig:2940:26: 
error: unable to translate C expr: unexpected token '##'
pub const __int_c_join = @compileError("unable to translate C expr: unexpected token '##'"); 
// /home/user/zig-linux-x86_64-0.10.0-dev.2351+b64a1d5ab/lib/include/stdint.h:282:9

Which refers to this line in stdint.h

#define __int_c_join(a, b) a ## b

The __int_c_join Macro fails because the LL suffix is now blank and the ## Concatenation Operator fails.

We redefine the __int_c_join Macro without the ## Concatenation Operator…

/// Import the LoRaWAN Library from C
const c = @cImport({
  // Workaround for "Unable to translate macro: undefined identifier `LL`"
  @cDefine("LL", "");
  @cDefine("__int_c_join(a, b)", "a");  //  Bypass zig/lib/include/stdint.h

(Source)

Now Zig Compiler successfully compiles our LoRaWAN Test App lorawan_test.zig

§23 Appendix: Struct Initialisation Error

(Note: We observed this issue with Zig Compiler version 0.10.0, it might have been fixed in later versions of the compiler)

When we initialise the Timer Struct at startup…

/// Timer to handle the application data transmission duty cycle
var TxTimer: c.TimerEvent_t = 
  std.mem.zeroes(c.TimerEvent_t);

(Source)

Zig Compiler crashes with this error…

TODO buf_write_value_bytes maybe typethread 11512 panic:
Unable to dump stack trace: debug info stripped

So we initialise the Timer Struct in the Main Function instead…

/// Timer to handle the application data transmission duty cycle.
/// Init the timer in Main Function.
var TxTimer: c.TimerEvent_t = undefined;

/// Main Function
pub export fn lorawan_test_main(
  _argc: c_int, 
  _argv: [*]const [*]const u8
) c_int {
  // Init the Timer Struct at startup
  TxTimer = std.mem.zeroes(c.TimerEvent_t);

(Source)