NuttX RTOS for PinePhone: LVGL Terminal for NSH Shell

đź“ť 3 Feb 2023

LVGL Terminal App on PinePhone with Apache NuttX RTOS

Apache NuttX RTOS (Real-Time Operating System) now boots on Pine64 PinePhone and runs Touchscreen Apps!

Today we’ll look inside a Touchscreen App that will be useful for NuttX Developers… Our Terminal App for NSH Shell. (Pic above)

(Watch the Demo on YouTube)

What’s NSH Shell?

NuttShell (NSH) is the Command-Line Interface for NuttX. (Works like a Linux Shell)

Previously we needed a special Serial Cable to access NSH Shell on PinePhone…

Now we can run NSH Commands through the Touchscreen! (Pic above)

(Super helpful for testing new NuttX Features on PinePhone!)

Read on to find out how we…

And how we might simplify the LVGL coding with the Zig Programming Language.

What’s NuttX? Why run it on PinePhone?

If we’re new to NuttX, here’s a gentle intro…

Flow of LVGL Terminal for PinePhone on Apache NuttX RTOS

§1 LVGL Terminal for NuttX

Before we dive in, let’s walk through the internals of our LVGL Terminal App for NuttX (pic above)…

  1. We start the NSH Shell as a NuttX Task

    (Which will execute our NSH Commands)

  2. An NSH Command is entered through the LVGL Keyboard Widget

    (Which goes to the Input Text Area Widget)

    Let’s say we type this NSH Command…

    ls
    
  3. When the Enter Key is pressed, we send the NSH Command ls to the NSH Input Pipe

  4. Which delivers the NSH Command to the NSH Shell

  5. NSH Shell executes our NSH Command…

    nsh> ls
    
  6. NSH Shell produces some Text Output, which is pushed to the NSH Output Pipe

    nsh> ls
    dev/
    var/
    
  7. We run an LVGL Timer that periodically polls the NSH Output Pipe for Text Output

  8. When it detects the Text Output, the LVGL Timer reads the data…

    And renders the output in the Output Text Area Widget.

    nsh> ls
    dev/
    var/
    

It looks like this…

LVGL Terminal for PinePhone on Apache NuttX RTOS

Whoa that looks complicated!

Yeah. But we’ll explain everything in this article…

And eventually we’ll understand the Source Code…

We begin by starting the NSH Task and piping a command to NSH Shell…

Pipe a Command to NSH Shell

§2 Pipe a Command to NSH Shell

Our Terminal App needs to…

We’ll redirect the NSH Input and Output with NuttX Pipes.

(Which will work like Linux Pipes)

Let’s find out how…

§2.1 Create the Pipes

How will we create the NuttX Pipes?

This is how we create a NuttX Pipe for NSH Input: lvglterm.c

// Create the NuttX Pipe for NSH Input
int nsh_stdin[2];
int ret = pipe(nsh_stdin);

// Check for error
if (ret < 0) {
  _err("stdin pipe failed: %d\n", errno); return;
}

NSH Shell will receive NSH Commands through this Pipe.

Why two elements in nsh_stdin?

That’s because a NuttX Pipe has Two Endpoints (in and out)…

NuttX Pipes are Unidirectional… Don’t mix up the endpoints!

To remind ourselves, we define the Read and Write Endpoints like so…

// pipe[0] for reading, pipe[1] for writing
#define READ_PIPE  0
#define WRITE_PIPE 1

We do the same to create the NuttX Pipes for NSH Output and NSH Error…

// Create the NuttX Pipe for NSH Output
int nsh_stdout[2];
ret = pipe(nsh_stdout);
if (ret < 0) { _err("stdout pipe failed: %d\n", errno); return; }

// Create the NuttX Pipe for NSH Error
int nsh_stderr[2];
ret = pipe(nsh_stderr);
if (ret < 0) { _err("stderr pipe failed: %d\n", errno); return; }

There’s a reason why we call _err instead of printf, we’ll find out next…

§2.2 Connect the Pipes

How will we connect the pipes to NSH Shell?

In a while we’ll start the NuttX Task for NSH Shell. But before that, we need some plumbing to connect the NuttX Pipes.

First we close the streams for Standard Input, Output and Error: lvglterm.c

// Close stdin, stdout and stderr
close(0);
close(1);
close(2);

That’s because NSH Shell will inherit our Standard I/O streams later.

Next we redirect the Standard I/O streams to the NuttX Pipes that we’ve created earlier…

// Redirect stdin, stdout and stderr to our NuttX Pipes.
// READ_PIPE is 0, WRITE_PIPE is 1
dup2(nsh_stdin[READ_PIPE],   0);  // Redirect stdin
dup2(nsh_stdout[WRITE_PIPE], 1);  // Redirect stdout
dup2(nsh_stderr[WRITE_PIPE], 2);  // Redirect stderr

When we do this, Standard I/O will no longer work with the NuttX Console.

Instead, all Standard I/O will go to our NuttX Pipes.

So printf will no longer print to the NuttX Console?

Exactly! That’s why we call _err and _info in this article.

These functions are hardwired to the NuttX Console. They will continue to work after we have redirected the Standard I/O streams.

§2.3 Create the Task

Our plumbing is done, let’s start the NuttX Task for NSH Shell: lvglterm.c

// Task ID will be returned here
pid_t pid;

// No arguments for the NuttX Task.
// argv[0] is always the Task Path.
static char * const argv[] = { "nsh", NULL };

// Start a NuttX Task for NSH Shell
int ret = posix_spawn(
  &pid,   // Returned Task ID
  "nsh",  // NSH Path
  NULL,   // Inherit stdin, stdout and stderr
  NULL,   // Default spawn attributes
  argv,   // Arguments
  NULL    // No environment
);

// Check for error
if (ret < 0) { _err("posix_spawn failed: %d\n", errno); return; }

// For Debugging: Wait a while for NSH Shell to start
sleep(1);

(posix_spawn should be enabled in menuconfig)

NSH Shell inherits our Standard I/O streams, which we’ve redirected to our NuttX Pipes.

We’re ready to test this!

§2.4 Test the Pipes

Finally we add some Test Code to verify that everything works: lvgldemo.c

// Send a command to NSH stdin
const char cmd[] = "ls\r";
ret = write(   // Write to the stream...
  nsh_stdin[WRITE_PIPE],  // NSH stdin (WRITE_PIPE is 1)
  cmd,         // Data to be written
  sizeof(cmd)  // Number of bytes
);

// Wait a while for NSH Shell to execute our command
sleep(1);

The code above sends the ls command to NSH Shell, by writing to our NuttX Pipe for NSH Standard Input.

NSH Shell runs the command and generates the command output.

We read the output from NSH Shell…

// Read the output from NSH stdout.
// TODO: This will block if there's nothing to read.
static char buf[64];
ret = read(        // Read from the stream...
  nsh_stdout[READ_PIPE],  // NSH stdout (READ_PIPE is 0)
  buf,             // Buffer to be read
  sizeof(buf) - 1  // Buffer size (needs terminating null)
);

// Print the output
if (ret > 0) {
  buf[ret] = 0;
  _info("%s\n", buf);
}

And it works! Here’s the NSH Shell auto-running the ls command received via our NuttX Pipe…

NuttShell (NSH) NuttX-12.0.0
nsh> ls
/:
 dev/
 var/
nsh>

(See the Complete Log)

What about NSH Error Output?

Normally we do this to read the NSH Error Output…

// Warning: This will block!
#ifdef NOTUSED
  // Read the output from NSH stderr.
  // TODO: This will block if there's nothing to read.
  ret = read(        // Read from the stream...
    nsh_stderr[READ_PIPE],  // NSH stderr (READ_PIPE is 0)
    buf,             // Buffer to be read
    sizeof(buf) - 1  // Buffer size (needs terminating null)
  );

  // Print the output
  if (ret > 0) { buf[ret] = 0; _info("%s\n", buf); }
#endif

But there’s a problem…

Calling read() on nsh_stderr will block the execution if there’s no NSH Error Output ready to be read!

(Same for nsh_stdout)

Instead let’s check if there’s NSH Output ready to be read. We do this by calling poll()…

Poll for NSH Output

§3 Poll for NSH Output

In the previous section we started an NSH Shell that will execute NSH Commands that we pipe to it…

But there’s a problem: Calling read() on nsh_stdout will block if there’s no NSH Output to be read.

(We can’t block our LVGL App, since LVGL needs to handle User Interface Events periodically)

Solution: We call has_input to check if NSH Shell has data ready to be read, before we actually read the data: lvglterm.c

// If NSH stdout has data to be read...
if (has_input(nsh_stdout[READ_PIPE])) {

  // Read the data from NSH stdout
  static char buf[64];
  ret = read(
    nsh_stdout[READ_PIPE],
    buf,
    sizeof(buf) - 1
  );

  // Print the data
  if (ret > 0) { buf[ret] = 0; _info("%s\n", buf); }
}

(We do the same for nsh_stderr)

has_input calls poll() on nsh_stdout to check if NSH Shell has data ready to be read: lvglterm.c

// Return true if the File Descriptor has data to be read
static bool has_input(
  int fd  // File Descriptor to be checked
) {
  // Define the Poll Struct
  struct pollfd fdp;
  fdp.fd     = fd;      // File Descriptor to be checked
  fdp.events = POLLIN;  // Check for Input

  // Poll the File Descriptor for Input
  int ret = poll(
    &fdp,  // File Descriptors
    1,     // Number of File Descriptors
    0      // Poll Timeout (Milliseconds)
  );

Note that we set the Poll Timeout to 0.

Thus poll() returns immediately with the result, without blocking.

We decode the result of poll() like so…

  if (ret > 0) {
    // If Poll is OK and there's Input...
    if ((fdp.revents & POLLIN) != 0) {
      // Report that there's Input
      _info("has input: fd=%d\n", fd);
      return true;
    }

    // Else report No Input
    _info("no input: fd=%d\n", fd);
    return false;

  } else if (ret == 0) {
    // If Timeout, report No Input
    _info("timeout: fd=%d\n", fd);
    return false;

  } else if (ret < 0) {
    // Handle Error
    _err("poll failed: %d, fd=%d\n", ret, fd);
    return false;
  }

What happens when we run this?

If NSH Shell has data waiting to be read, has_input returns True…

has_input: has input: fd=8

And if there’s nothing waiting to be read, has_input returns False (due to timeout)…

has_input: timeout: fd=8

(See the Complete Log)

We’ve solved our problem of Blocking Reads from NSH Output, by polling for NSH Output!

This polling for NSH Output needs to be done in an LVGL Timer, here’s why…

Timer for LVGL Terminal

§4 Timer for LVGL Terminal

How will we poll for NSH Output and display it?

We started an NSH Shell that will execute NSH Commands that we pipe to it.

Now we need to periodically poll for NSH Output, and write the output to the LVGL display.

Every couple of milliseconds we…

We do this with an LVGL Timer that’s triggered every 100 milliseconds: lvglterm.c

// Create an LVGL Terminal that will let us
// interact with NuttX NSH Shell
static void create_terminal(void) {

  // Create an LVGL Timer to poll for output from NSH Shell
  lv_timer_t *timer = lv_timer_create(
    timer_callback,  // Callback Function
    100,             // Timer Period (Milliseconds)
    &user_data       // Callback Data
  );

(user_data is unused for now)

timer_callback is our Callback Function for the LVGL Timer.

Inside the callback, we poll for NSH Output, read the output and display it: lvglterm.c

// Callback Function for LVGL Timer
static void timer_callback(lv_timer_t *timer) {

  // If NSH stdout has data to be read...
  if (has_input(nsh_stdout[READ_PIPE])) {

    // Read the output from NSH stdout
    static char buf[64];
    int ret = read(
      nsh_stdout[READ_PIPE],
      buf,
      sizeof(buf) - 1
    );

    // Add to NSH Output Text Area
    if (ret > 0) {
      buf[ret] = 0;
      remove_escape_codes(buf, ret);
      lv_textarea_add_text(output, buf);
    }
  }

(We’ve seen has_input earlier)

(lv_textarea_add_text comes from LVGL)

We’ll talk about remove_escape_codes in a while.

How do we test this Timer Callback?

Without LVGL Widgets, testing the LVGL Timer Callback will be tricky. Here’s how we tested by manipulating the LVGL Timer…

Why poll for NSH Output? Why not run a Background Thread that will block on NSH Output?

Even if we ran a Background Thread that will block until NSH Output is available, we still need to write the NSH Output to an LVGL Widget for display.

But LVGL is NOT Thread-Safe. Thus we need a Mutex to lock the LVGL Widget, which gets messy.

For now, it’s simpler to run an LVGL Timer to poll for NSH Output.

Now that our Background Processing is ready, let’s render the LVGL Widgets for our terminal…

Render Terminal with LVGL Widgets

§5 Render Terminal with LVGL Widgets

How will we render the Terminal with LVGL?

Our Terminal will have 3 LVGL Widgets…

Like this…

LVGL Terminal App

This is how we create the 3 LVGL Widgets: lvglterm.c

// LVGL Text Area Widgets for NSH Input and Output
static lv_obj_t *input;
static lv_obj_t *output;

// Create the LVGL Widgets for the LVGL Terminal
static void create_widgets(void) {
  ...
  // Create an LVGL Text Area Widget for NSH Output
  output = lv_textarea_create(
    col  // Parent is LVGL Column Container
  );

(We’ll explain col)

In the code above, we begin by creating the LVGL Text Area Widget to display the NSH Output.

Next we create another LVGL Text Area Widget to show the NSH Input…

  // Create an LVGL Text Area Widget for NSH Input
  input = lv_textarea_create(
    col  // Parent is LVGL Column Container
  );

Then we create the LVGL Keyboard Widget to allow touchscreen entry of NSH Commands…

  // Create an LVGL Keyboard Widget
  lv_obj_t *kb = lv_keyboard_create(
    col  // Parent is LVGL Column Container
  );

  // No padding around the Keyboard Widget
  lv_obj_set_style_pad_all(kb, 0, 0);

We register a Callback Function for NSH Input, to detect the pressing of the Enter Key…

  // Register the Callback Function for NSH Input
  lv_obj_add_event_cb(
    input,  // LVGL Text Area Widget for NSH Input
    input_callback,  // Callback Function
    LV_EVENT_ALL,    // Callback for All Events
    NULL             // Callback Argument
  );

input_callback is the Callback Function for NSH Input. Which we’ll cover in a while.

Finally we set the Keyboard Widget to populate the NSH Input Text Area…

  // Set the Keyboard to populate the NSH Input Text Area
  lv_keyboard_set_textarea(
    kb,    // LVGL Keyboard Widget
    input  // LVGL Text Area Widget for NSH Input
  );
}

That’s how we create the 3 LVGL Widgets for our Terminal App!

What’s col?

col is an LVGL Column Container that contains our 3 LVGL Widgets: lvglterm.c

  // Create an LVGL Container with Column Flex Direction
  col = lv_obj_create(lv_scr_act());

  // Column is 100% of Screen Width, 100% of Screen Height
  // (720 x 1440 pixels)
  lv_obj_set_size(col, LV_PCT(100), LV_PCT(100));

  // Widgets in the column will have Flex Layout
  lv_obj_set_flex_flow(col, LV_FLEX_FLOW_COLUMN);

  // No padding around the column
  lv_obj_set_style_pad_all(col, 0, 0);

We’re using LVGL Flex Layout so that the LVGL Widgets will be Auto-Positioned, based on the Screen Size of our device.

(Hence the code will render correctly on devices other than PinePhone)

Remember we have 3 LVGL Widgets: NSH Output, NSH Input and Keyboard. NSH Input and Keyboard have Fixed Height.

But NSH Output has Flexible Height and will fill the vertical space in the column…

  // Create an LVGL Text Area Widget for NSH Output
  output = lv_textarea_create(col);

  // Width of NSH Output is 100% of Column Width
  lv_obj_set_width(output, LV_PCT(100));

  // Height of NSH Output will fill the Column Height
  lv_obj_set_flex_grow(output, 1);

NSH Input and Keyboard have Fixed Height…

  // Create an LVGL Text Area Widget for NSH Input
  input = lv_textarea_create(col);

  // Width of NSH Input is 100% of Column Width.
  // Height is fixed. (One line of text)
  lv_obj_set_size(input, LV_PCT(100), LV_SIZE_CONTENT);

  // Create an LVGL Keyboard Widget.
  // Use Fixed Width and Height.
  kb = lv_keyboard_create(col);

Thus we have arranged our LVGL Widgets neatly, to fit any Screen Size!

Note that we’re using the LVGL Default Font for all 3 LVGL Widgets. Which has a problem…

Set Default Font to Monospace

§6 Set Terminal Font to Monospace

Like any Terminal App, our LVGL Terminal looks nicer with a Monospaced Font. (Instead of a Proportional Font)

So we change the Default LVGL Font to a Monospaced Font?

But watch what happens if we change the LVGL Default Font from Montserrat 20 (Proportional) to UNSCII 16 (Monospaced)…

The LVGL Keyboard has missing symbols! Enter, Backspace, …

The symbols are undefined in the UNSCII 16 Font. (Pic above)

Thus we set the LVGL Default Font back to Montserrat 20.

And instead we set the Font Style for NSH Input and Output to UNSCII 16: lvglterm.c

// Set the Font Style for NSH Input and Output
// to a Monospaced Font: UNSCII 16
static lv_style_t terminal_style;
lv_style_init(&terminal_style);
lv_style_set_text_font(&terminal_style, &lv_font_unscii_16);

// Create an LVGL Text Area Widget for NSH Output
output = lv_textarea_create(lv_scr_act());
// Set the Font Style for NSH Output
lv_obj_add_style(output, &terminal_style, 0);
...

// Create an LVGL Text Area Widget for NSH Input
input = lv_textarea_create(lv_scr_act());
// Set the Font Style for NSH Input
lv_obj_add_style(input, &terminal_style, 0);
...

Now we see the LVGL Keyboard without missing symbols (when rendered with Montserrat 20)…

Set Terminal Font to Monospace

Let’s look at our Callback Function for NSH Input…

Handle Input from LVGL Keyboard

§7 Handle Input from LVGL Keyboard

How will we check if the Enter Key has been pressed?

Remember earlier we registered a Callback Function for NSH Input Text Area, to detect the pressing of the Enter Key: lvglterm.c

// Register the Callback Function for NSH Input
lv_obj_add_event_cb(
  input,  // LVGL Text Area Widget for NSH Input
  input_callback,  // Callback Function
  LV_EVENT_ALL,    // Callback for All Events
  kb               // Callback Argument (Keyboard)
);

input_callback is the Callback Function for NSH Input.

It waits for the Enter Key to be pressed, then it sends the typed command to NSH Shell via the NuttX Pipe: lvglterm.c

// Callback Function for NSH Input Text Area
static void input_callback(lv_event_t *e) {

  // Decode the LVGL Event
  const lv_event_code_t code = lv_event_get_code(e);

  // If NSH Input Text Area has been updated...
  if (code == LV_EVENT_VALUE_CHANGED) {

    // Get the Keyboard Widget from the LVGL Event
    const lv_obj_t *kb = lv_event_get_user_data(e);

    // Get the Button Index of the Key Pressed
    const uint16_t id = lv_keyboard_get_selected_btn(kb);

    // Get the Text of the Key Pressed
    const char *key = lv_keyboard_get_btn_text(kb, id);

The Enter Key has a Text Label of EF A2 A2.

We match the Key Pressed with that label…

    // If Key Pressed is Enter...
    if (key[0] == 0xef && key[1] == 0xa2 && key[2] == 0xa2) {

      // Read the NSH Input
      const char *cmd = lv_textarea_get_text(input);
      if (cmd == NULL || cmd[0] == 0) { return; }

      // Send the Command to NSH stdin
      int ret = write(
        nsh_stdin[WRITE_PIPE],
        cmd,
        strlen(cmd)
      );

      // Erase the NSH Input
      lv_textarea_set_text(input, "");
    }
  }
}

And we send the NSH Input Text to the NSH Shell for execution.

(The NSH Input Text is already terminated by a Newline Character 0x0A, which works fine with NSH Shell)

The command runs in NSH Shell and produces some Output Text. Which is handled by our LVGL Timer Callback Function…

Timer for LVGL Terminal

§8 Handle Output from NSH Shell

Earlier we’ve created an LVGL Timer that polls periodically for output generated by NSH Shell…

If it detects NSH Output, the LVGL Timer Callback Function writes the output to the NSH Output Text Area: lvglterm.c

// Callback Function for LVGL Timer
static void timer_callback(lv_timer_t *timer) {

  // If NSH stdout has data to be read...
  if (has_input(nsh_stdout[READ_PIPE])) {

    // Read the output from NSH stdout
    static char buf[64];
    int ret = read(
      nsh_stdout[READ_PIPE],
      buf,
      sizeof(buf) - 1
    );

    // Add to NSH Output Text Area
    if (ret > 0) {
      buf[ret] = 0;
      remove_escape_codes(buf, ret);
      lv_textarea_add_text(output, buf);
    }
  }

(We’ve seen has_input earlier)

(lv_textarea_add_text comes from LVGL)

What’s remove_escape_codes?

NSH is configured to work with VT100 and ANSI Terminals. So the NSH Output will contain ANSI Escape Codes like…

nsh> <ESC>[K

(Source)

Which is the ANSI Command for “Erase In Line”. (Clear to the end of line)

remove_escape_codes searches for Escape Codes in the NSH Output and removes them.

But the NSH Output looks laggy…

Yeah we used an LVGL Text Area to render the NSH Output. Which is probably not optimal for scrollable, static text.

An LVGL Label Widget might work better.

(Remember to set LV_LABEL_LONG_TXT_HINT)

Also we might have to change polling to Multithreaded Blocking.

Which means we need Mutexes to lock the LVGL Widgets.

LVGL Programming in Zig

§9 LVGL Programming in Zig

As we add more features to our LVGL Terminal…

Will it become too complex to extend and maintain?

It might! That’s why we should consider coding our LVGL Terminal App in the Zig Programming Language.

LVGL works with Zig?

Yep! The pic above shows an LVGL Program in Zig that runs OK with PinePhone and NuttX.

To simpify the code, we created a simple LVGL Wrapper in Zig that makes LVGL feel more “object-oriented”.

(More about this)

Compiling an LVGL Program in Zig

How will we compile our LVGL Program in Zig?

Do we need to manually import every single LVGL Function from C?

Zig Compiler automatically imports all LVGL Functions from C into Zig!

The pic above shows how we compile our LVGL Program in Zig. Which will auto-import all LVGL Functions from the C Header Files.

(More about this)

Sorry ChatGPT… Please try harder

Sorry ChatGPT… Please try harder

§10 What’s Next

Now we can run NuttX Console Apps on PinePhone, without a Serial Cable!

I hope this will make it a little easier to experiment with NuttX on PinePhone. Lemme know what you’re building!

Meanwhile please check out the other articles on NuttX for PinePhone…

Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/terminal.md

§11 Notes

  1. NSH Architecture-Specific Initialization CONFIG_NSH_ARCHINIT should be disabled…

    Application Configuration
      > NSH Library 
        > Have architecture-specific initialization
    

    That’s because LVGL Terminal starts a new task for NSH, which will redo the Architecture-Specific Initialization if CONFIG_NSH_ARCHINIT is enabled.

    LVGL Terminal will handle the initialization when CONFIG_NSH_ARCHINIT is disabled.

  2. On Arm64 Platforms: This (harmless) warning appears when starting the NSH Task…

    mkfatfs: command not found
    

    This will be fixed in a future patch.