đź“ť 3 Feb 2023
Apache NuttX RTOS (Real-Time Operating System) now boots on Pine64 PinePhone and runs Touchscreen Apps!
Today we’ll look inside a Touchscreen App that will be useful for NuttX Developers… Our Terminal App for NSH Shell. (Pic above)
What’s NSH Shell?
NuttShell (NSH) is the Command-Line Interface for NuttX. (Works like a Linux Shell)
Previously we needed a special Serial Cable to access NSH Shell on PinePhone…
Now we can run NSH Commands through the Touchscreen! (Pic above)
(Super helpful for testing new NuttX Features on PinePhone!)
Read on to find out how we…
Pipe a Command to NSH Shell
Poll for NSH Output
Render the Terminal with LVGL Widgets
Handle Input from LVGL Keyboard
Handle Output from NSH Shell
And how we might simplify the LVGL coding with the Zig Programming Language.
What’s NuttX? Why run it on PinePhone?
If we’re new to NuttX, here’s a gentle intro…
Before we dive in, let’s walk through the internals of our LVGL Terminal App for NuttX (pic above)…
We start the NSH Shell as a NuttX Task
(Which will execute our NSH Commands)
An NSH Command is entered through the LVGL Keyboard Widget
(Which goes to the Input Text Area Widget)
Let’s say we type this NSH Command…
ls
When the Enter Key is pressed, we send the NSH Command ls to the NSH Input Pipe
Which delivers the NSH Command to the NSH Shell
NSH Shell executes our NSH Command…
nsh> ls
NSH Shell produces some Text Output, which is pushed to the NSH Output Pipe
nsh> ls
dev/
var/
We run an LVGL Timer that periodically polls the NSH Output Pipe for Text Output
When it detects the Text Output, the LVGL Timer reads the data…
And renders the output in the Output Text Area Widget.
nsh> ls
dev/
var/
It looks like this…
Whoa that looks complicated!
Yeah. But we’ll explain everything in this article…
How we start a NuttX Task
What are NuttX Pipes and how we use them
How we render LVGL Widgets and handle events
And eventually we’ll understand the Source Code…
We begin by starting the NSH Task and piping a command to NSH Shell…
Our Terminal App needs to…
Start the NuttX Task for NSH Shell
(Which will execute our NSH Commands)
Redirect the NSH Shell Input
(To receive the NSH Commands that we typed)
Redirect the NSH Shell Output
(To render the output of NSH Commands)
We’ll redirect the NSH Input and Output with NuttX Pipes.
(Which will work like Linux Pipes)
Let’s find out how…
How will we create the NuttX Pipes?
This is how we create a NuttX Pipe for NSH Input: lvglterm.c
// Create the NuttX Pipe for NSH Input
int nsh_stdin[2];
int ret = pipe(nsh_stdin);
// Check for error
if (ret < 0) {
_err("stdin pipe failed: %d\n", errno); return;
}
NSH Shell will receive NSH Commands through this Pipe.
Why two elements in nsh_stdin?
That’s because a NuttX Pipe has Two Endpoints (in and out)…
nsh_stdin[0] reads from the Pipe
nsh_stdin[1] writes to the Pipe
NuttX Pipes are Unidirectional… Don’t mix up the endpoints!
To remind ourselves, we define the Read and Write Endpoints like so…
// pipe[0] for reading, pipe[1] for writing
#define READ_PIPE 0
#define WRITE_PIPE 1
We do the same to create the NuttX Pipes for NSH Output and NSH Error…
// Create the NuttX Pipe for NSH Output
int nsh_stdout[2];
ret = pipe(nsh_stdout);
if (ret < 0) { _err("stdout pipe failed: %d\n", errno); return; }
// Create the NuttX Pipe for NSH Error
int nsh_stderr[2];
ret = pipe(nsh_stderr);
if (ret < 0) { _err("stderr pipe failed: %d\n", errno); return; }
There’s a reason why we call _err instead of printf, we’ll find out next…
How will we connect the pipes to NSH Shell?
In a while we’ll start the NuttX Task for NSH Shell. But before that, we need some plumbing to connect the NuttX Pipes.
First we close the streams for Standard Input, Output and Error: lvglterm.c
// Close stdin, stdout and stderr
close(0);
close(1);
close(2);
That’s because NSH Shell will inherit our Standard I/O streams later.
Next we redirect the Standard I/O streams to the NuttX Pipes that we’ve created earlier…
// Redirect stdin, stdout and stderr to our NuttX Pipes.
// READ_PIPE is 0, WRITE_PIPE is 1
dup2(nsh_stdin[READ_PIPE], 0); // Redirect stdin
dup2(nsh_stdout[WRITE_PIPE], 1); // Redirect stdout
dup2(nsh_stderr[WRITE_PIPE], 2); // Redirect stderr
When we do this, Standard I/O will no longer work with the NuttX Console.
Instead, all Standard I/O will go to our NuttX Pipes.
So printf will no longer print to the NuttX Console?
Exactly! That’s why we call _err and _info in this article.
These functions are hardwired to the NuttX Console. They will continue to work after we have redirected the Standard I/O streams.
Our plumbing is done, let’s start the NuttX Task for NSH Shell: lvglterm.c
// Task ID will be returned here
pid_t pid;
// No arguments for the NuttX Task.
// argv[0] is always the Task Path.
static char * const argv[] = { "nsh", NULL };
// Start a NuttX Task for NSH Shell
int ret = posix_spawn(
&pid, // Returned Task ID
"nsh", // NSH Path
NULL, // Inherit stdin, stdout and stderr
NULL, // Default spawn attributes
argv, // Arguments
NULL // No environment
);
// Check for error
if (ret < 0) { _err("posix_spawn failed: %d\n", errno); return; }
// For Debugging: Wait a while for NSH Shell to start
sleep(1);
(posix_spawn should be enabled in menuconfig)
NSH Shell inherits our Standard I/O streams, which we’ve redirected to our NuttX Pipes.
We’re ready to test this!
Finally we add some Test Code to verify that everything works: lvgldemo.c
// Send a command to NSH stdin
const char cmd[] = "ls\r";
ret = write( // Write to the stream...
nsh_stdin[WRITE_PIPE], // NSH stdin (WRITE_PIPE is 1)
cmd, // Data to be written
sizeof(cmd) // Number of bytes
);
// Wait a while for NSH Shell to execute our command
sleep(1);
The code above sends the ls command to NSH Shell, by writing to our NuttX Pipe for NSH Standard Input.
NSH Shell runs the command and generates the command output.
We read the output from NSH Shell…
// Read the output from NSH stdout.
// TODO: This will block if there's nothing to read.
static char buf[64];
ret = read( // Read from the stream...
nsh_stdout[READ_PIPE], // NSH stdout (READ_PIPE is 0)
buf, // Buffer to be read
sizeof(buf) - 1 // Buffer size (needs terminating null)
);
// Print the output
if (ret > 0) {
buf[ret] = 0;
_info("%s\n", buf);
}
And it works! Here’s the NSH Shell auto-running the ls command received via our NuttX Pipe…
NuttShell (NSH) NuttX-12.0.0
nsh> ls
/:
dev/
var/
nsh>
What about NSH Error Output?
Normally we do this to read the NSH Error Output…
// Warning: This will block!
#ifdef NOTUSED
// Read the output from NSH stderr.
// TODO: This will block if there's nothing to read.
ret = read( // Read from the stream...
nsh_stderr[READ_PIPE], // NSH stderr (READ_PIPE is 0)
buf, // Buffer to be read
sizeof(buf) - 1 // Buffer size (needs terminating null)
);
// Print the output
if (ret > 0) { buf[ret] = 0; _info("%s\n", buf); }
#endif
But there’s a problem…
Calling read() on nsh_stderr will block the execution if there’s no NSH Error Output ready to be read!
(Same for nsh_stdout)
Instead let’s check if there’s NSH Output ready to be read. We do this by calling poll()…
In the previous section we started an NSH Shell that will execute NSH Commands that we pipe to it…
But there’s a problem: Calling read() on nsh_stdout will block if there’s no NSH Output to be read.
(We can’t block our LVGL App, since LVGL needs to handle User Interface Events periodically)
Solution: We call has_input to check if NSH Shell has data ready to be read, before we actually read the data: lvglterm.c
// If NSH stdout has data to be read...
if (has_input(nsh_stdout[READ_PIPE])) {
// Read the data from NSH stdout
static char buf[64];
ret = read(
nsh_stdout[READ_PIPE],
buf,
sizeof(buf) - 1
);
// Print the data
if (ret > 0) { buf[ret] = 0; _info("%s\n", buf); }
}
(We do the same for nsh_stderr)
has_input calls poll() on nsh_stdout to check if NSH Shell has data ready to be read: lvglterm.c
// Return true if the File Descriptor has data to be read
static bool has_input(
int fd // File Descriptor to be checked
) {
// Define the Poll Struct
struct pollfd fdp;
fdp.fd = fd; // File Descriptor to be checked
fdp.events = POLLIN; // Check for Input
// Poll the File Descriptor for Input
int ret = poll(
&fdp, // File Descriptors
1, // Number of File Descriptors
0 // Poll Timeout (Milliseconds)
);
Note that we set the Poll Timeout to 0.
Thus poll() returns immediately with the result, without blocking.
We decode the result of poll() like so…
if (ret > 0) {
// If Poll is OK and there's Input...
if ((fdp.revents & POLLIN) != 0) {
// Report that there's Input
_info("has input: fd=%d\n", fd);
return true;
}
// Else report No Input
_info("no input: fd=%d\n", fd);
return false;
} else if (ret == 0) {
// If Timeout, report No Input
_info("timeout: fd=%d\n", fd);
return false;
} else if (ret < 0) {
// Handle Error
_err("poll failed: %d, fd=%d\n", ret, fd);
return false;
}
What happens when we run this?
If NSH Shell has data waiting to be read, has_input returns True…
has_input: has input: fd=8
And if there’s nothing waiting to be read, has_input returns False (due to timeout)…
has_input: timeout: fd=8
We’ve solved our problem of Blocking Reads from NSH Output, by polling for NSH Output!
This polling for NSH Output needs to be done in an LVGL Timer, here’s why…
How will we poll for NSH Output and display it?
We started an NSH Shell that will execute NSH Commands that we pipe to it.
Now we need to periodically poll for NSH Output, and write the output to the LVGL display.
Every couple of milliseconds we…
Poll the NSH Shell to check if it has output data
Read the output from NSH Shell
Display the output in an LVGL Widget
We do this with an LVGL Timer that’s triggered every 100 milliseconds: lvglterm.c
// Create an LVGL Terminal that will let us
// interact with NuttX NSH Shell
static void create_terminal(void) {
// Create an LVGL Timer to poll for output from NSH Shell
lv_timer_t *timer = lv_timer_create(
timer_callback, // Callback Function
100, // Timer Period (Milliseconds)
&user_data // Callback Data
);
(user_data is unused for now)
timer_callback is our Callback Function for the LVGL Timer.
Inside the callback, we poll for NSH Output, read the output and display it: lvglterm.c
// Callback Function for LVGL Timer
static void timer_callback(lv_timer_t *timer) {
// If NSH stdout has data to be read...
if (has_input(nsh_stdout[READ_PIPE])) {
// Read the output from NSH stdout
static char buf[64];
int ret = read(
nsh_stdout[READ_PIPE],
buf,
sizeof(buf) - 1
);
// Add to NSH Output Text Area
if (ret > 0) {
buf[ret] = 0;
remove_escape_codes(buf, ret);
lv_textarea_add_text(output, buf);
}
}
(We’ve seen has_input earlier)
(lv_textarea_add_text comes from LVGL)
We’ll talk about remove_escape_codes in a while.
How do we test this Timer Callback?
Without LVGL Widgets, testing the LVGL Timer Callback will be tricky. Here’s how we tested by manipulating the LVGL Timer…
Why poll for NSH Output? Why not run a Background Thread that will block on NSH Output?
Even if we ran a Background Thread that will block until NSH Output is available, we still need to write the NSH Output to an LVGL Widget for display.
But LVGL is NOT Thread-Safe. Thus we need a Mutex to lock the LVGL Widget, which gets messy.
For now, it’s simpler to run an LVGL Timer to poll for NSH Output.
Now that our Background Processing is ready, let’s render the LVGL Widgets for our terminal…
How will we render the Terminal with LVGL?
Our Terminal will have 3 LVGL Widgets…
LVGL Text Area Widget that shows the NSH Output
(At the top)
LVGL Text Area Widget for NSH Input, to enter commands
(At the middle)
LVGL Keyboard Widget for typing commands into NSH Input
(At the bottom)
Like this…
This is how we create the 3 LVGL Widgets: lvglterm.c
// LVGL Text Area Widgets for NSH Input and Output
static lv_obj_t *input;
static lv_obj_t *output;
// Create the LVGL Widgets for the LVGL Terminal
static void create_widgets(void) {
...
// Create an LVGL Text Area Widget for NSH Output
output = lv_textarea_create(
col // Parent is LVGL Column Container
);
(We’ll explain col)
In the code above, we begin by creating the LVGL Text Area Widget to display the NSH Output.
Next we create another LVGL Text Area Widget to show the NSH Input…
// Create an LVGL Text Area Widget for NSH Input
input = lv_textarea_create(
col // Parent is LVGL Column Container
);
Then we create the LVGL Keyboard Widget to allow touchscreen entry of NSH Commands…
// Create an LVGL Keyboard Widget
lv_obj_t *kb = lv_keyboard_create(
col // Parent is LVGL Column Container
);
// No padding around the Keyboard Widget
lv_obj_set_style_pad_all(kb, 0, 0);
We register a Callback Function for NSH Input, to detect the pressing of the Enter Key…
// Register the Callback Function for NSH Input
lv_obj_add_event_cb(
input, // LVGL Text Area Widget for NSH Input
input_callback, // Callback Function
LV_EVENT_ALL, // Callback for All Events
NULL // Callback Argument
);
input_callback is the Callback Function for NSH Input. Which we’ll cover in a while.
Finally we set the Keyboard Widget to populate the NSH Input Text Area…
// Set the Keyboard to populate the NSH Input Text Area
lv_keyboard_set_textarea(
kb, // LVGL Keyboard Widget
input // LVGL Text Area Widget for NSH Input
);
}
That’s how we create the 3 LVGL Widgets for our Terminal App!
What’s col?
col is an LVGL Column Container that contains our 3 LVGL Widgets: lvglterm.c
// Create an LVGL Container with Column Flex Direction
col = lv_obj_create(lv_scr_act());
// Column is 100% of Screen Width, 100% of Screen Height
// (720 x 1440 pixels)
lv_obj_set_size(col, LV_PCT(100), LV_PCT(100));
// Widgets in the column will have Flex Layout
lv_obj_set_flex_flow(col, LV_FLEX_FLOW_COLUMN);
// No padding around the column
lv_obj_set_style_pad_all(col, 0, 0);
We’re using LVGL Flex Layout so that the LVGL Widgets will be Auto-Positioned, based on the Screen Size of our device.
(Hence the code will render correctly on devices other than PinePhone)
Remember we have 3 LVGL Widgets: NSH Output, NSH Input and Keyboard. NSH Input and Keyboard have Fixed Height.
But NSH Output has Flexible Height and will fill the vertical space in the column…
// Create an LVGL Text Area Widget for NSH Output
output = lv_textarea_create(col);
// Width of NSH Output is 100% of Column Width
lv_obj_set_width(output, LV_PCT(100));
// Height of NSH Output will fill the Column Height
lv_obj_set_flex_grow(output, 1);
NSH Input and Keyboard have Fixed Height…
// Create an LVGL Text Area Widget for NSH Input
input = lv_textarea_create(col);
// Width of NSH Input is 100% of Column Width.
// Height is fixed. (One line of text)
lv_obj_set_size(input, LV_PCT(100), LV_SIZE_CONTENT);
// Create an LVGL Keyboard Widget.
// Use Fixed Width and Height.
kb = lv_keyboard_create(col);
Thus we have arranged our LVGL Widgets neatly, to fit any Screen Size!
Note that we’re using the LVGL Default Font for all 3 LVGL Widgets. Which has a problem…
Like any Terminal App, our LVGL Terminal looks nicer with a Monospaced Font. (Instead of a Proportional Font)
So we change the Default LVGL Font to a Monospaced Font?
But watch what happens if we change the LVGL Default Font from Montserrat 20 (Proportional) to UNSCII 16 (Monospaced)…
The LVGL Keyboard has missing symbols! Enter, Backspace, …
The symbols are undefined in the UNSCII 16 Font. (Pic above)
Thus we set the LVGL Default Font back to Montserrat 20.
And instead we set the Font Style for NSH Input and Output to UNSCII 16: lvglterm.c
// Set the Font Style for NSH Input and Output
// to a Monospaced Font: UNSCII 16
static lv_style_t terminal_style;
lv_style_init(&terminal_style);
lv_style_set_text_font(&terminal_style, &lv_font_unscii_16);
// Create an LVGL Text Area Widget for NSH Output
output = lv_textarea_create(lv_scr_act());
// Set the Font Style for NSH Output
lv_obj_add_style(output, &terminal_style, 0);
...
// Create an LVGL Text Area Widget for NSH Input
input = lv_textarea_create(lv_scr_act());
// Set the Font Style for NSH Input
lv_obj_add_style(input, &terminal_style, 0);
...
Now we see the LVGL Keyboard without missing symbols (when rendered with Montserrat 20)…
Let’s look at our Callback Function for NSH Input…
How will we check if the Enter Key has been pressed?
Remember earlier we registered a Callback Function for NSH Input Text Area, to detect the pressing of the Enter Key: lvglterm.c
// Register the Callback Function for NSH Input
lv_obj_add_event_cb(
input, // LVGL Text Area Widget for NSH Input
input_callback, // Callback Function
LV_EVENT_ALL, // Callback for All Events
kb // Callback Argument (Keyboard)
);
input_callback is the Callback Function for NSH Input.
It waits for the Enter Key to be pressed, then it sends the typed command to NSH Shell via the NuttX Pipe: lvglterm.c
// Callback Function for NSH Input Text Area
static void input_callback(lv_event_t *e) {
// Decode the LVGL Event
const lv_event_code_t code = lv_event_get_code(e);
// If NSH Input Text Area has been updated...
if (code == LV_EVENT_VALUE_CHANGED) {
// Get the Keyboard Widget from the LVGL Event
const lv_obj_t *kb = lv_event_get_user_data(e);
// Get the Button Index of the Key Pressed
const uint16_t id = lv_keyboard_get_selected_btn(kb);
// Get the Text of the Key Pressed
const char *key = lv_keyboard_get_btn_text(kb, id);
The Enter Key has a Text Label of EF A2 A2.
We match the Key Pressed with that label…
// If Key Pressed is Enter...
if (key[0] == 0xef && key[1] == 0xa2 && key[2] == 0xa2) {
// Read the NSH Input
const char *cmd = lv_textarea_get_text(input);
if (cmd == NULL || cmd[0] == 0) { return; }
// Send the Command to NSH stdin
int ret = write(
nsh_stdin[WRITE_PIPE],
cmd,
strlen(cmd)
);
// Erase the NSH Input
lv_textarea_set_text(input, "");
}
}
}
And we send the NSH Input Text to the NSH Shell for execution.
(The NSH Input Text is already terminated by a Newline Character 0x0A, which works fine with NSH Shell)
The command runs in NSH Shell and produces some Output Text. Which is handled by our LVGL Timer Callback Function…
Earlier we’ve created an LVGL Timer that polls periodically for output generated by NSH Shell…
If it detects NSH Output, the LVGL Timer Callback Function writes the output to the NSH Output Text Area: lvglterm.c
// Callback Function for LVGL Timer
static void timer_callback(lv_timer_t *timer) {
// If NSH stdout has data to be read...
if (has_input(nsh_stdout[READ_PIPE])) {
// Read the output from NSH stdout
static char buf[64];
int ret = read(
nsh_stdout[READ_PIPE],
buf,
sizeof(buf) - 1
);
// Add to NSH Output Text Area
if (ret > 0) {
buf[ret] = 0;
remove_escape_codes(buf, ret);
lv_textarea_add_text(output, buf);
}
}
(We’ve seen has_input earlier)
(lv_textarea_add_text comes from LVGL)
What’s remove_escape_codes?
NSH is configured to work with VT100 and ANSI Terminals. So the NSH Output will contain ANSI Escape Codes like…
nsh> <ESC>[K
Which is the ANSI Command for “Erase In Line”. (Clear to the end of line)
remove_escape_codes searches for Escape Codes in the NSH Output and removes them.
But the NSH Output looks laggy…
Yeah we used an LVGL Text Area to render the NSH Output. Which is probably not optimal for scrollable, static text.
An LVGL Label Widget might work better.
(Remember to set LV_LABEL_LONG_TXT_HINT)
Also we might have to change polling to Multithreaded Blocking.
Which means we need Mutexes to lock the LVGL Widgets.
As we add more features to our LVGL Terminal…
Will it become too complex to extend and maintain?
It might! That’s why we should consider coding our LVGL Terminal App in the Zig Programming Language.
LVGL works with Zig?
Yep! The pic above shows an LVGL Program in Zig that runs OK with PinePhone and NuttX.
To simpify the code, we created a simple LVGL Wrapper in Zig that makes LVGL feel more “object-oriented”.
How will we compile our LVGL Program in Zig?
Do we need to manually import every single LVGL Function from C?
Zig Compiler automatically imports all LVGL Functions from C into Zig!
The pic above shows how we compile our LVGL Program in Zig. Which will auto-import all LVGL Functions from the C Header Files.
Sorry ChatGPT… Please try harder
Now we can run NuttX Console Apps on PinePhone, without a Serial Cable!
I hope this will make it a little easier to experiment with NuttX on PinePhone. Lemme know what you’re building!
Meanwhile please check out the other articles on NuttX for PinePhone…
Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.
Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…
lupyuen.github.io/src/terminal.md
NSH Architecture-Specific Initialization CONFIG_NSH_ARCHINIT should be disabled…
Application Configuration
> NSH Library
> Have architecture-specific initialization
That’s because LVGL Terminal starts a new task for NSH, which will redo the Architecture-Specific Initialization if CONFIG_NSH_ARCHINIT is enabled.
LVGL Terminal will handle the initialization when CONFIG_NSH_ARCHINIT is disabled.
On Arm64 Platforms: This (harmless) warning appears when starting the NSH Task…
mkfatfs: command not found
This will be fixed in a future patch.