đź“ť 22 Jan 2023
Apache NuttX RTOS (Real-Time Operating System) now boots on Pine64 PinePhone and runs Touchscreen Apps! (Pic above)
Does it need a special Serial Cable for PinePhone?
Not any more… NuttX will auto-boot into an LVGL Touchscreen App, without a Serial Cable!
All we need is a microSD Card for booting NuttX on PinePhone. NuttX won’t touch the eMMC Storage in PinePhone.
(Perfect for exploring the internals of PinePhone)
What’s LVGL?
LVGL is a popular library for rendering Graphical User Interfaces on Microcontrollers.
Now we have “upsized” LVGL for a Smartphone. And it works great!
So we can create our own Touchscreen App for PinePhone?
Yep! With LVGL, NuttX on PinePhone runs Touchscreen Apps almost like a regular Smartphone.
(Though much much simpler: It won’t make phone calls or browse the web)
In this article we shall…
Make a Bootable microSD with NuttX inside
Configure NuttX to boot an LVGL App
Make LVGL Apps more Touch-Friendly on PinePhone
Take a peek at the LVGL Demo Apps available for PinePhone
And explore how we might create our own Touchscreen App for PinePhone.
What’s NuttX? Why run it on PinePhone?
If we’re new to NuttX, here’s a gentle intro…
We begin by making a Bootable microSD…
Let’s make a Bootable NuttX microSD that will start an LVGL Touchscreen App on our PinePhone…
Download the PinePhone Jumpdrive Image pine64-pinephone.img.xz from…
Write the downloaded image to a microSD Card with Balena Etcher
Download Image.gz from the NuttX Release…
Image.gz: NuttX Image for PinePhone
(If we prefer to build NuttX ourselves: Follow these steps)
Copy the downloaded Image.gz and overwrite the file on the microSD Card.
(Pic above)
Insert the microSD Card into PinePhone and power up PinePhone.
NuttX boots on PinePhone and shows a Test Pattern.
(Very briefly)
The LVGL Touchscreen Demo appears on PinePhone! (Like this)
Tap around and play with the LVGL Widgets (UI Controls).
Something doesn’t work right…
Yeah there are some limitations in our Touch Panel Driver: Scrolling and swiping won’t work right now.
Someday we might fix these issues in our driver…
Let’s find out how we made NuttX boot to LVGL…
PinePhone with USB Serial Debug Cable
How did we configure NuttX to boot an LVGL App?
Normally NuttX boots to the NSH Shell. Which lets us execute Console Commands through a USB Serial Debug Cable. (Pic above)
But for today’s demo we configured NuttX to boot instead with the LVGL Demo App. In the NuttX Project Folder, we ran…
make menuconfig
And we set these options…
In “RTOS Features > Tasks and Scheduling”…
Set “Application Entry Point” to lvgldemo_main
(Which sets CONFIG_INIT_ENTRYPOINT)
Set “Application Entry Name” to lvgldemo_main
(Which sets CONFIG_INIT_ENTRYNAME)
In “Application Configuration > NSH Library”…
Disable “Have Architecture-Specific Initialization”
(Which disables CONFIG_NSH_ARCHINIT)
Save the configuration and exit menuconfig
Which will start the lvgldemo app (instead of nsh) when NuttX boots.
Doesn’t lvgldemo require a Command-Line Argument?
lvgldemo doesn’t require a Command-Line Argument if we make sure that only one LVGL Demo is selected. (Because of this)
We’ll talk about the available LVGL Demos in a while.
Why disable “NSH Architecture-Specific Initialization”?
Usually the NSH Shell initialises the drivers for LCD Display and Touch Panel on PinePhone.
But since we’re not running NSH Shell, we configured NuttX to initialise the drivers in our LVGL Demo App.
The Default LVGL Demo is a little hard to use, let’s talk about it…
Default LVGL Widget Demo is not quite so Touch-Friendly
Is there a problem with the LVGL Demo App?
The pic above shows the LVGL Demo App with the Default Settings. The dense screen is a little hard to use with my thick shaky fingers…
Let’s tweak the LVGL Demo Code to make our app more accessible.
We modify this LVGL Source File: apps/graphics/lvgl/lvgl/ demos/widgets/lv_demo_widgets.c
// Insert this
#include <stdio.h>
// Modify this function
void lv_demo_widgets(void) {
// Note: PinePhone has width 720 pixels.
// LVGL will set Display Size to Large, which looks really tiny.
// Shouldn't this code depend on DPI? (267 DPI for PinePhone)
if(LV_HOR_RES <= 320) disp_size = DISP_SMALL;
else if(LV_HOR_RES < 720) disp_size = DISP_MEDIUM;
else disp_size = DISP_LARGE;
// Insert this: Change Display Size from Large to Medium,
// to make Widgets easier to tap
disp_size = DISP_MEDIUM;
// Insert this: Print warning if font is missing
#undef LV_LOG_WARN
#define LV_LOG_WARN(s) puts(s)
The first part of the code above comes from LVGL. Since PinePhone has 720 Horizontal Pixels, the code sets Display Size to Large. Which squishes everything on PinePhone.
That’s why in the code above we override and set Display Size to Medium. Which makes the screen less dense.
Shouldn’t the Display Size be computed based on Screen DPI?
Yeah probably. PinePhone’s Display has 267 DPI, we should use it in the code above to compute the Display Size.
In the next part of the code, we ask LVGL to…
Increase the Tab Height
(For earlier tapping)
Use Font Size 20
(Instead of Font Size 14)
// Existing Code
font_large = LV_FONT_DEFAULT;
font_normal = LV_FONT_DEFAULT;
lv_coord_t tab_h;
// For Large Display Size (unused)...
if(disp_size == DISP_LARGE) {
...
}
// For Medium Display Size...
else if(disp_size == DISP_MEDIUM) {
// Change this: Increase Tab Height from
// 45 to 70, to make Tabs easier to tap
tab_h = 70;
// Previously: tab_h = 45;
#if LV_FONT_MONTSERRAT_20
font_large = &lv_font_montserrat_20;
#else
LV_LOG_WARN("LV_FONT_MONTSERRAT_20 is not enabled for the widgets demo. Using LV_FONT_DEFAULT instead.");
#endif
#if LV_FONT_MONTSERRAT_14
// Change this: Use the default font Montserrat 20
// (instead of Montserrat 14)
// Previously: font_normal = &lv_font_montserrat_14;
#else
LV_LOG_WARN("LV_FONT_MONTSERRAT_14 is not enabled for the widgets demo. Using LV_FONT_DEFAULT instead.");
#endif
}
We set the Default Font to Montserrat 20 (previously Montserrat 14) in the LVGL Configuration for NuttX: configs/lvgl/defconfig
## Set the LVGL Default Font to Montserrat 20
## (Previously Montserrat 14)
CONFIG_LV_FONT_DEFAULT_MONTSERRAT_20=y
Which will make (most) LVGL Apps more legible on PinePhone.
The LVGL Demo App is now less dense and easier to touch (pic below)…
(Too bad the scrolling isn’t working yet)
Let’s take a peek at the other LVGL Demos…
LVGL Widget Demo is Touch-Friendly now
We’ve seen the LVGL Widget Demo. What about other demos?
There are 5 LVGL Demos available in make menuconfig…
Browse into “Application Configuration > Graphics Support > Light and Versatile Graphics Library (LVGL) > LVGL Configuration”
In “Demos”: Select ONE of the these demos…
“Show Some Widgets”
“Demonstrate Usage of Encoder and Keyboard”
“Benchmark Your System”
“Stress Test for LVGL”
“Music Player Demo”
For Music Player: We need extra fonts…
Browse into “LVGL > LVGL Configuration”
In “Font usage”, select…
“Montserrat 16”
“Montserrat 20”
“Montserrat 22”
“Montserrat 32”
We’ve seen the LVGL Widget Demo…
Here’s the LVGL Music Player Demo (pic above)…
And the LVGL Benchmark Demo (pic below)…
Which gives us some useful numbers…
How well does LVGL perform on PinePhone?
From the last video (pic above) we see the LVGL Benchmark Numbers…
| Slow but common cases | Frames Per Sec |
|---|---|
| Image RGB | 19 |
| Image RGB + Opa | 17 |
| Image ARGB | 18 |
| Image ARGB + Opa | 17 |
| Image ARGB Recolor | 17 |
| Image ARGB Recolor + Opa | 16 |
| Substr Image | 19 |
| All Cases | Frames Per Sec |
|---|---|
| Rectangle | 24 |
| Rectangle + Opa | 23 |
| Rectangle Rounded | 23 |
| Rectangle Rounded + Opa | 21 |
| Circle | 23 |
| Circle + Opa | 20 |
| Border | 24 |
| Border + Opa | 24 |
| Border Rounded | 24 |
| (Many many more) |
So LVGL Performance on PinePhone looks OK.
After all, LVGL is simply blasting pixels into a RAM Framebuffer and the rest is done by PinePhone’s Display Hardware…
We’re finally ready to create our own LVGL App for PinePhone!
We’ve seen the LVGL Demo Apps for PinePhone…
Can we create our own Touchscreen App?
Yep! Simplest way to create our own app: We take the LVGL Widget Demo and modify it.
Inside our NuttX Project, look for the Widget Demo Source Code…
apps/graphics/lvgl/lvgl/demos/widgets/lv_demo_widgets.c
Modify the function lv_demo_widgets to create our own LVGL Widgets…
// Create a Button, set the Width and Height
void lv_demo_widgets(void) {
lv_obj_t *btn = lv_btn_create(lv_scr_act());
lv_obj_set_height(btn, LV_SIZE_CONTENT);
lv_obj_set_width(btn, 120);
}
(lv_demo_widgets is called by LVGL Demo App lvgldemo_main)
For details, check out the LVGL Widget Docs.
But coding LVGL Apps in C looks cumbersome…
We could consider coding in Zig to simplify our LVGL Apps (pic above)…
And Zig has helpful Runtime Safety Checks too.
What apps will we create for PinePhone and NuttX?
Maybe we can build an LVGL Terminal App? That will let us interact with the NSH NuttX Shell, without a Serial Debug Cable?
LVGL already provides an Onscreen Keyboard that works on PinePhone.
We might build the app in Zig. And we’ll redirect the NSH Console Input / Output to LVGL like so: nxterm_main.c and redirect_test.c also maybe pty_test.c
(Our LVGL Terminal will probably work like NxTerm)
What about porting a Graphical IDE to PinePhone and NuttX?
Yeah perhaps Lisp? Or Smalltalk?
Our LVGL App doesn’t appear and PinePhone’s LED turns white. What happened?
This happens if our LVGL App lvgldemo fails to start.
Check for Error Messages with a USB Serial Debug Cable.
Now we can finally build and test NuttX Apps on PinePhone… All we need is a microSD Card!
What will you create? Lemme know!
Please check out the other articles on NuttX for PinePhone…
Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.
Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…
lupyuen.github.io/src/lvgl2.md
PinePhone with USB Serial Debug Cable
The easiest way to run Apache NuttX RTOS on PinePhone is to download the NuttX Image and boot it on PinePhone…
But if we’re keen to build NuttX ourselves, here are the steps…
Install the Build Prerequisites, skip the RISC-V Toolchain…
Download the ARM64 Toolchain for
AArch64 Bare-Metal Target aarch64-none-elf
(Skip the section for Beta Releases)
Add the downloaded toolchain to the PATH Environment Variable…
gcc-arm-...-aarch64-none-elf/bin
Check the ARM64 Toolchain…
aarch64-none-elf-gcc -v
Download and configure NuttX…
mkdir nuttx
cd nuttx
git clone https://github.com/apache/nuttx nuttx
git clone https://github.com/apache/nuttx-apps apps
cd nuttx
tools/configure.sh pinephone:lvgl
By default, NuttX boots into the NSH Shell.
(Which requires a USB Serial Debug Cable for PinePhone)
If we wish to boot an LVGL App, follow the instructions here…
Build the NuttX Project…
make
With the default settings, the LVGL Widget Demo isn’t quite so Touch-Friendly. (See this)
To fix this, look for this LVGL Source File…
apps/graphics/lvgl/lvgl/demos/widgets/lv_demo_widgets.c
And replace by the contents of this file: lv_demo_widgets.c
If we wish to boot a different LVGL Demo (instead of the Widget Demo), follow the steps here…
To boot the LVGL Terminal App…
Rebuild NuttX and compress the NuttX Image…
make
cp nuttx.bin Image
rm -f Image.gz
gzip Image
This produces the file Image.gz, which will be copied to PinePhone.
If the build fails with…
token "@" is not valid in preprocessor
Then look for this file in the ARM64 Toolchain…
aarch64-none-elf/include/_newlib_version.h
And apply this patch, so that it looks like this…
// Near the end of _newlib_version.h, insert this...
#define _NEWLIB_VERSION "4.2.0"
#define __NEWLIB__ 4
#define __NEWLIB_MINOR__ 2
#endif /* !_NEWLIB_VERSION_H__ */
Follow the steps in the next section to boot the NuttX Image…
In the previous section we’ve built the NuttX Image Image.gz.
Let’s boot the NuttX Image on PinePhone, assuming we have a USB Serial Debug Cable…
Download the PinePhone Jumpdrive Image pine64-pinephone.img.xz from…
Write the downloaded image to a microSD Card with Balena Etcher or GNOME Disks.
Copy the file Image.gz from the previous section.
Overwrite the file on the microSD Card.
(Pic above)
On PinePhone, set Privacy Switch 6 (Headphone) to Off.
Connect PinePhone to our computer with the Serial Debug Cable.
On our computer, start a Serial Terminal and connect to the USB Serial Port at 115.2 kbps.
Insert the microSD Card into PinePhone and power up PinePhone.
NuttX boots on PinePhone and shows a Test Pattern.
NuttShell nsh appears in the Serial Console. (Pic below)
To see the available commands in NuttShell…
help
To run the LVGL Widget Demo…
lvgldemo widgets
And that’s how we build and boot NuttX for PinePhone!
