NuttX RTOS for PinePhone: Boot to LVGL

đź“ť 22 Jan 2023

NuttX on PinePhone now boots to the LVGL Touchscreen Demo, without a Serial Cable

Apache NuttX RTOS (Real-Time Operating System) now boots on Pine64 PinePhone and runs Touchscreen Apps! (Pic above)

Does it need a special Serial Cable for PinePhone?

Not any more… NuttX will auto-boot into an LVGL Touchscreen App, without a Serial Cable!

All we need is a microSD Card for booting NuttX on PinePhone. NuttX won’t touch the eMMC Storage in PinePhone.

(Perfect for exploring the internals of PinePhone)

What’s LVGL?

LVGL is a popular library for rendering Graphical User Interfaces on Microcontrollers.

Now we have “upsized” LVGL for a Smartphone. And it works great!

So we can create our own Touchscreen App for PinePhone?

Yep! With LVGL, NuttX on PinePhone runs Touchscreen Apps almost like a regular Smartphone.

(Though much much simpler: It won’t make phone calls or browse the web)

In this article we shall…

And explore how we might create our own Touchscreen App for PinePhone.

What’s NuttX? Why run it on PinePhone?

If we’re new to NuttX, here’s a gentle intro…

We begin by making a Bootable microSD…

PinePhone Jumpdrive on microSD

1 Boot NuttX on PinePhone

Let’s make a Bootable NuttX microSD that will start an LVGL Touchscreen App on our PinePhone…

  1. Download the PinePhone Jumpdrive Image pine64-pinephone.img.xz from…

    dreemurrs-embedded/Jumpdrive

    Write the downloaded image to a microSD Card with Balena Etcher

  2. Download Image.gz from the NuttX Release…

    Image.gz: NuttX Image for PinePhone

    (If we prefer to build NuttX ourselves: Follow these steps)

  3. Copy the downloaded Image.gz and overwrite the file on the microSD Card.

    (Pic above)

  4. Insert the microSD Card into PinePhone and power up PinePhone.

    NuttX boots on PinePhone and shows a Test Pattern.

    (Very briefly)

  5. The LVGL Touchscreen Demo appears on PinePhone! (Like this)

    Tap around and play with the LVGL Widgets (UI Controls).

    (Watch the demo on YouTube)

Something doesn’t work right…

Yeah there are some limitations in our Touch Panel Driver: Scrolling and swiping won’t work right now.

Someday we might fix these issues in our driver…

Let’s find out how we made NuttX boot to LVGL…

PinePhone with USB Serial Debug Cable

PinePhone with USB Serial Debug Cable

2 Boot to LVGL

How did we configure NuttX to boot an LVGL App?

Normally NuttX boots to the NSH Shell. Which lets us execute Console Commands through a USB Serial Debug Cable. (Pic above)

But for today’s demo we configured NuttX to boot instead with the LVGL Demo App. In the NuttX Project Folder, we ran…

make menuconfig

And we set these options…

  1. In “RTOS Features > Tasks and Scheduling”…

    Set “Application Entry Point” to lvgldemo_main

    (Which sets CONFIG_INIT_ENTRYPOINT)

    Set “Application Entry Name” to lvgldemo_main

    (Which sets CONFIG_INIT_ENTRYNAME)

  2. In “Application Configuration > NSH Library”…

    Disable “Have Architecture-Specific Initialization”

    (Which disables CONFIG_NSH_ARCHINIT)

  3. Save the configuration and exit menuconfig

Which will start the lvgldemo app (instead of nsh) when NuttX boots.

Doesn’t lvgldemo require a Command-Line Argument?

lvgldemo doesn’t require a Command-Line Argument if we make sure that only one LVGL Demo is selected. (Because of this)

We’ll talk about the available LVGL Demos in a while.

Why disable “NSH Architecture-Specific Initialization”?

Usually the NSH Shell initialises the drivers for LCD Display and Touch Panel on PinePhone.

But since we’re not running NSH Shell, we configured NuttX to initialise the drivers in our LVGL Demo App.

(More about this)

The Default LVGL Demo is a little hard to use, let’s talk about it…

Default LVGL Widget Demo is not quite so Touch-Friendly

Default LVGL Widget Demo is not quite so Touch-Friendly

3 Touch-Friendly LVGL

Is there a problem with the LVGL Demo App?

The pic above shows the LVGL Demo App with the Default Settings. The dense screen is a little hard to use with my thick shaky fingers…

Let’s tweak the LVGL Demo Code to make our app more accessible.

We modify this LVGL Source File: apps/graphics/lvgl/lvgl/ demos/widgets/lv_demo_widgets.c

// Insert this
#include <stdio.h>

// Modify this function
void lv_demo_widgets(void) {
  // Note: PinePhone has width 720 pixels.
  // LVGL will set Display Size to Large, which looks really tiny.
  // Shouldn't this code depend on DPI? (267 DPI for PinePhone)
  if(LV_HOR_RES <= 320) disp_size = DISP_SMALL;
  else if(LV_HOR_RES < 720) disp_size = DISP_MEDIUM;
  else disp_size = DISP_LARGE;

  // Insert this: Change Display Size from Large to Medium,
  // to make Widgets easier to tap
  disp_size = DISP_MEDIUM;

  // Insert this: Print warning if font is missing
  #undef LV_LOG_WARN
  #define LV_LOG_WARN(s) puts(s)

The first part of the code above comes from LVGL. Since PinePhone has 720 Horizontal Pixels, the code sets Display Size to Large. Which squishes everything on PinePhone.

That’s why in the code above we override and set Display Size to Medium. Which makes the screen less dense.

Shouldn’t the Display Size be computed based on Screen DPI?

Yeah probably. PinePhone’s Display has 267 DPI, we should use it in the code above to compute the Display Size.

In the next part of the code, we ask LVGL to…

  // Existing Code
  font_large = LV_FONT_DEFAULT;
  font_normal = LV_FONT_DEFAULT;
  lv_coord_t tab_h;

  // For Large Display Size (unused)...
  if(disp_size == DISP_LARGE) {
    ...
  }
  // For Medium Display Size...
  else if(disp_size == DISP_MEDIUM) {
    // Change this: Increase Tab Height from 
    // 45 to 70, to make Tabs easier to tap
    tab_h = 70;
    // Previously: tab_h = 45;

#if LV_FONT_MONTSERRAT_20
    font_large = &lv_font_montserrat_20;
#else
    LV_LOG_WARN("LV_FONT_MONTSERRAT_20 is not enabled for the widgets demo. Using LV_FONT_DEFAULT instead.");
#endif

#if LV_FONT_MONTSERRAT_14
    // Change this: Use the default font Montserrat 20 
    // (instead of Montserrat 14)
    // Previously: font_normal = &lv_font_montserrat_14;
#else
    LV_LOG_WARN("LV_FONT_MONTSERRAT_14 is not enabled for the widgets demo. Using LV_FONT_DEFAULT instead.");
#endif
  }

We set the Default Font to Montserrat 20 (previously Montserrat 14) in the LVGL Configuration for NuttX: configs/lvgl/defconfig

## Set the LVGL Default Font to Montserrat 20
## (Previously Montserrat 14)
CONFIG_LV_FONT_DEFAULT_MONTSERRAT_20=y

Which will make (most) LVGL Apps more legible on PinePhone.

The LVGL Demo App is now less dense and easier to touch (pic below)…

(Too bad the scrolling isn’t working yet)

Let’s take a peek at the other LVGL Demos…

LVGL Widget Demo is Touch-Friendly now

LVGL Widget Demo is Touch-Friendly now

4 LVGL Demos

We’ve seen the LVGL Widget Demo. What about other demos?

There are 5 LVGL Demos available in make menuconfig…

  1. Browse into “Application Configuration > Graphics Support > Light and Versatile Graphics Library (LVGL) > LVGL Configuration”

  2. In “Demos”: Select ONE of the these demos…

    “Show Some Widgets”

    “Demonstrate Usage of Encoder and Keyboard”

    “Benchmark Your System”

    “Stress Test for LVGL”

    “Music Player Demo”

    (LVGL won’t boot if we select 2 or more demos)

  3. For Music Player: We need extra fonts…

    Browse into “LVGL > LVGL Configuration”

    In “Font usage”, select…

    “Montserrat 16”

    “Montserrat 20”

    “Montserrat 22”

    “Montserrat 32”

LVGL Music Player Demo

LVGL Music Player Demo

We’ve seen the LVGL Widget Demo…

Here’s the LVGL Music Player Demo (pic above)…

And the LVGL Benchmark Demo (pic below)…

Which gives us some useful numbers…

LVGL Benchmark Demo

LVGL Benchmark Demo

5 LVGL Performance

How well does LVGL perform on PinePhone?

From the last video (pic above) we see the LVGL Benchmark Numbers…

Slow but common casesFrames Per Sec
Image RGB19
Image RGB + Opa17
Image ARGB18
Image ARGB + Opa17
Image ARGB Recolor17
Image ARGB Recolor + Opa16
Substr Image19
All CasesFrames Per Sec
Rectangle24
Rectangle + Opa23
Rectangle Rounded23
Rectangle Rounded + Opa21
Circle23
Circle + Opa20
Border24
Border + Opa24
Border Rounded24
(Many many more)

So LVGL Performance on PinePhone looks OK.

After all, LVGL is simply blasting pixels into a RAM Framebuffer and the rest is done by PinePhone’s Display Hardware…

We’re finally ready to create our own LVGL App for PinePhone!

LVGL Programming in Zig

LVGL Programming in Zig

6 Create a Touchscreen App

We’ve seen the LVGL Demo Apps for PinePhone…

Can we create our own Touchscreen App?

Yep! Simplest way to create our own app: We take the LVGL Widget Demo and modify it.

Inside our NuttX Project, look for the Widget Demo Source Code…

apps/graphics/lvgl/lvgl/demos/widgets/lv_demo_widgets.c

Modify the function lv_demo_widgets to create our own LVGL Widgets…

// Create a Button, set the Width and Height
void lv_demo_widgets(void) {
  lv_obj_t *btn = lv_btn_create(lv_scr_act());
  lv_obj_set_height(btn, LV_SIZE_CONTENT);
  lv_obj_set_width(btn, 120);
}

(lv_demo_widgets is called by LVGL Demo App lvgldemo_main)

For details, check out the LVGL Widget Docs.

But coding LVGL Apps in C looks cumbersome…

We could consider coding in Zig to simplify our LVGL Apps (pic above)…

And Zig has helpful Runtime Safety Checks too.

(More details here)

What apps will we create for PinePhone and NuttX?

Maybe we can build an LVGL Terminal App? That will let us interact with the NSH NuttX Shell, without a Serial Debug Cable?

LVGL already provides an Onscreen Keyboard that works on PinePhone.

We might build the app in Zig. And we’ll redirect the NSH Console Input / Output to LVGL like so: nxterm_main.c and redirect_test.c also maybe pty_test.c

(Our LVGL Terminal will probably work like NxTerm)

What about porting a Graphical IDE to PinePhone and NuttX?

Yeah perhaps Lisp? Or Smalltalk?

Our LVGL App doesn’t appear and PinePhone’s LED turns white. What happened?

This happens if our LVGL App lvgldemo fails to start.

Check for Error Messages with a USB Serial Debug Cable.

NuttX on PinePhone in the wild

7 What’s Next

Now we can finally build and test NuttX Apps on PinePhone… All we need is a microSD Card!

What will you create? Lemme know!

Please check out the other articles on NuttX for PinePhone…

Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/lvgl2.md

PinePhone with USB Serial Debug Cable

PinePhone with USB Serial Debug Cable

8 Appendix: Build Apache NuttX RTOS for PinePhone

The easiest way to run Apache NuttX RTOS on PinePhone is to download the NuttX Image and boot it on PinePhone…

But if we’re keen to build NuttX ourselves, here are the steps…

  1. Install the Build Prerequisites, skip the RISC-V Toolchain…

    “Install Prerequisites”

  2. Download the ARM64 Toolchain for AArch64 Bare-Metal Target aarch64-none-elf

    Arm GNU Toolchain Downloads

    (Skip the section for Beta Releases)

  3. Add the downloaded toolchain to the PATH Environment Variable…

    gcc-arm-...-aarch64-none-elf/bin
    

    Check the ARM64 Toolchain…

    aarch64-none-elf-gcc -v
    
  4. Download and configure NuttX…

    mkdir nuttx
    cd nuttx
    git clone https://github.com/apache/nuttx nuttx
    git clone https://github.com/apache/nuttx-apps apps
    
    cd nuttx
    tools/configure.sh pinephone:lvgl
    
  5. By default, NuttX boots into the NSH Shell.

    (Which requires a USB Serial Debug Cable for PinePhone)

    If we wish to boot an LVGL App, follow the instructions here…

    “Boot to LVGL”

  6. Build the NuttX Project…

    make
    

    (See the Build Log)

    (Missing math.h? See this)

  7. With the default settings, the LVGL Widget Demo isn’t quite so Touch-Friendly. (See this)

    To fix this, look for this LVGL Source File…

    apps/graphics/lvgl/lvgl/demos/widgets/lv_demo_widgets.c
    

    And replace by the contents of this file: lv_demo_widgets.c

  8. If we wish to boot a different LVGL Demo (instead of the Widget Demo), follow the steps here…

    “LVGL Demos”

  9. To boot the LVGL Terminal App…

    “LVGL Terminal”

  10. Rebuild NuttX and compress the NuttX Image…

    make
    cp nuttx.bin Image
    rm -f Image.gz
    gzip Image
    

    This produces the file Image.gz, which will be copied to PinePhone.

  11. If the build fails with…

    token "@" is not valid in preprocessor
    

    Then look for this file in the ARM64 Toolchain…

    aarch64-none-elf/include/_newlib_version.h
    

    And apply this patch, so that it looks like this…

    // Near the end of _newlib_version.h, insert this...
    #define _NEWLIB_VERSION "4.2.0"
    #define __NEWLIB__ 4
    #define __NEWLIB_MINOR__ 2
    
    #endif /* !_NEWLIB_VERSION_H__ */
    

Follow the steps in the next section to boot the NuttX Image…

PinePhone Jumpdrive on microSD

9 Appendix: Boot Apache NuttX RTOS on PinePhone

(Watch the Demo on YouTube)

In the previous section we’ve built the NuttX Image Image.gz.

Let’s boot the NuttX Image on PinePhone, assuming we have a USB Serial Debug Cable…

  1. Download the PinePhone Jumpdrive Image pine64-pinephone.img.xz from…

    dreemurrs-embedded/Jumpdrive

    Write the downloaded image to a microSD Card with Balena Etcher or GNOME Disks.

  2. Copy the file Image.gz from the previous section.

    Overwrite the file on the microSD Card.

    (Pic above)

  3. On PinePhone, set Privacy Switch 6 (Headphone) to Off.

    Connect PinePhone to our computer with the Serial Debug Cable.

    On our computer, start a Serial Terminal and connect to the USB Serial Port at 115.2 kbps.

  4. Insert the microSD Card into PinePhone and power up PinePhone.

    NuttX boots on PinePhone and shows a Test Pattern.

    NuttShell nsh appears in the Serial Console. (Pic below)

    (See the Boot Log)

  5. To see the available commands in NuttShell…

    help
    

    To run the LVGL Widget Demo…

    lvgldemo widgets
    

    (We should see this)

    (Other LVGL Demos)

And that’s how we build and boot NuttX for PinePhone!

Booting Apache NuttX RTOS on PinePhone

(See the Boot Log)