NuttX RTOS for PinePhone: Touch Panel

đź“ť 12 Jan 2023

Apache NuttX RTOS reads the PinePhone Touch Panel

We’re porting Apache NuttX RTOS (Real-Time Operating System) to Pine64 PinePhone…

Now we can render LVGL Graphical User Interfaces… But it won’t work yet with Touch Input!

Let’s talk about the Capacitive Touch Panel inside PinePhone…

We begin with the internals of the Touch Panel…

Capacitive Touch Panel in PinePhone Schematic (Pages 9 and 11)

Capacitive Touch Panel in PinePhone Schematic (Pages 9 and 11)

1 Goodix GT917S Touch Panel

Inside PinePhone is the Goodix GT917S Capacitive Touch Panel (CTP) that talks over I2C.

According to the PinePhone Schematic Pages 9 and 11 (pic above)…

What are PH4 and PH11?

Just think of them as GPIOs on the Allwinner A64 SoC.

(Allwinner calls them PIOs)

Does it need special power?

Please remember to power up LDO (3.3V) through the Power Management Integrated Circuit…

PinePhone’s Touch Panel doesn’t seem to be the Power-Saving type like PineTime’s CST816S.

How do we program the Touch Panel?

The datasheet doesn’t say much about programming the Touch Panel…

So we’ll create the driver by replicating the I2C Read / Write Operations from the official Android Driver gt9xx.c.

(Or the unofficial simpler driver GT911.c)

So PinePhone’s Touch Panel is actually undocumented?

Yeah it’s strangely common for Touch Panels to be undocumented.

(Just like PineTime’s CST816S Touch Panel)

Let’s experiment with PinePhone’s Touch Panel to understand how it works…

(I think Touch Panels are poorly documented because of Apple’s patent on Multitouch)

Reading the Product ID from Touch Panel

2 Read the Product ID

What’s the simplest thing we can do with PinePhone’s Touch Panel?

Let’s read the Product ID from the Touch Panel.

We experimented with the Touch Panel (Bare Metal with NuttX) and discovered these I2C Settings…

Based on the above settings, we wrote this Test Code that runs in the NuttX Kernel: pinephone_bringup.c

// Read Product ID from Touch Panel over I2C
static void touch_panel_read(
  struct i2c_master_s *i2c  // NuttX I2C Bus (Port TWI0)
) {
  uint32_t freq = 400000;  // I2C Frequency: 400 kHz
  uint16_t addr = 0x5d;    // Default I2C Address for Goodix GT917S
  uint16_t reg  = 0x8140;  // Register Address: Read Product ID

  // Swap the Register Address, MSB first
  uint8_t regbuf[2] = {
    reg >> 8,   // First Byte: MSB
    reg & 0xff  // Second Byte: LSB
  };

  // Erase the Receive Buffer (4 bytes)
  uint8_t buf[4];
  memset(buf, 0xff, sizeof(buf));

  // Compose the I2C Messages
  struct i2c_msg_s msgv[2] = {
    // Send the 16-bit Register Address (MSB first)
    {
      .frequency = freq,
      .addr      = addr,
      .flags     = 0,
      .buffer    = regbuf,
      .length    = sizeof(regbuf)
    },
    // Receive the Register Data (4 bytes)
    {
      .frequency = freq,
      .addr      = addr,
      .flags     = I2C_M_READ,
      .buffer    = buf,
      .length    = sizeof(buf)
    }
  };

  // Execute the I2C Transfer
  int ret = I2C_TRANSFER(i2c, msgv, 2);
  DEBUGASSERT(ret == OK);

  // Dump the Receive Buffer
  infodumpbuffer("buf", buf, buflen);
  // Shows "39 31 37 53" or "917S"
}

This is what we see (with TWI0 Logging Enabled)…

Read Product ID from Touch Panel

Yep the I2C Response is correct…

39 31 37 53

Which is ASCII for “917S”!

(Goodix GT917S Touch Panel)

How’s the code above called by NuttX Kernel?

Read on to find out how we poll the Touch Panel and read the Product ID…

Polling the Touch Panel

3 Poll the Touch Panel

PinePhone’s Touch Panel will trigger interrupts right?

To detect Touch Events, we’ll need to handle the interrupts triggered by Touch Panel.

Based on our research, PinePhone’s Touch Panel Interrupt (CTP-INT) is connected at PH4.

But to simplify our first experiment, let’s poll PH4. (Instead of handling interrupts)

How do we poll PH4?

We read PH4 as a GPIO Input. When we touch the Touch Panel, PH4 goes from Low to High.

This is how we poll PH4: pinephone_bringup.c

// Test Touch Panel Interrupt by Polling as GPIO Input.
// Touch Panel Interrupt (CTP-INT) is at PH4.
// We configure it for GPIO Input.
#define CTP_INT (PIO_INPUT | PIO_PORT_PIOH | PIO_PIN4)

// Poll for Touch Panel Interrupt (PH4) by reading as GPIO Input
void touch_panel_initialize(
  struct i2c_master_s *i2c  // NuttX I2C Bus (Port TWI0)
) {

  // Configure the Touch Panel Interrupt for GPIO Input
  int ret = a64_pio_config(CTP_INT);
  DEBUGASSERT(ret == OK);

  // Poll the Touch Panel Interrupt as GPIO Input
  bool prev_val = false;
  for (int i = 0; i < 6000; i++) {  // Poll for 60 seconds

    // Read the GPIO Input
    bool val = a64_pio_read(CTP_INT);

    // If value has changed...
    if (val != prev_val) {

      // Print the transition
      if (val) { up_putc('+'); }  // PH4 goes Low to High
      else     { up_putc('-'); }  // PH4 goes High to Low
      prev_val = val;

      // If PH4 has just transitioned from Low to High...
      if (val) {

        // Read the Touch Panel over I2C
        touch_panel_read(i2c);
      }
    }

    // Wait a while
    up_mdelay(10);
  }
}

(a64_pio_config configures PH4 as an Input Pin)

(a64_pio_read reads PH4 as an Input Pin)

The loop above watches for PH4 shifting from Low to High…

Thus our simple loop simulates an Interrupt Handler!

How do we open the I2C Port?

On NuttX, this is how we open the I2C Port and pass it to the above loop: pinephone_bringup.c

// Open Allwinner A64 Port TWI0 for I2C
struct i2c_master_s *i2c =
  a64_i2cbus_initialize(0);  // 0 for TWI0

// Pass the I2C Port to the above loop
touch_panel_initialize(i2c);

We insert this code at the end of the PinePhone Bringup Function, so that NuttX Kernel will run it at the end of startup.

(Yes it sounds hacky, but it’s a simple way to do Kernel Experiments)

Now that we can poll our Touch Panel, let’s read a Touch Point!

Reading a Touch Point

4 Read a Touch Point

When the Touch Panel is touched, how do we read the Touch Coordinates?

Based on the GT911 Reference Code, here are the steps to read a Touch Point…

  1. Read the Touch Panel Status (1 byte) at I2C Register 0x814E

    Status Code is Bit 7 of Touch Panel Status

    Touched Points is Bits 0 to 3 of Touch Panel Status

  2. If Status Code is non-zero and Touched Points is 1 or more…

    Read the Touch Coordinates (6 bytes) at I2C Register 0x8150

    First 2 Bytes (LSB First) are the X Coordinate (0 to 720)

    Next 2 Bytes (LSB First) are the Y Coordinate (0 to 1440)

    (What’s in the 2 remaining bytes? Doesn’t seem to indicate Touch Up / Touch Down)

  3. To acknowledge the Touch Point, set the Touch Panel Status to 0…

    Write 0 to I2C Register 0x814E

(This won’t support Multitouch, more about this later)

Here is our code: pinephone_bringup.c

// I2C Registers for Touch Panel
#define GTP_READ_COORD_ADDR 0x814E  // Touch Panel Status
#define GTP_POINT1          0x8150  // First Touch Point

// Read Touch Panel over I2C
static void touch_panel_read(
  struct i2c_master_s *i2c  // NuttX I2C Bus (Port TWI0)
) {

  // Read the Touch Panel Status
  uint8_t status[1];
  touch_panel_i2c_read(   // Read from I2C Touch Panel...
    i2c,                  // NuttX I2C Bus (Port TWI0)
    GTP_READ_COORD_ADDR,  // I2C Register: 0x814E
    status,               // Receive Buffer
    sizeof(status)        // Buffer Size
  );
  // Receives "81"

  // Decode the Status Code and the Touched Points
  const uint8_t status_code    = status[0] & 0x80;  // Set to 0x80
  const uint8_t touched_points = status[0] & 0x0f;  // Set to 0x01

  if (status_code != 0 &&     // If Status Code is OK and...
      touched_points >= 1) {  // Touched Points is 1 or more

    // Read the First Touch Coordinates
    uint8_t touch[6];
    touch_panel_i2c_read(  // Read from I2C Touch Panel...
      i2c,                 // NuttX I2C Bus (Port TWI0)
      GTP_POINT1,          // I2C Register: 0x8150
      touch,               // Receive Buffer
      sizeof(touch)        // Buffer Size
    );
    // Receives "92 02 59 05 1b 00"

    // Decode the Touch Coordinates
    const uint16_t x = touch[0] + (touch[1] << 8);
    const uint16_t y = touch[2] + (touch[3] << 8);
    _info("touch x=%d, y=%d\n", x, y);
    // Shows "touch x=658, y=1369"
  }

  // Set the Touch Panel Status to 0
  touch_panel_set_status(i2c, 0);
}

(touch_panel_i2c_read reads from the I2C Touch Panel)

(touch_panel_set_status sets the I2C Touch Panel Status)

Let’s run the code…

Reading Touch Points with Polling

When we tap the screen, we see “-+” which means that PH4 has shifted from Low to High.

Followed by the reading of the Touch Panel Status…

-+
twi_transfer: TWI0 count: 2
twi_wait: TWI0 Waiting...
twi_put_addr: TWI address 7bits+r/w = 0xba
twi_put_addr: TWI address 7bits+r/w = 0xbb
twi_wait: TWI0 Awakened with result: 0
0000  81                                               .               

(Source)

Touch Panel Status is 0x81. Which means the status is OK and there’s One Touch Point detected.

Our code reads the Touch Coordinates…

twi_transfer: TWI0 count: 2
twi_wait: TWI0 Waiting...
twi_put_addr: TWI address 7bits+r/w = 0xba
twi_put_addr: TWI address 7bits+r/w = 0xbb
twi_wait: TWI0 Awakened with result: 0
0000  92 02 59 05 1b 00                                ..Y...          
touch_panel_read: touch x=658, y=1369

(Source)

This says that the Touch Point is at…

x=658, y=1369

Which is quite close to the Lower Right Corner. (Screen size is 720 x 1440)

Yep we can read the Touch Coordinates correctly, through polling! (But not so efficiently)

Let’s handle interrupts from the Touch Panel…

Attaching our Interrupt Handler

5 Interrupt Handler for Touch Panel

We’ve done polling with the Touch Panel…

Can we handle interrupts from the Touch Panel?

Earlier we’ve read the Touch Panel by polling… Which is easier but inefficient.

So we tried reading the Touch Panel with an Interrupt Handler…

But there’s a problem: The Touch Panel only fires an interrupt once.

It won’t trigger interrupts correctly when we touch the screen.

Is this a showstopper for our Touch Panel Driver?

Not really, polling will work fine for now.

In a while we’ll run the LVGL Demo App, which uses polling. (Instead of interrupts)

Now we dive inside our NuttX Touch Panel Driver that will be called by NuttX Apps..

6 NuttX Touch Panel Driver

What’s inside our NuttX Touch Panel Driver for PinePhone?

We took the code from above and wrapped it inside our NuttX Touch Panel Driver for PinePhone…

NuttX Apps will access our driver at /dev/input0, which exposes the following File Operations: gt9xx.c

// File Operations supported by the Touch Panel
struct file_operations g_gt9xx_fileops = {
  gt9xx_open,   // Open the Touch Panel
  gt9xx_close,  // Close the Touch Panel
  gt9xx_read,   // Read a Touch Sample
  gt9xx_poll    // Setup Poll for Touch Sample

NuttX Apps will call these Touch Panel Operations through the POSIX Standard Functions open(), close(), read() and poll().

(Later we’ll see how LVGL Apps do this)

How do we start the Touch Panel Driver?

This is how we start the Touch Panel Driver when NuttX boots: pinephone_touch.c

// Default I2C Address for Goodix GT917S
#define CTP_I2C_ADDR 0x5d

// Register the Touch Panel Driver
ret = gt9xx_register(
  "/dev/input0",      // Device Path
  i2c,                // I2C Bus
  CTP_I2C_ADDR,       // I2C Address of Touch Panel
  &g_pinephone_gt9xx  // Callbacks for PinePhone Operations
);
DEBUGASSERT(ret == OK);

(gt9xx_register comes from our Touch Panel Driver)

(g_pinephone_gt9xx defines the Interrupt Callbacks)

The Touch Panel operations are explained in the Appendix…

The driver code looks familiar?

We borrowed the logic from the NuttX Driver for Cypress MBR3108.

(Which is also an I2C Input Device)

Let’s test our Touch Panel Driver with a NuttX App…

LVGL Demo App on PinePhone

7 LVGL Calls Our Driver

Have we tested our driver with NuttX Apps?

Our NuttX Touch Panel Driver works great with the LVGL Demo App! (Pic above)

Here are the LVGL Settings for NuttX…

  1. Enable “Application Configuration > Graphics Support > Light and Versatile Graphics Library (LVGL)”

  2. Enable “LVGL > Enable Framebuffer Port”

  3. Enable “LVGL > Enable Touchpad Port”

  4. Browse into “LVGL > LVGL Configuration”

  5. Enable “Application Configuration > Examples > LVGL Demo”

Also we need to set in .config…

CONFIG_LV_TICK_CUSTOM=y
CONFIG_LV_TICK_CUSTOM_INCLUDE="port/lv_port_tick.h"

Which is advised by FASTSHIFT…

“The tick of LVGL should not be placed in the same thread as the rendering, because the execution time of lv_timer_handler is not deterministic, which will cause a large error in LVGL tick.”

“We should let LVGL use the system timestamp provided by lv_port_tick, just need to set two options (above)”

(Thank you so much FASTSHIFT!)

How does LVGL call our Touch Panel Driver?

The LVGL App begins by opening our Touch Panel Driver at /dev/input0: lv_port_touchpad.c

// From lv_port_touchpad_init()...
// Open the Touch Panel Device
int fd = open(
  "/dev/input0",         // Path of Touch Panel Device
  O_RDONLY | O_NONBLOCK  // Read-Only Access
);

The app runs an Event Loop that periodically reads a Touch Sample from our driver: lv_port_touchpad.c

// From touchpad_read()...
// Struct for Touch Sample
struct touch_sample_s sample;

// Read a Touch Sample from Touch Panel
read(
  fd,       // File Descriptor from `open("/dev/input0")`
  &sample,  // Touch Sample
  sizeof(struct touch_sample_s)  // Size of Touch Sample
);

(More about the Event Loop in a while)

We’ll receive a Touch Sample that contains 0 or 1 Touch Points. (More about Touch Samples)

(The Read Operation above is Non-Blocking. It returns 0 Touch Points if the screen hasn’t been touched)

We extract the First Touch Point (inside the Touch Sample) and return it to LVGL: lv_port_touchpad.c

// From touchpad_read()...
// Get the First Touch Event from the Touch Sample
uint8_t touch_flags = sample.point[0].flags;

// If the Touch Event is Touch Down or Touch Move...
if (touch_flags & TOUCH_DOWN || touch_flags & TOUCH_MOVE) {
  // Report it as LVGL Press
  // with the Touch Coordinates
  touchpad_obj->last_state = LV_INDEV_STATE_PR;
  touchpad_obj->last_x = sample.point[0].x;
  touchpad_obj->last_y = sample.point[0].y;
  ...
} else if (touch_flags & TOUCH_UP) {
  // If the Touch Event is Touch Up,
  // report it as LVGL Release with
  // the previous Touch Coordinates
  touchpad_obj->last_state = LV_INDEV_STATE_REL;
}

And that’s how LVGL polls our driver to handle Touch Events!

(LVGL polling our driver is not so efficient, but it works!)

How to create our own LVGL Touchscreen App?

Inside our NuttX Project, look for the LVGL Demo Source Code…

Modify the function lv_demo_widgets to create our own LVGL Widgets…

// Create a Button, set the Width and Height
void lv_demo_widgets(void) {
  lv_obj_t *btn = lv_btn_create(lv_scr_act());
  lv_obj_set_height(btn, LV_SIZE_CONTENT);
  lv_obj_set_width(btn, 120);
}

For details, check out the LVGL Widget Docs.

Can we improve the rendering speed?

Yep we need to flush the CPU Cache to fix a rendering issue with the Allwinner A64 Display Engine.

This ought to improve the rendering speed and make LVGL more responsive.

(More about this)

Where’s this LVGL Event Loop that periodically reads a Touch Sample from our driver?

The LVGL Event Loop comes from the Main Function of our LVGL Demo App: lvgldemo.c

// Loop forever handling LVGL Event...
while (1) {

  // Execute LVGL Background Tasks
  uint32_t idle = lv_timer_handler();

  // Minimum sleep of 1ms
  idle = idle ? idle : 1;
  usleep(idle * 1000);
}

lv_timer_handler will periodically execute LVGL Background Tasks to…

8 Driver Limitations

Is there anything missing in our NuttX Touch Panel Driver for PinePhone?

Yep our driver has limitations, since the Touch Panel Hardware is poorly documented…

  1. Our driver doesn’t support Multitouch and Swiping.

    Someday we might fix this when we decipher the (undocumented) Official Android Driver.

    (2,000 lines of code!)

  2. But the LVGL Demo doesn’t support Multitouch either.

    (So we might put on hold for now)

  3. PinePhone’s Touch Panel seems to trigger too few interrupts. (See this)

    Again we’ll have to decipher the (still undocumented) Official Android Driver to fix this.

  4. Every read() forces an I2C Read AND Write.

    This feels expensive. We should fix this with our Interrupt Handler.

    (After fixing our Interrupt Handler)

  5. Note to Future Self: poll() won’t work correctly for awaiting Touch Points!

    That’s because the Touch Panel won’t generate interrupts for every Touch Point. (See this)

    (More about the Poll Setup)

  6. The LVGL Demo doesn’t call poll(), it only calls non-blocking read().

    So we’re good for now.

  7. As we add more features to our Touch Panel Driver, we should reuse the Touchscreen Upper Half Driver: touchscreen_upper.c

9 What’s Next

PinePhone on NuttX will soon support LVGL Touchscreen Apps!

Now we need to tidy up the LVGL Demo App so that it’s more Touch-Friendly for a Phone Form Factor. We’ll talk more about this…

(Pardon me while I learn to shoot a decent video of a Touchscreen App with a Mirrorless Camera)

Please check out the other articles on NuttX for PinePhone…

Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/touch2.md

10 Appendix: NuttX Touch Panel Driver for PinePhone

What’s inside our NuttX Touch Panel Driver for PinePhone?

We took the code from above and wrapped it inside our NuttX Touch Panel Driver for PinePhone…

NuttX Apps will access our driver at /dev/input0, which exposes the following File Operations: gt9xx.c

// File Operations supported by the Touch Panel
struct file_operations g_gt9xx_fileops = {
  gt9xx_open,   // Open the Touch Panel
  gt9xx_close,  // Close the Touch Panel
  gt9xx_read,   // Read a Touch Sample
  gt9xx_poll    // Setup Poll for Touch Sample

NuttX Apps will call these Touch Panel Operations through the POSIX Standard Functions open(), close(), read() and poll().

How do we start the Touch Panel Driver?

This is how we start the Touch Panel Driver when NuttX boots: pinephone_bringup.c

// Default I2C Address for Goodix GT917S
#define CTP_I2C_ADDR 0x5d

// Register the Touch Panel Driver
ret = gt9xx_register(
  "/dev/input0",      // Device Path
  i2c,                // I2C Bus
  CTP_I2C_ADDR,       // I2C Address of Touch Panel
  &g_pinephone_gt9xx  // Callbacks for PinePhone Operations
);
DEBUGASSERT(ret == OK);

(gt9xx_register comes from our Touch Panel Driver)

(g_pinephone_gt9xx defines the Interrupt Callbacks)

The driver code looks familiar?

We borrowed the logic from the NuttX Driver for Cypress MBR3108.

(Which is also an I2C Input Device)

UPDATE: We should reuse the Touchscreen Upper Half Driver: touchscreen_upper.c

Let’s talk about the Touch Panel operations…

10.1 Register Touch Panel Driver

At startup, pinephone_bringup registers our Touch Panel Driver at /dev/input0 by calling…

Which will…

  1. Initialise the Struct for Touch Panel

  2. Register the Touch Panel Driver with NuttX

    (At /dev/input0)

  3. Attach the Interrupt Handler with NuttX

    (Implemented as pinephone_gt9xx_irq_attach)

    (As explained here)

    (Interrupt Handler is gt9xx_isr_handler)

  4. Disable Interrupts from the Touch Panel

    (We’ll enable interrupts when we open the Touch Panel)

Now watch what happens when a NuttX App opens the Touch Panel…

10.2 Open the Touch Panel

When a NuttX App calls open() on /dev/input0, NuttX Kernel invokes this operation on our driver…

Inside the Open Operation we…

  1. Power On the Touch Panel

    (Implemented as pinephone_gt9xx_set_power)

  2. Probe the Touch Panel on the I2C Bus, to verify that it exists

    (Implemented as gt9xx_probe_device)

    (Which reads the Product ID)

    (By calling gt9xx_i2c_read)

  3. Enable Interrupts from the Touch Panel

    (Implemented as pinephone_gt9xx_irq_enable)

    (As explained here)

The Actual Flow looks more complicated because we do Reference Counting.

(We do the above steps only on the first call to open())

Let’s read some touch data…

10.3 Read a Touch Sample

What’s a Touch Sample?

When a NuttX App reads data from our Touch Panel, the app passes a Touch Sample Struct…

// Struct for Touch Sample
struct touch_sample_s sample;

// Read a Touch Sample from Touch Panel
read(
  fd,       // File Descriptor from `open("/dev/input0")`
  &sample,  // Touch Sample
  sizeof(struct touch_sample_s)  // Size of Touch Sample
);

(Source)

A Touch Sample contains One Touch Point (by default): touchscreen.h

// Touch Sample Struct
struct touch_sample_s {
  int npoints;  // Number of Touch Points in point[]
  struct touch_point_s point[1];  // Touch Points of length npoints
};

A Touch Point contains the X and Y Coordinates, also indicates whether it’s Touch Up or Touch Down: touchscreen.h

// Touch Point Struct
struct touch_point_s {
  uint8_t  id;     // Identifies the finger touched (Multitouch)
  uint8_t  flags;  // Touch Up or Touch Down
  int16_t  x;      // X Coordinate of the Touch Point
  int16_t  y;      // Y Coordinate of the Touch Point
  ...

When the app calls read(), NuttX Kernel calls our driver at…

Which does this…

  1. If the Last Result was Touch Down…

    We return the Last Touch Point, now changed to Touch Up.

    (We simulate the Touch Up because our LVGL Demo expects it)

  2. If the Last Result was NOT Touch Down…

    We clear the Interrupt Pending Flag, read the Touch Point from the Touch Panel and return it.

    (Implemented as gt9xx_read_touch_data)

    (As explained here)

    (Which calls gt9xx_set_status to set the status)

    (Which calls gt9xx_i2c_write to write over I2C)

    We ignore Duplicate Touch Points.

    (Otherwise we’ll see duplicates like this)

    (Also fixes the duplicate keypresses at the end of this video)

Since our driver doesn’t support Multitouch, the Read Operation will return either 0 or 1 Touch Points.

Why the Duplicate Touch Points?

Right now we ignore Duplicate Touch Points, because we saw the Touch Panel generating duplicate points. (See this)

(We added the logs here)

The Touch Panel seems to be producing Touch Up Events… Even though the 6-byte Touch Data looks identical for the Touch Down and Touch Up Events. (See this)

Eventually we’ll have to decode the Touch Up Events. And then remove our Simulated Touch Up Event.

Let’s talk about the Interrupt Pending Flag…

10.4 Interrupt Handler

This is our Interrupt Handler for Touch Panel Interrupts…

Inside the Interrupt Handler we…

  1. Set the Interrupt Pending Flag

    (Which is protected by a NuttX Critical Section)

  2. Notify the Poll Waiters (Background Threads)

    (As explained here)

Now we talk about the Poll Waiters…

10.5 Setup Poll for Touch Sample

A NuttX App calls poll() to set up (or tear down) a Poll for Touch Sample.

This enables the app to suspend itself and block until a Touch Panel Interrupt has been triggered. (And there’s a Touch Point available)

When an app calls poll(), the NuttX Kernel calls our driver at…

For Poll Setup:

  1. We find an Available Slot for the Poll Waiter

    (Poll Waiter Slots are defined in gt9xx_dev_s)

    (INPUT_GT9XX_NPOLLWAITERS is the max number of slots, set to 1)

  2. We bind the Poll Struct and this Slot

  3. If Interrupt Pending is set, we notify the Poll Waiters

For Poll Teardown: We unbind the Poll Setup

10.6 Close the Touch Panel

When a NuttX App calls close() on /dev/input0, NuttX Kernel invokes this operation on our driver…

Inside the Close Operation we…

  1. Disable Interrupts from the Touch Panel

    (Implemented as pinephone_gt9xx_irq_enable)

  2. Power Off the Touch Panel

    (Implemented as pinephone_gt9xx_set_power)

We do this only if the Reference Count decrements to 0.

(Which indicates the final close() for our driver)

Attaching our Interrupt Handler

11 Appendix: Interrupt Handler for Touch Panel

Earlier we’ve read the Touch Panel by polling… Which is easier but inefficient.

So we tried reading the Touch Panel with an Interrupt Handler… But there’s a problem: The Touch Panel only fires an interrupt once!

It won’t trigger interrupts correctly when we touch the screen.

Is this a showstopper for our Touch Panel Driver?

Not really, polling will work fine for now.

Earlier we saw that the LVGL Demo App runs OK because it uses polling. (Instead of interrupts)

This section talks about our experiments with Touch Panel Interrupts…

11.1 Attach our Interrupt Handler

Earlier we said that PinePhone’s Touch Panel fires an interrupt at PH4 when it’s touched…

This is how we attach our Interrupt Handler to PH4 in NuttX: pinephone_bringup.c

// Touch Panel Interrupt (CTP-INT) is at PH4
#define CTP_INT ( \
  PIO_EINT      | \  /* PIO External Interrupt */
  PIO_PORT_PIOH | \  /* PIO Port H */
  PIO_PIN4        \  /* PIO Pin 4 */
)

// Register the Interrupt Handler for Touch Panel
void touch_panel_initialize(void) {

  // Attach the PIO Interrupt Handler for Port PH
  int ret = irq_attach(     // Attach a NuttX Interrupt Handler...
    A64_IRQ_PH_EINT,        // Interrupt Number for Port PH: 53
    touch_panel_interrupt,  // Interrupt Handler
    NULL                    // Argument for Interrupt Handler
  );
  DEBUGASSERT(ret == OK);

  // Set Interrupt Priority in Generic Interrupt Controller v2
  arm64_gic_irq_set_priority(
    A64_IRQ_PH_EINT,  // Interrupt Number for Port PH: 53
    2,                // Interrupt Priority
    IRQ_TYPE_EDGE     // Trigger on Low-High Transition
  );

  // Enable the PIO Interrupt for Port PH.
  // A64_IRQ_PH_EINT is 53.
  up_enable_irq(A64_IRQ_PH_EINT);

  // Configure the Touch Panel Interrupt for Pin PH4
  ret = a64_pio_config(CTP_INT);
  DEBUGASSERT(ret == OK);

  // Enable the Touch Panel Interrupt for Pin PH4
  ret = a64_pio_irqenable(CTP_INT);
  DEBUGASSERT(ret == OK);
}

(arm64_gic_irq_set_priority configures the Generic Interrupt Controller)

(a64_pio_config configures PH4 as an Interrupt Pin)

(a64_pio_irqenable enables interrupts on Pin PH4)

Why call both up_enable_irq and a64_pio_irqenable?

Allwinner A64 does Two-Tier Interrupts, by Port and Pin…

Which means that our Interrupt Handler will be shared by all Pins on Port PH.

(When we enable them in future)

What’s touch_panel_interrupt?

touch_panel_interrupt is our Interrupt Handler. Let’s do a simple one…

// Interrupt Handler for Touch Panel
static int touch_panel_interrupt(int irq, void *context, void *arg) {

  // Print something when interrupt is triggered
  up_putc('.');
  return OK;
}

This Interrupt Handler simply prints “.” whenever the Touch Panel triggers an interrupt.

But our Interrupt Handler won’t actually read the Touch Coordinates.

(Because Interrupt Handlers can’t make I2C calls)

We’ll fix this in the next section.

It’s OK to call up_putc in an Interrupt Handler?

Yep it’s perfectly OK, because up_putc simply writes to the UART Output Register.

(It won’t trigger another interrupt)

Handling Interrupts from Touch Panel

11.2 Handle Interrupts from Touch Panel

Here’s the actual Interrupt Handler in our Touch Panel Driver: gt9xx.c

// Interrupt Handler for Touch Panel
static int gt9xx_isr_handler(int irq, FAR void *context, FAR void *arg) {

  // For Testing: Print something when interrupt is triggered
  up_putc('.');

  // Get the Touch Panel Device
  FAR struct gt9xx_dev_s *priv = (FAR struct gt9xx_dev_s *)arg;

  // Begin Critical Section
  flags = enter_critical_section();

  // Set the Interrupt Pending Flag
  priv->int_pending = true;

  // End Critical Section
  leave_critical_section(flags);

  // Notify the Poll Waiters
  poll_notify(  // Notify these File Descriptors...
    priv->fds,  // File Descriptors to notify
    1,          // Max 1 File Descriptor supported
    POLLIN      // Poll Event to be notified
  );
  return 0;
}

(gt9xx_dev_s is the Touch Panel Device)

Our Interrupt Handler won’t actually read the Touch Coordinates. (Because Interrupt Handlers can’t make I2C calls)

Instead our Interrupt Handler sets the Interrupt Pending Flag and notifies the Background Thread (via Poll Waiters) that there’s a Touch Event waiting to be processed.

The Background Thread calls poll(), suspends itself and waits for the notification before processing the Touch Event over I2C.

(Thanks to gt9xx_poll)

Let’s test our new and improved Interrupt Handler…

Testing our Interrupt Handler

11.3 Test our Interrupt Handler

How do we test our Interrupt Handler?

We could start a Background Thread that will be notified when the screen is touched…

Or we can run a simple loop that checks whether the Interrupt Pending Flag is set by our Interrupt Handler.

Let’s test the simple way: pinephone_bringup.c

// Poll for Touch Panel Interrupt
for (int i = 0; i < 6000; i++) {  // Poll for 60 seconds

  // If Touch Panel Interrupt has been triggered...
  if (priv->int_pending) {

    // Read the Touch Panel over I2C
    touch_panel_read(i2c_dev);

    // Reset the Interrupt Pending Flag
    priv->int_pending = false;
  }

  // Wait a while
  up_mdelay(10);  // 10 milliseconds
}

(This loop works only when Interrupt Trigger is set to IRQ_TYPE_LEVEL. See the next section)

Note that we call touch_panel_read to read the Touch Coordinates. (After the Touch Interrupt has been triggered)

And it works! (Pic above)

0000  81                                               .               
0000  19 01 e6 02 2a 00                                ....*.          
touch_panel_read: touch x=281, y=742

0000  81                                               .               
0000  81 02 33 00 25 00                                ..3.%.          
touch_panel_read: touch x=641, y=51

0000  81                                               .               
0000  0f 00 72 05 14 00                                ..r...          
touch_panel_read: touch x=15, y=1394

(Source)

The log shows that we’ve read the Touch Panel Status 0x81, followed by the Touch Coordinates. Yep we’ve tested our Interrupt Handler successfully!

But there’s a problem…

11.4 Too Few Interrupts

So Touch Panel Interrupts are working OK?

There’s a problem: The Touch Panel only fires an interrupt once!

It won’t trigger interrupts correctly when we touch the screen.

How do we know this?

The Debug Log shows that “.” is printed only once… Which means our Interrupt Handler is only triggered once!

gt9xx_probe_device (0x40b1ba18):
0000  39 31 37 53                                      917S            
pinephone_gt9xx_irq_enable: enable=1
gt9xx_set_status: status=0
gt9xx_i2c_write: reg=0x814e, val=0
touchpad /dev/input0 open success
touchpad_init
.
Before: disp_size=2
After: disp_size=1
gt9xx_read: buflen=32
gt9xx_read_touch_data: 
gt9xx_i2c_read: reg=0x814e, buflen=1
gt9xx_i2c_read (0x40b1bab0):
0000  80

(See the Debug Log)

We might need to study the Generic Interrupt Controller to learn how it handles such interrupts.

Is this a showstopper for our Touch Panel Driver?

Not really, polling will work fine for now.

Earlier we saw that the LVGL Demo App runs OK because it uses polling. (Instead of interrupts)

But let’s consider an alternative setup (with too many interrupts)…

Touch Panel triggers our Interrupt Handler Non-Stop

11.5 Too Many Interrupts

Why did we set the Interrupt Trigger to IRQ_TYPE_EDGE?

// Set Interrupt Priority in Generic Interrupt Controller v2
arm64_gic_irq_set_priority(
  A64_IRQ_PH_EINT,  // Interrupt Number for Port PH: 53
  0,                // Interrupt Priority
  IRQ_TYPE_EDGE     // Trigger on Low-High Transition
);

(Source)

IRQ_TYPE_EDGE means that the interrupt is triggered on Low-High Transitions.

When we tested this, the Touch Panel triggers an interrupt only once.

(We’re not sure why it doesn’t trigger further interrupts)

Let’s try out IRQ_TYPE_LEVEL, which triggers an interrupt by Low-High Level…

// Set Interrupt Priority in Generic Interrupt Controller v2
arm64_gic_irq_set_priority(
  A64_IRQ_PH_EINT,  // Interrupt Number for Port PH: 53
  2,                // Interrupt Priority
  IRQ_TYPE_LEVEL    // Trigger by Level
);

What happens when we run it?

When we run the code, it generates a never-ending stream of “.” characters…

Without us touching the screen! (Pic above)

Is this a bad thing?

Yes it’s terrible! This means that the Touch Panel fires Touch Input Interrupts continuously…

NuttX will be overwhelmed handling Touch Input Interrupts 100% of the time. No time for other tasks!

What if we limit the interrupts?

Previously we tried throttling the interrupts from the Touch Panel. We disable the Touch Panel Interrupt if we’re still waiting for it to be processed: gt9xx.c

// Interrupt Handler for Touch Panel, with Throttling and Forwarding
static int gt9xx_isr_handler(int irq, FAR void *context, FAR void *arg) {

  // Print "." when Interrupt Handler is triggered
  up_putc('.');

  // Get the Touch Panel Device
  FAR struct gt9xx_dev_s *priv = (FAR struct gt9xx_dev_s *)arg;

  // If the Touch Panel Interrupt has not been processed...
  if (priv->int_pending) { 

    // Disable the Touch Panel Interrupt
    priv->board->irq_enable(priv->board, false); 
  }

  // Omitted: Set the Interrupt Pending Flag
  // and notify the Poll Waiters

It seems to work… But it doesn’t look right to throttle interrupts in an Interrupt Handler.

Is it OK to throttle interrupts?

Between calls to read(), our driver might fail to detect some Touch Input Events.

This happens because we throttle the Touch Panel Interrupts, and we re-enable them only when read() is called. (Like this)

Interrupts that fire before read() will likely get ignored.

Maybe we didn’t set the Touch Panel Status correctly? Causing the Excessive Interrupts?

We checked that the Touch Panel Status was correctly set to 0 after every interrupt. Yet we’re still receiving Excessive Interrupts. (See this)

(Why does Status 0x81 change to 0x80, instead of 0?)

Thus we need to stick with IRQ_TYPE_EDGE and figure out how to trigger interrupts correctly on Touch Input.

(Perhaps by studying the Generic Interrupt Controller)

11.6 Interrupt Priority

Why did we set Interrupt Priority to 2?

// Set Interrupt Priority in Generic Interrupt Controller v2
arm64_gic_irq_set_priority(
  A64_IRQ_PH_EINT,  // Interrupt Number for Port PH: 53
  2,                // Interrupt Priority
  IRQ_TYPE_EDGE     // Trigger on Low-High Transition
);

(Source)

We set the Interrupt Priority to 2 for Legacy Reasons…

The code below comes from the Early Days of NuttX Arm64: arch/arm64/src/qemu/qemu_serial.c

// Attach Interrupt Handler for Arm64 QEMU UART
static int qemu_pl011_attach(struct uart_dev_s *dev) {
  ...
  // Set Interrupt Priority for Arm64 QEMU UART
  arm64_gic_irq_set_priority(
    sport->irq_num,  // Interrupt Number for UART
    IRQ_TYPE_LEVEL,  // Interrupt Priority is 2 ???
    0                // Interrupt Trigger by Level ???
  );

There seems to be a mix-up in the arguments: Interrupt Priority vs Interrupt Trigger.

IRQ_TYPE_LEVEL is 2, so the code above sets the Interrupt Priority to 2, with the Default Interrupt Trigger (by Level).

That’s why we configured our Touch Panel Interrupt for Priority 2, to be consistent with the existing calls.

Someday we should fix all existing calls to arm64_gic_irq_set_priority…

To this…

  // Set Interrupt Priority with Correct Arguments
  arm64_gic_irq_set_priority(
    irq_num,        // Interrupt Number
    0,              // Interrupt Priority
    IRQ_TYPE_LEVEL  // Interrupt Trigger by Level
  );

(UPDATE: Interrupt Priority is now 0 for the Touch Panel)