[GH-ISSUE #420] 2.75 on Lilygo T-panel freezes after a couple of minutes, requires reboot #2459

Closed
opened 2026-03-20 21:06:44 +01:00 by sascha_hemi · 16 comments
Owner

Originally created by @DJFliX on GitHub (Dec 25, 2024).
Original GitHub issue: https://github.com/OpenEPaperLink/OpenEPaperLink/issues/420

Describe the bug
I received a new T-Panel that I wanted to use to replace my Mini AP v2. However most tags I used with the old AP either don´t connect at all ("No AP found") or connect once/twice and never check in again. By turning on my old AP I can see that all tags prefer to connect to the old AP even if the old AP is two floors away and the T-Panel is in the same room. The OEPL web UI is not available and the TFT on the T-Panel stops being updated after a minute or two.

To Reproduce

  1. Install OpenEPaperlink firmware for the S3 chip from https://install.openepaperlink.org/ (I held both H2 and S3 boot buttons to make sure they don't interfere with each other just in case). S3 USB plugged in to my computer, selected T-Panel AP option and the flash was successful. It was a bit slow around 79%.
  2. Power cycle the T-Panel, move USB cable to H2 USB and kept both H2 and S3 boot buttons again. Selected T-Panel AP option again and this time the flash was a lot faster
  3. Disconnect old AP from power, connect T-Panel to power
  4. Configure WiFi on T-Panel, reboot
  5. Update S3 firmware from test5 to latest (2.75) from the OEPL UI on the T-Panel (this succeeds)
  6. Set T-Panel to display AP stats
  7. Wait for M2 (Solum 2.9 / Solum 1.54) tags to connect
  8. Wait a bit more before the T-Panel AP becomes unreachable.

Expected behavior
The tags do their 40 second checkin and occassionally skip a checkin. The OEPL web UI of the T-Panel remains available. The TFT on the T-Panel keeps updating AP statistics. Edit: the web UI thing is due to a networking issue on my end. Changing the DHCP IP reservation for the AP solved this particular issue.

Screenshots
Old AP (TagAP) showing that the T-Panel AP is discoverable, however in this state the T-Panel UI is not available and tags are not making use of the T-Panel AP any more
image
Old AP showing that the T-Panel AP (on the right) does still check in regularly:
image

Additional context
I've reflashed the firmware (both S3 and H2) a couple of times just to make sure it wasn't a faulty flash. The only thing I can think of is that there is now a mismatch between the version of the firmware flashed to the H2 (test5) and the S3 (2.75). But as far as I can tell there is no way for me to flash 2.75 directly from https://install.openepaperlink.org/.
Updates from Home Assistant to the T-Panel for tags connected to the S2 AP work fine until the T-Panel S3 locks up...

Originally created by @DJFliX on GitHub (Dec 25, 2024). Original GitHub issue: https://github.com/OpenEPaperLink/OpenEPaperLink/issues/420 **Describe the bug** I received a new T-Panel that I wanted to use to replace my Mini AP v2. However most tags I used with the old AP either don´t connect at all ("No AP found") or connect once/twice and never check in again. By turning on my old AP I can see that all tags prefer to connect to the old AP even if the old AP is two floors away and the T-Panel is in the same room. The OEPL web UI is not available and the TFT on the T-Panel stops being updated after a minute or two. **To Reproduce** 1. Install OpenEPaperlink firmware for the S3 chip from https://install.openepaperlink.org/ (I held both H2 and S3 boot buttons to make sure they don't interfere with each other just in case). S3 USB plugged in to my computer, selected T-Panel AP option and the flash was successful. It was a bit slow around 79%. 2. Power cycle the T-Panel, move USB cable to H2 USB and kept both H2 and S3 boot buttons again. Selected T-Panel AP option again and this time the flash was a lot faster 3. Disconnect old AP from power, connect T-Panel to power 4. Configure WiFi on T-Panel, reboot 5. Update S3 firmware from test5 to latest (2.75) from the OEPL UI on the T-Panel (this succeeds) 6. Set T-Panel to display AP stats 7. Wait for M2 (Solum 2.9 / Solum 1.54) tags to connect 8. Wait a bit more before the T-Panel AP becomes unreachable. **Expected behavior** The tags do their 40 second checkin and occassionally skip a checkin. The OEPL web UI of the T-Panel remains available. The TFT on the T-Panel keeps updating AP statistics. **Edit: the web UI thing is due to a networking issue on my end. Changing the DHCP IP reservation for the AP solved this particular issue.** **Screenshots** Old AP (`TagAP`) showing that the T-Panel AP is discoverable, however in this state the T-Panel UI is not available and tags are not making use of the T-Panel AP any more ![image](https://github.com/user-attachments/assets/90bc04f6-62e0-4935-8ba1-23780afc601c) Old AP showing that the T-Panel AP (on the right) does still check in regularly: ![image](https://github.com/user-attachments/assets/cb61588c-2ff5-4e61-950f-d755136b4895) **Additional context** I've reflashed the firmware (both S3 and H2) a couple of times just to make sure it wasn't a faulty flash. The only thing I can think of is that there is now a mismatch between the version of the firmware flashed to the H2 (test5) and the S3 (2.75). But as far as I can tell there is no way for me to flash 2.75 directly from https://install.openepaperlink.org/. Updates from Home Assistant to the T-Panel for tags connected to the S2 AP work fine until the T-Panel S3 locks up...
sascha_hemi added the bug label 2026-03-20 21:06:44 +01:00
Author
Owner

@nlimper commented on GitHub (Dec 25, 2024):

It sounds like you enabled Bluetooth (BLE) in the config ofvtye AP. That one is not stable currently.

<!-- gh-comment-id:2561944647 --> @nlimper commented on GitHub (Dec 25, 2024): It sounds like you enabled Bluetooth (BLE) in the config ofvtye AP. That one is not stable currently.
Author
Owner

@DJFliX commented on GitHub (Dec 25, 2024):

Hey @nlimper! Thanks for your reply. For my initial attempt this was indeed the case. However for my second attempt I only changed the settings as I described in the "how to reproduce" steps. All other settings (tft brightness, bluetooth, ap name, channel) were left untouched to eliminate config changes as a cause of this behavior.

Is it possible that the bluetooth toggle somehow caused a change that survived a flash of both the S3 and H2?

<!-- gh-comment-id:2561985259 --> @DJFliX commented on GitHub (Dec 25, 2024): Hey @nlimper! Thanks for your reply. For my initial attempt this was indeed the case. However for my second attempt I only changed the settings as I described in the "how to reproduce" steps. All other settings (tft brightness, bluetooth, ap name, channel) were left untouched to eliminate config changes as a cause of this behavior. Is it possible that the bluetooth toggle somehow caused a change that survived a flash of both the S3 and H2?
Author
Owner

@nlimper commented on GitHub (Dec 25, 2024):

If you didn't touch the config panel at all, the status of the settings is undefined. Please visit (and save) the config panel once, to be sure the BLE is not the cause.
Also, I'm not sure about the correct version of the H2 firmware. @skiphansen can tell you more about that one.

<!-- gh-comment-id:2562000832 --> @nlimper commented on GitHub (Dec 25, 2024): If you didn't touch the config panel at all, the status of the settings is undefined. Please visit (and save) the config panel once, to be sure the BLE is not the cause. Also, I'm not sure about the correct version of the H2 firmware. @skiphansen can tell you more about that one.
Author
Owner

@skiphansen commented on GitHub (Dec 26, 2024):

Version 001f is the current version. I'm not sure what version the web flasher is using
currently, hopefully that's it, @jonasniesner can verify.

AFAIK no version of the H2 firmware causes the AP Web Gui to hang, but some versions can certainly cause major communications problems with 805.15.4 tags.

I'm planning on doing some testing of OTA update support for the Lilygo over the next few days and I'll update this ticket with anything I find.

<!-- gh-comment-id:2562026812 --> @skiphansen commented on GitHub (Dec 26, 2024): Version [001f](https://github.com/OpenEPaperLink/OpenEPaperLink/blob/master/binaries/ESP32-H2/OpenEPaperLink_esp32_H2.bin) is the current version. I'm not sure what version the web flasher is using currently, hopefully that's it, @jonasniesner can verify. AFAIK no version of the H2 firmware causes the AP Web Gui to hang, but some versions can certainly cause major communications problems with 805.15.4 tags. I'm planning on doing some testing of OTA update support for the Lilygo over the next few days and I'll update this ticket with anything I find.
Author
Owner

@skiphansen commented on GitHub (Dec 26, 2024):

I followed your steps and H2 firmware version 001d was installed which definitely has problems with 805.15.4 tags. See #410 for details.

Unfortunately OTA of the H2 is not supported yet. You can use developer tools to flash the H2 or wait for @jonasniesner to update the H2 image installed by https://install.openepaperlink.org.

<!-- gh-comment-id:2562788043 --> @skiphansen commented on GitHub (Dec 26, 2024): I followed your steps and H2 firmware version 001d was installed which definitely has problems with 805.15.4 tags. See #410 for details. Unfortunately OTA of the H2 is not supported yet. You can use developer tools to flash the H2 or wait for @jonasniesner to update the H2 image installed by https://install.openepaperlink.org.
Author
Owner

@DJFliX commented on GitHub (Dec 30, 2024):

Thanks for the responses! I just tried to flash 001f to the H2 using https://espressif.github.io/esptool-js/ but I can seem to get the offset right. I tried the default and 0x10000 (as found in firmware_H2.json). Either way it ended up in a boot loop on the H2 so I'm assuming I'm just too inexperienced with ESP32 tinkering on this level and it's probably wiser to wait for the web UI to be updated to reference the current image.

<!-- gh-comment-id:2565633408 --> @DJFliX commented on GitHub (Dec 30, 2024): Thanks for the responses! I just tried to flash 001f to the H2 using https://espressif.github.io/esptool-js/ but I can seem to get the offset right. I tried the default and `0x10000` (as found in firmware_H2.json). Either way it ended up in a boot loop on the H2 so I'm assuming I'm just too inexperienced with ESP32 tinkering on this level and it's probably wiser to wait for the web UI to be updated to reference the current image.
Author
Owner

@skiphansen commented on GitHub (Dec 30, 2024):

Yes flashing it using the dev tools is tricky. I'm working on finishing up the H2 OTA support for the T-panel but the "one last bug" is taking forever.

BTW I've seen no instability as far as lockups go. Are you sure you don't have a power supply problem?

<!-- gh-comment-id:2565640721 --> @skiphansen commented on GitHub (Dec 30, 2024): Yes flashing it using the dev tools is tricky. I'm working on finishing up the H2 OTA support for the T-panel but the "one last bug" is taking forever. BTW I've seen no instability as far as lockups go. Are you sure you don't have a power supply problem?
Author
Owner

@DJFliX commented on GitHub (Dec 31, 2024):

BTW I've seen no instability as far as lockups go. Are you sure you don't have a power supply problem?

So I've tried various supplies and cables (one of them being a 65W power brick that should do 15W over the USB-A ports) and the results seemed to be consistent. However what I did notice is that my issues seem to be isolated to the IP address rather than the hardware/firmware: I initially removed the DHCP reservation for my old AP and assigned that IP to the new AP (T-panel). Then I encountered the issues and I realised I had actually been encountering the same issues with the old AP as well but I wrote it off as instability due to limited RAM available on the ESP32. Pinging also doesn't work in this state.

When the T-panel ended up not working I switched the DHCP reservation back to the old AP (which is now showing the lock-up behavior) and the T-panel is fine on a new and previously unused IP address.

Not sure what is causing this but is feels like the web UI thing is definitely a network issue on my end.

<!-- gh-comment-id:2566314897 --> @DJFliX commented on GitHub (Dec 31, 2024): > BTW I've seen no instability as far as lockups go. Are you sure you don't have a power supply problem? So I've tried various supplies and cables (one of them being a 65W power brick that should do 15W over the USB-A ports) and the results seemed to be consistent. However what I did notice is that my issues seem to be isolated to the IP address rather than the hardware/firmware: I initially removed the DHCP reservation for my old AP and assigned that IP to the new AP (T-panel). Then I encountered the issues and I realised I had actually been encountering the same issues with the old AP as well but I wrote it off as instability due to limited RAM available on the ESP32. Pinging also doesn't work in this state. When the T-panel ended up not working I switched the DHCP reservation back to the old AP (which is now showing the lock-up behavior) and the T-panel is fine on a new and previously unused IP address. Not sure what is causing this but is feels like the web UI thing is definitely a network issue on my end.
Author
Owner

@skiphansen commented on GitHub (Dec 31, 2024):

Interesting, thanks for the update.

<!-- gh-comment-id:2566520229 --> @skiphansen commented on GitHub (Dec 31, 2024): Interesting, thanks for the update.
Author
Owner

@skiphansen commented on GitHub (Jan 4, 2025):

@DJFliX I think I may have a fix for you. I've implemented OTA updates of the H2 processor on the Lilygo and it works fine for me. Installation instructions are here: https://github.com/skiphansen/OpenEPaperLink/wiki/Installing-the-Coprocessor-OTA-Beta .

Please give it a try and let me know how it goes.

<!-- gh-comment-id:2571349345 --> @skiphansen commented on GitHub (Jan 4, 2025): @DJFliX I think I may have a fix for you. I've implemented OTA updates of the H2 processor on the Lilygo and it works fine for me. Installation instructions are here: https://github.com/skiphansen/OpenEPaperLink/wiki/Installing-the-Coprocessor-OTA-Beta . Please give it a try and let me know how it goes.
Author
Owner

@DJFliX commented on GitHub (Jan 5, 2025):

@skiphansen Thank you so much for the detailed guide. Everything went as described. I disabled cache in the dev tools and followed along with the steps. Flashing the H2 worked and uodated it from 001d to 001f. And as a result the tags seem to play nice with this AP even after a couple of minutes. I'll try using this AP as "only AP" and hopefully all the tags will switch over.

<!-- gh-comment-id:2571606938 --> @DJFliX commented on GitHub (Jan 5, 2025): @skiphansen Thank you so much for the detailed guide. Everything went as described. I disabled cache in the dev tools and followed along with the steps. Flashing the H2 worked and uodated it from 001d to 001f. And as a result the tags seem to play nice with this AP even after a couple of minutes. I'll try using this AP as "only AP" and hopefully all the tags will switch over.
Author
Owner

@skiphansen commented on GitHub (Jan 5, 2025):

Excellent, that's great news!

<!-- gh-comment-id:2571656791 --> @skiphansen commented on GitHub (Jan 5, 2025): Excellent, that's great news!
Author
Owner

@minajevs commented on GitHub (Jan 6, 2025):

@skiphansen hey, a quick thanks from my end! I am a bit of a newbie in this topic, but could setup my new lilygo t-panel with 3 tags, thanks to your guide - https://github.com/skiphansen/OpenEPaperLink/wiki/Installing-the-Coprocessor-OTA-Beta .

<!-- gh-comment-id:2572949312 --> @minajevs commented on GitHub (Jan 6, 2025): @skiphansen hey, a quick thanks from my end! I am a bit of a newbie in this topic, but could setup my new lilygo t-panel with 3 tags, thanks to your guide - https://github.com/skiphansen/OpenEPaperLink/wiki/Installing-the-Coprocessor-OTA-Beta .
Author
Owner

@skiphansen commented on GitHub (Jan 6, 2025):

Thanks for the feedback and welcome to the OpenEPaperLink world.

<!-- gh-comment-id:2573115567 --> @skiphansen commented on GitHub (Jan 6, 2025): Thanks for the feedback and welcome to the OpenEPaperLink world.
Author
Owner

@DJFliX commented on GitHub (Mar 25, 2025):

Hey @nlimper! The issue has been marked as completed, but I can't find any releases that seem to include the fix. As of now I'm still on the "coprocessor beta". Do you know if this fix will be released in an upcoming oepl release?

<!-- gh-comment-id:2751202385 --> @DJFliX commented on GitHub (Mar 25, 2025): Hey @nlimper! The issue has been marked as completed, but I can't find any releases that seem to include the fix. As of now I'm still on the "coprocessor beta". Do you know if this fix will be released in an upcoming oepl release?
Author
Owner

@skiphansen commented on GitHub (Mar 25, 2025):

Yes it will be.

commit bb36185066eeee3e57c404a6338053d6ecc834af
Author: Skip Hansen <skip@gfrn.org>
Date:   Tue Dec 17 07:17:47 2024 -0800

    C6 Ver 001f - fix #410 & other C6 improvements

    1. Only call esp_ieee802154_receive_handle_done() when ack != NULL.
    2. Add 2 millisecond timeout for all SubGhz MISO wait loops to
       prevent watchdog timeouts when/if bad things happen (none observed
       during testing)
    3. Make CC1101 detection more robust and less intrusive by testing
       MISO and CSn before trying to read chip version number.
    4. Remove useless "error" messages which occur during normal operation.
    5. Added missing \r in some log messages for EOL consistency.
<!-- gh-comment-id:2751378041 --> @skiphansen commented on GitHub (Mar 25, 2025): Yes it will be. ``` commit bb36185066eeee3e57c404a6338053d6ecc834af Author: Skip Hansen <skip@gfrn.org> Date: Tue Dec 17 07:17:47 2024 -0800 C6 Ver 001f - fix #410 & other C6 improvements 1. Only call esp_ieee802154_receive_handle_done() when ack != NULL. 2. Add 2 millisecond timeout for all SubGhz MISO wait loops to prevent watchdog timeouts when/if bad things happen (none observed during testing) 3. Make CC1101 detection more robust and less intrusive by testing MISO and CSn before trying to read chip version number. 4. Remove useless "error" messages which occur during normal operation. 5. Added missing \r in some log messages for EOL consistency. ```
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/OpenEPaperLink#2459