Merge branch 'master' of https://github.com/jjwbruijn/solum-esl-alternative-proto

README.md

@@ -1,7 +1,7 @@
 # solum-esl-alternative-proto
 
 ## ⚠️⚠️⚠️THIS IS NOT PRODUCTION READY!⚠️⚠️⚠️
-This is not a final, polished codebase. Not by a long shot. It's a complicated system, and it is not unlikely to be a dead end. There are drawbacks to a slotted protocol, it's hard to debug, and it might not work all that well in unfavourable RF conditions. You'll need some knowledge on the use of these tags. A very good place to start is here: https://github.com/atc1441/ZBS_Flasher. You'll need to fix issues yourself, troubleshoot stuff. Once again: this is not for everyone.
+This is not a final, polished codebase. Not by a long shot. You'll need some knowledge on the use of these tags. A very good place to start is here: https://github.com/atc1441/ZBS_Flasher. You'll need to fix issues yourself, troubleshoot stuff. Once again: this is not for everyone.
 
 This is an alternative protocol for the ZBS243-based Electronic Shelf Labels - ESL / price tags by Solum / Samsung.
 
@@ -12,10 +12,11 @@ It is currently compatible with the following tags:
 * 1.54"
 
 ### Aims
-- Low power (currently around 8µA with 30 second latency)
-- Low latency (tags can check for new data every 30 seconds)
+- Low power (currently around 9µA with a minimum of 40 second latency)
+- Even lower power when there's no AP around
+- Low latency (tags can check for new data every 40 seconds)
 - High transfer speeds - It can do about 5kbyte/s in favorable RF conditions. This allows for lower power
-- RF-spectrum friendly - We don't need to acknowledge EVERY packet, and we don't need to transfer data we already have
+- RF-friendly - We don't need to acknowledge EVERY packet, and we don't need to transfer data we already have
 
 The entire setup requires a few tags, and an ESP32. A (preferably, but not necessarily) broken tag is used as an 802.15.4 radio for the ESP32. You'll need a ZBS_Flasher in order to flash both the AP with its firmware, and the tags. Using the 'mac' option on ZBS_Flasher makes sure a tag flashed with a custom firmware has a valid mac address; it used the stock mac address assigned to the tag if it hasn't been flashed before. If you want to set it yourself, you can edit the mac address in the infopage. The AP expects a tag with a mac that starts with 00:00, followed by 6 bytes. The MAC-address also needs to be set on the AP-tag.
 
@@ -23,15 +24,10 @@ Once flashed, you can hook the AP tag up to the ESP32 by connecting the tags ser
 
 You can access the ESP32 with any web browser after connecting it to your WiFi Network. The file browser is located at <ip>/edit. For sending data to tags, you'll need to upload the information in 'data' to the ESP32's filesystem. After uploading, you can access the status screen at <ip>/index.html. If everything is working, you should be able to see tags synchronising to the network. After uploading a suitable .bmp file to the filesystem, this file can be sent to the tag by entering it's 6-byte mac address and filename.
 
-## The synchronized/slotted protocol explained, kinda. This is simplified, the reality is even more convoluted
-
-- Every 30 seconds, the AP sends a 250ms long 'burst' of packets, each marked with a sequential 802.15.4 sequence number. The tag, when synchronized to the network, listens for a short while every 30 seconds, to receive exactly one packet. Based on the sequence number received by the tag, the tag can calculate clock drift and window-offset to the 30 seconds interval, in order to remain synchronized.
-- At bootup, the tag will try to find an access point, and get timing information. It will start a calibration cycle in order to characterize the difference between the AP's clock and the internal (RC) oscillator. This allows for a fairly precise reception of the sync burst. After initial calibration, the tag will slowly adjust its calibration value to compensate for drift in the RC oscillator frequency. 
-- When synced, the tag only listens for a -very- short period of time, which allows for both low power use and low latency.
-- The ESP32 can send a 'pendingData' message to the AP, indicating it wants a specific tag to check-in to the AP.
-- AP adds this pendingData struc to a queue, and adds these tags to the sync-burst
-- The sync-burst packets contain up to (currently) 2 mac addresses of tags that the AP wants to talk to, and an 'offset'. Tag receives a MAC address, see if it matches its own, and when it does, it sleeps for [offset] ms. This allows for minimal power consumption and maximum throughput, as the AP determines which tags are allowed to talk.
-- When addressed by AP in the sync-burst, the tag wakes up after the offset period, and requests information from the AP. The AP responds with some metadata, such as size, MD5 checksum, and data type.
+## The protocol explained
+- The tag checks in with the AP every 40+ seconds. Actual check-in interval is highly dependent on RF conditions
+- The AP holds a list with tag MAC's that have pending transfers.
+- If a tag checks in, the AP replies with either no data, or information about a pending transfer
 - The tag checks if this information is already downloaded to EEPROM, or is already displayed. If this is the case, the transfer is immediately cancelled by issuing a 'transfer complete' packet to the AP.
 - The tag then proceeds to request data in 'blocks' of 4096 bytes. The AP responds with an ACK on the request, and specifies how long it will spend to gather the data. The tag sleeps until the AP will send the data
 - The AP requests its block-buffer to be filled by the ESP32, specifying MD5 and blockID
@@ -50,13 +46,14 @@ You can access the ESP32 with any web browser after connecting it to your WiFi N
 - Implement battery reading
 - Implement RSSI/LQI to be sent to the AP
 ### AP:
+- Important! The AP needs to be able to tell a tag to try again later if it's already doing comms with another tag. The AP can't handle concurrent checkins/download due to memory constraints!
 - More reliable serial comms (sometimes bytes are dropped)
 - Include source mac with blockrequest struct 
 ### ESP32:
 - Do more with status info as sent by the tags
 
 ## Known issues:
-- The RC oscillator has some jitter, especially on longer sleep times. I find it difficult to have the tag sleep for much longer than 30s without losing synchronization to the network
+
 - For some reason, the screen needs to be reset and put to sleep -EVERY TIME- the tag wakes up. This is a relatively slow process; it would really help if we could find out what causes this. Some glitch on the reset line of the EPD would be my guess...
 - The ZBS CPU should be able to sleep during the EPD-draw command; however, this currently (for some reason) increases the sleep-current-draw
 - Some tags work better as AP's than others. Your range may suck. The boards on these tags are tiny and fragile. For instance, a dab of hot-glue on a board is enough to warp it pretty severely, and will damage the components that are soldered on there. Reportedly, segmented-display solum tags work well. 

ap_fw/main.c

@@ -49,6 +49,7 @@ struct MacFrameBcast {
 #define PKT_BLOCK_PART 0xE8
 #define PKT_XFER_COMPLETE 0xEA
 #define PKT_XFER_COMPLETE_ACK 0xEB
+#define PKT_CANCEL_XFER 0xEC
 
 struct AvailDataReq {
     uint8_t checksum;
@@ -113,7 +114,6 @@ struct blockRequest {
 struct blockRequestAck {
     uint8_t checksum;
     uint16_t pleaseWaitMs;
-    uint8_t cancelXfer;
 } __packed;
 
 struct espPendingData {
@@ -168,8 +168,10 @@ extern bool __idata serialBypassActive;               // if the serial bypass is
 uint32_t __xdata nextBlockAttempt = 0;                // reference time for when the AP can request a new block from the ESP32
 uint8_t seq = 0;                                      // holds current sequence number for transmission
 uint8_t __xdata blockbuffer[BLOCK_XFER_BUFFER_SIZE];  // block transfer buffer
+uint8_t lastAckMac[8] = {0};
 
 void sendXferCompleteAck(uint8_t *dst);
+void sendCancelXfer(uint8_t *dst);
 
 // tools
 void addCRC(void *p, uint8_t len) {
@@ -231,7 +233,9 @@ uint8_t __xdata getPacketType(void *__xdata buffer) {
 int8_t findSlotForMac(const uint8_t *mac) {
     for (uint8_t __xdata c = 0; c < MAX_PENDING_MACS; c++) {
         if (u64_isEq((uint64_t __xdata *)mac, (uint64_t __xdata *)&(pendingDataArr[c].targetMac))) {  // this costs 1 sloc :(
-            return c;
+            if (pendingDataArr[c].attemptsLeft != 0) {
+                return c;
+            }
         }
     }
     return -1;
@@ -244,10 +248,27 @@ int8_t findFreeSlot() {
     }
     return -1;
 }
+int8_t findSlotForVer(const uint8_t *ver) {
+    for (uint8_t __xdata c = 0; c < MAX_PENDING_MACS; c++) {
+        // if (u64_isEq((uint64_t __xdata *)ver, (uint64_t __xdata *)&(pendingDataArr[c].availdatainfo.dataVer))) {
+        if (memcmp(ver, ((uint8_t __xdata *)&(pendingDataArr[c].availdatainfo.dataVer)), 8) == 0) {
+            if (pendingDataArr[c].attemptsLeft != 0) return c;
+        }
+    }
+    return -1;
+}
+void deleteAllPendingDataForVer(const uint8_t *ver) {
+    int8_t slot = -1;
+    do {
+        slot = findSlotForVer(ver);
+        if (slot != -1) pendingDataArr[slot].attemptsLeft = 0;
+    } while (slot != -1);
+}
 
 // processing serial data
 #define ZBS_RX_WAIT_HEADER 0
 #define ZBS_RX_WAIT_SDA 1
+#define ZBS_RX_WAIT_CANCEL 2
 void processSerial(uint8_t lastchar) {
     // uartTx(lastchar); echo
     switch (RXState) {
@@ -263,6 +284,12 @@ void processSerial(uint8_t lastchar) {
                 serialbufferp = serialbuffer;
                 break;
             }
+            if (strncmp(cmdbuffer, "CXD>", 4) == 0) {
+                RXState = ZBS_RX_WAIT_CANCEL;
+                bytesRemain = sizeof(struct pendingData);
+                serialbufferp = serialbuffer;
+                break;
+            }
             if (strncmp(cmdbuffer, "VER?", 4) == 0) {
                 pr("VER>%04X\n", version);
             }
@@ -294,6 +321,22 @@ void processSerial(uint8_t lastchar) {
                 RXState = ZBS_RX_WAIT_HEADER;
             }
             break;
+        case ZBS_RX_WAIT_CANCEL:
+            *serialbufferp = lastchar;
+            serialbufferp++;
+            bytesRemain--;
+            if (bytesRemain == 0) {
+                if (checkCRC(serialbuffer, sizeof(struct pendingData))) {
+                    struct pendingData *pd = (struct pendingData *)serialbuffer;
+                    deleteAllPendingDataForVer((uint8_t *)&pd->availdatainfo.dataVer);
+                    pr("ACK>\n");
+                } else {
+                    pr("NOK>\n");
+                }
+
+                RXState = ZBS_RX_WAIT_HEADER;
+            }
+            break;
     }
 }
 
@@ -348,7 +391,16 @@ void processBlockRequest(const uint8_t *buffer, uint8_t forceBlockDownload) {
     struct MacFrameNormal *__xdata rxHeader = (struct MacFrameNormal *)buffer;
     struct blockRequest *__xdata blockReq = (struct blockRequest *)(buffer + sizeof(struct MacFrameNormal) + 1);
     if (!checkCRC(blockReq, sizeof(struct blockRequest))) return;
-    // todo: actually do something with the block request
+
+
+    // check if we have data for this mac
+    if(findSlotForMac(rxHeader->src)==-1){
+        // no data for this mac, politely tell it to fuck off
+        sendCancelXfer(rxHeader->src);
+        return;
+    }
+
+
     bool __xdata requestDataDownload = false;
     if ((blockReq->blockId != requestedData.blockId) || (!u64_isEq((const uint64_t __xdata *)&blockReq->ver, (const uint64_t __xdata *)&requestedData.ver))) {
         // requested block isn't already in the buffer
@@ -390,8 +442,6 @@ void processBlockRequest(const uint8_t *buffer, uint8_t forceBlockDownload) {
     }
     blockStartTimer = timerGet() + blockRequestAck->pleaseWaitMs * TIMER_TICKS_PER_MS;
 
-    blockRequestAck->cancelXfer = 0;
-
     memcpy(txHeader->src, mSelfMac, 8);
     memcpy(txHeader->dst, rxHeader->src, 8);
 
@@ -437,7 +487,7 @@ void processAvailDataReq(uint8_t *buffer) {
         return;
 
     // prepare tx buffer to send a response
-    memset(radiotxbuffer, 0, sizeof(struct MacFrameNormal)+sizeof(struct AvailDataInfo)+2);//120);
+    memset(radiotxbuffer, 0, sizeof(struct MacFrameNormal) + sizeof(struct AvailDataInfo) + 2);  // 120);
     struct MacFrameNormal *txHeader = (struct MacFrameNormal *)(radiotxbuffer + 1);
     struct AvailDataInfo *availDataInfo = (struct AvailDataInfo *)(radiotxbuffer + sizeof(struct MacFrameNormal) + 2);
     radiotxbuffer[0] = sizeof(struct MacFrameNormal) + 1 + sizeof(struct AvailDataInfo) + RAW_PKT_PADDING;
@@ -468,14 +518,18 @@ void processAvailDataReq(uint8_t *buffer) {
     txHeader->seq = seq++;
     addCRC(availDataInfo, sizeof(struct AvailDataInfo));
     radioTx(radiotxbuffer);
+    memset(lastAckMac, 0, 8);  // reset lastAckMac, so we can record if we've received exactly one ack packet
     espNotifyAvailDataReq(availDataReq, rxHeader->src);
 }
 void processXferComplete(uint8_t *buffer) {
     struct MacFrameNormal *rxHeader = (struct MacFrameNormal *)buffer;
     sendXferCompleteAck(rxHeader->src);
-    espNotifyXferComplete(rxHeader->src);
-    int8_t slot = findSlotForMac(rxHeader->src);
-    if (slot != -1) pendingDataArr[slot].attemptsLeft = 0;
+    if (memcmp(lastAckMac, rxHeader->src, 8) != 0) {
+        xMemCopyShort((void *__xdata)lastAckMac, (void *__xdata)rxHeader->src, 8);
+        espNotifyXferComplete(rxHeader->src);
+        int8_t slot = findSlotForMac(rxHeader->src);
+        if (slot != -1) pendingDataArr[slot].attemptsLeft = 0;
+    }
 }
 
 // send block data to the tag
@@ -525,6 +579,21 @@ void sendXferCompleteAck(uint8_t *dst) {
     frameHeader->pan = dstPan;
     radioTx(radiotxbuffer);
 }
+void sendCancelXfer(uint8_t *dst) {
+    struct MacFrameNormal *frameHeader = (struct MacFrameNormal *)(radiotxbuffer + 1);
+    memset(radiotxbuffer + 1, 0, sizeof(struct blockPart) + sizeof(struct MacFrameNormal));
+    radiotxbuffer[sizeof(struct MacFrameNormal) + 1] = PKT_CANCEL_XFER;
+    radiotxbuffer[0] = sizeof(struct MacFrameNormal) + 1 + RAW_PKT_PADDING;
+    memcpy(frameHeader->src, mSelfMac, 8);
+    memcpy(frameHeader->dst, dst, 8);
+    frameHeader->fcs.frameType = 1;
+    frameHeader->fcs.panIdCompressed = 1;
+    frameHeader->fcs.destAddrType = 3;
+    frameHeader->fcs.srcAddrType = 3;
+    frameHeader->seq = seq++;
+    frameHeader->pan = dstPan;
+    radioTx(radiotxbuffer);
+}
 
 // main loop
 void main(void) {

ap_fw/soc/zbs243/uart.c

@@ -21,7 +21,6 @@ void uartInit(void) {
 }
 
 extern uint8_t __xdata blockbuffer[];
-extern bool __xdata blockRequestInProgress;
 
 volatile uint8_t txtail = 0;
 volatile uint8_t txhead = 0;
@@ -79,7 +78,6 @@ void UART_IRQ1(void) __interrupt(0) {
             *blockp++ = UARTBUF;
             if (blockp == (blockbuffer+4100)) {
                 serialBypassActive = false;
-                blockRequestInProgress = false;
             }
         } else {
             rxbuf[rxhead] = UARTBUF;

tag_fw/syncedproto.c

@@ -49,8 +49,7 @@ struct MacFrameBcast {
     uint8_t src[8];
 } __packed;
 
-#define PKT_TIMING_REQ 0xE0
-#define PKT_TIMING_RESPONSE 0xE1
+
 #define PKT_AVAIL_DATA_REQ 0xE5
 #define PKT_AVAIL_DATA_INFO 0xE6
 #define PKT_BLOCK_PARTIAL_REQUEST 0xE7
@@ -59,17 +58,7 @@ struct MacFrameBcast {
 #define PKT_BLOCK_PART 0xE8
 #define PKT_XFER_COMPLETE 0xEA
 #define PKT_XFER_COMPLETE_ACK 0xEB
-#define PKT_SYNC_BURST 0xEF
-
-struct timingResponse {
-    uint8_t checksum;
-    uint32_t burstInterval;           // time between burst-start
-    uint8_t burstLength;              // in packets; due to use of sequence field, limited to a 256-packet burst
-    uint16_t burstLengthMs;           // burst length in ms
-    uint32_t timerValue;              // current timer value (used to sync up other RC timers/oscillators)
-    uint32_t burstIntervalRemaining;  // time until the next sync burst
-    uint8_t dataAvailable;
-} __packed;
+#define PKT_CANCEL_XFER 0xEC
 
 struct AvailDataReq {
     uint8_t checksum;
@@ -129,11 +118,9 @@ struct blockRequest {
 struct blockRequestAck {
     uint8_t checksum;
     uint16_t pleaseWaitMs;
-    uint8_t cancelXfer;
 } __packed;
 
 #define TIMER_TICKS_PER_MS 1333UL
-#define RX_WINDOW_SIZE 10UL  // ms
 // #define DEBUGBLOCKS
 
 // download-stuff
@@ -149,26 +136,27 @@ uint32_t __xdata curHighSlotId = 0;
 uint8_t __xdata nextImgSlot = 0;
 uint8_t __xdata imgSlots = 0;
 
-// stuff we need to keep track of because of the network
+// stuff we need to keep track of related to the network/AP
 uint8_t __xdata APmac[8] = {0};
 uint16_t __xdata APsrcPan = 0;
 uint8_t __xdata mSelfMac[8] = {0};
 uint8_t __xdata seq = 0;
 
-// timing stuff
-bool __xdata isSynced = false;  // are we synced to the network?
-#define SYNC_CALIB_TIME 10000UL
-#define MAX_NEXT_ATTEMPT_DELAY_MS 300000UL
-#define MIN_NEXT_ATTEMPT_DELAY_MS 1000UL   // wait at least this time
-uint32_t __xdata nextAttempt = 1000;       // delay before next attempt;
-bool __xdata isCalibrated = false;         // is the RC oscillator calibrated against the AP?
-uint16_t __xdata calib = SYNC_CALIB_TIME;  // RC oscillator calibration value
-uint32_t __xdata APburstInterval = 0;
-int8_t __xdata calibVector = 0;
-uint16_t __xdata APburstLengthMs = 0;  // burst length in ms
-uint8_t __xdata APburstLength = 0;
+// power saving algorithm
+#define INTERVAL_BASE 40                                           // interval (in seconds) (when 1 packet is sent/received) for target current (7.2µA)
+#define INTERVAL_AT_MAX_ATTEMPTS 600                               // interval (in seconds) (at max attempts) for target average current
+#define INTERVAL_NO_SIGNAL 1800                                    // interval (in seconds) when no answer for POWER_SAVING_SMOOTHING attempts,
+                                                                   // (INTERVAL_AT_MAX_ATTEMPTS * POWER_SAVING_SMOOTHING) seconds
+#define DATA_REQ_RX_WINDOW_SIZE 5UL                                // How many milliseconds we should wait for a packet during the data_request.
+                                                                   // If the AP holds a long list of data for tags, it may need a little more time to lookup the mac address
+#define DATA_REQ_MAX_ATTEMPTS 14                                   // How many attempts (at most) we should do to get something back from the AP
+#define POWER_SAVING_SMOOTHING 8                                   // How many samples we should use to smooth the data request interval
+#define MINIMUM_INTERVAL 45                                        // IMPORTANT: Minimum interval for check-in; this determines overal battery life!
+uint16_t __xdata dataReqAttemptArr[POWER_SAVING_SMOOTHING] = {0};  // Holds the amount of attempts required per data_req/check-in
+uint8_t __xdata dataReqAttemptArrayIndex = 0;
+uint8_t __xdata dataReqLastAttempt = 0;
 
-// buffer we use to prepare packets
+// buffer we use to prepare/read packets
 // static uint8_t __xdata mRxBuf[130];
 static uint8_t __xdata inBuffer[128] = {0};
 static uint8_t __xdata outBuffer[128] = {0};
@@ -187,13 +175,6 @@ uint8_t __xdata getPacketType(void *__xdata buffer) {
     }
     return 0;
 }
-uint32_t __xdata getNextAttemptDelay() {
-    nextAttempt *= 2;
-    if (nextAttempt > MAX_NEXT_ATTEMPT_DELAY_MS) {
-        nextAttempt = MAX_NEXT_ATTEMPT_DELAY_MS;
-    }
-    return nextAttempt;
-}
 void dump(uint8_t *__xdata a, uint16_t __xdata l) {
     pr("\n        ");
 #define ROWS 16
@@ -261,15 +242,36 @@ void initAfterWake() {
     initRadio();
 }
 void doSleep(uint32_t __xdata t) {
+    pr("s=%lu\n ", t / 1000);
     powerPortsDownForSleep();
     // sleepy
-    sleepForMsec(t * SYNC_CALIB_TIME / (uint32_t)calib);
+    sleepForMsec(t);
     initAfterWake();
 }
+uint16_t getNextSleep() {
+    uint16_t __xdata curval = INTERVAL_AT_MAX_ATTEMPTS - INTERVAL_BASE;
+    curval *= dataReqLastAttempt;
+    curval /= DATA_REQ_MAX_ATTEMPTS;
+    curval += INTERVAL_BASE;
+    dataReqAttemptArr[dataReqAttemptArrayIndex % POWER_SAVING_SMOOTHING] = curval;
+    dataReqAttemptArrayIndex++;
+
+    uint16_t avg = 0;
+    bool noNetwork = true;
+    for (uint8_t c = 0; c < POWER_SAVING_SMOOTHING; c++) {
+        avg += dataReqAttemptArr[c];
+        if (dataReqAttemptArr[c] != INTERVAL_AT_MAX_ATTEMPTS) {
+            noNetwork = false;
+        }
+    }
+    if (noNetwork == true) return INTERVAL_NO_SIGNAL;
+    avg /= POWER_SAVING_SMOOTHING;
+    return avg;
+}
 
 // data xfer stuff
 void sendAvailDataReq() {
-    struct MacFrameBcast __xdata *txframe = (struct MacFrameBcast *)(outBuffer+1);
+    struct MacFrameBcast __xdata *txframe = (struct MacFrameBcast *)(outBuffer + 1);
     memset(outBuffer, 0, sizeof(struct MacFrameBcast) + sizeof(struct AvailDataReq) + 2 + 4);
     struct AvailDataReq *__xdata availreq = (struct AvailDataReq *)(outBuffer + 2 + sizeof(struct MacFrameBcast));
     outBuffer[0] = sizeof(struct MacFrameBcast) + sizeof(struct AvailDataReq) + 2 + 2;
@@ -290,9 +292,9 @@ void sendAvailDataReq() {
 }
 struct AvailDataInfo *__xdata getAvailDataInfo() {
     uint32_t __xdata t;
-    for (uint8_t c = 0; c < 15; c++) {
+    for (uint8_t c = 0; c < DATA_REQ_MAX_ATTEMPTS; c++) {
         sendAvailDataReq();
-        t = timerGet() + (TIMER_TICKS_PER_MS * 10UL);
+        t = timerGet() + (TIMER_TICKS_PER_MS * DATA_REQ_RX_WINDOW_SIZE);
         while (timerGet() < t) {
             int8_t __xdata ret = commsRxUnencrypted(inBuffer);
             if (ret > 1) {
@@ -301,24 +303,28 @@ struct AvailDataInfo *__xdata getAvailDataInfo() {
                         struct MacFrameNormal *__xdata f = (struct MacFrameNormal *)inBuffer;
                         memcpy(APmac, f->src, 8);
                         APsrcPan = f->pan;
+                        // pr("RSSI: %d\n", commsGetLastPacketRSSI());
+                        // pr("LQI: %d\n", commsGetLastPacketLQI());
+                        dataReqLastAttempt = c;
                         return (struct AvailDataInfo *)(inBuffer + sizeof(struct MacFrameNormal) + 1);
                     }
                 }
             }
         }
     }
+    dataReqLastAttempt = DATA_REQ_MAX_ATTEMPTS;
     return NULL;
 }
-void processBlockPart(struct blockPart *bp) {
+bool processBlockPart(struct blockPart *bp) {
     uint16_t __xdata start = bp->blockPart * BLOCK_PART_DATA_SIZE;
     uint16_t __xdata size = BLOCK_PART_DATA_SIZE;
     // validate if it's okay to copy data
     if (bp->blockId != curBlock.blockId) {
         // pr("got a packet for block %02X\n", bp->blockId);
-        return;
+        return false;
     }
-    if (start >= (sizeof(blockXferBuffer) - 1)) return;
-    if (bp->blockPart > BLOCK_MAX_PARTS) return;
+    if (start >= (sizeof(blockXferBuffer) - 1)) return false;
+    if (bp->blockPart > BLOCK_MAX_PARTS) return false;
     if ((start + size) > sizeof(blockXferBuffer)) {
         size = sizeof(blockXferBuffer) - start;
     }
@@ -327,6 +333,9 @@ void processBlockPart(struct blockPart *bp) {
         xMemCopy((void *)(blockXferBuffer + start), (const void *)bp->data, size);
         // we don't need this block anymore, set bit to 0 so we don't request it again
         curBlock.requestedParts[bp->blockPart / 8] &= ~(1 << (bp->blockPart % 8));
+        return true;
+    } else {
+        return false;
     }
 }
 bool blockRxLoop(uint32_t timeout) {
@@ -338,9 +347,8 @@ bool blockRxLoop(uint32_t timeout) {
         int8_t __xdata ret = commsRxUnencrypted(inBuffer);
         if (ret > 1) {
             if (getPacketType(inBuffer) == PKT_BLOCK_PART) {
-                success = true;
                 struct blockPart *bp = (struct blockPart *)(inBuffer + sizeof(struct MacFrameNormal) + 1);
-                processBlockPart(bp);
+                success = processBlockPart(bp);
             }
         }
     }
@@ -351,7 +359,6 @@ bool blockRxLoop(uint32_t timeout) {
 struct blockRequestAck *__xdata continueToRX() {
     struct blockRequestAck *ack = (struct blockRequestAck *)(inBuffer + sizeof(struct MacFrameNormal) + 1);
     ack->pleaseWaitMs = 0;
-    ack->cancelXfer = 0;
     return ack;
 }
 void sendBlockRequest() {
@@ -359,7 +366,8 @@ void sendBlockRequest() {
     struct MacFrameNormal *__xdata f = (struct MacFrameNormal *)(outBuffer + 1);
     struct blockRequest *__xdata blockreq = (struct blockRequest *)(outBuffer + 2 + sizeof(struct MacFrameNormal));
     outBuffer[0] = sizeof(struct MacFrameNormal) + sizeof(struct blockRequest) + 2 + 2;
-    if (requestPartialBlock) {;
+    if (requestPartialBlock) {
+        ;
         outBuffer[sizeof(struct MacFrameNormal) + 1] = PKT_BLOCK_PARTIAL_REQUEST;
     } else {
         outBuffer[sizeof(struct MacFrameNormal) + 1] = PKT_BLOCK_REQUEST;
@@ -398,10 +406,12 @@ struct blockRequestAck *__xdata performBlockRequest() {
                         break;
                     case PKT_BLOCK_PART:
                         // block already started while we were waiting for a get block reply
-                        //pr("!");
-                        //processBlockPart((struct blockPart *)(inBuffer + sizeof(struct MacFrameNormal) + 1));
+                        // pr("!");
+                        // processBlockPart((struct blockPart *)(inBuffer + sizeof(struct MacFrameNormal) + 1));
                         return continueToRX();
                         break;
+                    case PKT_CANCEL_XFER:
+                        return NULL;
                     default:
                         pr("pkt w/type %02X\n", getPacketType(inBuffer));
                         break;
@@ -411,7 +421,7 @@ struct blockRequestAck *__xdata performBlockRequest() {
         } while (timerGet() < t);
     }
     return continueToRX();
-    //return NULL;
+    // return NULL;
 }
 void sendXferCompletePacket() {
     memset(outBuffer, 0, sizeof(struct MacFrameNormal) + 2 + 4);
@@ -451,6 +461,15 @@ void sendXferComplete() {
     pr("XFC NACK!\n");
     return;
 }
+bool validateBlockData() {
+    struct blockData *bd = (struct blockData *)blockXferBuffer;
+    // pr("expected len = %04X, checksum=%04X\n", bd->size, bd->checksum);
+    uint16_t t = 0;
+    for (uint16_t c = 0; c < bd->size; c++) {
+        t += bd->data[c];
+    }
+    return bd->checksum == t;
+}
 
 const uint8_t epd_bitmap_ant[] = {
     0xff, 0xbf, 0xfd, 0xdf, 0xfe, 0xdf, 0xee, 0xdf, 0xee, 0xdf, 0xed, 0xdf, 0xd7, 0xbf, 0xd7, 0xff,
@@ -529,7 +548,6 @@ void saveUpdateBlockData(uint8_t blockId) {
     if (!eepromWrite(EEPROM_UPDATA_AREA_START + (blockId * BLOCK_DATA_SIZE), blockXferBuffer + sizeof(struct blockData), BLOCK_DATA_SIZE))
         pr("EEPROM write failed\n");
 }
-
 void saveImgBlockData(uint8_t blockId) {
     uint16_t length = EEPROM_IMG_EACH - (sizeof(struct EepromImageHeader) + (blockId * BLOCK_DATA_SIZE));
     if (length > 4096) length = 4096;
@@ -562,19 +580,9 @@ uint32_t getHighSlotId() {
     return temp;
 }
 
-bool validateBlockData() {
-    struct blockData *bd = (struct blockData *)blockXferBuffer;
-    // pr("expected len = %04X, checksum=%04X\n", bd->size, bd->checksum);
-    uint16_t t = 0;
-    for (uint16_t c = 0; c < bd->size; c++) {
-        t += bd->data[c];
-    }
-    return bd->checksum == t;
-}
-
 #define DEBUGBLOCKS
 // Main download function
-void doDataDownload(struct AvailDataInfo* __xdata avail){
+void doDataDownload(struct AvailDataInfo *__xdata avail) {
     // this is the main function for the download process
 
     if (!eepromInit()) {  // we'll need the eeprom here, init it.
@@ -664,9 +672,9 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
 
     // do transfer!
     uint8_t __xdata blockRequestAttempt = 0;
+    uint8_t __xdata blockValidateAttempt = 0;
     while (!curXferComplete) {
         // this while loop loops until the transfer has been completed, or we get tired for other reasons
-        blockRequestAttempt = 0;
     startdownload:;
 #ifdef DEBUGBLOCKS
         pr("REQ %d[", curBlock.blockId);
@@ -681,20 +689,15 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
         pr("]\n");
 #endif
 
-        //timerDelay(TIMER_TICKS_PER_MS*100);
+        // timerDelay(TIMER_TICKS_PER_MS*100);
 
         // DO BLOCK REQUEST - request a block, get an ack with timing info (hopefully)
         struct blockRequestAck *__xdata ack = performBlockRequest();
         if (ack == NULL) {
-            pr("no reply on gblockrequest\n");
-            // didn't get an ack :( we'll probably try again later
-            return;
-        } else if (ack->cancelXfer == 1) {
-            // we were asked to cancel the transfer by the AP
-            pr("transfer cancelled by AP\n");
+            pr("Cancelled request\n");
             return;
         } else {
-            // got an ack
+            // got an ack!
         }
 
         // SLEEP - until the AP is ready with the data
@@ -711,14 +714,14 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
         }
 
         // BLOCK RX LOOP - receive a block, until the timeout has passed
-        if (!blockRxLoop(440)) { // was 340
+        if (!blockRxLoop(440)) {  // was 340
             // didn't receive packets
             blockRequestAttempt++;
             if (blockRequestAttempt > 5) {
                 pr("bailing on download, 0 blockparts rx'd\n");
                 return;
             } else {
-                //goto startdownload;
+                // goto startdownload;
             }
         } else {
             // successfull block RX loop
@@ -747,7 +750,8 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
         if (blockComplete) {
             if (validateBlockData()) {
                 // checked and found okay
-                requestPartialBlock = false;
+                requestPartialBlock = false; // next block is going to be requested from the ESP32 by the AP
+                blockValidateAttempt = 0;
                 switch (curBlock.type) {
                     case DATATYPE_IMG:
                     case DATATYPE_IMGRAW:
@@ -759,6 +763,11 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
                 }
             } else {
                 // block checked, but failed validation. Mark all parts for this block as 'request'
+                blockValidateAttempt++;
+                if (blockValidateAttempt > 5) {
+                    pr("bailing on download, 0 blockparts rx'd\n");
+                    return;
+                }
                 for (uint8_t c = 0; c < partsThisBlock; c++) {
                     curBlock.requestedParts[c / 8] |= (1 << (c % 8));
                 }
@@ -815,7 +824,7 @@ void doDataDownload(struct AvailDataInfo* __xdata avail){
                         sendXferComplete();
                         killRadio();
                         eepromReadStart(EEPROM_UPDATA_AREA_START);
-                        //wdtDeviceReset();
+                        // wdtDeviceReset();
                         selfUpdate();
 
                         break;
@@ -856,8 +865,6 @@ void mainProtocolLoop(void) {
     // i2ctest();
 
     pr("BOOTED> (new version!)\n\n");
-    isSynced = false;
-    isCalibrated = false;
 
     if (!eepromInit()) {
         pr("failed to init eeprom\n");
@@ -869,20 +876,27 @@ void mainProtocolLoop(void) {
         curHighSlotId = getHighSlotId();
     }
 
+    // initialize attempt-array with the default value;
+    for (uint8_t c = 0; c < POWER_SAVING_SMOOTHING; c++) {
+        dataReqAttemptArr[c] = INTERVAL_BASE;
+    }
+
     screenSleep();
     eepromDeepPowerDown();
     initRadio();
     // drawPartial();
     // i2ctest();
-
+    // doSleep(10000);
     while (1) {
         radioRxEnable(true, true);
         struct AvailDataInfo *__xdata avail = getAvailDataInfo();
-        if(avail->dataType!=DATATYPE_NOUPDATE){
-            doDataDownload(avail);
+        if (avail == NULL) {
         } else {
-            pr("nothing.\n");
+            if (avail->dataType != DATATYPE_NOUPDATE) {
+                doDataDownload(avail);
+            } else {
+            }
         }
-        doSleep(10000);
+        doSleep(getNextSleep() * 1000UL);
     }
 }