Beacon Ack — Group 22, Code 3 (proposed)

Status of this proposal

Why it is needed

Since Tells are retained for 30 days and fetching no longer deletes them, a client sees the same notifications again on every poll that starts at or before their timestamp. Today the client hides that by deduplicating against its local database, which works but has costs:

1. Every poll on a busy mailbox carries the backlog. A fresh cursor, a cursor held back for skew, or any batch that was cut returns Tells the device already has. On a mailbox with hundreds of retained Tells that is hundreds of records per poll, against a 1 MiB cap.
2. The server cannot tell delivered from undelivered. There is no signal from any device that a Tell reached it, so the server can neither report delivery to an operator nor prioritise Tells that no device has fetched.
3. Multi-device mailboxes have no per-device state. A phone and a desktop polling the same mailbox each need their own view of “what is new to me”. Only the server can hold that view; a per-device cursor alone cannot express “I have this one but not that older one”.

Ack gives the server exactly that signal, scoped to a device, without touching the retention or the legacy commands. It is idempotent and batched so a client can send it after each local commit or once a minute.

Quick reference

Wire packet

QMail Ack request packet, bytes on TCP

+-------------------------------+-----------------------------------------+----------------------+
| RAIDA request header           | encrypted request body                  | plaintext terminator |
| 32 bytes                       | 48 + N*20 bytes                         | 2 bytes              |
+-------------------------------+-----------------------------------------+----------------------+
| command group=22, command=3    | challenge/CRC(16)                       | 3E 3E                |
| encryption type=1              | QMail identity/auth block(32)           |                      |
| body_size=50+N*20              | entry 1: email_id(16) version(4)        |                      |
| nonce(8)                       | entry 2 ... entry N                     |                      |
+-------------------------------+-----------------------------------------+----------------------+

The 32-byte request header is the standard header used by every QMail command (see the Ping page) with command group 0x16, command code 0x03 and a body size of 50 + 20N.

Request body (50 + 20N bytes)

Decrypted request body layout, N = 2 shown

+-------------------+--------------------------+------------------------+------------------------+------------+
| challenge/CRC     | QMail identity/auth      | entry 1                | entry 2                | terminator |
| 16 bytes          | 32 bytes                 | 20 bytes               | 20 bytes               | 2 bytes    |
| encrypted         | encrypted                | encrypted              | encrypted              | plaintext  |
+-------------------+--------------------------+------------------------+------------------------+------------+
| body[0..15]       | body[16..47]             | [48..67]               | [68..87]               | [88..89]   |
+-------------------+--------------------------+------------------------+------------------------+------------+

Response packet

QMail Ack success response, bytes on TCP

+-------------------------------+-----------------------------------------+----------------------+
| RAIDA response header          | encrypted payload                       | plaintext terminator |
| 32 bytes                       | N bytes                                 | 2 bytes              |
+-------------------------------+-----------------------------------------+----------------------+
| status=250, group=6            | result[0] result[1] ... result[N-1]     | 3E 3E                |
| body_size=N+2                  |                                         |                      |
+-------------------------------+-----------------------------------------+----------------------+

The whole request is authenticated once; individual entries never fail the batch. The status is 250 whenever the preamble validates, and per-entry outcomes are in the payload.

Semantics

• Scope. An acknowledgement is a row (mailbox, device_id, email_id, version) in the beacon’s retention index. It has no effect on any other device and no effect on the file itself.
• Retention. Unchanged. A Tell acknowledged by every known device still lives until its 30-day expiry. The amendment’s rule that fetch masks are not durable acknowledgements is kept: only an explicit Ack counts, and only for the device that sent it.
• Effect on Wait. A Wait from the same device with flags.hide_acked omits acknowledged versions and does not count them in total_tells. A Wait without that flag, and any Ping or Peek, behaves as if no Ack existed.
• Recovery. The acknowledgement rows are advisory. If the client loses its database it starts with a zero cursor and hide_acked = 0 and receives everything retained.
• Timing. The client sends Ack only after its local database commit, never from a callback that precedes the commit. Sending it late is safe; sending it early loses mail on a crash.
• Device retirement. A device that stops polling leaves rows that expire with the Tells they reference. No separate cleanup is needed.

Status codes

Compatibility

• Existing clients: unaffected. They never send group 22 and never set hide_acked, so they keep seeing every retained Tell.
• Existing servers: answer 5. The client treats that as “Ack unsupported here” and relies on local deduplication, exactly as today. Beacon Status reports support in advance.
• Device identity: the Device ID is authenticated in the sense that only the mailbox owner (AN holder) can send it, but it is chosen by the client. Two installs that pick the same value share one acknowledgement view. The client must generate a stable per-install value; rest_core’s hard-coded 1 is not sufficient and must change before it uses Ack.
• Legacy Ping/Peek: never consult Ack rows. This keeps the amendment’s promise that legacy Ping stays non-destructive and complete.
• Storage: one small row per (device, Tell) in the retention index; bounded by 255 devices × retained Tells per mailbox and reaped with the Tell.

Common mistakes