Beacon Wait — Group 22, Code 2 (proposed)

Status of this proposal

Why it is needed

Ping and Peek filter Tells by the timestamp the sender wrote into the Tell header, and the client advances its cursor to the newest such timestamp it has seen. Three defects follow from that design, and none can be fixed inside the existing packets without changing what a zero byte means to a deployed client:

1. Sender clock skew hides mail. If sender A’s clock runs ten minutes fast, a recipient who receives A’s Tell moves its cursor ten minutes into the future. Everything sender B sends in the next ten minutes carries a timestamp at or below that cursor and is never returned. The server cannot correct this because the filter value is the sender’s, not its own.
2. Capped batches strand Tells. The scan stops at 1 MiB in directory order and sets no flag; its one-byte count is a plain cast of an int, so 256 records advertise as 0 and 257 as 1. A client that advances to the newest timestamp in a cut batch never asks for the Tells that were left out. Retention does not help, because the cursor has moved past them. Re-sending the same cursor returns the same prefix, so no timestamp-only client strategy can page through a large backlog.
3. The hold time is the server’s choice. The server parks a Ping for 15–16 minutes. rest_core’s socket budget is 10 minutes with a 15-minute ceiling, so every idle Ping expires on the client side and is logged as a failure. Neither side can fix this alone: raising the client limit races the server’s sweep, lowering the server hold changes behaviour for every existing client.

Wait fixes all three at the root: the cursor is assigned by the server from its own publication order (so skew and same-second ordering do not matter), the response says how many Tells matched and returns the cursor of the last one included (so a cut batch is resumed, not skipped), and the client states its own maximum hold (so the poll always completes inside the client’s socket budget). It also folds Peek in: a flag asks the server not to park, so one command serves both cases and the client keeps one code path.

Quick reference

Wire packet

The RAIDA packet header is the 32 plaintext bytes at the front of the TCP packet. The 16-byte challenge/CRC is encrypted with the rest of the request body. raidax groups that challenge with the QMail identity block and calls those first 48 decrypted body bytes qmail_preamble_t; Wait uses the preamble unchanged so the existing decrypt, CRC and AN checks apply as they are.

QMail Wait request packet, bytes on TCP

+-------------------------------+-----------------------------------------+----------------------+
| RAIDA request header           | encrypted request body                  | plaintext terminator |
| 32 bytes                       | 60 bytes                                | 2 bytes              |
+-------------------------------+-----------------------------------------+----------------------+
| command group=22, command=2    | challenge/CRC(16)                       | 3E 3E                |
| encryption type=1              | QMail identity/auth block(32)           |                      |
| body_size=62                   | cursor(8)                               |                      |
| nonce(8)                       | max_hold_seconds(2)                     |                      |
|                                | max_records(1) flags(1)                 |                      |
+-------------------------------+-----------------------------------------+----------------------+

The 32-byte request header is identical to Ping’s except for the command group (0x16), the command code (0x02) and the body size (00 3E, 62).

Request body (62 bytes)

Decrypted request body layout

+-------------------+--------------------------+-----------+-------------+-----------+-------+------------+
| challenge/CRC     | QMail identity/auth      | cursor    | max_hold_s  | max_recs  | flags | terminator |
| 16 bytes          | 32 bytes                 | 8 bytes   | 2 bytes     | 1 byte    | 1     | 2 bytes    |
| encrypted         | encrypted                | encrypted | encrypted   | encrypted | enc.  | plaintext  |
+-------------------+--------------------------+-----------+-------------+-----------+-------+------------+
| body[0..15]       | body[16..47]             | [48..55]  | [56..57]    | [58]      | [59]  | [60..61]   |
+-------------------+--------------------------+-----------+-------------+-----------+-------+------------+

Cursor

The cursor is assigned by the beacon from its retention index, which already records when each Tell was published on this server. The recommended encoding, opaque to the client, is:

cursor (8 bytes, big-endian fields)

+-------------+----------+-------------------------------+---------------------------+
| raida_id    | index_gen| publish_time                  | publish_seq               |
| 1 byte      | 1 byte   | 4 bytes, server Unix seconds  | 2 bytes, per-second seq   |
+-------------+----------+-------------------------------+---------------------------+

The RAIDA tag stops a cursor from one beacon being replayed at another; the index generation lets the server invalidate cursors after an index rebuild. Two Tells published in the same second get different sequence numbers, so ordering is total and a page boundary inside one second is representable. A Tell republished under the same GUID (an edit) gets a new cursor and is returned again; the client deduplicates by GUID and version. Because the value is server time, sender clock skew has no effect, and because the server sorts by cursor before filling the reply, a cut batch always drops the newest Tells and the returned next_cursor lets the client resume exactly where the cut happened.

Two servers do not share cursors. A client keeps one cursor per beacon RAIDA, as rest_core already keeps one timestamp per beacon in qmail_beacon_ts.dat.

Response packet

QMail Wait success response, bytes on TCP

+-------------------------------+-----------------------------------------+----------------------+
| RAIDA response header          | encrypted payload                       | plaintext terminator |
| 32 bytes                       | 16 + records                            | 2 bytes              |
+-------------------------------+-----------------------------------------+----------------------+
| status=250, group=6            | array header(16)                        | 3E 3E                |
| body_size=payload_len+2        | record 1                                |                      |
| challenge signature            | record 2 ...                            |                      |
+-------------------------------+-----------------------------------------+----------------------+

The response header is the standard 32-byte legacy header described on the Ping page. The payload array header grows from 8 to 16 bytes; the Tell records that follow are byte-for-byte the records Ping and Peek return, so the existing record parser is reused.

Long-poll mechanics

1. Validate body length (62), terminator, decryption, challenge CRC and AN exactly as Ping does.
2. Reject unknown flags bits with ERROR_INVALID_PARAMETER.
3. Query the retention index for Tells of this mailbox with cursor > request cursor, ascending; drop those acknowledged by this device when hide_acked is set; count them as total_tells.
4. If at least one exists, copy records until max_records or the byte cap, set truncated if any were left, set next_cursor to the last included record, reply 250.
5. If none exist and no_park is set, reply 250 with tell_count = 0 at once.
6. Otherwise arm the inotify watcher before the final scan (arm, rescan once, then park) so a Tell published between the scan and the watch registration cannot be missed; give the watcher a per-connection monotonic deadline of min(max_hold_seconds, server_max_hold) and expire it at that deadline, not on a once-a-minute sweep with a 60 s slop.
7. On wake, every parked connection watching that inbox (one per device) is completed: each re-runs step 3 with its own cursor and replies as in step 4 or, if nothing matched, with tell_count = 0 and woken_empty.
8. On deadline, reply 250 with tell_count = 0, expired and the unchanged cursor. Wait never uses status 245; a reply that carries a cursor is more useful to the client than a bare status.

Status codes

A server that predates group 22 answers ERROR_INVALID_COMMAND_GROUP (5) from the dispatcher before any handler runs. A server that has the group but not this command answers ERROR_INVALID_COMMAND (6). Treat both as “unsupported here”.

Compatibility

• Existing clients: unaffected. Ping and Peek are not modified. A client that never sends group 22 sees no change.
• Existing servers: a server without group 22 answers ERROR_INVALID_COMMAND_GROUP (5). A client should send Beacon Status once per beacon per session, read its commands bit set, and fall back to Ping/Peek when Wait is absent.
• Cursor scope: a cursor is valid only on the RAIDA that issued it. A client keeps one per beacon and never sends RAIDA 11’s cursor to RAIDA 14. The server rejects a cursor whose embedded RAIDA tag does not match itself with ERROR_INVALID_PARAMETER.
• Cursor migration: a client switching from Ping to Wait sends an all-zero cursor once and deduplicates the retained backlog by GUID, or starts from newest_cursor reported by Beacon Status when it only wants mail from now on.
• Expired cursor: a cursor older than the oldest retained Tell is still valid: the server returns from the oldest retained record and sets flags.cursor_expired so the client knows a gap may exist. It never rejects a stale cursor.
• Index rebuild: publish_seq is persisted in the retention index. If the index is rebuilt from files, sequence numbers are reassigned in file order within each second, and the server bumps an index generation embedded in the cursor so that pre-rebuild cursors are treated as expired (full replay from oldest retained) rather than mis-ordered.
• Retention semantics: Wait never deletes. Per-device suppression through Ack is optional and does not shorten retention.

Common mistakes