QMail Object Transfer v1 — Overview

Command family

Group 6 commands 76 through 84 make up Object Transfer v1. Each has its own detailed page with request/response packet diagrams:

All nine commands require TCP and AES-128 QMail authentication. A server receiving one over UDP returns ERROR_TCP_REQUIRED before allocating a request body. Every command payload follows the normal 48-byte QMail preamble and begins with the same 16-byte common prefix (protocol_version, command_header_length, flags, request_id) — documented once on each command page since the offsets are identical everywhere.

Goals

• Support objects larger than 4 GiB using unsigned 64-bit object sizes and byte offsets.
• Let the client request an upload chunk size while the server may reduce it to a configured limit.
• Let download range size differ from upload chunk size.
• Support resume, out-of-order ranges, idempotent retries, and integrity verification.
• Support atomic replacement and owner-authorized deletion while keeping a stable object ID.
• Keep storage implementation independent of the wire protocol — RAM, tmpfs, disk, and distributed storage are server policy.
• Do not encode a permanent maximum object size in the protocol.

The protocol does not promise that every QMail server accepts every object. A client selects suitable servers from capabilities (command 83) and must tolerate individual storage servers rejecting or losing a copy.

Terminology

Typical flow

A sender storing one object on one server, end to end:

1. begin (76) — declare total size, hash algorithm, storage class, retention, and (for a replace) the expected/target generation. The server reserves capacity and returns an accepted chunk size.
2. put_range (77) — repeated for each chunk, in any order, possibly concurrently. Overlapping identical bytes are accepted idempotently; overlapping different bytes are rejected.
3. transfer_status (78) — optional, used to resume after a dropped connection by listing which ranges the server already has.
4. commit (79) — once every byte is uploaded, the server verifies total size and the whole-object hash, then atomically publishes the target generation.
5. abort (80) — instead of commit, releases an incomplete transfer and its reservations.

Reading a committed object from any client, at any later time:

1. object_info (81) — look up committed metadata (size, hash, generation, expiry) for the latest or a specific generation.
2. get_range (82) — fetch byte ranges, pinned to the generation returned by object_info so a concurrent replace can never mix bytes from two generations in one reconstruction.

object_capabilities (83) can be called at any time, without an existing transfer, to learn the live server’s current limits (max object size, chunk size, active-transfer quota, storage classes). It overrides stale cached discovery data. object_delete (84) is owner-only and publishes a tombstone generation; it does not imply a payment refund.

Tell integration

QMail announces object-stored mail with the standard manifest v1 tell (manifest_version=1, 16-byte entries) — see the cmd 71 Tell spec. There is no Object-Transfer-specific manifest entry on the wire. Instead, the object key is fully derivable from the tell itself:

object_id   = the email GUID (the email_id field — first 16 bytes of the
              tell routing header, absolute bytes 48-63; mirrored at
              file-header offset 0-15)
file_type   = manifest entry file_type (0x00 meta, 0x01 body, 0x0A+ attachments)
generation  = 1 (senders always commit message objects at generation 1)

The sender uploads every message file (private meta, body, attachments) under that key via commands 76–79 before sending the tell. The receiver fetches each file via object_info (cmd 81) and get_range (cmd 82) against the servers listed in the tell’s server-location entries, reconstructs the logical file from its stripes, and verifies it against the manifest entry’s sender-authored CRC32 plus the per-stripe SHA-256 hashes returned by cmd 81. A receiver that cannot find the object (older striped-storage senders) falls back to the legacy paged download path (cmd 74; the legacy paged-storage family is cmds 70/74/75). Committing message objects at a fixed generation 1 keeps delivered mail immutable — a replacement upload would require a higher target generation, which a receiver never requests for mail objects.

Payment model (Phase I)

Phase I uses legacy_locker_marker payment mode: begin starts the existing asynchronous locker-consumption path and records the existing reuse marker. Payment is charged once per server for the tuple (authenticated owner, object_id, locker_code) and is shared by every file_type under that Object ID and independent of Transfer ID. Each stored object still receives its own capacity reservation and transfer state. Replacements using the same tuple reuse the existing payment marker; changing the locker code creates a different payment key. Deletion does not refund payment.

The payment record has durable pending, paid, and failed states. The first begin writes pending before dispatching locker consumption. Other begins for the same payment key return ERROR_PAYMENT_PROCESSING while pending, reuse it when paid, and return ERROR_PAYMENT_REQUIRED when failed. Command 83 and future DRD records advertise the payment mode and fee-schedule identifiers so a later payment mode can be added without changing transfer packet layouts.

Access control (Phase I)

• READ and owner-neutral object_info: allow-all for a caller who knows the object identifier and can make a valid encrypted QMail request.
• REPLACE, DELETE, and future SET_ACL: authenticated owner only.
• Allow-all does not mean anonymous plaintext access; transport authentication and encryption still apply.
• The server stores owner identity and ACL version now so a future phase can add account groups, resource groups, and explicit permissions without changing object ownership.

Status code range

Values 218 through 235 are assigned to Object Transfer v1 (value 217 is intentionally skipped — it is already published as RAIDA_CONNECTION). Each command page lists the specific status codes it can return; see any individual command page for the full table, e.g. object_begin status codes.

Server configuration

Limits are local policy and are not compiled into the wire protocol. A server exposes its effective values live through command 83. Example configuration keys (deployment examples, not protocol defaults):

qmail_object_transfer_enabled = false
qmail_max_object_bytes = 26843545600
qmail_preferred_chunk_bytes = 1048576
qmail_max_chunk_bytes = 8388608
qmail_max_download_range_bytes = 8388608
qmail_max_active_transfers = 256
qmail_max_active_transfers_per_identity = 8
qmail_max_reserved_bytes_per_identity = 107374182400
qmail_transfer_ttl_seconds = 86400
qmail_transfer_tombstone_ttl_seconds = 604800
qmail_default_storage_class = 1
qmail_delete_grace_seconds = 300
qmail_payment_mode = "legacy_locker_marker"

A server implements only its local storage promise. It does not calculate or coordinate the client’s horizontal, vertical, or diagonal parity. Storage class is a local service tier, not a claim about global QMail durability.