QMail Upload — Group 6, Code 70

Quick reference

Purpose

The sender splits each QMail object into stripes using Reed-Solomon-style erasure coding (typically 7 data + 1 parity = 8 stripes). One upload request is sent to each storage RAIDA in parallel, each carrying one stripe. The recipient later downloads the stripes from those RAIDAs and reassembles the object.

An email is normally a group of multiple files under the same GUID: private CBDF meta (file_type = 0), body/content (file_type = 1), and zero to several attachments (file_type = 10..255). Each file is uploaded separately under the same GUID; the storage server distinguishes them by the file_type byte, which selects the on-disk file extension.

Request body

All offsets are relative to the start of the decrypted body (after the 32-byte routing header that the executor framework wraps every request in).

Preamble (48 bytes, offsets 0–47)

The standard QMail preamble. See QMail Overview — Universal preamble for the byte-level breakdown.

Command payload (38 bytes, offsets 48–85)

Total body length: 48 + 38 + Data Length + 2. The server validates this exactly — one byte off and the request is rejected with ERROR_INVALID_PACKET_LENGTH.

File types

The file_type byte at offset 80 picks the storage filename suffix. The server’s mapping (cmd_qmail_upload in cmd_qmail.c):

The same file_type value is later carried in the tell file manifest entry for this file, and the recipient passes it back in download to select which stored file to fetch. New QMail beta messages must upload file_type=0 private meta and file_type=1 body/content; attachments use file type 10 (0x0A) and up. The Tell manifest lists private meta first, body second, then attachments.

Payment & pricing

Command 70 does not put a fee amount on the wire. The server is paid by the coins in the locker identified by the 16-byte locker code. Payment behavior is controlled by the server’s [qmail] payment_mode_upload setting (off | shadow | enforce). Tell and object-transfer payments are not enforced in this release.

Payment modes

Enforce ordering (CMD 70)

1. Welfare / zero-fee short-circuit — if the computed fee for this stripe is 0, no claim is performed; an all-zero locker is accepted.
2. All-zero locker gate (fee due) — when a fee is due, an all-zero locker is rejected with ERROR_PAYMENT_REQUIRED before payment-record lookup or claim. (Zero-fee / welfare path already returned in step 1.)
3. Durable payment-record lookup — records are keyed (locker, GUID, sender). A prior successful claim for the same tuple covers retries of the same GUID for free. The same locker presented for a different GUID finds no covering record (coins already taken) and fails unpaid.
4. Synchronous claim-by-download — the server downloads the entire locker contents (exclusive: coins can be taken only once). This is not escrow; nothing is left in the locker for reuse. The client funds each locker with the exact computed fee; if the locker holds more than the fee, the excess is still consumed.
5. Value check — claimed value < required fee → ERROR_PAYMENT_REQUIRED and nothing is stored. Transport or backend claim failure → ERROR_PAYMENT_PROCESSING (retryable; no durable zero-value record is written for transport failures).

In enforce mode, unpaid or underfunded content that was accepted outside the synchronous path can be deleted by a sweep after a grace window (default 3600 s). Content stored before enforcement cutover is permanently protected via legacy markers.

Storage fee formula (interim, until DRD serves prices)

Rate and welfare are server-configurable ([qmail] upload_fee_per_mb, upload_welfare_mb); the client mirrors the same formula. Defaults:

• Rate: 0.01 CC per MiB (2²⁰ bytes) of per-server stripe data, per server.
• Rounding: always ceil up to the next 0.01 CC (at most two decimal places in the billed amount).
• Welfare: if per-server stripe size ≤ welfare allowance (default 1 MiB), fee is 0 for that server.
• Representation: fee amounts are decimal CloudCoins; the representation layer supports 0.0001 CC granularity. All REST amounts are exact decimal JSON numbers (not double-rounded units fields).

Example: 25 MiB striped over 10 servers ⇒ 2.5 MiB per server ⇒ 2.5 × 0.01 = 0.025 ⇒ ceil to 0.03 CC per server.

When the fee is zero (welfare or rate 0), the client sends an all-zero locker code and the server accepts it. When a fee is due, the client funds a locker with exactly that amount and places its key in the locker-code field.

Response

No body. The status byte in the response header indicates the outcome.

Status codes

Server-side storage layout

The storage path is sharded by the first two bytes of the GUID:

Root: /opt/raidax/QMail_Data/public_uploads/
Path: ROOT/{G1}/{G2}/{GUID_hex}/00000000{GUID_hex}.{suffix}

  G1       = hex(file_guid[0])    e.g. "a3"
  G2       = hex(file_guid[1])    e.g. "f7"
  GUID_hex = hex(file_guid)       32 lowercase hex chars
  suffix   = chosen by file_type  (.qmail, .meta, .{N}.bin, .blob)

Beside each stored file the server writes a .acl sidecar:

{file_path}.acl  (22 bytes, Phase I "allow all" format)
  byte 0      version       = 0x01
  byte 1      flags         = 0x01 (ACL_ALLOW_ALL)
  bytes 2-17  owner_guid    = the email GUID (16 bytes)
  byte 18     owner_denom   = sender denomination
  bytes 19-22 owner_sn      = sender SN (4 bytes)

The ACL sidecar exists today as a placeholder; download currently logs its absence but does not enforce. Future phases will read the sidecar to gate downloads by (recipient SN, recipient AN).

Example request body (hex)

Preamble (48 bytes, offsets 0-47):
A1 B2 C3 D4 E5 F6 07 08 09 0A 0B 0C XX XX XX XX  <- challenge: 12 random + 4 CRC32 BE
00 00 00 00 00 00 00 00                            <- session_id (zeros)
00 06                                              <- coin_type
01                                                 <- denomination = 1
00 00 0B 19                                        <- serial_number = 2841 BE
00                                                 <- reserved (was Device ID)
[16 bytes of authenticity AN]                      <- AN

Payload (38 bytes, offsets 48-85):
[16 bytes File GUID]
58 37 4B 51 2D 4D 33 50 4C 2D 39 52 56 42 00 00    <- locker_code "X7KQ-M3PL-9RVB" + 2 nulls
01                                                 <- file_type = 1 (.qmail)
00                                                 <- storage_duration
00 00 00 50                                        <- data_length = 80 BE

Stripe data (80 bytes):
[80 bytes of stripe content]

Terminator:
3E 3E

Total body size: 48 + 38 + 80 + 2 = 168 bytes.

Common mistakes

Points of confusion

1. Preamble size. The current value is QMAIL_PREAMBLE_SIZE = 48. Older specs say 49 — that was a typo from when byte 31 was Device ID and counted as a separate field. Byte range 0–47 is 48 bytes; the server enforces 48.
2. Reserved byte 31. Documented as “Reserved (was Device ID)”. The server reads it but does nothing with it. Available for protocol extension; clients should write zero today.
3. File type 2 vs 10. File type 2 is reserved for non-email blob storage (web pages, IM, etc.) and gets the .blob suffix. Attachments start at file type 10. Don’t use 2 for attachments.
4. Locker payment is mode-dependent, not always async. Default payment_mode_upload=off skips payment entirely. shadow peeks and logs only. enforce runs a synchronous claim-by-download before store (welfare → reject all-zero when fee due → payment record → claim → value check). Claim failures are ERROR_PAYMENT_PROCESSING (retryable) or ERROR_PAYMENT_REQUIRED (unpaid/underfunded). See Payment & pricing.
5. Storage duration is currently advisory. The byte is logged but the server does not yet enforce retention. Don’t rely on the server to delete expired files.