Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Peer serving & the fleet

This layer is one Xe Computer device serving another’s apps — a publisher exposes locally served apps over an authenticated tunnel, a consumer materializes them as ordinary local origins its browser can visit — and the fleet catalog that lets a device see every peer’s apps in one place. It rides directly on Transport & identity.

As-built truth. Citations are repo-relative in agent54/xenon-launcher at main @56e94575. Ports follow the current decisions d/100 / d/102 (browser plane on 5454; internal peer forward on 5198); earlier port choices in d/078/d/080 are lineage, superseded. Where the design is honestly unfinished, this chapter says so and links the review.

Primer: the words this chapter assumes

What is peer serving here? Not file sync and not a CDN: a live TCP tunnel from consumer to publisher over iroh QUIC, carrying whatever HTTP the consumer’s browser plane sends. The publisher’s app never knows it is being reached from another device.

What is a bearer secret? A credential that works for whoever holds it — like a coat-check token. No identity is checked; possession is the whole proof. Today’s serving tickets are bearer credentials, which is the central honest limitation of this layer.

Lineage. Peer-serve replaced an earlier experiment built on n0’s dumbpipe (P-158). The current implementation is native: crates/launcher/src/peer_serve.rs + fleet_catalog.rs.

Zoom 1 — the big picture: publish → ticket → consume → browse

sequenceDiagram
    participant Op as Operator (out-of-band)
    participant Pub as Publisher (xe peer publish)
    participant Con as Consumer (xe peer consume)
    participant Br as Browser / workerd
    Pub->>Pub: mint fresh 32-byte secret (getrandom)
    Pub->>Pub: publish_index → published-index.json + .sha256
    Pub->>Pub: write ticket (node_addr, secret, index_sha256, origin_id, device, origin_port)
    Pub->>Pub: 0600/DACL file; print ticket_path only
    Op-->>Con: carry ticket file (OUT OF BAND — G5)
    Con->>Con: read ticket (refuse group/other-readable file, unix)
    Con->>Pub: dial SERVE_ALPN + length-prefixed secret
    Pub->>Pub: constant-time compare vs expected
    Pub-->>Con: ACCEPT(1) / REJECT(0)
    Con->>Pub: GET /xe/api/index (Host rewritten to origin_port)
    Pub-->>Con: index bytes
    Con->>Con: verify sha256 == ticket.index_sha256; cache; register_peer
    Con->>Con: bind 127.0.0.1:port; liveness = live
    Br->>Con: HTTP to peer-backed localhost origin
    Con->>Pub: authenticated tunnel → 127.0.0.1:target_port
    Pub-->>Br: response bytes (raw pipe)
    Note over Pub,Con: Publisher restart → new secret → all tickets void (the only revocation today — G1)

In words: xe peer publish mints a one-session secret, writes it into a tightly permissioned ticket file, and starts accepting tunnels on the xenon/serve/tcp/1 ALPN. The operator carries the ticket to another device. xe peer consume dials the publisher by its iroh key, presents the secret, verifies the app index against the digest pinned in the ticket, and binds a local port. From then on the consumer’s browser plane treats the remote apps as local origins.

Zoom 2 — subsystems

The ticket: mint and custody

  • Fresh secret per publish. Every publish mints a new 32-byte secret via getrandom (peer_serve.rs:430-433). The code comment is explicit that persisting or reusing it “would remove restart-as-revocation” (:428-429); the --rotate-secret parameter exists but is deliberately inert (_rotate_secret, :426) until a real revocation design lands.
  • Unix custody. Ticket files are created 0600 (:216-225). On read, open_private_ticket_file refuses any ticket whose mode has group/other bits set (mode & 0o077 != 0) and tells the operator to chmod 600 (:287-304). Refusal is a tripwire: an over-shared credential is a signal, not something to silently paper over.
  • Windows custody. Creation resolves the current-user SID (validated, :147-175), replaces the parent-directory and file DACLs with a fresh owner-only rule (restrict_windows_acl, :180-213), opens create_new with share_mode(0), and deletes the half-written file if ACL setup fails (:266-284). Filenames carry PID + a 16-byte nonce against path reuse (:236-262). On read, however, Windows silently repairs a broad ACL rather than refusing (:308-323) — an asymmetry with the Unix tripwire, registered as G2 in the review.
  • The secret never hits stdout. Publish prints ticket_path=… plus non-secret status only (node_key, relay, target, device, origin_id, index_sha256, ready) (:463-477). Distribution of the file itself is out of band — undefined by the product today (G5).

The forward path and the two ports

The publisher forwards each accepted tunnel to 127.0.0.1:target_port (:557); target_port is required (P-209 item 1). The consumer binds 127.0.0.1:port for its side (:634). Two constants keep the planes distinct (d/100, d/102):

PortConstantRole
5454LOCAL_PLANE_PORT (fleet_catalog.rs:24)The browser-facing local plane — what URLs carry
5198PEER_FORWARD_PORT (origin.rs:728)Internal peer-forward target — never user-visible

Host rewrite: decoupling the two authorities

The ticket carries an optional origin_port (P-209 item 4): the consumer’s index probe rewrites the upstream Host: header to 127.0.0.1:<origin_port or 5454> (:804-808), so the publisher’s HTTP authority port and the consumer’s browser port need not match. Tickets minted before this field behave as a plain byte pipe (PeerTicket.origin_port doc, :101-104). The fuller seam — the plane owning Host normalization rather than the consume forward — is queued as the d/102 follow-up seam.

Restart-as-revocation

Because the secret is per-session and never persisted, restarting the publisher invalidates every outstanding ticket. That is the only revocation mechanism today (:428-429; P-209 STATE). It is a correct stop — a deliberate refusal to ship durable secrets on a route with no revocation list, epoch, or lifetime — but it is all-or-nothing: any restart strands every consumer. The design space is G1 in the review.

Liveness

A side channel, not part of the wire protocol (crates/launcher/src/peer_liveness.rs): replace-in-place JSON per origin with four states — unknown / live / degraded / offline — mapped 1:1 from observations: a real tunnel served, or a 5-second probe cycle with a 10-second dial budget (peer_serve.rs:570-573, probe_loop, :741-783). A data-freshness stamp is deferred (P-169 Q3); the code carries an honest note that “reachable” only approximates “fresh” (FRESHNESS_DEFERRAL_NOTE).

Zoom 2 — the fleet catalog

The catalog adds no new protocol. The publisher writes an immutable JSON app index into its origin store; its SHA-256 rides in the ticket; the consumer fetches the index as ordinary HTTP over the already-authenticated tunnel, verifies the digest, caches it; a route worker merges every cached index with liveness into one view.

flowchart TD
    subgraph Publisher
      REG[App registry] -->|publish_index| IDX[published-index.json + .sha256]
      IDX -->|sha256 into ticket| TIX[PeerTicket.index_sha256]
    end
    subgraph Consumer store
      TIX -->|register_peer| PEERS[peers.json + mirror xe/peers.json]
      PROBE[probe loop, 5 s] -->|GET /xe/api/index over tunnel| VER{sha256 == ticket?}
      VER -->|yes| CACHE[xe/indexes/peer_key.json]
      VER -->|no| KEEP[keep last-good copy]
      PROBE -->|observation| LIVE[xe/liveness/*.json<br/>unknown/live/degraded/offline]
    end
    subgraph Route worker
      PEERS --> MERGE[serveMergedCatalog]
      CACHE --> MERGE
      LIVE --> MERGE
      MERGE --> CAT[/xe/api/catalog: peers + apps + status + last_seen/]
    end
  • Publisher index. publish_index (fleet_catalog.rs:158-231) enumerates the app registry deterministically (name-sorted), writes xe/published-index.json plus a .sha256 sidecar, and appends a hash-chained audit record peer_catalog_index_published.
  • Peer identity, derived not asserted. publisher_origin_id = "iroh-" + sha256(json(node_addr)) (:152-155) — only public address metadata is hashed, never the secret. From it: peer_key = sha256(origin_id) and connector = "peer_" + peer_key[..16] (:254-260).
  • register_peer (:234-294) rejects port == 0 and port == 5454 (never squat the plane port), rejects a port already owned by a different peer, upserts by id, sorts, and writes durable peers.json and the workerd mirror xe/peers.json atomically.
  • cache_verified_index (:298-343) bounds the index at 1 MiB (MAX_INDEX_BYTES), validates schema/identity/label/order (validate_index, :410-440), audits peer_catalog_index_verified — and on digest mismatch never overwrites the last-good copy (:308-326). That last rule is what keeps offline peers’ apps visible in the catalog.
  • Merged view. The route worker (assets/route-worker.js:102-130) merges each peer’s cached index with its liveness snapshot into /xe/api/catalog. binaryPeerStatus collapses live|degraded → "live", else "offline" (:74-76). Offline peers are retained with last_seen_unix_ms — nothing prunes them (a deliberate visibility choice; retention policy is an open edge). Fetching an offline peer’s app returns a 503 that honestly admits it cannot defeat the app’s own service-worker cache (:35-44).

Zoom 3 — the wire, precisely

Consumer                          Publisher   (ALPN xenon/serve/tcp/1)
  dial_alpn(node_addr, SERVE_ALPN)  ── QUIC bi-stream open ──►
  u8 len (=32) ‖ secret[len]        ──────────────────────►  read u8 len (1..=64)
                                                             read len bytes
                                                             constant-time compare vs 32-byte secret
        ◄──────────── u8 ACK: ACCEPT(1) | REJECT(0) ───────  (≤10 s preamble deadline)
  if ACCEPT:  copy_bidirectional(local TCP ⇄ iroh stream) ⇄  connect 127.0.0.1:target_port

A 1-byte length-prefixed secret preamble, then a raw untyped byte pipe carrying whatever HTTP the browser plane sends. Constants: ACCEPT=1, REJECT=0, MAX_SECRET_LEN=64, MAX_TUNNELS=64. The index probe reuses the same authenticated stream but writes a literal GET /xe/api/index HTTP/1.1 with the rewritten Host (:788-839) — HTTP-over-tunnel for the catalog, opaque pipe for apps.

Full field tables for the ticket, the app index, and this protocol are Part V candidates: Xe Peer Ticket v1, Xe Fleet App-Index v1, Peer-serve wire protocol v1.

Honest edges

Named here, argued in the review:

  • Serving auth is membership-blind — a bearer secret, while a full signed membership plane sits unwired one crate away (G4, held in the transport review).
  • Revocation is publisher-restart only; --rotate-secret is inert scaffolding awaiting the G1 decision.
  • The ticket’s whole security rests on an undefined out-of-band transfer channel (G5).
  • Windows repairs an over-shared ticket where Unix refuses (G2).
  • Peer proof: Windows publish+consume native (P-206, PR#62); macOS proven (P-208); the fleet is real across all three desktop platforms.