Sol design review — Transport & peer serving

Part of the site’s external review; reproduced verbatim below the header. sol’s positions are folded into the design frontier as External review input lines.

• Date: 2026-07-18
• Reviewer: gpt-5.6-sol (codex model_reasoning_effort=xhigh; ran on lane-182 / 10.56.0.182; 104,702 tokens)
• Scope: Part II transport-identity.md + peer-serving.md; Part III reviews of both; the transport/peer spec-candidate block of Part V emerging-specs.md (Peer Ticket v1, Fleet App-Index v1, peer-serve wire protocol v1, ALPN & signing-domain registry v1). Material inlined; reviewer had no repo access.

Orchestrator-facing summary (5 lines)

1. Verdict on membership-gating: directionally right but overstated — group membership is an authenticated ACL, not a full capability system; first gated release should be membership AND a per-route, audience-bound signed grant, not membership replacing the ticket. “Reserve UCAN for non-members” is the wrong boundary; the real boundary is transitive delegation/attenuation. Ship gating on a new ALPN xenon/serve/tcp/2, never mutate /1 in place.
2. Verdict on the v1/v2 fork: ship v1 (crates/net/group) as the first canonical serving-gate lineage — it is what join/gossip/revocation/gating already compile against; v2’s nicer data model has no evidence of a complete lifecycle (“a larger version number is not a readiness argument”). Front it with a lineage-independent authorization interface; require a signed one-way v1→v2 transition; never authorize from the union of both lineages.
3. Strongest criticism: the four spec candidates are not yet sufficient for an independent interoperable implementation — the Peer Ticket has no on-wire version/type discriminator, JSON + base64url parsing profiles are unspecified, and the single-byte ACK conflates ~7 distinct failure classes while the 1-byte “length” is not real extensibility.
4. Second-order flags: origin_id = sha256(json(node_addr)) is encoding-unstable — derive identity from the endpoint public key; and single-key lifecycle coupling means one rotation changes transport, membership, and catalog identity at once (domain separation does not fix this).
5. Top-5 proposals: (1) ratify one membership authority + serving-policy profile; (2) xenon/serve/tcp/2 structured handshake; (3) replace session-wide bearer with sender-constrained grants; (4) publish normative schemas + parsing profiles + conformance vectors; (5) separate stable identity from routing and catalog freshness.

1. Protocol-design weaknesses in the spec candidates.

The candidates are useful as-built extractions, but they are not yet sufficient for independent interoperable implementations.

• Peer Ticket v1 lacks an on-wire version. “v1” appears only in the document title; the JSON has no schema or version discriminator and the token has no type prefix. Add a mandatory schema value, define whether future versions use a new prefix or schema, and specify whether unknown fields are ignored, preserved, or rejected.

• The JSON parsing profile is unspecified. All JSON formats need rules for UTF-8 validity, duplicate keys, trailing data, numbers, unknown fields, unknown enum values, and nesting/string/array limits. Duplicate keys should be rejected; otherwise different parsers can disagree about which value is authoritative. Base64url decoding also needs an exact alphabet, padding policy, whitespace policy, and decoded-size limit—not merely a file-size cap.

• The ticket table contradicts itself. index_sha256 is marked optional but required by the consumer. That should be either mandatory in v1 or assigned to a named legacy profile with explicitly reduced behavior. The same issue affects optional origin_id and device_name: the spec does not say what validation is skipped when they are absent.

• node_addr is not actually specified. {provider, address} does not define the types, bounds, ordering, or supported address variants inside address; it merely names an implementation-dependent iroh serialization. Pin an iroh-independent representation or a specific upstream schema version. Define behavior for unknown providers and unsupported transport-address types.

• origin_id is unstable and encoding-dependent. Hashing json(node_addr) makes identity depend on JSON serialization, transport-hint ordering, and potentially changing direct or relay addresses. Stable identity should derive from the iroh endpoint public key alone; reachability hints should be separately replaceable metadata. If hashing JSON remains, a canonical JSON encoding and exact preimage must be normative.

• Ticket fields need stricter bounds and normalization. The exact secret-decoding result must be 32 bytes; accepting arbitrary decoded lengths is inconsistent with the credential type. origin_id, provider strings, address collections, and relay URLs have no bounds. device_name needs a byte limit as well as a character limit, control-character policy, Unicode normalization decision, and exact comparison rule. origin_port needs validity rules, including whether zero is invalid.

• The ticket has no lifecycle vocabulary. There is no ticket/grant ID, issuer, audience, scope, issuance time, not-before time, expiry, group identifier, authorization epoch, or replacement relationship. Consequently, replay is simply reusable presentation, rotation is global publisher restart, and revocation cannot identify one consumer, ticket, origin, or app. Key rotation is more disruptive still: because the same Ed25519 key is also the node address and apparent origin identity, rotation changes transport identity, membership identity, and catalog identity simultaneously.

• Fleet App-Index v1 is only partially typed. PublishedApp.version, sha256, names, URLs, timestamps, observations, and several durable-row fields have no complete type or bound. The referent of each app’s sha256 is not stated. Maximum app count, unique-name requirements, total string sizes, and maximum peer count are absent.

• “Deterministic name order” is underspecified. Define the comparator over encoded bytes or Unicode scalar values, normalization, case sensitivity, and duplicate-name handling. The localhost-label grammar must be normative. Unknown kind values currently have no evolution rule: rejecting them makes every new kind a schema break, while accepting them requires fallback semantics.

• Digest pinning does not replace canonicalization. Pinning exact bytes is sufficient to detect a byte mismatch, but publishers still need a deterministic serialization profile for cross-implementation generation and test vectors. Specify key ordering, whitespace, escaping, number encoding, final newline, and whether the digest covers compressed or uncompressed representation.

• The index has no publication sequence or freshness model. “Immutable” appears to mean that each generated object is content-addressed, but there is no revision, previous digest, issuance time, expiry, or relation to a group head. A valid old index is therefore indistinguishable from the current index. Retaining last-good data is good availability behavior, but the catalog needs separate cached, stale, withdrawn, and authorization-invalid concepts rather than treating all retained entries as merely offline.

• Disk state and public API are mixed into one spec. FleetPeer, peers.json, the workerd mirror, and the merged HTTP view have different compatibility and migration requirements. They need separate schemas and versioning rules. The stated xe/peer-list/v1 schema is also ambiguous about whether it labels a row or a top-level collection.

• Derived local identifiers need collision behavior. A connector based on 64 bits of a digest needs a deterministic collision-resolution rule, not just an assumption of uniqueness. URL construction needs escaping rules for ?xe-peer=<id>, and duplicated origin_id, device-name, and per-app fields need explicit equality checks against their enclosing index.

• Peer-serve v1 has no extensible handshake. ALPN provides a clean incompatible-version boundary, but the stream itself has no message type, feature bits, auth-method discriminator, request ID, or reserved extension area. The one-byte secret length is not meaningful extensibility: the publisher accepts lengths 1..=64, while the only defined credential is exactly 32 bytes.

• The QUIC connection/stream model is unclear. The spec does not state whether one QUIC connection carries one tunnel or many, whether every stream repeats authentication, or whether MAX_TUNNELS applies per process, publisher, peer, connection, or origin. Queueing behavior, refusal behavior at capacity, stream limits, idle timeouts, and connection reuse are absent.

• The ACK is too coarse and its timing is ambiguous. REJECT conflates malformed preamble, unsupported auth method, invalid credential, non-member, revoked member, capacity exhaustion, and unavailable backend. It is also unclear whether ACCEPT means “credential accepted” or “backend connection established.” Define a small stable status frame, coarse peer-visible categories, detailed local diagnostics, QUIC application close codes, and retryability.

• Malformed-input and timeout behavior is missing. Specify handling for length zero, length over 64, early EOF, unknown ACK values, partial writes, preamble timeout scope, backend-connect timeout, and cancellation. Half-close propagation and whether one direction may continue after the other finishes also need definition.

• Rotation and revocation semantics omit live connections. Publisher restart incidentally closes them, but future exp, per-ticket revocation, membership changes, and secret rotation must state whether existing streams close immediately, receive a grace period, or remain authorized until disconnect. Race behavior at a group-head or epoch transition must be deterministic.

• The catalog’s literal HTTP probe is not sufficiently described. The spec needs accepted status codes, header and body limits, Content-Length versus chunked versus EOF framing, redirect behavior, response timeout, connection-close behavior, and how the 1 MiB cap is enforced while streaming. Host syntax and normalization also belong in the protocol, not only in implementation citations.

• The ALPN registry records policy that can vary at runtime. “Membership if provider built with it” means the same ALPN can have materially different authorization semantics on two deployments. Authentication prerequisites should be fixed by the protocol profile; a mandatory-gated serving protocol should receive a new ALPN rather than silently changing /1.

• The registry lacks normal registry metadata. Add exact octets, owner, stability status, reference, roles, framing, mandatory authorization profile, implementation versions, deprecation date, and replacement. Test-only identifiers should live in a clearly non-production namespace.

• The signing-domain list is not a complete signing specification. Wildcards such as xenon/interface-* and brace notation are unsuitable as allocations; enumerate every exact byte string. For each domain, define the signed structure, canonical encoding, concatenation or length-prefix rule, hash/prehash mode, Ed25519 verification profile, and test vectors. A trailing NUL is useful separation only when the exact construction is normative.

• Domain separation does not solve single-key lifecycle coupling. It prevents one signed message type from being accepted as another, but compromise, loss, and rotation still affect transport, membership, recovery statements, and every application signature at once. The registry needs a root/transport/signing-key lifecycle and algorithm-agility policy, even if v1 continues using one key.

2. The membership-blind-serving question.

The review’s verdict is directionally correct but stated too strongly. The existing membership plane should be reused before introducing a general delegation system, but group membership is an authenticated ACL, not “already a capability system” in the normal sense. It expresses coarse admission and revocation; it does not express an origin, app, port, action, attenuation chain, or per-consumer grant.

For the first gated release, I would use membership AND a route-specific grant, not membership as a replacement for the ticket:

• The authenticated QUIC remote key must be an active group member authorized for consume.
• The publisher must be an active member authorized for publish.
• A route grant should identify the publisher, consumer key, origin or service, validity interval, and grant ID. Initially this can be a simple publisher-signed structure; it does not need UCAN’s delegation machinery.

This distinction matters. Membership-only serving means every member who learns a publisher’s address may receive every exposed route unless another policy exists. Conversely, membership plus the current unbound bearer secret still leaves per-ticket replay and active-member sharing semantics unresolved. Group revocation fixes device removal, but it does not fully “collapse G1”: it cannot revoke one route while retaining the device, and a copied secret remains usable by another authorized member unless the grant is bound to the connection’s remote key.

“Reserve UCAN for a non-member” is also the wrong boundary. The relevant boundary is transitive delegation and attenuation, not membership status. A one-hop grant to a non-member does not require UCAN; a member who may delegate one app to another party might. Adopt UCAN only when Xe has a real requirement for proof chains, attenuation, or third-party delegation.

The integration is not a “small change.” Serving and group currently have separate endpoints and accept loops, so enforcement requires a defined source of membership truth, startup readiness, persistence, head freshness, active-stream registration, and behavior during partitions or chain-update failure. Both inbound and outbound checks need the same role semantics.

The v1/v2 fork must not remain implicit when gating lands. Otherwise:

• A v1 boolean member check may grant publish rights to a v2 Companion.
• Endpoint-binding semantics may be enforced on one path and ignored on another.
• Revocation accepted by one lineage may not close connections authorized through the other.
• Persisted low-water marks and chain heads may describe different authority histories.
• Deployments may silently disagree about who is a member while sharing the same serving ALPN.

I would first ratify a lineage-independent authorization interface—group ID, canonical member key, member kind, publish/consume rights, status, head version/hash, and readiness state—then provide exactly one authoritative adapter. Ship mandatory gating in xenon/serve/tcp/2 with structured status messages. During migration, advertise /1 only under an explicit legacy policy, prefer /2, audit legacy use, and set a removal point; do not change /1’s contract in place.

3. The v1 (crates/net/group) vs v2 (crates/xe-sync) group-document fork.

I would apply these decision criteria:

• Lifecycle completeness: join, succession, gossip, revocation, active disconnect, persistence, and restart anti-rollback must operate against the same model.
• Normative clarity: canonical bytes, signing inputs, chain rules, group identity, member identity, role semantics, and test vectors.
• Serving-policy fit: explicit publish/consume permissions, companion behavior, and any necessary endpoint binding.
• Operational maturity: integration tests, restart behavior, corrupt-state handling, observability, and proven deployment paths.
• Migration safety: a signed transition from existing persisted chains, prevention of dual-authority acceptance, and low-water-mark continuity.
• Dependency ownership: one crate must own the canonical model without circular adapters or two independently evolving validators.
• Extensibility: additions should not require interpreting unknown statements as authorization; unknown authorization-affecting data should fail closed.
• Recovery compatibility: authority rotation and future recovery must preserve a single comprehensible chain of authority.

Given the material, these criteria point to v1 for the first shipped membership-gated serving path. V1 is the lineage used by join, gossip, revocation, provider gating, live-connection closure, canonical encoding, and persisted anti-rollback. V2 has attractive data-model additions, but the material supplies no equivalent evidence that its join, chain, gossip, persistence, and serving semantics are complete. A larger version number is not a readiness argument.

That should be a bounded decision, not permanent format entrenchment. Ship v1 through a stable authorization interface; do not admit a Companion concept until its rights are normative. Treat v2 as a proposed successor, and require a signed v1-to-v2 transition that binds the v2 genesis to a final v1 head, migrates low-water state, supplies cross-version test vectors, and establishes a point after which v1 documents are no longer accepted. Never authorize from the union of both lineages.

4. Prior art & standards alignment.

• iroh: Using ALPN to route multiple protocols over an authenticated QUIC endpoint is directly aligned with iroh’s model. Iroh recommends a shared endpoint/router for multiple protocols, however, while Xe runs separate endpoints and accept loops under the same key; that divergence adds lifecycle and policy-consistency work without an articulated benefit. Current iroh guidance also treats the endpoint ID as stable and transport addresses as quickly changing hints, reinforcing that origin_id should derive from the endpoint key rather than serialized address metadata. Iroh endpoints

• iroh tickets: Combining dialing information with application data is an established iroh convention, and short-lived, manually transferred tickets fit the current publish session. Xe diverges by using unprefixed base64url JSON instead of iroh’s usual typed prefix plus compact encoding. Human-readable JSON is defensible, but omitting a type/version prefix is not; iroh also recommends endpoint IDs plus discovery rather than stale full tickets when an application already has coordination or gossip. Iroh ticket guidance

• TLS/ALPN: A distinct, versioned ALPN per incompatible protocol is correct. ALPN selects a protocol; it does not define authorization, and RFC 7301 requires precise, non-empty protocol-identification octets and treats the selected protocol as definitive for that connection. The IANA ALPN registry uses Expert Review and requires an interoperable reference. Keeping unregistered names during a closed experiment is understandable, but ratified externally implemented Xe protocols should seek registration and publish exact octets. RFC 7301, IANA ALPN registry

• OAuth bearer practice: The 32-byte secret has conventional bearer semantics: possession alone authorizes use, exactly the property formalized by RFC 6750. Strong file custody and encrypted transport align with bearer-token practice; missing expiry, scope, audience, token ID, standardized rejection categories, and refresh/rotation semantics do not. OAuth’s DPoP work demonstrates the established alternative of sender-constraining a token to a public key. Xe can obtain much of that benefit more simply by comparing a grant’s audience key to the already-authenticated iroh remote identity. RFC 6750, RFC 9449

• UCAN: UCAN is a plausible fit for public-key-verifiable delegation, attenuation, time bounds, and offline proof chains. The draft’s “signed JWT-like token” description should not be ratified without pinning a UCAN generation: the current specification is divided into delegation, invocation, promise, and revocation sub-specifications, with current containers serialized as signed DAG-CBOR rather than simply an ad hoc JWT. Audience alone is not a complete proof-of-possession story; UCAN invocation signs the requested use, while Xe could alternatively bind a delegation to the authenticated QUIC peer. UCAN’s CID revocation also still requires monotonic distribution and local checking, so it does not make the revocation channel disappear. UCAN specification, UCAN revocation

• Macaroons and Biscuit: Macaroons offer contextual caveats and attenuation but are fundamentally service-rooted and commonly bearer-oriented; their symmetric-root model is a poor match for Xe’s existing public device identities. Biscuit is closer—public-key verification, offline attenuation, structured policy, and revocation identifiers—but remains a bearer token unless the policy binds it to a proven peer, and revocation still needs external state. Both are excessive for “active fleet member may connect,” though Biscuit becomes relevant if Xe needs a rich policy language rather than a few fixed permissions. Macaroons paper, Biscuit specification

• Willow/Meadowcap: Meadowcap is useful prior art for rigor: it explicitly defines receiver, namespace, access mode, granted area, recursive delegation, restrictions, canonical encodings, and concrete parameter bounds. Its actual resource model is Willow data—read/write access over namespaces, paths, subspaces, and timestamps—not a live TCP origin, so adopting it directly would be unjustified. Xe should copy its semantic discipline, not its data model. Meadowcap specification

• did:key: Ed25519 keys can be represented as did:key without network resolution, but it is not literally free: Xe must implement and test multicodec, multibase, canonical key validation, and DID comparison. More importantly, did:key does not support update or deactivation and its own specification discourages unqualified long-term use. It is justified when interoperating with UCAN or DID tooling; otherwise the raw iroh endpoint key is simpler and has fewer representations. did:key v0.9

• Capability-system terminology: Xe’s group document is a strong signed membership ACL with prompt revocation. Calling it a capability system obscures the distinction between ambient group authority and an attenuable, transferable grant to a particular resource. Reusing membership for fleet admission is justified; claiming that it subsumes route capabilities is not.

• One key across planes: Reusing one iroh identity across ALPN-routed protocols is consistent with iroh’s endpoint model, and signing-domain separation is good practice. Reusing the same long-lived signing authority for transport identity, group authority, recovery, and application statements creates correlated identity and coupled rotation. A better long-term design is a stable device root that certifies purpose-specific transport and signing keys, with explicit migration; retaining one key in v1 is defensible only if that lifecycle limitation is documented.

5. Top-5 concrete improvement proposals, ranked.

1. Ratify one membership authority and serving-policy profile. Ship v1 as the initial canonical engine, define group ID, head readiness, member-key identity, publish/consume rights, and fail behavior, and explicitly exclude unresolved companion semantics. Specify a signed, one-way migration before v2 can replace it.

2. Introduce xenon/serve/tcp/2 with a structured handshake. Authenticate the QUIC remote key, require canonical membership, carry a typed route grant, and return bounded status codes distinguishing retryable capacity/backend conditions from non-retryable authorization failures. Define connection reuse, per-stream authentication, limits, timeouts, half-close behavior, and live revocation.

3. Replace session-wide bearer authority with sender-constrained grants. Give each grant a random ID, publisher key, audience endpoint key, group ID, origin/app scope, iat, optional nbf, exp, and authorization epoch; compare the audience to the authenticated iroh peer. Use a simple signed grant first and adopt UCAN only when transitive delegation or attenuation is an approved requirement.

4. Publish normative schemas, parsing profiles, and conformance vectors. Define every type, byte bound, ordering rule, Unicode rule, duplicate-key rule, unknown-field rule, canonical encoding, digest preimage, and error category for tickets, indexes, durable state, and signing inputs. Provide positive, negative, cross-version, and canonical-byte test vectors.

5. Separate stable identity from routing and catalog freshness. Derive origin_id from the endpoint public key, keep transport addresses replaceable, and add index revision, previous digest, issuance time, withdrawal/tombstone, and explicit stale status. Define how device-key rotation preserves catalog continuity through a signed old-key-to-new-key transition.

6. Questions missing from the session agenda.

• What is the exact authorization matrix for publish, consume, index fetch, app launch, and raw tunnel creation? Is membership fleet-wide authority or merely eligibility for a separate route grant?

• Will mandatory gating change /1 in place, or will Xe introduce a new ALPN with dual-advertisement and retirement rules? What prevents a mixed-version fleet from silently falling back to the membership-blind path?

• What membership state is sufficiently fresh to authorize serving? What happens at startup, during a partition, after low-water state is present but no current head is available, or when group persistence is damaged?

• How are route grants or non-secret discovery records delivered, acknowledged, audited, refreshed, and withdrawn? G5’s ticket-distribution channel is a protocol and product question even if bearer secrets disappear.

• What is the stable group identifier, and can the same device key participate in multiple groups? If so, which group and policy context does a serving connection select?

• Is a served authority one configured TCP port, one origin, one app, or an arbitrary byte tunnel to the publisher’s local plane? Where are Host normalization, allowed targets, app scope, and backend readiness enforced?

• What are the normative clock assumptions for exp, membership challenges, and catalog freshness? What skew is allowed, and what is the behavior when a device clock is unreliable?

• How do grant expiry, group revocation, authority rotation, and publisher restart affect existing QUIC connections and streams? Are closure semantics immediate, graceful, or epoch-based?

• How are device-key loss and rotation handled across iroh addressing, group membership, endpoint bindings, origin IDs, cached indexes, audit identity, and outstanding grants? The idle recovery vocabulary does not answer this operationally.

• Should the two current endpoints be consolidated behind one iroh router so every ALPN shares one membership snapshot, relay policy, shutdown lifecycle, and connection identity? If they remain separate, what invariant keeps their policies synchronized?

• What are the public error taxonomy and retry contract? Which failures are visible to the peer, which appear only in local audit logs, and how do operators distinguish authorization, stale membership, capacity, relay, and backend failures?

• How are catalog updates, app removals, device removal, stale last-good indexes, and intentional peer forgetting represented? “Never prune” is an availability policy, not a complete deletion or fleet-exit lifecycle.

• What interoperability scope is intended? If third-party implementations are expected, when will Xe freeze exact encodings, register stable ALPN identifiers, publish test vectors, and replace the historical xenon/ namespace if necessary?

SOLREV-TRANSPORT-DONE