mux.cool and XUDP

mux.cool is Xray’s multiplexing layer. One proxy connection, the carrier, holds many independent sessions, and each session is a TCP stream or a UDP association. Every piece of session data travels in a small frame that names its session. XUDP is the UDP extension: each packet of a UDP session may name its own peer, so one session can talk to many addresses. The module etemenanki_protocols::mux ports Xray’s common/mux server side. Its core is a sans-I/O demultiplexer, Demux, which the Trojan, VLESS and VMess server cores switch to once a request turns out to be a carrier.

This page is for contributors who change protocols/src/mux/ or the carrier paths of those three cores. It covers how each protocol signals a carrier, the frame layout, the session table and its key generations, how uplink frames become runtime effects without copying, how a frame that straddles VMess chunks is held, how downlink bytes are framed and reserved for, and the tests that pin each rule. Nothing on this page is configurable: every Trojan, VLESS and VMess inbound accepts a carrier, and no setting turns that off. The only inbound setting that reaches the sub-flows is sniffing. The user guide pages of the three protocols describe their inbound settings.

server side only The crate demultiplexes carriers it receives. It never multiplexes its own outbound traffic: there is no mux client.

Responsibilities

Concern	Where	What it does
Carrier address	protocols/src/mux/mod.rs → MUX_ADDRESS, mux_destination, is_mux_destination	The pseudo-destination v1.mux.cool, and the check Trojan uses to detect a carrier
Frame codec	protocols/src/mux/frame.rs → parse_meta, parse_frame, encode_keep, encode_end and their async and staging variants	Parses and builds frames and enforces the metadata and data length caps
Session table	protocols/src/mux/demux.rs → Demux	Maps wire session ids to outbound keys, opens and retires sub-flows, and declines excess or unknown sessions
Uplink dispatch	Demux::feed, Demux::feed_chunks	Turns frames into Effect::Open, Forward, ForwardHeld, SendTo, SendToHeld and Shutdown
Downlink framing	Demux::on_outbound, on_datagram, on_outbound_gone, on_transport_eof	Frames outbound bytes and packets as Keep frames, and a finished sub-flow as End
Outbound keys	protocols/src/core/mod.rs → FlowKey, SubKey	The key type the three carrier cores give the runtime
Carrier detection and staging	TrojanCore, VlessCore, VMessCore	Enter the Mux state, feed the demultiplexer, and stage or seal its output inside a declared staging reserve

What the module leaves to others:

• Routing and dialing. Each sub-flow reaches the program’s connector as its own Flow through Effect::Open. In etemenanki-app, AppConnector routes a stream sub-flow once, and turns a UDP sub-flow into a FanOutLink that routes packet by packet. katana drives the same three cores, so every mux sub-flow also passes through its connector’s admission, routing, audit and traffic metering on its own.
• I/O and backpressure. The per-connection runtime applies the effects, holds the read buffer while a forward is pending, and guarantees staging room before it delivers an event.
• Carrier crypto and transport. VMess opens and seals its chunk stream itself. Trojan and VLESS rely on the transport under them (TLS, WebSocket, gRPC) for confidentiality.

Signalling a carrier

The three protocols signal multiplexing in two ways. VLESS and VMess have a mux command. Trojan has none, so a client signals it by address.

Carrier	Signal	Carrier Flow::destination	Staged at once
VLESS	Command byte CMD_MUX (0x03). The header ends at the command byte, with no address	mux_destination(): TCP, v1.mux.cool, port 0	RESPONSE_HEADER
VMess	Command 0x03 in the sealed request header, with no address	mux_destination()	The sealed response header, through VMessCore::reply
Trojan	A request whose command is not CMD_UDP_ASSOCIATE (0x03), in practice CMD_TCP_CONNECT (0x01), and whose address satisfies is_mux_destination	The address as sent, for example v1.mux.cool port 9527 from Xray	Nothing, because Trojan never replies

protocols/src/mux/mod.rs

pub const MUX_ADDRESS: &str = "v1.mux.cool";

pub fn mux_destination() -> Destination

pub const MUX_PORT: u16 = 9527;

pub fn is_mux_destination(dest: &Destination) -> bool

is_mux_destination matches a Remote::Domain equal to MUX_ADDRESS ignoring ASCII case, and ignores the port, as Xray’s mux.Server.Dispatch does. MUX_PORT is informational. Only TrojanCore calls the check, and only for a request that is not CMD_UDP_ASSOCIATE. A SOCKS, HTTP or Shadowsocks CONNECT to v1.mux.cool is dialed like any other domain. The VLESS and VMess header parsers substitute mux_destination() for the missing address. Command::network maps Mux to DialNetwork::Tcp, because the carrier itself is a stream.

On a carrier, the core builds its Flow as usual (user, source, destination) and passes it to Demux::new. That flow is never handed to the connector. It is only the template for every sub-flow:

protocols/src/flow.rs

impl<T> Flow<T> {
    pub fn toward(&self, destination: Destination) -> Self
}

toward copies the carrier’s user and source, sets the sub-flow’s own destination, and resets sniffed to None. Every sub-flow is therefore attributed to the user who authenticated the carrier.

After detection, each core sets State::Mux(Demux::new(flow, self.sniff)) and enters Phase::Relay, which arms RELAY_IDLE_TIMEOUT (300 s). That one idle deadline covers the whole carrier. Timing::touch re-arms it on every transport and outbound byte event, so a KeepAlive frame keeps a quiet carrier open. When the deadline passes, Timing::expired pushes Effect::Finish and the carrier ends with all its sub-flows. Sub-flows have no idle timer of their own.

Frame format

A frame is a length-prefixed metadata block, optionally followed by a length-prefixed data block. All integers are big-endian.

Field	Size (bytes)	Meaning
Metadata length	2	Length of the metadata block that follows. Must be at least MIN_META_LEN (4) and at most MAX_META_LEN (512)
Session id	2	The sub-flow this frame belongs to, chosen by the client
Status	1	0x01 New, 0x02 Keep, 0x03 End, 0x04 KeepAlive
Option	1	Bit OPTION_DATA (0x01): a data block follows. Bit OPTION_ERROR (0x02): the peer reports an error on this session
Network	1	Present on every New frame, and on a Keep frame only when this byte is NETWORK_UDP. 0x01 TCP, 0x02 UDP
Address	5 to 259	The target (New) or per-packet peer (UDP Keep), in AddressCodec::VMESS layout: a 2-byte port, then type 0x01 IPv4 (4 bytes), 0x02 domain (1 length byte and up to 255 bytes) or 0x03 IPv6 (16 bytes)
Global id	8	XUDP session identity. Read only on a New frame that opens a UDP session with OPTION_DATA set, when at least 8 bytes remain in the metadata
Rest of metadata	variable	Ignored. The metadata length accounts for it
Data length	2	Present only with OPTION_DATA. At most MAX_DATA_LEN (8192)
Data	data length	The sub-flow’s payload: stream bytes, or one UDP packet

The address codec is the one VLESS and VMess use (frame::ADDR is AddressCodec::VMESS), with the port first. AddressCodec::MAX_LEN is 259: the type byte, a length byte, a 255-byte domain and the port.

A Keep frame on a stream session carries no network or address. parse_meta looks at the byte after the option first. Only NETWORK_UDP there makes it parse an address, as upstream does, because on a stream session the metadata ends at the option byte.

protocols/src/mux/frame.rs

pub const ADDR: AddressCodec = AddressCodec::VMESS;

pub const STATUS_NEW: u8 = 0x01;
pub const STATUS_KEEP: u8 = 0x02;
pub const STATUS_END: u8 = 0x03;
pub const STATUS_KEEP_ALIVE: u8 = 0x04;

pub const OPTION_DATA: u8 = 0x01;
pub const OPTION_ERROR: u8 = 0x02;

pub const NETWORK_TCP: u8 = 0x01;
pub const NETWORK_UDP: u8 = 0x02;

pub const MAX_META_LEN: usize = 512;
pub const MAX_DATA_LEN: usize = 8 * 1024;
const MIN_META_LEN: usize = 4;
pub const FRAME_OVERHEAD_MAX: usize = 2 + MIN_META_LEN + 1 + AddressCodec::MAX_LEN + 2;
const GLOBAL_ID_LEN: usize = 8;

MAX_META_LEN matches frame.go’s metaLen > 512 rejection and bounds the per-frame allocation from an untrusted peer. MAX_DATA_LEN matches the 8 KiB split in upstream’s writer. The largest uplink frame is therefore 2 + 512 + 2 + 8192 = 8708 bytes, which fits the 16 KiB read buffer of the Trojan and VLESS runtimes. FRAME_OVERHEAD_MAX is 268: the most a server-emitted frame adds beyond its payload (a Keep with a UDP peer and a domain address).

Decoded types

protocols/src/mux/frame.rs

pub enum SessionStatus {
    New,
    Keep,
    End,
    KeepAlive,
}

pub struct FrameMeta {
    pub session_id: u16,
    pub status: SessionStatus,
    pub option: u8,
    pub target: Option<Destination>,
    pub global_id: Option<[u8; GLOBAL_ID_LEN]>,
}

impl FrameMeta {
    pub fn has_data(&self) -> bool
}

pub struct Frame {
    pub meta: FrameMeta,
    pub data: Option<Range<usize>>,
    pub consumed: usize,
}

FrameMeta::target is the destination on a New frame, the per-packet peer on a UDP Keep frame, and None otherwise. Frame::data is the payload’s range inside the parsed buffer, not a copy, and consumed is the whole frame’s length.

Codec functions

protocols/src/mux/frame.rs

pub fn parse_meta(b: &[u8]) -> io::Result<FrameMeta>
pub fn parse_frame(buf: &[u8]) -> io::Result<Option<Frame>>

pub async fn read_meta<R: AsyncRead + Unpin>(r: &mut R) -> io::Result<FrameMeta>
pub async fn read_data<R: AsyncRead + Unpin>(r: &mut R) -> io::Result<Bytes>

pub fn encode_keep(session_id: u16, udp_peer: Option<&Destination>, payload: &[u8]) -> Bytes
pub fn encode_end(session_id: u16) -> Bytes

pub fn encode_keep_into(
    session_id: u16,
    udp_peer: Option<&Destination>,
    payload: &[u8],
    out: &mut Staging<'_>,
) -> Option<()>
pub fn encode_end_into(session_id: u16, out: &mut Staging<'_>) -> Option<()>

• parse_frame is the sans-I/O parser the demultiplexer uses. It returns Ok(None) while the buffer holds only part of a frame. It checks both length fields as soon as they are visible, before it waits for the body, so an oversized length fails at once instead of making the caller buffer up to 64 KiB.
• read_meta and read_data are the async equivalents, with the same caps. The server paths do not use them; the unit tests use them to read frames back.
• encode_keep and encode_end build server-to-client frames. The server never emits New or KeepAlive: upstream’s NewResponseWriter starts in follow-up mode, so every response is a Keep, and a session is closed with an End whose option is 0. encode_keep writes the payload length as a u16, so its callers keep payloads within MAX_DATA_LEN.
• encode_keep_into and encode_end_into write the same bytes straight into a Staging area. They return None and stage nothing when the room is short or the payload does not fit a u16. At this revision only the unit tests call them; Demux builds its downlink with encode_keep and encode_end.

The demultiplexer

protocols/src/mux/demux.rs

pub const MAX_SESSIONS: usize = 256;

pub const fn downlink_overhead(read_size: usize) -> usize

struct Sub {
    key: SubKey,
    peer: Option<Destination>,
}

enum Origin {
    Slice(usize),
    Held(usize),
}

pub struct Demux<T> {
    sessions: BTreeMap<u16, Sub>,
    generation: u32,
    straddle: Vec<u8>,
    partial_at: usize,
    out: Vec<u8>,
    flow: Flow<T>,
    sniff: bool,
}

impl<T> Demux<T> {
    pub fn new(flow: Flow<T>, sniff: bool) -> Self
    pub fn held(&self) -> &[u8]
    pub fn out(&self) -> &[u8]
    pub fn keys(&self) -> impl Iterator<Item = FlowKey> + '_
    pub fn is_empty(&self) -> bool
}

impl<T: Send + Sync + 'static> Demux<T> {
    pub fn feed<C>(
        &mut self,
        plain: &[u8],
        base: usize,
        fx: &mut Effects<'_, C>,
    ) -> io::Result<usize>
    where
        C: ProxyCoreDecode<Key = FlowKey, Target = Flow<T>>;

    pub fn feed_chunks<C>(
        &mut self,
        data: &[u8],
        chunks: &[std::ops::Range<usize>],
        fx: &mut Effects<'_, C>,
    ) -> io::Result<()>
    where
        C: ProxyCoreDecode<Key = FlowKey, Target = Flow<T>>;

    pub fn take_out(&mut self) -> Vec<u8>
    pub fn on_outbound(&mut self, key: SubKey, data: &[u8])
    pub fn on_datagram(&mut self, key: SubKey, from: &Destination, data: &[u8])

    pub fn on_outbound_gone<C>(&mut self, key: SubKey, fx: &mut Effects<'_, C>)
    where
        C: ProxyCoreDecode<Key = FlowKey, Target = Flow<T>>;

    pub fn on_transport_eof<C>(&mut self, fx: &mut Effects<'_, C>)
    where
        C: ProxyCoreDecode<Key = FlowKey, Target = Flow<T>>;
}

Field	Holds
sessions	One Sub per live wire session id: its current SubKey, and for a UDP session the target from its New frame (peer), used as the fallback peer
generation	The last generation minted. It starts at 0 and is incremented with wrapping_add on every accepted New, so the first sub-flow gets generation 1
straddle	The held buffer: the frames completed from held bytes during the current byte event, which its held forwards reference, then the partial tail of a frame split across VMess chunks
partial_at	Where the partial tail starts inside straddle
out	The frames of the last call only: the downlink frames of a downlink call, or the declines of an uplink call
flow, sniff	The carrier’s flow template, and whether sub-flows are sniffed

keys (documented as the live keys “for closing them all”) and is_empty are not called by the three cores at this revision; they close every sub-flow through on_transport_eof.

Keys and generations

protocols/src/core/mod.rs

pub enum FlowKey {
    Direct,
    Sub(SubKey),
}

pub struct SubKey {
    pub id: u16,
    pub generation: u32,
}

The three carrier cores declare type Key = FlowKey. FlowKey::Direct is the connection’s own flow on a plain request, and FlowKey::Sub is one mux sub-flow.

The generation exists because of the runtime’s key rule: an Effect::Open on a key that is still live fails the connection with RuntimeError::DuplicateKey. A runtime key stays live until the outbound is gone, which for a half-closed stream means until its read side reaches EOF. A mux client may reuse a session id right after it ends it. When the client sends End, the demultiplexer removes the id from sessions and pushes Effect::Shutdown, but the runtime still holds the old outbound until it drains. A New on the same id then gets a fresh generation, so its SubKey differs and the two outbounds never collide.

Demux::session_id(key) maps a key back to its wire id only while sessions[key.id] still holds that exact key. Downlink bytes from a retired generation therefore frame nothing and are dropped. This is how an outbound that is still draining after the client’s End is kept off the wire.

Uplink dispatch

Every parsed frame goes through Demux::dispatch. The payload is never copied on this path: the demultiplexer pushes an effect that names a range, either in the event’s slice (Origin::Slice) or in the held buffer (Origin::Held).

flowchart TB
  f["parse_frame"] --> s{"status"}
  s -- New --> n{"MAX_SESSIONS reached or id live?"}
  n -- yes --> dec["queue End for the id, drop the payload"]
  n -- no --> open["mint SubKey, sniff payload, push Open"]
  open --> del["deliver the payload"]
  s -- "Keep with data" --> k{"id live?"}
  k -- no --> dec
  k -- yes --> del
  s -- End --> e["remove the id, push Shutdown"]
  s -- "KeepAlive or Keep without data" --> x["nothing"]
  del --> u{"UDP session?"}
  u -- no --> fw["Forward or ForwardHeld"]
  u -- yes --> st["SendTo or SendToHeld, to the frame's peer or the session target"]

The rules, status by status:

• New. target is required; parse_meta always reads it for New, and dispatch still fails with mux: new session without a target if it is missing. If sessions.len() >= MAX_SESSIONS, or the id is already live, the demultiplexer queues an End for that id and discards the frame’s payload. A duplicate New leaves the live session under that id as it is. Otherwise it mints SubKey { id, generation } and builds the sub-flow with flow.toward(target). It pushes Effect::Open { key: FlowKey::Sub(key), target }, records the Sub (with peer set to the target for UDP), and delivers the payload if there is one.
• Keep. A Keep with data on a live id is delivered. A Keep with data on an unknown id, for example one the server already ended, is answered with an End, as upstream does, so the client stops sending on it. A Keep without data is ignored.
• End. The id is removed and Effect::Shutdown half-closes the outbound’s write side. An End for an unknown id is ignored. An End that carries a data block has the block skipped. OPTION_ERROR is parsed but does not change what End does.
• KeepAlive. Nothing happens in the demultiplexer. The transport event that carried it has already re-armed the idle deadline.

deliver chooses the effect from the session type and the origin of the bytes:

Session	Bytes in the event slice	Bytes in the held buffer
Stream (peer is None)	Effect::Forward { key, range }	Effect::ForwardHeld { key, range }
UDP (peer is the New target)	Effect::SendTo { key, to, range }	Effect::SendToHeld { key, to, range }

For a UDP session, to is the frame’s own address when the Keep frame carried one, and the session’s New target otherwise. That is XUDP’s per-packet addressing: one sub-flow, one outbound key, many peers. Each UDP frame’s data block is exactly one datagram. A stream session’s frames that carry an address are delivered as stream bytes, and the address is not used.

Frame ranges are absolute. feed takes base, the offset of plain inside the event’s slice, and adds it to every range. feed_chunks takes each chunk as a range of the event’s slice data, so its ranges are absolute already and it has no base argument.

Sniffing sub-flows

With sniff on, a New whose target is an IP address (sniff::worth_sniffing) has its payload passed to sniff::sniff, which tries the TLS SNI sniffer and then the HTTP Host sniffer. A hit sets the sub-flow’s Flow::sniffed before Effect::Open is pushed. The demultiplexer sniffs only the payload of the New frame. It does not wait for more bytes and does not arm SNIFF_TIMEOUT, because the sub-flow is routed the moment it opens, and holding the carrier for one sub-flow would stall all the others. A New without data, or one whose first bytes do not show the name, opens unsniffed. The same check runs for UDP sub-flows, where the TLS and HTTP sniffers normally find nothing. How sniffing works on single-flow connections is described in Sniffing.

Two ways to feed: feed and feed_chunks

The carrier cores see mux frames in two shapes, and the demultiplexer has one entry point for each.

	feed	feed_chunks
Caller	TrojanCore, VlessCore	VMessCore, once per byte event with all opened chunks
Input	The whole unparsed transport region, base 0	Every opened chunk’s range in data
Trailing partial frame	Not consumed. feed returns the bytes used, and the runtime hands the tail back, prefixed to the next read	Consumed. It is copied into straddle and completed by a later chunk, of the same event or the next
Effects for complete frames	Forward / SendTo over the read buffer	Forward / SendTo over the read buffer
Effects for a completed straddling frame	Not applicable	ForwardHeld / SendToHeld over straddle

A Trojan or VLESS carrier is plaintext at this layer, so a partial frame can stay in the runtime’s read buffer. VMess cannot do that: ChunkDecoder opens each AEAD chunk in place and consumes it whole, so an unconsumed tail would come back already decrypted. feed_chunks therefore handles the event’s chunks one by one in the private feed_chunk, which copies a frame’s leading part into straddle instead of leaving it behind. When a partial frame is held from partial_at, frame_need works out how many bytes it still needs (2 for the metadata length, then the metadata, then the data length, then the data), and the loop copies only that many bytes from the chunk. The frame is completed in place, after any frames completed earlier in the same event, and dispatched with Origin::Held(start), where start is its offset partial_at. partial_at then moves to the end of straddle, and the rest of the chunk is parsed in place. A chunk that ends inside a frame appends that frame’s leading part at partial_at, for the next chunk of the event or the next event to complete.

The held buffer follows the runtime’s pin rule. A held range is resolved when the runtime applies the effect, and until every queued held effect is applied, the runtime delivers no byte event to the core. trim_held runs once at the start of feed and of feed_chunks, that is once per byte event and never between the chunks of one event: it drains the completed frames before partial_at and keeps the partial tail. VMessCore passes every chunk that one read opened in a single feed_chunks call for this reason, because the held forwards pushed for earlier chunks of the read are applied only after the core returns. VMessCore::held, VlessCore::held and TrojanCore::held return Demux::held in the Mux state and the sniff prefix otherwise. The server core contract and the pin rule are described in Server core and Server runtime.

During one byte event, straddle holds every frame completed from held bytes in that event (at most one per chunk), then the partial tail. Its size is therefore bounded by the tail carried in from the previous event plus the plaintext of one read. The carried tail is part of one frame, bounded by the length caps, which parse_frame checks before frame_need trusts a length, so it is shorter than 8708 bytes.

Downlink framing

Outbound events for a sub-flow reach the carrier core with FlowKey::Sub(key), and the core passes them to the demultiplexer:

Core event	Demux call	Frames in out
Event::Outbound	on_outbound(key, data)	One Keep per MAX_DATA_LEN piece of data, without an address
Event::Datagram	on_datagram(key, &from, data)	One Keep with from as the UDP peer address. A packet longer than MAX_DATA_LEN is dropped
Event::OutboundEof, ConnectFailed, OutboundError	on_outbound_gone(key, fx)	One End, plus Effect::Close for the key, when the key is still the live generation
Event::TransportEof (the carrier’s uplink ended)	on_transport_eof(fx)	None. Effect::Close for every live sub-flow

Every Demux call that writes out clears it first: the downlink calls on_outbound, on_datagram and on_outbound_gone, and the uplink calls feed and feed_chunks. out therefore only ever holds the frames of the last call, and after an uplink call it contains only that call’s End declines, never downlink frames a carrier has already staged. A downlink call frames nothing when session_id(key) does not match. The from address on a reply is the packet’s source as the datagram link reports it. A link that resolves domains reports the IP it received from, so a client that sent to a domain gets replies tagged with an IP. The XUDP client uses that address to attribute each reply to a peer. The three cores declare MAX_DATAGRAM = 8192, and the runtime receives an outbound packet into a buffer of at most that many bytes, so on_datagram never sees more than MAX_DATA_LEN from the runtime. Its length check is a backstop.

After a sub-flow’s outbound ends, the demultiplexer sends End and closes the key in both directions. A mux session has no half-close: once either side ends it, the whole session is over.

The carrier cores deliver out in two ways:

• TrojanCore and VlessCore stage demux.out() verbatim with fx.stage. After an uplink feed, they stage demux.take_out() when it is not empty.
• VMessCore takes out with take_out after every call and seals it with seal_frames, which splits it into body chunks of at most MAX_PAYLOAD (1966) bytes. Chunk boundaries and frame boundaries are independent in both directions.

Reserving staging room: downlink_overhead

The runtime delivers an outbound read of n bytes only when STAGING_RESERVE + n bytes of staging are free, and an outbound read is at most BUF_SIZE bytes. Framing adds headers, so each core declares their worst case in its reserve:

protocols/src/mux/demux.rs

pub const fn downlink_overhead(read_size: usize) -> usize {
    read_size
        .div_ceil(MAX_DATA_LEN)
        .saturating_mul(FRAME_OVERHEAD_MAX)
}

Core	BUF_SIZE	STAGING_RESERVE	MAX_DATAGRAM
TrojanCore	16384	PACKET_HEADER_MAX.next_multiple_of(16) + downlink_overhead(Self::BUF_SIZE) = 272 + 536 = 808	MAX_LENGTH = 8192
VlessCore	16384	272 + downlink_overhead(Self::BUF_SIZE) = 808	8192
VMessCore	32768	4096, a flat value for the response header and the chunk overhead of one read. It does not call downlink_overhead: a stream Keep header is 8 bytes, so the mux framing of one read is small next to the chunk overhead	8192

A reserve that is too small shows up as staging_full(), the error staging room below the core's declared reserve, which ends the carrier. An uplink event only guarantees STAGING_RESERVE, so the End frames queued by declines (6 bytes each) must fit the room left at that moment.

A carrier from start to finish

The sequence below shows a VLESS carrier with one stream sub-flow. Trojan differs only in its signal and in staging no response header. VMess seals every staged byte into chunks.

sequenceDiagram
  participant C as Xray client
  participant R as Runtime
  participant K as VlessCore
  participant D as Demux
  participant O as Outbound
  C->>R: VLESS header with command 0x03
  R->>K: Event::Transport
  K->>R: stage RESPONSE_HEADER, enter Relay
  K->>D: Demux::new(flow, sniff)
  C->>R: New id 1, TCP target, data
  R->>K: Event::Transport
  K->>D: feed(data, 0, fx)
  D-->>R: Open Sub(1, gen 1), Forward range
  R->>O: connect the sub-flow
  O-->>R: connected, Forward applied
  O-->>R: reply bytes
  R->>K: Event::Outbound for Sub(1, gen 1)
  K->>D: on_outbound(key, data)
  K->>R: stage Keep frames from out
  R-->>C: Keep id 1 with data
  O-->>R: read EOF
  R->>K: Event::OutboundEof
  K->>D: on_outbound_gone(key, fx)
  D-->>R: Close Sub(1, gen 1)
  R-->>C: End id 1
  C->>R: transport EOF
  R->>K: Event::TransportEof
  K->>D: on_transport_eof(fx)
  K->>R: ShutdownTransport, Finish

A Forward pushed for a sub-flow that is still connecting waits for the connect. The runtime applies effects in order and holds every effect behind one that cannot complete, so the carrier’s uplink waits too, while downlink from other sub-flows keeps flowing. The concepts test stalled_outbound_holds_uplink_but_not_other_downlink pins that behaviour with a toy mux protocol.

Session lifecycle

One SubKey (a session id plus a generation) goes through these states:

stateDiagram-v2
  [*] --> Live: New accepted, push Open
  Live --> Live: Keep with data, push Forward or SendTo
  Live --> Live: outbound bytes or packet, stage Keep
  Live --> Retiring: End from the client, push Shutdown
  Live --> [*]: outbound EOF or error, stage End, push Close
  Live --> [*]: carrier EOF, push Close
  Retiring --> [*]: outbound read EOF or error, or carrier end

In Retiring, the id is already free in sessions, the outbound’s write side is shut down, and anything it still sends is dropped because session_id no longer matches. A New on the same id starts a new SubKey in Live with the next generation.

Uplink frames that do not open a key never reach this diagram. A New over MAX_SESSIONS, a duplicate New, and a Keep with data for an unknown id are each answered with one End on the downlink and have no other effect.

XUDP

XUDP adds two things to mux.cool UDP sessions. The server implements the first and parses the second.

1. Per-packet addresses. Each Keep frame of a UDP session may carry its own peer, and each downlink Keep names the peer the packet came from. parse_meta reads the address on a Keep frame when the network byte is NETWORK_UDP. deliver sends to that address, or to the session’s New target when a frame has none. on_datagram tags every reply with its source. In etemenanki-app, the UDP sub-flow’s outbound is a FanOutLink (app/src/outbound/udp_fanout.rs), which routes each packet on its own destination, so one XUDP session can reach peers behind different outbounds.
2. The global id. A New frame that opens a UDP session with data may append an 8-byte global id after the address. Xray uses it to let a UDP session continue over a new carrier. parse_meta decodes it into FrameMeta::global_id so the metadata block is fully accounted for, but the demultiplexer does not look at it: every New opens a fresh session, and a UDP session does not survive the loss of its carrier.

Invariants

Invariant	Mechanism	Pinned by
A frame’s lengths are capped before the parser waits for its body	parse_frame and read_meta / read_data check MAX_META_LEN, MIN_META_LEN and MAX_DATA_LEN as soon as each length is visible	read_meta_enforces_the_length_caps, frames_parse_from_slices_and_stage_into_buffers
Unknown status and network bytes, and metadata that ends inside a field, are errors	SessionStatus::from_byte, dial_network, take / take_array returning ProtocolError::Truncated	rejects_unknown_status_and_network, rejects_truncated_metadata
A stream Keep frame is never read as an address	parse_meta reads a Keep target only when the network byte is NETWORK_UDP	keep_frame_carries_an_address_only_when_flagged_udp, end_and_keepalive_carry_no_target
The global id is read only on a UDP New with data	parse_meta’s global_id match	parses_new_udp_session_with_global_id, parses_new_tcp_session
At most MAX_SESSIONS live session ids per carrier; excess and unknown sessions are declined without ending the carrier	The sessions.len() and contains_key checks in dispatch, and decline	unknown_or_excess_sessions_are_declined_with_end
A reused session id never produces a duplicate runtime key	A fresh generation per accepted New; session_id matches the exact SubKey	stream_sessions_open_forward_and_end_with_fresh_generations
A retired generation frames nothing on the downlink	session_id(key) returns None for a key that is not the live one	stream_sessions_open_forward_and_end_with_fresh_generations
UDP payloads go to the frame’s peer, or the session target when none is given, and replies carry their source	deliver with Sub::peer as the fallback; on_datagram passes from to encode_keep	udp_sessions_address_each_packet_and_tag_replies, encoded_udp_keep_carries_the_peer_address, vless_xudp_datagram_roundtrip
Uplink payloads are forwarded by range, never copied, unless a frame straddles VMess chunks	Origin::Slice in feed; Origin::Held only for a frame completed in straddle	stream_sessions_open_forward_and_end_with_fresh_generations, a_frame_straddling_chunks_is_held_and_forwarded_from_the_held_buffer
Held bytes stay in place until the held forward is applied	trim_held runs only at the start of the next byte event, once per feed or feed_chunks call; the runtime’s pin rule	a_frame_straddling_chunks_is_held_and_forwarded_from_the_held_buffer, held_bytes_are_forwarded_after_the_dial_and_survive_later_rewrites
Every frame one read completes from held bytes stays in straddle until the next byte event	VMessCore passes all chunks of a read to one feed_chunks call; feed_chunk completes each held frame after the ones before it and does not trim	vmess_keeps_every_frame_one_read_completes, vmess_mux_payload_spans_both_framings
Each downlink frame is staged once	out.clear() at the start of every call that writes out, uplink and downlink	a_downlink_frame_is_not_sent_again_by_the_next_uplink, vless_answers_mux_at_once_and_demultiplexes (the next uplink stages nothing), xudp_attributes_replies_to_the_right_peer
No stream downlink frame exceeds MAX_DATA_LEN	on_outbound splits with chunks(MAX_DATA_LEN)	vmess_demultiplexes_across_chunk_boundaries (10,000 bytes become two Keep frames)
Downlink framing never exceeds the staging room	downlink_overhead in STAGING_RESERVE for Trojan and VLESS; the runtime’s reserve check before an outbound read	frames_parse_from_slices_and_stage_into_buffers (a domain Keep fits FRAME_OVERHEAD_MAX exactly)
The carrier is answered before any sub-flow connects	VLESS and VMess stage their response header on entering Mux	vless_answers_mux_at_once_and_demultiplexes, vmess_demultiplexes_across_chunk_boundaries
Trojan detects a carrier by address, ignoring the port	is_mux_destination	trojan_carries_mux_when_the_connect_names_the_carrier, trojan_mux_tcp_single_stream
A VLESS mux request carries no address	parse_request_header substitutes mux_destination() after the command byte	mux_command_synthesises_its_destination, mux_request_header_roundtrips

Failure paths and cancellation

• Malformed frames end the carrier. Every error from parse_frame (a length cap, an unknown status or network, a truncated field, an address the codec rejects) is returned as an io::Error from the core’s handle. The runtime then ends the connection, and every sub-flow with it. The messages are mux: metadata length N exceeds 512, mux: metadata length N below 4, mux: data length N exceeds 8192, mux: unknown session status N, mux: unknown target network N, and truncated input: … for a field cut short inside a complete metadata block. An address the codec rejects fails with the codec’s own message.
• Per-session problems do not end the carrier. Excess, duplicate and unknown sessions are declined with End. A failed dial or an outbound error on one sub-flow reaches the core as ConnectFailed or OutboundError for that FlowKey::Sub, and on_outbound_gone sends End and closes only that key. A refused datagram send arrives as Event::SendFailed, is logged at debug level and changes nothing.
• The carrier’s uplink ends. On Event::TransportEof, or the VMess uplink terminator chunk, the core calls on_transport_eof, which pushes Effect::Close for every live sub-flow, then moves to Done and pushes ShutdownTransport and Finish. VMess also seals its downlink terminator with terminate before ShutdownTransport. Sub-flows in Retiring are not in sessions; Finish ends the runtime and drops them with it.
• Idle carrier. RELAY_IDLE_TIMEOUT passes with no byte event in either direction. Timing::expired pushes Finish, and the carrier ends.
• Staging shortfall. A fx.stage that finds less room than promised returns staging_full() and ends the carrier. A core that keeps its STAGING_RESERVE honest never reaches this on the downlink. On the uplink it can happen when declines exceed the room left.
• Cancellation. The demultiplexer owns no task, timer or socket. Dropping the runtime drops the core, the Demux and every outbound with it.

Limits

Constant	Value	Where	Effect
MAX_META_LEN	512 bytes	frame.rs	Longer metadata is an error
MIN_META_LEN	4 bytes	frame.rs (private)	Shorter metadata is an error
MAX_DATA_LEN	8192 bytes	frame.rs	Longer uplink data is an error; downlink stream bytes are split at it; a longer downlink packet is dropped
FRAME_OVERHEAD_MAX	268 bytes	frame.rs	Worst-case header of a server-emitted frame
MAX_SESSIONS	256	demux.rs	Live session ids per carrier; further News are declined
downlink_overhead(16384)	536 bytes	demux.rs	Staging reserve share for framing one Trojan or VLESS outbound read
MUX_PORT	9527	mod.rs	Informational only
RELAY_IDLE_TIMEOUT	300 s	protocols/src/core/mod.rs	Idle limit of the whole carrier
MAX_PAYLOAD	1966 bytes	protocols/src/vmess/framing.rs	Largest VMess body chunk that downlink frames are sealed into

Known limitations

These are functional limitations at the revision this page describes. They are by design:

• Head-of-line blocking on the uplink. One sub-flow whose outbound is still connecting or not writable holds the carrier’s uplink for all sub-flows. With a held forward pending (a VMess frame that straddled chunks), the pin rule also holds the downlink until it is applied.
• No per-sub-flow idle timeout. A live sub-flow lasts until the client ends it, its outbound ends, or the carrier ends.
• No XUDP session resumption. The global id is decoded and ignored.
• A duplicate New is declined with an End for the live id. The live session is left open on the server, while the client is told that the id has ended.
• Server side only. The crate has no mux client, so etemenanki-app and katana outbounds never multiplex.

History: etemenanki-protocols 2.0.0 sent downlink frames a second time on VLESS and Trojan carriers and scrambled VMess mux uploads. 2.0.1 fixes both, and katana 3.0.1 moves its lockfile to 2.0.1, so its nodes no longer do either (Known issues).

Tests

Test	File	Pins
parses_new_tcp_session	protocols/tests/unit/mux/frame.rs	New metadata with a port-first IPv4 target; no global id on TCP
parses_new_udp_session_with_global_id	same	The 8-byte global id after a UDP target
keep_frame_carries_an_address_only_when_flagged_udp	same	A UDP Keep’s peer address; nothing parsed from a stream Keep
end_and_keepalive_carry_no_target	same	End and KeepAlive metadata
rejects_unknown_status_and_network	same	Status 0x09 and network 0x07 are errors
rejects_truncated_metadata	same	Short metadata and a cut IPv4 address are errors
read_meta_enforces_the_length_caps	same	513-byte and 3-byte metadata, and 8193-byte data, are refused
encoded_keep_round_trips_through_the_reader	same	encode_keep for a stream reads back exactly
encoded_udp_keep_carries_the_peer_address	same	encode_keep with a UDP peer
encoded_end_has_no_data_block	same	encode_end layout
frames_parse_from_slices_and_stage_into_buffers	same	parse_frame waiting on partial frames, the staging encoders, and FRAME_OVERHEAD_MAX fitting a 255-byte domain exactly
stream_sessions_open_forward_and_end_with_fresh_generations	protocols/tests/unit/mux/demux.rs	Open and Forward order, absolute ranges, the partial frame left unconsumed, Shutdown on End, a new generation on reuse, a retired key framing nothing, Close and End on on_outbound_gone
unknown_or_excess_sessions_are_declined_with_end	same	End for an unknown Keep; exactly MAX_SESSIONS Opens and an End for the next id
udp_sessions_address_each_packet_and_tag_replies	same	Per-frame peers, the fallback to the New target, reply tagging
a_downlink_frame_is_not_sent_again_by_the_next_uplink	same	After a UDP reply is read in place with out(), as Trojan and VLESS stage it, the next uplink feed leaves nothing in out
a_frame_straddling_chunks_is_held_and_forwarded_from_the_held_buffer	same	feed_chunks holding a tail; the next event’s chunk, at offset 1000 of its read, completing it; ForwardHeld over the held buffer and an absolute Forward for the following frame; trimming at the next event
trojan_carries_mux_when_the_connect_names_the_carrier	same	A Trojan CONNECT to v1.mux.cool:9527 becomes a carrier; framed reply; End and Close on outbound EOF; Finish on transport EOF
vless_answers_mux_at_once_and_demultiplexes	same	RESPONSE_HEADER staged at once; a UDP sub-flow’s SendTo and its tagged reply; a next uplink packet to another peer that stages nothing
vmess_demultiplexes_across_chunk_boundaries	same	A frame split across two sealed chunks; downlink framed and sealed into chunks a client can open
vmess_keeps_every_frame_one_read_completes	same	One read of three sealed chunks that completes two straddling frames: both ForwardHeld ranges resolve to their own payloads, and the last frame is forwarded in place
mux_command_synthesises_its_destination	protocols/tests/unit/vless/protocol.rs	A VLESS mux header yields mux_destination() and leaves the first frame unread
mux_request_header_roundtrips	same	A VLESS mux header is 19 bytes with no address
stalled_outbound_holds_uplink_but_not_other_downlink	concepts/tests/runtime.rs	A stalled forward holds the uplink, not other keys’ downlink
held_bytes_are_forwarded_after_the_dial_and_survive_later_rewrites	same	The pin rule a held forward relies on
a_held_range_past_the_buffer_is_rejected	same	RangeOutOfBounds for a held range past held()
vless_mux_tcp_single_stream	app/tests/integration/e2e_xray_mux.rs	One stream over a VLESS carrier from a real Xray client
vless_mux_tcp_concurrent_streams_stay_separate	same	Eight concurrent 4 KiB streams on one carrier each get their own bytes
vless_mux_over_ws_tls	same	A carrier under WebSocket and TLS
vless_xudp_datagram_roundtrip	same	UDP over a VLESS carrier with xudpConcurrency
vmess_mux_tcp_single_stream	same	Mux frames inside the VMess chunk stream
vmess_mux_payload_spans_both_framings	same	A 64 KiB echo split by both framings
vmess_xudp_datagram_roundtrip	same	UDP over a VMess carrier
xudp_attributes_replies_to_the_right_peer	same	Two peers on one XUDP session, each reply attributed by its frame address
trojan_mux_tcp_single_stream	same	The address-based Trojan signal over WebSocket and TLS

Run them from the Etemenanki workspace:

cargo test -p etemenanki-protocols mux
cargo test -p etemenanki-concepts --test runtime
cargo test -p etemenanki-app --test integration mux

The integration tests build Xray from the reference tree with Go. Without Go, or when the build fails, they print a SKIP: line and pass. The xudp two-peer test paces its two exchanges on purpose: back to back, they trip a buffer-aliasing race in Xray’s own mux client, which reports an earlier packet with a newer address. How the suites are organised is described in Testing.

Related

Server coreProxyCoreDecode, events, effects, keys and the held buffer.

Server runtimeHow effects are applied, the staging reserve and the pin rule.

TrojanThe carrier that signals mux by address.

VLESSThe carrier that signals mux with command 0x03.

VMess wire and coreThe chunk stream a VMess mux carrier travels inside.

SniffingThe TLS and HTTP sniffers a New payload is passed to.

App outboundsWhere FanOutLink routes XUDP packets one by one.

Known issuesFunctional limitations across the kernel and katana.