Transports: TCP and TLS

The transports module in etemenanki-protocols sits between a TCP socket and a protocol. On the accept side, InboundTransport turns one accepted socket into one or more byte streams and hands each to a sink. On the dial side, TransportConnector resolves an upstream proxy server, connects to it and wraps the socket in the same layers. Both produce a TransportStream, so every server core and client codec is written once against a single stream type and never learns whether TLS, WebSocket or HTTP/2 sits underneath.

This page covers the module’s frame (the two enums, the stream type, keepalive and the handshake timeout), the plain TCP and TLS layers in full, and the app code that maps a [inbound.stream] or [outbound.stream] block onto these types. The WebSocket and gRPC layers have their own page, Transports: WebSocket and gRPC; they are mentioned here only where they share code with TCP and TLS.

Responsibilities

Concern	Owner	Deliberately left to
Keepalive on every TCP socket that passes through InboundTransport::accept or TransportConnector::dial	protocols/src/transports/keepalive.rs → set_keepalive	nothing; applied unconditionally on those two paths
Bounding the accept-side transport handshake	protocols/src/transports/accept.rs → within	the protocol core for its own handshake
Turning one socket into one or many streams	InboundTransport::accept	the caller’s sink, which spawns the per-stream task
Resolving and connecting to an upstream server	TransportConnector::dial	Resolver, AddressFamilyStrategy, TcpDialer
TLS protocol versions and cipher policy	protocols/src/transports/tls/config.rs	callers choose certificates, trust and ALPN only
Deciding which transport a config asks for	app/src/transport.rs → resolve_stream	the inbound and outbound builders turn the shape into values

The module never parses a proxy protocol, never routes and never counts bytes. A core that receives a TransportStream sees AsyncRead + AsyncWrite + Unpin and nothing else.

The layers

Every transport starts from a tokio TcpStream. TLS wraps it in tokio_openssl::SslStream. WebSocket and gRPC run over MaybeTlsStream, which is either the plain socket or the TLS stream, so they share one I/O path with and without TLS. TransportStream is the enum at the top that the rest of the code sees.

flowchart BT
  tcp["TcpStream"]
  ssl["SslStream of TcpStream"]
  mts["MaybeTlsStream"]
  ws["WsStream of MaybeTlsStream"]
  grpc["GrpcStream"]
  ts["TransportStream"]
  tcp -- "Tcp" --> ts
  tcp --> ssl
  ssl -- "Tls" --> ts
  tcp -- "Plain" --> mts
  ssl -- "Tls" --> mts
  mts --> ws
  mts --> grpc
  ws -- "Ws" --> ts
  grpc -- "Grpc" --> ts

The labels on the edges are the enum variants: TransportStream::Tcp, TransportStream::Tls and so on on the way into TransportStream, and MaybeTlsStream::Plain / MaybeTlsStream::Tls on the way into MaybeTlsStream.

Key types

TransportStream

protocols/src/transports/stream.rs → TransportStream

pub enum TransportStream {
    Tcp(TcpStream),
    Tls(Box<SslStream<TcpStream>>),
    Ws(Box<WsStream<MaybeTlsStream>>),
    Grpc(Box<GrpcStream>),
}

TransportStream implements AsyncRead and AsyncWrite by matching on the variant and forwarding poll_read, poll_write, poll_flush and poll_shutdown to the inner stream through Pin::new. Every inner type is Unpin, so no pin projection is needed. It does not override poll_write_vectored, so vectored writes fall back to the default single-buffer write.

Variant	Inner type	Boxed	Produced by
Tcp	TcpStream	no	InboundTransport::Tcp, TransportKind::Tcp
Tls	SslStream<TcpStream>	yes, OpenSSL’s wrapper is large	InboundTransport::Tls, TransportKind::Tls
Ws	WsStream<MaybeTlsStream>	yes	InboundTransport::Ws, TransportKind::Ws
Grpc	GrpcStream	yes	InboundTransport::Grpc, TransportKind::Grpc

protocols/src/transports/accept.rs also names the accept-side alias:

pub type Accepted = TransportStream;

InboundTransport and accept

protocols/src/transports/accept.rs → InboundTransport

#[derive(Clone)]
pub enum InboundTransport {
    Tcp,
    Tls(ServerConfig),
    Ws {
        route: WsRoute,
        tls: Option<ServerConfig>,
    },
    Grpc {
        paths: Arc<GrpcPaths>,
        tls: Option<ServerConfig>,
    },
}

impl InboundTransport {
    pub fn ws(path: impl AsRef<str>, host: Option<&str>, tls: Option<ServerConfig>) -> Self;
    pub fn grpc(service: impl AsRef<str>, tls: Option<ServerConfig>) -> Self;
    pub async fn accept<F>(&self, tcp: TcpStream, mut sink: F) -> io::Result<()>
    where
        F: FnMut(Accepted);
}

InboundTransport is Clone and cheap to clone: ServerConfig holds an Arc<SslAcceptor>, WsRoute holds Arc<str> values, and the gRPC paths sit behind an Arc. The app builds one value per inbound and clones it for each accepted socket, so every connection shares one OpenSSL context.

accept does three things, in this order:

1. It calls set_keepalive(&tcp) on the accepted socket.
2. It runs the transport’s own handshake inside within, which wraps the future in tokio::time::timeout(TRANSPORT_HANDSHAKE_TIMEOUT, …).
3. It hands each resulting stream to sink.

Variant	Step bounded by TRANSPORT_HANDSHAKE_TIMEOUT	within label	Streams yielded	accept returns
Tcp	none, there is no handshake	none	exactly one	right after the sink call
Tls	ServerConfig::accept	"tls"	exactly one	right after the sink call
Ws	optional TLS, then WsStream::accept (the HTTP upgrade)	"websocket"	exactly one	right after the sink call
Grpc	optional TLS, then the HTTP/2 server handshake from configured_server_builder()	"grpc"	one per accepted HTTP/2 stream	when the HTTP/2 connection ends

For Ws and Grpc the TLS handshake and the layer above it share one 10-second budget, because both run inside the same within future. On expiry the error is io::ErrorKind::TimedOut with the text "<label> handshake timed out", for example tls handshake timed out.

The limit covers only the transport step. The protocol handshake that follows on the yielded stream (a Trojan password, a VLESS header) is bounded by the protocol layer: for the core-based protocols, app/src/serve.rs → drive watches it with etemenanki_protocols::core::HANDSHAKE_TIMEOUT (also 10 s), and the SOCKS driver has a timeout of its own (see Serving). The budgets are separate: the protocol handshake’s clock starts only after accept has handed the stream to the sink.

pub const TRANSPORT_HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(10);

async fn within<T>(what: &str, fut: impl Future<Output = io::Result<T>>) -> io::Result<T>;

gRPC: the sink is called per HTTP/2 stream

For Grpc, accept hands the handshaken h2::server::Connection to a private driver:

async fn serve_h2<T, F>(
    mut conn: Connection<T, Bytes>,
    paths: &GrpcPaths,
    sink: &mut F,
) -> io::Result<()>
where
    T: tokio::io::AsyncRead + tokio::io::AsyncWrite + Unpin,
    F: FnMut(Accepted);

The driver loops over a tokio::select! with three arms:

• conn.accept(): a new request. GrpcPaths::classify maps the request path to GrpcMode::Gun (/<service>/Tun) or GrpcMode::Multi (/<service>/TunMulti). An unknown path gets RST_STREAM with REFUSED_STREAM, and the loop continues. A known path gets the gRPC response headers, and the stream is wrapped as GrpcStream::served(send, recv, mode, &count) and passed to sink. When conn.accept() returns None the loop ends with Ok(()). An error from it ends the whole connection with that error.
• count.changed(), enabled only while streams are live: a stream ended. StreamCount is a shared AtomicUsize of live streams plus a Notify; each served GrpcStream holds a guard that increments the count on creation and decrements it and notifies on drop.
• liveness.watch(idle): Liveness gives an HTTP/2 connection that carries no streams H2_IDLE_TIMEOUT (300 s) to open one, and sends a PING every H2_KEEPALIVE_INTERVAL (60 s) that must be answered within H2_KEEPALIVE_TIMEOUT (20 s). Both of the other arms call note_progress, which restarts the idle deadline. When Liveness judges the peer gone, the loop ends with Ok(()).

The details of Liveness, the gRPC framing and the HTTP/2 settings are on Transports: WebSocket and gRPC.

sequenceDiagram
  participant L as Accept loop
  participant T as InboundTransport::accept
  participant H as serve_h2
  participant S as sink
  L->>T: accept(tcp, sink)
  T->>T: set_keepalive
  T->>T: within("grpc", TLS + h2 handshake)
  T->>H: serve_h2(conn, paths, sink)
  loop each HTTP/2 stream
    H->>H: classify(path)
    alt unknown path
      H-->>H: RST_STREAM REFUSED_STREAM
    else Tun or TunMulti
      H->>S: sink(TransportStream::Grpc)
      S-->>L: spawns a task for the stream
    end
  end
  H-->>T: connection ended or peer judged dead
  T-->>L: Ok(())

The sink must not block

sink is a synchronous FnMut(Accepted), and for gRPC it runs inside the loop that drives the whole HTTP/2 connection. Anything slow in it stalls every stream on that connection. The sink should only spawn a task and return. app/src/serve.rs → serve_socket does exactly that: it calls spawn_scoped with serve_connection for each stream.

TransportKind and TransportConnector

protocols/src/transports/connect.rs → TransportKind, TransportConnector

#[derive(Clone)]
pub enum TransportKind {
    Tcp,
    Tls(ClientConfig),
    Ws {
        target: WsTarget,
        tls: Option<ClientConfig>,
    },
    Grpc {
        authority: Arc<str>,
        service: Arc<str>,
        mode: GrpcMode,
        user_agent: Option<Arc<str>>,
        tls: Option<ClientConfig>,
    },
}

impl TransportKind {
    pub fn ws(host: impl AsRef<str>, path: impl AsRef<str>, tls: Option<ClientConfig>) -> Self;
    pub fn grpc(
        authority: impl AsRef<str>,
        service: impl AsRef<str>,
        tls: Option<ClientConfig>,
    ) -> Self;
    pub fn multi(mut self) -> Self;
    pub fn user_agent(mut self, agent: Option<&str>) -> Self;
    async fn wrap(&self, tcp: TcpStream) -> io::Result<TransportStream>;
}

#[derive(Clone)]
pub struct TransportConnector {
    kind: Arc<TransportKind>,
    dialer: Dialer,
    resolver: Resolver,
    strategy: AddressFamilyStrategy,
}

impl TransportConnector {
    pub fn new(
        kind: TransportKind,
        dialer: Dialer,
        resolver: Resolver,
        strategy: AddressFamilyStrategy,
    ) -> Self;
    pub fn tcp() -> Self;
    pub async fn dial(&self, dest: &Destination) -> io::Result<TransportStream>;
}

TransportKind::grpc starts in GrpcMode::Gun with DEFAULT_USER_AGENT (a desktop Chrome user agent). multi switches to GrpcMode::Multi, and user_agent(None) drops the header. Both builders do nothing on a non-gRPC kind. The app calls neither, so its gRPC outbounds always open /<service>/Tun with the default user agent; the tests call multi directly. TransportConnector::tcp() is plain TCP with Dialer::default(), Resolver::default() (the system resolver) and AddressFamilyStrategy::default() (Auto).

dial runs four steps:

flowchart LR
  d["dial(dest)"] --> u{"dest.network is Udp?"}
  u -- "yes" --> e["Err Unsupported"]
  u -- "no" --> r["destination_to_socketaddrs"]
  r --> c["TcpDialer::connect_any"]
  c --> k["set_keepalive"]
  k --> w["TransportKind::wrap"]
  w --> s["TransportStream"]

1. Refuse UDP. A Destination whose network is DialNetwork::Udp fails at once with io::ErrorKind::Unsupported and the text a proxy transport carries no datagrams of its own. Every other DialNetwork value is dialed as TCP.
2. Resolve. protocols/src/helpers/address_family.rs → destination_to_socketaddrs calls resolve_candidates("dial", …) with the connector’s Resolver and AddressFamilyStrategy and FamilySupport::both(). An IP literal skips the resolver. Every answer is kept, not only the first, and filtered by the strategy; PreferIpv4 and PreferIpv6 reorder with a stable sort and keep the other family as a fallback.
3. Connect. environment/src/dial/tcp.rs → TcpDialer::connect_any tries the addresses in order, one at a time. Each attempt is bounded by the dialer’s connect timeout (DEFAULT_CONNECT_TIMEOUT, 10 s). The first success wins; if all fail, the error lists every address with its failure.
4. Keepalive and wrap. set_keepalive runs on the connected socket, then TransportKind::wrap layers TLS, WebSocket or gRPC on top.

TransportConnector implements the concepts crate’s Connector for a Destination:

concepts/src/link.rs

pub trait Connector<Target> {
    type Stream: AsyncRead + AsyncWrite + Unpin;
    type Datagram: DatagramLink;
    type Future: Future<Output = io::Result<Outbound<Self::Stream, Self::Datagram>>>;

    fn connect(&mut self, target: Target) -> Self::Future;
}

pub enum NoDatagram {
    Never(Infallible),
}

// protocols/src/transports/connect.rs
type DialFuture =
    Pin<Box<dyn Future<Output = io::Result<Outbound<TransportStream, NoDatagram>>> + Send>>;

impl Connector<Destination> for TransportConnector {
    type Stream = TransportStream;
    type Datagram = NoDatagram;
    type Future = DialFuture;

    fn connect(&mut self, dest: Destination) -> DialFuture;
}

connect clones the connector (an Arc bump for the kind, plus clones of the dialer and resolver) into a boxed 'static + Send future, so concurrent dials share nothing mutable. It maps the result to Outbound::Stream. NoDatagram is uninhabited, so Outbound::Datagram cannot be built from this connector at all. A proxy’s UDP travels inside its stream: the client codec, not the transport, frames it.

This connector is the Conn parameter of concepts/src/client.rs → ProxyClientConnector for every stream-based client in the app (app/src/outbound/proxy.rs holds a ProxyClientConnector<BUF, Make<S, D>, TransportConnector, Destination>). The SOCKS outbound also calls dial directly to open its UDP ASSOCIATE control stream. katana consumes InboundTransport and TransportConnector from the published crate: its src/inbound.rs → build_transport builds an InboundTransport from the panel’s node description, and its src/outbound/proxy.rs uses the same ProxyClientConnector shape as the app. Its outbounds (src/outbound/mod.rs → build_outbound) always use TransportKind::Tcp: katana v3.0.1 builds no TLS, WebSocket or gRPC dial transport. katana’s mapping from panel fields to these types is its own and is not described here.

TLS: ServerConfig, ClientConfig, Alpn, VerifyMode

TLS uses the system OpenSSL through the openssl and tokio-openssl crates. The module keeps the OpenSSL builders private: callers choose the certificate material, the trust policy and ALPN, and protocols/src/transports/tls/config.rs owns the profile and the protocol versions.

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum Alpn {
    None,
    Http1,
    Http2,
    Http2ThenHttp1,
}

#[derive(Clone, Debug, Eq, PartialEq)]
pub enum VerifyMode {
    System,
    CustomCa(Vec<u8>),
    Insecure,
}

#[derive(Clone)]
pub struct ServerConfig {
    acceptor: Arc<SslAcceptor>,
}

impl ServerConfig {
    pub fn from_pem(cert_pem: &[u8], key_pem: &[u8], alpn: Alpn) -> io::Result<Self>;
    pub async fn accept(&self, tcp: TcpStream) -> io::Result<SslStream<TcpStream>>;
}

#[derive(Clone)]
pub struct ClientConfig {
    connector: Arc<SslConnector>,
    sni: Arc<str>,
    verify_hostname: bool,
}

impl ClientConfig {
    pub fn new(sni: impl AsRef<str>, verify: bool, alpn: Alpn) -> io::Result<Self>;
    pub fn with_verify_mode(
        sni: impl AsRef<str>,
        verify_mode: VerifyMode,
        alpn: Alpn,
    ) -> io::Result<Self>;
    pub async fn wrap(&self, tcp: TcpStream) -> io::Result<SslStream<TcpStream>>;
}

ALPN

Alpn maps to fixed wire bytes: length-prefixed protocol names, as ALPN encodes them.

Variant	Constant	Wire bytes
Alpn::None	none	ALPN is not configured
Alpn::Http1	ALPN_HTTP1	\x08http/1.1
Alpn::Http2	ALPN_H2	\x02h2
Alpn::Http2ThenHttp1	ALPN_H2_HTTP1	\x02h2\x08http/1.1

A server with ALPN installs a select callback that runs select_next_proto(server_protos, client_protos). When the client offers nothing in common, the callback returns AlpnError::NOACK: the handshake goes on without an ALPN extension instead of failing. A client with ALPN calls set_alpn_protos and offers the list. Alpn::Http2ThenHttp1 exists in the API, but no caller in the workspace or in katana uses it at this revision.

Server side

ServerConfig::from_pem builds the acceptor in this order:

1. Parse the PEM bundle with X509::stack_from_pem. The first certificate is the leaf; if there is none, it fails with InvalidInput and no certificate in PEM bundle.
2. Parse the private key with PKey::private_key_from_pem.
3. Start from SslAcceptor::mozilla_intermediate_v5(SslMethod::tls()), the openssl crate’s rendering of Mozilla’s intermediate server profile (v5). It sets NO_TLSV1 | NO_TLSV1_1, loads the RFC 7919 ffdhe2048 DH group, and fixes the cipher lists shown below. It leaves the key-exchange groups to OpenSSL’s defaults.
4. Set the minimum version to TLS 1.2 explicitly with set_min_proto_version(Some(SslVersion::TLS1_2)), so the floor does not depend on the option bits alone. TLS 1.3 stays enabled.
5. Install the key and the leaf, add every further certificate in the bundle with add_extra_chain_cert, and run check_private_key, so a key that does not match the leaf fails at build time.
6. Install the ALPN select callback if alpn is not Alpn::None.

Neither side sets a cipher list of its own, so the cipher configuration comes from the openssl crate (0.10.81 in Cargo.lock at this revision), not from this repository:

Side	TLS 1.2 cipher list	TLS 1.3 suites
Server (mozilla_intermediate_v5)	ECDHE-ECDSA-AES128-GCM-SHA256, ECDHE-RSA-AES128-GCM-SHA256, ECDHE-ECDSA-AES256-GCM-SHA384, ECDHE-RSA-AES256-GCM-SHA384, ECDHE-ECDSA-CHACHA20-POLY1305, ECDHE-RSA-CHACHA20-POLY1305, DHE-RSA-AES128-GCM-SHA256, DHE-RSA-AES256-GCM-SHA384	TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256
Client (SslConnector::builder)	DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:!aDSS:!SRP:!PSK	OpenSSL’s defaults

Both builders also start from the crate’s common context options, which include NO_COMPRESSION, NO_SSLV2 and NO_SSLV3. An upgrade of the openssl crate can change these lists; check them when you bump it.

accept creates a fresh Ssl from the shared context for each connection and drives the server handshake with SslStream::accept. OpenSSL errors become io::Error::other.

Client side

ClientConfig::with_verify_mode starts from SslConnector::builder(SslMethod::tls()), which already enables peer verification and loads the default trust paths, and sets the minimum version to TLS 1.2. The verification policy then decides the rest:

VerifyMode	Chain verified against	Hostname checked	Builder calls
System	the platform/OpenSSL default trust store	yes	set_default_verify_paths
CustomCa(pem)	the default trust store plus every certificate in pem	yes	set_default_verify_paths, then cert_store_mut().add_cert per certificate
Insecure	nothing	no	set_verify(SslVerifyMode::NONE)

CustomCa adds the given CAs to the system roots; it does not replace them. A bundle with no certificate in it fails with InvalidInput and no certificate in CA PEM bundle.

ClientConfig::new(sni, verify, alpn) is shorthand: verify = true means VerifyMode::System, and false means VerifyMode::Insecure.

Hostname checking is set per session, not on the builder. OpenSSL’s ConnectConfiguration turns it on by default, so ClientConfig stores verify_hostname and wrap calls set_verify_hostname(false) for Insecure. wrap then calls into_ssl(&self.sni), which sends the name as SNI and, when hostname checking is on, checks the certificate against it. Both behaviours come from the openssl crate:

• a DNS name is sent as SNI and checked against the certificate’s DNS names, with partial wildcards refused;
• an IP literal is not sent as SNI (the extension carries only host names) and is checked against the certificate’s IP addresses.

Other users of the TLS types

ClientConfig and VerifyMode are not private to the transports. protocols/src/dns/mod.rs → Resolver::new builds a ClientConfig for the DNS-over-TLS backend (Alpn::None) and the DNS-over-HTTPS backend (Alpn::Http1), with CustomCa when a CA is configured and System otherwise. protocols/src/hysteria/connection.rs reuses the VerifyMode enum as its trust vocabulary but translates it into a rustls configuration, because QUIC does not run over OpenSSL here. A change to VerifyMode’s meaning therefore reaches three places: the TCP transports, the DNS resolver and the Hysteria 2 client.

MaybeTlsStream and the helpers

protocols/src/transports/tls/stream.rs

pub enum MaybeTlsStream {
    Plain(TcpStream),
    Tls(Box<SslStream<TcpStream>>),
}

pub fn tcp_from_std(tcp: std::net::TcpStream) -> io::Result<TcpStream>;

pub async fn accept_optional_tcp(
    tcp: std::net::TcpStream,
    server: Option<ServerConfig>,
) -> io::Result<MaybeTlsStream>;

pub async fn wrap_optional_tcp(
    tcp: TcpStream,
    client: Option<ClientConfig>,
) -> io::Result<MaybeTlsStream>;

MaybeTlsStream forwards AsyncRead and AsyncWrite the same way TransportStream does. accept.rs and connect.rs each have a private maybe_tls that builds it from an Option of the server or client config; that is how the WebSocket and gRPC variants get optional TLS.

tcp_from_std sets a std socket non-blocking, adopts it into tokio and applies set_keepalive. accept_optional_tcp (which goes through tcp_from_std) and wrap_optional_tcp are the public, owned-config versions of maybe_tls. The doc comment on tcp_from_std calls it the single point every inbound transport converts through, but at this revision nothing in the workspace or in katana calls these three helpers: InboundTransport::accept takes a tokio TcpStream and sets keepalive itself.

Keepalive

protocols/src/transports/keepalive.rs

const TCP_KEEPALIVE_IDLE: Duration = Duration::from_secs(120);
const TCP_KEEPALIVE_INTERVAL: Duration = Duration::from_secs(30);
const TCP_KEEPALIVE_RETRIES: u32 = 3;

pub fn set_keepalive(stream: &TcpStream);

set_keepalive applies a socket2::TcpKeepalive with the three values above through SockRef::from(stream). The kernel sends the first probe after 120 s of silence and repeats it every 30 s; after 3 unanswered probes it resets the socket. A peer that vanished without a FIN is therefore noticed about 210 s after the last traffic, instead of the Linux default of more than two hours.

The call is best-effort. If the platform refuses the option, it logs could not enable TCP keepalive: … at debug and the connection goes on. Keepalive only catches dead peers: a live peer that answers probes but sends nothing is reclaimed by the idle timeouts above this layer, not here. A socket that has already sent its FIN is not probed at all.

environment/src/dial/socket.rs → SocketOptions has its own tcp_keepalive: Option<Duration>, which TcpDialer::socket applies before connecting. On a socket dialed through TransportConnector, set_keepalive runs after the connect and overwrites it, so the fixed schedule above always wins. The app passes Dialer::default(), where that field is None, so the two do not meet today.

Only the two paths in this module call set_keepalive. A socket that never passes through them does not get the schedule: the freedom outbound (app/src/outbound/freedom.rs → FreedomConnector) dials its destination with TcpDialer::connect_any directly, so a direct connection keeps the kernel’s keepalive defaults.

From configuration to transport

app/src/transport.rs holds the stream-settings rules that the inbound and outbound builders share. inbound::build_inbound_transport and outbound::build_transport are near-duplicates of each other, and a rule written into only one of them is a rule the other lacks. So both call resolve_stream, and each only turns the resulting StreamShape into values. A new rule about which fields a network needs belongs in resolve_stream, not in a builder.

flowchart TB
  cfg["StreamConfig"] --> rs["resolve_stream"]
  rs --> tl["tls_layer"]
  rs --> shape["StreamShape"]
  shape --> ib["build_inbound_transport"]
  shape --> ob["build_transport"]
  ib --> it["InboundTransport"]
  ob --> tk["TransportKind"]
  tk --> tc["TransportConnector::new"]
  cfg -.->|"protocols without a transport"| rej["reject_stream_settings"]

StreamConfig

app/src/config.rs → StreamConfig, TlsConfig. Both are #[serde(deny_unknown_fields)], so a misspelt key fails parsing.

pub struct StreamConfig {
    pub network: Option<String>,
    pub security: Option<String>,
    pub tls: TlsConfig,
    pub ws: WsStreamConfig,
    pub grpc: GrpcStreamConfig,
}

pub struct TlsConfig {
    pub server_name: Option<String>,
    pub allow_insecure: bool,
    pub ca_file: Option<String>,
    pub cert_file: Option<String>,
    pub key_file: Option<String>,
}

The user-facing description of these keys is in the Transports guide.

tls_layer: validating security against network

pub fn tls_layer(network: &str, security: Option<&str>, ctx: &str) -> io::Result<bool>;

tls_layer answers “does TLS go under this network?”. It validates security against the network, instead of comparing it to the literal "tls", and fails closed. If an unrecognised value quietly meant “no TLS”, a listener or dialer would come up in plaintext against the operator’s intent. The only symptom would be a failed handshake after the proxy credential had already crossed the wire.

security is trimmed and then matched case-sensitively: None, "" and "none" mean no TLS, "tls" means TLS, and anything else, including "TLS", "reality" and "xtls", is an error.

network	security absent, "" or "none"	security = "tls"	any other security
tcp	plain TCP	error	error
tls	TLS	TLS (accepted as redundant)	error
ws	plain	TLS under WebSocket	error
grpc	plain	TLS under HTTP/2	error
anything else	Ok(false), left to the caller	Ok(false), left to the caller	Ok(false), left to the caller

The error texts, where ctx is inbound <tag> or outbound <tag>:

• <ctx>: unknown stream security "<value>" (expected "tls" or "none")
• <ctx>: security = "tls" is not valid with network = "tcp"; use network = "tls" for TLS over plain TCP (security = "tls" layers TLS under network = "ws" or "grpc")

network = "tcp" with security = "tls" is Xray’s way of writing TLS over TCP. Here it is refused with a message that names the local spelling, rather than being built as plaintext. A redundant security = "tls" beside network = "tls" is accepted, because a config carried over from Xray often writes both and saying TLS twice is not a contradiction. An unknown network returns Ok(false) on purpose, even when security is also invalid, so that resolve_stream can report the network with its own, clearer message.

network itself is not trimmed: " ws" is an unknown network.

resolve_stream and StreamShape

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StreamShape {
    Tcp,
    Tls,
    Ws {
        path: String,
        host: Option<String>,
        tls: bool,
    },
    Grpc {
        service: String,
        authority: Option<String>,
        tls: bool,
    },
}

pub fn resolve_stream(stream: &StreamConfig, ctx: &str) -> io::Result<StreamShape>;

resolve_stream defaults network to "tcp", runs tls_layer, and then builds the shape:

network	Shape	Required and defaulted fields	Error
"tcp"	StreamShape::Tcp	none	none
"tls"	StreamShape::Tls	none here; the builders read the TLS keys	none
"ws"	StreamShape::Ws	path defaults to "/"; host stays optional	none
"grpc"	StreamShape::Grpc	service from grpc.service_name is required	<ctx>: grpc stream needs grpc.service_name
other	none	none	<ctx>: unknown stream network "<value>"

Building each side

Inbound

app/src/inbound/mod.rs → build_inbound_transport(cfg: &InboundConfig) -> io::Result<InboundTransport>

Shape	InboundTransport	Server ALPN
Tcp	InboundTransport::Tcp	none
Tls	InboundTransport::Tls(ServerConfig::from_pem(…))	Alpn::None
Ws { tls: true, .. }	InboundTransport::ws(path, host, Some(…))	Alpn::Http1
Ws { tls: false, .. }	InboundTransport::ws(path, host, None)	none
Grpc { tls: true, .. }	InboundTransport::grpc(service, Some(…))	Alpn::Http2
Grpc { tls: false, .. }	InboundTransport::grpc(service, None)	none

Certificates are read by read_inbound_cert_key, only when the shape needs TLS. The inbound side reads only tls.cert_file and tls.key_file; tls.server_name, tls.allow_insecure and tls.ca_file are accepted by the parser and ignored. The TLS keys decide nothing on their own either: a ws or grpc stream with cert_file set but no security = "tls" is served in plaintext. It requires tls.cert_file and tls.key_file and reports inbound <tag>: tls stream needs tls.cert_file (or tls.key_file) when one is missing. A gRPC inbound ignores the shape’s authority: the server does not check the authority a client sends. A WebSocket inbound has no Host fallback; with no ws.host, any Host is accepted.

transport_for wraps the builder. On a Unix-socket listen, a transport has no meaning, so it calls reject_stream_settings with the protocol name "<proto> over a unix socket", and it runs the builder lazily so a Unix listener never reads certificate files.

Outbound

app/src/outbound/mod.rs → build_transport(cfg: &OutboundConfig, resolver: &Resolver) -> io::Result<TransportConnector>

Shape	TransportKind	Client ALPN	Name used
Tcp	TransportKind::Tcp	none	none
Tls	TransportKind::Tls(…)	Alpn::None	SNI from require_sni
Ws	TransportKind::ws(host, path, tls)	Alpn::Http1 when TLS	Host: ws.host, else tls.server_name, else server
Grpc	TransportKind::grpc(authority, service, tls)	Alpn::Http2 when TLS	:authority: grpc.authority, else tls.server_name, else server

require_sni uses tls.server_name and falls back to server. It refuses to guess when neither is set: <ctx>: <network> stream needs tls.server_name or server, where <network> is tls, ws+tls or grpc+tls. The WebSocket and gRPC name fallbacks fail the same way: <ctx>: ws stream needs ws.host or server, <ctx>: grpc stream needs grpc.authority or server.

client_tls_config picks the VerifyMode:

tls.allow_insecure	tls.ca_file	VerifyMode
false	not set	System
false	set	CustomCa(<file bytes>)
true	not set	Insecure
true	set	error: outbound <tag>: tls.allow_insecure and tls.ca_file cannot both be set

The outbound side never reads tls.cert_file or tls.key_file: ClientConfig has no client certificate, so those keys are accepted and ignored.

The connector gets Dialer::default(), the resolver the app built from its DNS configuration, and the strategy from the outbound’s address_family (default AddressFamilyStrategy::Auto; an unparsable value fails with outbound <tag>: invalid address_family "<value>").

reject_stream_settings

pub fn reject_stream_settings(stream: &StreamConfig, proto: &str, ctx: &str) -> io::Result<()>;

Some protocols never consult the stream block. The SOCKS, Shadowsocks, Hysteria 2 and TUN inbounds own their listener, are wired to a fixed transport or own a network interface; the freedom, blackhole, wireguard and hysteria2 outbounds dial on their own. For these, the builder calls reject_stream_settings (through a small reject_stream wrapper in each builder module that supplies the inbound <tag> or outbound <tag> context). It accepts a missing, empty or "tcp" network and a missing, empty or "none" security (both trimmed), and otherwise fails with <ctx>: protocol <proto> does not support stream network "<value>" or … does not support stream security "<value>". Without it, an operator who asked for WebSocket would get a bare port and no error.

Invariants

Invariant	Enforced by	Pinned by
Every stream a core sees has the same type, whatever the transport	TransportStream is the only Accepted type and the only Connector::Stream of TransportConnector	tcp_round_trip, tls_round_trip, ws_round_trip, grpc_round_trip in protocols/tests/pipeline/transports.rs
A TCP socket accepted by InboundTransport or dialed by TransportConnector always gets the keepalive schedule (best-effort)	set_keepalive at the top of InboundTransport::accept and after connect_any in TransportConnector::dial	no test pins it
The accept-side transport step cannot take longer than 10 s	within wraps the TLS, upgrade and HTTP/2 handshakes in tokio::time::timeout(TRANSPORT_HANDSHAKE_TIMEOUT, …)	no test pins it
A gRPC connection that opens no stream is released	Liveness in serve_h2	a_connection_that_opens_no_stream_is_given_up_on in protocols/tests/unit/transports/accept.rs
One gRPC connection yields one stream per HTTP/2 stream	serve_h2 calls sink for each accepted request	one_grpc_connection_carries_many_streams in protocols/tests/pipeline/transports.rs
A TLS server never negotiates below TLS 1.2	mozilla_intermediate_v5 plus set_min_proto_version(Some(SslVersion::TLS1_2))	server_rejects_tls10, server_rejects_tls11, server_accepts_tls12, server_accepts_tls13 in protocols/tests/unit/transports/tls_config.rs
A custom CA extends the trust store rather than skipping verification	VerifyMode::CustomCa loads default paths, then adds the certificates; hostname checking stays on	tls_with_a_pinned_ca_round_trip in protocols/tests/pipeline/transports.rs
A proxy transport never yields datagrams	dial refuses DialNetwork::Udp; Datagram = NoDatagram is uninhabited	transport_connector_refuses_udp in protocols/tests/pipeline/transports.rs
An unrecognised security never means plaintext	tls_layer returns an error for every value other than absent, "", "none" and "tls"	an_unknown_security_is_rejected_on_every_network, security_is_trimmed_before_matching in app/tests/unit/transport.rs
Xray’s tcp + tls spelling is refused with the fix in the message	tls_layer’s "tcp" if requested arm	tcp_with_tls_is_rejected_and_names_the_fix in app/tests/unit/transport.rs
network = "tls" always carries TLS; ws and grpc carry it only when asked	tls_layer’s match network	tls_network_carries_tls_with_or_without_a_redundant_security, ws_and_grpc_layer_tls_only_when_asked, tcp_without_security_is_plaintext in app/tests/unit/transport.rs
An unknown network is reported by resolve_stream, not tls_layer	tls_layer returns Ok(false) for it	an_unknown_network_is_left_to_the_caller in app/tests/unit/transport.rs
Inbound and outbound apply the same stream rules	both builders go through resolve_stream	structural; the tls_layer tests cover both
A protocol that cannot carry a transport refuses a stream block	reject_stream_settings	a_default_or_plain_tcp_stream_is_accepted, a_transport_the_protocol_cannot_honour_is_rejected, security_on_a_protocol_without_a_transport_is_rejected in app/tests/unit/transport.rs
An outbound never verifies a certificate against a guessed name	require_sni fails when neither tls.server_name nor server is set	no test pins it
allow_insecure and ca_file cannot be combined	client_tls_config	no test pins it for the TCP-based outbounds (the Hysteria 2 outbound’s own check is pinned by hysteria2_refuses_contradictory_certificate_settings in app/tests/unit/outbound.rs)

Failure paths and cancellation

Accept side

Failure	Error	Where it surfaces
Transport handshake exceeds 10 s	TimedOut, tls handshake timed out / websocket handshake timed out / grpc handshake timed out	returned from accept; the socket is dropped
TLS handshake fails (no shared version, bad ClientHello)	OpenSSL error as io::Error::other	returned from accept
gRPC request on an unknown path	none; the stream is reset with REFUSED_STREAM	the connection keeps serving
gRPC send_response fails	none; the stream is skipped	the connection keeps serving
conn.accept() returns an HTTP/2 error	io::Error::other	returned from accept; the connection ends

app/src/serve.rs → serve_socket logs any error from accept at debug as inbound transport failed: … and serves nothing further on that socket. Streams already handed to the sink run in their own tasks.

accept is a plain future, so dropping it cancels everything it owns. Before the sink runs, that is only the socket and the half-finished handshake. For gRPC, serve_h2 is the only thing that drives the HTTP/2 connection, so once it is dropped, the streams it yielded can no longer send or receive. In the app, both the accept future and every per-stream task run under spawn_scoped with the generation’s cancellation token; see Serving.

Dial side

Failure	Error kind	Text
UDP destination	Unsupported	a proxy transport carries no datagrams of its own
The resolver fails	as returned by Resolver::resolve	see DNS
Domain resolves to nothing	NotFound	dial: destination did not resolve
Nothing left after the family filter	AddrNotAvailable	dial: no usable <strategy> destination address for <remote>:<port>
Every address failed, for any reason	ConnectionRefused	failed to connect to any address (<addr>: <error>; …)
TLS handshake or verification fails	io::Error::other	OpenSSL’s error stack

connect_any never returns a single attempt’s error: an attempt that exceeds the connect timeout shows up only inside that list, as <addr>: connect to <addr> timed out, and the error kind is ConnectionRefused even when every attempt timed out.

The future returned by connect owns everything it touches, so dropping it at any await (the client runtime giving up, or the flow being cancelled) closes the socket and abandons the handshake.

Build time

Most configuration errors carry inbound <tag> or outbound <tag>, as listed above. Errors raised inside ServerConfig::from_pem, ClientConfig::with_verify_mode or the certificate file reads do not: an empty certificate bundle reports only no certificate in PEM bundle, a non-PEM CA file only no certificate in CA PEM bundle, and a missing file only the OS error. When you add a new failure to these functions, keep in mind that the operator sees it without the tag.

Limits

Constant	Value	Defined in	Applies to
TRANSPORT_HANDSHAKE_TIMEOUT	10 s	protocols/src/transports/accept.rs	accept-side TLS, WebSocket upgrade and HTTP/2 handshake, per socket
TCP_KEEPALIVE_IDLE	120 s	protocols/src/transports/keepalive.rs	silence before the first keepalive probe
TCP_KEEPALIVE_INTERVAL	30 s	protocols/src/transports/keepalive.rs	interval between probes
TCP_KEEPALIVE_RETRIES	3	protocols/src/transports/keepalive.rs	unanswered probes before the kernel resets the socket
DEFAULT_CONNECT_TIMEOUT	10 s	environment/src/dial/tcp.rs	each TCP connect attempt in connect_any
H2_IDLE_TIMEOUT	300 s	protocols/src/transports/grpc/settings.rs	a served gRPC connection with no live streams
H2_KEEPALIVE_INTERVAL	60 s	protocols/src/transports/grpc/settings.rs	how often a served gRPC connection sends a PING
H2_KEEPALIVE_TIMEOUT	20 s	protocols/src/transports/grpc/settings.rs	time for the peer to answer a PING on a served gRPC connection
Minimum TLS version	TLS 1.2	protocols/src/transports/tls/config.rs	both ServerConfig and ClientConfig

With several resolved addresses, the worst-case connect time is the number of addresses times DEFAULT_CONNECT_TIMEOUT, because the attempts in connect_any are sequential, not raced (this is not Happy Eyeballs).

Tests

File	Tests	What they pin
protocols/tests/pipeline/transports.rs	tcp_round_trip, tls_round_trip, tls_with_a_pinned_ca_round_trip, ws_round_trip, ws_over_tls_with_early_data_round_trip, ws_early_data_is_the_first_bytes_the_server_reads, ws_rejects_a_wrong_path_at_the_upgrade, grpc_round_trip, grpc_multi_mode_over_tls_round_trip, one_grpc_connection_carries_many_streams, transport_connector_refuses_udp	Every InboundTransport accepts what the matching TransportKind dials. assert_echo sends hello and a 200 000-byte payload, shuts down the write side, and expects a clean end of stream.
protocols/tests/unit/transports/tls_config.rs	server_accepts_tls12, server_accepts_tls13, server_rejects_tls10, server_rejects_tls11	The server’s version floor, using a raw SslConnector pinned to one version.
protocols/tests/unit/transports/accept.rs	a_connection_that_opens_no_stream_is_given_up_on	serve_h2 gives up on an idle HTTP/2 peer after H2_IDLE_TIMEOUT and not before, on a paused clock over tokio::io::duplex.
app/tests/unit/transport.rs	tcp_without_security_is_plaintext, tcp_with_tls_is_rejected_and_names_the_fix, tls_network_carries_tls_with_or_without_a_redundant_security, ws_and_grpc_layer_tls_only_when_asked, an_unknown_security_is_rejected_on_every_network, security_is_trimmed_before_matching, an_unknown_network_is_left_to_the_caller, a_default_or_plain_tcp_stream_is_accepted, a_transport_the_protocol_cannot_honour_is_rejected, security_on_a_protocol_without_a_transport_is_rejected	The (network, security) matrix row by row, and reject_stream_settings.
app/tests/integration/e2e_xray.rs	app_client_tls_xray_server_tls, app_server_tls_xray_client_tls	network = "tls" interoperates with Xray’s tcp + tls in both directions. These tests skip themselves when Go is not installed.

The pipeline tests share self_signed_pem, tcp_dest and udp_dest from protocols/tests/support. tls_pair builds a server and an Insecure client for localhost; the pinned-CA test instead passes the server’s own certificate as VerifyMode::CustomCa.

cargo test -p etemenanki-protocols --test pipeline transports
cargo test -p etemenanki-protocols --lib transports
cargo test -p etemenanki-app --bin etemenanki-app transport

Gaps worth closing

No test pins the keepalive schedule, TRANSPORT_HANDSHAKE_TIMEOUT, require_sni or the allow_insecure + ca_file refusal for TCP-based outbounds. A change to any of them should add one. For the timeout, the paused-clock pattern of a_connection_that_opens_no_stream_is_given_up_on works over tokio::io::duplex, but accept takes a real TcpStream, so such a test needs a loopback listener.

Related

Transports: WebSocket and gRPCThe upgrade, early data, gRPC framing, HTTP/2 settings and Liveness.

DialersTcpDialer, SocketOptions and the shared socket policy.

DNSThe Resolver that TransportConnector dials through.

Links and connector typesConnector, Outbound, NoDatagram and Destination.

Client runtimeProxyClientConnector, which uses TransportConnector as its Conn.

ServingThe accept loop that calls InboundTransport::accept and spawns each stream.

Transports (user guide)The stream keys as an operator writes them.