HTTP proxy
Source files: 19 · checked against Etemenanki 596916d · katana v3.0.1
Etemenanki/protocols/src/http/mod.rsEtemenanki/protocols/src/http/config.rsEtemenanki/protocols/src/http/protocol.rsEtemenanki/protocols/src/http/core.rsEtemenanki/protocols/src/http/codec.rsEtemenanki/protocols/src/core/mod.rsEtemenanki/protocols/src/sniff/mod.rsEtemenanki/protocols/src/helpers/address.rsEtemenanki/concepts/src/runtime.rsEtemenanki/concepts/src/client.rsEtemenanki/protocols/tests/unit/http/core.rsEtemenanki/protocols/tests/unit/http/protocol.rsEtemenanki/protocols/tests/unit/http/codec.rsEtemenanki/protocols/tests/pipeline/http.rsEtemenanki/app/src/config.rsEtemenanki/app/src/inbound/mod.rsEtemenanki/app/src/serve.rsEtemenanki/app/src/outbound/mod.rskatana/src/outbound/mod.rs
The HTTP proxy lives in protocols/src/http/. It has two halves that share one set of wire helpers:
HttpCore, the server side, is aProxyCoreDecodethat serves one HTTP/1.x proxy request per connection: aCONNECTtunnel, or a plain request with an absolute URI that it rewrites and forwards to the origin.HttpConnect, the client side, is aProxyCoreEncodecodec that sends oneCONNECTto an upstream proxy, waits for a200, and then passes bytes through unchanged.
This page is for contributors who change either half. It assumes you know the server-core contract (events, effects, the held buffer and the staging reserve) from Server cores, and the shared Timing, SniffPrefix and Passthrough helpers from Protocol foundations. The user-facing settings are on the HTTP proxy guide page.
Responsibilities
Section titled “Responsibilities”| Component | Does | Leaves to others |
|---|---|---|
protocol.rs |
Finds and parses request and response heads, holds the fixed responses, rewrites a forwarded request, builds a CONNECT request, checks Proxy-Authorization. |
All I/O. The helpers are pure functions over byte slices, except the unused async read_head. |
HttpCore |
Parses the first request head, authenticates it, opens exactly one outbound flow, answers CONNECT, stages the fixed error responses, relays both directions verbatim afterwards. |
Dialing, routing, reading and writing sockets, and the watchdog for a client that never sends a byte (the runtime and the application). |
HttpConnect |
Stages one CONNECT request, parses the upstream’s status line, then seals and opens bytes verbatim. |
Dialing the upstream proxy and any TLS around it (the client runtime and the outbound’s transport). UDP: HTTP proxies carry none. |
HttpServerConfig |
Holds the account map, the anonymous payload and allow_transparent. |
Parsing TOML. The app builds it from HttpInboundSettings in app/src/inbound/mod.rs. |
The helpers are a port of the HTTP proxy in Xray-core (proxy/http and common/protocol/http/headers.go); the sans-I/O core and codec around them are native to this crate.
Key types
Section titled “Key types”Configuration
Section titled “Configuration”protocols/src/http/config.rs → HttpServerConfig:
pub struct HttpServerConfig<T> { pub accounts: HashMap<CompactString, (CompactString, Arc<T>)>, pub anonymous: Arc<T>, pub allow_transparent: bool,}| Field | Meaning |
|---|---|
accounts |
username -> (password, payload). An empty map turns authentication off. |
anonymous |
The payload every flow carries when accounts is empty. It is ignored otherwise. |
allow_transparent |
Accept origin-form request targets (GET /path) and take the destination from Host. Default sets it to false. |
Clone is implemented by hand so that T needs no Clone bound (every payload sits behind an Arc). Default requires T: Default, because it builds the anonymous payload with T::default(). The core receives the config as Arc<HttpServerConfig<T>>, so one config is shared by every connection of a listener.
The app maps its settings one to one: each [[inbound.settings.accounts]] entry (Account { user, pass }) becomes a map entry with payload Arc::new(()), and allow_transparent is copied from HttpInboundSettings, which is #[serde(deny_unknown_fields, default)].
The server core
Section titled “The server core”protocols/src/http/core.rs → HttpCore:
pub struct HttpCore<T> { /* private */ }
impl<T> HttpCore<T> { pub const BUF_SIZE: usize = MAX_HEAD;
pub fn new(config: Arc<HttpServerConfig<T>>, sniff: bool, source: Option<IpAddr>) -> Self; pub fn is_established(&self) -> bool;}
impl<T: Send + Sync + 'static> ProxyCoreDecode for HttpCore<T> { type Key = Single; type Target = Flow<T>; type Error = io::Error; type TransportAddr = ();
const STAGING_RESERVE: usize = 256;
fn handle( &mut self, event: Event<'_, Self>, fx: &mut Effects<'_, Self>, ) -> Result<usize, io::Error>;
fn held(&self) -> &[u8];}The private fields tell you where every piece of state lives:
| Field | Type | Role |
|---|---|---|
config |
Arc<HttpServerConfig<T>> |
Accounts and allow_transparent. |
sniff |
bool |
Whether the inbound has sniffing on (the app passes the inbound’s sniffing setting, which defaults to true). Only a CONNECT whose target is an IP literal sniffs. |
source |
Option<IpAddr> |
The client address, copied into every Flow for routing. |
timing |
Timing |
The single deadline, armed per phase. is_established() reads it. |
prefix |
SniffPrefix |
The first tunnel bytes of a sniffing CONNECT, held until the flow opens. |
rewritten |
Vec<u8> |
The origin-form head of a forwarded plain request. |
state |
State<T> |
The state machine below. |
enum State<T> { Handshake, Sniff(Flow<T>), Relay { relay: Passthrough<Single>, reply: bool, }, Done,}reply is true only for a CONNECT whose 200 is still owed: the core stages it on Connected. A sniffing CONNECT and a plain request both enter Relay with reply: false.
held() returns rewritten when it is non-empty and the sniff prefix otherwise. At most one of the two is ever filled on a connection, because a plain request never sniffs and a CONNECT never rewrites.
BUF_SIZE is the read-buffer size a runtime over this core needs. The app instantiates it as drive::<{ HttpCore::<()>::BUF_SIZE }, _, _> in app/src/serve.rs, and the pipeline tests use the same constant.
The client codec
Section titled “The client codec”protocols/src/http/codec.rs → HttpConnect:
pub struct HttpConnect { /* private: request: Vec<u8> */ }
impl HttpConnect { pub const REQUEST_MAX: usize = 1024;
pub fn new(dest: &Destination, auth: Option<(&str, &str)>) -> Self;}
impl ProxyCoreEncodeHandshake for HttpConnect { type Target = Destination; type Error = io::Error; const STAGING_RESERVE: usize = Self::REQUEST_MAX;
fn start(&mut self, out: &mut Staging<'_>) -> io::Result<Handshake>; fn reply(&mut self, wire: &mut [u8], _: &mut Staging<'_>) -> io::Result<Reply>; fn finish(&mut self, _: &mut Staging<'_>) -> io::Result<()>;}
impl ProxyCoreEncode for HttpConnect { fn seal(&mut self, plain: &[u8], out: &mut Staging<'_>) -> io::Result<usize>; fn open(&mut self, wire: &mut [u8]) -> io::Result<Opened>;}new builds the whole request up front, so the codec’s only state is that byte vector, which start clears once staged. Both the app and katana wrap it as ProxyClient<HTTP_BUF, HttpConnect, NoUdp>; see Outbounds for the app and the katana outbound pool in Inbound and outbound.
Wire helpers
Section titled “Wire helpers”protocols/src/http/protocol.rs:
pub struct Header { pub name: String, pub value: Vec<u8>,}
pub struct RequestHead { pub method: String, pub target: String, pub headers: Vec<Header>,}
pub fn find_head_end(buf: &[u8]) -> Option<usize>;pub fn parse_request_head(head: &[u8]) -> io::Result<RequestHead>;pub fn parse_response_status(head: &[u8]) -> io::Result<u16>;pub fn build_connect_request(dest: &Destination, auth: Option<(&str, &str)>) -> Vec<u8>;pub fn header_value<'a>(headers: &'a [Header], name: &str) -> Option<&'a str>;pub fn parse_request_target(target: &str) -> (bool, Option<&str>, &str);pub fn build_forward_request( method: &str, origin_target: &str, host: &str, headers: &[Header],) -> Vec<u8>;pub fn check_proxy_auth<T>( headers: &[Header], accounts: &HashMap<CompactString, (CompactString, Arc<T>)>,) -> Option<(String, Arc<T>)>;pub async fn read_head<R>(reader: &mut R) -> io::Result<Vec<u8>>where R: AsyncRead + Unpin;| Helper | Behaviour |
|---|---|
find_head_end |
Offset just past the first \r\n\r\n, or None. It scans the whole slice each call. |
parse_request_head |
httparse::Request over at most MAX_HEADERS (128) headers, copied into owned Headers. A partial parse is an error, so the caller must pass a complete head. |
parse_response_status |
httparse::Response over the same header budget; returns only the status code. |
header_value |
First header whose name matches case-insensitively, as UTF-8. A non-UTF-8 value reads as absent. |
parse_request_target |
Splits a target into (is_https, authority, origin_form). The http:// and https:// prefixes match case-insensitively; the authority ends at the first /, ? or #; an absolute URI with nothing after the authority yields origin form /. Anything else returns (false, None, target). |
build_forward_request |
Builds the origin-form head a plain request is forwarded as (see the forwarded head). |
check_proxy_auth |
Verifies Proxy-Authorization: Basic … against the account map (see authentication). |
read_head |
An async byte-at-a-time head reader capped at MAX_HEAD. Nothing in the workspace calls it; the core uses find_head_end on the runtime’s read buffer instead. |
parse_authority and format_authority in protocols/src/helpers/address.rs convert between a host[:port] string and a TCP Destination. parse_authority(raw, default_port) trims surrounding whitespace, accepts a bracketed IPv6 literal with or without a port, takes default_port when the port is absent, keeps a domain unresolved, and rejects an unbracketed IPv6 literal, an invalid or empty port and an empty host. A host that does not parse as an IP address becomes a domain, including text inside brackets.
Wire format
Section titled “Wire format”Fixed responses
Section titled “Fixed responses”The server core never builds a response dynamically. It stages one of four constants from protocol.rs, all of which fit the 256-byte STAGING_RESERVE:
| Constant | Bytes | Sent when | Then |
|---|---|---|---|
CONNECT_ESTABLISHED |
39 | A CONNECT target connected, or immediately for a sniffing CONNECT |
Relay (Sniff for a sniffing CONNECT) |
RESP_407 |
106 | Credentials are required and missing or wrong | ShutdownTransport, Finish |
RESP_400 |
72 | A plain request is not absolute-form and allow_transparent is off |
ShutdownTransport, Finish |
RESP_502 |
47 | A CONNECT target failed to connect and the 200 is still owed |
ShutdownTransport, Finish |
HTTP/1.1 200 Connection established
HTTP/1.1 407 Proxy Authentication RequiredProxy-Authenticate: Basic realm="proxy"Connection: close
HTTP/1.1 400 Bad RequestProxy-Connection: closeConnection: close
HTTP/1.1 502 Bad GatewayConnection: closeEvery line ends in \r\n, and every response ends with an empty line and no body.
The CONNECT request the codec sends
Section titled “The CONNECT request the codec sends”build_connect_request formats the target with format_authority (IPv6 in brackets, port always present) and writes:
| Line | Present | Content |
|---|---|---|
| Request line | always | CONNECT <host>:<port> HTTP/1.1 |
Host |
always | The same <host>:<port>. |
Proxy-Authorization |
when auth is Some |
Basic followed by standard base64 of user:pass. |
Proxy-Connection |
always | Keep-Alive |
| Terminator | always | An empty line. |
The request must fit REQUEST_MAX (1024 bytes): the authority twice plus one credential. start checks the length before staging and otherwise fails with InvalidInput and the text http: CONNECT request exceeds the codec's reserve.
The forwarded head
Section titled “The forwarded head”For a plain request, build_forward_request produces this head, which the core keeps in rewritten:
| Part | Content |
|---|---|
| Request line | <method> <origin-form target> HTTP/1.1. The method is copied; the version is always HTTP/1.1. |
Host |
The authority from the absolute URI, or the client’s Host value when the URI had none (transparent mode). |
| Other headers | Every client header in its original order, with its name spelled as the client sent it, written as name: value, except those dropped below. |
Connection |
Always close, appended last. |
| Terminator | An empty line. |
Headers dropped from the client’s head:
- every
Hostheader (replaced by the line above); - every name in
HOP_BY_HOP:proxy-connection,proxy-authenticate,proxy-authorization,te,trailers,transfer-encoding,upgrade,connection,keep-alive; - every name listed in the client’s
Connectionheader values, compared in lower case.
Data flow
Section titled “Data flow”The state machine
Section titled “The state machine”stateDiagram-v2 [*] --> Handshake Handshake --> Handshake: head incomplete, consume 0 Handshake --> Relay: CONNECT, Open, reply owed Handshake --> Relay: plain request, Open and ForwardHeld Handshake --> Sniff: CONNECT to an IP with sniffing, 200 staged Sniff --> Relay: verdict, sniff deadline or client EOF Handshake --> Done: 407, 400 or client EOF Relay --> Done: ConnectFailed Done --> [*]
Timing runs alongside with its own phase: Handshake until the flow opens, Sniff while the prefix is collected, and Relay from the moment open pushes Effect::Open. That is why is_established() turns true at Open, before the outbound has connected. Relay ends through the Passthrough half-close bookkeeping or the idle deadline, both of which push Effect::Finish without changing state.
Parsing the head
Section titled “Parsing the head”On Event::Transport in Handshake, on_transport:
- Calls
find_head_endon the whole unparsed region. With no terminator it returnsOk(0), so the runtime keeps the bytes and reads more, unless the region has reachedMAX_HEAD, which is an error. - Parses exactly
data[..end]withparse_request_head. Bytes afterend(a request body, or tunnel bytes a client sent early) stay in the runtime’s buffer. - Calls
on_head, which authenticates first and then branches on the method, compared case-insensitively withCONNECT. - Returns
Ok(end), consuming only the head.
CONNECT
Section titled “CONNECT”The target is authority-form and goes straight to parse_authority(&head.target, 443); the Host header is not consulted. What happens next depends on whether the core has to sniff.
The core opens the flow at once and owes the 200 until the outbound connects:
sequenceDiagram
participant C as Client
participant R as Server runtime
participant H as HttpCore
participant O as Outbound
C->>R: CONNECT example.com:443 HTTP/1.1 and headers
R->>H: Event Transport, whole head
H->>R: Effect Open, reply owed, deadline RELAY_IDLE_TIMEOUT
R->>O: connect through the router
alt connected
O-->>R: connected
R->>H: Event Connected
H->>R: stage CONNECT_ESTABLISHED
R-->>C: HTTP/1.1 200 Connection established
C->>R: tunnel bytes
R->>H: Event Transport
H->>R: Effect Forward, whole slice
R->>O: bytes, verbatim
else connect failed
R->>H: Event ConnectFailed
H->>R: stage RESP_502, ShutdownTransport, Finish
R-->>C: HTTP/1.1 502 Bad Gateway
endA client sends nothing after CONNECT until it sees the 200, so there would be nothing to sniff. The core therefore stages CONNECT_ESTABLISHED in the same call that parsed the head, parks the flow in State::Sniff and arms SNIFF_TIMEOUT:
sequenceDiagram participant C as Client participant R as Server runtime participant H as HttpCore participant O as Outbound C->>R: CONNECT 192.0.2.10:443 HTTP/1.1 R->>H: Event Transport, whole head H->>R: stage CONNECT_ESTABLISHED, deadline SNIFF_TIMEOUT R-->>C: HTTP/1.1 200 Connection established C->>R: TLS ClientHello R->>H: Event Transport H->>H: SniffPrefix push, domain found H->>R: Effect Open with sniffed domain, ForwardHeld of the prefix R->>O: connect, then the held prefix R->>H: Event Connected Note over H: reply is false, nothing staged
The flow also opens when the prefix reaches SNIFF_LIMIT (Verdict::Exhausted), when the sniff deadline passes (Expired::Sniff), or when the client half-closes during the window (TransportEof opens the flow and then half-closes it). In every case open_sniffed copies SniffPrefix::result() into Flow::sniffed, and open pushes Effect::ForwardHeld for the held prefix when it is non-empty. How the sniffers recover a domain is on Sniffing.
Once in Relay, the core is a Passthrough<Single>: every transport slice becomes one Effect::Forward of the whole slice, and every outbound slice is staged verbatim toward the client.
Plain forwarding
Section titled “Plain forwarding”For any other method, on_head resolves the destination and the forwarded Host separately:
flowchart TB
target["parse_request_target(head.target)"]
abs{"authority present and non-empty?"}
transparent{"allow_transparent?"}
r400["stage RESP_400, close"]
host{"Host header non-empty?"}
fromHost["destination from Host"]
fromUri["destination from URI authority"]
empty["error: missing target host"]
open["parse_authority, build_forward_request, open"]
target --> abs
abs -- yes --> host
abs -- no --> transparent
transparent -- no --> r400
transparent -- yes --> host
host -- yes --> fromHost --> open
host -- no --> fromUri
fromUri -- "authority present" --> open
fromUri -- "no authority" --> empty
The rules, precisely:
- Destination. The
Hostheader wins over the URI authority when it is non-empty. The default port is 443 for anhttps://target and 80 otherwise. - Forwarded
Host. The URI authority wins over theHostheader. In transparent mode there is no authority, so the client’sHostis forwarded. - Forwarded target. The origin-form part of the URI (
/path?q=1), or the target unchanged in transparent mode.
The core stores the rewritten head in rewritten and calls open(flow, false, fx), which pushes Effect::Open followed by Effect::ForwardHeld { range: 0..held }. The runtime applies the held forward once the outbound is connected, so the new head reaches the origin before any body byte:
sequenceDiagram participant C as Client participant R as Server runtime participant H as HttpCore participant O as Origin C->>R: GET http://example.com/path?q=1 HTTP/1.1, headers, body R->>H: Event Transport H->>H: build_forward_request into rewritten H->>R: Effect Open, ForwardHeld of rewritten, consumed = head only R->>O: connect, then GET /path?q=1 HTTP/1.1 ... Connection close R->>H: Event Transport, body bytes H->>H: clear rewritten H->>R: Effect Forward, whole slice R->>O: body, verbatim O-->>R: response, then EOF R->>H: Event Outbound, then OutboundEof H->>R: stage response, ShutdownTransport R-->>C: HTTP/1.1 200 OK ...
The core parses one head per connection. After it, everything the client sends is relayed to the same outbound without inspection, and the Connection: close the rewrite appends asks the origin to close after its response. A client that wants a second request has to open a new connection. The core does not stage a 200 for a plain request: the origin’s own response is the reply.
Authentication
Section titled “Authentication”authenticate runs before the method branch, so it guards CONNECT and plain requests alike:
- With an empty
accountsmap, every request is accepted as username""with theanonymouspayload. - Otherwise
check_proxy_authmust returnSome. It reads the firstProxy-Authorizationheader, strips aBasicorbasicprefix (any other capitalisation of the scheme is refused), trims the rest, decodes standard base64, requires UTF-8, splits at the first:(so a password may contain colons, a username may not), looks up the username and requires the stored password to be equal. Any failure along that chain yieldsNone.
A None sets State::Done and calls refuse(RESP_407, fx). On success the flow carries NetworkUser with UserAuthorization::UsernamePassword { username, password }, where password is always empty: the credential stops at the core. For plain requests the Proxy-Authorization header is also removed from the forwarded head by HOP_BY_HOP.
The client codec
Section titled “The client codec”sequenceDiagram
participant A as Client runtime
participant X as HttpConnect
participant U as Upstream proxy
A->>X: start
X->>A: stage CONNECT request, Handshake AwaitReply
A->>U: CONNECT example.com:443 HTTP/1.1
U-->>A: HTTP/1.1 200 ...
A->>X: reply with the unparsed bytes
alt no terminator yet
X->>A: Reply NeedMore
else status 200
X->>A: Reply Step, consumed = head, next Done
else any other status
X->>A: error ConnectionRefused
end
A->>X: seal and open, verbatim
replyaccepts exactly status200; any other code, including other2xxcodes, is refused withConnectionRefusedand the textproxy responded with status <code>. The client runtime wraps every codec error asInvalidDatabut keeps its text, which is how the pipeline test sees407in the error from the plaintext side.consumedis the head length only. Bytes the upstream sent after the head stay in the client runtime’s read buffer and become the first tunnel bytes.sealstages the whole plaintext slice it is given andopenreturns the whole wire slice as one frame.finishstages nothing: the wire’s own EOF carries the half-close.
Invariants
Section titled “Invariants”| Invariant | Enforced by | Pinned by |
|---|---|---|
| Nothing is consumed until a whole head has arrived. | on_transport returns Ok(0) while find_head_end is None. |
connect_to_a_domain_answers_200_once_connected (a 20-byte prefix consumes 0), head_end_is_found_only_once_the_blank_line_arrives |
| A request head always fits the read buffer, and a larger one fails in the core rather than stalling. | BUF_SIZE = MAX_HEAD; on_transport errors when the region reaches MAX_HEAD with no terminator, before the runtime’s own frame-size check. |
an_oversized_head_is_refused |
| Only the head is consumed; a body or early tunnel bytes stay for the relay. | on_transport returns end, not data.len(). |
plain_request_is_rewritten_into_the_held_buffer (consumed == wire.len() - 4) |
| No flow opens and no byte reaches an outbound before authentication. | on_head calls authenticate first; failure goes to refuse(RESP_407, …). |
missing_or_wrong_credentials_are_a_407, new_server_refuses_bad_credentials_with_407 |
A CONNECT gets its 200 only after the target connected, unless it sniffs. |
State::Relay { reply: true }; Event::Connected stages CONNECT_ESTABLISHED and clears the flag. |
connect_to_a_domain_answers_200_once_connected |
A CONNECT is answered exactly once. |
The sniffing path stages the 200 in on_head and opens with reply: false; Connected then stages nothing. |
connect_to_an_ip_with_sniffing_replies_early_and_holds_the_prefix (“no second 200”), new_server_connect_to_an_ip_answers_before_the_first_bytes, new_server_vs_new_client_tcp (a second 200 would corrupt the echoed bytes) |
A failed CONNECT whose 200 is owed is answered 502 and closed. |
Event::ConnectFailed stages RESP_502 when reply is set, then ShutdownTransport and Finish. |
connect_refused_answers_502_and_closes |
| The rewritten head reaches the origin before the body. | open pushes ForwardHeld right after Open; the runtime delivers no byte event until the held effect is applied. |
plain_request_is_rewritten_into_the_held_buffer, new_server_forwards_a_plain_request |
| The held buffer is never cleared under a queued held range. | rewritten and prefix are cleared only at the start of a Relay byte event (Transport or Outbound), which the runtime’s pin rule delays until held effects are applied. |
plain_request_is_rewritten_into_the_held_buffer checks that held() is empty after the body; the pin rule itself belongs to the server runtime. |
| The rewritten head carries no proxy credentials and no hop-by-hop headers. | build_forward_request skips HOP_BY_HOP names, names listed in Connection, and Host. |
forward_request_strips_hop_by_hop, plain_request_is_rewritten_into_the_held_buffer |
| A plain request without an absolute URI is refused unless transparent mode is on. | on_head checks authority.is_none() && !allow_transparent. |
origin_form_without_transparent_is_a_400 |
| Every fixed response fits the staging reserve. | STAGING_RESERVE = 256; the largest response (RESP_407) is 106 bytes. A shortfall is a core bug and surfaces as staging_full(). |
Covered implicitly by every test that stages a response. |
| The codec’s request never exceeds its declared reserve. | STAGING_RESERVE = REQUEST_MAX; start checks the length first. |
No test builds an oversized request. |
Only a 200 opens a client tunnel, and the tail after it is preserved. |
reply matches 200 and returns consumed: end. |
connect_codec_refuses_a_non_200, connect_codec_awaits_a_200_then_passes_through |
Failure paths and cancellation
Section titled “Failure paths and cancellation”Every error that handle returns ends the connection through the runtime without a response to the client. The fixed responses cover the cases where the core has something useful to say.
| Situation | What the client sees | Mechanism |
|---|---|---|
Head reaches MAX_HEAD with no terminator |
Connection closed | InvalidData, http head exceeds maximum size |
| Malformed head, or more than 128 headers | Connection closed | InvalidData, malformed http request: <httparse error> |
Head made only of blank lines (for example a stray \r\n\r\n before the request) |
Connection closed | InvalidData, incomplete http request head |
| Unparsable authority | Connection closed | parse_authority: invalid port, empty authority host, ambiguous authority (bracket IPv6 literals), malformed IPv6 authority, trailing data after IPv6 authority |
| Missing or wrong credentials | 407, then close |
refuse(RESP_407, …), State::Done |
Origin-form target, allow_transparent off |
400, then close |
refuse(RESP_400, …), State::Done |
Transparent mode, no Host and no authority |
Connection closed | InvalidData, missing target host |
CONNECT target fails to connect, 200 owed |
502, then close |
Event::ConnectFailed stages RESP_502 |
Sniffing CONNECT or plain request fails to connect |
Connection closed, no response | Event::ConnectFailed with reply: false: ShutdownTransport, Finish |
| Outbound errors during the relay | Connection closed after staged bytes drain | Passthrough::on_outbound_gone |
| Client closes before a complete head | Connection closed | TransportEof in Handshake pushes Finish |
| Client stalls after starting its head | Connection closed | Timing armed HANDSHAKE_TIMEOUT at the first byte; Expired::Handshake returns handshake_timed_out(): TimedOut, client did not complete its request in time |
No bytes either way for RELAY_IDLE_TIMEOUT |
Connection closed | Timing::expired pushes Finish (Expired::Idle) |
refuse stages the response and then pushes Effect::ShutdownTransport and Effect::Finish, so the runtime writes the staged bytes before it closes the write side. ConnectFailed and OutboundError are logged at debug level as http: connect failed: … and http: outbound failed: ….
Half-closes follow Passthrough: client EOF becomes Effect::Shutdown on the outbound, outbound EOF becomes ShutdownTransport, and Finish follows once both halves have closed.
The core holds no task, lock or channel, so there is nothing of its own to cancel: when the runtime is dropped, the core goes with it. A client that connects and never sends a byte produces no event and so never arms the core’s deadline. The app’s drive in app/src/serve.rs covers that case by wrapping each runtime step in tokio::time::timeout(HANDSHAKE_TIMEOUT, …) until is_established() is true (failing with inbound handshake timed out after 10s), and releases the handshake permit at that point; see Serving.
On the client side, every handshake error fails the plaintext stream. The last two rows come from the client runtime, the others from the codec:
| Situation | Error |
|---|---|
Request longer than REQUEST_MAX |
InvalidInput, http: CONNECT request exceeds the codec's reserve |
Status other than 200 |
ConnectionRefused, proxy responded with status <code> |
Response head that httparse rejects |
InvalidData, malformed proxy response: <httparse error> |
| Response head without a status line (only blank lines) | InvalidData, proxy response missing status code |
Response head with no terminator within MAX_HEAD |
InvalidData, http: proxy response head exceeds maximum size |
| Response head larger than the client runtime’s read buffer | InvalidData, upstream frame larger than the client runtime's buffer |
| Upstream closes before a complete reply | UnexpectedEof, upstream closed during the handshake |
The client runtime re-wraps the codec’s errors as InvalidData and keeps their text. The app and katana size that runtime with HTTP_BUF (16 KiB), smaller than MAX_HEAD, so for them an oversized response head fails in the runtime before the codec’s own check can fire.
Limits
Section titled “Limits”| Constant | Value | Where | Governs |
|---|---|---|---|
MAX_HEAD |
64 KiB (64 * 1024) |
protocols/src/http/protocol.rs |
Largest request head the core, or response head the codec, will buffer. |
HttpCore::BUF_SIZE |
MAX_HEAD |
protocols/src/http/core.rs |
Size of each of the server runtime’s three buffers (transport read, transport staging, outbound scratch). |
MAX_HEADERS |
128 | protocols/src/http/protocol.rs |
Header slots given to httparse, for requests and responses. |
HttpCore STAGING_RESERVE |
256 bytes | protocols/src/http/core.rs |
Room guaranteed for a fixed response. |
HttpConnect::REQUEST_MAX |
1024 bytes | protocols/src/http/codec.rs |
Largest CONNECT request, and the codec’s STAGING_RESERVE. |
HANDSHAKE_TIMEOUT |
10 s | protocols/src/core/mod.rs |
From the first byte to a complete head. |
SNIFF_TIMEOUT |
300 ms | protocols/src/sniff/mod.rs |
Sniffing window of a CONNECT to an IP. |
SNIFF_LIMIT |
4 KiB | protocols/src/sniff/mod.rs |
Largest sniff prefix held. |
RELAY_IDLE_TIMEOUT |
300 s | protocols/src/core/mod.rs |
Idle limit of a relay, refreshed on every Transport and Outbound event. |
HTTP_BUF |
16 KiB | app/src/outbound/mod.rs, katana src/outbound/mod.rs |
Client runtime buffer of the HTTP outbound; bounds the upstream’s response head in practice. |
| Default ports | 443 for CONNECT and https://; 80 otherwise |
protocols/src/http/core.rs |
Port used when the authority has none. |
Allocation per connection is bounded by these numbers: rewritten is at most one head plus a request line, a Host line and Connection: close, and the sniff prefix is capped by SNIFF_LIMIT.
There are two kinds of tests. The unit tests in protocols/tests/unit/http/ are compiled into the crate through #[path] modules, so they can reach private items; the pipeline tests run the real runtimes over loopback TCP.
| File | Test | Pins |
|---|---|---|
protocols/tests/unit/http/core.rs |
connect_to_a_domain_answers_200_once_connected |
Partial heads consume nothing; a domain CONNECT opens immediately, stages nothing until Connected, and is established at Open. |
connect_refused_answers_502_and_closes |
ConnectFailed stages RESP_502, then ShutdownTransport and Finish. |
|
connect_to_an_ip_with_sniffing_replies_early_and_holds_the_prefix |
The early 200, the SNIFF_TIMEOUT deadline, the sniffed domain on Open, the ForwardHeld of the prefix, and no second 200. |
|
plain_request_is_rewritten_into_the_held_buffer |
Origin-form rewrite, header stripping, Connection: close, head-only consumption, body forwarded after it, held buffer cleared. |
|
origin_form_without_transparent_is_a_400 |
RESP_400 and Finish without opening. |
|
missing_or_wrong_credentials_are_a_407 |
RESP_407 without credentials; a correct Basic header opens a flow as alice. |
|
an_oversized_head_is_refused |
MAX_HEAD bytes without a terminator is an error. |
|
protocols/tests/unit/http/protocol.rs |
target_absolute_form, target_absolute_form_no_path, target_origin_form |
parse_request_target splitting and the https flag. |
forward_request_strips_hop_by_hop |
HOP_BY_HOP and Connection-listed headers are dropped. |
|
head_end_is_found_only_once_the_blank_line_arrives |
find_head_end offsets. |
|
request_head_is_parsed_into_owned_parts |
parse_request_head, header_value, check_proxy_auth, and rejection of garbage. |
|
connect_request_and_response_status_round_trip |
build_connect_request parses back; parse_response_status reads 200, 407 and 502 from the fixed responses. |
|
protocols/tests/unit/http/codec.rs |
connect_codec_awaits_a_200_then_passes_through |
The request line and credential, NeedMore on a partial head, the tail after the head, verbatim open and seal. |
connect_codec_refuses_a_non_200 |
A 407 is ConnectionRefused. |
|
protocols/tests/pipeline/http.rs |
new_server_vs_new_client_tcp |
HttpConnect against HttpCore with credentials, to an IP target with sniffing on (so the early-200 path): 100,000 bytes echoed, clean close. |
new_server_refuses_bad_credentials_with_407 |
A wrong password surfaces as an error containing 407. |
|
new_server_forwards_a_plain_request |
A real origin sees GET /path?q=1 HTTP/1.1 and the client gets its response. |
|
new_server_connect_to_an_ip_answers_before_the_first_bytes |
With sniffing on, the 200 arrives before the client sends payload. |
Run them from the Etemenanki workspace:
cargo test -p etemenanki-protocols --lib -- http::core http::protocol http::codeccargo test -p etemenanki-protocols --test pipeline http::A bare --lib http:: filter would also run the HTTP sniffer’s tests in sniff::http. The core tests drive HttpCore through CoreHarness, and the file’s moves helper filters SetDeadline effects out so assertions stay about behaviour. A change to an event path belongs in tests/unit/http/core.rs; a change to what a real client or origin sees on the wire also needs a pipeline test. Testing describes the harnesses.