Commentary: http-from-sockets

An HTTP/1.1 server on a raw TCP socket — and the discovery that curl will print a complete-looking body to your terminal and then exit 28, while the response it exits 0 on is the one that lied to it.

cd http-from-sockets
python3 demo.py                  # the aha (§6) -- needs curl on PATH
python3 test_http_server.py      # 17 tests, one of them on a real socket

You can also drive one framing by hand, which is the best way to feel it:

python3 http_server.py --mode none --after hold --port 8127
curl -s --max-time 1 http://127.0.0.1:8127/hello ; echo "exit=$?"

1. Orientation

This toy is a working HTTP/1.1 server with no framework, no http.server, and no third-party anything: a listening socket, a read loop, some string splitting, and a sendall of bytes you assembled by hand. That part takes about forty lines and is not the point.

The point is the other hundred and fifty: ten different ways to end a response body, only some of which actually say where it ended. TCP will not tell the client. TCP has no messages — it has bytes, in order, and no idea that any group of them belongs together.

By the end you should be able to:

• name the three things that can end an HTTP/1.1 response body, and say which one applies when two of them are present;
• explain why a client can print your entire body to the screen and still report failure, and why the reverse is more dangerous;
• say what Connection: close does, and — more usefully — what it does not;
• explain why keep-alive is what turned framing from a detail into a requirement;
• look at a list of recv sizes and know what it does, and does not, tell you about the network.

2. The problem this mechanism exists to solve

TCP gives you a reliable, ordered byte stream. That is a genuinely hard thing to build and it is worth exactly what it costs — but read the promise carefully, because of the four words reliable, ordered, byte, stream, the last two are the ones that hurt. You will get every byte, in order. You will not be told where anything begins or ends. recv() returns "whatever is in the buffer right now," which may be half a request, or three responses, or one byte.

So any protocol running on TCP has to invent its own idea of "message." That is framing, and there are only three families of answer:

• Send the length first. The receiver reads a counter, then reads exactly that many bytes. This is Content-Length. Cheap and exact, and it requires you to know the size before you send the first byte.
• Delimit it. Put a marker at the end that cannot appear inside. HTTP uses this for the head — a blank line ends the headers — and chunked encoding uses a length-prefixed sequence with a zero-length chunk as the terminator, which is really a hybrid: many small counted frames, then a delimiter.
• Close the connection. EOF is the one framing signal TCP hands you for free. It works exactly once per connection, which is the problem.

HTTP/1.0 mostly used the third. One request, one response, close, done — so "the body is everything until FIN" was a complete answer and nobody had to think about it. HTTP/1.1 made connections persistent by default, because setting up a TCP connection costs a round trip and a page needs thirty objects. The moment the connection survives the response, EOF stops being available as a terminator: if the client waited for FIN it would wait forever, and if it guessed early it would read your next response as this one's body.

That is the trade this toy is about. Persistent connections bought you a round trip and charged you a framing header. Everything in the matrix in §6 is a way of not paying that charge correctly.

3. Background you need

The two flagged rows are the whole page. The first decides what happens; the third decides what you are entitled to say about why.

4. The mental model

The asymmetry to hold on to: rules 1 and 2 are promises the server makes; rule 3 is an event the client waits for. A promise can be wrong. An event can simply never happen.

5. Reading the source

5.1 The entire wire format, in eleven lines

def head(status_line, *headers):
    """Status line + headers + the blank line that ends the head."""
    out = status_line + "\r\n"
    for header in headers:
        out += header + "\r\n"
    return (out + "\r\n").encode()


def chunk(part):
    """One chunk of a chunked body: hex length, CRLF, the bytes, CRLF."""
    return b"%x\r\n" % len(part) + part + b"\r\n"

There is no HTTP library under this. head() is string concatenation and one .encode(); the protocol's entire message syntax is "lines ending in CRLF, then an empty one." The + "\r\n" on the return line is the blank line, and it is the only delimiter in HTTP/1.1 that is a delimiter — everything after it is counted, not scanned.

chunk() is worth a second look because the %x is not decoration. Chunk lengths are hexadecimal, so the 13-byte and 14-byte pieces in frame_chunked go on the wire as d and e, not 13 and 14. A client reading them as decimal would ask for 13 bytes and 14 bytes anyway for the first one and get away with it, then ask for 14 and get 20. Bugs like that are why the test suite asserts the exact bytes.

5.2 Three framings that differ only in a header

def frame_correct(body):
    return head("HTTP/1.1 200 OK", "Content-Type: text/plain",
                "Content-Length: %d" % len(body)) + body


def frame_none(body):
    return head("HTTP/1.1 200 OK", "Content-Type: text/plain") + body


def frame_short(body):
    return head("HTTP/1.1 200 OK", "Content-Type: text/plain",
                "Content-Length: 10") + body

Three functions, one body, and every byte after the head is identical in all three. frame_correct computes the length from the body it is about to send, which is the only version where the header cannot drift. frame_short hard-codes 10 — the thing a real server does implicitly the moment someone adds a suffix to the body after the length was computed, or a gzip layer recompresses it, or a proxy rewrites it.

frame_none is not a broken server so much as an unfinished one. It is exactly what you write first, before you have heard of framing, and it is the version that works perfectly in every manual test where you close the socket afterwards.

5.3 The five bytes that decide chunked

def frame_chunked(body):
    return (head("HTTP/1.1 200 OK", "Transfer-Encoding: chunked")
            + chunk(body[:13]) + chunk(body[13:]) + b"0\r\n\r\n")


def frame_chunked_noterm(body):
    return (head("HTTP/1.1 200 OK", "Transfer-Encoding: chunked")
            + chunk(body[:13]) + chunk(body[13:]))

The two functions are one expression apart. frame_chunked is 89 bytes, frame_chunked_noterm is 84, and the difference is literally b"0\r\n\r\n" — measured, not asserted; see CF2 in §6.4. Chunked encoding does not need the terminator to deliver the body. It needs it to end. A streaming server that crashes after its last chunk has sent you every byte it owed you and still produced an unterminated message.

5.4 The reader: two loops, because two different rules

def read_request(conn, bufsize=4096):
    """Read one request; return (request_line, headers, body, recv_sizes).

    Two loops, because two different rules end two different things. The head
    ends at a blank line -- a delimiter you scan for, so you must be willing
    to call recv() again mid-header. The body ends after Content-Length bytes
    -- a count you subtract from. There is no third rule and, if the header is
    absent, no rule at all: `want` is 0 and any body sits unread in the
    kernel's buffer. That asymmetry is the whole toy, seen from the other end.
    """
    buf, sizes = b"", []
    while b"\r\n\r\n" not in buf:
        data = conn.recv(bufsize)
        if not data:                            # client closed: no request
            return None, {}, b"", sizes
        sizes.append(len(data))
        buf += data
    raw_head, rest = buf.split(b"\r\n\r\n", 1)

The condition is while b"\r\n\r\n" not in buf and not if — the head may arrive in pieces, and a server that assumes one recv gives it a whole head works flawlessly against every small client and falls over against a large one. Note also buf.split(..., 1): whatever came in past the blank line is kept in rest rather than discarded, because a single recv can easily hand you the head and the body and the beginning of the next request together.

The if not data: return None is the honest reading of "recv returned zero bytes": on a socket that means the peer has closed, not that it is being quiet. Deleting it costs you an infinite loop the first time something connects and disconnects without speaking — which, as §6.5 shows, happens on every single run of the demo, because demo.py's own readiness probe does exactly that.

want = int(headers.get("content-length", 0))
while len(rest) < want:
    data = conn.recv(bufsize)
    if not data:
        break
    sizes.append(len(data))
    rest += data
return request_line, headers, rest, sizes

headers.get("content-length", 0) defaulting to 0 is the request-side echo of the whole page: with no length header there is no body, full stop. RFC 9112 §6.3 rule 8 makes this asymmetric on purpose — for a request, "no framing" means length zero; for a response, it means read until close. A request cannot use EOF as its terminator, because the client still wants to read the answer on that same connection.

Deleting these seven lines is the difference between receiving 20 000 bytes and receiving 3 939 of them. That is CF5 in §6.4, and it is measured.

5.5 What happens after the response

while True:
    conn, _ = lsock.accept()
    conns += 1
    log("connection %d accepted" % conns)
    while True:
        request_line, _, req_body, sizes = read_request(conn, bufsize)
        if request_line is None:
            log("connection %d closed by client" % conns)
            break
        log("  %s | recv sizes %s | request body %d bytes"
            % (request_line, sizes, len(req_body)))
        conn.sendall(build(body))
        if after == "keepalive":
            continue
        if after == "hold":
            time.sleep(HOLD_SECONDS)
        conn.close()
        break

Two nested while Trues, and the inner one is the interesting one: it is persistence. after == "keepalive" means continue — go round and read another request off the same socket, which is what HTTP/1.1 does by default and what makes framing mandatory.

after == "hold" is the experimental apparatus. It writes the response and then simply does nothing for HOLD_SECONDS, keeping the socket open. That is not a pathological server; it is an ordinary keep-alive server that has finished its work and is waiting for your next request. The client cannot tell those two situations apart, and §6 is a table of what it does about that.

6. The demo, and what it proves

Everything below is a verbatim slice of one run of python3 demo.py.

6.1 The pair

The body is b"the framing is the message\n" — 26 characters and a newline, 27 bytes. Read the two failing-looking rows against each other:

• none sent all 27 bytes and curl printed all 27. Then, having no rule that says the body ended, it fell through to rule 3 — read until FIN — and the server (an ordinary keep-alive server, doing nothing wrong except omitting a header) never sent one. At --max-time 1 curl gave up: exit 28, CURLE_OPERATION_TIMEDOUT. Every byte arrived. The transfer failed.
• short declared Content-Length: 10, so curl read exactly 10 bytes, considered the message complete, and stopped: exit 0, and it handed the caller 'the framin'. The other 17 bytes — 27 − 10 = 17 — were never delivered to anybody.

If you are writing a script around curl, the first case wakes you up at 3am for a response that was fine and the second silently corrupts your data. The exit code is a statement about framing, not about the body.

6.2 Every framing, held open vs. closed

Four things in that table are worth more than the headline.

close-header-only (hold) exits 28. The response says Connection: close in as many words, and the client still hangs. The header is an announcement of intent about the connection; it is not a body terminator. What ends the body under rule 3 is the FIN — an actual TCP segment, produced by an actual close(). Say you will close and then don't, and you have told the client nothing it can use. The fin row is the same response with a real close() after it: exit 0. The header does not frame. The FIN frames.

http10 (hold) exits 28 too. HTTP/1.0's default was close-when-done, so it is tempting to think the version string tells the client "expect an EOF". It does not change what the client does — it still waits for the FIN, and one that never comes times out identically. The version in the status line is not a framing mechanism either.

long and chunked-noterm swap 28 for 18 when the socket closes. long declares Content-Length: 47 (that is 27 + 20, from frame_long) and sends 27. Held open, curl waits for the missing 20 bytes and times out: 28. Closed, the FIN arrives while curl is still 20 bytes short of a promise, and it can say something much more precise — exit 18, CURLE_PARTIAL_FILE, "shorter or larger than expected." Same for the unterminated chunked response. That is the difference between "I gave up" and "you lied to me," and you only get the second diagnosis because there was a promise to check against. none has no promise, so closing it produces exit 0 and no complaint at all — the same bytes, an entirely clean bill of health.

both and both-te-first exit 0 with the correct body. Those responses carry Content-Length: 99 and Transfer-Encoding: chunked, in both possible orders. 99 is a lie about a 27-byte body, and curl ignores it completely, in both orders, because RFC 9112 §6.3 rule 3 says Transfer-Encoding wins and the Content-Length must be dropped. That precedence rule is why the combination is the classic request-smuggling primitive: it is only safe as long as every box in the chain implements the same tie-break. A front-end that believes the length and a back-end that believes the chunks will disagree about where one request ends and the next begins — and the attacker chooses the bytes in the gap.

6.3 curl's own account of the truncation

This is the whole -v trace, not a selection. Two lines in it settle the question of whether curl was fooled:

{ [27 bytes data] — it received all 27. The bytes were never lost, delayed, or dropped; they were sitting in the buffer.

Excess found writing body: excess = 17, size = 10, maxdownload = 10 — and it noticed. 27 − 10 = 17. curl knows perfectly well that the server sent more than it promised. It discards the excess, closes the connection rather than reuse a stream it can no longer find its place in, and reports success. That is a defensible choice — the alternative, keeping the connection and parsing g is the message\n as the next status line, is response smuggling — but the caller is told nothing. §6.5's keep-alive run is what that defensive close looks like from the outside.

6.4 Counterfactuals

These come from counterfactuals.py, which imports the shipped http_server.py. Real output, not reasoning.

CF1 — which Content-Length value flips the exit code? Sweep the declared length over a body that is really 27 bytes:

The flip is between 27 and 28 — one digit in one header. And the two sides are not symmetric in the slightest. Under-declare by any amount and you get exit 0 with a body truncated to exactly the number you claimed; the client has no way to know, because you never told it about the rest. Over-declare by one byte and you get a hang. Being too small is silent; being too large is loud. Almost every framing bug you will meet in production is on the silent side of that boundary.

CF2 — the five bytes.

89 − 84 = 5. Both deliver the identical 27-byte body to stdout. Five bytes of pure metadata are the entire difference between a successful transfer and a timeout.

CF3 — does the client insist on CRLF?

This one surprised me, and it is the most quietly instructive result here. Send the head with bare \n line endings, in flat violation of the grammar, and curl accepts it and exits 0. The delimiter it forgives; the count it does not. Robustness in real clients is not evenly distributed — it is concentrated wherever being strict would break existing servers, and absent wherever being lenient would mean guessing where a message ends. You cannot guess a length.

CF4 — parse the request head by scanning for \n\n.

Change the delimiter in read_request from b"\r\n\r\n" to b"\n\n" and the server never emits a response at all, because CR LF CR LF contains no two adjacent LFs. Note which side is lenient here: curl-the-client accepted my malformed LF-only response in CF3, and curl-the-client's own request is strictly CRLF. Be conservative in what you send.

CF5 — delete the body-reading loop.

Remove while len(rest) < want: — i.e. assume one recv hands you the whole request — and a 20 000-byte upload arrives as 3 939 bytes. The arithmetic: one recv(4096) returned 4 096 bytes, of which 157 were the head, leaving 4096 − 157 = 3 939 of body. No error is raised anywhere. The server just silently processes a fifth of the upload.

(That head is 157 bytes here and 156 in §6.6, because these counterfactuals run on ephemeral five-digit ports while the demo uses the fixed four-digit 8127, and the port number lives inside the Host: header. Framing arithmetic is unforgiving that way.)

CF6 — the one number that moves between runs.

1001 vs 1002. This message is the clearest single sentence in the whole toy — "timed out … with 27 bytes received", the failure and the complete body in one line — and it is also the reason demo.py runs curl with -s and never prints elapsed time. That millisecond figure is the only thing standing between this page and a reproducible transcript.

6.5 Two requests down one connection

This is the price of the lie, and it is paid in connections. With correct framing, curl url/a url/b fetches both over one connection (number 2 — number 1 is the demo's readiness probe, which connects and closes without speaking, and is a real illustration of why read_request handles a zero-byte recv). With Content-Length: 10, curl uses two connections, 2 and 3, because after each response it finds 17 unaccounted-for bytes and refuses to reuse a stream it has lost its place in.

So a framing bug does not only corrupt the body. It quietly destroys keep-alive — the entire reason the framing header was needed — and converts every request into a fresh TCP handshake. On loopback that is free. Across the Atlantic it is another round trip per request, forever, and it will show up in your latency graphs as a mystery rather than as an error, because every one of those requests exits 0.

6.6 That 4096 is mine, not the network's

Identical client, identical upload, three different answers to "how many pieces did it arrive in": 5, 20, or 1.

The arithmetic. Total bytes read is 20 156 every time: 20 000 of body plus a 156-byte request head (POST /upload HTTP/1.1, Host: 127.0.0.1:8127, User-Agent, Accept, Content-Length: 20000, Content-Type, and the blank line). Then:

• ⌈20156 / 4096⌉ = ⌈4.92⌉ = 5 — and indeed 4×4096 + 3772 = 20156.
• ⌈20156 / 1024⌉ = ⌈19.68⌉ = 20 — and 19×1024 + 700 = 20156.
• ⌈20156 / 65536⌉ = 1 — the whole request in a single call.

What the experiment does prove is the thing that matters for writing servers: the number of recv calls is a property of your code, not of the message. A reader that treats "one recv" as "one message" is not making an optimistic assumption about the network; it is making a false statement about an API. That is why read_request loops, and why test_recv_count_is_the_buffer_bound_not_the_wire asserts len(sizes) == ceil(len(raw) / bufsize) against an in-memory fake with no network under it at all — the relationship holds because it is arithmetic about a buffer, not physics about a wire.

Note also that ⌈·⌉ is a lower bound in general. On a slower link a recv can legitimately return less than bufsize because that is all that has arrived yet, giving more calls. It can never give fewer. On loopback the buffer is always full when we ask, so the bound is tight and the numbers above were identical across five consecutive runs.

6.7 Reproducibility

A toy whose result depends on a port, a client, a timeout and a socket has an obvious problem: the repo requires byte-identical transcripts. What was done about it, concretely:

• A fixed port, 8127, checked with port_is_free() before anything starts and refused with a one-line message if busy. An ephemeral port would be more polite and would put a different number into the transcript on every run — including, as CF5 showed, a different request length.
• Readiness by connect-poll, not by sleeping (start_server, lines 45–62). A sleep(0.5) is both slower and occasionally wrong; polling connect() until it succeeds is neither. The cost is the phantom connection 1 in §6.5, which is honest and is left visible rather than filtered out.
• curl -s, always. The progress meter is the single largest source of transcript noise and it is timing-dependent.
• --max-time 1, verified sufficient for every timeout case here.
• No elapsed time printed anywhere. This is the load-bearing one. Elapsed seconds, and curl's own after 1001 milliseconds message, are the only fields that differ between otherwise identical runs (CF6).

With those five in place, five consecutive runs of demo.py produce identical output — 5 704 bytes, SHA-256 8de9778b45af6499… each time.

What remains environment-dependent, and is therefore declared in the demo's own banner rather than hidden:

Every exit code on this page is curl 8.7.1's judgement. A different client — or a different curl — may frame the same bytes the same way and still report differently; the Excess found writing body behaviour in particular is libcurl's policy, not the RFC's. The banner prints your local version, so if your numbers differ from this page's, the first line of the output tells you why.

7. Design decisions and roads not taken

curl is a hard dependency, on purpose. The obvious way to make this toy self-contained is to write a fifty-line Python client. It was rejected outright: a client I wrote, checking framing rules I also wrote, proves nothing except that I am internally consistent. The entire evidentiary value of §6 is that an independent implementation of RFC 9112 — one that predates this toy by twenty-five years and has no idea it exists — reads my bytes and disagrees with me. demo.py therefore checks for curl up front and exits with a message naming it rather than degrading to a fallback.

Ten framings in the core file, rather than one correct server plus broken variants in the demo. The comparison is the toy; the same call as rate-limiter carrying both algorithms and lru-cache carrying both policies. A single correct server would be about ninety lines and would teach the thing everyone already believes.

hold rather than a background thread pool. The response and what happens to the socket afterwards are two independent axes, so they are two independent parameters, and the demo crosses them. Making "hold" an explicit mode rather than an accident of scheduling is what turns "curl sometimes hangs" into a 2×10 table.

One connection at a time, no threads, no selectors. Concurrency is a different mechanism and would double the line count while teaching nothing about framing. The hold mode does block the accept loop for 30 seconds; the demo starts a fresh server per case, which costs about 10ms and keeps the toy honest about what it is.

Chunk sizes 13 and 14, hard-coded. They exist to make the hex length visible (d and e, not 13 and 14) and to prove the client reassembles across chunk boundaries. A single chunk would work and would demonstrate less.

Content-Length: 99 for the both cases. A wrong number, deliberately: if it were 27 you could not tell whether curl honoured it or ignored it. It has to be a lie for the precedence rule to be observable.

Not built: a --max-time-free demo. Without a timeout the unframed cases hang forever and there is no aha, only a stuck terminal. The timeout is not a workaround; the number 28 is the result.

8. What's simplified vs. the real thing

• One connection at a time. A real server is epoll/kqueue plus a worker pool or an event loop. Framing gets harder there, not easier: the parser must be resumable, because a partial head arrives, the loop moves on to another socket, and it must come back and continue. This toy can block in recv because it has nothing else to do; nginx cannot, which is why real HTTP parsers are state machines rather than while ... not in buf.
• No request routing, methods, or status codes. Every request gets the same 200. The whole point is that framing is orthogonal to all of it.
• Transfer-Encoding is treated as chunked-or-nothing. Real implementations must parse a list (gzip, chunked), reject chunked when it is not the final coding (RFC 9112 §6.3 rule 5 makes that a 400), and handle chunk extensions and trailers.
• No Expect: 100-continue. A real server reading a large request body must decide whether to accept it before the client sends it. That is a fourth framing-adjacent negotiation this toy skips entirely.
• No HEAD, 204, or 304. Those responses carry a Content-Length that describes a body that is not there (RFC 9112 §6.3 rule 1). A real client must not wait for it; a naive one built on the model in §4 would hang.
• No limits on anything. read_request will happily accumulate a head of any size, which is a denial-of-service in one line. Real servers cap header size, header count, and body size, and those caps are the first thing you add after framing.
• No timeouts on the server side. The toy's socket has no settimeout, so a client that connects and says nothing pins the accept loop forever. The mirror image of the client-side problem this whole page is about.
• HTTP/1.1 only, and that is the era being taught. HTTP/2 makes this entire page obsolete by design: it defines binary frames with explicit length fields at the transport layer, so "where does this message end" is answered once, structurally, instead of by three competing header conventions. Read §6.2 as the argument for why HTTP/2 exists.
• No TLS. It changes nothing about framing — TLS is another byte stream and the same three rules apply on top of it — but it does mean this toy cannot talk to anything that requires https.

9. Check yourself

Answer before expanding. Each answer is derivable from the source.

10. Further reading

• RFC 9112 §6.3, "Message Body Length" — the eight numbered rules a client applies, in order, to decide where a body ends. This page is essentially a set of experiments against rules 3 (Transfer-Encoding beats Content-Length), 7 (a valid length is the length), and 8 (otherwise, read until close). Short, and unusually readable for a spec.
• RFC 9112 §9, "Connection Management" — persistence, Connection: close, and why 1.1 defaults the other way from 1.0. The context for §6.2's hold/fin split.
• Python socket programming HOWTO — the canonical statement of the problem in §6.6: "there is no EOT on a socket," and it is your responsibility to call recv again until your message is complete. Read the myreceive example next to read_request.
• libcurl error codes — what 18 (CURLE_PARTIAL_FILE, "shorter or larger than expected") and 28 (CURLE_OPERATION_TIMEDOUT) actually mean. Worth skimming the whole list; it is a catalogue of the ways a byte stream can disappoint you.
• MDN: Transfer-Encoding — chunked encoding with a worked example, including the terminating zero-length chunk that CF2 deletes.
• PortSwigger: HTTP request smuggling — what §6.2's both row becomes when a front-end and a back-end resolve the Content-Length/Transfer-Encoding tie-break differently. CL.TE, TE.CL and TE.TE, with concrete payloads.
• curl man page — for --max-time, -s, and -v, the three flags that make §6 reproducible.