Day 7 · TCP Server

By the end of Day 6 kvlite is fast, thread-safe, and durable — but you can only talk to it from its own REPL. Today it becomes a server: a process that listens on a TCP port and answers many clients at once, each on its own connection, speaking the same line protocol the REPL already uses. We do it with std::net and one OS thread per connection — no async, no tokio. That deliberate choice is the lesson: thread-per-connection is the simplest correct network server, and feeling exactly where it stops scaling is the best possible setup for async in Project 3.

The shape of a blocking server

A TCP server is a loop around one blocking call. TcpListener::accept waits until a client connects, hands you a connected socket, and you go again:

   use std::net::TcpListener;

   pub fn serve_on(listener: TcpListener, db: Db) -> Result<()> {
       for incoming in listener.incoming() {       // blocks until a client connects
           let stream = incoming?;
           let db = db.clone();                     // cheap: an Arc bump (Day 5)
           std::thread::spawn(move || {             // one thread owns this connection
               if let Err(e) = handle_connection(stream, db) {
                   eprintln!("connection ended: {e}");
               }
           });
       }
       Ok(())
   }

The whole architecture is in those lines:

   main thread:  accept ──▶ accept ──▶ accept ──▶ ...   (loops forever, never blocks on a client)
                    │           │          │
                    ▼           ▼          ▼
                 thread 1    thread 2    thread 3        (each: read line ▸ run ▸ reply ▸ repeat)
                    └───────────┴──────────┘
                                │
                          one shared Db   (Arc<RwLock<..>> + the log)

Each connection gets its own thread, and each thread gets a clone of the Db handle. Because that handle is Arc<RwLock<MemStore>> plus an Arc<Mutex<Wal>> (Days 5–6), every clone points at the same map and the same log — so two clients see one consistent store, and the locks serialize their writes. The move keyword transfers the handle into the closure; it’s only a handle, so the data isn’t copied.

Send is what makes thread::spawn accept the closure

thread::spawn requires its closure to be Send — everything captured must be safe to move to another thread. The closure captures stream (a TcpStream, Send) and db (a Db built from Arc<RwLock>/ Arc<Mutex>, which are Send + Sync). So it compiles. Capture something thread-unsafe — say an Rc — and you’d get the same compile error from Day 5. The network layer didn’t need any new safety machinery; the types you already built carry their thread-safety with them.

One connection: two buffered views of one socket

A TcpStream is bidirectional, but we want a buffered reader (to read whole lines) and a buffered writer (to batch the reply) at the same time. try_clone gives us a second handle to the same underlying socket so we can wrap each direction independently:

   pub fn handle_connection(stream: TcpStream, db: Db) -> Result<()> {
       let reader = BufReader::new(stream.try_clone()?);  // for reading lines
       let mut writer = BufWriter::new(stream);           // for writing replies

       for line in reader.lines() {        // iterator over the socket, one line per request
           let line = line?;
           let trimmed = line.trim();
           if trimmed.is_empty() { continue; }
           if trimmed.eq_ignore_ascii_case("QUIT") {
               writeln!(writer, "BYE")?; writer.flush()?; break;
           }
           let response = handle_line(&db, trimmed);   // parse → execute → Response (Day 6/7)
           writeln!(writer, "{response}")?;            // one reply line
           writer.flush()?;                            // send it now, don't sit in the buffer
       }
       Ok(())   // loop ends on EOF (client closed) — thread exits, Db clone drops
   }

reader.lines() is the same iterator idea as log replay, now over a socket: it yields one request per line until the client hangs up (EOF), at which point the loop ends, the thread returns, and this connection’s Db clone drops (one Arc decrement). The flush() after each reply matters — without it, the response could sit in the BufWriter and the client would hang waiting for an answer that’s stuck in a buffer.

The line protocol

The grammar is deliberately tiny and text-based so a human can speak it. handle_line parses one line into a typed Request, runs it against the Db, and formats a Response — and crucially, a bad request becomes an ERR line, not a dropped connection:

   request            response
   ───────            ────────
   SET k v rest...    OK
   GET k              VALUE <v>   | NIL
   DEL k              DELETED     | NOT_FOUND
   PING               PONG
   QUIT               BYE   (and the connection closes)
   <anything else>    ERR protocol error: unknown command "..."

Keeping parse and execute as separate functions (Day 6) pays off here: the server is a thin shell that does socket I/O and delegates all meaning to the same code the REPL uses. The protocol is written once and tested without a socket.

Under the hood — what accept actually returns

TcpListener::accept isn’t where a connection is made — by the time it returns, the kernel has already completed TCP’s three-way handshake (the client’s SYN, the server’s SYN-ACK, the client’s ACK) and placed the finished connection on the listener’s accept queue (the backlog). accept simply dequeues one completed connection and hands you its socket; if the queue is empty it blocks the calling thread until one arrives. That blocking is the defining trait of this model: the main thread is parked in accept doing nothing while it waits, and each connection thread is parked in read doing nothing between a client’s requests. Simple and correct — and, as we’re about to see, expensive at scale.

Why thread-per-connection, and where it breaks

This model is the right first answer: it’s easy to reason about (each connection is a plain top-to-bottom function), it uses real OS parallelism across cores, and it’s perfect for tens to a few hundred connections. Its ceiling is just as real, and it comes from two costs that scale with the number of connections, not the amount of work:

• Memory: every thread has its own stack. Rust’s default spawned-thread stack is 2 MiB.
• Scheduling: the OS must context-switch among all those threads; past a point, the scheduler spends more time switching than working.

By the numbers — the memory wall

Suppose you want 10,000 simultaneous connections — the classic target — with one thread each:

   stack per thread  = 2 MiB         (Rust std default)
   connections       = 10,000
   stack memory      = 2 MiB × 10,000 = 20,000 MiB ≈ 20 GiB   (just for stacks!)

Twenty gigabytes of RAM before storing a single key — and that ignores per-thread kernel bookkeeping and the context-switch tax. Mostly-idle connections (a chat server, long-lived clients) make it worse: you pay for the threads whether or not they’re doing anything. This is precisely the wall async was invented to climb.

It actually happened — the C10k problem

Around 1999, Dan Kegel named and documented “The C10K problem”: the difficulty of serving ten thousand concurrent connections on the hardware of the day, where the dominant thread-(or process-) per-connection model simply ran out of memory and scheduler. His write-up catalogued the alternative — event-driven, non-blocking I/O built on epoll (Linux) and kqueue (BSD), where one thread multiplexes thousands of sockets and never blocks on any one of them. That lineage runs straight to modern async runtimes: Rust’s tokio, which is exactly what Project 3 uses. kvlite’s thread-per-connection server is the honest “before” picture — build it, feel the wall, and async will land as a solution to a problem you’ve actually had, not a buzzword.

Build it: a networked kvlite, two clients sharing one store

The Day 7 deliverable is the real thing. Start the server:

   $ cargo run -- serve
   replayed 0 record(s) from kvlite.wal
   kvlite listening on 127.0.0.1:6380

In a second terminal, connect with nc and talk to it by hand:

   $ nc 127.0.0.1 6380
   SET name ada lovelace
   OK
   GET name
   VALUE ada lovelace
   PING
   PONG

Now open a third terminal and nc in again — a separate connection on a separate server thread — and GET name. You’ll get VALUE ada lovelace: both clients are hitting one shared, locked Db. That’s the payoff of the whole project visible in one command. The crate’s tests/server.rs automates exactly this: it binds to an OS-chosen port, spawns the server, connects two clients, has one SET and the other GET, and asserts they see the same value. (The restart-survival half is already covered by the Day 6 tests — the server writes through the same Db.)

The thread, tied off

Step back and look at what four days forced you to understand — and what the compiler protected you from at each layer. Day 4: a trait and generics, so the engine is reusable and the borrow rules are written into its signatures. Day 5: Arc<RwLock> and the Send/Sync traits, so sharing the store across threads is type-checked free of data races — fearless concurrency, made concrete. Day 6: Result/? and a thiserror enum, so every disk failure is a value you must handle, and durability is a dial you set with open eyes. Day 7: std::net and one thread per connection, so the network is just more of the same ownership and Send rules you already knew — until the C10k wall hands you the reason async exists.

You built a real key-value store, and at no point could you have shipped the most dangerous bug in systems programming, because the compiler wouldn’t let the program that contained it exist. That’s the whole playbook in one project.

→ Next: Project 3 · apilite — An Async Web API · Prev: Day 6 · Persistence & Errors · Back to the Project 2 overview

Check your understanding

1. In the serve_on loop, what exactly is cloned and moved into each connection thread, and why does that give every client a consistent view of one store rather than separate copies?
2. Why does handle_connection call stream.try_clone(), and what would go wrong if you forgot the flush() after writing a reply?
3. By the time accept returns a socket, what has already happened at the TCP level, and what does accept do when no connection is waiting?
4. Thread-per-connection has a hard ceiling. Name the two costs that scale with the number of connections, and use the 2 MiB default stack to estimate the stack memory for 10,000 connections.
5. What was the C10k problem, and what architectural shift did it push the industry toward — the one Project 3 will use?

Show answers

1. A clone of the Db handle is moved in — and Db is Arc<RwLock<MemStore>> plus Arc<Mutex<Wal>>, so cloning just bumps Arc counts. Every clone points at the same map and log, and the locks serialize access, so all clients share one consistent store rather than independent copies.
2. try_clone gives a second handle to the same socket, so we can wrap one in a BufReader (reading lines) and the other in a BufWriter (writing replies) simultaneously. Without the flush(), a reply could sit in the BufWriter’s buffer and never reach the client, so the client would hang waiting for an answer that was never sent.
3. The kernel has already completed TCP’s three-way handshake (SYN, SYN-ACK, ACK) and put the finished connection on the listener’s accept/backlog queue. accept dequeues one completed connection; if the queue is empty it blocks the calling thread until a connection arrives.
4. The two costs are memory (one stack per thread) and scheduling (context-switching among all the threads). At 2 MiB per thread, 10,000 connections need 2 MiB × 10,000 ≈ 20 GiB of stack memory alone — before storing any data.
5. The C10K problem (Dan Kegel, ~1999) was serving ten thousand concurrent connections when thread/process-per-connection ran out of memory and scheduler. It pushed the industry toward event-driven, non-blocking I/O (epoll/kqueue) where one thread multiplexes many sockets — the lineage of async runtimes like tokio, which Project 3 (apilite) uses.