Day 12 · Calling the Claude API

In Project 3 you wrote the server side of HTTP. Today you cross to the other side and become a client: askr will take a question, POST it to the Claude Messages API, and print the answer. The mechanics are the same Result-and-? discipline you already know, plus one new idea that shapes the whole project — putting the network behind a trait so the program is testable without it.

Why a trait is the first thing we build

The naive version of this program calls reqwest directly from main. It works exactly once: the first time you run cargo test, every test tries to open a socket to api.anthropic.com, needs a real API key, costs money, and fails on a plane. The fix is to make the network an interface, not a hardcoded fact:

   main / tests  ─────►  trait LlmClient  ◄─────┐
                          complete_streaming      │
                              ▲          ▲        │
                              │          │        │
                       ClaudeClient   MockLlm  (more impls later)
                       (real HTTP)   (offline, deterministic)

Callers depend on the trait. Tests inject MockLlm; production injects ClaudeClient. This is the same “depend on a capability, not a concretion” move you’d reach for in any language — but in Rust the compiler enforces it: a Box<dyn LlmClient> can only call methods the trait declares, so a test can’t accidentally trip over a real socket.

pub type TokenSink<'a> = dyn FnMut(&str) + Send + 'a;

#[async_trait]
pub trait LlmClient: Send + Sync {
    async fn complete_streaming(
        &self,
        request: &CompletionRequest,
        sink: &mut TokenSink<'_>,
    ) -> Result<String>;
}

Why #[async_trait]?

Rust has native async fn in traits, but a trait that you want to use as Box<dyn LlmClient> (dynamic dispatch) needs each async method to return a boxed future so the trait stays object-safe. The async-trait crate rewrites async fn into exactly that boxing for you. It’s the standard, boring choice — reach for it whenever you want dyn + async together.

The Messages API shape — get this exactly right

This is the stable contract. A completion is one HTTP request:

   POST https://api.anthropic.com/v1/messages

   headers:
     x-api-key:          <your key>
     anthropic-version:  2023-06-01
     content-type:       application/json

   body:
     {
       "model":      "<a current Claude model id>",
       "max_tokens": 1024,
       "messages":   [ { "role": "user", "content": "..." } ]
     }

The response carries the answer in a content array of blocks; for a plain text reply you read the text of the first block. (Tomorrow we switch this same endpoint to streaming, which changes the response into a sequence of events — but the request shape is identical.)

In Rust, serde turns your structs into that JSON and back. We model the request as plain types and a private “wire” struct that matches the body byte-for-byte:

#[derive(Debug, Clone, Serialize)]
pub struct Message {
    pub role: Role,        // serializes to "user" / "assistant"
    pub content: String,
}

#[derive(Serialize)]
struct WireRequest<'a> {
    model: &'a str,
    max_tokens: u32,
    #[serde(skip_serializing_if = "Option::is_none")]
    system: Option<&'a str>,
    messages: &'a [Message],
    stream: bool,
}

The #[serde(rename_all = "lowercase")] on Role is what makes the enum variant User serialize as the string "user" — the API speaks lowercase, your code speaks Rust, and serde is the translator. The skip_serializing_if means an absent system prompt simply doesn’t appear in the JSON, rather than showing up as null.

Keep the model id out of the code

Notice model is a value we pass in, never a constant. Model ids are versioned and get retired; baking one into the binary bakes in an expiry date. askr reads it from an environment variable with a clearly-labelled placeholder default:

pub fn model() -> String {
    // placeholder default — set ANTHROPIC_MODEL to a current Claude model id.
    // See the model list at https://docs.anthropic.com (Models overview).
    env::var("ANTHROPIC_MODEL").unwrap_or_else(|_| "claude-opus-4-8".to_string())
}

Treat the default as a value to override, not a promise. Pricing and model names move; your config shouldn’t be a source-code edit.

The real client

ClaudeClient holds a reused reqwest::Client (it pools TCP/TLS connections, so a second request is cheaper) and the key. The send is unremarkable — and that’s the point; the interesting design work was the trait:

let resp = self.http
    .post(ANTHROPIC_API_URL)
    .header("x-api-key", &self.api_key)
    .header("anthropic-version", ANTHROPIC_VERSION)
    .header("content-type", "application/json")
    .json(&body)            // serde serializes WireRequest here
    .send()
    .await?;

if !resp.status().is_success() {
    let status = resp.status().as_u16();
    let raw = resp.text().await.unwrap_or_default();
    let message = extract_api_error(&raw).unwrap_or(raw);
    return Err(AskrError::Api { status, message });
}

Two things the compiler made us do, both of which are real bugs in other languages:

• .await? on every step. send() can fail (DNS, TLS, timeout); text() can fail. Each returns a Result, and ? forces you to either handle it or propagate it. You cannot accidentally ignore a network error.
• Check the status before trusting the body. A non-2xx response still has a body — usually a JSON error like {"type":"error","error":{"message":"..."}}. We surface that message in a typed AskrError::Api { status, message } instead of parsing an error body as if it were an answer.

The error type

One enum carries every failure, with #[from] so ? converts foreign errors automatically:

#[derive(Debug, Error)]
pub enum AskrError {
    #[error("missing environment variable: {0}")]
    MissingEnv(String),
    #[error("HTTP request failed: {0}")]
    Http(#[from] reqwest::Error),
    #[error("Claude API error ({status}): {message}")]
    Api { status: u16, message: String },
    #[error("could not read the documents directory or a file: {0}")]
    Io(#[from] std::io::Error),
    #[error("JSON (de)serialization failed: {0}")]
    Json(#[from] serde_json::Error),
}

This is the thiserror pattern from kvlite, now spanning the network: a reqwest::Error, an io::Error, and a serde_json::Error all flow through one ? into one type your main can match on.

The mock — why the tests are hermetic

MockLlm implements the same trait with no socket: it returns a canned reply (or echoes the question), streamed word-by-word so callers exercise the identical code path. Because tests depend on LlmClient, they never know the difference:

#[tokio::test]
async fn mock_fixed_reply_via_complete() {
    let client = MockLlm::new("forty-two");
    let answer = client.complete(&req("the meaning of life?")).await.unwrap();
    assert_eq!(answer, "forty-two");
}

cargo test is green with no ANTHROPIC_API_KEY and no network. That’s the requirement the trait was there to satisfy.

Secrets come from the environment, never the source

The key is read at the boundary and nowhere else:

pub fn api_key() -> Result<String> {
    env::var("ANTHROPIC_API_KEY")
        .map_err(|_| AskrError::MissingEnv("ANTHROPIC_API_KEY".to_string()))
}

A hardcoded key is a one-line mistake with a long tail.

It actually happened — secrets in public commits

GitGuardian’s 2023 State of Secrets Sprawl report found roughly 10 million secrets — API keys, database credentials, tokens — committed to public GitHub repositories during 2022, a figure that has risen every year since. Once a key is in a commit it’s effectively public forever (git history, forks, mirrors, scrapers), and automated bots scan new commits for credentials within seconds. Reading the key from the environment, and keeping it out of the repo, is the cheapest security control you will ever apply.

Under the hood — what reqwest is doing before your await returns

.send().await is a lot of machinery compressed into one line. Under it, reqwest (on tokio) resolves DNS, opens a TCP connection, runs the TLS handshake (we built with rustls, so no system OpenSSL is needed), writes the HTTP request, and yields the task back to the runtime while it waits for bytes. That yield is the whole reason async exists here: a network round-trip is mostly waiting, and an await point lets the runtime do other work meanwhile instead of blocking a thread. The borrow checker has already proven that everything you’re holding across that await is safe to hold — which is exactly the guarantee that makes “thousands of in-flight requests on a handful of threads” sound rather than terrifying.

Build it

Today’s slice — the one-shot ask:

1. cargo new askr --bin, add tokio, reqwest (json + stream + rustls-tls), serde, serde_json, thiserror, async-trait.
2. Define LlmClient, CompletionRequest, Message, Role, and AskrError.
3. Implement ClaudeClient (real POST) and MockLlm (offline).
4. In main, pick the client: real if ANTHROPIC_API_KEY is set, otherwise the mock (and say so).
5. cargo test green offline; cargo run -- "your question" prints an answer.

Tomorrow we make the answer stream, and teach askr to read your local files first.

Check your understanding

1. Why does askr hide the network behind the LlmClient trait instead of calling reqwest directly from main? Name two concrete things it buys.
2. List the four parts of a Claude Messages API request a client must get right (endpoint, the three headers, and the required body fields).
3. Why is the model id read from ANTHROPIC_MODEL rather than written as a constant in the code?
4. After .send().await?, why does ClaudeClient check resp.status().is_success() before reading the body as an answer? What would go wrong if it didn’t?
5. Why can cargo test pass with no API key and no network — what is actually under test in the mock path?

Show answers

1. So the program is testable and runnable without the network. Concretely: (a) cargo test swaps in MockLlm, so the suite needs no API key, no socket, and no money and runs fast/hermetically; (b) the binary can fall back to the mock when no key is set, and swapping providers later is a new impl, not a rewrite. The compiler enforces that callers only use what the trait declares.
2. Endpoint: POST https://api.anthropic.com/v1/messages. Headers: x-api-key: <key>, anthropic-version: 2023-06-01, content-type: application/json. Body: model, max_tokens, and a messages array of { "role", "content" } objects.
3. Model ids are versioned and eventually retired, so hardcoding one bakes an expiry date into the binary. Reading it from an env var (with a clearly-labelled placeholder default) makes the model a config value you can change without editing and recompiling source.
4. A non-2xx response still has a body, but it’s a JSON error object, not an answer. Without the status check you’d parse the error as a successful reply — surfacing garbage or a confusing parse failure instead of the real problem. The check lets askr return a typed AskrError::Api { status, message } carrying the server’s actual error message.
5. The mock path exercises the trait, request building, message/role serialization, and the streaming sink — all the logic that isn’t the socket. MockLlm implements LlmClient with a deterministic, in-memory reply, so everything except the actual HTTP call is covered without touching the network.

---

Prev: Overview — Project 4 · Next: Day 13 · Streaming and RAG →