Serving Models with FastAPI

This is the framing post for the whole day, so it stays at the level of definitions and mental models before any deployment or scaling concern appears. The single most useful idea to internalize is that serving a model means turning a Python function into a network service: something other systems can reach over HTTP, not just code you import in your own script.

Getting this clear early prevents the most common beginner mistake — treating a saved model file as if it were already a product. The .pkl is the start of the deployment work, not the end of it. Once you see 'make the model callable by the rest of the world' as the real task, FastAPI, Pydantic, and Uvicorn stop looking like arbitrary dependencies and become the obvious pieces of that one job.

The core definition: serving a model means putting its predict step behind a network interface so other systems can use it. Instead of importing the model object directly, a calling application sends an HTTP request carrying input features and receives a prediction back as JSON. That indirection is the whole point — it decouples the consumer from the model's implementation.

This framing matters because it explains why serving is language-agnostic. A JavaScript frontend, a Go backend, or a mobile app can all call an HTTP endpoint, none of them needing to know the model is a scikit-learn random forest in Python. The model becomes a service with a stable contract, and that contract is what the rest of your architecture depends on rather than the model internals.

This slide drives home that a notebook or a loose .pkl file is not a service. A trained model sitting in a cell or a pickle can only be used by you, on your machine, in that session. To power a real product it has to be reachable over the network, always on, and callable by other code without a human running a cell. Serving is the step that bridges that gap.

The reframing matters because it resets where the work goes. The model artifact is necessary but inert — like a compiled library nobody has linked against yet. Wrapping it in an always-on API is what makes it usable, and everything else in the day (validation, concurrency, health checks) exists to make that API trustworthy under real conditions.

The pipeline diagram lays out the request lifecycle as four stages: a client sends JSON, the server validates it against a schema, the model runs inference, and a JSON response goes back. This is the heartbeat of every serving system — every request, no matter how complex the model, follows this same shape.

Seeing the lifecycle explicitly is the organizing idea for the whole day. The validate stage is why Pydantic exists; the infer stage is where your model lives; the respond stage is the typed contract callers depend on. Each later post zooms into one of these stages, and the deeper point is that each stage exists to remove one source of failure — bad input, wrong model, or an unpredictable response — from the path between caller and prediction.

This names the three pieces you'll keep meeting all day: FastAPI defines the routes and orchestrates validation; Pydantic types and validates the request body; Uvicorn is the ASGI server that actually runs the application and owns the network socket. They're a stack, not interchangeable — each does one job.

The distinction is worth pinning down early because beginners often conflate them. FastAPI is not a server — it's an application framework that Uvicorn runs, which is why you launch with 'uvicorn main:app'. Pydantic isn't part of FastAPI either; it's a separate validation library FastAPI leans on. Knowing which tool owns which responsibility is what makes the next posts' mechanics legible rather than a blur of names.

This slide answers 'why FastAPI specifically' at a conceptual level. It's built on ASGI, so it's async and can handle many concurrent requests without dedicating a blocked worker to each one. It uses ordinary Python type hints to validate input automatically, and it generates interactive API documentation for free from those same hints. For serving, the payoff is less boilerplate and a typed contract between the caller and the model.

The deeper reason this matters is that the typed contract pushes a whole class of errors to the boundary. Instead of malformed input crashing somewhere deep inside NumPy, it's rejected at the door with a clear message. That single property — validate before inference — is most of why FastAPI became the default for model serving in Python, and the next post defends it in detail.

The code shows the smallest possible serving API so the shape is unmistakable: create a FastAPI app, declare a Pydantic Input model with one typed field, and define a POST /predict handler that receives a validated Input and returns a JSON dict. The launch command in the comment shows Uvicorn running it.

The thing to notice is how little ceremony separates 'a function' from 'a service.' The type hint on the handler parameter is what wires in validation — FastAPI reads it, builds the request parser, and rejects anything that doesn't match before your code runs. This trivial example already contains all three pieces (FastAPI, Pydantic, Uvicorn) doing their jobs, which is exactly why it's worth memorizing as the skeleton everything else grows from.

This compare slide draws the line between the two serving modes you'll choose between constantly. Online (real-time) serving handles one request at a time over a live endpoint where latency is the priority — a fraud check that must answer in milliseconds. Batch (offline) serving scores many rows at once in a scheduled job where throughput, not latency, is what matters — nightly scoring of every customer.

The distinction shapes everything downstream. Online serving is where FastAPI shines and where this whole day focuses, because it's the harder, latency-sensitive case. Batch serving often doesn't need an API at all — a scheduled script reading and writing a database can be the better tool. Knowing which mode your problem is in stops you from building a real-time endpoint for a job that should have been a cron job, or vice versa.

This slide states the single most important serving pattern: load the model once, predict many times. The trained model is loaded into memory when the server process starts and lives as a long-lived object; each incoming request reuses that same object rather than reconstructing it. Loading per request would re-read the file from disk and re-initialize the model every call — often hundreds of milliseconds of pure waste.

The principle generalizes to any expensive per-process resource: database connection pools, tokenizers, feature transformers. Initialize them once at startup, share them across requests. This is also the seed of mistake number one in the final post — loading inside the handler is the most common reason a fast model becomes a slow endpoint, and recognizing the pattern now is what makes that mistake obvious later.

The flow diagram traces a single prediction end to end: an HTTP POST arrives with a JSON body, Pydantic validates and parses it into a typed object, the model's predict() runs on that input, and the result is serialized back to JSON. It's the same lifecycle from the earlier pipeline slide, now drawn as the concrete data path through your code.

Seeing it as a linear flow makes the responsibilities obvious: each arrow is a handoff where one component finishes its job and passes a cleaner artifact to the next. The raw bytes become a validated object, the validated object becomes a prediction, the prediction becomes JSON. Debugging a serving issue is almost always a matter of figuring out which arrow in this flow broke, which is why the mental model is worth carrying into the rest of the day.

The closing tips condense the post into the mental model that makes everything else follow. A model in production is a service, not a file — that reframing is the whole foundation. Validate input before it reaches the model, because the model trusts whatever it's given. Load the model once at startup, because reloading is the classic latency killer. JSON in and JSON out keeps the interface language-agnostic. And the API is a contract, so keeping it stable protects every caller that depends on it.

Internalizing 'service, not file' changes how you approach the rest of the day. You stop thinking about the model in isolation and start thinking about the system around it — the validation, the always-on process, the stable contract. Day 99's next post builds directly on this by defending why FastAPI specifically is the tool that makes this system cheap to build correctly.

This wraps the conceptual groundwork for the day. You now have the vocabulary — serving, online vs batch, the FastAPI/Pydantic/Uvicorn stack, load-once-predict-many — and the central mental model that a production model is a service with a stable contract rather than an inert file. That foundation is what makes the next four posts legible instead of a pile of framework trivia.

The next post builds directly on this by examining why FastAPI in particular pays off: how typed validation, async concurrency, and auto-generated docs each defuse a whole class of real serving failures that you'd otherwise debug the hard way in production.