OpenAI & Claude APIs

This post reframes the provider choice from a shopping decision into an engineering one. The cover sets that tone deliberately: the question is not 'which is better' in the abstract, but 'which constraints am I committing my product to'. Every subsequent slide unpacks one of those constraints.

The core argument is that your provider choice couples to far more code than you expect. Prompt formatting, tool/function schemas, streaming handlers, response parsing, and even error types are all provider-specific. By the time a product is live, hundreds of small assumptions about one API's shape are baked in across the codebase.

That is why a late switch is a migration project, not a config change. The defensive move — isolating the provider behind one interface from the start — costs almost nothing early and saves enormous pain later. Treat 'which API' with the same gravity as 'which database'.

LLM economics differ fundamentally from typical SaaS. There are no seats; cost scales with the volume of tokens flowing through every request. A feature that looks free in a demo with ten users can become the single largest line item at ten thousand.

Crucially, input and output tokens are priced separately, and output is typically three to five times more expensive than input. A chatty system prompt inflates input cost on every call, while verbose model responses inflate the pricier output side. Designing for brevity — tight prompts, capped outputs, summarized history — is therefore a direct cost lever, not just a style preference.

The bar chart ranks the main drivers of your bill so you can target optimization where it pays. Output tokens dominate because of their price multiplier, which is why trimming response length usually beats trimming the prompt. Calls per user matters because an over-eager agent that makes five API calls where one would do multiplies everything.

Model tier is the other big lever: a large model can cost an order of magnitude more than a small one for the same request. The actionable read is to right-size each task — use a small fast model for classification or routing, and reserve the expensive model for work that genuinely needs it.

Latency is treated here as a product feature rather than a backend metric, because users experience it directly. Time-to-first-token determines whether a chat feels responsive; total generation time determines how long they wait for a complete answer. Larger models and larger contexts both increase latency.

Streaming mitigates the perception problem by showing tokens as they arrive, but it cannot reduce the underlying compute time. The design consequences are concrete: stream wherever the UI allows, keep contexts as lean as the task permits, and consider a smaller model when interactivity matters more than raw quality.

Data governance is where many enterprise deals live or die, so this slide stays precise. Both OpenAI and Anthropic state that data sent through their standard APIs is not used to train their models by default — a meaningful difference from their consumer chat products. Enterprise tiers layer on data-residency, retention controls, and agreements like a BAA or DPA where regulation requires them.

The non-negotiable advice is to read the current policy rather than rely on hearsay, because these terms change. And remember the division of responsibility: the provider controls what happens on their side, but logging, redaction, and PII handling on your own side remain entirely your responsibility.

This compares two architectural stances. Committing to a single provider yields simpler code, one bill, and one support relationship, at the cost of lock-in and a single point of failure — if they have an outage, you have an outage. Building an abstraction layer lets you swap models, fail over during incidents, and benchmark competitors, but it adds glue code and tends to limit you to features common to both providers.

There is no universally correct answer. Early-stage products usually benefit from the speed of a single provider; mature, high-availability products often justify the abstraction. The key is to choose consciously and revisit the choice as the product's reliability requirements grow.

This slide captures a fast-moving reality: which model is 'best' rotates every few months as each lab ships new versions. Coding ability, long-context reasoning, vision, latency, and tool use each have a current leader, and those leaders change independently.

The strategic implication is to optimize for re-evaluability rather than for today's winner. Keep a small benchmark of your actual tasks, run it against candidates periodically, and structure the code so switching is cheap. Teams that hardwire today's leaderboard into their architecture pay for it when the landscape shifts under them.

The code makes the abstract advice concrete by turning the model and its prices into configuration rather than scattered literals. A single config module names the model and its per-token input and output prices, and a cost() helper turns a usage object into dollars.

This pattern delivers two wins at once. First, switching models or updating prices is a one-file change. Second, computing real cost per request becomes trivial, which makes the cost arguments from earlier slides measurable instead of theoretical — you can log dollars per call and find the expensive paths immediately.

The decision tree offers a starting heuristic, explicitly framed as a starting point rather than a verdict. If your workload is many tiny, latency-sensitive calls, a small fast model from either provider usually wins. If it involves long documents and careful reasoning, Claude's large context and reasoning strength are a natural fit. Otherwise, the honest answer is to benchmark both on your own task.

The tree's real lesson is in its final leaf: generic leaderboards are a weak proxy for your specific workload. A short evaluation on representative inputs beats any blog ranking, and it takes an afternoon to set up.

The closing mistake names the failure mode this entire post is designed to prevent: choosing a provider casually, hardcoding it everywhere, and discovering the cost or capability consequences only in production. The pattern is common because the early friction of doing it 'right' feels unnecessary when everything works in the demo.

The remedy is structural and cheap: route every model call through a single function or interface from day one, so the provider decision remains reversible. You are not committing to multi-provider complexity — you are simply refusing to scatter an irreversible choice across your codebase.

This post deliberately avoided syntax to keep the focus on consequences, because the consequences are what teams underestimate. Cost, latency, governance, lock-in, and the shifting capability frontier are all downstream of the provider choice, and all of them are easier to manage when the choice is isolated and measured.

With the 'why' established, the next post zooms all the way in to the mechanics: how a request actually becomes tokens, how the model generates each one, and why replies sometimes stop early.