LLM Development Roadmap: Tokens, Costs, and Prompt Engineering Basics

There isn't one canonical way to learn LLM development, but there is a bad order: optimizing prompts before you can measure their cost, or reaching for structured output before you understand what a token even is. This roadmap fixes that by sequencing four stages so each one supplies a prerequisite for the next — count tokens correctly, turn those counts into a real monthly bill, cut that bill without breaking behavior, then lock down output shape once cost and reliability both matter enough to justify the extra rigor. Skip a stage and the next one turns into guesswork.

Everything downstream — pricing, context limits, prompt optimization — is denominated in tokens, so starting anywhere else means working with the wrong unit. A token is not a word and not a character; it's a sub-word chunk produced by byte-pair encoding (BPE), and the vocabulary that produces it is specific to a model family. That's why "internationalization" might be one token in one tokenizer and four in another — the two models learned different merge tables from different training corpora. Word count and character count are both cheap proxies; neither is the number a provider actually bills you on or the number that fills your context window.

This is the first thing to get right because two very different problems depend on it being exact rather than approximate: billing (providers charge per token, not per word) and context-window limits (a model truncates or rejects input once it runs out of tokens, not once it runs out of characters). Guessing at either with a word-count heuristic produces estimates that are wrong in a way you won't notice until a request gets truncated or a bill comes in higher than expected.

Start with the token counter guide, which walks through how BPE tokenization actually works and why the same prompt tokenizes differently across model families. Anyone serious about learning LLM development should be able to look at a prompt and know, not guess, how many tokens it costs.

Once token counts are exact, the natural next question is what they cost at the scale you actually run at — a single request's price is nearly meaningless; a workload's monthly bill is what a budget owner asks about. Modeling that bill means multiplying token counts by requests per day, and then by 30, but the part most back-of-envelope estimates get wrong is treating input and output tokens as interchangeable. They aren't — output tokens are typically priced several times higher than input tokens across every major provider, so a workload that generates long completions from short prompts costs far more than the same token count would suggest if you averaged the two rates together.

The single biggest lever most estimates ignore entirely is prompt caching. If your system prompt, tool definitions, or retrieved context are large and repeated across requests, a provider that caches that prefix charges a fraction of the standard input rate for the cached portion on every call after the first. For workloads with a big, stable prefix and a small variable suffix — the common shape for RAG and agent systems — caching isn't an optimization at the margins, it's the difference between a bill that scales linearly with traffic and one that doesn't.

workload: 5,000 requests/day
  input:  1,800 tokens/request  (1,600 of it a repeated system prompt)
  output:   400 tokens/request

naive estimate (input + output averaged at one rate): understates output cost,
ignores that 1,600 of the 1,800 input tokens repeat identically every call

correct model:
  uncached input tokens × input rate
+ cached input tokens   × cached-input rate (often ~10-25% of standard rate)
+ output tokens          × output rate  (typically 3-5x the input rate)
× requests/day × 30

This is stage 2, not stage 1, because you need reliable token counts before a cost model means anything — garbage tokens in, garbage dollars out. Work through the LLM pricing calculator guide to model your own workload's actual monthly bill, input/output split included, before deciding anything needs to change. LLM cost management starts with an honest number, not a vibe.

Once you have a real monthly figure from stage 2, the obvious next move is to bring it down — and the obvious mistake at this stage is treating “shorter prompt” and “better prompt” as the same problem solved the same way. There are two fundamentally different approaches to shrinking a prompt, and they carry very different risk.

A deterministic, rule-based tightening pass — collapsing redundant whitespace, removing filler phrases that don't constrain the model's behavior, deduplicating repeated instructions — is inspectable and reversible. You can diff the before and after, see exactly what changed, and revert if something looks off. Sending the same prompt to another LLM and asking it to “improve” or shorten it is a different category of operation entirely: it's nondeterministic, the same input can produce a different rewrite on a different run, and there's a real risk it silently drops a constraint or nuance that was load-bearing for correctness. A rewritten prompt that's 20% shorter and produces subtly worse outputs isn't an optimization — it's a regression that took a token-count screenshot to look like progress.

This is prompt engineering basics that matter in production, not in a demo: know which class of edit you're making, and prefer the deterministic one whenever the prompt's exact wording is doing real work. The prompt optimizer guide covers exactly this distinction — what a rule-based tightening pass actually changes, what it deliberately leaves alone, and why that's the safer default before you ever hand a prompt to a model to rewrite.

With token costs measured and prompts trimmed of waste, the remaining failure mode is reliability of the output's shape itself — and this is deliberately the last stage, because it only becomes worth the extra rigor once a prompt is stable and its cost is under control. Asking a model nicely in the prompt to “respond with valid JSON matching this shape” works in a demo and fails in production: it drifts under load, degrades with longer context, and offers no hard guarantee the field you depend on downstream will exist, let alone have the right type.

OpenAI's structured outputs feature closes that gap, but it does so with a strict subset of JSON Schema, not the full spec. The model is constrained at generation time to produce output that validates against your schema — a real guarantee, not a hope — but only if the schema itself sticks to the keywords that subset supports. This is the detail that catches people: a schema can be perfectly valid JSON Schema by the spec and still get rejected with a 400 error, because it uses a keyword — an unsupported combination of oneOf, a bare format constraint, or an unrestricted additionalProperties— that falls outside what OpenAI's strict mode accepts.

{
  "type": "object",
  "properties": {
    "email": { "type": "string", "format": "email" }
  },
  "additionalProperties": true
}
→ 400: schema uses unsupported keyword combination for strict mode
  (open additionalProperties + certain format constraints aren't in the
   strict-mode subset — the schema is valid JSON Schema, just not valid
   *strict* JSON Schema)

Learning that OpenAI strict JSON schema subset — which keywords are supported, which get silently restricted, and how to write around the gap — is exactly what the JSON schema generator guide walks through. Reaching this stage last isn't arbitrary: structured output constraints are worth designing carefully only once you already know what the request costs and the prompt driving it is stable, which is exactly what stages 2 and 3 established.