What is structured output from an LLM?
Structured output means getting a language model to return data in an exact, machine-readable format, usually JSON that matches a schema you define, instead of free text. It is how LLM output becomes usable by code: extracting fields from documents, classifying records or producing arguments for another system.
The moment code reads model output, "usually correct format" is not good enough. One stray sentence before the JSON, or a missing field, breaks the pipeline. Structured output is the set of techniques that close that gap.
What are the ways to get structured output?
There are four common approaches, and they differ in how strongly they guarantee the format.
| Approach | How it works | Format guarantee |
|---|---|---|
| Prompt only | Describe the JSON and show an example | Weak: usually right, sometimes not |
| JSON mode | The API guarantees syntactically valid JSON | Valid JSON, but not necessarily your fields |
| Schema-constrained output | You pass a JSON Schema; decoding is restricted so output must match it | Strong: matches the schema's structure |
| Tool or function calling | The model fills in arguments for a tool defined with a schema | Strong on providers that enforce the schema |
Schema-constrained output works by constrained decoding: at each step, the server only allows tokens that keep the output valid against the schema. OpenAI introduced this as Structured Outputs in August 2024 and reported that its constrained mode scored 100% on its schema-following evaluation, compared with under 40% for an older model using prompting alone. Other providers, including Anthropic and Google, offer schema-enforced output too. Check your provider's documentation for which schema features are supported, since most support only a subset of JSON Schema.
What does a structured output request look like?
You define a schema, then ask for data that fits it. For example, extracting an invoice:
{
"type": "object",
"properties": {
"vendor": { "type": "string" },
"invoice_date": { "type": "string", "description": "ISO date, YYYY-MM-DD" },
"total": { "type": "number" },
"currency": { "type": "string", "enum": ["USD", "EUR", "GBP", "INR"] },
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": { "type": "string" },
"amount": { "type": "number" }
},
"required": ["description", "amount"]
}
}
},
"required": ["vendor", "invoice_date", "total", "currency", "line_items"]
}
Field descriptions matter. The model reads them, so "ISO date, YYYY-MM-DD" does more than the type string alone.
Does valid JSON mean correct data?
No. Constrained decoding guarantees the shape, not the truth. A model can return a perfectly valid invoice object with the wrong total, a hallucinated vendor, or a date in the wrong year. It can also be pushed to fill a required field when the source does not contain it, inventing a value rather than leaving it empty.
- Do: Make fields nullable, or add an explicit "not_found" option, when the data may be missing
- Do: Validate values in code: dates parse, totals equal the sum of line items, IDs exist
- Do: Keep schemas flat and small; deeply nested schemas are harder for the model to fill correctly
- Do: Put reasoning in a field before the answer fields if the task needs thinking
- Avoid: Assume a schema-valid object is accurate
- Avoid: Mark every field required when real inputs sometimes lack it
- Avoid: Change a schema without re-running your evals
Should the model reason before producing structured output?
For tasks that need judgement, yes. Forcing the answer straight into a JSON field removes the room to think that chain-of-thought prompting provides. Two patterns fix this: add a reasoning string field as the first property in the schema, so it is generated before the answer fields, or use a model with built-in reasoning, which thinks before emitting the structured result.
How does structured output relate to tool calling?
They use the same machinery. When a model calls a tool, it produces a structured set of arguments that matches the tool's input schema. Some teams get structured output by defining a single "record_result" tool and forcing the model to call it. Tool calling is the right choice when the model should decide whether and which action to take; schema-constrained output is simpler when you always want one fixed shape back.
Frequently asked questions
What is JSON mode?
JSON mode is an API setting that guarantees the model's output is valid JSON. It does not guarantee the JSON has the fields or types you need; schema-constrained output does.
How do I make an LLM always return valid JSON?
Use your provider's schema-enforced structured output or tool calling with a JSON Schema, then parse and validate the result in code. Prompt-only instructions work most of the time but not always.
Can structured output cause hallucinations?
It can encourage them if the schema requires fields the input does not contain, because the model must fill them. Allow null or "not found" values for anything that may be missing.
What is constrained decoding?
Constrained decoding restricts which tokens a model may generate at each step so the output always matches a given grammar or schema. It is how providers guarantee schema-valid JSON.
Last checked for accuracy on . Written by the solidcoder team.