Best Practices Advanced

This final lesson consolidates production-ready guidelines for structured output. Follow these practices to build reliable, maintainable systems that extract structured data from AI models at scale.

Choosing the Right Format

Use Case Recommended Approach Why
Simple data extraction JSON mode + Pydantic Type-safe, validated, widely supported
Complex nested data Structured outputs (schema-enforced) Guarantees exact schema match
Mixed text + data XML with tag extraction Naturally supports interleaved content
Multi-provider support Instructor library Same code works across OpenAI, Anthropic, Google
Actions + structured data Function calling Designed for structured arguments with execution

Schema Design Guidelines

Keep Schemas Simple:
  • Use flat structures when possible; avoid deep nesting (3+ levels)
  • Use enums/Literal types for fields with known values
  • Make fields Optional with defaults rather than required when the data may not exist
  • Add Field descriptions — the model reads them to understand what you want
  • Keep the total number of fields under 15–20 for best reliability

Schema Evolution

As your application evolves, your output schemas will change. Handle this carefully:

  • Add new fields as Optional: This is backward-compatible. Old responses still validate.
  • Never remove required fields: This breaks existing consumers. Deprecate first, then remove.
  • Version your schemas: Include a schema_version field so consumers know which version to expect.
  • Test with real model output: Schema changes can affect model behavior. Always test with actual API calls.

Common Mistakes

❌

Overly Complex Schemas

A 50-field deeply nested schema will produce unreliable results. Split into multiple focused extraction steps instead.

❌

Missing Error Handling

Calling json.loads() without try/except will crash your application. Always handle parse failures, even with JSON mode.

❌

No Semantic Validation

Valid JSON with nonsense values (age: -5, date: "yesterday") passes structural validation. Add semantic checks.

❌

Blind Trust in Output

Even with structured output, the model can hallucinate values. Cross-reference extracted data with source material when accuracy matters.

Production Monitoring

What to Monitor:
  • Parse success rate: Percentage of responses that pass all validation layers
  • Retry rate: How often retries are needed (target: under 5%)
  • Failure types: Which validation layer fails most (syntax, schema, semantic)
  • Latency impact: How much retries add to total response time
  • Cost impact: Additional token cost from retries and error feedback
  • Schema drift: Whether model output patterns are changing over time (model updates)

Decision Flowchart

Decision Guide 
Need structured output from AI?
  |
  |-- Need to call external functions? → Use Function Calling
  |
  |-- Need mixed text + data? → Use XML Output
  |
  |-- Using OpenAI?
  |     |-- Need strict schema? → Structured Outputs (json_schema)
  |     |-- Simple JSON? → JSON Mode (json_object)
  |
  |-- Using multiple providers? → Instructor + Pydantic
  |
  |-- Using Anthropic?
        |-- Need strict schema? → Tool Use with forced tool
        |-- Simple extraction? → XML tags or Instructor

Course Complete!

You now have a comprehensive understanding of structured output across all major AI providers. Start with the simplest approach that meets your needs, add validation layers as you scale, and monitor parse metrics in production.

← Validation Course Overview →