Writing AI Requirements Intermediate

AI requirements go beyond traditional user stories. You must specify data needs, accuracy thresholds, latency constraints, edge case handling, and fairness criteria. This lesson teaches you how to write requirements that give your ML team clarity while maintaining the flexibility that AI development demands.

The AI Requirements Template

An effective AI product requirement includes these components:

Component Traditional PM AI PM Addition
User Story As a user, I want X so that Y Same, but include AI behavior expectations
Acceptance Criteria Pass/fail conditions Statistical thresholds (e.g., >95% precision)
Data Requirements Typically not needed Training data specs, volume, labeling requirements
Performance Response time, uptime Inference latency, throughput, model size constraints
Error Handling Error messages, retry logic Confidence thresholds, fallback behavior, human escalation
Fairness Accessibility standards Bias testing criteria, protected group performance parity

Defining Accuracy Targets

Setting the right accuracy target requires understanding the business context:

  1. Understand the baseline

    What is the current accuracy without AI? If humans handle the task today, measure their accuracy. If there is an existing heuristic, benchmark it. Your AI must beat this baseline to justify the investment.

  2. Assess error costs

    Not all errors are equal. A false positive in spam detection (marking a legitimate email as spam) has a different cost than a false negative (letting spam through). Define acceptable rates for each type of error.

  3. Set tiered targets

    Define minimum viable accuracy (MVP threshold), target accuracy (good enough for full launch), and aspirational accuracy (best-in-class). This gives the ML team clear goalposts.

  4. Specify evaluation methodology

    Define how accuracy will be measured: which test dataset, what metrics (precision, recall, F1, AUC), and how often evaluation occurs.

Data Requirements Specification

Clearly document what data the AI feature needs:

  • Training data: Volume needed, format, labeling requirements, and sources
  • Input data: What data the model receives at inference time, format, and quality expectations
  • Output data: What the model returns, confidence scores, and structured output format
  • Refresh requirements: How often does the model need new data? How quickly must it adapt?

Edge Cases and Failure Modes

AI products need explicit handling for situations where the model struggles:

  • Low confidence: What happens when the model is not sure? Show alternatives? Ask for human input?
  • Out-of-distribution inputs: What if the input is unlike anything in the training data?
  • Adversarial inputs: How should the system handle intentionally misleading inputs?
  • Missing data: What if required input fields are empty or malformed?
  • Model unavailability: What is the fallback if the model service is down?
Requirements Tip: Write requirements as ranges rather than absolutes. Instead of "the model must be 95% accurate," write "the model must achieve at least 90% precision and 85% recall on the standard test set, with a target of 95% precision and 92% recall." This gives the ML team room to optimize the right trade-offs.
Collaboration is Key: AI requirements should be co-authored by the product manager, ML lead, and domain expert. The PM brings the user context, the ML lead brings technical feasibility, and the domain expert validates that the requirements capture the nuances of the problem.

Ready for AI Development?

The next lesson covers managing the AI development cycle, including data pipelines, model training, evaluation, and working with ML teams.

Next: AI Development →
← AI Product Discovery AI Development →