Glossary
AI-in-production terms, as we use them
Precise definitions you can hold a vendor to, including us. Each one links to where it matters on this site.
- Agent
- A model that calls tools in a loop with a stopping rule. Anything else is a workflow. We scope an agent’s tools and approval points before it reaches production. Where it matters
- Assessment
- A 2–3 week fixed-scope, fixed-price engagement that ends in a ranked roadmap of AI use cases with effort and risk per item, or a written no. Where it matters
- Audit
- How platform work starts: what exists, who can operate it, what it costs, and where product or AI work is blocked. Ends in the smallest platform design that removes the blockers. Where it matters
- Decision record
- One short document per model, prompt or data decision: what was chosen, what was rejected, and the evidence. Written so your team can defend it later. Where it matters
- Evaluation suite
- A set of real cases with expected answers, run in CI against every prompt, model or retrieval change. It is how “better” is measured instead of guessed. Where it matters
- Golden set
- A starting set of real cases with the correct answer written down, drawn from the work of the people who will use the result. Where it matters
- Guardrail
- A check between the model and the world: input validation, output constraints, a human approval, or a rule that refuses. Measured in the evaluation suite.
- Handover
- Not a phase at the end. Code, infrastructure and docs are in your accounts from the first commit, and your engineers pair with ours until they run it alone. Where it matters
- Infrastructure as code
- Your cloud environment described as code, usually Terraform, so it can be rebuilt, audited and rolled back. Where it matters
- Milestone pricing
- Pilot or scale work quoted per milestone. Scope is written down before work starts. Where it matters
- Pilot
- One use case taken to production, typically in 4–8 weeks, with an evaluation suite in CI, security review, monitoring and a runbook. A demo every two weeks. Where it matters
- Prompt version
- Prompts are deployed like code: versioned and reversible. A prompt change is a release. Where it matters
- Retrieval
- Looking up your own documents or records to ground a model’s answer. We use it where it earns its keep, which the evaluation suite decides. Where it matters
- Rollback
- Returning to the previous model, prompt or infrastructure version. Versioned, reversible, and documented in the runbook. Where it matters
- Runbook
- The document the person on call reads: what each alert means, the first action, the rollback command, who to call. Where it matters
- Time and materials with a weekly cap
- The alternative to milestones: time and materials with a weekly cap, reviewed at every two-week demo. Where it matters
- Two-week iteration
- The unit of work after the pilot. Each ends in a demo in your environment and continues only while it earns its keep. Where it matters
- Written no
- The assessment’s other outcome: the use case, the blocker, and what would have to change. Delivered in writing, as the assessment’s result. Where it matters
A term we should define?
Tell us which word a vendor used that you could not pin down.
Book a free 30-minute scoping callWe reply within two business days. No deck or RFP needed.