The AI Plumber
Glossary
The reference guide for AI infrastructure terminology. Written by practitioners, not marketers.
A
Agent Identity & Auditability
The practice of assigning unique, traceable identities to AI agents operating in production, ensuring every action taken by an agent can be attributed, audited, and rolled back. Essential for SOC2 and EU AI Act compliance.
AI Observability
Full-stack tracing of LLM inputs, outputs, latency, cost, and drift across agent pipelines. Traditional APM tools track request/response cycles. AI observability tracks token consumption, prompt effectiveness, hallucination rates, model routing decisions, and cost per inference. It answers: "Why did this agent choose GPT-4 instead of Claude for this request?" and "How much did that decision cost us?"
Without observability, teams fly blind — debugging production AI issues becomes archaeological excavation instead of log analysis.
In Practice
A fintech team discovers their LLM costs doubled in a week. With AI observability, they trace it to a routing change that sent 80% of simple queries to the most expensive model. Without it, they'd still be guessing.
C
Context Debt
The compounding cost of undocumented architectural decisions. Unlike technical debt (bad code you chose to ship), context debt is invisible. It lives in Slack threads, hallway conversations, and the memories of engineers who've since left. Every time a team member asks "why did we choose this model?" and nobody can answer, that's context debt compounding.
It manifests as re-opened tickets, repeated debates, slowed onboarding, and the quiet tax of teams re-litigating yesterday instead of building tomorrow. The antidote is Architecture Decision Records (ADRs).
In Practice
A team spends three weeks reverse-engineering why their agent uses GPT-4 for a pipeline that Haiku could handle. The original architect left. The ADR was never written. That's three weeks of context debt repayment — with interest.
E
EU AI Act Compliance (for SaaS)
The regulatory framework governing AI systems deployed within the European Union. For SaaS companies, compliance requires risk classification of AI features, transparency obligations, human oversight mechanisms, and documentation of training data provenance.
G
Governance-First Framework
A deployment methodology that treats compliance, observability, and access control as prerequisites — not afterthoughts — before any AI agent gets write access to production systems. Most teams bolt governance on after shipping. Governance-first inverts this: you build the guardrails before you build the agent.
This means identity management, audit logging, cost controls, and human escalation paths are architected from day one. The framework reduces the 70% failure rate caused by governance gaps to near-zero by making ungoverned deployment architecturally impossible.
In Practice
A SaaS company implements governance-first for their AI features. Before any agent code is written, they deploy identity management, audit logging, and cost guardrails. When the agent ships 60 days later, it's production-ready on day one — no post-launch scramble.
L
LLM Cost Guardrails
Rate limiting and budget enforcement mechanisms that prevent runaway API costs in LLM-powered systems. Without guardrails, a single misconfigured agent loop can generate thousands of dollars in API calls within minutes.
M
MLOps Pipeline
The infrastructure layer responsible for deploying, monitoring, versioning, and scaling machine learning models in production. For LLM workloads, this extends to prompt versioning, model routing, A/B testing, and cost optimization.
Model Routing Engine
A middleware layer that dynamically selects the optimal LLM for each request based on complexity, cost, latency requirements, and model capabilities. Model routing prevents the common anti-pattern of sending every request to the most expensive model.
Multi-Agent Orchestration
The coordination layer that manages communication, task delegation, and state sharing between multiple AI agents operating within a single system. Orchestration determines which agent handles which task, how agents share context, and how conflicts between agent outputs are resolved.
S
SaaS Staircase (AI Maturity Model)
An AI maturity model that maps the progression from basic automation through governed AI to autonomous agent systems. Each step on the staircase requires specific infrastructure, governance, and observability capabilities before advancement is safe.
Go Deeper
Need these concepts applied to your stack?
Book a 30-min Architecture Review