In plain terms
An orchestra has excellent players and still needs a conductor: someone who decides who comes in when. In an AI system the players are models, tools and agents. Orchestration is the conducting: passing work along, waiting for an approval, retrying a step that failed.
Why it matters
A demo needs a model; a product needs orchestration. Reliability, cost control and auditability live in this layer: what happens when a step fails, where a human signs off, which model handles which job. When a vendor says “agent platform”, this layer is most of what you are buying.
Example
A loan application passes through document extraction, a credit-policy check, a fraud model and a human underwriter. The orchestration layer routes each file, pauses for the underwriter's decision, retries a failed extraction and records every step for the regulator.
Most often confused with
Orchestration vs. Agent
An agent is a worker that decides its own steps. Orchestration is the arrangement around workers: who gets which task and what happens to the result. The orchestrator can be plain code, or itself an agent that delegates to subagents.
Under the hood
Two styles. In code-driven orchestration, control flow is written as a graph or state machine with explicit nodes, edges and persisted state; it is predictable and testable. In model-driven orchestration, a lead agent plans and delegates at run time; it is flexible and harder to bound. Production concerns are the same in both: durable state so long runs survive restarts, retries and timeouts, interrupts for human approval, routing to cheaper models where possible, and tracing so every step can be inspected afterwards.