Who sells the model
Margin's claim
Orchestra
Builds the evals and builds a specialist model where needed; its privacy notice lists Fireworks AI for model hosting, inference and fine-tuning when that workflow is enabled. Calls to Anthropic and OpenAI run on your own provider accounts.
Margin
Sells no model and hosts no inference. Whichever vendor wins on your evals, our flat fee is the same, so we have no stake in which model you run, theirs included.
Sources: Orchestra home, Orchestra privacy notice (updated 2026-09-30), Orchestra terms (updated 2026-05-22), Margin pricing. Read 2026-11-02.
Where it sits
Different designs
Orchestra
A gateway in the request path: you point the OpenAI or Anthropic SDK at its base URL, api.understudylabs.com, with an Orchestra key.
Margin
Not in the request path. A meter beside your calls, so an outage on our side cannot stop your agents. A change arrives as a pull request you merge.
Sources: Orchestra platform, Margin security. Read 2026-11-02.
Getting started
Different designs
Orchestra
Change your SDK's base URL to its gateway and add an Orchestra key, so every production call goes through it.
Margin
Connect a repository or drop a folder. On OpenRouter, turn on Broadcast and paste one endpoint and one header. No code change either way, and no production call ever routes through Margin.
Sources: Orchestra platform, Margin: start, Margin docs: OpenRouter Broadcast. Read 2026-11-02.
The promotion bar
Margin's claim
Orchestra
A new model ships when it meets quality, cost and latency standards, tested against held-out examples. On the pages we read we did not find a numeric bar or the statistic behind one.
Margin
Promoted only when it costs less and the lowest quality your paired evals can vouch for at 95% confidence is at least 0.97 of the incumbent’s. Below that it is held, and the refusal stays on the record.
Sources: Orchestra home, Orchestra platform, Margin: the Parity Gate. Read 2026-11-02.
Held-out confirmation
Both state it
Orchestra
States it: each model is tested against held-out examples.
Margin
A switch picked on some of your cases is confirmed on cases that never chose it, split by a hash of each case, or it is held.
Sources: Orchestra home, Margin: the Parity Gate. Read 2026-11-02.
Size of a change
Margin's claim
Orchestra
Routing rules implement a given model and keep a fallback in place. On the pages we read we did not find whether a change covers one step or a whole route.
Margin
One job at a time. A face-off can move a single step of an agent, every other step as it ran, and the change is a pull request that edits one entry in a model-pin file you own. On our founder's own agents it moved one step this way.
Sources: Orchestra platform, Margin pull request #13 on our founder's own agents (merged 2026-11-02). Read 2026-11-02.
Response time
Both state it
Orchestra
Evaluates quality alongside response time and cost of the complete route.
Margin
Times both arms of a face-off. The pull request prints the median and p90 per model call for each, or says the calls were not timed.
Sources: Orchestra platform, Margin's switch pull request, speed section. Read 2026-11-02.
After the switch
Margin's claim
Orchestra
Ships approved changes with a retained fallback; it can introduce a new route gradually and hold onto the prior route when evidence calls for it. On the pages we read we did not find an automatic revert.
Margin
Watches quality on live traffic after the merge and opens the revert as a pull request when it slips. Undoing a one-job change removes that one entry.
Sources: Orchestra platform, Margin: the Governor. Read 2026-11-02.
Value, not only cost
Margin's claim
Orchestra
The results we read are reported as cost per task, per call or per table, and as speed.
Margin
Price an outcome and Margin holds a switch whose lost passes are worth more than the dollars it saves. With no value supplied we report cost per outcome and say so.
Sources: Orchestra results, Margin: cost per verified outcome. Read 2026-11-02.
When prices or models move
Margin's claim
Orchestra
Not described on the pages we read.
Margin
A daily job re-tests a model when its price drops or its last answer goes stale, and tests a newly released one, with nobody asking. It has run every day since 2026-10-14.
Sources: Margin's production job log (19 daily runs, 2026-10-14 to 2026-11-01). Read 2026-11-02.
Whose model wins
Margin's claim
Orchestra
Trains its own specialist models on your traces, then evaluates them against your current model.
Margin
Builds no model, by design. It picks candidates from the live catalogue of ten labs, and for a one-job face-off you can name any model id your provider serves. It has nothing to gain from which one wins.
Sources: Orchestra home, Orchestra platform. Read 2026-11-02.