Braintrust vs LangSmith on pricing with worked examples, tracing, evals, agents, CI, hosting and data export, checked against both vendors' docs.
Braintrust and LangSmith both trace an LLM application in production, run evals on datasets and live traffic, manage prompts and route outputs to human reviewers. Choose LangSmith if you build on LangChain or LangGraph, or want trajectory evaluators, judge alignment and an agent runtime ready-made in one product. Choose Braintrust if your team works through experiments and CI, wants unlimited users with no seat price, or writes in Ruby or C#.
They bill on different meters. As of November 2026, LangSmith Plus is $39 per seat a month plus $5 per 1,000 traces beyond the first 10,000. Braintrust Pro is $249 a month for any number of users, plus $3 per GB of data and $1.50 per 1,000 scores beyond its allowances. For five people and a million traces a month, that is $5,145 on LangSmith at list price and roughly $340 to $3,800 on Braintrust, depending on how large your traces are and how many you score.
Both are proprietary, and neither can be self-hosted without an Enterprise contract. Neither documents a statistical test on the difference between two experiments.
This post compares them on pricing with worked examples, tracing, evaluation, agents, CI, hosting and getting your data out. It also checks what each vendor's comparison page says about the other against the other's docs.
A disclosure: I build EvalShift. It replaces neither platform, and it appears only in the section on what both leave to you. Every claim here comes from the vendors' own pricing pages, docs and announcements, read on November 13, 2026.
Braintrust vs LangSmith at a glance #
| Braintrust | LangSmith | |
|---|---|---|
| Company | Braintrust; $80M Series B ↗ , February 2026 | LangChain; $300M Series C ↗ at a $3B valuation, November 13, 2026 |
| License | Proprietary; autoevals and SDKs open source | Proprietary; openevals, agentevals and SDKs open source |
| Free plan | Unlimited users, 1 GB, 10k scores, 14 days of logs | 1 seat, 5k traces, 14 days |
| First paid plan | Pro: $249 a month, unlimited users | Plus: $39 per seat a month |
| What is metered | GB ingested, and scores | Seats, and traces |
| Log retention | 14 days (Starter), 30 days (Pro), up to 180 paid | 14 days base, 180 days extended |
| SDKs | TypeScript, Python, Go, Java, Ruby, C# | Python, TypeScript, Go, Java |
| OpenTelemetry | Ingests OTLP | Ingests OTLP |
| Offline evals | Eval() with data, task and scores |
evaluate() , or pytest, Vitest and Jest tests |
| Ready-made agent evaluators | Write your own trace-level scorers | agentevals trajectory match and judge |
| Judge alignment | No dedicated feature in the docs | Align Evaluator |
| CI | Official GitHub Action that comments on the pull request | Your own workflow running the test suite |
| Agent deployment | No | LangSmith Agent Server |
| Self-hosting | Enterprise BYOC, operated by Braintrust | Enterprise: self-hosted, or BYOC operated by LangChain |
| Cloud regions | US, EU | US, EU, APAC |
Sources: Braintrust pricing ↗, Braintrust plans and limits ↗, LangSmith pricing ↗, LangSmith usage and billing ↗.
Pricing: seats and traces against data and scores #
LangSmith
| Developer | Plus | Enterprise | |
|---|---|---|---|
| Price | Free | $39 per seat a month | Custom |
| Seats | 1 | Unlimited | Custom |
| Included traces | 5k a month | 10k a month | Custom |
| Hosting | Cloud | Cloud | Cloud, BYOC or self-hosted |
Beyond the included traces, both self-serve plans charge $5 per 1,000 base traces, kept for 14 days. Upgrading a trace to extended retention, 180 days, costs a further $2.50 per 1,000. The rate is in a tooltip on the pricing page ↗, quoted in LangSmith units, where one unit is one dollar. It has changed during 2026, and several comparison posts still quote older figures.
A trace is one request end to end, however many model and tool calls it contains. Two details move the bill:
- +Evaluators extend retention by default. A new online evaluator or automation rule upgrades the traces it matches to extended retention unless you opt out, and experiment runs are extended by default. Scoring a trace therefore adds $2.50 per 1,000, though LangSmith has no per-score charge.
- +Other meters. Engine, Fleet, the agent runtime and sandboxes are billed separately in the same units.
Braintrust
| Starter | Pro | Enterprise | |
|---|---|---|---|
| Price | Free | $249 a month | Custom |
| Users | Unlimited | Unlimited | Unlimited |
| Processed data | 1 GB, then $4 per GB | 5 GB, then $3 per GB | Custom |
| Scores | 10k, then $2.50 per 1,000 | 50k, then $1.50 per 1,000 | Custom |
| Log retention | 14 days | 30 days; up to 180 at $0.50 per GB a month | Custom |
Processed data is every byte you ingest across logs, experiments and datasets, measured at ingestion; deleting it later does not reduce the count. Scores are counted across offline evals, online evals and human review. Traces and spans are not counted, and neither are seats. Experiments are kept for up to 365 days on every plan.
What the same team pays on each
The two cannot be put on one unit, so a comparison needs assumptions. Mine: five people, traces of 50 KB each, and two online evaluators scoring 10% of traces. I count a GB as a million KB, and the judge model's own token cost is extra on both.
50,000 traces a month
- +LangSmith Plus: $195 for five seats, plus 40,000 traces beyond the included 10,000 at $5 per 1,000, which is $200. Extended retention on the 5,000 scored traces adds $12.50. About**$408 a month** .
- +Braintrust Starter: 2.5 GB is 1.5 GB over the allowance, $6. The 10,000 scores are included. About**$6 a month** , with 14 days of logs. On Pro it is the $249 base, with 30 days.
1,000,000 traces a month
- +LangSmith Plus: $195 for seats, plus 990,000 traces at $5 per 1,000, which is $4,950:$5,145 a month . Extended retention on the 100,000 scored traces adds $250. Extending every trace to 180 days adds $2,500.
- +Braintrust Pro: $249, plus 45 GB over the allowance at $3, which is $135, plus 150,000 scores over the allowance at $1.50 per 1,000, which is $225:$609 a month .
Three things change the picture:
- +Trace size. One LangSmith base trace costs $0.005, which buys 1.67 MB of Braintrust data at the Pro rate. If your traces average less than that, and most do, Braintrust's data meter is the cheaper of the two. At 10 KB a trace and one evaluator, the million-trace month is about $339. At 200 KB it is about $1,059.
- +How much you score. Score every trace with two evaluators and Braintrust counts 2 million scores: the same month becomes about $3,309 at 50 KB a trace and $3,759 at 200 KB. LangSmith would add $2,500 for the retention upgrade.
- +Headcount. Twenty more people cost $780 a month on LangSmith and nothing on Braintrust.
At list prices Braintrust is cheaper in every case I worked through. Two cautions: a team sending a million traces a month to either vendor would negotiate an Enterprise price, and neither publishes those. And the comparison ignores what LangSmith's bill includes that Braintrust does not sell, such as the agent runtime.
Tracing #
Both accept OpenTelemetry, so an application instrumented with OTel can send spans to either by repointing its exporter: Braintrust OTel ↗, LangSmith OTel ↗. Neither documents exporting OTLP back out.
SDKs. Braintrust documents TypeScript, Python, Go, Java, Ruby and C#. LangSmith has Python, TypeScript, Go and Java.
Does LangSmith need LangChain? No. Its integrations page ↗ covers the OpenAI, Anthropic, Gemini and Bedrock SDKs and agent frameworks that are not LangChain's, including the OpenAI Agents SDK, the Claude Agent SDK, Pydantic AI, CrewAI and the Vercel AI SDK. Where it is LangChain-specific is further up the stack: the Agent Server is a runtime for LangGraph-style agents, and Engine's code fixes target agents built on LangChain's frameworks.
Units. LangSmith groups runs into traces and traces into threads, and bills by the trace. Braintrust stores traces of typed spans, writes scores onto spans, and bills by the byte. Long agent runs with large contexts cost more on Braintrust; many small requests cost more on LangSmith.
Gateways. Both now have an LLM gateway in preview: Braintrust Gateway, which replaced its AI proxy, and LangSmith's LLM Gateway, in beta.
Moving from one to the other. Braintrust ships LangSmith wrappers ↗ that send LangSmith tracing calls to both platforms or only to Braintrust. Its changelog calls them experimental.
Evaluation #
An offline experiment looks much the same in both: a dataset, a function under test and a list of scorers.
Braintrust:
from autoevals import Factuality
from braintrust import Eval, init_dataset
from app import reply # your function that calls the model
Eval(
"Support replies",
data=init_dataset(project="Support", name="Order questions"),
task=lambda input: reply(input),
scores=[Factuality],
trial_count=3,
)
LangSmith:
from langsmith import Client
from app import reply # your function that calls the model
client = Client()
def correct(outputs: dict, reference_outputs: dict) -> bool:
return outputs["answer"] == reference_outputs["answer"]
client.evaluate(
lambda inputs: {"answer": reply(inputs["question"])},
data="Order questions",
evaluators=[correct],
num_repetitions=3,
)
LangSmith can also run a dataset as pytest ↗, Vitest or Jest tests, where each test syncs to a dataset example and each run creates an experiment.
Where they differ:
| Braintrust | LangSmith | |
|---|---|---|
| Scorer library | autoevals (MIT): factuality, similarity, JSON, SQL and more | openevals and agentevals (MIT): judge prompts, trajectory match |
| Trace-level scoring | Scorers that read the whole trace or thread | Thread-level evaluators; trajectory evaluators |
| Pairwise | A human pairwise card | Pairwise evaluators, human or automated |
| Judge alignment | Not found in the docs | Align Evaluator ↗ : label, test the judge against the labels, iterate |
| Human review | Review mode with assignment, several reviewers, blind reviews | Annotation queues ↗ for runs or threads, with rubrics |
| Datasets | From logs, traces, CSV or Loop; named snapshots on Pro | Every change creates a version; rules add matching traces |
| Online evals | Scoring rules with a filter and a sampling rate | Online evaluators and automation rules with a filter and a sampling rate |
| Prompts | Playgrounds, versioned prompts, environments | Playground, Prompt Hub |
The clearest gap is judge alignment. LangSmith has a workflow for checking an LLM judge against human labels and feeding corrections back into it. In Braintrust you would assemble that from human review and a scorer run over labeled logs. If LLM judges carry your evals, that matters: when to trust an LLM judge explains why an unvalidated judge is a guess.
Comparing two experiments
Both let you pick a baseline and see what moved.
Braintrust ↗ matches rows on input, shows a score delta per row, filters to regressions or improvements, and gives the experiment a grade: improvement, regression, trade-off or tie, weighing scores with latency, cost and errors.
LangSmith ↗ colours each example red or green against the source experiment for each feedback key, and counts how many did better or worse at the top of each column.
Repeats and statistics
Both can run each example several times: trial_count in Braintrust, num_repetitions in
LangSmith. Braintrust groups the results by input. LangSmith shows the average and, on a click,
the standard deviation across repetitions ↗.
That is where both stop. I searched both vendors' complete documentation for a confidence interval, a p-value or a significance test on an experiment comparison and found none. Braintrust's grade is not described as a statistical test. LangChain's comparison page says LangSmith experiments have statistical grounding and that Braintrust documents no such thing; the second half is right, and it applies to LangSmith too.
It matters because experiment scores are noisy. On 200 cases, a pass rate of 86% has a 95% confidence interval of roughly 80.5% to 90.1%, so a three-point drop between two experiments can be nothing. How many eval cases do you need has the arithmetic.
Agents #
This is where LangSmith has gone furthest, and where the two companies are heading in different directions.
LangSmith treats the agent as the product. Beyond tracing it has:
- +agentevals ↗ : match a trajectory against a reference in strict, unordered, subset or superset mode, or have a judge grade it.
- +Thread-level online evaluators and multi-turn simulation.
- +Insights, which categorizes production traces, and Chat, formerly Polly, for questions about them.
- +Engine, which clusters production failures into issues and proposes fixes as pull requests.
- +Agent Server, formerly LangSmith Deployment, to run agents, and Fleet, formerly Agent Builder, to build them without code.
Braintrust treats the eval loop as the product:
- +Loop ↗ , an agent that queries logs, writes scorers, runs experiments and builds dashboards, now with scheduled automations.
- +Topics and Patterns, which classify logs and record recurring problems.
- +A thread view, and online scoring across a session logged as several traces.
Braintrust has no agent runtime, and its docs show no ready-made trajectory-match scorer: you write tool-call checks as code or judge scorers at trace level. Multi-turn offline evals are a pattern you build, with one dataset row per conversation.
If you want trajectory checks on day one, LangSmith has them. They are also MIT-licensed functions you can call from any test, which is worth knowing before you pick a platform for them. Evaluating agent tool calls covers what those checks catch.
CI #
Braintrust has an official eval-action ↗. It
runs your evals and creates or updates one comment on the pull request, with a link and a table of
scores showing improvements and regressions against the baseline.
One thing to know before you rely on it as a gate: the action has no threshold input. Per the CI docs ↗, the run fails when an eval throws an error, and failing on a score needs a custom reporter that you write. Braintrust's own comparison article describes merges blocked on score thresholds; that is something you build on the action, not a setting in it.
LangSmith has no official eval action. The documented path is to run the pytest, Vitest or Jest suite in your own workflow, where a failed assertion fails the job. That is less to show on a pull request and a more direct gate.
Either way, the gate is yours to design. LLM regression testing in CI covers what to block on.
Hosting, data and compliance #
| Braintrust | LangSmith | |
|---|---|---|
| Cloud regions | US or EU, chosen when the organization is created | US, EU and APAC, with AWS and Azure options in the US |
| Run it in your cloud | BYOC ↗ on Enterprise; Braintrust operates the data plane | BYOC ↗ on Enterprise; LangChain operates the data plane |
| Operate it yourself | Documented only for teams already running a data plane | Self-hosted ↗ on Kubernetes, an Enterprise add-on |
| Control plane under BYOC | Stays with Braintrust | Stays in LangChain's cloud |
| Compliance | SOC 2 Type II; HIPAA with a BAA on Enterprise | SOC 2 Type II; HIPAA; GDPR |
| Use of your data | Terms limit it to providing the service | Pricing FAQ says it is not used to train models |
For a team that must run the platform itself, LangSmith is the only one of the two that offers it to new customers, on an Enterprise contract. For everyone else the options are the same shape: the vendor's cloud, or the vendor operating a data plane in yours. If you need to self-host without an Enterprise deal, neither fits; LangSmith alternatives and Braintrust alternatives cover Langfuse, Opik and Phoenix.
Getting your data out #
Short retention makes this more urgent than it sounds: on both, logs past their window are gone.
- +Braintrust: download logs, datasets and experiments as JSON or CSV from the UI, with
experiment downloads capped at 1,000 rows, or pull more through the API, SQL or the
btCLI.Scheduled export ↗ to S3 or GCS is Enterprise only. - +LangSmith: query runs through the API or CLI, and download datasets as CSV or JSONL.Bulk export ↗ to S3-compatible storage is Enterprise only for organizations that signed up after August 3, 2026.
Neither ships an importer for the other's datasets or experiments. Datasets move as files. Scorers built on autoevals, openevals or agentevals move with your code, since all three are MIT libraries. Evaluators configured in either UI have to be rebuilt, and experiment history does not move at all.
What each vendor says about the other #
Both publish a comparison page: Braintrust's ↗, from April 2026, and LangChain's ↗, from June 2026. Both products have shipped a lot since, and several claims no longer match the other side's docs.
| Claim | What the docs say in November 2026 |
|---|---|
| Braintrust: LangSmith has no CLI | A LangSmith CLI is documented |
| Braintrust: LangSmith has no gateway | LLM Gateway is in beta |
| Braintrust: turning traces into datasets is manual in LangSmith | Automation rules add matching traces to a dataset |
| Braintrust: its own action blocks merges on thresholds | The action has no threshold input; it needs a custom reporter |
| LangChain: Braintrust has no native multi-turn evaluation | Trace-level scorers, session-level online scoring and a thread view exist; offline multi-turn is a pattern you build |
| LangChain: Braintrust prompts are static and cannot be parameterized | Versioned prompts, environments and parameters are documented |
| LangChain: Braintrust has no equivalent to Align Evaluator | Correct; none found |
| LangChain: Braintrust documents no statistical significance | Correct, and LangSmith's docs describe none either |
The lesson is not that either vendor is careless. A comparison page is a snapshot written by one side, and these products change monthly. Check a claim that matters to you against the other vendor's docs, this post's claims included.
What both leave to you #
Braintrust and LangSmith answer "what is production doing?" and "how does this dataset score?" well. The question a model upgrade raises is narrower: if I swap the model under this agent, does it still call the same tools with the same arguments, and is the difference in scores real or noise?
Both show you the two experiments side by side. Neither tests the difference, and neither ships a gate that fails a pull request on it without code you write.
That is the job EvalShift does. It runs your captured production cases through both models, scores tool selection, arguments and trace rules on both sides, and tests each per-example difference for significance: a paired t-test or Wilcoxon, an effect size with a confidence interval, and a false-discovery-rate correction across the run. A migration policy turns that into a verdict that passes or fails the pull request.
In one published run, the candidate model was 58.8% cheaper and 75.2% faster with no failed calls, and the verdict was still FAIL: it picked a different tool on 25% of comparisons against a 10% limit. A dashboard of cost, latency and errors showed a clean win.
It runs alongside whichever platform you choose. What it does not do: trace production, monitor live traffic or manage prompts. The CLI and SDK are free and open source; Pro, the hosted gate with run history and baselines, is $199 a month. EvalShift vs LangSmith goes dimension by dimension.
How to choose #
| If you | Look at |
|---|---|
| Build on LangChain or LangGraph | LangSmith |
| Want to deploy agents on the same platform | LangSmith |
| Want trajectory evaluators and judge alignment ready-made | LangSmith |
| Need to operate the platform yourself, on an Enterprise contract | LangSmith |
| Have many reviewers, product managers and domain experts who need access | Braintrust |
| Send many small traces | Braintrust |
| Want a pull request comment with score changes out of the box | Braintrust |
| Write in Ruby or C# | Braintrust |
| Score every production trace with several evaluators | Price both; the score meter narrows the gap |
| Need APAC data residency | LangSmith |
| Need to self-host without an Enterprise deal | Neither: Langfuse, Opik or Phoenix |
| Need to gate a model swap before it ships | EvalShift, alongside either |
Then send the same week of traffic from one real agent to both free plans and compare the trace views, the eval setup and what the bill would have been.
FAQ #
Is Braintrust or LangSmith better? Neither is better overall. LangSmith is the better fit for teams on LangChain or LangGraph and for agent work, with trajectory evaluators, judge alignment and an agent runtime. Braintrust is the better fit for experiment-driven teams that want unlimited users, a pull request comment from an official GitHub Action and SDKs in six languages.
What is the difference between Braintrust and LangSmith? Both are proprietary platforms for tracing and evaluating LLM applications. LangSmith bills per seat and per trace and adds agent deployment and ready-made agent evaluators. Braintrust bills per GB of data and per score, has no seat price, and centers on comparing experiments.
Which is cheaper, Braintrust or LangSmith? At November 2026 list prices, usually Braintrust. Five people sending one million 50 KB traces a month, with 10% scored by two evaluators, pay about $5,145 on LangSmith Plus and about $609 on Braintrust Pro. Scoring every trace raises Braintrust to about $3,309. Enterprise prices on both are negotiated.
How much does LangSmith cost? The Developer plan is free with one seat and 5,000 traces a month. Plus is $39 per seat a month with 10,000 traces included, then $5 per 1,000 traces kept for 14 days and a further $2.50 per 1,000 to keep them 180 days. Enterprise is custom.
How much does Braintrust cost? Starter is free with unlimited users, 1 GB of data and 10,000 scores a month, then $4 per GB and $2.50 per 1,000 scores. Pro is $249 a month with 5 GB and 50,000 scores, then $3 per GB and $1.50 per 1,000 scores. Enterprise is custom.
Does LangSmith only work with LangChain? No. It traces the OpenAI, Anthropic, Gemini and Bedrock SDKs, agent frameworks such as the OpenAI Agents SDK and Pydantic AI, and anything that emits OpenTelemetry. Its agent runtime and some newer features are built around LangChain's own frameworks.
Can you self-host Braintrust or LangSmith? Only on an Enterprise plan. LangSmith offers a self-hosted install on Kubernetes and BYOC operated by LangChain. Braintrust offers BYOC, where Braintrust operates the data plane in your cloud account; customer-operated self-hosting is documented only for teams that already run one.
Do Braintrust or LangSmith test whether an experiment difference is statistically significant? Neither documents one. Braintrust grades an experiment as an improvement, regression, trade-off or tie against a baseline. LangSmith shows the average and standard deviation across repetitions. Neither describes a confidence interval or significance test on the comparison.
Keep reading #
- +Braintrust alternatives : Braintrust vs Langfuse, with a worked cost example, and the other platforms to consider.
- +LangSmith alternatives : Langfuse vs LangSmith on pricing, self-hosting and retention.
- +The best LLM evaluation tools : frameworks and platforms compared by the job they do.
- +How many eval cases do you need : why a three-point move between two experiments is often noise.