Braintrust

source

closed

license

Apache-2.0(OSI, client SDKs only, Python/TS/Go/Ruby/C#)

platform

proprietary(Brainstore custom DB, evals/tracing/automation all SaaS)

self-host

enterprise-only(hybrid)

Open SDKs over a closed SaaS core: the Brainstore backend, the dashboards and the automation layer are all proprietary. There is no open source core, so this is closed rather than open core, the same shape as LangSmith and Galileo. The Apache-2.0 client SDK is instrumentation only and does not open the product; braintrustdata/braintrust-sdk now describes itself as holding Braintrust's JavaScript and TypeScript SDKs and integrations rather than a five-language set, but it remains Apache-2.0 and remains a client SDK, so neither the score nor the class turns on it.

• https://github.com/braintrustdata/braintrust-sdk recorded 2026-08-13

  Repository README - "This repository contains Braintrust's JavaScript/TypeScript SDKs and integrations", with the core SDK described as being "for logging, tracing, evals, and CLI". Its License section reads Apache-2.0. Nothing in the repository is the platform itself.

• https://www.braintrust.dev/ recorded 2026-08-13

  The platform, and the proprietary backend the class rests on - "Brainstore, the database built for AI data at scale ... designed specifically for agent observability so you can query millions of traces quickly", with no source published for it. The only self-host route offered is a hybrid one - "Deploy Brainstore data plane on your own infrastructure" - alongside SOC 2 Type II certification.

The braintrust package draws 7,366,065 PyPI downloads a month, inside the 1M-10M band and the strongest adoption of the closed platforms in this category. Of the users named in the record, Notion and Coursera are on the homepage while Vercel, Dropbox and Replit are not, so the band rests on the download count.

• https://pypistats.org/packages/braintrust recorded 2026-08-13

  Downloads last month 7,366,065 for the braintrust package (last week 1,612,417)

• https://www.braintrust.dev/ recorded 2026-08-13

  Named customer stories on the homepage - "How Coursera builds next-generation learning tools" and "How Notion evaluates AI at scale across 70 engineers". Vercel, Dropbox and Replit no longer appear anywhere on the page.

The deepest evaluation and automation surface in this category: tracing that supports real-time inspection across millions of logs and tool calls; LLM, code and human scorers running experiments against datasets; prompt management and a playground; automation covering pattern discovery, continuous scoring and CI-style quality gates that block releases; datasets and annotation; and OpenTelemetry, MCP and framework-agnostic SDKs. The quality-gate and automation depth goes beyond the MLflow and Langfuse anchors at 4, which is why this is a 5 - a frontier-definer for evaluation-first LLM observability.

• https://www.braintrust.dev/ recorded 2026-08-13

  Homepage feature copy - "Automate pattern discovery with Topics", "Topics surfaces patterns in real time across task, issues, and sentiment, online scoring catches regressions, and quality gates block bad releases", plus "Trace everything - Inspect prompts, responses, and tool calls in real time" and scoring "with LLMs, code, or humans" against versioned datasets.