Patronus Evaluation Platform

platform

closed(hosted SaaS

source

partial(patronus-py is a client wiring to the hosted API, not the engine)

client-SDKs

open(patronus-py public on GitHub, v0.1.25, but only 7 stars and wires to the hosted API)

The open-source SDK is a thin client to a closed hosted evaluation backend: the scoring engine and the managed Lynx and Glider evaluators are proprietary, not self-hostable, and the SDK repo carries no license text. Rather than an open core withholding a paid tier, this is an open periphery around a runtime that never ships, keeping the source partial rather than closed.

• https://api.github.com/repos/patronus-ai/patronus-py recorded 2026-08-13

  stargazers_count = 8, pushed_at 2026-01-09, and the repo's `license` field is null - GitHub detects no license file on the SDK at all.

• https://docs.patronus.ai/docs recorded 2026-06-04

  hosted platform: built-in Lynx/Glider evaluators, experiments, production monitoring, SDK access to cloud service

• https://docs.patronus.ai/docs recorded 2026-08-13

  An end-to-end hosted system for evaluating, monitoring and improving performance, with Lynx and Glider as in-house evaluators, experiments, real-time production monitoring and Python/TypeScript SDKs into the cloud service. No self-hostable or open-source evaluation engine is offered anywhere in it.

• https://github.com/patronus-ai/patronus-py recorded 2026-06-04

  Python SDK whose remote evaluators 'run remotely on Patronus infrastructure'; v0.1.25 Jan 2026; 7 stars

• https://raw.githubusercontent.com/patronus-ai/patronus-py/main/README.md recorded 2026-08-13

  SDK README's own example carries the comment '# This evaluator runs remotely on Patronus infrastructure.' - the published code wires to the hosted engine rather than being it.

The homepage and pricing page publish no user, customer or usage-volume count, and the SDK's GitHub stars belong to a client library that wires to the hosted service rather than the platform itself, so no level is assigned rather than one inferred from marketing.

• https://api.github.com/repos/patronus-ai/patronus-py recorded 2026-08-13

  stargazers_count = 8 on the public SDK, which is not an honest usage figure for the hosted platform

• https://docs.patronus.ai/docs recorded 2026-08-13

  No customer count, no usage volume and no seat count published. Three named case studies (Nova AI, Etsy, Weaviate) carry no quantitative adoption data.

• https://github.com/patronus-ai/patronus-py recorded 2026-06-04

  7 stars on the public SDK (no honest usage figure for the hosted platform)

• https://www.patronus.ai recorded 2026-09-29

  The homepage lists "Handling complex customer support cases end-to-end" among its task areas and one count, "5k+ expert contributors", which counts data labelers rather than users; searched the body for downloads, million and trusted by: none, so it publishes no user, customer or usage figure.

• https://www.patronus.ai/pricing recorded 2026-09-29

  The pricing page compares plans with rows such as "Customizable options to meet your specific needs" and "Number of Users"; searched the body for downloads, million and trusted by: none, so it publishes no customer or usage figure.

A solid, broad commercial eval feature set - evaluators, experiments, monitoring and agent evaluation - but not frontier-defining within the category, and the closed backend limits reproducibility. Mid-tier on features overall.

• https://docs.patronus.ai/docs recorded 2026-06-04

  Lynx/Glider evaluators, RAG/agent/NLP/OWASP metrics, experiments, red-teaming, monitoring

• https://docs.patronus.ai/docs recorded 2026-08-13

  Lynx and Glider in-house evaluators plus custom ones, experiments for A/B testing, real-time production monitoring through tracing and alerts, and Python/TypeScript SDKs. Agent evaluation and Percival present. Nothing independently reproducible, the engine being closed.