Ragas
Exploding GradientsEvaluation framework built for retrieval-augmented generation pipelines, with more than thirty metrics covering retrieval quality, generation faithfulness and answer relevance, plus automatic test-set generation. It splits pipeline failure into retrieval and generation buckets, which general harnesses do not, and integrates with LangChain and observability tooling.
The project moved from explodinggradients to the vibrantlabsai organization and has not been pushed since February 2026. A hosted tier is sold beside the library. Verified 2026-08-13 via the vibrantlabsai/ragas repository, its license endpoint and its README.
Openness
5 high confidence- license
- Apache-2.0(OSI)
- source
- public(full metrics + test-gen library)
- core-gated
- ungated
Core library is fully OSI (Apache-2.0) with no feature-gated functionality; could be argued open_core given the hosted cloud product, but the eval library itself is complete and open, so scored open_source. The repo the product declares is vibrantlabsai/ragas, the cited explodinggradients path redirecting there after the org rename; the README installs and runs the whole metric suite from PyPI or a git checkout with no account, no hosted tier and no license key mentioned anywhere in it.
- https://github.com/explodinggradients/ragas recorded 2026-06-04
Apache-2.0 license; LLM-app eval metrics + test-set generation; v0.4.3 (Jan 2026); 14.2k stars
- https://api.github.com/repos/vibrantlabsai/ragas/license recorded 2026-08-13
GitHub's license endpoint reports spdx_id Apache-2.0 for vibrantlabsai/ragas and returns the Apache-2.0 text as the LICENSE body.
- https://raw.githubusercontent.com/vibrantlabsai/ragas/main/README.md recorded 2026-08-13
README carries the Apache-2.0 badge and installs with `pip install ragas` or straight from the git repo; the metric suite and test-set generation ship in the package. No pricing page, hosted tier, enterprise edition or license key is mentioned - even the app.ragas.io cloud product goes unnamed in it.
Adoption
4 high confidenceragas draws 1,587,521 PyPI downloads in the trailing 30 days, solidly inside the 1M-10M monthly band, and it is the de facto standard for RAG and LLM-application evaluation, widely integrated with LangChain, LlamaIndex and observability platforms. The repo's 15.3k stars, under its new vibrantlabsai org, corroborate.
- https://pypistats.org/packages/ragas recorded 2026-06-04
1,513,553 downloads in the last month
- https://github.com/explodinggradients/ragas recorded 2026-06-04
14.2k stars; LangChain/observability integrations
- https://pypistats.org/api/packages/ragas/recent recorded 2026-08-13
last_month = 1,587,521 downloads of the ragas package (last_week 335,773, last_day 58,054)
Capability
4 medium confidenceFrontier within the application and RAG evaluation niche on metric breadth and synthetic test-data generation, but not a general academic-benchmark model harness. That is why it scores 4 rather than 5: the coverage is domain-scoped, and the academic-benchmark harnesses such as HELM cover ground it does not.
- https://github.com/explodinggradients/ragas recorded 2026-06-04
objective metrics (LLM-based + traditional), Aspect Critique, automatic test-data generation, integrations
- https://raw.githubusercontent.com/vibrantlabsai/ragas/main/README.md recorded 2026-08-13
README still leads on the same three capabilities - 'Objective Metrics: Evaluate your LLM applications with precision using both LLM-based and traditional metrics', 'Test Data Generation: Automatically create comprehensive test datasets', and data-driven insights - and demonstrates Aspect Critique through `DiscreteMetric`.
Verified 2026-08-13