Product / UX / Telemetry & observability

AgentOps

Python SDK for AI agent monitoring, LLM cost tracking, and benchmarking. Purpose-built for the agent era, tracks multi-step agent sessions, tool calls, and costs across frameworks (CrewAI, Agno, OpenAI Agents SDK, LangChain, Autogen). Lightweight instrumentation with two-line setup. 5.6K GitHub stars.

agentops v0.4.21 (last release 29 Aug 2025, PyPI), no 2026 release at scoring time (>9 months stale). SDK MIT-licensed; primary product is the app.agentops.ai SaaS dashboard.

Openness

4 medium confidence

4.0

license: MIT(OSI, SDK)
source: public(SDK + self-hostable app dir)
commercial: app.agentops.ai-SaaS-dashboard-is-the-primary-product

SDK is MIT/OSI and self-host is possible, but the hosted dashboard is the product surface, closer to open_core than pure open_source. Scored 4.

https://github.com/AgentOps-AI/agentops recorded 2026-06-04
MIT-licensed SDK; complementary SaaS dashboard at app.agentops.ai; self-host option

Adoption

3 high confidence

3.0

agentops ~585k PyPI downloads/month; multi-framework integrations (CrewAI, AG2, OpenAI Agents SDK, LangChain, Camel); ~5.6k stars corroborates only. 100K-1M monthly-download band.

https://pypistats.org/packages/agentops recorded 2026-06-04
~584,603 downloads last month
https://github.com/AgentOps-AI/agentops recorded 2026-06-04
~5.6k stars; agent tracing + cost tracking + integrations

Capability

3 medium confidence

3.0

Solid agent-tracing + cost + basic evals, but narrower feature matrix than MLflow/Langfuse/Opik/Phoenix; stale SDK (no 2026 release) reinforces a mid score.

https://github.com/AgentOps-AI/agentops recorded 2026-06-04
tracing, session replays, cost tracking, eval metrics, integrations

Unchanged since 2026-06-24 (last edited, not re-checked)