Report · agent-observability · updated 2026-09-24
Agent observability and guardrails
Platforms that record agent traces, run evaluations, or enforce runtime guardrails. Included only with public, dated sources. Helicone is included because its acquisition and maintenance-mode statement are themselves evidence of a shipped platform — not a prediction of future investment.
Placements
10 of 10 entities have a Jev placement. Axes: Proven Execution × Validated Direction.
| Entity | Status | Proven Execution | Validated Direction | uX | uY | Region | P(region) |
|---|---|---|---|---|---|---|---|
| LangSmith | scored | 0.547 | 0.585 | 0.346 | 0.324 | OPERATIONAL | OPERATIONAL:0.76 DIRECTED:0.07 FORMING:0.02 ANCHORED:0.15 |
| Langfuse | scored | 0.535 | 0.535 | 0.327 | 0.301 | OPERATIONAL | OPERATIONAL:0.82 ANCHORED:0.12 FORMING:0.02 DIRECTED:0.04 |
| Arize Phoenix | scored | 0.375 | 0.427 | 0.408 | 0.362 | OPERATIONAL | ANCHORED:0.02 OPERATIONAL:0.62 FORMING:0.28 DIRECTED:0.08 |
| Braintrust | scored | 0.635 | 0.443 | 0.340 | 0.351 | OPERATIONAL | OPERATIONAL:0.39 ANCHORED:0.34 FORMING:0.02 DIRECTED:0.25 |
| Helicone | scored | 0.728 | 0.282 | 0.357 | 0.368 | OPERATIONAL | FORMING:0.04 ANCHORED:0.15 OPERATIONAL:0.78 DIRECTED:0.03 |
| Galileo | scored | 0.362 | 0.292 | 0.376 | 0.384 | OPERATIONAL | FORMING:0.20 OPERATIONAL:0.52 DIRECTED:0.24 ANCHORED:0.04 |
| Patronus AI | scored | 0.302 | 0.275 | 0.356 | 0.365 | FORMING | DIRECTED:0.32 ANCHORED:0.01 FORMING:0.48 OPERATIONAL:0.19 |
| AgentOps | scored | 0.185 | 0.180 | 0.331 | 0.332 | FORMING | DIRECTED:0.04 FORMING:0.66 ANCHORED:0.00 OPERATIONAL:0.30 |
| Lakera | scored | 0.190 | 0.193 | 0.333 | 0.330 | FORMING | DIRECTED:0.09 OPERATIONAL:0.18 ANCHORED:0.00 FORMING:0.73 |
| Guardrails AI | scored | 0.235 | 0.188 | 0.348 | 0.328 | FORMING | OPERATIONAL:0.20 ANCHORED:0.00 FORMING:0.73 DIRECTED:0.07 |
Entities
- LangSmith
LangChain’s tracing, evaluation, and agent-operations platform.
3 evidence · scored · OPERATIONAL
- Langfuse
Open-source LLM and agent observability with cloud and self-host options.
3 evidence · scored · OPERATIONAL
- Arize Phoenix
Open-source AI observability with Phoenix Intelligence (PXI) agent.
2 evidence · scored · OPERATIONAL
- Braintrust
Evaluation-first observability and CI quality gates for production AI.
2 evidence · scored · OPERATIONAL
- Helicone
Open-source LLM gateway and observability platform; acquired by Mintlify.
2 evidence · scored · OPERATIONAL
- Galileo
Evaluation, observability, and runtime protection for GenAI applications.
1 evidence · scored · OPERATIONAL
- Patronus AI
Evaluator models and domain-specific testing for LLM applications.
1 evidence · scored · FORMING
- AgentOps
Session-level observability aimed at agent runs.
1 evidence · scored · FORMING
- Lakera
API-first AI security and prompt-injection guardrails.
1 evidence · scored · FORMING
- Guardrails AI
Open-source and commercial validators for LLM outputs.
1 evidence · scored · FORMING
Payment does not affect scores. Submit evidence or read the rubric.