Braintrustbraintrust.dev
Evaluation first AI platform for experiments, scoring, prompt playgrounds and production tracing of LLM apps.
- Green flag: Free plan
- Green flag: No card for trial
- Red flag: Credits reset monthly
LLM observability & evals
7 tools · 7 with a free plan · 7 with an API · middle starting price $60 a month
$1$10$100$1,000
Filter
Arize AI (Phoenix / AX)arize.com
AI observability and evaluation: open source Phoenix for local tracing/evals plus hosted Arize AX for production.
- Green flag: Free plan
- Green flag: No card for trial
LangSmithlangchain.com/langsmith
LangChain's platform for tracing, evaluating, and deploying LLM apps and agents, framework agnostic.
- Green flag: Free plan
- Green flag: Doesn't train on your data
Langfuselangfuse.com
Open source LLM engineering platform for tracing, evals, prompt management and cost tracking, cloud or self hosted.
- Green flag: Free plan
- Red flag: Acquired
Heliconehelicone.ai
Open source LLM observability and gateway: proxy based request logging, cost tracking, caching and prompt management.
- Green flag: Free plan
- Green flag: No card for trial
- Red flag: Acquired
Weights & Biases Weavewandb.ai/site/weave
W&B toolkit for tracing, evaluating and iterating on LLM apps and agents, now under CoreWeave Forge.
- Green flag: Free plan
- Red flag: Acquired
Datadog LLM Observability (Agent Observability)datadoghq.com/product/llm-observability
Datadog module tracing LLM apps and agents end to end with quality, safety and cost monitoring.
- Green flag: Free plan
- Red flag: Renamed
$200/mo
Free plan · Flat monthly
Visit site Datadog LLM Observability (Agent Observability) (opens the tool's site)No tools match all of those filters
Remove a filter to see more.
All tools in Agents and Automation: AI agents and automation tools