Langfuse

Building LLM apps and multi-step agents without observability is pure guesswork. Langfuse is an open-source engineering platform built to trace, evaluate, and optimize AI workflows from early prototypes straight into high-load production.

Created by Langfuse GmbH (a German team now under ClickHouse), the platform unpacks non-deterministic agent workflows. Instead of treating model calls like black boxes, it logs every prompt, tool execution, retrieval step, and completion into clean hierarchical trace trees.

Core Capabilities

• Granular tracing & session tracking: Maps multi-turn chats and agent loops into tidy sessions and user profiles. You can visualize the full agent graph, see exactly where the logic derailed, and spot latency bottlenecks alongside token costs for every single step.
• Multi-method evaluations: Run automated LLM-as-a-judge scoring, set up deterministic Python or TypeScript code evaluators, and queue human annotations to build golden datasets and catch hallucinations before users do.
• Decoupled prompt management: Create, version, and tag prompts in the UI or via API without touching codebase logic. Client-side and edge caching keep fetch latency close to zero.
• Offline experiments & CI/CD guardrails: Compare prompts and models side-by-side against test datasets, catch regressions early, and block deployments if benchmark scores dip below your predefined thresholds.
• Developer and agent ergonomics: Packs native SDKs for Python and JavaScript, standard OpenTelemetry support across languages, a command-line interface, and an MCP (Model Context Protocol) server so coding assistants in Cursor or VS Code can dig into traces directly.

Best of all, unlike proprietary tools locking telemetry behind closed doors, Langfuse's core engine is MIT-licensed. Under the hood, it ingests massive data streams asynchronously via Redis and object storage, storing structured traces in ClickHouse to query millions of events in milliseconds. You can also hook up gateways like eurouter.ai to pipe inference traces straight into Langfuse with zero fuss.