For years, the observability stack has been built on a familiar, seemingly stable foundation: logs, metrics, and maybe a dashboard or two. Developers are told that if they instrument their code correctly, aggregate enough data points, and set the right alerts, they will achieve “observability”—the holy grail of understanding their system’s internal state from its external outputs. But a growing, frustrated chorus of engineers is calling this a lie. The tools we’ve been sold are failing us. They generate oceans of data but deliver teaspoons of insight. When a critical incident occurs at 2 AM, developers are left drowning in a sea of disconnected logs and meaningless metric graphs, trying to perform forensic archaeology instead of engineering. It’s time to confront the uncomfortable truth: traditional logs and metrics are not enough.
The Broken Promise: From Data to Understanding
The core promise of observability is to allow you to ask arbitrary, novel questions about your system without having to ship new code. Logs and metrics, in their traditional form, spectacularly fail this test. They represent a pre-computed, predetermined view of the world.
The Tyranny of Pre-Aggregated Metrics
Metrics are aggregates. They are wonderful for tracking known-knowns: request rate, error count, 95th percentile latency. You must decide what to measure before you need to measure it. This is their fatal flaw. When a novel failure mode emerges—a specific combination of user attributes, service version, and database shard causing a slowdown—your pre-aggregated metrics are blind to it. You cannot retroactively query your metrics store to analyze latency by a specific user ID or a particular cache key that wasn’t part of your original aggregation key. You’re left with a graph showing “latency is up,” but zero ability to drill down into the why.
The Haystack of Unstructured Logs
Logs, on the other hand, contain the raw details but in a form that is notoriously difficult to navigate at scale. The standard practice of grepping through terabytes of text files is a relic of a simpler time. While structured logging and centralized platforms like the ELK stack have improved the situation, the fundamental problem remains: logs show you the “what” but rarely connect the “how.” A single user request might touch a dozen microservices, each generating its own log stream. Correlating these events to reconstruct a single transaction’s path is a manual, painful process of matching IDs across systems. Logs tell you that Service A failed with “Database connection timeout,” but they don’t show you the cascading failure that started with a thread pool exhaustion in Service D three hops upstream.
The Missing Pillar: Traces as a First-Class Citizen
The industry’s answer to this correlation problem has been distributed tracing. Yet, tragically, tracing is often treated as a niche tool for performance tuning, not as the foundational observability pillar it needs to be. A trace is the golden thread that connects logs and metrics to a real user action or business transaction.
- Traces Provide Causality: Unlike a log line, a trace shows parent-child relationships between operations across process and network boundaries. You can see the exact path and timing of a request.
- Traces Enable High-Cardinality Exploration: This is the critical difference. You can filter and group traces by virtually any attribute attached to them—user_id, device_type, deployment_version, specific feature flag. This allows you to ask those novel, unplanned questions: “Show me all traces for users in region EU-West-1 who experienced errors after we rolled out the new payment service.”
- Traces Contextualize Metrics and Logs: A spike in your error-rate metric (a metric) can be clicked to sample the underlying traces. Those traces contain their span IDs, which can be used to instantly pull the relevant, correlated logs from all involved services. Suddenly, the data has a narrative.
The failure is that most organizations implement tracing as an afterthought. It’s bolted onto a few “critical paths,” with poor instrumentation and low sampling rates that discard the very data needed during an incident. If your tracing solution isn’t capturing nearly 100% of traffic for key services and is not the primary lens through which you debug, you are missing the point.
The Tooling Disconnect: Dashboards Are Not Answers
Modern observability platforms are often dashboard factories. Teams spend weeks crafting beautiful, real-time Grafana boards filled with graphs. These dashboards are excellent for monitoring the health of known, specific components. They are terrible for diagnosis.
A dashboard tells you which system is sick. It rarely tells you why. It pushes the burden of synthesis onto the on-call engineer, who must now mentally map a dozen red graphs to a hypothetical failure in the system architecture. This is not a tool problem; it’s a philosophy problem. The tooling is built for monitoring (watching known thresholds), not for observability (exploring the unknown).
The Alert Storm Anti-Pattern
This disconnect culminates in the alert storm. A single root cause—like a network partition—triggers hundreds of downstream alerts: database latency high, cache miss rate up, service health-check failing, error logs spiking. The engineer’s phone becomes a useless brick of notifications, all symptoms of one cause. The tools have faithfully reported every measurable consequence but have provided zero help in identifying the single, underlying fault. They add noise, not signal.
What Developers Actually Need: The Three Shifts
For observability to stop failing developers, a fundamental shift in tooling and practice is required.
1. Shift from Monitoring Systems to Debugging Workflows
Tools must be built for the investigative workflow of an engineer under fire. This means:
- Start with a Question: The interface should be a query bar, not a dashboard. “Why are checkout requests failing?”
- Prioritize Traces: The primary result of that query should be a set of exemplar traces showing both successful and failed checkouts, with the ability to diff them.
- Unify the Data Silo: Clicking on a failed span should seamlessly reveal the logs from that specific service for that specific request, and the relevant metrics for that service’s underlying host at that precise moment.
2. Shift from Sampling to Intelligent Fidelity
The old argument for sampling traces (due to cost and volume) is collapsing. With intelligent, tail-based sampling, you can capture 100% of traffic for all error paths and anomalous latency, while sampling down the boring, successful requests. You cannot debug what you do not record. The tooling must make this both affordable and automatic.
3. Shift from Static Alerts to Dynamic Baselines
Alerts need to move beyond static thresholds. Tooling should leverage the high-cardinality context in traces to alert on changes in behavior. For example: “Alert me if the latency for users on the ‘premium’ tier degrades relative to other tiers,” or “Alert me if the error pattern for a new deployment version is statistically different from the old one.” This turns observability data into a proactive detection engine.
Conclusion: Demand Better Tools
The status quo of stitching together logs, metrics, and a sidecar of tracing is a burden placed on developers by inadequate tooling. We are data-rich and insight-poor. True observability is not about collecting more data; it’s about having the ability to navigate that data to find answers to questions you didn’t know you were going to ask.
Developers must stop accepting tools that simply warehouse their logs and plot their metrics. Demand tools that are built for exploration, not just reporting. Insist that distributed tracing is not a premium feature but the core data model. Choose platforms that let you start with a question about user experience and follow a breadcrumb trail through traces, logs, and code—without switching contexts.
Logs tell you an event occurred. Metrics tell you it’s part of a trend. Only traces connected to context-rich, explorable data can tell you why. Until our tools embrace this reality, they will continue to fail us, leaving developers to perform the hardest work of synthesis with the least helpful tools. The era of the dashboard is over. The era of the debugger must begin.
