Cover art for Why Your UX Metrics Are Lying to You

Published: April 17th

Why Your UX Metrics Are Lying to You

The numbers on your dashboard look great. Your users are struggling. Here's why.

UX Metrics = numbers that tell you how your product is performing.

The problem: most teams track numbers that feel important but don't tell you if users are actually getting things done.

Easy metrics go up: clicks, time on site, and daily users. Everyone nods. The product quietly gets worse.

The rule: a metric that is easy to move is rarely the one that matters.

The 7 Metrics Lying to You

For each one: what it is in plain English, why it lies, and what to track instead.

1. Time on Site

What it is: How long users spend in your app.

The lie: More time could mean users love it, or they're completely lost and can't find anything.

Track instead: Task Completion Time vs. Benchmark, how long does it take to finish your top 5 tasks compared to a target? Falling time = improving design.

2. Page Views

What it is: How many screens users visit per session.

The lie: More pages could mean engaged exploring, or broken navigation, forcing users to wander.

Track instead: Navigation Efficiency Score, pages viewed divided by tasks completed. Score of 1 = perfect. Score of 8 = lost.

3. Daily Active Users (DAU)

What it is: How many people opened the app today?

The lie: A user who opens the app, sees nothing useful, and closes it still counts as a DAU.

Track instead: Active Value Days, days where the user actually completed something meaningful (sent a doc, finished a workflow, hit a goal).

4. Bounce Rate

What it is: Users who leave after viewing just one page.

The lie: A user who found the phone number they needed in 10 seconds and left = 100% bounce rate and a perfect session.

Track instead: Intent-Completion Rate by Entry Point, did users achieve what they came for, regardless of how many pages they visited?

5. NPS (Net Promoter Score)

What it is: "Would you recommend us?" on a scale of 0–10.

The lie: It tells you how people felt when you asked. Not what you should design differently.

Track instead: Customer Effort Score (CES) at task completion, one question: "How easy was that?" right after they finish something.

6. Feature Adoption Rate

What it is: How many users tried a feature.

The lie: A feature with 80% adoption and 80% abandonment after one use is a failure — not a win.

Track instead: Feature Retention at 30 Days, of users who tried it, how many came back? High retention = real value. High adoption + low retention = novelty.

7. Error Rate (Standalone)

What it is: How often errors occur.

The lie: A low error rate can mean great design, or that the product is so restrictive users can't do anything that might fail.

Track instead: Error Recovery Rate, when errors happen, do users successfully recover and complete their task? That's the real quality signal.

The Metrics That Actually Matter

Three tiers:

  • Tier 1 — Outcome Metrics: Did users achieve their goal? (Task success rate, goal completion, confidence)
  • Tier 2 — Experience Metrics: How did it feel? (Customer Effort Score, SUS score, error recovery)
  • Tier 3 — Signal Metrics: Where should we investigate? (Navigation efficiency, 30-day retention, rage clicks)

Most teams only have Tier 3. They see signals but never connect them to outcomes.

The Replacement Map

ReplaceTrack instead
Time on siteTask completion time vs. benchmark
Page viewsNavigation efficiency score
DAUActive Value Days
Bounce rateIntent-completion rate by entry point
NPSCustomer Effort Score at task completion
Feature adoptionFeature retention at 30 days
Error rateError recovery rate

AI Products Need New Metrics

Old metrics were built for predictable apps. AI is unpredictable.

New metrics to track:

  • Edit Distance: How much did users change the AI's output? Low edits = AI understood context.
  • Delegation Rate: Are users trusting the AI with more over time? Rising = growing trust.
  • Intervention Frequency: How often do users stop or correct the AI? High = trust problem.
  • Recovery Success Rate: When the AI fails, do users still complete their goal?

The 4-Question Audit

Before keeping any metric on your dashboard, ask:

  1. If this number went up 20%, would we make a different design decision? (If no — it's decorative.)
  2. Does this measure what users achieved, or just what they did?
  3. Can we tell if changes were caused by design or by outside factors?
  4. Does this align with what users told us they actually care about?

If a metric fails questions 1 or 2, remove it from your primary dashboard.

Get the PDF