Disputes and Efficacy

How a verdict is challenged and how accuracy is measured

A score is a prediction, and a prediction needs to be checked against the truth. Two things supply that truth: disputes you file when a verdict is wrong, and the efficacy dashboard that measures every verdict against verified outcomes. This guide covers both, and how a label earns the right to overrule a score.

Filing a dispute

Open any session from the Sessions view and dispute its verdict. A dispute states the verdict you believe is correct — human or bot — a reason, and optional free-text detail up to 2,000 characters. You can dispute up to 50 sessions at once when a whole segment was misjudged.

Dispute reasons

Reason When to use it
known_customer You recognise the visitor as a real customer or contact.
conversion The session led to a genuine sale, sign-up or lead.
internal_traffic Your own team, office or testing produced the session.
vpn_proxy A real person on a VPN or proxy who was scored as suspect.
known_bot You know the session was automated, against a human or suspect score.
suspicious_behaviour You saw behaviour that points to a bot the score missed.
no_real_action The visitor did nothing a genuine prospect would do.
other Anything else. Use the free-text details to explain.

What happens to a dispute

A new dispute starts in the pending state. You can delete a pending dispute you filed to undo it. Resolving it, from the dashboard by someone with access to the site, accepts or rejects it.

  • Accepted. The session is marked as verified ground truth from a complaint source, and it becomes a training label. The label is recorded at 0.8 confidence, below the certainty of a honeypot catch or a hand-verified session, because it is an asserted outcome rather than a first-party trap.
  • Rejected. The dispute is closed and no label is created. The score stands.

The session's original score is never rewritten. The verified label is stored alongside it, so the efficacy dashboard can compare what the engine predicted against what was later confirmed.

A dispute does not leak your visitors' data into a model. When an accepted dispute becomes a training label, the session's feature vector has calibrated Laplace noise added before it is stored, so no single visitor's exact feature values can be read back out of the training set.

The efficacy dashboard

The efficacy dashboard measures scoring against verified outcomes: leads your CRM confirms or rejects, disputes you file, honeypot catches, and sessions verified by hand. It is the difference between asserting accuracy and showing it.

Confusion matrix

Verified sessions cross-tabulated as what they were verified to be (human or bot) against what ClickLens predicted (human, suspect or bot). The diagonal is correct, the off-diagonal is where a score and the truth disagreed.

Precision, recall and F1

Computed for the human class and the bot class. Precision is how often a prediction was right; recall is how much of the verified class was caught; F1 is their harmonic mean. A verified human scored suspect counts against human recall, so the metric does not let a hedge to suspect hide a miss.

Accuracy and coverage

Accuracy is the share of verified sessions that landed on the right verdict. Coverage is the share of all sessions that have a verified label at all, so you can see how much of the picture the metrics rest on.

Score distribution and by-source

Verified humans and bots bucketed across the 0-100 score range, and a breakdown of how many labels came from each source. A healthy detector separates the two populations across the score axis.

Flag analysis

For each detection flag, how often it fired, how often on verified bots versus verified humans, and the resulting precision of that flag. A flag that fires mostly on verified humans is a false-positive source you can see.

A daily job also records a lighter snapshot — human precision and bot recall for the day — so accuracy is tracked over time as well as on demand. And every scoring change is replayed against the bank of verified sessions before it ships: a tweak that would reclassify a known human as a bot is caught and rejected before release, not discovered in your account.

Ground-truth precedence

A session can be verified from four sources, and the source is recorded with the label:

  • honeypot — a first-party trap a real visitor would never trigger, the hardest of the four to fake.
  • manual — a session verified by hand.
  • complaint — an accepted dispute, the path described above, recorded at 0.8 confidence.
  • conversion — a reported conversion treated as positive evidence of a human.

The rule the engine enforces is the one that matters: a reported conversion is the weakest of the four and never overwrites a session that was already verified another way. When a conversion fires on a session that did not independently clear the bar, it is logged for review but mints no human label. So a bot that converts cannot whitewash itself into your ground-truth set, and the efficacy metrics stay anchored to the labels that are harder to fake.