Writing on analytics, evidence and attribution

The arguments

The dashboard was never the product

Every analytics tool ships a screen full of charts and calls it the deliverable. The actual deliverable is a sentence: here is what changed and here is what it was worth. Charts are how you check the sentence, which makes them evidence rather than output.

Never let a language model do arithmetic

Ask a model to compute a conversion rate from raw events and it will produce something plausible and occasionally wrong, with identical confidence either way. The fix is architectural: deterministic metrics underneath, explanation on top, and no path from the model to the store that skips the definitions.

Most of your traffic report is not customers

Search crawlers, AI crawlers, scrapers, uptime monitors, vulnerability scanners and outright attacks. A tool that reports the total as traffic is not being simple, it is being wrong — and the error flatters you exactly when you can least afford it.

An AI crawler and an AI referral are opposites

One is a bot reading your documentation. The other is a person arriving because an assistant recommended you, usually already mid-decision and converting well above average. Filing both under 'referral' loses the second inside the first.

Attribution is a model, and pretending otherwise is the problem

Last touch, linear and position-based produce three different answers from identical data. None is the truth; each is a rule. The failure is not picking one — it is presenting the output as a discovered fact rather than as the consequence of a choice somebody made.

Your conversion number is wrong until refunds are in it

Almost every analytics setup records the sale and never hears about the reversal. Revenue stays permanently overstated, worst in exactly the channels that convert impulsively, and the team optimises toward the traffic that returns the goods.

Analytics should not be able to slow your site down

Collection that sits on the request path turns a pipeline problem into a visitor problem. Writing the record aside from the request — durable spool, drained afterwards — makes an outage boring, which is the only acceptable design near production traffic.

Simple analytics and complex analytics fail the same way

One hands you 42 reports and leaves you to find the right one. The other hands you a number and no way to ask why it moved. Both skip the same step: somebody looking at the data and saying what happened.