Free interactive tool
Match-Rate Drop Calculator: Traffic Mix vs Real Change
When your visitor-identification match rate falls, part of the fall can come from traffic moving toward segments that always match less (more mobile, more paid social, more countries the vendor covers thinly), and part from a segment whose own rate went down. Only the second part points at a broken tag, a consent change or the vendor. Enter sessions and identified sessions for each segment in two periods, and this calculator splits the change exactly into those two parts, segment by segment, and tests each segment’s rate change against sampling noise. It is the arithmetic behind step 1 of the match-rate drop diagnosis.
The formula
share w = segment sessions ÷ all sessions in that period
rate r = segment identified ÷ segment sessions
overall R = Σ w·r (before: R0, after: R1)
mix effect of a segment = (w1 − w0) × ((r0 + r1)/2 − (R0 + R1)/2)
rate effect of a segment = (r1 − r0) × (w0 + w1)/2
Σ mix + Σ rate = R1 − R0 (exact: nothing is left unexplained)
noise check: z = (r1 − r0) ÷ √( r0(1−r0)/n0 + r1(1−r1)/n1 )
|z| < 2 → the rate change is within sampling noiseThis is Evelyn Kitagawa’s decomposition of the difference between two rates (“Components of a Difference Between Two Rates”, Journal of the American Statistical Association 50, no. 272, 1955), which demographers use to separate a change in population make-up from a change in the rate itself. Weighting by the average of the two periods makes the split symmetric and leaves no residual. The per-segment mix effect is measured against the average overall rate, so a segment that grew and matches below average shows a negative number; the mix total is the same either way. A segment with traffic in only one period counts entirely as mix, because it has no rate change to measure.
Run your numbers
One row per segment. The segments must cover all of the traffic in both periods, with the same page scope and filters, so add an “Everything else” row if needed. Counts come from your own analytics property and the vendor’s identified-session export.
| Segment | Sessions before | Identified before | Sessions after | Identified after | Remove |
|---|---|---|---|---|---|
- Overall match rate before
- 31.3%
- Overall match rate after
- 26.3%
- Change
- −4.9 pts
- Explained by traffic mix
- −3.8 pts
- Explained by segment rates moving
- −1.1 pts
| Segment | Share of sessions | Match rate | Mix effect | Rate effect | Is the rate change real? |
|---|---|---|---|---|---|
| Organic search | 50.0% → 37.1% | 35.0% → 35.0% | −0.8 pts | 0.0 pts | Within noise (z = 0.0) |
| Paid social | 12.5% → 33.3% | 15.0% → 15.0% | −2.9 pts | 0.0 pts | Within noise (z = 0.0) |
| Direct | 25.0% → 20.0% | 30.0% → 25.0% | +0.1 pts | −1.1 pts | Yes (z = -5.1) |
| Email and referral | 12.5% → 9.5% | 35.0% → 35.0% | −0.2 pts | 0.0 pts | Within noise (z = 0.0) |
Match-rate change split — visitorops.com/tools/match-rate-drop-decomposer Overall: 31.3% → 26.3% (−4.9 pts) Traffic mix: −3.8 pts · Segment rates: −1.1 pts Segment | share before → after | rate before → after | mix | rate | noise check Organic search | 50.0% → 37.1% | 35.0% → 35.0% | −0.8 pts | 0.0 pts | within noise Paid social | 12.5% → 33.3% | 15.0% → 15.0% | −2.9 pts | 0.0 pts | within noise Direct | 25.0% → 20.0% | 30.0% → 25.0% | +0.1 pts | −1.1 pts | real (z = -5.1) Email and referral | 12.5% → 9.5% | 35.0% → 35.0% | −0.2 pts | 0.0 pts | within noise
The pre-filled rows are a hypothetical example, not a benchmark. Sessions: 16,000 before, 21,000 after. Nothing you enter leaves your browser.
Worked example (hypothetical numbers)
A team’s weekly routine shows the match rate on filtered human sessions falling from 31.3% (5,000 of 16,000 sessions) to 26.3% (5,530 of 21,000), a drop of 4.9 points, in the week a paid-social campaign launched (hypothetical numbers, the tool’s default input). Paid social went from 2,000 to 7,000 sessions at the same 15% rate. Direct traffic went from 30% to 25%. Organic and email held at 35%.
The split: 3.8 points are traffic mix and 1.1 points are a real rate change. Most of the mix effect is paid social (−2.9 points): a segment that grew from 12.5% to 33.3% of sessions and matches at half the average rate. The only real rate change is direct (−1.1 points), and it is far outside noise (z = −5.1 on about 4,000 sessions per period). Nobody needs to call the vendor about the campaign traffic. The team should ask what changed for direct visitors that week (a consent-banner update, a redesign of the pages direct visitors land on) and run the tag QA on those pages.
What should you do with the result?
- Mostly mix, no real rate change. Report the drop as a traffic change, with the segment that caused it. Compare vendors or months on a fixed segment, such as desktop organic, so the headline rate stops moving with the media plan.
- A real change in one segment. Work the diagnosis order on that segment only: consent first, then tag firing on the pages that segment lands on, then the vendor’s changelog. If the fall is in one country or one browser and nothing changed on your site, take the dated numbers to the vendor.
- Real changes everywhere, in the same direction. Suspect something global: the tag, the consent banner, a Content-Security-Policy change, or the vendor.
- Nothing passes the noise check. The segments are too small for a one-week comparison. Use four weeks per period, or fewer, larger segments.
Which segments should you use?
Split by the thing you suspect changed, and keep the list short. Channel (organic, paid, direct, email and referral), device (desktop, mobile), and country or consent region are the three to try first, because identification depends on the network and browser a visit comes from. The split can only see mix between the segments you enter: a shift inside a segment (more mobile visits inside paid social, say) shows up as that segment’s rate change. If a segment shows a real change, run the tool again inside it, with device or country as the new segments.
Use the same denominator in both periods: the same page scope, bot filter and exclusions. The match-rate calculator explains the four denominators. A denominator that changes between periods shows up here as a rate change that is not real.
When does the split not apply?
- Different definitions of “identified”. If the vendor changed what it counts (company-level only, or a new confidence cut-off) between the periods, the split is comparing two different metrics. Check the vendor’s changelog first.
- The noise check is a floor, not a proof. It treats every session as independent. Repeat visits from one company are not, so real noise is larger than the z value suggests. Treat a z between 2 and 3 as “watch next week”, not as a finding.
- A match is not a correct match. The split works on the raw rate. If precision changed too, score a ground-truth sample with the match-quality audit.
- Sessions the tag never saw. If a consent or blocker change stopped the tag from counting some sessions, the analytics property may not count them either. The tag coverage calculator compares tag page views with your server logs to find them.