Studies
Definition
A visitor arrives with a question: are returns free, do you ship to Austria, do you have this in XL. If the site gives no usable answer, that intent has failed. FIR is the share of those questions that fail:
Standard analytics can't see this. GA4 records that a session ended, not that the visitor was looking for a returns policy that was buried in a PDF. Two sessions that look the same in a dashboard can fail for completely different reasons, and each reason has its own fix.
I defined the metric in May 2026 in Failed Intent Rate: the KPI your GA4 dashboard is missing, on the blog of AskVio, the website assistant I build, and presented it publicly for the first time at MeasureCamp Los Angeles on 30 May 2026. That post measures FIR on live traffic. For the 2026 DACH study I needed a version that works from outside, on any site, without an assistant installed. That gives two variants of the same idea.
Two variants: live FIR and audit FIR
| Live FIR | Audit FIR | |
|---|---|---|
| What it measures | How often real visitors asked for something and did not get it | How often a site cannot answer a fixed set of common questions |
| Where questions come from | Real visitors, in their own words, through an on-site assistant | A fixed list, the same for every site (10 in the 2026 DACH study) |
| A question fails when | The answer has low confidence, or a product request matches nothing in the catalog | The public pages hold no specific, usable answer |
| Formula | (low-confidence answers + zero-catalog-match queries) ÷ all queries | questions not answered ÷ questions asked |
| Components | Content FIR and Catalog FIR, tracked separately | One number per site; catalog gaps are out of scope |
| What you need | An assistant on the site that logs questions and answers | Nothing on the site; an outside crawler reads public pages |
| Comparable across sites? | No: each site gets different questions | Yes: same questions, same crawler, same scoring |
| Best for | Finding what to fix first on your own site, week by week | Benchmarks, studies, and checking a site before an assistant goes live |
The two numbers are not interchangeable. Live FIR depends on what a site's own visitors happen to ask, so a 12% live FIR and a 12% audit FIR say different things. When FIR is reported, it should say which variant it is.
They work well together. Audit FIR asks: can a site answer the basics a shopper needs, before any visitor shows up? Live FIR asks: once visitors arrive, what do they actually ask that the site can't answer? A site can score well on the first and still fail on the second, for example because shoppers keep asking for a size or product it doesn't sell.
Where to go next
- The 2026 DACH study: results, shop by shop, and its method: questions, crawler, scoring, validation and limits.
- How do you know an AI judge is right? How the model that scores audit FIR was checked against human review.
- The original FIR post on the AskVio blog. Live FIR, its two components and the three kinds of failed intent.
Disclosure
I build AskVio, which reports live FIR to the shops that use it. Audit FIR needs no AskVio and no assistant of any kind, and a shop's audit result does not depend on using one. The scoring tool, fir-audit, will be open source from 20 November 2026.