How to track citations.
This is a how-to for building a minimum-viable AI citation tracker yourself. It's meant for engineers who want to understand the shape of the problem before deciding whether to build or buy. AIRank does all of this and more, but the DIY version is a useful reference.
The short answer
Tracking AI citations yourself means issuing a frozen panel of 20–50 buyer queries against four platforms in clean sessions on a fixed schedule, parsing both the sources panel and 15-token prose spans, matching them to your pages by URL and by embedding similarity around 0.88 cosine, then deduplicating regenerations before you count anything.
Before you start
This is an engineering project, not a configuration task. You need billing-enabled API access on four platforms with web search or grounding available on each tier you are paying for, an embedding model and somewhere to keep the vectors, a store that handles time series without pain, and a scheduler you trust to run unattended. You also need a canonical list of your own URLs — a sitemap will do — and a maintained list of competitor domains.
- Four API keys with browsing or grounding enabled, and headroom on the rate limits for 50 queries every six hours.
- A pre-computed embedding index of your own pages, rebuilt whenever you publish.
- Durable storage for raw responses — your parser will be wrong and re-parsing history is the only fix.
- A frozen query panel written before any site change, so a baseline exists.
A weekend gets you to a working v1 if you have done retrieval work before. After that, expect two to four hours a week of maintenance as API contracts drift, tool-use response shapes change, and competitors move domains. That ongoing cost is the part people underestimate, because it never ends and it never gets a sprint allocated to it. Decide up front who owns it when the person who built it is on holiday.
Why this works
What you are building is not a rank tracker, and trying to make it one is the fastest route to nonsense. There is no ranking to measure. Two engines asked the same question return different answers, and the same engine asked twice returns different answers. The only stable quantity is your own site's rate of appearance across a fixed panel over a fixed window, which is why the panel has to be frozen before you change anything.
The clean-session rule and the embedding threshold exist for the same underlying reason: you are trying to measure the model's baseline prior rather than an artefact of how you asked. Memory turns the measurement into a personalization result. Pure URL matching turns it into a link count, and most real citations are paraphrases with no link at all — which is why a similarity threshold around 0.88 does more work in this pipeline than any other single number.
You are not measuring where you rank. You are measuring how often a non-deterministic system reaches for you, which only becomes a number after you average it.
The steps
- 01
Build a query panel
Pick 20–50 buyer questions your ideal customers might ask an AI. These are your probes, and their quality is the ceiling on everything downstream. Bad queries: your brand name, which always returns you and therefore carries no signal; and pure technical lookups, which get routed to the docs site of whoever owns the underlying technology. Good queries have contested answers — 'what's the best X for Y under $Z', 'how do I do Z', 'is A a good alternative to B'. Freeze the list before you change anything on your site.
Check
You'll know it worked when no query in the panel contains your brand name and at least half of them return an answer naming some competitor — a contested answer is what makes a probe informative.
- 02
Call the APIs in a clean session
OpenAI with browsing enabled via a web-search tool, Anthropic with Claude's web search tool, Perplexity's Sonar API, and Google's Gemini with grounding. Run each query at temperature 0 with no memory, no system persona and no conversation history — one query, one response, one session. Session reuse is the single most common way a DIY tracker produces numbers that look great and mean nothing, because an earlier answer in the thread biases every later one. Record the full raw response body, not just your parse of it.
Check
You'll know it worked when the same query issued twice in a row returns two different responses with only partially overlapping source lists — that non-determinism is what you will be averaging over.
- 03
Parse the response
Each response has prose plus structured citation blocks, and the two need different handling. Extract URLs from the citation blocks and sources panel. Extract candidate snippets of 15 or more tokens from the prose, since anything shorter is too common a phrase to attribute safely. Store both, plus the untouched raw response, because your parser will be wrong in ways you cannot anticipate and the only recovery is to re-parse history. Normalise URLs at this stage — strip tracking parameters, resolve redirects, canonicalise trailing slashes.
Check
You'll know it worked when your parser recovers both the sources-panel URLs and at least one 15-token verbatim span from a response you can eyeball against the stored raw log.
- 04
Match against your site
For each URL, check whether it falls in your site's URL space, remembering that subdomains, help centres and CDN-hosted assets may or may not count as you. For each snippet, compute a sentence embedding and compare it against a pre-computed index of your pages, using roughly 0.88 cosine similarity as the paraphrase threshold. Below that you collect coincidences; far above it you only catch verbatim copies and miss most real citations, which are paraphrases. Keep the score on every match so you can re-tune the threshold later without re-running anything.
Check
You'll know it worked when a paragraph you deliberately paraphrased from your own docs scores above 0.88 while an unrelated paragraph from a competitor's site scores well below it.
- 05
Schedule it
Run every six hours and store results in something time-series shaped, with one row per query per platform per run and the raw response retained separately. Variance is high — a 10–15% week-over-week swing on any single query is normal — so never alert on a single-day change. Smooth over a 7-day trailing window for anything a human looks at, and keep the raw per-query series available underneath for debugging. Rate limits and transient tool failures are routine, so record run failures explicitly rather than letting them look like zero citations.
Check
You'll know it worked when your time-series table holds four runs per query per day and the 7-day trailing average is visibly flatter than the raw series plotted beside it.
- 06
Build the deduper and competitor tracker
This is the hardest part and where most DIY implementations fall over. Near-duplicate paragraphs from model regenerations need collapsing into one citation event, or your counts inflate every time a platform retries. Competitor citations need detecting against a maintained competitor URL list run through the same matcher, since 'who else got cited on this query' is most of the actionable signal. Both need to survive competitors changing domains, acquiring each other, and publishing on Medium. Budget more time here than for everything above combined.
Check
You'll know it worked when two near-identical regenerations of the same answer collapse into a single citation event in your table instead of two.
How to verify it worked
Verify the instrument before you trust anything it says about your site. Plant a control: a page you know is indexed and a query you are confident it answers. If the pipeline never catches that citation, your matcher is broken, not your content — and until you have proved otherwise, every zero in your dashboard is ambiguous between 'no citation' and 'missed citation'.
Then check the tracker against a human. Take ten responses from one run, read them yourself, and mark the citations by hand. Compare against what the pipeline recorded, and treat any disagreement as a parser bug until proven otherwise. The floor of what your instrument can resolve is the platform's own noise — around 10–15% week-over-week on a single query — so anything smaller than that is not a finding.
Failure recording
Log run failures explicitly. A rate-limited request that silently records zero citations is indistinguishable from a real decline, and it will cost you a week.
Three ways this goes wrong
- The session contamination bug. Symptom: citation rates are implausibly high and climb the longer a run goes on. Cause: you reused a conversation, a thread id, or a cached assistant state across queries, so answers from earlier probes are priming later ones and your own domain keeps getting re-cited. Fix: issue every query in a genuinely new session with no history and no persona, and add an assertion that the request payload contains exactly one user message before it goes out.
- The regeneration double-count. Symptom: your daily counts spike on days when a platform was flaky. Cause: retried or regenerated responses produce near-identical paragraphs, and without deduplication each one lands as its own citation event. Fix: fingerprint responses on normalised text and collapse anything above a high similarity threshold within the same query and run window. Count distinct citation events per query per run, never raw matched rows, and keep both numbers so you can see the gap.
- The domain-match blind spot. Symptom: your tracker reports far fewer citations than a manual read of the same responses finds. Cause: you are matching on URLs only, so every paraphrase without a link is invisible, and your help centre or docs subdomain may not even be in your URL space. Fix: run the embedding matcher on prose spans as well as links, and explicitly enumerate every hostname that counts as you — apex, www, docs, help, and any CDN path.
What to expect
A DIY tracker will cost you roughly $80–150/month in API calls for a single site, assuming 50 queries × 4 platforms × 4 runs/day. Maintenance runs 2–4 hours a week as API contracts drift. Our honest take: build it if you want to understand the shape; otherwise pay us $49/month and point the engineering time at something that compounds.
Can I just use the ChatGPT web UI instead of the API?
Not for a tracker. The web UI carries memory, personalization and account history, so what you measure is what the model thinks about you given everything you have ever asked it. It is also not scriptable at the volume you need. Use the UI for spot-checks and eyeballing responses, and the API for anything you intend to plot or compare across weeks.
What temperature should I run the queries at?
Zero, though it will not make responses deterministic. Retrieval layers re-rank between runs, indexes refresh, and grounding sources rotate, so identical requests still diverge. Temperature 0 removes one source of variance rather than all of them. The real defence against noise is sample count and a trailing average, not sampling parameters.
How many queries do I need before the numbers mean anything?
Twenty is the floor, fifty is comfortable. Below twenty, a single query flipping moves your headline rate by five percentage points and you will chase it. The panel also has to span intents — comparison, how-to, and alternative-to phrasings — because a panel of twenty variations on one question measures one page rather than your site.
Do I need embeddings, or is string matching enough?
You need embeddings. Most citations are paraphrases: the model restates your claim in its own words with no link, and exact string matching finds none of those. Verbatim spans of 15 or more tokens are worth catching separately because they are unambiguous, but they are the minority. Around 0.88 cosine similarity is a reasonable starting threshold for paraphrase detection.
What does this actually cost to run?
The published estimate on this page is roughly $80–150 a month in API calls for one site at fifty queries across four platforms four times daily, plus two to four hours a week of maintenance. The API bill is the predictable part. The engineering time is not, and it is the reason most teams who build this end up buying something instead.
Written by
The AIRank Editorial Team
Research & editorial, AIRank
The AIRank editorial team runs the 47-point scanner, the Observer pings, and the GEO research programme every week. Writing is reviewed by the core engineers who build the Injector, Blaster, and Surgeon agents.
Last reviewed
About the team →