How to llms.txt.
llms.txt is a single Markdown file at your domain root that tells AI crawlers what your site is about and which pages matter. It's a 2024 proposal from Jeremy Howard / Answer.AI that's now honored by Claude, Perplexity, and Bing's grounding layer.
- 01
Create the file
Make a file called `llms.txt`. It lives at your site root — `https://yoursite.com/llms.txt`. It's plain Markdown. Start with an H1 that's your site name.
- 02
Write the one-sentence description
Directly under the H1, write a Markdown blockquote with one sentence explaining what the site is. Example: `> AIRRNK tracks where ChatGPT, Claude, and Perplexity cite your brand, and ships automatic fixes.` This is the highest-signal line in the file.
- 03
Add a context paragraph (optional but recommended)
One paragraph of prose explaining the product or site. No marketing language — this isn't for humans, it's for retrieval. Aim for 30–80 words.
- 04
List your primary URLs
Under an H2 heading (e.g. `## Docs`), list the URLs that best explain the product, one per line as `[Title](https://...) : one-line description`. Aim for 10–30 URLs total across sections. Group them: Docs, Examples, Optional.
- 05
Deploy and reference it in robots.txt
Put the file at your root so it's served at `/llms.txt`. Add a line to your `robots.txt`: `LLMs: /llms.txt`. This helps crawlers discover it without guessing.
- 06
Consider /llms-full.txt
For API docs and reference pages, add a companion `/llms-full.txt` with the full content of each URL inlined. This is expensive (often 100KB+) but is the format Claude's retrieval prefers — and our data shows 40–60% higher citation rates on sites that ship it.
What to expect
llms.txt shows up in crawler access patterns within about 48 hours of deployment. The citation impact lags by 1–2 weeks as the retrieval signals are re-indexed. The largest wins are on technical/documentation queries — marketing-page impact is smaller.
What is How to set up llms.txt in the context of AI SEO?
How to set up llms.txt describes one piece of the larger Generative Engine Optimization (GEO) problem — measuring and fixing how ChatGPT, Claude, Perplexity, and Gemini talk about a business. GEO differs from classical SEO because LLM answers do not return a list of links; they return a paraphrase, and the signals that get you inside that paraphrase are different.
How does AIRank measure how to set up llms.txt?
AIRank's Observer agent queries ChatGPT, Claude, Perplexity, and Gemini daily with the prompts your customers actually use and logs every mention. The Scanner agent then walks your site the way an LLM does — 47 signals across headings, schema, entity mesh, and source trust — and flags the specific gaps driving the result.
Why does how to set up llms.txt matter for AI visibility?
Roughly 42% of B2B buyer research now starts inside an LLM (Forrester 2026). Pages that do not satisfy the GEO signal set get paraphrased without attribution or omitted from answers entirely — a situation Aggarwal et al. (Princeton, 2023) measured as a 30-40% citation gap against pages that do.
What is the fastest way to improve how to set up llms.txt?
Start by running a free AIRank scan to surface the three highest-leverage fixes for your domain, then ship them through the Injector agent in a single click. Most teams see their first fix land within 12 minutes of install; citation lift typically shows up in weeks two and three once assistants re-crawl the edge-rewritten HTML.
Written by
The AIRank Editorial Team
Research & editorial, AIRank
The AIRank editorial team runs the 47-point scanner, the Observer pings, and the GEO research programme every week. Writing is reviewed by the core engineers who build the Injector, Blaster, and Surgeon agents.
About the team →