What is the difference between llms.txt and robots.txt?

robots.txt tells bots what they may access (allow/deny); llms.txt tells AI models what your site is (summary, key pages, topics). One is the doorman, the other is the brochure.

How do I verify my llms.txt works?

Open yoursite.com/llms.txt in a browser: you should see plain text, not an HTML page. GEONI's free scan also checks llms.txt presence and AI crawler access automatically.

// GEONI Guides

What Is llms.txt and How Do You Add It?

Q: Is llms.txt mandatory?

No — it is a voluntary standard. But the cost is near zero and the potential return is high: you give AI models the chance to summarize your site correctly, and the convention is standardizing quickly.

Last updated: July 2026 · GEONI Team

llms.txt is a plain-text (Markdown) file that lives at your website's root and gives AI models a summary of what your site is, its key pages and its main topics. Where robots.txt tells bots where they may go, llms.txt tells them how to understand you.

Why add one?

AI engines process a site with a limited context budget. llms.txt spends that budget in your favor: it points the model to your own accurate summary and your most important pages. Without it, the model summarizes your site by guesswork — incompletely or incorrectly.

The cost is a single text file; the return is a better chance of being described correctly inside AI answers.

Format: a minimal example

# Brand Name
> One-sentence definition: what you do, for whom.

## Key Pages
- [Home](https://yoursite.com/): short description
- [Products](https://yoursite.com/products): short description
- [About](https://yoursite.com/about): short description

## Main Topics
- A topic you're an authority on
- Another topic you're an authority on

The rules are simple: Markdown syntax, # for the brand name, > for a one-sentence definition, then link lists. For a live example, see geoni.ai/llms.txt.

Adding it in 3 steps

Write the file: an honest summary of your site in the format above. Clarity wins, not hype.
Upload to the root: the file must be reachable at yoursite.com/llms.txt. On WordPress use a file-manager plugin; elsewhere use FTP or your hosting panel — same level as your index file.
Verify: open yoursite.com/llms.txt in a browser — you should see plain text. If you see a redirect or an HTML page, a server/rewrite rule is swallowing the file.

Its companion: allowing AI bots in robots.txt

The bots you want reading your llms.txt must be able to reach your site at all. Allow the major AI crawlers in robots.txt:

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

A critical detail: many sites block AI bots with Disallow without realizing it — then wonder why they never appear in AI answers. Check access first.

Frequently asked questions

Is llms.txt mandatory?

No — a voluntary standard. But the cost is near zero and the return is high: you give the model a chance to summarize you correctly.

How is it different from robots.txt?

robots.txt is the doorman (who may enter); llms.txt is the brochure (what this place is). They complement each other.

How do I verify it works?

Open yoursite.com/llms.txt in a browser: plain text should appear. GEONI's free scan also checks llms.txt presence and AI crawler access automatically.

Is your site open to AI bots?

A GEONI scan checks your robots.txt rules, llms.txt presence and your visibility across AI engines in 60 seconds. We can also generate the files for you.

Scan Free →