Does llms.txt actually do anything?

For Google, no: Google says its Search and AI features do not use llms.txt at all. For other AI engines, no major company has publicly said it reads the file when deciding what to cite. Where llms.txt does help today is narrower: AI tools and agents reading documentation. It costs almost nothing, so publish one if you like, but do not expect it to change your visibility.

We build an llms.txt generator, so this is the sceptical companion to our own tool. Here is what the file is, who actually reads it, and when it is worth your time.

What llms.txt is

llms.txt is a proposal published by Jeremy Howard on September 3, 2024, at llmstxt.org. The idea is simple: a website is full of navigation, scripts and cookie banners, and an AI system with limited room to read would do better with one short, curated summary.

The file is plain Markdown at the root of your domain. The format is light:

• An H1 with the site or project name. This is the only required part.
• A short blockquote summarising what the site is.
• Optional notes in plain paragraphs.
• H2 sections of links, each link with a short note on what it covers. A section called “Optional” marks links a reader can skip when space is short.

The proposal also suggests offering clean Markdown versions of pages, at the same URL with .md added. That part matters mostly for documentation sites.

Who actually reads it

Google: no, and it says so

Google’s guide to generative AI features is direct. Under things you can ignore, it lists llms.txt:

“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.”

It adds that it is “completely fine” to keep an llms.txt for other services, and that doing so “will neither harm nor help your site’s visibility or rankings in Google Search.” So for AI Overviews and AI Mode, the question is settled.

OpenAI, Anthropic, Perplexity: no public commitment

None of them has said publicly that its crawler or search product reads llms.txt when choosing what to cite. Their published crawler documentation covers robots.txt, not llms.txt. That may change, but today anyone telling you llms.txt improves your visibility in ChatGPT or Claude is making a claim nobody can support.

Where it genuinely helps

That pattern explains who the file is really for today. It is most useful when an AI tool is pointed at a site on purpose, rather than crawling it:

• Coding assistants and agents reading documentation. A developer asks a tool to use a library, and the tool fetches that library’s docs. A clean llms.txt and Markdown pages are far easier to work with than HTML.
• Someone pasting your URL into an assistant. A short, accurate summary at a predictable address gives a tool a better starting point.
• Smaller tools that have adopted it. Some AI products and agent frameworks look for the file. None of them decide how the big assistants answer.

So the sites that benefit most are developer products with real documentation. For a typical marketing site, the benefit today is small.

What it costs

Very little. A useful llms.txt for most businesses is a few hundred words and a dozen links, and takes minutes to write. The only ongoing cost is keeping it accurate. A file that still lists last year’s products or dead pages is worse than no file, because it is the one place an AI tool is told to trust.

Where it gets expensive is the full version: Markdown copies of every page, or an llms-full.txt with your whole site in it. That is worth doing for documentation. For a marketing site it is a second website to maintain, for a benefit nobody can currently measure.

What a good one looks like

A short example for a software company:

# Acme Analytics

> Acme Analytics is product analytics software for B2B SaaS teams. It tracks feature usage and shows which accounts are likely to churn.

Acme is used mainly by product and customer success teams. Plans start at $49/month; there is a free tier.

## Product

- [Features](https://acme.example/features): what Acme tracks and how
- [Pricing](https://acme.example/pricing): plans, limits and what each includes
- [Integrations](https://acme.example/integrations): Segment, HubSpot, Salesforce

## Guides

- [Getting started](https://acme.example/docs/start): install and first dashboard in 10 minutes
- [Churn prediction](https://acme.example/docs/churn): how the churn score is calculated

## Optional

- [Blog](https://acme.example/blog): product updates and analysis
- [Careers](https://acme.example/careers)

Notice what is not there: no marketing slogans, no list of every page, nothing that goes stale weekly. It says plainly what the company is and points to the pages that answer the questions people actually ask.

So should you publish one?

• Developer product with documentation: yes. This is the use case the file was designed for, and tools already use it.
• Any other site: optional. It takes minutes and cannot hurt you in Google. Just do not count it as AI visibility work.
• If you have limited time: skip it and fix the things that do matter. Make sure AI crawlers are not blocked (see should you block GPTBot?), that your content is in the HTML, and that your pages open with a direct answer.

Our free llms.txt generator builds the file from your site’s real pages, so it cannot invent anything. For the work that actually moves AI visibility, start with the answer engine optimization guide.

The summary

llms.txt is a sensible idea with uneven adoption. Google has said plainly that its Search and AI features do not use it. No other major AI engine has said it reads the file to decide what to cite. It genuinely helps AI tools read documentation, and it costs almost nothing. Publish one if it is easy, keep it short and accurate, and spend your real effort elsewhere.

Questions people ask