Google says llms.txt doesn’t affect rankings. Here’s what it actually does, who’s using it well, and how to build one worth your time in 2026.
Sometime around the end of 2024, a proposal for creating a file called llms.txt began circulating in SEO communities. By early 2025, roughly half the Internet was either building one or saying there’s no use building one because that will take a whole Tuesday afternoon. In general, both sides were right. The fact that both sides were right is enough to write about.
A llms.txt file is simply a plain-text Markdown file located at your site’s root directory that describes your site’s content and points large language model systems to your most important pages, similar to how a Table of Contents directs a reader to a Chapter. Importantly, it does not restrict crawler access (robots.txt continues to perform this function). As of mid-2026, none of the major search or AI providers have indicated they currently consider llms.txt in their rankings or citations. However, llms.txt provides much greater value elsewhere. We’ll explain exactly where later.
What is llms.txt Really?
Jeremy Howard’s suggestion to create such a file in September 2024 was straightforward: large language models struggle to explore a large and disorganized website like humans do. So provide those large language models with a guided tour instead. There are generally two types of root files:
- A short /llms.txt list of your site’s main topics and related URLs.
- Optionally, a longer /llms-full.txt file listing all content on your site.
Both the name “llms” and the structure of the file borrow heavily from robots.txt. This was done intentionally. No one would ever even attempt to implement a “robotic.txt.” A shared location where other developers could guess where such a file existed was necessary for widespread adoption of any of these formats.
Is llms.txt Used for Ranking in Google AI Overviews?
Google has stated that llms.txt does not affect search engine rankings, which includes AI Overviews and AI Mode. Google issued an update to their guidance on AI Search on June 15, 2026 stating clearly that llms.txt does not impact Google Search rankings. This is consistent with prior statements from Google’s John Mueller regarding why no major AI service has indicated they utilize the file. Mueller also noted that server logs support his position since the crawlers generally don’t request it.
Traffic data independently supports this statement as well. A GEO firm analyzed more than 500 million visits from various AI bots over a 90 day period and found less than 1% requested /llms.txt from the crawlers that generate citations, GPTBot, ClaudeBot, PerplexityBot, and OAI-SearchBot being some examples. Additionally, a separate Semrush study did not find any statistical relationship between having the file present and appearing better in AI generated responses.
Therefore if the sole purpose of utilizing llms.txt is to appear in ChatGPT responses or Google AI Overviews then llms.txt is not going to be the tool. Structuring your content properly via structured content and schema markup along with earning high-quality citations on the web remain the primary methods used to achieve this, and no file at your root will alter this equation.
Why are Companies Like Stripe, Cloudflare and Anthropic Implementing llms.txt Files Then?
As mentioned previously, many organizations receive traffic that is generated by non-human entities (i.e., codebots and other automated tools) rather than human users searching for information online. These include things like developer’s AI coding assistants (e.g. Cursor), GitHub Copilot, Claude Code, etc. Anthropic notes multiple crawlers for different purposes: ClaudeBot for training, Claude-SearchBot for indexing/search functionality, and Claude-User for live requests when a user asks Claude to fetch a specific page. That latter type is precisely where llms.txt begins to demonstrate its true potential. Creating and maintaining a clean /llms.txt file makes the difference between an AI agent successfully calling an API versus the agent confidently guessing an API endpoint that has been deprecated for two releases.
We chuckled slightly when we initially encountered this phenomenon: Google’s own Search Team indicates that llms.txt doesn’t matter for ranking while another group within Google quietly pushed llms.txt files onto developer documentation for several months starting in December 2024. When two parts of the same organization cannot agree on whether something has utility, you are permitted to be perplexed as well.
Where llms.txt Actually Earns Its Keep
For most SME websites, the honest use cases split into three buckets:
AI coding and dev-tool assistance. If your product has an API, SDK, or developer docs, an llms.txt file genuinely reduces the token-waste and hallucination risk when someone’s coding assistant tries to integrate with you. This is the one use case with strong, repeated evidence behind it.
Agentic commerce. As shopping agents start acting on instructions like “find running shoes under $150 that ship by Friday,” they need a clean, machine-readable path to your catalog, pricing rules, and shipping policy rather than parsing cluttered category-page HTML. Most retailers haven’t done this yet, which makes it a real gap for the ones who do.
Brand identity layer. Chrome’s Lighthouse tool added an “Agentic Browsing” audit category in its May 2026 release that checks for the file’s presence, on the theory that without it, agents spend more crawl time and tokens figuring out a site’s structure from scratch. That’s a technical signal worth having, separate from any ranking claim.
If none of those three describe your business, llms.txt is a low-priority item. If one of them does, it’s a low-effort, low-risk addition worth an afternoon.
How Do You Build One That’s Actually Worth Reading?
An audit of production llms.txt files in May 2026 found the most common failure mode was treating the file as a second sitemap, stuffed with 800 to 1,200 unsorted links. That defeats the entire point. The proposal was explicitly meant to hold expert-level, curated information in one place, not everything you’ve ever published.
A working file for most SME sites should include:
- One short, factual summary of the business (no marketing superlatives, AI systems tend to filter those out anyway)
- Core product or service pages
- Pricing or plan pages, if public
- Buyer-risk policies (privacy, security, shipping, returns, SLAs)
- Technical documentation or API references, if applicable
- Top FAQ or help-center content
- A link to your XML sitemap for anything not explicitly called out
Aim for roughly 20 to 50 high-signal links. If you’re listing more than that, you’re building a sitemap with extra steps, not an llms.txt file.
What to Avoid
Don’t treat it as an access-control tool. llms.txt can’t block a crawler from reading anything. That job still belongs to robots.txt, and confusing the two leaves gaps in your actual crawler governance.
Don’t auto-generate Markdown copies of every page. Some CMS plugins now offer one-click llms.txt generation that mirrors your entire site into Markdown files. If those mirrored files are indexable, you’ve just created duplicate content at scale, which can quietly drag down the rankings of your original pages.
Don’t expect it to fix weak content. A clean llms.txt file pointing to a thin, outdated product page doesn’t make that page stronger. It just gets an AI system there faster to notice it’s thin.
Don’t skip the maintenance. A stale file pointing to dead URLs or discontinued pricing is worse than no file at all, because it actively feeds bad information to anything reading it.
Practical Application: What to Do This Week
If you run developer docs, an API, or anything an AI coding assistant might query, build the file this week. It’s a few hours of work with real, documented upside.
If you run a content-heavy SME site with no developer audience and no agentic-commerce ambitions yet, park it. Spend that afternoon instead auditing whether your five most important pages actually answer the questions people are asking them, in plain language, near the top of the page. That does more for AI visibility than a perfectly formatted llms.txt file ever will.
Frequently Asked Questions
Question: Does llms.txt improve my Google rankings?
Answer: No. Google confirmed in June 2026 that llms.txt files do not influence Search rankings, including AI Overviews and AI Mode. Independent traffic studies have found no meaningful correlation between having the file and improved AI citation rates either.
Question: Is llms.txt the same as robots.txt?
Answer: No. Robots.txt controls which crawlers may access which parts of your site and is legally and technically recognized as a standard. Llms.txt is a navigation aid with no access-control function and no formal standards backing, adopted unevenly across AI providers.
Question: Which AI crawlers actually read llms.txt?
Answer: Adoption is inconsistent. Search-and-answer bots like GPTBot, ClaudeBot, and PerplexityBot show minimal traffic to the file in independent log analysis. AI coding assistants such as Cursor, GitHub Copilot, and Claude Code are the strongest documented users, since they fetch documentation live during development sessions.
Question: Is it worth building an llms.txt file in 2026?
Answer: It depends on your business. If you have developer documentation, an API, or e-commerce ambitions involving shopping agents, yes, it’s a low-effort addition with real upside. If neither applies, your time is better spent strengthening the content and structure of your core pages.
Question: What should I include in an llms.txt file?
Answer: A short factual summary of your business, core product and service pages, pricing if public, key policies like shipping and privacy, technical documentation if relevant, and top FAQ content. Keep it to roughly 20 to 50 high-signal links rather than a full site map.