Llms.txt Playbook: What It Actually Does for AI Crawlers
Sometime around the end of 2024, a proposal for creating a file called llms.txt began circulating in SEO communities. By early 2025, roughly half the Internet was either building one or saying there’s no use building one because that will take a whole Tuesday afternoon. In general, both sides were right. The fact that both sides were right is enough to write about.
A llms.txt file is simply a plain-text Markdown file located at your site’s root directory that describes your site’s content and points large language model systems to your most important pages, similar to how a Table of Contents directs a reader to a Chapter. Importantly, it does not restrict crawler access (robots.txt continues to perform this function). As of mid-2026, none of the major search or AI providers have indicated they currently consider llms.txt in their rankings or citations. However, llms.txt provides much greater value elsewhere.
Llms.txt Playbook: What It Actually Does for AI Crawlers Read More »
