llms.txt

llms.txt is a plain-text file served at the root of a website (e.g., bizaiready.com/llms.txt) following the open spec proposed by Jeremy Howard in September 2024. It uses Markdown formatting and lists the site's most important pages with one-line descriptions, organized under H2 section headers like "Core offering," "Documentation," and "Optional."

The goal is the same as sitemap.xml — help machines discover the right URLs — but the audience is different. Sitemap.xml is for search engine crawlers (Google, Bing). llms.txt is for AI assistants, agent frameworks, and developer tools that need a curated index of what matters most on a site, not an exhaustive crawl.

Adoption as of mid-2026 is real but uneven. Anthropic, Perplexity, Hugging Face, Cloudflare, Vercel, Cursor, and Mintlify all publish llms.txt files. No major AI lab (OpenAI, Anthropic, Google) has officially confirmed using llms.txt for ranking or citation. Treat it as cheap insurance — a 5-minute setup that some agentic tools already consume and that costs nothing if the standard never takes off.

The minimum-viable llms.txt has an H1 with the site name, an optional blockquote summary, and one or more H2 sections listing key URLs as Markdown links with one-line descriptions per link.