How to Create an llms.txt File for Your Website

If you’ve heard of robots.txt and sitemap.xml, meet their newer cousin: llms.txt. It’s a small text file that tells AI tools like ChatGPT, Claude, and Perplexity what your website is about, so they can summarize and cite it accurately.

Here’s what it is, whether you need one, and exactly how to create it.

What Is llms.txt?

llms.txt is a Markdown file placed at your website’s root (yoursite.com/llms.txt) that gives AI models a clean, structured summary of your site — what it does, and links to the pages that matter most.

The idea was proposed in September 2024 by Jeremy Howard, co-founder of fast.ai, as a machine-readable complement to sitemaps and robots.txt. The problem it solves: AI models have limited context windows and can’t read an entire website. Raw HTML is also full of navigation menus, ads, and scripts that waste space and confuse extraction. llms.txt skips all of that and hands the model a curated map instead.

Think of it this way:

• robots.txt tells crawlers what not to crawl
• sitemap.xml lists every page, for search engines
• llms.txt tells AI models what your site actually contains and how to interpret it

Do You Actually Need One?

Worth setting expectations before you build one. Adoption is still early — over 2,000 sites have implemented it, including Anthropic and Cloudflare, but it’s a community proposal, not an official web standard. There’s no IETF or W3C spec behind it, and unlike robots.txt, AI systems don’t automatically crawl for it. There’s no guarantee any given tool checks for the file at all.

That said, it’s cheap to create, doesn’t hurt anything, and positions your site well if adoption grows. Treat it like early schema markup: low-cost, plausibly useful, no promised return.

The File Format

A valid llms.txt file follows a specific structure, in this order:

1. H1 heading — your site or project name (required, the only required part)
2. Blockquote — a one-sentence summary right under the H1
3. Optional context — a short paragraph or two expanding on the site (no headings allowed here)
4. H2 sections — grouping links by category, each with a short description

Here’s a minimal example:

# Clearflow Plumbing
> Residential and commercial plumbing services in Cebu City.
Clearflow Plumbing handles repairs, installations, and emergency
callouts for homes and businesses across Cebu.
## Services
- [Emergency Repairs](https://clearflowplumbing.com/emergency): 24/7 leak and burst pipe response
- [Drain Cleaning](https://clearflowplumbing.com/drain-cleaning): Clogged drain and sewer line clearing
- [Water Heater Installation](https://clearflowplumbing.com/water-heaters): Installation and replacement services
## About
- [About Us](https://clearflowplumbing.com/about): Company background and service area

A few formatting rules worth following:

• Keep the whole file under 10KB — it’s a curated index, not a full copy of your site
• Use full URLs, not relative paths
• Write factual descriptions, not marketing copy
• Only include your 5–15 most important pages; skip legal boilerplate, login pages, and one-off old posts

The llms-full.txt Companion

For content-heavy sites — documentation, extensive blogs, API references — you’ll often see a second file: llms-full.txt. Where llms.txt is a map of links with short descriptions, llms-full.txt inlines the actual page text, cleaned and concatenated, so a model can absorb everything in one fetch. Anthropic, Vercel, and others ship both. Marketing sites with under 10 key pages usually don’t need it.

How to Create One

Option 1: Write It by Hand

For small sites, the fastest path is writing the file yourself using the template above, then uploading it to your site’s root directory via FTP, SFTP, or your host’s file manager, so it resolves at yoursite.com/llms.txt.

Option 2: Prompt an AI to Generate It

You can hand an AI model your site’s structure and have it draft the file. A prompt like this works well:

Create an llms.txt file for my website following the llmstxt.org spec.
Site name: Clearflow Plumbing
Site URL: https://clearflowplumbing.com
One-line description: Residential and commercial plumbing services in Cebu City
Key pages to include (most important first):
- [Page title] — [URL] — [one-line description]
- [Page title] — [URL] — [one-line description]
Format requirements:
- H1 with site name, blockquote summary underneath
- H2 sections grouping links logically
- Each link as: - [Title](URL): short description
- Under 10KB, factual descriptions, no marketing language
- Output only the raw markdown file, nothing else

Always review the output before publishing — verify every URL is real and correctly cased, and trim anything that reads like ad copy.

Option 3: Use a Generator Tool or Platform Feature

Several dedicated generators exist that crawl your site and draft the file automatically (Firecrawl and Mintlify are two examples for hosted docs). Squarespace now auto-generates and hosts llms.txt natively. If you’re on WordPress, Yoast SEO has this built in: enabling the feature creates the file at your site root and refreshes it weekly, automatically pulling in your most recently updated pages/posts and top categories. Treat any auto-generated output as a first draft — trim it and add the per-link descriptions that make the file actually useful.

Common Mistakes to Avoid

• Wrong location — placing the file in a subdirectory instead of root (a subpath is technically allowed by the spec, but root is what most tools check first)
• No H1 or blockquote — these are the only required elements; skipping them breaks the format
• Listing everything — an llms.txt with your entire sitemap defeats the purpose; keep it curated
• Marketing fluff — write descriptions like documentation, not ad copy
• Broken links — dead links inside the file undermine the whole point
• Set-and-forget — if your pages change, update the file; a stale map is worse than no map

Bottom Line

llms.txt takes an hour or two to set up properly and costs nothing to maintain if your platform automates it. It won’t guarantee AI citations or traffic, but it’s a small, forward-looking addition to your site’s technical foundation — similar to how early structured data adopters got ahead before it became standard practice.