[{"data":1,"prerenderedAt":268},["ShallowReactive",2],{"learn-geo-/en/learn/geo/llms-txt-en":3},{"id":4,"title":5,"body":6,"description":227,"extension":228,"meta":229,"navigation":106,"path":262,"seo":263,"stem":266,"__hash__":267},"content_en/5.learn/geo/llms-txt.md","llms.txt",{"type":7,"value":8,"toc":221},"minimark",[9,25,30,47,56,63,69,73,84,154,161,165,191,195,217],[10,11,12,16,17,20,21,24],"p",{},[13,14,15],"strong",{},"TL;DR"," — ",[18,19,5],"code",{}," is an emerging standard (llmstxt.org, 2024) analogous to ",[18,22,23],{},"robots.txt"," but specifically for AI crawlers. It provides a curated, LLM-friendly summary of your site in Markdown. Perplexity already supports it. It's a 30-minute implementation with long-term AI discoverability upside.",[26,27,29],"h2",{"id":28},"why-llmstxt-matters-for-ai-engines","Why llms.txt Matters for AI Engines",[10,31,32,33,35,36,43,44,46],{},"The ",[18,34,5],{}," specification (introduced at ",[37,38,42],"a",{"href":39,"rel":40},"https://llmstxt.org",[41],"nofollow","llmstxt.org"," in 2024) addresses a fundamental problem: AI engines that crawl the web to answer queries receive all content equally, without knowing which pages are most important or authoritative for a given site. The ",[18,45,5],{}," file solves this by providing a curated, LLM-readable index of your site's most important content.",[10,48,49,50,52,53,55],{},"Unlike ",[18,51,23],{},", which tells crawlers what not to access, ",[18,54,5],{}," tells AI engines what is most important and how to understand your site. It's a structured summary in plain Markdown format — site name, description, and a list of key pages with brief descriptions.",[10,57,58,59,62],{},"The spec also defines ",[18,60,61],{},"llms-full.txt"," — a companion file containing the complete content of your key pages in a single, LLM-consumable document. This is particularly valuable for AI engines that build knowledge graphs of sites: rather than crawling 500 pages, they can read one file that contains everything important.",[10,64,65,66,68],{},"Perplexity announced support for ",[18,67,5],{}," in early 2025. As the spec gains adoption, AI engines that support it will have a more accurate, curated understanding of your site compared to AI engines that have to infer structure from crawling.",[26,70,72],{"id":71},"how-to-implement","How to Implement",[10,74,75,76,79,80,83],{},"Create ",[18,77,78],{},"/llms.txt"," at your site root, accessible at ",[18,81,82],{},"https://yourdomain.com/llms.txt",":",[85,86,91],"pre",{"className":87,"code":88,"language":89,"meta":90,"style":90},"language-markdown shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Your Company Name\n\n> One paragraph description of your site: what it does, who it's for, and what makes it authoritative.\n\n## Key Pages\n\n- [Home](https://yourdomain.com/): Main landing page — overview of what we do\n- [Product](https://yourdomain.com/product): Full product description and features\n- [Blog](https://yourdomain.com/blog): Articles and guides on [your topic]\n- [About](https://yourdomain.com/about): Team, mission, and company background\n","markdown","",[18,92,93,101,108,114,119,125,130,136,142,148],{"__ignoreMap":90},[94,95,98],"span",{"class":96,"line":97},"line",1,[94,99,100],{},"# Your Company Name\n",[94,102,104],{"class":96,"line":103},2,[94,105,107],{"emptyLinePlaceholder":106},true,"\n",[94,109,111],{"class":96,"line":110},3,[94,112,113],{},"> One paragraph description of your site: what it does, who it's for, and what makes it authoritative.\n",[94,115,117],{"class":96,"line":116},4,[94,118,107],{"emptyLinePlaceholder":106},[94,120,122],{"class":96,"line":121},5,[94,123,124],{},"## Key Pages\n",[94,126,128],{"class":96,"line":127},6,[94,129,107],{"emptyLinePlaceholder":106},[94,131,133],{"class":96,"line":132},7,[94,134,135],{},"- [Home](https://yourdomain.com/): Main landing page — overview of what we do\n",[94,137,139],{"class":96,"line":138},8,[94,140,141],{},"- [Product](https://yourdomain.com/product): Full product description and features\n",[94,143,145],{"class":96,"line":144},9,[94,146,147],{},"- [Blog](https://yourdomain.com/blog): Articles and guides on [your topic]\n",[94,149,151],{"class":96,"line":150},10,[94,152,153],{},"- [About](https://yourdomain.com/about): Team, mission, and company background\n",[10,155,156,157,160],{},"Optionally, create ",[18,158,159],{},"/llms-full.txt"," with the complete text content of your most important pages.",[26,162,164],{"id":163},"common-mistakes","Common Mistakes",[166,167,168,175,183],"ul",{},[169,170,171,174],"li",{},[13,172,173],{},"Using HTML instead of plain Markdown"," — the spec explicitly requires plain Markdown; HTML tags will confuse LLM parsers",[169,176,177,16,180,182],{},[13,178,179],{},"Not updating when major pages are added",[18,181,5],{}," should reflect your current site structure; treat it like a sitemap",[169,184,185,16,188,190],{},[13,186,187],{},"Including pages you don't want AI engines to use",[18,189,5],{}," is a positive curation tool; only include pages you actively want AI engines to read and cite",[26,192,194],{"id":193},"sources","Sources",[166,196,197,203,210],{},[169,198,199],{},[37,200,202],{"href":39,"rel":201},[41],"llmstxt.org specification",[169,204,205],{},[37,206,209],{"href":207,"rel":208},"https://www.anthropic.com/research/crawling-policy",[41],"Anthropic crawl policy",[169,211,212],{},[37,213,216],{"href":214,"rel":215},"https://platform.openai.com/docs/gptbot",[41],"OpenAI GPTBot documentation",[218,219,220],"style",{},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":90,"searchDepth":103,"depth":103,"links":222},[223,224,225,226],{"id":28,"depth":103,"text":29},{"id":71,"depth":103,"text":72},{"id":163,"depth":103,"text":164},{"id":193,"depth":103,"text":194},"A plain-text file at /llms.txt that tells AI crawlers what your site is about and what they may use.","md",{"publishedAt":230,"badge":231,"type":233,"faq":234,"related":244,"cta":257},"2026-03-31",{"label":232},"Authority","guide",[235,238,241],{"question":236,"answer":237},"Is llms.txt an official standard or still experimental?","As of 2025, llms.txt is a community-driven specification that major AI companies have not officially mandated, but several AI engines (notably Perplexity) have announced support. It's analogous to the early days of robots.txt — not yet universal, but implemented by enough tooling to be worth the 30-minute setup cost.",{"question":239,"answer":240},"Does llms.txt replace robots.txt for AI crawlers?","No. robots.txt controls access (what crawlers can and cannot fetch). llms.txt is a curation guide (what content is most important and how to understand the site). Both serve different purposes and should both be present. robots.txt for access control, llms.txt for content prioritisation.",{"question":242,"answer":243},"What should I put in the site description in llms.txt?","Write it as if you were explaining your site to someone who will summarise it in one paragraph. Include: what the site/company does, who it serves, what makes it authoritative or unique, and the primary topics covered. 2–4 sentences is ideal. This description may be directly used by AI engines when generating summaries of your site.",[245,249,253],{"title":246,"url":247,"description":248},"AI Bot Access via robots.txt","/learn/geo/ai-bot-access","robots.txt access control — the prerequisite for llms.txt to have any effect.",{"title":250,"url":251,"description":252},"Schema Markup for AI Engines","/learn/geo/schema-markup","Structured data that complements llms.txt for individual page context.",{"title":254,"url":255,"description":256},"Intro Summary","/learn/geo/intro-summary","Page-level intro summaries that should mirror your llms.txt page descriptions.",{"title":258,"description":259,"label":260,"url":261},"Does your site have an llms.txt file?","TrustData checks for llms.txt presence, format validity, and whether your key pages are included.","Audit my pages","https://app.trustdata.tech","/learn/geo/llms-txt",{"title":264,"description":265},"llms.txt — AI Crawler Guidance File | GEO Optimisation Guide","The llms.txt spec (2024) gives AI engines a curated, Markdown-formatted summary of your site. Perplexity already supports it. A 30-minute implementation with long-term discoverability upside.","5.learn/geo/llms-txt","i33baq32I7tfceHwo9wwaONZCwXIH3fUWupW4Y_zzUg",1777026714087]