We now support Meta Muse Spark. Track your Spark prompts

How to Add llms.txt for AI Search Optimization in Record Time

Learn how to add llms.txt & llms-full.txt to boost your website’s visibility in AI search engines like ChatGPT, Perplexity, and Deepseek. A step-by-step guide to structuring your content for AI indexing in under 60 minutes.

Mathias Ptacek•
llms.txt guideAI website optimizationLLMOGEOLarge Language ModelsAI content structuringenhancing AI interaction
How to Add llms.txt for AI Search Optimization in Record Time

How I Added llms.txt & llms-full.txt in No Time - And Why You Should Too

Rankscale is all about fast and agile improvements that sets it apart in the ever-changing AI landscape. Recently, I rolled outllms.txtandllms-full.txt!

Here’s a behind-the-scenes look at what they are, how they work, and a quick rundown of the implementation process inspired directly by the guidelines at llmstxt.org [2].

What’s the Deal with llms.txt?

llms.txtis a markdown-formatted text file that lives in your site’s root directory. Its job? Give AI a structured summary of your site’s content—without all the extra clutter. Withllms.txt, you tell AI exactly what your site is about, ensuring it picks up the right information.

Key Takeaway:Traditional crawlers (like Google) are great at indexing websites,but AI-based search engines and their simple crawlers often ignore JavaScript-rendered content. That means sites built on React, Vue, Angular, or similar frameworks might not show up properly in AI-generated responses. So in LLMO terms, by usingllms.txtandllms-full.txtyou canboost your relevance in AI-driven search results dramatically.

Why This Matters

If you want AI-powered search engines and chatbots (like ChatGPT, Perplexity,DeepSeekor Claude) to pull accurate information from your site,llms.txtis essential.Here’s why:

  • AI Search Engines Are on the RiseAI search results are becoming the more and more relevant in 2025 (e.g.ChatGPT has just unlocked Web Search for free, nonregistered users[3]) — and they work differently from traditional search engines. While Google indexes all pages, most LLMs don’t process JavaScript-rendered content. If your site relies on client-side rendering, it’s effectively invisible to AI search tools in the field ofGenerative Engine Optimization (GEO).
  • Your Competitors are already optimizing for AILooking at thedirectory.llmstxt.cloud, more and more companies are adapting to the standard [2]
  • Accurate Information = Better VisibilityWithout a structured file, LLMs might misinterpret or completely miss your content. You don’t want AI chatbots making assumptions about your business, products, or services - as your customers already use AI answers heavily.
  • Faster AI Indexing:With our new llms.txt files, AI systems can immediately pick up what our site is about. This means better visibility and more accurate search results for client-rendered content that might otherwise be overlooked.
  • It’s Fast, Easy, and Future-ProofCreatingllms.txtis ridiculously simple -it’s just a well-structured markdown file. But that small step future-proofs your site for AI-driven search and discovery. A few minutes of work, a couple of new files, and we’re ahead in the digital game!

How to do it: Following the llms.txt Format

I based my implementation on the straightforward, effective format specified at llmstxt.org [1] while looking at other implementations at directory.llmstxt.cloud [2]

Here’s the step-by-step breakdown:

  1. Create allms.txtfile in your root directoryOpen a plain text editor (VS Code, Notepad++, or any IDE).
  2. Save the file asllms.txtin UTF-8 encoding.
  3. Follow the Recommended Format, here my Rankscale example (simplified):Start with anH1 header("# YourSiteName - optional tagline").
  4. Add ablockquote summary("> A short description of your site").
  5. Includestructured sections("## YourSection") with links to important content.

# Rankscale.ai - Monitor Your Website or SaaS or Brand or Service or Product or Client in AI Search Engines > Analyze and Optimize your AI Search Visibility with Results Tracking, Deep Insights and Actionable Recommendations - in Seconds. Here's what you can do with Rankscale: - Audit your website to see how AI understands it - Identify Gaps and Content Optimization Potential - Track your Presence in AI Search Results over Time - Analyze AI Result Citations & Brand Sentiments - Uncover, Survey, and Outpace Competitors - Rankscale works accross: ChatGPT, AI Overviews, Perplexity, Claude, Gemini, Deepseek [Rankscale Landing Page](https://rankscale.ai): Rankscale's landing page. ## Rankscale Features - [Features](https://rankscale.ai/features.md): Learn about the features of Rankscale. ## Why use Rankscale? - [Why Rankscale?](https://rankscale.ai/whyrankscale.md): Learn why Rankscale is the best tool for AI search compared to traditional SEO tools. ## Frequently Asked Questions - [Frequently Asked Questions](https://rankscale.ai/faq.md): Answers to common questions about Rankscale, AI search, and Generative Engine Optimization (GEO). ## Blog - [RankScale Blog](https://rankscale.ai/blog): AI SEO, visibility tracking, and brand monitoring insights. Learn about AEO, GEO, and AI search optimization. ## Blog Posts - [AI SEO: Mastering AEO & GEO](https://rankscale.ai/blog/explosive_facts_and_steps_to_master_ai_search_engine_optimization_in_2025): Master AI search engine optimization (AEO and GEO) in 2025: a guide. - [DeepSeek R1: Revolutionizing AI](https://rankscale.ai/blog/deepseek_r1_revolutionizing_ai_with_transparent_reasoning_and_integrated_web_search): DeepSeek R1: Revolutionizing AI with Transparent Reasoning and Integrated Web Search

  1. Add markdown files to extend/route to in your sections In"## YourSection", integrate a link (format:"[title](URL):description") to the extended markdown content by using an additional "*.md"file --> See my "features.md" example for "- [Features](https://rankscale.ai/features.md): Learn about the features of Rankscale" below.

# Features ## Website Audits Perform In-Depth AI Audits of Your Website Uncover how AI search engines perceive your content, pinpoint gaps, and discover optimal paths to greater visibility. - Content Scoring: Get detailed insights on relevance, authority, engagement, and technical structure that affect AI search visibility. - Gap Analysis & Recommendations: Pinpoint which areas need improvement — with content recommendations on critical sections. - Comprehensive Crawls: Select between single-page or multi-page crawling to fully reveal your site's AI readiness. - Customizable Bot Settings: Simulate AI bots accurately by toggling anti-bot measures for realistic audits ## Performance Tracking Monitor and Understand Your AI Presence See where and how often your brand appears in AI search results. - Specify your own search terms: Design custom AI search prompts that fit to your brand and industry - or leverage AI to propose them for you. - Real-Time Updates: Analyze trends and changes to your brand's AI search rankings accross time, topics and regions. - Cross-Channel Insights: Get a unified view and compare results across different AI answer engines and industry topics. - Sentiment Context: Understand how AI search engines perceive your brand and industry topics. ## Competitor Benchmarking Find and Track your Competitors Identify the key players in your AI search landscape, monitor their performance, and stay one step ahead. - Competitor Discovery: Automatically uncover direct and indirect competitors for relevant AI search terms and industry topics. - Side-by-Side Comparisons: Compare visibility, sentiment, rankings and other important metrics to gain a holistic view of your competitors. - Competitor Citations: See how often your competitors are cited and which sources are responsible for their visibility. - Benchmarking: Identify Competitor's strenghts and weaknesses accross industry topics and AI engines. [...]

  1. Upload thellms.txtand "*.md"file(s) to your site's root directoryIf using cPanel or FTP, place it at /public_html/llms.txt or /public/llms.txt
  2. If using Git, push it to your production build.
  3. This makes thempublicly accessibleto AI crawlers.
  4. Optionally you could create a path in your sitemap.xml as well (just a personal opinion)
  5. Verify accessibilityOpen https://yourwebsite.com/llms.txt in your browser.
  6. Ensure it displays correctly and isn’t blocked by robots.txt.
  7. Optionally, add llms-full.txtfor detailed AI indexing Copy all structured markdown from your site into llms-full.txt.
  8. This provides AI engines a full content snapshot for deeper indexing.
  9. Update as neededJust likerobots.txt, keep itfresh and accurateas your site evolves.
  10. Use tools like Mintlify to automate the creation and any updates [4]

The Bottom Line

If your websiterelies on JavaScript, or you simply want to ensure AI models interpret your content correctly, addingllms.txtor evenllms-full.txtis a quick, no-brainer improvement.I did it in minutes (ok, ok, let's say <60 minutes - I'm a slow thinker), and now Rankscale's content is properly structured for AI.

Want your site to rank higher in AI search results?

  1. Followthis guide
  2. Implementllms.txt
  3. Optimize and monitoryour AI search visibilitywithRankscale.ai[5]

Sources

Back to Blog