/llms.txt ; A New Standard for LLM-Optimized Website Information

Jeremy Howard proposes a new standard: a /llms.txt file for websites to provide information optimized for Large Language Models (LLMs). This Markdown file, located at the root of a website, offers a concise overview and links to more detailed Markdown files, making it easier for LLMs to access key information.

The proposal aims to improve how LLMs interact with websites, addressing limitations of context windows and the challenges of converting complex HTML. The format includes key sections, a summary, and links to further documentation, allowing both human and LLM readability. The /llms.txt standard is intended to work alongside existing web standards like robots.txt and sitemap.xml.

Proposal