Robots & llms.txt

This module covers two adjacent files that control how automated visitors read your site: the long-standing robots.txt, and the newer llms.txt convention aimed specifically at AI crawlers and agents.

Robots.txt enhancements#

Seofyme adds sensible defaults to robots.txt — pointing at your sitemap index, blocking WordPress admin paths, and giving you a settings screen to add custom rules without editing a file directly.

Auto-generated llms.txt#

Seofyme can generate a /llms.txt file for your site summarizing what it is and linking to its most important pages, in the plain-text format AI agents and LLM-based crawlers are starting to look for — similar in spirit to how seofyme.com exposes its own `/llms.txt` for AI agents evaluating Seofyme itself.