Markdown for agents
AI agents, and people working with them, read Markdown more easily than HTML: it’s smaller, and it keeps headings, lists, links and code without your site’s navigation and scripts. If your site publishes a Markdown copy of each page, Evolving Edge can serve that copy from the page’s own address to any request that asks for Markdown. Browsers keep getting your HTML.
GET /cli/Accept: text/markdown
→ 200, Content-Type: text/markdown; charset=utf-8 (your site's /cli.md)Nothing is converted: the network serves the .md files your build writes. It’s off until you turn it on, and it works on Level 0 (public) sites only.
Turning it on
Section titled “Turning it on”Three steps, in this order: build the copies, turn the setting on, then deploy and check.
-
Build a Markdown copy beside each page. Your build writes the copies into its output, next to the pages they belong to. See Where the copies go.
-
Add one line to your
_redirectsfile:# ee:markdown onIt’s a comment, so the file still works on other hosts.
-
Deploy, then check with
curl, as in Checking it.
Pages without a copy are served as HTML, as before, so you can add copies one section at a time.
Where the copies go
Section titled “Where the copies go”For each page, the network looks for a copy in these places, in order, and serves the first one it finds:
| Page address | Page file | Copy looked for |
|---|---|---|
/cli/ |
cli/index.html |
cli.md, then cli/index.md |
/ |
index.html |
index.md |
/about.html |
about.html |
about.md |
/about (with clean URLs) |
about.html |
about.md |
How you write the copies depends on your generator. Astro can write them from the same content as your pages with an endpoint (a file such as src/pages/[...slug].md.ts). With most generators, you can also copy your Markdown sources into the output with a short script after the build. A copy should read well on its own: start with the page’s title and a link back to the page.
Copies are ordinary files too. /cli.md is served at its own address as well, so you can link to it.
What agents get
Section titled “What agents get”-
Only requests that prefer Markdown. The copy is served when the request’s
Acceptheader rankstext/markdownabovetext/html. A request that ranks them equally, or that sends*/*, gets HTML. Browsers never ask for Markdown, so visitors always see your pages. -
The copy as it is. Its own content type (
text/markdown), its ownETag, and nothing added to it. -
Your
_headersrules for the.mdpath. A rule for/*.mdapplies to the copy, wherever it’s served from. For example, to keep the copies out of search results:.md X-Robots-Tag: noindex -
Your
_redirectsrules first. A page address that a rule redirects stays redirected. A copy that a rule redirects, or hides with a404, at its own address isn’t served from the page’s address either.
What changes on your pages
Section titled “What changes on your pages”Each page that has a copy gets two extra response headers, whichever version is served:
Vary: Accept, so caches between your site and the reader keep the HTML and the Markdown apart.Link: </cli.md>; rel="alternate"; type="text/markdown", on the HTML only, so an agent that fetched the page can find its copy.
If your _headers file sets its own Vary or Link on a page, both are kept. Pages without a copy get neither header.
The copy is cached like the page: a browser or cache checks for a new version on each request, so a deploy shows at once. Requests for copies count towards your usage like any other request.
Checking it
Section titled “Checking it”Ask for a page as Markdown, then as a browser would:
curl -s -H 'Accept: text/markdown' https://www.example.com/about/ | headcurl -sI https://www.example.com/about/ | grep -iE '^(content-type|vary|link):'The first should print your copy. The second should show text/html, Vary: Accept and the Link header. If you get HTML back from the first:
- Check that the setting is on. The deploy prints what it found in
_redirects, for example_redirects: 2 rule(s), trailing-slash: on, clean-urls: off, markdown: on. A tool that strips comments can remove the line without any other sign. The deploy also warns if the setting is on but your site has no.mdfiles. - Check that the copy exists at one of the addresses in Where the copies go: request it directly, for example
/about.md.
Scope and limits
Section titled “Scope and limits”- Level 0 sites only. Encrypted sites (Level 1 and Level 2) don’t read
_redirects. See encryption levels. - Pages only. Only addresses that serve an HTML page negotiate. A path that a
200rule rewrites to another file, such as a single-page app’s shell, is left alone, because the shell’s copy wouldn’t describe the route. - No conversion. The network never turns HTML into Markdown. Pages without a copy are always served as HTML.
llms.txtis up to your build. An index of your copies for agents, such asllms.txt, is a file your build writes like any other. This site publishes one at /llms.txt.
Next: Coverage & Performance >
