Give agents a Markdown version of your site

We remain in this transitional period where we’re still figuring out how to make websites agent-ready. Between robots.txt, structured data, MCPs, and a smattering of other conventions (hi, WebMCP), it feels like there is a lot to consider and no single checklist (although Cloudflare’s isitagentready.com might not be a bad place to start).

Once an agent is looking at a site, an obviously important first job is making sure the agent understands it. What is this site for? Who is it for? What does it mean?

Between reading metadata, parsing HTML, and analyzing screenshots, an agent can infer a lot with websites as they exist today, but it’s not exactly efficient.

If the agent primarily communicates in text, a Markdown representation of the site could be a useful interface. So how do we provide that and where does it fit alongside other agent interfaces?

The interface depends on where the agent is

When we’re communicating with humans, we offer affordances for their current context: metadata for search results, unfurls for chat, Open Graph metadata for social media, ARIA labels for accessibility, responsive layouts for different screen sizes, and more.

We’re seeing a similar dynamic play out for agents. An agent might encounter a URL in a search result. It might be working in a browser tab. It might have access to your service with authentication and permissions. Each situation calls for a different kind of interface, but not necessarily the same interface that a human might consume.

Here’s a non-exhaustive list to illustrate:

Agent context	Ideal interface	What it provides
Fetching a URL	Markdown	Efficient representation of page content
Using a browser tab	WebMCP	Structured access to a page’s features
Credentialed access	MCP or API	Deeper, durable service capabilities

These interfaces can (and usually will) sit alongside each other. A Markdown representation helps an agent understand. WebMCP exposes a structured surface for the current page. MCP and conventional APIs are appropriate when a system has capabilities to offer independently of one page.

I wrote recently about WebMCP for browser agents. It lets a page register structured tools for an agent already working in the browser. The tools can expose read-only lookups or pre-defined interactive actions.

There are many agentic use cases that do not require a live tab or a tool call. An agent may just need to read the site. That’s where the Markdown representation comes in.

HTML is structured, but would you read it?

Crack open Chrome DevTools and look at the HTML for a typical homepage. It’s structured, but it’s also huge.

An agent can read whatever tangle of HTML you throw at it, but it’s going to do a lot of throwaway work in the process, sifting through your navigation, repeated calls to action, responsive variants, footer links, image treatment, and layout code (like all of your nested divs).

Most of your HTML representation is for browsers that use all of this code to render the page. For an agent, it’s often a ton of overhead.

Here’s a (very!) simplified example of a homepage for a community site. The important information is buried in the markup.

<div class="site">
  <header>
    <nav>
      <ul>
        <li><a href="/">Home</a></li>
        <li><a href="/events">Events</a></li>
        <li><a href="/members">Members</a></li>
      </ul>
    </nav>
  </header>

  <main>
    <article>
      <h1>Kyoto Tech Meetup</h1>
      <p>Welcome to the Kyoto Tech Meetup community!</p>

      <section id="next-event">
        <h2>Next Event</h2>
        <p>Join us for our next meetup on August 15th at 7 PM JST.</p>
        <p>Location: FabCafe Kyoto</p>
        <a href="/rsvp">RSVP here</a>
      </section>

      <section id="upcoming-events">
        <!-- You get the idea -->
      </section>

      <section id="member-publications">
        <!-- More markup here -->
      </section>

      <footer>
        <p>&copy; 2026 Kyoto Tech Meetup. All rights reserved.</p>
      </footer>
    </article>
  </main>
</div>

Now, imagine a Markdown representation of the same information. It is concise, focused, and easy to read.

# Kyoto Tech Meetup

Welcome to the Kyoto Tech Meetup community!

## Next Event

Join us for our next meetup on **August 15th at 7 PM JST**.
**Location:** FabCafe Kyoto
[RSVP here](/rsvp)

The HTML will look great rendered in the browser for our human guests, but the Markdown is easier for an agent to read and understand. It distills the essential information without the noise.

One URL, two representations

I recently added a Markdown representation to the homepage of the Kyoto Tech Meetup website. The standard browser request still receives HTML. A client that explicitly asks for Markdown receives a concise, localized document instead.

At request time, the Cloudflare Pages Function chooses the representation:

This is ordinary HTTP content negotiation. A small Cloudflare Pages Function at the edge inspects the request and chooses between two static representations.

We don’t need to introduce a new public endpoint or API service. The implementation stays small, the human-facing site remains static, and the canonical URL remains intact.

A small route function handles the choice. If Markdown is requested, it serves the Markdown file for the requested path. Other requests fall through to the normal static site.

export function handleMarkdownRequest(context, assetPath) {
  return acceptsMarkdown(context.request.headers.get("Accept"))
    ? serveMarkdown(context, assetPath)
    : context.env.ASSETS.fetch(context.request);
}

The response includes the expected MIME type and cache variation:

Content-Type: text/markdown; charset=utf-8
Vary: Accept

The Vary: Accept HTTP header keeps this arrangement safe for caches. It tells a CDN or browser cache that the same URL can produce different responses depending on what the client asked for. Without it, a shared cache could serve the Markdown representation to an HTML request, or the other way around.

Markdown is returned only when a client explicitly requests it. Requests with a generic */* header continue to receive HTML, and text/markdown;q=0 excludes Markdown altogether.

Static files, shared data

The routing layer stays small because it only has static files to serve. The Kyoto Tech Meetup website is built with Astro. The Markdown build writes assets to public/; Astro copies them, alongside the HTML it renders, into the deployed dist/ output.

The final build includes:

dist/index.html
dist/ja/index.html
dist/agent-home.en.md
dist/agent-home.ja.md

At build time, the relevant parts of the process look like this:

Both representations draw on the same content data. The Kyoto Tech Meetup website’s internationalized Astro template uses meetup event and member RSS feed snapshots to render HTML. The Markdown generator reads the same snapshots rather than creating another source of content to maintain.

That gives us these benefits:

• There is a single source of truth for the content, which is important for maintainers.
• Agents get the same information as the website’s regular visitors.
• Both versions can be inspected as build artifacts and live reviewed in a browser.

The single source of truth is what makes this approach sustainable.

Why do both Markdown and WebMCP?

To state the obvious, Markdown is read-only. There’s nothing interactive going on here.

For the Kyoto Tech Meetup site, our WebMCP implementation is also read-only, but that’s just because we haven’t exposed any interactive tools yet. We could do so in the future. (For reference, Chrome’s WebMCP documentation describes tools for form input, navigation, and state changes.)

So why do both? Partly because Markdown for Agents and WebMCP meet agents at different points in their journey. But if I’m honest, it’s also because a small developer community site is a fantastic playground for experimenting with new toys cutting edge technologies.

Work backwards from the agent’s context

Assuming you want agents on your site (Kyoto Tech Meetup does!), we first need to consider the context in which the agent is interacting with the site. The interface we provide should be appropriate for contexts that we want to support.

Markdown for Agents is a good fit when the context is an HTTP request and we want to provide an efficient representation of the content.

WebMCP may be a good fit (this is still an entirely experimental technology, so we don’t really know) when we want to expose well-defined capabilities to an agent that is actively interacting with the site in a browser context.

An API or MCP server is a good fit when a capability needs to work beyond an open tab and there is a durable service behind it, like, for example, an RSVP system. For our community site, we don’t currently offer such capabilities, so we don’t provide an API or MCP server.

These three interfaces can coexist but they are not all required. They also aren’t the only ways to make a site agent-ready, but they are a good starting point for helping an agent understand a site and its capabilities.