Markdown for Agents on Apache: How to Hit 100/100 on Cloudflare's AI Audit, Free

From 50 to 100/100 on Cloudflare's AI Compatibility audit, Level 5 Agent-Native, in 30 lines of configuration and zero CDN subscription.

For twenty years, web pages have been optimized for two audiences: humans and Google bots. The first reads with their eyes, the second crawls with an HTML parser that understands layout very well. Everyone got what they needed.

LLMs change the game. When ClaudeBot or GPTBot fetches a page to answer a user, it does not render visually. It internally converts raw HTML into a workable text representation, then passes that representation to the model, which consumes tokens at every character. For this use case, HTML is a verbose and noisy format, whereas Markdown was designed from day one to be readable by both humans and machines, with no parasitic layout.

The hidden HTML cost for an AI agent

Take a concrete case. A rankit.fr article page weighs about 180 KB of HTML with inline CSS, ARIA tags, scripts, og: meta, and JSON-LD. The same page converted to clean Markdown weighs about 18 KB. Ten times less to process.

For the model, that difference translates into tokens consumed. On the page above:

A 90% reduction in consumed context is not a detail. For an agent that must synthesize ten sources before answering, it can be the difference between fitting in the context window and arbitrarily truncating sources. And a truncating agent is an agent that cites poorly, or not at all.

A quality signal for the ecosystem

Beyond token savings, serving a negotiated Markdown version sends three converging signals to agent developers and AI engines.

First signal: technical maturity. The site understands AI consumers' constraints, knows how to configure HTTP content negotiation, and is willing to produce two consistent representations of the same content. That's a rare attention marker in 2026.

Second signal: intent to be read. Serving explicit Markdown tells LLMs yes, you can consume this content, here is the version that suits you. Conversely, a site that only replies in HTML and ships a restrictive robots.txt sends the opposite message. Agents picking between ten sources will mechanically favor the first group.

Third signal: durability. Cloudflare published its spec in February 2026, OpenAI and Anthropic adopted it shortly after. The wave is on. Serving Markdown today is positioning yourself before it becomes an implicit standard, the way <meta name="description"> did twenty years ago.

The HTTP mechanics: Accept and Content-Type

This content negotiation mechanism has been around for a while: it's standardized by the IETF in RFC 7231, ratified in 1999. The principle: the client states its preferences via the Accept header, the server responds with the most appropriate format and signals its choice via Content-Type.

In practice, when a browser visits your site, it typically sends:

GET /my-article HTTP/1.1
Host: www.rankit.fr
Accept: text/html,application/xhtml+xml,...

The server replies with HTML, business as usual. But when a modern AI crawler hits the same URL, it can now send:

GET /my-article HTTP/1.1
Host: www.rankit.fr
Accept: text/markdown
User-Agent: ClaudeBot/1.0 (+claude.ai/bot)

If the server knows how to respond, it serves a Markdown file with Content-Type: text/markdown; charset=utf-8. Otherwise, it falls back to HTML, which works too but burns ten times more tokens at the crawler. That difference between the two behaviors is exactly what Cloudflare's AI Compatibility audit measures, and exactly what we'll enable in .htaccess in the rest of this article.

Cloudflare launched a free audit in spring 2026 called the AI Compatibility Audit. The tool fires a battery of tests at the provided URL to measure whether a site is built for AI agents. Not for Google, not for humans. For agents.

The tests cover 17 points across 5 categories: Discoverability, Content Accessibility, Bot Access Control, API, Auth, MCP, and Commerce. Not every test is enabled on every site, that's what Cloudflare calls a partial scan. For rankit.fr, an editorial site with no public API or built-in payment, I picked only the 7 relevant checks out of the 17 available.

The 17 Cloudflare checks and the 7 selected for rankit.fr

Here's the full map of the audit's checks, useful to understand what follows. On rankit.fr, I enabled the seven checks in the top half (Discoverability, Content, Bot Access Control), and disabled the ten in the bottom half (API/Auth/MCP, Commerce) which don't apply to an editorial site.

The 7-check scope I picked is what you can call the "agent-ready core": the configuration items every website should have, whether editorial, commerce, or app. For SaaS publishers exposing a public API, the natural extension is to also enable apiCatalog (RFC 9727) and oauthDiscovery. For an e-commerce site running Stripe or similar, Commerce checks become relevant the day these HTTP 402-based protocols are supported by payment processors, which is not yet the case in 2026.

The exact URL that reproduces the measurement done on rankit.fr is:

https://isitagentready.com/www.rankit.fr?checks=robotsTxt,
sitemap,linkHeaders,markdownNegotiation,robotsTxtAiRules,
contentSignals,webBotAuth

Initial result on rankit.fr

Before any work, the audit returns a mixed verdict. The global score is 50/100, putting rankit.fr at Level 1: Basic Web Presence. The detail that matters here is in the Content section, which specifically measures Markdown negotiation:

The only indicators I hold as fully certain here are the global score of 50 and the Content 0/1 failure. Cloudflare also exposes per-category sub-scores (Discoverability, Bot Access Control), but their exact computation depends on internal weights the public docs don't detail. I'd rather focus on what's verifiable and on what changes after the intervention, than invent a per-category breakdown that could be misleading.

The exact test: what Cloudflare does

The audit runs an extremely simple test on the site root:

# Request sent by Cloudflare
GET / HTTP/2
Host: www.rankit.fr
Accept: text/markdown

# Expected response: text/markdown
# Actual response before intervention:
HTTP/2 200
content-type: text/html; charset=UTF-8

# Cloudflare verdict:
# "Site does not support Markdown for Agents"

Everything sits in the third line of the response. The server completely ignores the client's preference and serves HTML, even though Markdown was explicitly requested. To Cloudflare, this is a flat failure, and the Content score drops to zero.

Good news: this verdict, this 0/1 score, is fully reversible with a basic Apache configuration. That's what we'll see now, after a detour through the official Cloudflare solution and its pricing.

Cloudflare's official documentation describes Markdown for Agents in very clear terms. When an AI crawler sends Accept: text/markdown on an eligible zone, the Cloudflare network intercepts the request, fetches the original HTML, converts it to Markdown via an internal converter, and serves the Markdown directly to the client. The origin server doesn't change, the work happens at the CDN level.

The setup is genuinely elegant. No conversion to program, no .md file to create, no consistency to maintain between HTML and Markdown: everything is computed on the fly by Cloudflare. The feature even adds a bonus header, x-markdown-tokens, telling the client the estimated number of tokens in the served Markdown version.

Another refinement to know: Cloudflare systematically adds a content-signal: ai-train=yes, search=yes, ai-input=yes response header in Markdown for Agents conversions. This header aligns with the same-named directive that now lives in robots.txt via the contentsignals.org framework. Key point: the Content-Signal directive in robots.txt is the main standardized mechanism, the HTTP header is just a per-request confirmation. We'll come back to this in the bonus section on signals.

How to enable Markdown for Agents on Cloudflare

Three routes are possible: dashboard interface, API, or configuration rules on specific subdomains. The dashboard route is the simplest:

1. Sign in to the Cloudflare dashboard with an account hosting a Pro plan zone or higher
2. Select the zone, navigate to AI Crawl Control
3. Flip the Markdown for Agents toggle
4. Wait a few minutes for propagation

For multi-tenant SaaS using Cloudflare for SaaS with custom hostnames, activation via configuration rule is possible, but requires an advanced subscription with access to custom metadata, which is itself an Enterprise tier.

The actual price of the feature

Here is where it pinches. Cloudflare's documentation is unambiguous:

Three words make all the difference: Pro, Business and Enterprise. The Free plan, which hosts the vast majority of Cloudflare-protected sites worldwide, is explicitly excluded. The at no cost qualifier at the end means at no extra cost on top of the plan, not free. To benefit from Markdown for Agents, you must move your site to a paid plan.

Cloudflare's public pricing as of April 2026 (sources: Cloudflare documentation, Vendr and CheckThat comparisons):

The Pro monthly plan costs $25 instead of $20 annual, that's $300/year. For an average editorial site that doesn't need the advanced WAF, nor Image Resize, nor Pro features, paying $240 to $300 per year to enable a single checkbox is economically absurd. Especially when that checkbox replicates in .htaccess.

When the Cloudflare solution remains relevant

Let's be honest: Cloudflare Pro has real upsides that can justify the price if you need them. For a site combining several of these factors, the subscription pays for itself:

• Massive global traffic requiring advanced CDN cache on 330+ POPs
• Significant attack surface requiring managed OWASP WAF, bot management, exposed credentials check
• Automatic image optimization (Polish, Mirage) on e-commerce or media sites
• Very high page volume where manual HTML→Markdown conversion becomes industrially costly
• Fully static architecture where you have no control over the server layer (Pages, Vercel, Netlify hosting)

For a standard editorial site hosted on shared Apache or LiteSpeed, controlling its .htaccess and adding .md files progressively, these advantages don't apply. And $240/year is the price of an excellent dinner or a month of SEO tools at a peer's. It's not the price of an HTTP configuration toggle.

Let's see how to enable the same feature, at zero cost, on any Apache or LiteSpeed server.

The goal is simple. When an AI crawler visits https://www.rankit.fr/my-article with the Accept: text/markdown header:

1. The server detects the client's Markdown preference
2. It checks that a my-article.md file exists at the root
3. If yes, it serves it with Content-Type: text/markdown; charset=utf-8
4. Otherwise, it serves the normal HTML, no error
5. In every case, it adds Vary: Accept to inform caches

This logic breaks down into three distinct .htaccess blocks, which we'll build and explain one by one before assembling them.

Block 1: the mod_rewrite rewrite rule

This is the heart of the setup. Three cases must be covered to handle every URL form on a modern site: the root /, clean URLs without extension like /my-slug, and URLs with explicit HTML extension like /my-slug.html.

# ================================================
# MARKDOWN CONTENT NEGOTIATION (AI crawlers)
# ================================================
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /

  # Case 1: root "/" with Accept: text/markdown
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/?$
  RewriteCond %{DOCUMENT_ROOT}/index.md -f
  RewriteRule ^$ /index.md [L,E=IS_MARKDOWN:1]

  # Case 2: /slug URL (no extension)
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/([^.]+)$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^([^.]+)$ /$1.md [L,E=IS_MARKDOWN:1]

  # Case 3: /slug.html URL with extension
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/(.+)\.html$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^(.+)\.html$ /$1.md [L,E=IS_MARKDOWN:1]
</IfModule>

Let's break down the directives used, because each one matters.

RewriteCond %{HTTP_ACCEPT} text/markdown [NC]: this is the sentinel condition. The [NC] flag makes the comparison case-insensitive, which matches RFC 7231 on media types. The condition triggers as soon as the string text/markdown appears in the client's Accept header, even in the middle of a list like text/html, text/markdown, */*.

RewriteCond %{REQUEST_URI} ^/?$: targets the root. The pattern accepts both / and /index.html depending on server configuration. For Cases 2 and 3, the regex captures the slug into a numbered backreference %1 used in the next condition.

RewriteCond %{DOCUMENT_ROOT}/%1.md -f: the critical condition. We check the physical existence of the twin .md file on disk, via the -f (regular file) test. If the file doesn't exist, the rule doesn't apply, and the server falls back to standard HTML. No 404, no breakage: the degradation is graceful.

RewriteRule ... [L,E=IS_MARKDOWN:1]: the action. The [L] flag stops processing after this rule (Last). The [E=IS_MARKDOWN:1] flag sets an environment variable that block 2 will read to apply the right Content-Type. This variable is the bridge between mod_rewrite and mod_headers.

Block 2: HTTP response headers

Once the rewrite is done, we need to make sure the response goes out with the right headers. Three headers are mandatory, plus an optional but recommended one.

# Headers for .md files served via negotiation
<IfModule mod_headers.c>
  Header set Content-Type "text/markdown; charset=utf-8" env=IS_MARKDOWN
  Header set X-Content-Negotiation "markdown-served" env=IS_MARKDOWN

  # Vary: Accept critical for CDN caches (LiteSpeed, Cloudflare)
  <FilesMatch "\.(html|md)$">
    Header append Vary "Accept"
  </FilesMatch>
</IfModule>

Header set Content-Type "text/markdown; charset=utf-8" env=IS_MARKDOWN: we force the right MIME type when the IS_MARKDOWN variable has been set by block 1. The charset=utf-8 qualifier matters for content with accented characters.

Header set X-Content-Negotiation "markdown-served" env=IS_MARKDOWN: a very handy debug header. It lets you verify at a glance, via browser dev tools or curl -I, that the server actually served Markdown via negotiation. It's not a standard header, but the X- prefix remains allowed for specific headers.

Block 3: the MIME type for .md files

By default, Apache doesn't know that a .md file is Markdown. It treats it as unknown, or even serves it as application/octet-stream, which forces a download. So we need to teach it.

# MIME configuration for .md (text/markdown, not text/plain)
<IfModule mod_mime.c>
  AddType text/markdown .md
  AddCharset UTF-8 .md
</IfModule>

# Inline reading (no forced download)
<FilesMatch "\.md$">
  Header set Content-Disposition "inline"
</FilesMatch>

# Prevent Google indexing (AI crawlers ignore this header)
<FilesMatch "\.md$">
  Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch>

AddType text/markdown .md: the key directive. It's what aligns the MIME type with what Cloudflare and every AI crawler expect. Many online tutorials wrongly recommend text/plain. The Cloudflare test only validates text/markdown. Don't get this wrong.

Content-Disposition: inline: prevents the browser from offering download when a human accesses a .md URL directly. The page then displays as plain text, consistent with a Markdown source.

X-Robots-Tag: noindex, nofollow: the clever detail. Google indexes .md files if it finds them, which would create duplicate content. So we ask it not to index. But AI crawlers ignore X-Robots-Tag: they don't index in Google's sense, they consume content. The noindex directive does not prevent them from using the served Markdown pages. Result: Google stays on the HTML, AI agents get their Markdown version, everyone is happy.

The twin-file rule: how it fits together

Before assembling everything, an important pedagogical point. The .htaccess doesn't convert anything on the fly: it acts as a switcher between two files that must coexist at the site root. For every published page, you maintain two versions of the same content, sharing the same slug.

Take the URL https://www.rankit.fr/livre-blanc-geo-greenred-2026. For content negotiation to work, two files must physically exist on the server:

The .htaccess mechanism can be described in one sentence: if the request arrives with Accept: text/markdown and the .md file exists, the server serves the .md; in any other case, the server serves the .html. Degradation is graceful: a missing .md never prevents HTML from being served.

Special case of the site root: for https://www.rankit.fr/ to return Markdown to AIs, you need an index.md next to the index.html. The .htaccess handles this root as a dedicated case in its first rewrite rule.

In practice, on rankit.fr, the file tree looks like this:

www.rankit.fr/
├── .htaccess
├── index.html              # humans + Googlebot
├── index.md                # AI crawlers
├── livre-blanc-geo-greenred-2026.html
├── livre-blanc-geo-greenred-2026.md
├── markdown-for-agents-apache-htaccess-free.html
├── markdown-for-agents-apache-htaccess-free.md
├── brave-search-grail-22-signaux-moteur-ia.html
├── brave-search-grail-22-signaux-moteur-ia.md
└── ...

This one-to-one matching may feel heavy at first, but it has a virtue: the Markdown version stays under your control. You can enrich it, trim it, tailor it to LLM context, whereas Cloudflare auto-converts HTML without giving you any say. For high-volume editorial sites, conversion can be automated via script (see the dedicated section below).

Final assembly: the ultimate 30 lines

Here are the three blocks assembled, ready to paste into your existing .htaccess. Place them after your HTTPS/www rules but before your application rules.

# ================================================================
# MARKDOWN CONTENT NEGOTIATION: Markdown for Agents in pure Apache
# Compatible with Apache 2.4+, LiteSpeed, O2switch, Hostinger, OVH
# ================================================================

# --- Block 1: rewrite if Accept: text/markdown ---
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /

  # Case 1: root "/"
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/?$
  RewriteCond %{DOCUMENT_ROOT}/index.md -f
  RewriteRule ^$ /index.md [L,E=IS_MARKDOWN:1]

  # Case 2: /slug URL
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/([^.]+)$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^([^.]+)$ /$1.md [L,E=IS_MARKDOWN:1]

  # Case 3: /slug.html URL
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/(.+)\.html$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^(.+)\.html$ /$1.md [L,E=IS_MARKDOWN:1]
</IfModule>

# --- Block 2: HTTP headers ---
<IfModule mod_headers.c>
  Header set Content-Type "text/markdown; charset=utf-8" env=IS_MARKDOWN
  Header set X-Content-Negotiation "markdown-served" env=IS_MARKDOWN
  <FilesMatch "\.(html|md)$">
    Header append Vary "Accept"
  </FilesMatch>
</IfModule>

# --- Block 3: MIME and .md configuration ---
<IfModule mod_mime.c>
  AddType text/markdown .md
  AddCharset UTF-8 .md
</IfModule>

<FilesMatch "\.md$">
  Header set Content-Disposition "inline"
  Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch>

Thirty effective lines, comments excluded. Thirty lines that do exactly the work Cloudflare bills $240/year for. Thirty lines that pass the AI Compatibility audit with a perfect Content score. And thirty lines you keep under your control, editable at will, with no external dependency.

The test takes no more than a minute. Open a terminal on your machine, and run the four commands below one after the other.

Test 1: a human must always get HTML

First crucial check: content negotiation must not break anything for regular visitors. A standard browser sends Accept: text/html, ... and must get HTML.

# Command to run
curl -I https://www.rankit.fr/

# Expected response (key fragments)
HTTP/2 200
content-type: text/html; charset=UTF-8
vary: Accept

The content-type must remain text/html, and the vary: Accept header must be present. If not, block 2 of the .htaccess is not being applied correctly.

Test 2: an AI crawler must get Markdown

This is the critical test, the one Cloudflare runs in its audit. We simulate an AI crawler by forcing the Accept header.

# Command to run
curl -I -H "Accept: text/markdown" https://www.rankit.fr/

# Expected response
HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept
x-content-negotiation: markdown-served

Three elements to validate: content-type: text/markdown; charset=utf-8 (the Cloudflare criterion), vary: Accept (cache safety), and the debug header x-content-negotiation: markdown-served which confirms block 2 was triggered by the environment variable.

Test 3: a specific article in AI mode

If the root works, verify that a specific article works too. That's the most common case in production.

# Clean URL without extension
curl -I -H "Accept: text/markdown" \
  https://www.rankit.fr/livre-blanc-geo-greenred-2026

# URL with .html extension (must also work)
curl -I -H "Accept: text/markdown" \
  https://www.rankit.fr/livre-blanc-geo-greenred-2026.html

Both URL variants must return text/markdown. If either fails, check that Cases 2 and 3 of block 1 are both present in your .htaccess.

Test 4: inspecting the served Markdown content

Final check: fetch the actual body to confirm we're really getting clean, well-formed Markdown.

# Fetch the response body in Markdown mode
curl -H "Accept: text/markdown" https://www.rankit.fr/ | head -30

# Expected output: clean Markdown
# Rankit.fr: SEO and GEO Consultant

Frédéric Jézégou, senior freelance SEO based in Pornichet...

## Services offered

- **Technical SEO audit**
- **GEO strategy**
- **Presence in AI engines**
...

You should see readable Markdown: headings with hash signs, lists with dashes, bold with double asterisks. If you see raw HTML with tags, block 1 didn't rewrite the URL and the request hit the default HTML file. If you see text without any formatting, your .md file is likely empty or malformed.

A few minutes after pushing the new .htaccess version to O2switch production, I re-ran the Cloudflare AI Compatibility audit on the same URL. Then twice more after each improvement. The result tells itself in four numbered tiers.

Tier 2: from 50 to 67 with the .htaccess

The technical test detail, captured after the change, shows that the GET / Accept: text/markdown request now returns content-type: text/markdown; charset=utf-8, with Cloudflare's positive verdict: "Response content-type is text/markdown; charset=utf-8, site supports markdown negotiation". Content section went from 0/1 to 1/1, a +17 point gain on the global score.

Tier 3: 83 and Level 5 Agent-Native after adding the robots.txt

The story doesn't stop at 67. A few minutes after that first audit, I pushed to production the robots.txt enriched with the Content-Signal: search=yes, ai-train=yes, ai-input=yes directive described in section 8 of this article. New audit scan, and the result makes a spectacular jump: 67 → 83, and the tier goes directly from Basic Web Presence to Agent-Native, the highest classification in Cloudflare's system.

The jump has a simple explanation. The Content-Signal directive in robots.txt, combined with the Allow: / mention and the absence of explicit blocking for legitimate AI crawlers, takes the Bot Access Control section from partial to 2/2. The global calculation rises to 83.

The methodological lesson is interesting. The .htaccess alone, despite all its technical ingenuity, capped at 67 because it only handles one criterion: Markdown content negotiation. It's the addition of the declarative robots.txt, which talks to the crawler's brain and not just its HTTP client, that unlocks the higher tier. The two levers are complementary, not redundant.

Tier 4: 100/100 after adding RFC 8288 Link headers

One last criterion remains to hit the maximum. The Discoverability section caps at 2/3 because the homepage is missing a Link: HTTP header compliant with RFC 8288 (Web Linking). It's a standardized mechanism that lets AI agents discover, on the very first HTTP call, the machine-readable resources describing the site: sitemap, robots.txt, API documentation, llms.txt, etc.

On rankit.fr, I added three lines to the mod_headers block of the .htaccess pointing to the two machine-readable resources my site already exposes: /sitemap.xml and /robots.txt. The subsection on Link headers in section 9 details the correct syntax and the technical gotcha to avoid under LiteSpeed. Once deployed, new scan: 83 → 100, Discoverability, Content and Bot Access Control sections all at their maximum.

What's next after 100/100?

Once you hit 100 on the Content profile, two routes remain to push further on other audit profiles. Cloudflare offers more demanding profiles like full, which enable all 17 checks instead of 7, testing additional criteria like API, Auth, MCP & Skill Discovery and Commerce. For an editorial site like rankit.fr, these are out of scope. But for a SaaS publisher, an e-commerce site, or a platform offering a public API, they're concrete levers.

• API catalog (rel="api-catalog"): add a /.well-known/api-catalog file compliant with RFC 9727 and a Link header pointing to it. The official mechanism to advertise the existence of a public API to agents.
• llms.txt: add an llms.txt file at the site root, compliant with the emerging norm, that aggregates the site's main documentation in agent-friendly Markdown form.

But what just happened on rankit.fr is already the full delivery of the promise "replicate for free what Cloudflare charges $240/year for". Perfect score, maximum tier, with zero external dependency, in under an hour of configuration. Let's be clear on the economic stakes of this result.

Let's do the math calmly. The question is not to bash Cloudflare, which ships a technically excellent product useful for many use cases. The question is to measure what we spend to flip this specific checkbox, and what we get in return.

Single-site case: a standard editorial site

Assumption: a classic editorial site (professional blog, consultant homepage, media site), hosted on shared Apache or LiteSpeed, wanting to enable Markdown for Agents to stay competitive on GEO.

Over 5 years, the minimum gap is $1,200. At current rates, that's about €1,100 staying in the company's cash instead of going to a CDN subscription. For a freelance or SMB, that's three good monitors, a training week, or the budget of a small redesign.

Multi-site case: a consultant or agency

For a freelance like me running multiple domains, or for an agency hosting client sites, the gap multiplies linearly. My own web properties: rankit.fr, dicocitations.com, vultifrine.com, greenred.fr, four active domains.

For an independent SEO/GEO consultant or a small agency with four clients on shared hosting, that's about $4,800 (~€4,400) that can be reinvested elsewhere: a GEO tracking tool, a premium AI subscription for content production, or simply a healthier cash position.

The hidden cost of Cloudflare dependency

Beyond the monthly check, there's a cost that doesn't show up in pricing tables: dependency on a third-party provider for basic HTTP features.

If tomorrow Cloudflare changes its pricing, adds usage quotas, or decides to bump Markdown for Agents into Premium only, you take the decision with no recourse. If Cloudflare has a global outage (it happened in 2020, 2022 and 2024), your content negotiation falls with them. If you want to add business logic on the conversion (custom cache, content transformation, access control), you're capped by their API.

With .htaccess, the code is yours. It runs in your stack, under your control, editable in milliseconds. You can add whatever you want, as we'll see now with two bonus headers Cloudflare doesn't offer by default.

Markdown for Agents is the foundation. But once the mechanism is in place, you can enrich the setup with two complementary signals that speak an even more precise language to modern AI agents. The first, x-markdown-tokens, is an HTTP header you can compute in PHP when serving the .md. The second, Content-Signal, is a directive in your robots.txt that has nothing to do with .htaccess but completes the picture perfectly. Let's look at both.

The x-markdown-tokens header

This header, popularized by Cloudflare, tells the client the estimated number of tokens contained in the Markdown response. For an AI agent managing a limited context window (e.g. 200,000 tokens on Claude), knowing that a page will consume 4,500 tokens helps decide upfront how many sources to fetch before running inference.

Token count is an approximation. The commonly accepted rule of thumb for English is about 1 token per 4 characters. You can estimate it server-side with a lightweight PHP script.

<?php
// PHP router serving a .md with token computation
// Use instead of Block 1 if you want x-markdown-tokens
$accept = $_SERVER['HTTP_ACCEPT'] ?? '';
$uri    = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH);
$slug   = trim(preg_replace('#\.html$#', '', $uri), '/');

if (stripos($accept, 'text/markdown') !== false) {
    $md_path = __DIR__ . '/' . $slug . '.md';

    if (is_file($md_path)) {
        $content = file_get_contents($md_path);
        // Estimate: ~1 token per 4 characters in English
        $tokens  = (int) round(mb_strlen($content, 'UTF-8') / 3.5);

        header('Content-Type: text/markdown; charset=utf-8');
        header('Vary: Accept');
        header('X-Markdown-Tokens: ' . $tokens);
        header('X-Content-Negotiation: markdown-served');
        header('Last-Modified: ' . gmdate('D, d M Y H:i:s', filemtime($md_path)) . ' GMT');
        echo $content;
        exit;
    }
}

// Normal HTML fallback
$html_path = __DIR__ . '/' . $slug . '.html';
if (is_file($html_path)) {
    header('Vary: Accept');
    readfile($html_path);
    exit;
}

http_response_code(404);

This script replaces Block 1 when you want a precise computation. The .htaccess rule that routes to this file becomes:

# If Accept: text/markdown, route to md-router.php
RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^.]+)$ /md-router.php [L]

The Content-Signal directive in robots.txt

The Content Signals framework was kicked off by Cloudflare in September 2025 to let publishers express, in machine-readable form, their preferences about how AI consumes their content. First essential point: Content-Signal is not an HTTP header, it's a directive added to your robots.txt file. It's an extension of the Robots Exclusion Protocol, not an HTTP response header. Compliant AI crawlers look for this in robots.txt, where they're used to reading User-agent, Allow, and Disallow.

The framework defines three main signals:

• search: allow indexing and display in AI search results (values yes, no)
• ai-input: allow use as real-time agentic context, RAG or grounding (values yes, no)
• ai-train: allow use to train or fine-tune models (values yes, no)

On Cloudflare Free domains that have no existing robots.txt, Cloudflare automatically inserts the default Content-Signal: search=yes, ai-train=no directive. Domains that already have a robots.txt, like rankit.fr, are left untouched. So you have to add the directive yourself, and that's precisely what suits me better: I decide my policy, not Cloudflare in my place.

The permissive policy on rankit.fr

On rankit.fr, I went for the maximum permissive policy. My content exists to be read, indexed, cited, and incidentally to contribute to improving the models that cite me. It's an owned editorial decision, and it may differ for a paid media site or a scientific publication under embargo. Here's the snippet to paste in robots.txt.

# Content Signals policy (contentsignals.org)
# As a condition of accessing this site, you accept the following signals:
# (a) if content-signal = yes, you may collect the content for the matching use
# (b) if content-signal = no, you may not collect the content for the matching use
# (c) if the signal is absent, the site operator neither grants nor restricts use
#
# Meaning of the three signals:
# search: indexing and display in search results
# ai-input: use as real-time context for AI models (RAG, grounding)
# ai-train: use to train or fine-tune AI models

User-agent: *
Content-Signal: search=yes, ai-train=yes, ai-input=yes
Allow: /

Sitemap: https://www.rankit.fr/sitemap.xml

Comments are not mandatory but recommended by contentsignals.org: they make the policy readable by a human auditing your file, and they spell out the semantics of the three signals for anyone who doesn't yet know the framework. You can also generate a more complete custom version on contentsignals.org with their official tool.

For more restrictive policies, here are three examples matching common use cases:

# Case A: maximum permissive (rankit.fr choice)
# For sites that want to maximize their visibility in AI engines
Content-Signal: search=yes, ai-train=yes, ai-input=yes

# Case B: only refuse training (Cloudflare managed default)
# For sites that accept citation but refuse their content
# being used to train foundation models
Content-Signal: search=yes, ai-train=no, ai-input=yes

# Case C: classic search only
# For subscription media sites, embargoed scientific publications,
# sensitive legal content
Content-Signal: search=yes, ai-train=no, ai-input=no

Important note: the content-signal HTTP header is not the main mechanism

Technical clarification to avoid mistakes. When Cloudflare converts HTML to Markdown via Markdown for Agents, its HTTP response does include a content-signal: ai-train=yes, search=yes, ai-input=yes header. This HTTP header is consistent with what's declared in robots.txt, but it's not what crawlers check first. Compliant crawlers read robots.txt first, and the HTTP header is only the per-request confirmation of the global policy.

Concretely: if you add Header set Content-Signal to your .htaccess but not the directive in robots.txt, crawlers won't see your policy at the moment they decide whether to collect your content. The rule is therefore: the declaration lives in robots.txt, the HTTP header is secondary and rather aimed at clients that have already fetched the content.

The Lighthouse gotcha: why your SEO score may drop to 92

Let's be honest about a side effect you'll hit when deploying this directive. The moment you add Content-Signal: to your robots.txt, your Lighthouse SEO score will likely drop from 100 to 92. That's exactly what happened on rankit.fr. Search Console may also show a "Syntax not understood" warning on this line.

The reason is mechanical. Lighthouse runs eight automated SEO audits, each weighing roughly 12.5 points on the final score. One of those audits is "robots.txt is valid", which checks that every directive belongs to a known whitelist (User-agent, Allow, Disallow, Sitemap, Crawl-delay, etc.). Since Content-Signal isn't yet on that whitelist as of this writing, Lighthouse flags the directive as "Unknown directive", the audit fails, and the score drops by about eight points.

Cloudflare explicitly acknowledges the issue in its own documentation:

And the Chrome community has actually opened Pull Request #16767 on the Lighthouse repository to add Content-Signal to the safelist of recognized directives. The PR was opened in October 2025 and remains under review at the time of this publication. Once merged, the warning will disappear automatically and your score will return to 100 without you touching your robots.txt.

But the Lighthouse SEO score is not a Google ranking signal

Before panicking and removing the directive to recover your 100, read what John Mueller from Google's Search Relations team says regularly on this topic: the Lighthouse SEO score is not a ranking factor. Google doesn't use this score to decide where your pages rank. It's a diagnostic tool aimed at developers, not a ranking signal.

The unit of measure that really matters is Googlebot's actual behavior and effective indexing in Search Console, not an aggregate score computed by a third-party tool. Proof: on rankit.fr, despite the 92 Lighthouse score, Google indexing works perfectly, crawl is stable, and no URL has been deindexed because of this directive.

The trade-off becomes readable. On one side, a cosmetic Lighthouse score going from 100 to 92, with no impact on real Google ranking. On the other, a standardized GEO signal that expresses your content usage policy to 3.8 million Cloudflare domains and every compliant AI agent. The choice seems clear to me: the real GEO gain outweighs the cosmetic score, especially since the latter is a temporary regression whose fix is already underway on the Lighthouse side.

Your three strategies for this dilemma

For sites where the Lighthouse score has contractual or commercial weight (client presentations, agency audits, automated monitoring), here are the available options, from most pedagogical to most defensive:

• Keep the directive and explain. That's rankit.fr's choice. If someone challenges you on the 92 score, you show them the pending Lighthouse PR, you cite John Mueller, and you explain that you're about six months ahead of the safelist. It's a technical maturity argument, not a fault.
• Keep the directive and self-document the robots.txt. You add a comment in your robots.txt with an explanatory note that contextualizes the directive and anticipates the criticism. Anyone reading the file immediately understands why it's there, and where we are in the standardization cycle.
• Temporarily remove the directive. If you run an institutional site with high commercial stakes on aggregate scores, you can decide to wait for Lighthouse to merge the PR before deploying Content-Signal. You lose the marginal GEO signal but preserve your 100. It's a defensive choice, not a mistake.

On rankit.fr, I went with the hybrid strategy between the first two options: the directive is in place, and the robots.txt includes an explanatory comment. Lighthouse SEO score is 92, but Cloudflare AI Compatibility is 100/100, and Google ranking is intact. Three measurements pointing in different directions, together telling the story of a site that owns being ahead on the GEO transition.

The principle: advertise your machine-readable resources

Link headers have existed since RFC 5988 in 2010, updated by RFC 8288 in 2017. The concept is simple: add one or more Link: HTTP headers to the homepage response, pointing to related resources with a typed relation. The client reads these headers before even parsing the HTML, and immediately knows which other resources to consult.

For AI agents, it's a central discovery mechanism. Instead of scraping your entire tree to find the sitemap or robots.txt, they read your root's Link headers and know where to go. The industrial-strength version of "here's the site map, help yourself".

The Cloudflare AI Compatibility audit tests exactly this criterion under the name linkHeaders, in the Discoverability section. To validate it, the response to GET / must contain at least one Link: header with a relation recognized by the IANA Link Relations Registry and identified as agent-useful. The four main relations are:

For an editorial site like rankit.fr that doesn't expose a public API, the relation to use is describedby. It points to the resources describing the site, namely the XML sitemap and the robots.txt. For a SaaS site or an API publisher, you would combine describedby with api-catalog or service-desc.

The .htaccess code to add

Three lines to insert in the mod_headers block of the .htaccess, after the existing Markdown for Agents blocks:

# Block 4: RFC 8288 Link headers for agent discovery
# (linkHeaders criterion of the Cloudflare AI Compatibility audit)
<IfModule mod_headers.c>
    Header set Link '</sitemap.xml>; rel="describedby"; type="application/xml"'
    Header append Link '</robots.txt>; rel="describedby"; type="text/plain"'
</IfModule>

The Header set Link directive defines the first header, the Header append Link directive adds a second one separated by a comma (equivalent to two distinct Link: headers in the HTTP response, RFC-compliant). The result served on the root looks like:

HTTP/2 200
content-type: text/html; charset=UTF-8
link: </sitemap.xml>; rel="describedby"; type="application/xml"
link: </robots.txt>; rel="describedby"; type="text/plain"

The LiteSpeed gotcha: double quotes escaped with backslash

Here's a gotcha that cost me a failed audit on my first try. The classic Apache syntax for values with spaces uses external double quotes and internal double quotes escaped with backslashes:

# This syntax is valid Apache but broken on LiteSpeed
Header set Link "</sitemap.xml>; rel=\"describedby\"; type=\"application/xml\""

Under standard Apache 2.4, this line works and the served HTTP header is correctly decoded. But under LiteSpeed (the default server on most French shared hosting like O2switch, Infomaniak, or OVH Performance), the parser interprets the \" literally and includes them in the served value. Result in the HTTP response:

link: </sitemap.xml>; rel=\"describedby\"; type=\"application/xml\"

The backslashes are preserved literally, and RFC 8288 parsers (including Cloudflare's audit) fail to recognize the describedby relation because it's surrounded by \" instead of ". The audit returned the verdict "Link headers present but no agent-useful relation types found", which was technically accurate: no relation was parsable.

The solution is elegant. Apache mod_headers accepts single quotes as value delimiters, exactly like a Unix shell. Inside single quotes, double quotes are preserved as-is with no escaping required. This is the official syntax used in the reference Apache documentation for the IIIF framework.

Validation: getting Discoverability from 2/3 to 3/3

Once the snippet is deployed, the test runs in two steps. First, curl verification to make sure the served HTTP headers are clean:

# Output must contain no backslash
curl -I https://www.rankit.fr/ | grep -i "link:"

Then re-run the Cloudflare audit with the full profile so the Discoverability section is tested completely:

Run the Rankit audit (7 checks) on isitagentready.com

On rankit.fr, the final audit result is unequivocal: global score 100/100, Discoverability 3/3, Content 1/1, Bot Access Control 2/2, Level 5 Agent-Native tier confirmed. The full delivery of this article's promise. Three configuration files, fewer than forty lines total, zero external service, and the site clinches the maximum score on the Cloudflare AI Compatibility audit across the seven tested criteria.

Not everyone is on Apache. For Nginx sites (often used on VPS Hetzner, OVH Cloud, AWS Lightsail, or in modern stacks with PHP-FPM), the same logic transposes elegantly. Nginx has no direct equivalent of .htaccess; configuration happens in the site file (typically /etc/nginx/sites-available/your-site.conf).

# =============================================================
# At http {} level: map that detects Accept: text/markdown
# =============================================================
map $http_accept $wants_markdown {
    default            0;
    "~*text/markdown" 1;
}

# =============================================================
# At server {} level: MIME declaration and serving logic
# =============================================================
server {
    listen 443 ssl http2;
    server_name www.example.com;
    root /var/www/html;

    # MIME type for .md files
    types {
        text/markdown md;
    }
    charset utf-8;

    location / {
        # If Accept: text/markdown AND .md exists: serve it
        if ($wants_markdown) {
            rewrite ^/$ /index.md break;
            rewrite ^/(.+?)(\.html)?$ /$1.md break;
        }

        # Normal HTML fallback
        try_files $uri $uri.html $uri/ =404;
    }

    # Headers for served .md files
    location ~ \.md$ {
        add_header Content-Type "text/markdown; charset=utf-8" always;
        add_header Vary "Accept" always;
        add_header X-Content-Negotiation "markdown-served" always;
        add_header X-Robots-Tag "noindex, nofollow" always;
        add_header Content-Disposition "inline" always;
    }

    # Vary: Accept also on .html (cache consistency)
    location ~ \.html$ {
        add_header Vary "Accept" always;
    }
}

Three Nginx-specific points of attention.

The map block must be placed at the http {} level in nginx.conf, not inside the server block. That's an Nginx quirk: map directives are not allowed in the server context.

The if directive inside location is known for its pitfalls (if is evil), but here the use with rewrite ... break is safe and idiomatic. No risk of loops.

Reload the configuration after editing: sudo nginx -t to validate the syntax, then sudo systemctl reload nginx. On a properly configured VPS, the operation is non-disruptive.

The whole .htaccess mechanism rests on the existence of twin .md files alongside the .html files. If an article is named livre-blanc-geo-greenred-2026.html, you need a livre-blanc-geo-greenred-2026.md file at the root for content negotiation to work on this URL.

The logical question becomes: how to produce these Markdown files? Three approaches exist, each with its own upsides.

Manual approach during writing

This is the route I prefer for my important articles. As I write an article's HTML, I also write its source Markdown. Markdown effectively becomes the master format, and HTML is generated from it (by me, manually, or via a converter).

Upsides: maximum quality of served Markdown (no noise, structure faithful to authorial intent). Downsides: double effort at writing time, and risk of HTML/MD desync if you edit one without the other.

Automatic PHP script approach

For sites with many dynamically generated pages (CMS, e-commerce, high-volume editorial sites), automatic conversion at build time or via daily cron is more realistic. Several PHP libraries do the job:

• league/html-to-markdown (Composer): robust, well maintained, supports the main Markdown extensions
• turndown (Node.js, if you have a build pipeline): highly configurable, popular in the JS ecosystem
• pandoc on the CLI: the absolute reference, invoke it in post-build via pandoc -f html -t markdown_strict

The idea: on every deploy, a script walks the folder, reads each .html, extracts the main content (no nav, no sidebar, no footer), converts it to Markdown and writes the twin .md. The result is mechanically up-to-date.

Online converter approach: rankit.fr/md/

For editorial sites that don't feel like installing Composer or Pandoc, the simplest route is a converter that takes an HTML URL as input and returns clean Markdown as output. That's exactly what I built on rankit.fr/md/, after noticing existing tools were either verbose (kept too many tags) or too stripped (lost internal links).

The Rankit converter does three things you can't easily find elsewhere:

• It aggressively cleans non-semantic elements (inline CSS, scripts, ARIA, navigation blocks)
• It preserves semantic structure: headings, lists, quotes, code, links and tables in their standard Markdown form
• It respects encoding (UTF-8 with typographic apostrophes, accented characters, and other quirks)

The tool is free, no signup, designed to be the tool I wish I'd found when I started this project on rankit.fr. You paste your HTML page URL, you get formatted Markdown, you save it next to the .html. Three clicks, and your page is ready for content negotiation.

This is the consolidated and ready-to-deploy version. Each of the four blocks has been built and explained one by one in the previous sections. Here they are assembled in the right order, with detailed comments, ready to be pasted as-is into a fresh .htaccess at the site root.

To place after your HTTPS/www redirection rules but before any application rules (WordPress, Laravel, custom routing). Compatible Apache 2.4+, LiteSpeed (O2switch, Hostinger, Infomaniak, OVH Performance), and any server supporting .htaccess.

# ================================================================
# COMPLETE .HTACCESS · MARKDOWN FOR AGENTS · CLOUDFLARE 100/100
# ================================================================
# Tested in production on rankit.fr (O2switch / LiteSpeed)
# Cloudflare AI Compatibility audit: 100/100 · Level 5 Agent-Native
# Compatible Apache 2.4+, LiteSpeed, Nginx via separate translation
# ================================================================
#
# BLOCK 1 · mod_rewrite : Accept: text/markdown -> .md twin file
# BLOCK 2 · mod_headers : Content-Type, Vary, X-Content-Negotiation
# BLOCK 3 · mod_mime    : MIME type, charset, inline, noindex
# BLOCK 4 · mod_headers : RFC 8288 Link headers (Discoverability)
# ================================================================

# --- BLOCK 1: rewrite if Accept: text/markdown ---
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /

  # Case 1: root "/" with Accept: text/markdown
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/?$
  RewriteCond %{DOCUMENT_ROOT}/index.md -f
  RewriteRule ^$ /index.md [L,E=IS_MARKDOWN:1]

  # Case 2: clean URL /slug (no extension)
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/([^.]+)$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^([^.]+)$ /$1.md [L,E=IS_MARKDOWN:1]

  # Case 3: legacy URL /slug.html with explicit extension
  RewriteCond %{HTTP_ACCEPT} text/markdown [NC]
  RewriteCond %{REQUEST_URI} ^/(.+)\.html$
  RewriteCond %{DOCUMENT_ROOT}/%1.md -f
  RewriteRule ^(.+)\.html$ /$1.md [L,E=IS_MARKDOWN:1]
</IfModule>

# --- BLOCK 2: HTTP response headers ---
<IfModule mod_headers.c>
  Header set Content-Type "text/markdown; charset=utf-8" env=IS_MARKDOWN
  Header set X-Content-Negotiation "markdown-served" env=IS_MARKDOWN

  # Vary: Accept (mandatory) for CDN caches (LiteSpeed, Cloudflare)
  <FilesMatch "\.(html|md)$">
    Header append Vary "Accept"
  </FilesMatch>
</IfModule>

# --- BLOCK 3: MIME and .md configuration ---
<IfModule mod_mime.c>
  AddType text/markdown .md
  AddCharset UTF-8 .md
</IfModule>

<FilesMatch "\.md$">
  Header set Content-Disposition "inline"
  Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch>

# --- BLOCK 4: RFC 8288 Link headers (Discoverability 3/3) ---
# IMPORTANT: single quotes mandatory under LiteSpeed
# (escaping with \" breaks the parsing)
<IfModule mod_headers.c>
  Header set Link '</sitemap.xml>; rel="describedby"; type="application/xml"'
  Header append Link '</robots.txt>; rel="describedby"; type="text/plain"'
</IfModule>

Once this .htaccess is deployed at the site root and the robots.txt contains the Content-Signal directive, the Cloudflare AI Compatibility audit returns:

• Global score: 100/100 · Level 5 Agent-Native
• Discoverability: 3/3 · sitemap, robots.txt, and Link headers all detected
• Content: 1/1 · Markdown negotiation supported
• Bot Access Control: 2/2 · Content-Signal policy declared

Three files, around forty lines of code, zero CDN subscription. The same result Cloudflare bills at $240/year minimum on the Pro plan, here in pure Apache, under your full control, with the freedom to extend the policy whenever you want.

Our previous sections covered three formats served via content negotiation: HTML for humans, Markdown for AI crawlers, and JSON via the /api/post/{slug}.json endpoint. The natural next question becomes: how does an agent discover the JSON endpoint exists?

Three discovery routes coexist today, with different agent-readiness levels:

• The <link rel="alternate" type="application/json"> in the HTML <head>: the agent must download and parse the HTML first.
• The alternateFormat field inside the JSON itself: the agent must already have fetched the JSON to find it.
• The RFC 9727 well-known route: the agent fetches a single predictable URL and gets the full API catalog, no HTML parsing required.

The third route is the most agent-native. It's also the youngest : the standard is RFC 9727, ratified by the IETF in June 2025 by Kevin Smith (Vodafone) after thirteen revisions and two years of refinement.

The principle: a predictable well-known URI

The RFC defines a single well-known URI: /.well-known/api-catalog. Any compliant client can GET this URL on any domain and immediately know which APIs are exposed and where to find them. The mechanism leans on RFC 8615 (Well-Known URIs) and the RFC 9264 Linkset JSON format.

Three mandatory technical points to satisfy the RFC:

• The endpoint MUST be served over HTTPS (security recommendation §8 of the RFC).
• The response MUST be available in the application/linkset+json format defined by RFC 9264.
• The Content-Type SHOULD include the profile parameter profile="https://www.rfc-editor.org/info/rfc9727" so consumers can identify the document as a valid api-catalog.

The linkset.json file

The Linkset format (RFC 9264) is straightforward: a JSON document with a top-level linkset array, each entry containing an anchor URL and a set of typed link relations. For rankit.fr, the file is placed at the site root as /linkset.json.

{
  "linkset": [
    {
      "anchor": "https://www.rankit.fr/api/post/",
      "service-doc": [
        {
          "href": "https://www.rankit.fr/markdown-for-agents-apache-htaccess-free",
          "type": "text/html",
          "hreflang": ["en"],
          "title": "API documentation (English)"
        }
      ],
      "service-meta": [
        {
          "href": "https://www.rankit.fr/api/post/markdown-for-agents-apache-htaccess-free.json",
          "type": "application/json",
          "title": "Article JSON representation (Schema.org TechArticle)"
        }
      ],
      "license": [
        { "href": "https://creativecommons.org/licenses/by-sa/4.0/" }
      ]
    }
  ]
}

The link relations used (service-doc, service-meta, license) come from the IANA Link Relations registry. Their meaning is standardized: service-doc points to human-readable API documentation, service-meta to additional metadata about the API endpoints (here, the JSON content negotiation route).

The .htaccess code (Block 6)

Three things to do at the server level: route /.well-known/api-catalog to the physical file, serve the right Content-Type, and announce the catalog via a Link header on every HTML page.

# --- Block 6: api-catalog (RFC 9727) ---
<IfModule mod_rewrite.c>
  RewriteCond %{REQUEST_URI} ^/\.well-known/api-catalog/?$ [NC]
  RewriteRule ^\.well-known/api-catalog/?$ /linkset.json [L,E=IS_LINKSET:1]
</IfModule>

<IfModule mod_headers.c>
  Header set Content-Type "application/linkset+json; profile=\"https://www.rfc-editor.org/info/rfc9727\"" env=IS_LINKSET
  Header set Cache-Control "public, max-age=3600" env=IS_LINKSET

  # Announce the catalog via Link header on every HTML page
  <FilesMatch "\.html$|/$">
    Header append Link '</.well-known/api-catalog>; rel="api-catalog"; type="application/linkset+json"'
  </FilesMatch>
</IfModule>

curl validation

Three checks to confirm the deployment works:

# Test 1: the well-known endpoint serves the right Content-Type
curl -I https://www.rankit.fr/.well-known/api-catalog
# Expected: content-type: application/linkset+json; profile="..."

# Test 2: the JSON content is valid and parseable
curl https://www.rankit.fr/.well-known/api-catalog | jq .

# Test 3: the home page advertises the catalog via Link header
curl -I https://www.rankit.fr/ | grep -i "link:"
# Expected, among others: link: </.well-known/api-catalog>; rel="api-catalog"; ...

Honest take on the audit gain

To be fully transparent: deploying the api-catalog does not raise the global score above 100/100. You're already at the maximum. What it changes is the scope of the audit, which expands from three categories (Discoverability, Content, Bot Access Control) to four, with the new API/Auth/MCP/Skill Discovery category also at 100/100.

When you activate the API/Auth/MCP & Skill Discovery category in the audit, Cloudflare runs the apiCatalog check (this section, Block 6) and the agentSkills check (Block 7, see next section). Both pass for rankit.fr, giving a 2/2 on this category and confirming the deployment. The other potential checks documented in early specs (oauthDiscovery, oauthProtectedResource, mcpServerCard, a2aAgentCard, webMcp) are not part of the current audit scope as of April 2026.

So the strategic question becomes: do I need this for an editorial site? For rankit.fr, the answer is nuanced. It's not a SaaS with a public OAuth API, no MCP server, no agent skills exposed in the strict sense of the term. But the JSON output of each article via /api/post/{slug}.json is a public API in the architectural sense. Declaring it via the standardized RFC 9727 path is technically rigorous, costs nothing, and signals technical maturity to discerning agents.

The Agent Skills format originated with Anthropic and has been adopted by OpenAI Codex, OpenCode, and now Cloudflare. A skill is a folder containing a SKILL.md file with a YAML frontmatter (name and description fields) and Markdown instructions. AI agents can fetch the index, discover available skills, and progressively load the ones relevant to the task at hand.

For an editorial site like rankit.fr, exposing skills means publishing structured, machine-readable expertise that any compliant agent can consume directly. It's a step beyond articles meant for human reading: skills are operational knowledge an agent can apply immediately.

Folder structure

The Cloudflare draft specifies a predictable hierarchy under /.well-known/skills/:

/.well-known/skills/
├── index.json                              # Required: skills index
├── htaccess-markdown-negotiation/
│   └── SKILL.md                            # Skill 1: Markdown content negotiation
├── content-signal-robots-txt/
│   └── SKILL.md                            # Skill 2: Content-Signal robots.txt
└── rfc8288-link-headers-litespeed/
    └── SKILL.md                            # Skill 3: Link headers under LiteSpeed

The index.json file

The index lists every available skill with its name, description, and the files in its folder. Each skill name must follow the Agent Skills naming spec (1-64 lowercase alphanumeric characters and hyphens, no leading or trailing hyphen).

{
  "skills": [
    {
      "name": "htaccess-markdown-negotiation",
      "description": "Configure an Apache .htaccess (or LiteSpeed) to serve Markdown to AI crawlers via HTTP content negotiation when they send the Accept text/markdown header...",
      "files": ["SKILL.md"]
    },
    {
      "name": "content-signal-robots-txt",
      "description": "Add the Content-Signal directive to a robots.txt file to declare AI usage preferences...",
      "files": ["SKILL.md"]
    },
    {
      "name": "rfc8288-link-headers-litespeed",
      "description": "Configure RFC 8288 Link HTTP response headers in Apache .htaccess (or LiteSpeed)...",
      "files": ["SKILL.md"]
    }
  ]
}

A SKILL.md file

Each SKILL.md starts with a YAML frontmatter (between two --- lines) followed by Markdown instructions. The frontmatter must declare name and description; everything below is freeform Markdown read by the agent when the skill activates.

---
name: htaccess-markdown-negotiation
description: Configure an Apache .htaccess (or LiteSpeed) to serve
  Markdown to AI crawlers via HTTP content negotiation when they
  send the Accept text/markdown header. Use when setting up a
  website to be agent-ready, when implementing Markdown for Agents
  on shared hosting, or when targeting the Cloudflare AI Compatibility
  audit Content score (markdownNegotiation check).
---

# Apache Markdown Content Negotiation for AI Crawlers

## What this skill does

Configures an Apache HTTP server (or LiteSpeed, the default on most
French shared hosting like O2switch, Infomaniak, OVH Performance) to
serve Markdown content to AI crawlers when they send the HTTP header
`Accept: text/markdown`...

## When to use

- Setting up a website to be readable by AI crawlers
- Implementing GEO on Apache or LiteSpeed
- Reducing token consumption for AI agents fetching the site
- Passing the Content score on Cloudflare's `isitagentready.com` audit

## Implementation

[Full Apache .htaccess block, exactly the one from Section 4 of this article]

The .htaccess code (Block 7)

Same approach as Block 6: serve the right Content-Type for the index and for the SKILL.md files, and provide an alias from the older /.well-known/agent-skills/ URL to the official /.well-known/skills/ path so the audit and any future scanners both find the resources at the same location.

# --- Block 7: Agent Skills Discovery ---
<IfModule mod_rewrite.c>
  # Alias: /.well-known/agent-skills/ -> /.well-known/skills/
  RewriteCond %{REQUEST_URI} ^/\.well-known/agent-skills/(.+)$ [NC]
  RewriteRule ^\.well-known/agent-skills/(.+)$ /.well-known/skills/$1 [L]
</IfModule>

<IfModule mod_headers.c>
  # Content-Type for the index
  <Files "index.json">
    Header set Content-Type "application/json; charset=utf-8"
    Header set Cache-Control "public, max-age=3600"
    Header set Access-Control-Allow-Origin "*"
  </Files>

  # Content-Type for SKILL.md files
  <FilesMatch "^SKILL\.md$">
    Header set Content-Type "text/markdown; charset=utf-8"
    Header set Cache-Control "public, max-age=3600"
    Header set Access-Control-Allow-Origin "*"
  </FilesMatch>
</IfModule>

curl validation

# Test 1: index.json is served as application/json
curl -I https://www.rankit.fr/.well-known/skills/index.json
# Expected: content-type: application/json; charset=utf-8

# Test 2: alias /.well-known/agent-skills/ resolves to the same content
curl -I https://www.rankit.fr/.well-known/agent-skills/index.json
# Expected: same response as test 1

# Test 3: SKILL.md is served as text/markdown
curl -I https://www.rankit.fr/.well-known/skills/htaccess-markdown-negotiation/SKILL.md
# Expected: content-type: text/markdown; charset=utf-8

Audit result with the four categories

With Block 6 (api-catalog) and Block 7 (Agent Skills) deployed, the Cloudflare audit can run nine checks across four categories. Direct link to reproduce the rankit.fr scan:

Run the Rankit audit (9 checks) on isitagentready.com

Expected result:

• Global score: 100/100 · Level 5 Agent-Native
• Discoverability: 3/3 · sitemap, robots.txt, Link headers
• Content: 1/1 · Markdown negotiation
• Bot Access Control: 2/2 · Content-Signal policy
• API/Auth/MCP/Skill Discovery: 2/2 · API Catalog + Agent Skills index

This article demonstrates one simple thing. HTTP content negotiation, the mechanism the IETF standardized in 1999 in RFC 7231, is enough to do everything Cloudflare brands as Markdown for Agents. Thirty lines of Apache configuration. No external dependency. No subscription.

Yet, around this elementary mechanism, a narrative has been built in a few months whereby technical GEO would become the prerogative of those who can afford a premium CDN. That's the first thing to correct.

GEO must not turn into a captive market. The promise of AI engines is precisely that a well-built, well-structured, well-documented editorial site can be cited by Claude, ChatGPT or Perplexity on equal footing with major institutional sites. That promise collapses if AI visibility is conditioned on a $240/year subscription that many small businesses and freelances cannot justify.

The second thing to correct is the idea that GEO would be a radically new discipline. 80% of GEO rests on old technical fundamentals: clean semantic HTML, a coherent heading structure, thoughtful internal links, reasonable load speed, and now, correctly configured content negotiation. All of this is done with standard web tools, in the server you already have, in a few hours of well-directed work.

On rankit.fr, the Cloudflare AI Compatibility audit went from 50 to 100/100 in under an hour of actual work, across four progressive tiers: .htaccess Markdown for Agents (50 → 67), robots.txt with Content-Signal (67 → 83), RFC 8288 Link headers (83 → 100). Tier reached: Level 5 Agent-Native, the highest classification in Cloudflare's system. The equivalent of a light turning on a site's dashboard, telling AI agents: yes, you can consume me, and in the format you prefer. For an independent SEO/GEO consultant, this is exactly the kind of improvement I want to be able to reproduce for each of my SMB clients, billing for time and expertise, not for a third-party subscription.

Thirty lines of .htaccess. Zero annual extra cost. Plus 17 Cloudflare audit points in under an hour. That equation works for me, and I think it works for the vast majority of Apache and LiteSpeed sites in the world. The code is in this article. The Markdown converter is online. All the rest is just configuration time you save by following this guide.

And the day you want to go further with a custom Content-Signal policy in your robots.txt, a custom PHP token computation, or a path-differentiated access policy, you'll be glad to control your stack instead of depending on a CDN's general terms. Technical control, in this craft, is also a form of professional independence.

Frédéric Jézégou

Senior Freelance SEO · GEO Consultant · Rankit.fr

Senior Freelance SEO, 20 years of experience. I help SMBs strengthen their Google visibility while integrating the new challenges of AI engines. My expertise covers both classic SEO and the emerging strategies for AI presence (GEO), which I use as complementary approaches depending on the objectives. Data-driven methodology: server logs, competitive testing, site audits.

Rankit.fr LinkedIn HTML→MD Converter Book a call