Agent-facing documentation

When agents retrieve documentation, an omitted parameter or vague error can be copied into generated code and repeated across attempts. Agent-facing documentation states prerequisites, state, constraints, and failure modes explicitly while retaining the explanation a person needs.

The two readers, side by side

A human reading docs:

Uses headings and navigation to locate the relevant section.
Evaluates the product through its explanation, examples, and presentation.
May leave when the next action or prerequisite is unclear.
Can infer some missing context from prior knowledge.
Carries information between pages and sessions.

An AI agent reading docs:

Often receives retrieved chunks rather than reading the site in sequence.
Uses the retrieved text to answer a question or choose an action; it carries no accumulated view of the product unless the host supplies that context.
May retry until it reaches a tool, token, or context limit.
Performs better when prerequisites, state, and constraints are explicit and consistent.
Has only the context and memory supplied by its host, so each retrieved passage needs enough local context to stand alone.

Documentation must therefore support navigation and judgement for people while remaining complete when retrieved in sections.

Practical guidelines

One concept per page

Pages that combine unrelated concepts can reduce retrieval precision. Organise each page around one task or closely related set of questions; separate product definition, competitive comparison, and authentication when they require different context.

Lead with the answer

Put the direct answer near the start of the page. Retrieval systems often select opening passages when they match the query.

A short opening paragraph can state the answer, prerequisites, and scope before the detailed explanation. It also lets a person decide whether the page addresses their task.

Complete, runnable code samples

Bad:

# Configure the client
client = Foo(...)
# Make the call
result = client.do_something()

Good:

import os
from foo_sdk import FooClient

client = FooClient(
    api_key=os.environ["FOO_API_KEY"],
    region="us-east-1",
)

result = client.do_something(input="hello")
print(result.output)

The abbreviated example omits client configuration and a concrete input, so it cannot run when copied verbatim. The complete example gives both people and agents code they can execute and verify.

Explicit imports, environment variables, prerequisites

Every code sample should list:

All imports.
Required environment variables.
Any prerequisite installation steps.
Expected output (where deterministic).

Code may be copied without surrounding prose, so missing context makes the sample non-runnable.

Multiple language variants per sample

If your product supports Python, TypeScript, Go, Rust, and Ruby SDKs, maintain equivalent canonical examples for each supported language. Retrieval can then return an example that matches the developer’s requested language instead of translating one that may use different SDK behaviour.

Explicit versions and dates

dateModified in JSON-LD on every page.
Version in URLs (/docs/v3/...) or visibly in page metadata.
“Last verified against version X.Y” notes on tutorials.

Without explicit dates and versions, a retrieval system may combine current and deprecated instructions.

Canonical, persistent URLs

Keep documentation URLs stable. When a page moves, preserve existing citations and external links with redirects.

Schema.org / JSON-LD markup

Particularly:

Article with dateModified, dateCreated, author.
HowTo for tutorials.
FAQPage for FAQ-style content.
APIReference (less mature but emerging).
SoftwareApplication for product pages.

Schema markup exposes structured metadata to crawlers without requiring them to infer it from page layout.

Clear, consistent terminology

Pick a canonical name for each concept and use it consistently. If “workspace,” “project,” and “tenant” name different objects, define the distinction; if they are synonyms, choose one. Models may reproduce whichever terminology they retrieve.

Q&A and how-to formatting

Headings phrased as questions (“How do I X?”) match the way humans phrase queries to AI assistants. Agents can extract these passages cleanly into Q&A responses.

A useful exercise: imagine the actual question a developer would ask ChatGPT, and use that as the page heading.

Avoid burying key information in images

Include every material fact from a diagram in searchable text. Use alt text for the image and, when the relationships need explanation, a paragraph or list beside it.

Agents are improving at OCR-from-image but text is still substantially more reliable.

Don’t gate docs

Most AI crawlers cannot retrieve login-protected documentation. Keep documentation public when product, security, and contractual requirements allow it; otherwise document the access boundary clearly.

Keep `llms.txt` and `llms-full.txt` current

The convention is minimal and its effect on AI citations remains unproven. Generate it automatically where supported, then audit the links and descriptions periodically. See llms.txt.

Provide the OpenAPI spec publicly

Publish /openapi.yaml or /openapi.json at a stable URL so tools can inspect the API contract when generating integration code. Apply the same principle to GraphQL schemas, Protobuf definitions, and JSON Schema files.

Robots.txt: allow AI crawlers explicitly

Many teams accidentally block AI crawlers. Allow:

GPTBot (OpenAI training)
OAI-SearchBot (ChatGPT live retrieval)
ChatGPT-User (ChatGPT’s user-initiated browsing)
ClaudeBot (Anthropic training)
Claude-Web (Claude’s live retrieval)
Claude-User
PerplexityBot
Perplexity-User
Google-Extended (Gemini training)
Googlebot (Google + Gemini retrieval)

Each AI vendor publishes its bot names. Audit your robots.txt quarterly.

Server-side rendering

Agents that don’t execute JavaScript only see what’s in the initial HTML. If your docs are client-rendered, ensure they have a server-side rendered version or pre-rendered HTML available to bots.

Most modern docs platforms (Mintlify, Docusaurus, Astro, Hugo, Nextra) handle this by default. Custom docs sites often don’t.

Embed code that agents can extract

Plain Markdown code fences are easier to retrieve than code embedded in custom components or collapsed accordions. When the page uses an interactive component, also provide the code as searchable text.

Provide canonical “what is X?” answers

Maintain a stable page that states what the product does, who uses it, and its relevant constraints. This gives retrieval systems a canonical passage for basic product questions.

Provide canonical “X versus Y” comparisons

If your category has competitors, assistants will receive questions about the differences between them. A factual comparison page gives retrieval systems a first-party source; without one, they may rely on competitors or third-party material of uneven accuracy.

The comparison needs factual differences, explicit trade-offs, and enough detail to answer the query. Stripe’s “comparing payment processors” content is a useful example.

Review and retire stale material

A page that presents a deprecated endpoint as current can produce failing calls for both people and agents. Audit on a defined cadence, and remove deprecated material or mark it clearly with the supported replacement.

What humans still need on top of all of this

Machine-readable reference material does not cover every human need. People may also need:

Explanation of why the task or constraint matters.
Embedded video or animation when motion is material to the procedure.
Worked examples drawn from real use cases.
Authored guidance for evaluating alternatives and trade-offs.

Stripe, Twilio, Vercel, Cloudflare, and Anthropic’s Mintlify-hosted documentation combine structured reference material with examples, video, and authored explanation. The reference supports retrieval; the additional formats help people understand and evaluate the product.

Write the agent-readable version first and audit it for completeness. Then add the material that helps a person understand, choose, and remember the product.

A practical checklist

For each high-traffic doc page: