AI Catalogs Have Arrived Before the Conformance Harness

Three public implementations made the AI-facing edge of a website more concrete this week. Cloudflare's documentation site added /.well-known/ai-catalog.json on 24 August, DeepEval published agent-discovery endpoints the same day, and Weaviate updated its catalog and agent resources on 25 August.

These are not three copies of llms.txt. Each catalog is a typed directory that can point a client towards different machine-facing artifacts: Markdown documentation, OpenAPI descriptions, MCP server cards, agent skills, API catalogs, sitemaps, or another catalog. The emerging Common AI Catalog specification gives them one JSON envelope and uses media types to tell a client which parser should handle each entry.

The interesting result is not that another .well-known file exists. It is that the live implementations already expose the next engineering problem. Discovery is starting to converge while serving behaviour, media types, identity resolution, and artifact validation still vary. A catalog can be valid JSON, useful to an agent, and unsafe to treat as an executable inventory without further checks.

The repeated angle to avoid

The ten most recent posts here covered security-dashboard denominators, chat-adapter semantics, agent compute planes, vulnerability states, soft-navigation measurement, authorization control loops, credential containment, payment routing, MCP response metadata, and encrypted client hello. Older overlapping posts covered WebMCP tool boundaries, agent conversion tracking, agent skills as executable supply chain, AI-search content, and the distinction between readable MCP metadata and governed behaviour.

The weak version would repeat the old X needs Y formula: agent discovery needs governance. The sharper thesis is that a typed catalog creates a parser-selection boundary. If producers declare one artifact type but serve another representation, or if consumers turn discovery metadata directly into execution authority, the shared directory amplifies ambiguity instead of removing it.

This article therefore avoids re-explaining why websites should become agent-ready. It focuses on the mechanism between discovery and use: how a client should resolve a catalog entry, where current implementations diverge, and what a small conformance harness should prove.

Fresh evidence and background serve different jobs

The fresh primary evidence is the implementation cluster:

• Cloudflare's 24 August commit added a four-entry live AI catalog, a dedicated application/ai-catalog+json response type, permissive CORS, and tests that require HTTP(S) entry URLs.
• DeepEval's 24 August commit added a five-entry catalog alongside Markdown negotiation, an RFC 9727 API catalog, a skills index, and WebMCP tools.
• Weaviate's 25 August change expanded a ten-entry catalog that points to Markdown guides, agent skills, OpenAPI, pricing, and a sitemap.
• The specification repository itself changed on 27 August, extracting distribution mappings from the normative specification into separate guidance. Its README describes the work as a common standard under active collaboration, with steering-committee adoption still a future decision.

Older standards provide background rather than freshness. RFC 8615 explains well-known URIs, RFC 9110 defines HTTP representation metadata, and the DID Web method explains how a did:web identifier resolves. They help test the implementation; they are not evidence that this week's adoption cluster occurred.

A catalog is a typed directory, not a larger sitemap

A sitemap mostly answers, "Which pages exist?" An llms.txt file gives a language model a navigable text index. An OpenAPI document describes HTTP operations. An MCP server card describes how to connect to an MCP service. A skills index points to reusable instruction packages.

An AI catalog sits one layer above those assets:

Artifact	What it helps a client decide	What it does not prove
llms.txt or Markdown index	Where useful documentation lives	That the content is current, complete, or safe to follow
OpenAPI description	Which HTTP operations and schemas exist	That the caller is authorized or an operation is appropriate
MCP server card	How a client may connect to a server	That the server or its tools are approved for this user
Agent skills index	Which instruction packages are available	That a package is trusted, reviewed, or installed
AI catalog	Which artifact types a domain publishes and where to fetch them	That every entry is live, conformant, authentic, or executable

The catalog's most important field is therefore not its prose description. It is type. The working specification requires every entry to declare a media type so a client can select, filter, and route the entry without first parsing arbitrary content.

A minimal resolution pipeline looks like this:

discover catalog
  -> parse catalog envelope
  -> select supported entry type
  -> resolve entry URL
  -> fetch with bounded network policy
  -> compare served representation with declared type
  -> validate artifact-specific schema
  -> verify identity and integrity where required
  -> apply local authorization policy
  -> display, install, connect, or invoke

Skipping any arrow collapses separate claims. A description is not a schema. A media type is not a signature. A domain name is not authorization. Successful retrieval is not approval to execute.

Live catalogs already reveal soft interoperability

I probed the three live catalogs and all of their entries on 27 August. Every catalog returned HTTP 200 and parseable JSON. Every referenced entry also returned HTTP 200 at that moment. That is a useful baseline, but the response contracts were not identical.

Publisher	Catalog response	CORS	Entry-level observation
Cloudflare	application/ai-catalog+json	*	Three typed JSON artifacts were served as generic application/json; llms.txt matched text/plain
Weaviate	application/json	Not present	Markdown, plain text, and XML matched; two application/ai-skill entries resolved to GitHub HTML, and OpenAPI was served as generic JSON
DeepEval	application/json	*	Its API catalog matched application/linkset+json; two entries declared Markdown but were served as plain text, and its MCP card was generic JSON

These are not all equally severe failures. A generic JSON content type may still carry a perfectly valid OpenAPI description or MCP server card. text/plain can contain Markdown syntax. CORS matters to browser-based discovery clients but not to every server-side client. The specification's authoring guide says an entry URL should serve the content type declared in type; it does not justify treating every parameter or subtype variation as malicious.

The operational point is narrower: a client cannot assume that a valid catalog guarantees exact representation parity. It needs an explicit compatibility policy.

A sensible default is:

• accept an exact media-type match after removing parameters such as charset;
• allow a documented compatibility mapping only after validating the underlying artifact schema;
• never use content sniffing to turn HTML into an installable skill or executable tool definition;
• quarantine one bad entry rather than rejecting unrelated valid entries;
• record the mismatch so the publisher can fix the serving contract.

That last rule matters for adoption. If one stale skills URL causes an entire domain catalog to disappear, clients become brittle. If clients silently coerce every 200 response, publishers never see contract drift. Per-entry quarantine preserves useful discovery without normalising ambiguity.

Well-known does not yet mean universally registered

The working serving guide recommends /.well-known/ai-catalog.json, and the three fresh implementations use it. The specification also allows catalogs at arbitrary URLs and supports discovery through link relations or registries.

There is a sharp edge in the current wording. At the time of verification, the IANA Well-Known URI registry listed api-catalog but did not list ai-catalog, even though the AI Catalog serving guide described its path as registered. That makes the path a useful emerging convention, not yet a universally registered fact that a client should assume every domain understands.

This also changes failure handling. A 404 at the conventional path can mean "no catalog", "catalog advertised elsewhere", "old deployment", or "temporary routing error". A consumer that knows the page or HTTP Link relation should use that advertised location before falling back to the well-known path.

For publishers, the lesson is to expose both a predictable endpoint and an explicit discovery link when practical. For consumers, absence at one path should not trigger an unrestricted crawl looking for JSON files with promising names.

Identity-shaped strings are not verified identity

Cloudflare and DeepEval both place a did:web value in the catalog host identifier. Weaviate's implementation deliberately omits a host identifier; its earlier conformance commit explains that it did not want to claim a DID without serving the corresponding document.

That restraint is useful. A did:web:example.com identifier normally resolves through the DID Web rules to an HTTPS document on that domain. Direct probes of the corresponding .well-known/did.json paths for the Cloudflare documentation host and DeepEval returned HTTP 404 during this review.

That does not make either catalog unusable. The catalog still arrived over HTTPS from the expected domain, and the host string can remain a stable identifier. It does mean a consumer should not promote did:web syntax into "cryptographically verified publisher" without successful resolution and verification.

Keep these trust levels separate:

1. Located: a catalog was found at a domain or advertised URL.
2. Transport-authenticated: HTTPS connected to the expected host.
3. Structurally valid: the catalog and selected artifact passed their schemas.
4. Identity-resolved: the claimed identifier resolved under its method.
5. Integrity-bound: a digest or signature binds the fetched bytes to the claim.
6. Locally approved: policy permits this artifact, publisher, version, and action.

Only the final level should grant operational authority. The AI Catalog specification's optional Trust Manifest can carry identity, attestations, provenance, digests, and signatures. Even then, the specification's own threat analysis requires signed trust metadata to bind the artifact's URL, media type, and digest; a valid signature over unrelated metadata would not make substituted bytes safe.

Representative queries are routing hints

All three catalogs include human-readable descriptions, and many entries include representativeQueries: examples of questions that an artifact is suited to answer. This is useful retrieval metadata. A client can use it to rank a docs index for "How do I rotate an API key?" or an OpenAPI description for "Which endpoint updates a DNS record?"

It is also a new prompt-injection boundary. Catalog metadata comes from outside the consuming agent's local policy. A hostile or compromised catalog could describe a tool as the best answer for every task, place instructions inside a description, or steer a client towards an unrelated domain.

Treat those fields as search features, not commands:

• index descriptions and representative queries as untrusted text;
• cap their size and strip unsupported control content;
• do not let them override system instructions, tool allowlists, or confirmation rules;
• rank only entries whose type, publisher, and business purpose are already eligible;
• log why an entry was selected without copying sensitive user prompts into general analytics.

This is the same separation a search engine makes between retrieval and execution. Metadata can improve candidate selection. It cannot decide that a candidate may mutate customer data, spend money, deploy code, or install instructions.

Build a conformance harness around transitions

A useful harness should test the catalog as a graph of network and parser transitions, not only validate one JSON file. The following matrix is small enough for a CI job or scheduled monitor.

Gate	Test	Failure action
Discovery	Advertised URL or configured well-known path returns an expected status without an unsafe redirect	Keep previously approved inventory; do not discover new entries
Envelope	Body is bounded JSON with a supported specVersion and an entries array	Reject this catalog snapshot
Identity key	Entry identifiers are present, unique, and stable across ordinary URL moves	Quarantine duplicates; flag unexpected identifier churn
Location	Exactly one of url or inline data is present; URL scheme is allowed	Reject the entry
Fetch safety	DNS, redirects, response size, timeout, and private-network destinations satisfy policy	Reject the fetch and record the reason
Representation	Served Content-Type matches the declared type or an approved compatibility rule	Quarantine or validate under a narrow compatibility path
Artifact schema	OpenAPI, server card, skills index, nested catalog, or other artifact validates with the correct parser	Reject the entry, not unrelated entries
Recursion	Nested catalogs respect visited-URL, depth, entry-count, and byte limits	Stop recursion and retain validated results
Trust	Required DID, digest, signature, provenance, and freshness checks succeed	Allow discovery display only; deny installation or invocation
Policy	Publisher, artifact type, origin, version, data access, and action class are approved locally	Deny the operational action

The report should preserve observed and expected values rather than collapse everything into valid: true:

{
  "catalog_url": "https://example.com/.well-known/ai-catalog.json",
  "checked_at": "2026-08-27T15:00:00Z",
  "spec_revision": "pinned-commit-or-release",
  "catalog_status": "usable_with_quarantined_entries",
  "entries": [
    {
      "identifier": "urn:air:example.com:mcp:bookings",
      "declared_type": "application/mcp-server-card+json",
      "served_type": "application/json",
      "final_url": "https://example.com/mcp/server-card.json",
      "schema_valid": true,
      "identity_status": "unverified",
      "policy_status": "discovery_only",
      "findings": ["compatible-json-subtype", "no-approved-publisher-rule"]
    }
  ]
}

This shape separates compatibility from trust and trust from policy. An entry can be schema-valid but unapproved. It can be approved for display but not invocation. It can be fetched successfully while its publisher identity remains unresolved.

Fetching catalog URLs creates an SSRF surface

A catalog consumer is a URL fetcher controlled partly by external data. That makes server-side request forgery, oversized responses, redirect abuse, decompression bombs, and slow responses part of the feature's threat model.

Do not implement recursive discovery with a plain unrestricted fetch(entry.url) loop. Apply network controls before dereferencing:

• permit HTTPS by default and reject credential-bearing URLs;
• resolve and block loopback, link-local, private, metadata-service, and internal address ranges;
• re-check every redirect target rather than validating only the first URL;
• limit redirect count, response bytes, decompressed bytes, time, entries, and nesting depth;
• use a dedicated egress identity with no access to internal control planes;
• cache by validated artifact identity and version, not by an untrusted display name;
• avoid sending cookies, ambient authorization headers, or internal client certificates;
• parse each supported media type in an isolated, non-executing path.

A catalog that points to https://169.254.169.254/, an internal admin hostname, or a multi-gigabyte artifact should fail before the parser sees a byte. A nested catalog that links back to its parent should stop at the visited set instead of recursing forever.

Publisher checks should run before deployment

Publishing a catalog is not finished when the JSON schema passes. The referenced assets can drift independently, and several current implementations generate or serve them through different frameworks and domains.

A publisher-side CI check should:

1. validate the catalog against a pinned specification revision;
2. reject duplicate identifiers and unsupported URL schemes;
3. fetch every external entry from a clean network context;
4. assert status, final URL, and intended media type;
5. validate each artifact with its native schema;
6. verify that a downloadable artifact URL returns artifact bytes rather than a marketing or repository HTML page;
7. confirm CORS if browser-based discovery is supported;
8. resolve any claimed identity and fail claims that cannot be resolved;
9. verify digests after build output is final;
10. diff identifiers, URLs, types, permissions, and trust metadata against the last published catalog.

The diff is important. A newly added docs page is low risk. An MCP server URL moving to another origin, a skill changing from a reviewed archive to a mutable branch, or a read-only toolset gaining write capabilities deserves a release decision.

Catalog availability should also be monitored separately from artifact availability. A cached catalog can continue pointing to a dead asset; a healthy artifact can become undiscoverable because a route or deployment removed the catalog. Both paths need probes.

Decide whether a catalog is the right asset

Not every small business website should publish one. Use the least complex machine-facing contract that answers a real consumer need.

Site capability	Practical starting point
Public pages and articles only	Sitemap, clear internal links, structured data, and optional llms.txt or Markdown views
Documentation plus one API	Human docs and a validated OpenAPI description; add a catalog if clients need unified discovery
Multiple APIs, MCP servers, skills, or datasets	A catalog can provide useful typed routing across artifacts
Agent actions affecting leads, bookings, orders, or accounts	Catalog for discovery, plus separate authentication, authorization, confirmation, idempotency, and audit controls
Marketplace or multi-publisher registry	Catalog plus strong publisher identity, artifact integrity, review status, revocation, and version policy

The decision rule is simple:

Publish a catalog when it removes real ambiguity between multiple machine-facing artifacts. Do not publish one merely to signal that a site is "agent ready."

A four-entry catalog that accurately routes clients to docs, an API description, a server card, and reviewed skills is valuable. A hundred generated entries with dead URLs, guessed media types, mutable artifacts, and inflated representative queries create maintenance debt and a larger trust surface.

Failure modes worth rehearsing

The catalog becomes an auto-install list

A client discovers a skill archive and installs it because it came from the same domain as trusted docs. Keep discovery and installation separate. Require an approved publisher, immutable version or digest, package review, and explicit install policy.

Generic JSON is accepted as every JSON subtype

A response declares application/json, so the client tries OpenAPI, MCP, and skills parsers until one does not crash. Route from the catalog's declared type, apply an explicit compatibility rule, then validate exactly one expected schema. Parser roulette hides publisher mistakes and expands attack surface.

A 200 response contains HTML

Repository pages, login screens, bot challenges, and friendly error pages can all return HTML with status 200. Check media type and artifact bytes before caching. Never hand an HTML fallback to a package installer because the URL ended in a plausible name.

A stable name points to mutable authority

An identifier stays constant while an MCP endpoint changes origin or a skill URL follows a moving branch. Diff location, type, digest, version, and capability metadata. Stable identity should make change visible, not conceal it.

Catalog trust leaks into tool authorization

The catalog arrives from a known vendor, so every listed tool inherits broad credentials. Authorize each operational connection independently. The server card can describe an authentication mechanism; local policy decides which token, scopes, account, and actions are allowed.

A browser client works while server automation fails, or vice versa

CORS, redirects, content negotiation, bot controls, and regional edge rules can produce different results. Probe from each supported client class and record the request headers that define the contract.

The practical conclusion

The fresh Cloudflare, DeepEval, and Weaviate implementations show that AI discovery is becoming a real web surface rather than a diagram in a standards repository. They also show why early adoption should be paired with measurement. Typed entries are more useful than one undifferentiated text dump, but the type only earns its value when the referenced representation, parser, identity, and local policy agree.

The durable architecture is deliberately layered: catalogs discover, media types route, schemas validate, identities and digests support trust, and local policy grants authority. That lets a client use imperfect but useful early catalogs without converting every mismatch into an outage or every discovery claim into an execution path.

The conformance harness is therefore not cleanup work after the standard settles. It is the operational asset that makes a moving standard safe enough to learn from now.