Hosted Key Directories for Web Bot Auth

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 20 January 2027.¶

Copyright Notice

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

Web Bot Auth applies HTTP Message Signatures [RFC9421] to automated traffic. An agent signs requests with a private key; the corresponding public key is published in a key directory served from a well-known URI, and a verifier fetches the directory to validate the signature [I-D.meunier-webbotauth-httpsig-protocol] [I-D.meunier-webbotauth-httpsig-directory].¶

The deployment model implicit in these documents is self-hosting: the entity that operates the agent also operates the HTTPS origin that serves its key directory. This model fits large operators. It does not fit the long tail of agents operated by individual developers, small businesses, and research groups. Such operators frequently run agents from residential networks, ephemeral cloud workloads, or end-user devices; they may control no stable HTTPS origin at all. As origins increasingly gate automated traffic on verifiable identity, agents without a practical path to publishing keys are treated as anonymous automation and denied access, regardless of their behavior or the legitimacy of their purpose. The working group's use-case catalogue recognizes this asymmetry, observing that authenticating automated clients can "level the playing field for small bots" [I-D.nottingham-webbotauth-use-cases]; this document describes a deployment model that makes that levelling practical.¶

Today this gap is filled in two ways, both with significant drawbacks:¶

• Verifier-curated allowlists of large, well-known agent services. These scale to tens of entries, not to hundreds of thousands of independent operators.¶

• Platform-mediated identity, in which an agent inherits trust by running on a particular hosting platform whose infrastructure signs its egress traffic. This couples an agent's identity to its hosting choice: the identity, and any access arrangements origins have made for it, is lost when the agent moves. It also concentrates accountability in the platform rather than in the party that actually directs the agent's behavior.¶

This document describes a third model, the hosted key directory: a registry operator publishes key directories on behalf of many independent tenants, while each tenant generates and retains exclusive custody of its own private keys. The model is directly analogous to established practice in other trust systems: most DKIM [RFC6376] signing domains publish their keys through a handful of mail providers; most TLS certificates are issued by shared certificate authorities rather than self-run ones; and most DNS zones are served by hosting providers rather than by their registrants.¶

The registry document [I-D.meunier-webbotauth-registry] supplies discovery mechanics for this ecosystem: per-agent metadata cards, and registries that list them, which it explicitly contemplates being maintained by third parties. It deliberately leaves trust and accreditation policy to implementers. Hosted directories raise exactly those unaddressed operational questions: what principal a verifier should associate with a signature whose key lives in a shared directory; how possession of a key can be demonstrated when the directory host cannot sign with it; how quickly revocation must propagate through caches; and what a verifier is entitled to know about a directory operator's practices. This document analyzes each and offers guidance, informed by an independent open-source implementation of the current drafts.¶

2. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

Agent:

An automated client that originates HTTP requests and signs them per the Web Bot Auth profile of [RFC9421].¶

Operator (or Tenant):

The party that controls an agent's behavior and holds its private signing keys. In a hosted directory, each operator is a tenant of the directory host.¶

Hosted key directory:

A key directory served by a party other than the operators whose keys it advertises.¶

Directory host (or Registry operator):

The party operating one or more hosted key directories and the enrollment process behind them.¶

Verifier:

An origin, or infrastructure acting for origins, that validates Web Bot Auth signatures against a key directory.¶

Principal:

The identity a verifier associates with a validated signature for the purpose of policy and rate limiting.¶

3. Deployment Models for Key Directories

Three deployment models are observed or anticipated in practice. They differ in who serves the directory, who holds private keys, and what happens to an agent's identity when it changes infrastructure.¶

The hosted model preserves a property essential to origin policy: the operator's identity is stable across changes of hosting, cloud provider, model provider, and network location, because the operator, not the infrastructure, holds the key. Origins can therefore make per-operator access decisions that outlive any particular deployment.¶

4. The Principal Mapping Problem

The directory document [I-D.meunier-webbotauth-httpsig-directory] identifies a directory by the origin that serves it, and is silent on the relationship between a directory's keys and distinct agent identities. At least one widely deployed verifier program has resolved that silence by treating the origin as the principal: every key advertised by a directory is attributed to a single logical agent identity, registered and reviewed as a unit.¶

Applied unchanged to a hosted directory, this collapses all tenants into one principal. The consequences are unacceptable in both directions: a single misbehaving tenant taints the shared identity for every other tenant (shared fate), and per-tenant policy and rate limits become impossible (lost granularity).¶

Two remedies are available within the current protocol, and a third would benefit from working group attention:¶

1. Per-tenant directory authorities. The directory host serves one directory per tenant under a distinct authority, for example tenant-label.agents.registry.example. Because the Signature-Agent header carries the directory URI, each tenant presents a distinct principal to unmodified verifiers. This works today; its costs are DNS and certificate automation at the host, which are routine at scale (DKIM's selector convention and shared CA issuance demonstrate the pattern at internet scale). This is the RECOMMENDED deployment for hosted directories absent verifier support for key-level principals.¶

2. Key-level principals at the verifier. A verifier MAY treat the pair (directory authority, JWK thumbprint [RFC7638]) as the principal rather than the authority alone. This restores per-tenant granularity without protocol changes, at the cost of larger policy tables.¶

3. Per-tenant metadata via Signature Agent Cards. The registry document [I-D.meunier-webbotauth-registry] defines a per-agent metadata card resolvable from a stable URL, with key material separable via jwks_uri. Pairing a hosted directory with one card per tenant expresses tenancy explicitly without new protocol elements. This document suggests the working group consider making that pairing normative guidance for shared directories.¶

5. Proof of Key Possession

The directory document RECOMMENDS that a directory prove possession of the advertised keys by signing directory responses with each key. A hosted directory cannot follow this recommendation, and this document takes the position that it MUST NOT be able to: the security value of the hosted model rests on the directory host never holding tenant private keys. A host that can sign as its tenants is a single point of compromise for all of them.¶

Two substitutes preserve the recommendation's intent in the hosted model:¶

1. Enrollment-time proof. At enrollment, the host verifies possession by challenge: the prospective tenant signs a host-supplied nonce with the key being enrolled, binding the key thumbprint, the intended directory authority, and a timestamp. The host MUST perform such a challenge before first publication of a key and SHOULD repeat it at key rotation.¶

2. Tenant-produced directory assertions. The tenant periodically signs, with its own key, a detached assertion binding its key thumbprint to the directory authority and a validity window. The host publishes these assertions alongside the directory. A verifier that wishes to check possession can validate the assertion without the host ever holding the key. This construction is offered as a candidate for standardization; the author's implementation demonstrates enrollment-time proof today and can produce detached assertions if the working group standardizes a format.¶

Interoperability note: no verifier observed by the author currently enforces directory response signatures. The working group therefore has a window in which to specify a possession mechanism compatible with hosted directories before enforcement behavior ossifies.¶

6. Freshness, Caching, and Revocation

Directory caching directly bounds revocation latency: a verifier that cached a directory continues honoring a revoked key until the cache expires. The example cache lifetime in current material (on the order of a day) is reasonable for a single-operator directory whose keys change rarely. It is NOT RECOMMENDED for hosted directories, where tenant revocation is a routine, security-relevant event.¶

A hosted directory SHOULD serve cache lifetimes on the order of minutes (the author's implementation defaults to 300 seconds), SHOULD support conditional requests so that unchanged directories cost little to revalidate, and MUST document its revocation propagation target (the maximum interval between a tenant revocation request and the key's disappearance from all served responses). Push-style revocation signals between registries and high-volume verifiers are useful and complementary but are out of scope for this document.¶

7. Interoperability Observations from an Independent Implementation

The author operates an independent open-source implementation of the current protocol and directory drafts (signer, verifier, hosted directory, and enrollment). Implementation surfaced four observations relevant to deployment guidance; they are reported here neutrally as running-code input to the working group.¶

1. Signature-Agent representation. The protocol draft specifies a Structured Fields Dictionary form for the Signature-Agent header [RFC8941], while at least one major deployed verifier accepts only the earlier plain string form and rejects the Dictionary form. Until this converges, signers interoperate most widely by emitting the string form, and verifiers SHOULD accept both. Explicit transition guidance in the working group's deliverables would prevent divergent deployments.¶

2. Covered components. The drafts permit covering either @authority or @target-uri; at least one major deployed verifier requires @authority specifically. Signers maximize interoperability by covering @authority; verifiers SHOULD accept either.¶

3. Nonce encoding. Prose in current material describes the nonce as base64url while accompanying examples use standard base64 with padding. The author's implementation follows the examples. A one-line clarification would remove the ambiguity.¶

4. Nonce validation. No verifier observed by the author validates nonces for replay; within its validity window a captured signature replays against the same authority. Deployments SHOULD keep validity windows short (the author's implementation defaults to 60 seconds), and verifiers able to maintain per-authority nonce caches SHOULD do so. Operational guidance here would materially improve the profile's replay posture.¶

8. Operational Requirements for Hosted Directory Operators

A verifier extending trust to a hosted directory is extending trust to the host's operational practices. The following requirements make those practices legible. They are offered as candidate material for the working group's operational guidance.¶

A hosted directory operator:¶

• MUST NOT generate, receive, store, or otherwise possess tenant private keys;¶

• MUST verify possession of a key (per Section 5) before publishing it and upon rotation;¶

• MUST publish a description of its tenant verification practices, including any distinct assurance levels it offers (for example, possession-only; possession plus verified control of a domain; possession plus verified legal identity of the operator);¶

• MUST provide tenant-initiated and host-initiated revocation, and MUST document its revocation propagation target (Section 6);¶

• SHOULD serve each tenant under a distinct directory authority (Section 4);¶

• SHOULD maintain an append-only, auditable log of directory publication changes, so that unauthorized insertions or removals of keys are detectable after the fact;¶

• SHOULD publish availability information, since directory unavailability denies access to every tenant whose keys it serves.¶

9. Security Considerations

Key custody boundary. The defining security property of the hosted model is that compromise of the directory host does not yield tenant signing capability. A host that violates the custody requirement of Section 8 silently converts the model into platform-held identity with platform-sized blast radius.¶

Publication integrity. A compromised host can publish attacker keys under a tenant's authority or remove legitimate keys. Auditable publication logs and tenant-produced assertions (Section 5) make such actions detectable; short cache lifetimes (Section 6) bound their persistence after correction.¶

Enrollment integrity. Possession challenges MUST use fresh, unpredictable nonces and MUST bind the intended directory authority, preventing an attacker from replaying an enrollment performed for one directory against another.¶

Shared fate. Where verifiers apply policy by directory authority rather than by key, one tenant's abuse degrades service for all tenants of a shared authority. Per-tenant authorities and prompt host-initiated revocation are the mitigations; verifiers are encouraged toward key-level policy in any case.¶

Availability. A hosted directory is operationally an availability dependency for every tenant. Hosts should treat directory serving with the availability discipline of authoritative DNS.¶

All considerations of the underlying documents [RFC9421] [I-D.meunier-webbotauth-httpsig-protocol] [I-D.meunier-webbotauth-httpsig-directory] apply.¶

10. Privacy Considerations

A key directory is public; a hosted one enumerates, at minimum, the public keys of its tenants, and per-tenant authorities reveal tenant labels in DNS. Hosts SHOULD support pseudonymous tenant labels for operators that have verified attributes with the host but do not wish to disclose them publicly, disclosing attributes to verifiers only through whatever assurance-level publication the host offers (Section 8). The tension between operator accountability and operator privacy is real and merits working group discussion; this document does not resolve it.¶

11. IANA Considerations

This document has no IANA actions.¶

Acknowledgments

The observations in this document derive from implementing and operating the current Web Bot Auth drafts; the author thanks their authors and the deployed verifier operators whose public documentation made independent implementation possible.¶

Author's Address