]> Anthropic

kelleyk-ietf@kelleyk.net

Web and Internet Transport Web Bot Auth crawler bot IP ranges egress prefixes user-agent verification This document defines SAIL (Source Address Identity List), a metadata format served at a well-known URI in which the operator of automated HTTP clients publishes the IP prefixes its services send from, together with two optional assertions: that a service sends from nowhere else, and that nothing else of the operator's sends from a given set of prefixes. A site that receives a request claiming to be one of those services can use that metadata to confirm the claim or, under those assertions, to refute it. The mechanism is intended to complement reverse-DNS verification and HTTP Message Signatures. About This Document Status information for this document may be found at . Discussion of this document takes place on the Web Bot Auth Working Group mailing list (), which is archived at . Subscribe at .

Introduction This document defines SAIL (Source Address Identity List), a machine-readable and machine-discoverable metadata format that allows service operators to publish information about which addresses other parties should expect to receive inbound traffic from. It also defines a well-known URI for discovering that metadata. Among other uses, this mechanism is intended to allow a consumer that has received a request to determine whether the identity the request claims is consistent with the address it arrived from. For example, it allows a consumer that has received an HTTP request to determine whether the service named in the User-Agent header could have sent it, or whether the request may be an attempt to impersonate that service. Operators of well-behaved automated clients already publish documentation about the services that they operate, including e.g. IP prefixes; however, each publisher chooses its own schema and publication location today, which makes automatically consuming that metadata difficult. A SAIL document is published by a single operator and describes only traffic that operator itself originates. It identifies that operator's services, not the people or organisations on whose behalf a service may be acting: identity that is delegated, federated, or scoped more finely than the operator — per user, per customer, or per task — is outside the scope of this document. Site operators extend privileges to automated clients they recognise and trust, such as access that other clients are refused or higher rate limits, so impersonating such a client is valuable. This draft is intended to complement, and not to be an alternative to, reverse-DNS verification and HTTP Message Signatures as applied to automated clients by , which are two other widespread means of accomplishing the same outcome.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the following terms:

Publisher:

The operator that generates and serves a SAIL document describing its own services and the prefixes they send from.

Consumer:

A party that retrieves a SAIL document and acts on it, typically the operator of a site that receives automated requests, or an intermediary acting on such a site's behalf.

Holder:

Any party in possession of a copy of a SAIL document, however obtained: a consumer that fetched it from the publisher, or one that received it from a shared cache, a mirror, or a file. Every consumer is a holder; places obligations on holders generally.

Origin:

The scheme, host, and port of a URI, as defined in Section 4.3.1 of . A SAIL document speaks for the origin at whose well-known URI it was requested ().

Service:

A named source of outbound traffic operated by the publisher, typically an automated HTTP client such as a crawler or fetcher, described by an entry in the document's services array (). describes how a request claims a service.

Prefix:

An IPv4 or IPv6 address prefix in CIDR notation , or a single address, written as described in . "IP range" and "address list" in deployed practice refer to the same thing; arbitrary start–end ranges that are not a single prefix cannot be expressed.

Prefix set:

A named collection of prefixes published together, typically corresponding to one egress pool, and the unit over which exclusive is asserted ().

Location of the SAIL Document A publisher MUST serve its SAIL document at the well-known URI whose path component is "/.well-known/sail" on the origin by which it wishes its services to be identified (): The document MUST be served using the "https" scheme only; it MUST NOT be served with any other URI scheme, and a consumer MUST query it using HTTPS only. If the HTTPS request fails, a consumer MUST NOT reissue it over a non-secure connection. A publisher MUST serve it with status 200 (OK) and Content-Type application/sail+toml (). A consumer MUST NOT treat a response with any other status code as a SAIL document, and MUST accept a 200 response whose body parses and validates as a SAIL document () whatever Content-Type it carries. Retrieval MAY result in an HTTP redirect, and redirects are handled as (Section 2.3.1.2) handles them for robots.txt. A consumer SHOULD follow at least five consecutive redirects, even across origins; if the document is reached within five, it is fetched, validated, and applied in the context of the origin whose well-known URI was originally requested; if there are more than five, the consumer MAY treat the document as unavailable. Any such redirect MUST be to an "https" URI; a consumer MUST NOT follow a redirect to any other scheme. A publisher MAY redirect other schemes to "https". A document obtained by following redirects therefore speaks for the origin whose well-known URI was originally requested, not for the origin that ultimately serves it: a redirect is an act of the requested origin, and a consumer redirected from a.example to b.example has obtained a.example's SAIL document. This lets a publisher serve one document for several hosts, or from static hosting on another origin (). The well-known path is the only location this specification defines. Consumers MUST NOT look for the document elsewhere on the origin. Many automated clients include, in the comment portion of their User-Agent header field, a URL for a page describing the client and its operator . A publisher whose clients do so MUST also serve its SAIL document, directly or by redirect, at the well-known path on the host of that URL, so that https://<that-host>/.well-known/sail resolves as described above. A consumer that receives such a request can then derive the location of the publisher's SAIL document from the request alone, without prior configuration. A publisher SHOULD additionally link to the well-known URI from the human-readable page that describes its automated clients, for example alongside links to existing machine-readable lists of egress addresses it already publishes.

Document Format This section defines the serialization, schema, and prefix notation of a SAIL document, and how a consumer resolves its prefixes. gives a complete example.

Serialization A SAIL document is a TOML document and, as TOML requires, MUST be encoded in UTF-8 . Its date-time values are TOML offset date-times, which are timestamps. Its media type is application/sail+toml (; the +toml suffix is not yet registered, see the note there). We use TOML rather than alternatives such as JSON so that the document can carry comments. A document MUST NOT exceed 524288 octets (512 KiB). The limit applies to the TOML text itself, after any HTTP content coding has been removed, not to the size on the wire. A consumer MAY stop reading a response once the limit is passed. A document that exceeds the limit, is not valid UTF-8, or is not valid TOML is invalid (). [[EDNOTE: 512 KiB is a first guess. Also open is whether per-item limits (counts of prefix sets, services, or prefixes per set) are needed in addition.]]

Invalid Documents Wherever this specification says a document is invalid, a consumer MUST ignore it in its entirety and MUST NOT act on any part of it. In particular a consumer MUST NOT act on the readable portion of a truncated or partly parseable document. For example, an incomplete list under exhaustive would cause genuine traffic to be rejected. A final response with any status code other than 200 (OK), once redirects have been followed, is invalid. A consumer that already holds a valid document from the same origin SHOULD continue to use it until its effective expiry (), as if the invalid retrieval had not occurred.

Schema A SAIL document () is a single TOML table with the following top-level keys. Unrecognised keys at any level MUST be ignored.

schema (string, REQUIRED):

The specification the document conforms to; for this version, "draft-kelley-webbotauth-sail-00".

modified (offset date-time, REQUIRED):

When the document's content last changed, for the benefit of mirrors, caches, and other systems that track the metadata a SAIL document expresses. Regenerating or re-serving unchanged content, or changing only the expiry, MUST NOT advance it.

expires (offset date-time) / expires_after (integer seconds):

A publisher MUST include exactly one. expires is the absolute instant at which the content ceases to be valid (). expires_after obliges any holder to refresh the document within that many seconds of receiving it and not to rely on an unrefreshed copy beyond that; it MUST NOT exceed 2592000 (30 days), and a consumer that receives a larger value MUST treat it as 2592000. It is provided for publishers for whom regularly advancing an absolute expires is infeasible. A document with neither key is invalid (); a consumer that receives a document with both MUST use whichever yields the earlier effective expiry ().

prefix_sets (array of tables):

Each entry describes one egress pool:

• name (identifier, REQUIRED; see ) — unique within the document and meaningless outside it.
• ipv4_prefixes, ipv6_prefixes (arrays of strings) — prefixes in the syntax of , subject to .
• exclusive (boolean, default false) — the publisher asserts that all of its own use of these prefixes is described in this SAIL document ().

A prefix set that no service references and is not marked exclusive asserts nothing and MUST be ignored. One that no service references and is marked exclusive is meaningful: it states that none of the publisher's services send from those prefixes, so that any request claiming one of them from there is refuted (); a publisher can use this for address space it holds but does not send from. A prefix MAY appear in more than one set and in service entries; defines how occurrences combine, and a prefix in an unreferenced exclusive set that also appears elsewhere in the document takes its services from those other occurrences.

services (array of tables):

Each entry describes one source of outbound traffic:

• name (identifier, REQUIRED; see ) — unique within the document. This is the name a request claims ().
• prefix_sets (array of identifiers) — names of prefix sets in this document from which the service sends. An unknown name makes the document invalid.
• ipv4_prefixes, ipv6_prefixes (arrays of strings) — prefixes from which the service sends, listed directly. Exactly equivalent to an anonymous prefix set, referenced only by this service, whose other keys all take their default values.
• exhaustive (boolean, default false) — the publisher asserts that this service sends from nowhere outside its prefixes as resolved by (; see on when to set it).

A service with no prefix_sets and no directly listed prefixes has no prefixes.

A document with duplicate prefix-set names, duplicate service names, a dangling prefix-set reference, an identifier that violates , neither expiry key, or expires not later than modified is invalid ().

Identifiers Prefix-set names, service names, and the references to prefix sets in a service's prefix_sets array are identifiers. An identifier is a TOML string whose value:

• MUST be a token as defined in Section 5.6.2 of : one or more of the ASCII characters ALPHA, DIGIT, and ! # $ % & ' * + - . ^ _ ` | ~; and
• MUST NOT exceed 128 characters.

Length limits on strings in this specification are stated in characters and apply to the parsed TOML string value, after the document has been decoded and any TOML escapes resolved; only the overall document limit () is stated in octets, because it is checked before parsing. This is the same grammar as an HTTP product name, so every service name can appear verbatim in a User-Agent field value () and as a robots.txt product token, which likewise matches case-insensitively. Identifiers are compared ASCII-case-insensitively: two identifiers are equal if and only if they are identical, character for character, after each character in the range U+0041–U+005A ("A"–"Z") is replaced by the corresponding character in the range U+0061–U+007A ("a"–"z"). No other folding or normalisation is applied. This rule governs uniqueness of names within a document, resolution of prefix_sets references, and matching a service name against a User-Agent field value ().

Prefixes

Syntax Each element of ipv4_prefixes and ipv6_prefixes is a TOML string in exactly one of two forms:

• CIDR notation : an address in the family's textual form, "/", and a decimal prefix length ("192.0.2.0/24", "2001:db8::/32"); or
• a bare address with no "/", which denotes that single address and is equivalent to the same address with "/32" (IPv4) or "/128" (IPv6).

No other form is permitted: start–end ranges, netmask notation, wildcards, and lists within one string MUST NOT be used. Each element of ipv4_prefixes MUST be an IPv4 prefix and each element of ipv6_prefixes an IPv6 prefix. Publishers SHOULD write prefixes in canonical form, with all bits beyond the prefix length zero and, for IPv6, in the text representation of ; a consumer that receives one with host bits set MUST honour it as the prefix obtained by clearing those bits, and neither this nor a non-canonical text representation alone makes the document invalid.

Permitted Address Space A prefix MUST lie wholly within globally reachable unicast space. It MUST NOT overlap:

• any block whose "Globally Reachable" property is anything other than an unqualified True in the IANA IPv4 Special-Purpose Address Registry or IPv6 Special-Purpose Address Registry (established by and ; this covers private, shared, loopback, link-local, documentation, benchmarking, discard, and reserved space);
• multicast space (224.0.0.0/4 , ff00::/8 );
• IPv6 prefixes that embed IPv4 addresses (::ffff:0:0/96 and ::/96, Section 2.5.5 of ); or
• transition-mechanism and shared-anycast assignments from which no single operator originates ordinary unicast traffic, even where the registry marks them reachable: 6to4 (2002::/16 , and the deprecated relay anycast prefix 192.88.99.0/24 ), Teredo (2001::/32 ), the NAT64 well-known prefix (64:ff9b::/96 ), AS112 (192.31.196.0/24 and 2001:4:112::/48 ; 192.175.48.0/24 and 2620:4f:8000::/48 ), and AMT (192.52.193.0/24 and 2001:3::/32 ).

The registries are authoritative for the first item and consumers SHOULD track them as updated; the remaining items are fixed by this specification. A consumer MUST validate every prefix on receipt. A document containing any prefix that is malformed (), in the wrong family for its key, or outside the permitted space MUST be treated as invalid (). [[EDNOTE: Whole-document rejection is simple and consistent with the handling of unparseable and oversize documents, but one bad prefix then withdraws the publisher's entire document until corrected. More graceful degradation is probably possible, but I haven't thought about it carefully yet.]]

Resolving Prefixes A consumer resolves a document into, for each address family, a map from prefix to the set of services that send from it and whether it is exclusive, as follows.

1. Every prefix contributes equally, whether it appears in a named prefix set or in a service's anonymous set (). A prefix is associated with every service that references the set containing it (with no service, if it appears only in an unreferenced exclusive set), and is exclusive if that set is marked exclusive. Prefixes of unreferenced sets not marked exclusive do not contribute.
2. After clearing host bits (), identical prefixes are merged: the associated services are the union of those from each occurrence, and the prefix is exclusive if any occurrence is. Exact duplicates therefore carry no additional meaning and SHOULD be discarded.
3. For a given address, the governing prefix is the longest resolved prefix that contains it. The services and exclusive value that apply to the address are those of the governing prefix alone; any shorter prefix that also contains the address contributes nothing to it. A publisher can thus list a broad allocation under one service and carve a more specific prefix out of it for another without enumerating the remainder.

An address is "within S's prefixes", for the purposes of , if and only if S is among the services of its governing prefix. A consumer that observes a request's source as an IPv4-mapped IPv6 address (Section 2.5.5.2 of , ::ffff:0:0/96), as is common on dual-stack hosts, MUST match it against ipv4_prefixes as the embedded IPv4 address and MUST NOT match it against ipv6_prefixes.

Example The following is a complete SAIL document for a publisher that operates two crawlers from a dedicated egress pool, a fetcher that additionally uses shared corporate egress, holds a block it does not send from, and wishes to disown a name it has retired. The prefixes shown are documentation prefixes ; a real document could not contain them (). Based on the rules in :

• A request claiming ExampleBot from 192.0.2.10 is confirmed.
• One claiming ExampleBot from 198.51.100.70 is refuted, because ExampleBot is exhaustive and that address is not within its prefixes.
• A request claiming Example-Fetcher from 198.51.100.70 is confirmed.
• A request from 192.0.2.10 claiming ExampleBot-News is refuted twice over: ExampleBot-News is exhaustive and does not list that address, and the governing prefix 192.0.2.0/24 is exclusive and ExampleBot-News is not among its services.
• Every request claiming OldExampleBot is refuted.
• A request from 198.51.100.10 claiming any of these services is refuted: parked is exclusive and no service references it.
• The document says nothing about a request from 192.0.2.10 that claims no identity, or that claims some other operator's.

Validity Period HTTP caching governs when a consumer refetches the resource; modified and the expiry keys describe the data. The in-document keys exist because HTTP response metadata is not always passed along faithfully. The effective expiry of a document is the earliest of:

• the value of expires, if present;
• the time at which the holder received the document plus expires_after seconds, if present; and
• the time at which the holder received the document plus 2592000 seconds (30 days), in every case.

The last term bounds how long any copy can be relied on regardless of what the document says. A holder MUST NOT act on a document more than 30 days after receiving it without refreshing it, even if expires is later, and publishers SHOULD NOT set expires further ahead than that. This limits how long a copy the publisher cannot recall (in a cache, a mirror, or on disk) can continue to assert something the publisher has since withdrawn (). For expires_after each holder uses its own receipt time, from whichever source it fetched, so the obligation applies per hop. As a consequence, publishers SHOULD prefer to provide expires rather than expires_after; expires_after is provided for use in situations where regularly refreshing the expires timestamp is difficult. A SAIL document using expires_after and whose receipt time is unknown MUST be treated as expired. A consumer MUST NOT act on a document after its effective expiry, regardless of HTTP cache state, except in cases where the origin is unavailable. Consumers using stale metadata in that situation SHOULD exercise caution. When a document expires, confirmations and refutations () alike lapse. A consumer MAY refresh more often than expiry or HTTP freshness require, but is not obliged to. Publishers SHOULD, where feasible, send HTTP freshness information (Cache-Control: max-age or Expires) that agrees with the document's own expiry and in no case extends beyond it. Consumers MUST NOT refresh a non-expired SAIL document more frequently than every 1800 seconds, regardless of other caching signals.

Identity Claims A service name is not by itself an identity. Names are chosen freely by each publisher, and nothing prevents two publishers, or a publisher and an impostor, from using the same one. The identity that a SAIL document speaks to, and that a consumer confirms or refutes, is the pair (origin of the SAIL document, service name). Any trust or policy a consumer attaches to a service MUST be attached to that pair, with the origin established by means outside the request. In particular, a URL carried in a request (such as the User-Agent comment used for discovery in ) tells a consumer only where the sender says to look; it is not evidence of which origin is authoritative for a name (). The inferences in apply to a request that claims one of the publisher's services. For HTTP, that claim is carried in the request's User-Agent header field (Section 10.1.5 of ); a request claims service S when S's name appears there. Publishers SHOULD choose each service's name to be the product token its client sends, and for crawlers the token it answers to in robots.txt (Section 2.2.1 of ), so that one identifier serves throughout. [[EDNOTE: This is a little bit tricky because User-Agent headers routinely contain lots of different things. In practice, heuristic libraries are used to interpret these values. We should figure out if there's something more definite we can say here.]]

Permissible Inferences A SAIL document lets a consumer test a request's claim to be one of the publisher's services () against the address the request came from. It provides a basis for exactly the following conclusions, and for no others.

Confirmation:

A request that claims service S and originates from an address within S's prefixes is supported by the SAIL metadata. No assertion (e.g. exhaustive or exclusive) is required beyond S being listed.

Refutation by service (exhaustive):

If S is marked exhaustive, the publisher has stated that S sends from nowhere else. A request claiming S from an address outside S's prefixes is therefore not from the publisher, and the consumer MUST treat the claim as refuted, MUST NOT extend to the request any treatment it reserves for S, and MAY additionally reject the request. Absent exhaustive, no such conclusion follows; the listed prefixes are a subset. A consumer MAY nonetheless count the absence of a matching prefix as a weak signal against the claim, for example as one input to its own risk scoring, provided it does not reject or otherwise treat the request as refuted on that basis alone. A service that has no prefixes () and is marked exhaustive is valid and means that the service currently sends from nowhere, so every request claiming it is refuted. A publisher can use this to retire a service name, or to disown one it has never used but sees being impersonated.

Refutation by prefix (exclusive):

If the prefix governing an address () is exclusive, the publisher has stated that none of its other services send from there. A request from that address claiming any of the publisher's services other than those associated with that prefix (or claiming any of them at all, if the prefix has no associated service) is therefore not from the publisher. The consumer MUST treat the claim as refuted, MUST NOT extend to the request any treatment it reserves for the claimed service, and MAY additionally reject the request. Absent exclusive, no such conclusion follows.

Used together the two assertions close the mapping in both directions: exhaustive says "this service of ours, only from here"; exclusive says "from here, only these services of ours". Where either refutation permits a consumer to reject the request, this document does not specify the form the rejection takes (see the note at the end of this section). These conclusions serve permissive decisions as well as restrictive ones. A consumer MAY use confirmation to allowlist a service — for example to exempt requests that claim S and arrive from S's prefixes from challenges, rate limits, or blocks applied to other automated traffic — just as it uses the refutations to deny impostors. In every case the decision is keyed on the pair of claimed identity and source address, never on the address alone. A SAIL document provides no basis for any of the following, and a consumer that acts as though it did is acting on its own evidence, outside this format ():

• that traffic from a listed prefix which makes no identity claim, or claims some other operator's identity, is from the publisher — no key in the format speaks to unidentified traffic, and exclusive concerns only which of the publisher's own services use a prefix, not who else does;
• that the publisher owns, is the registered holder of, or is the sole occupant of any prefix it lists;
• anything about parties other than the publisher — two publishers may list overlapping prefixes, each truthfully;
• either refutation above when the corresponding flag is false or absent.

[[EDNOTE: Open issue: We should define response error codes and a way for signaling why SAIL metadata caused the request to be refused.]]

Publisher Responsibilities The value of a SAIL document to consumers, and the safety of its assertions, rest entirely on the publisher keeping it true.

Accuracy:

A publisher MUST NOT list prefixes it does not control, nor any prefix from which its traffic cannot actually originate, nor any prefix outside the space permitted by . It MUST NOT set exhaustive or exclusive unless the assertion is true when published and is expected to remain true until the document's effective expiry (). In particular, a publisher SHOULD add new egress addresses to the document, and allow the previously published document's validity period to elapse, before an exhaustive service begins sending from them. A false assertion harms no third party; its cost falls on the publisher's own traffic and on consumers that wanted it ().

Completeness:

A publisher SHOULD set exhaustive on every service for which it is true. A document without it lets consumers confirm genuine requests but never refute impostors, which forgoes the format's main benefit. A publisher MAY list, with no prefixes and exhaustive set, names it has retired or has never used but sees impersonated (), and MAY list, as an exclusive prefix set that no service references, address space it holds but sends nothing from ().

Shared address space:

Listing address space that the publisher shares with other parties — a provider's general-purpose egress pool, for example — allows those parties' traffic to be confirmed as the publisher's services, since confirmation rests on source address alone. A publisher SHOULD prefer dedicated egress for any service whose identity consumers are expected to act on, and SHOULD NOT mark shared space exclusive: although the assertion concerns only the publisher's own services and may be literally true, it signals dedicated use and invites the over-reading cautioned against in .

Stable origin:

Because consumers attach trust to the pair (origin, service name) (), a publisher SHOULD serve its SAIL document from one well-known origin, link to it from its crawler documentation, and keep the host in its clients' User-Agent comment stable, so that consumers can associate the pair once and are not exposed to the confusion described in .

Other publisher requirements appear where they arise: location and discovery in , choice of service names in , canonical prefix form in , HTTP freshness in , and granularity of disclosure in .

Security Considerations

Fetching from Request-Derived Locations A consumer that derives the location of a SAIL document from the host of the URL in a request's User-Agent comment () is acting on attacker-controlled input. Any client can name any host, turning the consumer into a fetcher of the attacker's choosing. A consumer doing so SHOULD NOT fetch inline with handling the triggering request, MUST fetch only via "https" on port 443 at exactly the well-known path, MUST rate-limit fetches, and SHOULD cache negative results, so that a stream of requests naming fresh hosts cannot be used for request amplification or to probe internal services. Only the host of the URL is used: any port, path, or query it carries is ignored, a URL whose scheme is not "http" or "https" is ignored, and a consumer MUST NOT fetch when the host is an IP literal.

Unverified Prefix Claims Nothing in a SAIL document establishes that the publisher controls, holds, or solely occupies any address it lists. The format carries no proof of holdership, and a careless or hostile publisher can list space belonging to, or shared with, someone else. The inferences in tolerate this because each is keyed on a claim of the publisher's own identity: a wrong listing causes the publisher's names to be honoured or refused in the wrong places and affects no one else. Any decision keyed on the address alone — admitting, blocking, or labelling traffic from a listed prefix regardless of what identity it claims — lacks that protection and lands on whoever actually occupies the space. A consumer that makes such a decision is acting on its own evidence, outside this format, and should first establish the publisher's control of the prefix by independent means (for example forward-confirmed reverse DNS, or RPKI or IRR origin data), bearing in mind that control does not imply sole occupancy.

False Assertions A false exhaustive (the service in fact also sends from elsewhere) or a false exclusive (another of the publisher's services in fact also sends from the set) causes consumers to refuse genuine traffic from the publisher the treatment it is due. The harm falls on the publisher, which is the party that made the assertion; third parties are unaffected because neither assertion concerns them. In-document expiry () bounds how long a once-true assertion can outlive the deployment it described.

Infrastructure Disclosure A SAIL document necessarily reveals something about the publisher's egress. Fine-grained prefix sets and service names can reveal considerably more: data-centre footprint, provider relationships, internal service boundaries, and the timing of deployments as entries appear and disappear. Publishers should treat the document as a public interface and choose its granularity accordingly.

Redirected Documents Because a document reached by redirect is attributed to the origin that issued the redirect (), an origin can point its well-known URI at some other publisher's document. Doing so is self-limiting: the redirecting origin thereby adopts that document's prefixes and assertions as its own statement about its own service names, and can cause only claims of (its own origin, name) to be confirmed or refuted by them (). It gains nothing it could not have said directly, and the other publisher's names are unaffected.

Counterfeit Documentation URLs A SAIL document speaks for the origin at whose well-known URI it was requested (), not for any product token that appears in it. Nothing prevents an impostor from sending and serving, at https://not-really-acme.example/.well-known/sail, a well-formed document that lists the impostor's own addresses under a service named AcmeBot with exhaustive = true. A consumer that derives the document's location from the User-Agent comment () and then checks the request against that document will find the claim "confirmed" — correctly, in the narrow sense that the operator of not-really-acme.example did send it — while learning nothing about whether Acme did. This is why defines the identity as the pair (origin, service name) and why a request-supplied URL is not evidence of authority. A consumer that has pinned (acme.example, AcmeBot) by means outside the request is unaffected: the counterfeit document speaks only for not-really-acme.example, and if Acme's genuine document marks AcmeBot exhaustive, the impostor's request is refuted by it regardless of what the counterfeit says, and the consumer SHOULD act on that refutation. A consumer with no pinned origin for AcmeBot learns from the counterfeit document only that some party at not-really-acme.example vouches for the request, and MUST NOT carry over to it any standing it associates with Acme. advises publishers on keeping their origin stable so that this confusion does not arise for their own names. [[EDNOTE: Should we discuss Forwarded and X-Forwarded-For headers?]]

IANA Considerations

Well-Known URI IANA is requested to register the following in the "Well-Known URIs" registry established by :

URI suffix:

sail

Change controller:

IETF

Specification document(s):

this document,

Status:

permanent

Related information:

N/A

Media Type IANA is requested to register the following in the "Media Types" registry, per the procedure in :

Type name:

application

Subtype name:

sail+toml

Required parameters:

N/A

Optional parameters:

N/A. The format version is carried in-document by the schema key ().

Encoding considerations:

8bit. A SAIL document is TOML, which is UTF-8 text ().

Security considerations:

see

Interoperability considerations:

Consumers accept a document that parses and validates regardless of the Content-Type it was served with (), to accommodate publishers on static hosting that cannot set this type.

Published specification:

this document

Applications that use this media type:

operators of automated HTTP clients publishing their egress addresses, and sites and intermediaries verifying the claimed identity of automated requests

Fragment identifier considerations:

N/A

Additional information:

Deprecated alias names for this type: N/A. Magic number(s): N/A. File extension(s): N/A. Macintosh file type code(s): N/A.

Person & email address to contact for further information:

see Authors' Addresses

Intended usage:

COMMON

Restrictions on usage:

N/A

Author:

see Authors' Addresses

Change controller:

IETF

Structured Syntax Suffix [[EDNOTE: +toml is not currently in the IANA "Structured Syntax Suffixes" registry. Either this document registers it per Section 6 of (Name: TOML; +suffix: +toml; References: ; encoding, fragment identifier, and security considerations as for TOML text; Contact and Change controller: IETF), or the media type above falls back to a bare application/sail. To be resolved before this document progresses.]]

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. ISO/IEC 10646-1 defines a large character set called the Universal Character Set (UCS) which encompasses most of the world's writing systems. The originally proposed encodings of the UCS, however, were not compatible with many current applications and protocols, and this has led to the development of UTF-8, the object of this memo. UTF-8 has the characteristic of preserving the full US-ASCII range, providing compatibility with file systems, parsers and other software that rely on US-ASCII values but are transparent to other values. This memo obsoletes and replaces RFC 2279. This specification defines the addressing architecture of the IP Version 6 (IPv6) protocol. The document includes the IPv6 addressing model, text representations of IPv6 addresses, definition of IPv6 unicast addresses, anycast addresses, and multicast addresses, and an IPv6 node's required addresses. This document obsoletes RFC 3513, "IP Version 6 Addressing Architecture". [STANDARDS-TRACK] This memo discusses the strategy for address assignment of the existing 32-bit IPv4 address space with a view toward conserving the address space and limiting the growth rate of global routing state. This document obsoletes the original Classless Inter-domain Routing (CIDR) spec in RFC 1519, with changes made both to clarify the concepts it introduced and, after more than twelve years, to update the Internet community on the results of deploying the technology described. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. As IPv6 deployment increases, there will be a dramatic increase in the need to use IPv6 addresses in text. While the IPv6 address architecture in Section 2.2 of RFC 4291 describes a flexible model for text representation of an IPv6 address, this flexibility has been causing problems for operators, system engineers, and users. This document defines a canonical textual representation format. It does not define a format for internal storage, such as within an application or database. It is expected that the canonical format will be followed by humans and systems when representing IPv6 addresses as text, but all implementations must accept and be able to handle any legitimate RFC 4291 format. [STANDARDS-TRACK] This memo reiterates the assignment of an IPv4 address block (192.0.0.0/24) to IANA. It also instructs IANA to restructure its IPv4 and IPv6 Special-Purpose Address Registries. Upon restructuring, the aforementioned registries will record all special-purpose address blocks, maintaining a common set of information regarding each address block. IANA IANA RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This memo defines a path prefix for "well-known locations", "/.well-known/", in selected Uniform Resource Identifier (URI) schemes. In doing so, it obsoletes RFC 5785 and updates the URI schemes defined in RFC 7230 to reserve that space. It also updates RFC 7595 to track URI schemes that support well-known URIs in their registry. The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document describes the overall architecture of HTTP, establishes common terminology, and defines aspects of the protocol that are shared by all versions. In this definition are core protocol elements, extensibility mechanisms, and the "http" and "https" Uniform Resource Identifier (URI) schemes. This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230. The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP caches and the associated header fields that control cache behavior or indicate cacheable response messages. This document obsoletes RFC 7234. This document specifies and extends the "Robots Exclusion Protocol" method originally defined by Martijn Koster in 1994 for service owners to control how content served by their services may be accessed, if at all, by automatic clients known as crawlers. Specifically, it adds definition language for the protocol, instructions for handling errors, and instructions for caching. Informative References This memo specifies an optional interim mechanism for IPv6 sites to communicate with each other over the IPv4 network without explicit tunnel setup, and for them to communicate with native IPv6 domains via relay routers. [STANDARDS-TRACK] This document defines a date and time format for use in Internet protocols that is a profile of the ISO 8601 standard for representation of dates and times using the Gregorian calendar. To reduce the likelihood of conflict and confusion when relating documented examples to deployed systems, an IPv6 unicast address prefix is reserved for use in examples in RFCs, books, documentation, and the like. Since site-local and link-local unicast addresses have special meaning in IPv6, these addresses cannot be used in many example situations. The document describes the use of the IPv6 address prefix 2001:DB8::/32 as a reserved prefix for use in documentation. This memo provides information for the Internet community. We propose here a service that enables nodes located behind one or more IPv4 Network Address Translations (NATs) to obtain IPv6 connectivity by tunneling packets over UDP; we call this the Teredo service. Running the service requires the help of "Teredo servers" and "Teredo relays". The Teredo servers are stateless, and only have to manage a small fraction of the traffic between Teredo clients; the Teredo relays act as IPv6 routers between the Teredo service and the "native" IPv6 Internet. The relays can also provide interoperability with hosts using other transition mechanisms such as "6to4". [STANDARDS-TRACK] Three IPv4 unicast address blocks are reserved for use in examples in specifications and other documents. This document describes the use of these blocks. This document is not an Internet Standards Track specification; it is published for informational purposes. This document provides guidance for the Internet Assigned Numbers Authority (IANA) in assigning IPv4 multicast addresses. It obsoletes RFC 3171 and RFC 3138 and updates RFC 2780. This memo documents an Internet Best Current Practice. This document discusses the algorithmic translation of an IPv6 address to a corresponding IPv4 address, and vice versa, using only statically configured information. It defines a well-known prefix for use in algorithmic translations, while allowing organizations to also use network-specific prefixes when appropriate. Algorithmic translation is used in IPv4/IPv6 translators, as well as other types of proxies and gateways (e.g., for DNS) used in IPv4/IPv6 scenarios. [STANDARDS-TRACK] This document describes an architecture for an infrastructure to support improved security of Internet routing. The foundation of this architecture is a Resource Public Key Infrastructure (RPKI) that represents the allocation hierarchy of IP address space and Autonomous System (AS) numbers; and a distributed repository system for storing and disseminating the data objects that comprise the RPKI, as well as other signed objects necessary for improved routing security. As an initial application of this architecture, the document describes how a legitimate holder of IP address space can explicitly and verifiably authorize one or more ASes to originate routes to that address space. Such verifiable authorizations could be used, for example, to more securely construct BGP route filters. This document is not an Internet Standards Track specification; it is published for informational purposes. This document defines procedures for the specification and registration of media types for use in HTTP, MIME, and other Internet protocols. This memo documents an Internet Best Current Practice. This document describes Automatic Multicast Tunneling (AMT), a protocol for delivering multicast traffic from sources in a multicast-enabled network to receivers that lack multicast connectivity to the source network. The protocol uses UDP encapsulation and unicast replication to provide this functionality. The AMT protocol is specifically designed to support rapid deployment by requiring minimal changes to existing network infrastructure. Experience with the 6to4 transition mechanism defined in RFC 3056 ("Connection of IPv6 Domains via IPv4 Clouds") has shown that the mechanism is unsuitable for widespread deployment and use in the Internet when used in its anycast mode. Therefore, this document requests that RFC 3068 ("An Anycast Prefix for 6to4 Relay Routers") and RFC 6732 ("6to4 Provider Managed Tunnels") be made obsolete and moved to Historic status. It recommends that future products should not support 6to4 anycast and that existing deployments should be reviewed. This complements the guidelines in RFC 6343. Many sites connected to the Internet make use of IPv4 addresses that are not globally unique. Examples are the addresses designated in RFC 1918 for private use within individual sites. Devices in such environments may occasionally originate Domain Name System (DNS) queries (so-called "reverse lookups") corresponding to those private-use addresses. Since the addresses concerned have only local significance, it is good practice for site administrators to ensure that such queries are answered locally. However, it is not uncommon for such queries to follow the normal delegation path in the public DNS instead of being answered within the site. It is not possible for public DNS servers to give useful answers to such queries. In addition, due to the wide deployment of private-use addresses and the continuing growth of the Internet, the volume of such queries is large and growing. The AS112 project aims to provide a distributed sink for such queries in order to reduce the load on the corresponding authoritative servers. The AS112 project is named after the Autonomous System Number (ASN) that was assigned to it. This document describes the steps required to install a new AS112 node and offers advice relating to such a node's operation. This document obsoletes RFC 6304. AS112 provides a mechanism for handling reverse lookups on IP addresses that are not unique (e.g., RFC 1918 addresses). This document describes modifications to the deployment and use of AS112 infrastructure that will allow zones to be added and dropped much more easily, using DNAME resource records. This approach makes it possible for any DNS zone administrator to sink traffic relating to parts of the global DNS namespace under their control to the AS112 infrastructure without coordination with the operators of AS112 infrastructure. This memo updates the IANA IPv4 and IPv6 Special-Purpose Address Registries to address issues raised by the definition of a "global" prefix. It also corrects several errors in registry entries to ensure the integrity of the IANA Special-Purpose Address Registries. This memo updates RFC 6890. This document describes a mechanism for creating, encoding, and verifying digital signatures or message authentication codes over components of an HTTP message. This mechanism supports use cases where the full HTTP message may not be known to the signer and where the message may be transformed (e.g., by intermediaries) before reaching the verifier. This document also describes a means for requesting that a signature be applied to a subsequent HTTP message in an ongoing HTTP exchange.

Acknowledgments TODO