]> Google, LLC

warren@kumari.net

Alston Networks

alston.networks@gmail.com

Cisco

evyncke@cisco.com

Cisco

suresh.krishnan@gmail.com

Independent

d3e3e3@gmail.com

Internet Internet Area Working Group Internet-Draft Documents describing protocols intended solely for use within "limited domains" often rely on edge filtering at every boundary node to prevent domain-internal traffic from leaking to the global Internet (and vice versa). Relying purely on administrative filtering creates "fail-open" designs that are susceptible to configuration errors, ACL bypass, and hardware table exhaustion. This document describes design principles and concrete mechanisms that allow limited-domain protocols to "fail-closed" by default. By leveraging Layer-2 encapsulation identifiers (such as dedicated or extended EtherTypes), link-local address scoping, and / or Hop-Limit boundaries, protocol designers can significantly reduce the operational and security risks associated with limited domain protocols. These mechanisms are not applicable to all protocols intended for use in a limited domain, but if implemented on certain classes of protocols, can significantly reduce the risks. Discussion Venues Discussion of this document takes place on the Internet Area Working Group Working Group mailing list (int-area@ietf.org), which is archived at . Source for this draft and an issue tracker can be found at .

Introduction discusses the concept of "limited domains", provides examples of limited domains, as well as Examples of Limited Domain Solutions, including Service Function Chaining (SFC ), Segment Routing, "Creative uses of IPv6 features" (including Extension headers, e.g., for in situ Operations, Administration, and maintenance ). In order to provide context, this document will quote extensively from , but it is assumed that the reader will actually read in its entirety. Section 3, notes:

• A common argument is that if a protocol is intended for limited use, the chances are very high that it will in fact be used (or misused) in other scenarios including the so-called open Internet. This is undoubtedly true and means that limited use is not an excuse for bad design or poor security. In fact, a limited use requirement potentially adds complexity to both the protocol and its security design, as discussed later.

Notably, in Section 2, states:

• Domain boundaries that are defined administratively (e.g., by address filtering rules in routers) are prone to leakage caused by human error, especially if the limited domain traffic appears otherwise normal to the boundary routers. In this case, the network operator needs to take active steps to protect the boundary. This form of leakage is much less likely if nodes must be explicitly configured to handle a given limited-domain protocol, for example, by installing a specific protocol handler.

In addition, Section 6, notes:

• Today, where limited domains exist, they are essentially created by careful configuration of boundary routers and firewalls. If a domain is characterized by one or more address prefixes, address assignment to hosts must also be carefully managed. This is an error-prone method, and a combination of configuration errors and default routing can lead to unwanted traffic escaping the domain. Our basic assumption is therefore that it should be possible for domains to be created and managed automatically, with minimal human configuration. We now discuss requirements for automating domain creation and management.

This document discusses some of the mechanisms which protocol designers can use to limit the scope of their protocols to a single link. If the protocol is intended to be used across multiple links, but should not be forwarded beyond a single administrative domain, then the protocol designer should consider making the protocol "fail-closed" rather than "fail-open", as described below. This is primarily targeted towards protocols which are intended to primarily be used within a single layer-2 broadcast domain, or for protocols which provide a transport type service (similar to MPLS or SRv6) and are not intended to remain within a single administrative domain.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Some types of limited domain protocols Section 3 discusses some examples of Limited Domains, based mainly on the network type (e.g. Home, Sensor Networks, Data Centers, etc). This section instead classifies the types of limited domain protocols based more on their intended use, and technology. Broadly speaking, there are two types of limited domain protocols:

• Layer-2 type limited domain protocols: These are protocols that are intended to be used within a single LAN segment.
• Transport type service (for example MPLS and SRv6): These protocols are intended to provide a transport service, and are intended to remain within a single administrative domain such as an Enterprise or a Service Provider network.

Fail-open versus Fail-closed Protocols can be broadly classified as either "fail-open" or "fail-closed". Fail-closed protocols are those that require explicit interface or device-wide configuration to enable them to be accepted or processed when received on an interface. A classic example of a fail-closed protocol is MPLS (): In order to allow MPLS to transit an interface, the operator must enable the MPLS protocol on that interface and on the device itself. This ensures that outside MPLS traffic does not leak in or out of the network / domain. Fail-open protocols are those that require explicit configuration in order to ensure that they do not leak out of a domain, for example, through the application of filters. An example of a fail-open protocol is SRv6 - in order to ensure that SRv6 traffic does not leak out of a network, the operator must explicitly filter this traffic, and, in order to ensure that SRv6 traffic does not leak in, the operator must explicitly filter SRv6 traffic. Fail-open protocols are inherently riskier than fail-closed protocols, as they rely on perfect configuration of filters on all interfaces at the boundary of a domain, and, if the filters are removed for any reason (for example, during troubleshooting), there is a risk of inbound or outbound leaks. In addition, some devices or interfaces may have limitations in the size and complexity of filters that can be applied, and so adding new filter entries to limit leaks of a new protocol may not be possible. Fail-closed protocols, on the other hand, do not require any explicit filtering. In order for the protocol to be accepted and processed when received on an interface, the operator must explicitly enable the protocol on that interface and on the device itself. In addition, there is less risk of operational mistakes, as it does not rely on filters that may be limited in number and complexity. Finally, fail-closed protocols do not require that operators of networks outside of the limited domain implement filters to protect their networks from the limited domain traffic.

IP Hop-Limit Limiting -=-=-=- # IP Hop-Limit Limiting Some limited-domain protocols are intended to operate strictly within a single IP subnet or link. In these cases, protocol specifications SHOULD require that the IP Hop-Limit (or IPv4 TTL) to be set to 1 upon transmission. As standard routers decrement the Hop-Limit upon forwarding and discard packets when the count reaches zero, setting the Hop-Limit to 1 ensures that the traffic cannot be forwarded off-link. This differs from the Generalized TTL Security Mechanism (GTSM, ), which sets the Hop-Limit to 255 to verify that the sender is on-link and protect against off-link spoofing. Which option to choose (if either) depends on the specific requirements of the protocol.

IPv4 Multicast Addressing Some protocols (e.g OSPF) use addresses from the IP Local Network Control Block , (224.0.0/24). In addition to providing a discovery mechanism, this traffic is not forwarded off-link, providing a simple and effective way to limit the scope of the protocol. In some (rare) cases, IPv4 "Link Local" addresses may be an appropriate mechanism to limit the scope of the protocol, but this such a niche case that it is not discussed further here.

IPv6 Link Local Addresses Link-Local IPv6 Unicast Addresses ( Section 2.5.6) are used for communication between nodes on a single link. They are not routable and are not forwarded by routers. In cases where a limited-domain protocol is intended to be used only within a single link, the use of IPv6 Link-Local addresses can be an effective way to limit the scope of the protocol.

Making a layer-3 type limited-domain protocol fail-closed One way to make a limited-domain protocol fail-closed is to assign it a unique layer-2 protocol identifier, usually an EtherType. This mechanism is used by MPLS. In modern router and hosts, if such a protocol identifier is not enabled on an interface, then the Ethernet chip-set will ignore the frame, and the node will not see or process it. Thus, it is necessary to specifically enable the layer-2 protocol identifier on all relevant interfaces inside the limited domain, and the protocol will be blocked at the domain boundary where the protocol has not been so enabled. This is a simple and effective mechanism to ensure that the protocol does not leak out of the limited domain if and when an operator makes a mistake in configuring filters based on identifiers appearing deeper in the frame such as IP addresses or IP protocol or header options. This layer-2 protocol identifier technique only works for transport-type limited domain protocols (i.e., protocols running at layer 3). Higher layer protocols cannot necessarily be protected in this way, and so cryptographically enforced mechanisms may need to be used instead (e.g., as done by ANIMA in and ).

Ethernet Protocol Identification Figure 1 shows the general format of Ethernet frames. The relevant protocol identification field occurs after the destination and source MAC addresses and any tags (such a VLAN tags). The alternatives for protocol identification are discussed in Section 3 of .

Ethernet Frame Format

This document considers EtherType protocol identification. An EtherType is an unsigned 16-bit field in an Ethernet frame with a value in the range of 0x0600 to 0xFFFF, and so it is a somewhat limited resource; however, there exists a special Extended EtherType (0x88B7) that can be suffixed by an Organizationally Unique Identifier (OUI) followed by a further 16-bits identifying the protocol relative to that OUI as discussed in Section 3 of . These alternatives of a direct EtherType or use of the Extended EtherType for the case of the IANA OUI are illustrated in Figure 2. The following subsections discuss the factors which may influence the choice between these alternatives when use of such layer 2 protocol identification, to make the isolation of a limited domain more robust, is warranted.

EtherType Based Protocol Identification

Because specific EtherTypes are a limited resource, an Extended EtherType SHOULD be used unless there is a strong reason why it will not work satisfactorily and a specific EtherType is required.

Extended EtherType Protocol Identification The main advantage of using an Extended EtherType with an IANA Protocol Number, as shown in Figure 2, is that such a number can be allocated by IANA with Expert Review based on an Internet Draft and is thus relatively easy to obtain. The main disadvantage is that the protocol identification is 5 bytes longer than a specific dedicated EtherType.

Hardware Forwarding Considerations While Extended EtherTypes () solve the namespace exhaustion issue for 16-bit EtherTypes, protocol designers must remain mindful of hardware parsing pipeline constraints. High-throughput "merchant silicon" and network processor ASICs are heavily optimized to parse standard 16-bit Layer-2 EtherTypes at line rate within a fixed-depth initial parsing window. In contrast, variable-length or extended EtherType encapsulations (e.g., SNAP headers or 8-octet Extended EtherTypes) can introduce forwarding trade-offs on certain switching silicon:

• Parser Depth and Recirculation: Some existing hardware engines cannot parse beyond a fixed offset in a single pass. Having an Extended EtherType before inner headers may trigger packet recirculation or force packets into a slower software/microcode exception path.
• TCAM and Flow Matching: Many Access Control Lists (ACLs) and flow-match engines have native primitives for matching 16-bit EtherTypes (e.g., IPv4, IPv6, MPLS), whereas matching on an OUI plus extended protocol ID may consume multiple TCAM lookups, or exceed key-generation capabilities.

Protocol designers should therefore balance the scarcity of standard 16-bit EtherTypes against the line-rate hardware parsing requirements of high-speed transit nodes.

Specific EtherType Protocol Identification The primary disadvantage of using a specific EtherType, as opposed to an Extended EtherType, is that assignment of such an EtherType is significantly more difficult than assignment of an Extended EtherType IANA protocol number. As discussed in , a specific EtherType can only be assigned by the IEEE Registration Authority under the following policy: "Since EtherTypes are a fairly scarce resource, the IEEE RAC has let us know that they will not assign a new EtherType to a new IETF protocol specification until the IESG has approved the protocol specification for publication as an RFC. In exceptional cases, the IEEE RA is willing to consider "early allocation" of an EtherType for an IETF protocol that is still under development as long as the request comes from and has been vetted by the IESG." ( Appendix B.1, citing ) During development and testing, a protocol can use a "Local Experimental Ethertype" (0x88b5 and 0x88b6 - ). Once the protocol is approved for publication, the IESG can request an EtherType from the IEEE. However, there is always a risk of some implementation using a Local Experimental EtherType not getting updated causing conflicts with a later different use of that experimental EtherType. The primary advantage of using a specific EtherType is the saving of 5 bytes relative to the use of the Extended EtherType with a protocol number under the IANA OUI.

Non-Ethernet Encapsulations and Layer-3 Overlays Many modern data centers, enterprise networks, and WANs deploy limited-domain features across routed multi-hop IP underlays using Layer-3 encapsulations (such as Geneve , VXLAN-GPE , GRE , or generic UDP encapsulation ). When designing limited-domain protocols for Layer-3 overlay environments, the same fundamental tension between fail-open and fail-closed designs applies:

The Fragility of IP-Level Filtering in Overlays Tunneling limited-domain traffic inside standard unicast IP/UDP packets without domain-specific transport identifiers exposes the traffic to the same leakage risks as native IP:

1. Routability by Default: If an inner packet is encapsulated in a standard UDP/IP header destined for an IP address that leaks into the global routing table, intermediate underlay routers will forward the encapsulated frame across domain boundaries unless explicitly filtered.
2. Transit Inspection Invisibility: Intermediate nodes that inspect only the outer IP/UDP headers cannot distinguish between legitimate transit traffic and sensitive limited-domain control payloads.

Security Considerations Protocols are designated as "limited domain" because something unexpected might happen if they leak outside of a domain with unified management. For example, VLAN or VPN or overlay identifiers may be misinterpreted resulting in the delivery of data to or the acceptance of data from unauthorized network nodes violating intended security constraints. The use of a layer-2 protocol identifier to provide a "fail closed" barrier at the domain border can significantly improve security by eliminating the opportunity for such misinterpretation.

IANA Considerations This document has no IANA actions.

References Normative References There is a noticeable trend towards network behaviors and semantics that are specific to a particular set of requirements applied within a limited region of the Internet. Policies, default parameters, the options supported, the style of network management, and security requirements may vary between such limited regions. This document reviews examples of such limited domains (also known as controlled environments), notes emerging solutions, and includes a related taxonomy. It then briefly discusses the standardization of protocols for limited domains. Finally, it shows the need for a precise definition of "limited domain membership" and for mechanisms to allow nodes to join a domain securely and to find other members, including boundary nodes. This document is the product of the research of the authors. It has been produced through discussions and consultation within the IETF but is not the product of IETF consensus. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References This document specifies a protocol for encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol. [STANDARDS-TRACK] This document specifies the architecture for Multiprotocol Label Switching (MPLS). [STANDARDS-TRACK] The use of a packet's Time to Live (TTL) (IPv4) or Hop Limit (IPv6) to protect a protocol stack from CPU-utilization based attacks has been proposed in many settings (see for example, RFC 2461). This document generalizes these techniques for use by other protocols such as BGP (RFC 1771), Multicast Source Discovery Protocol (MSDP), Bidirectional Forwarding Detection, and Label Distribution Protocol (LDP) (RFC 3036). While the Generalized TTL Security Mechanism (GTSM) is most effective in protecting directly connected protocol peers, it can also provide a lower level of protection to multi-hop sessions. GTSM is not directly applicable to protocols employing flooding mechanisms (e.g., multicast), and use of multi-hop GTSM should be considered on a case-by-case basis. This memo defines an Experimental Protocol for the Internet community. To participate in wide-area IP networking, a host needs to be configured with IP addresses for its interfaces, either manually by the user or automatically from a source on the network such as a Dynamic Host Configuration Protocol (DHCP) server. Unfortunately, such address configuration information may not always be available. It is therefore beneficial for a host to be able to depend on a useful subset of IP networking functions even when no address configuration is available. This document describes how a host may automatically configure an interface with an IPv4 address within the 169.254/16 prefix that is valid for communication with other devices connected to the same physical (or logical) link. IPv4 Link-Local addresses are not suitable for communication with devices not directly connected to the same physical (or logical) link, and are only used where stable, routable addresses are not available (such as on ad hoc or isolated networks). This document does not recommend that IPv4 Link-Local addresses and routable addresses be configured simultaneously on the same interface. [STANDARDS-TRACK] This specification defines the addressing architecture of the IP Version 6 (IPv6) protocol. The document includes the IPv6 addressing model, text representations of IPv6 addresses, definition of IPv6 unicast addresses, anycast addresses, and multicast addresses, and an IPv6 node's required addresses. This document obsoletes RFC 3513, "IP Version 6 Addressing Architecture". [STANDARDS-TRACK] This document provides guidance for the Internet Assigned Numbers Authority (IANA) in assigning IPv4 multicast addresses. It obsoletes RFC 3171 and RFC 3138 and updates RFC 2780. This memo documents an Internet Best Current Practice. This document describes an architecture for the specification, creation, and ongoing maintenance of Service Function Chains (SFCs) in a network. It includes architectural concepts, principles, and components used in the construction of composite services through deployment of SFCs, with a focus on those to be standardized in the IETF. This document does not propose solutions, protocols, or extensions to existing protocols. The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable. Network virtualization involves the cooperation of devices with a wide variety of capabilities such as software and hardware tunnel endpoints, transit fabrics, and centralized control clusters. As a result of their role in tying together different elements of the system, the requirements on tunnels are influenced by all of these components. Therefore, flexibility is the most important aspect of a tunneling protocol if it is to keep pace with the evolution of technology. This document describes Geneve, an encapsulation protocol designed to recognize and accommodate these changing capabilities and needs. Autonomic functions need a control plane to communicate, which depends on some addressing and routing. This Autonomic Control Plane should ideally be self-managing and be as independent as possible of configuration. This document defines such a plane and calls it the "Autonomic Control Plane", with the primary use as a control plane for autonomic functions. It also serves as a "virtual out-of-band channel" for Operations, Administration, and Management (OAM) communications over a network that provides automatically configured, hop-by-hop authenticated and encrypted communications via automatically configured IPv6 even when the network is not configured or is misconfigured. This document specifies automated bootstrapping of an Autonomic Control Plane. To do this, a Secure Key Infrastructure is bootstrapped. This is done using manufacturer-installed X.509 certificates, in combination with a manufacturer's authorizing service, both online and offline. We call this process the Bootstrapping Remote Secure Key Infrastructure (BRSKI) protocol. Bootstrapping a new device can occur when using a routable address and a cloud service, only link-local connectivity, or limited/disconnected networks. Support for deployment models with less stringent security requirements is included. Bootstrapping is complete when the cryptographic identity of the new key infrastructure is successfully deployed to the device. The established secure connection can be used to deploy a locally issued certificate to the device as well. In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document provides a framework for IOAM deployment and provides IOAM deployment considerations and guidance. Some IETF protocols make use of Ethernet frame formats and IEEE 802 parameters. This document discusses several aspects of such parameters and their use in IETF protocols, specifies IANA considerations for assignment of points under the IANA Organizationally Unique Identifier (OUI), and provides some values for use in documentation. This document obsoletes RFC 7042. This document describes a Same-Origin Policy (SOP) requirement for Resource Public Key Infrastructure (RPKI) Repository Delta Protocol (RRDP) servers and clients. Application of a SOP in RRDP client/server communication isolates resources such as Delta and Snapshot files from different Repository Servers, reducing possible attack vectors. This document updates RFC 8182.

Acknowledgments Much thanks to Deborah Brungard and Brian Carpenter, for their review and comments. Also much thanks to everyone else with whom we have discussed this topic; I've had numerous discussions with many many people on this, and I'm sure that I've forgotten some of them. Apologies if you were one of them.

Changelog

• • 02-07:
  • Add Security Considerations text about the risks of fail-open protocols and the benefits of fail-closed protocols.
  • Added Appendix contrasting MPLS and SRv6.
  • Added hardware processing notes.
    • Used Gemini to help me reformulate mess into something readable.
  • Rewrite Abstract.
• -02-03:
  • Add Security Considerations text about the risks of fail-open protocols and the benefits of fail-closed protocols.
• 01-02:
  • Add Donald Eastlake as an author.
  • Substantial re-write and expansion of material concerning specific and Extended EtherType protocol identification.
  • Add initial Security Considerations text.
• 00-01:
  • Deborah pointed out that "this only works for transport-type limited domain protocols (e.g., SRv6)" could be read as SRv6 fails-closed.

Appendix A: Structural vs. Policy-Enforced Boundaries (MPLS vs. SRv6) To illustrate the difference between fail-open and fail-closed designs, consider MPLS and Segment Routing over IPv6 (SRv6, ).

MPLS: Structural Fail-Closed Isolation MPLS provides an automatic fail-closed boundary by virtue of its Layer-2 encapsulation:

• Distinct Protocol Identifier: MPLS frames utilize dedicated EtherTypes (0x8847 for unicast, 0x8848 for multicast).
• Interface Enablement Requirement: A node will only process or forward MPLS frames on interfaces where MPLS forwarding is explicitly enabled.
• Drop by Default: If an MPLS-encapsulated packet is accidentally transmitted across a domain boundary to an un-configured interface, transit provider, or peering exchange, the receiving hardware cannot parse the payload as native IP and discards the frame at line rate.

Because the forwarding plane requires explicit protocol support and state at each hop, MPLS fails closed in the presence of configuration errors or interface miswiring.

SRv6: Policy-Enforced Fail-Open Exposure In contrast, SRv6 encapsulates domain-internal routing instructions within standard IPv6 headers:

• Standard Layer-2 Demuxing: SRv6 packets utilize standard IPv6 EtherType (0x86DD) and standard IPv6 Next Header values.
• Global Forwarding Compatibility: Any standard IPv6 router along a path will forward an SRv6 packet using ordinary routing procedures on the IPv6 Destination Address, regardless of whether that router participates in or recognizes Segment Routing.
• Reliance on Perimeter ACLs: As described in Section 5 of , preventing SRv6 packets from leaking outside the trusted domain (or preventing external packets from injecting unauthorized SIDs into the domain) requires strict, comprehensive ingress and egress filtering at every perimeter boundary.

If an edge filter is omitted, truncated due to TCAM limits, or temporarily removed during debugging, the network fails open: domain-internal packets are forwarded across the public Internet, and untrusted external entities may address internal function SIDs directly.