Hybrid Two-Step Performance Measurement Method

Introduction Successful resolution of challenges of automated network operation, as part of overall service orchestration and automated network operation, relies on a timely collection of accurate information that reflects the state of network elements. Because performing the analysis and act upon the collected information requires considerable computing and storage resources, the network operational state information is unlikely to be processed by the network elements themselves but will be exported to big data systems for processing and storing. The process of generating, and collecting network operational state information also referred to in this document as network telemetry, and transporting it for post-processing should work equally well with data flows or injected in the network test packets. describes a combination of elements of passive and active measurement as a hybrid measurement. Several technical methods have been proposed to enable the collection of network operational state information instantaneous to the packet processing, among them and . The instantaneous, i.e., in the data packet itself, collection of telemetry information simplifies the process of attribution of telemetry information to the particular monitored flow. On the other hand, this collection method impacts the data packets, potentially changing their treatment by the network elements. Also, the amount of information the instantaneous method collects might be incomplete because of the limited space it can be allotted. Other proposals defined methods to collect telemetry information in a separate packet from each node traversed by the monitored data flow. is an example of this approach to collecting telemetry information. These methods allow data collection from any arbitrary path and avoid directly impacting data packets. On the other hand, to unambiguously correlate telemetry information with the monitored flow requires explicit characteristic information about the monitored flow to be carried in the packet that carries telemetry information. This document introduces Hybrid Two-Step (HTS) as a new method of telemetry collection that improves accuracy of a measurement by separating the act of measuring or calculating the performance metric from the collecting and transporting this information while minimizing the overhead of the generated load in a network. HTS method extends the two-step mode of Residence Time Measurement (RTM) defined in to on-path network operational state collection and transport. HTS instruments data packets of the monitored flow or specially constructed test packets that are already equipped with a shim of on-path telemetry protocol to use as an HTS Trigger packet, making the process of attribution of telemetry to the data flow simple.

Conventions Used in This Document

Acronyms RTM Residence Time Measurement ECMP Equal Cost Multipath HMAC Hashed Message Authentication Code HTS Hybrid Two-Step () MTU Maximum Transmission Unit NSH Network Service Header RTT Round-Trip Time TLV Type-Length-Value ()

Terminology This document makes use of the following terms: Characteristic information - the interpretation in this document follows the definition of "Characteristic" in Section 3.2 in . HTS domain is an example of a "limited domain" as defined in . HTS domain may be identical to an IOAM domain (see Section 3 of ) or a controlled domain where the Alternate-Marking Method is used to measure performance metrics (see Section 7.1 of ). HTS ingress node, defined in , an HTS system that generates an HTS Trigger and HTS Follow-up packets. HTS intermediate node, defined in , an HTS system that receives the HTS Trigger and HTS Follow-up packets, HTS egress node, defined in , consumes HTS Trigger packet and terminates an HTS Follow-up packet. Telemetry is information reflecting the operational state of network elements, including performance metrics, collected and transported for analysis.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Problem Overview Performance measurement and network telemetry often require collecting operational state information - such as timestamps, queue occupancy, buffer utilization, interface load, or other metrics - at multiple points along a packet's forwarding path. Methods that embed telemetry directly into data packets simplify correlation with the monitored flow but may be limited by MTU constraints, may be incompatible with encrypted packets, and may introduce processing overhead that affects measurement accuracy. For encrypted packets, inserting telemetry information directly into the data packet may be impossible or may degrade measurement accuracy because the packet cannot be safely modified while in transit. Accurate comparison of timestamps depends on the availability of time synchronization across the HTS domain. In some networks, it is beneficial to collect telemetry at network elements other than the egress, depending on the performance metric and operational objective. For example, a head-end may optimize path selection based on upstream telemetry reflecting queue depth, buffer occupancy, or interface utilization. These challenges motivate the HTS method, which separates the act of measuring or collecting telemetry from the act of transporting it. HTS handles both performance measurements and telemetry information using the same two‑step mechanism. An HTS Trigger initiates telemetry collection, and one or more HTS Follow-up packets carry the collected information. These phases are described in .

Theory of Operation HTS measurement operates within an HTS domain consisting of an HTS Ingress node, zero or more HTS Intermediate nodes, and an HTS Egress node. HTS Trigger packets originate at the HTS Ingress node and are forwarded downstream through the HTS domain. HTS Follow‑up packets are generated by the HTS Egress node and sent upstream toward the HTS Ingress node. A reference HTS topology is illustrated in . HTS telemetry is ultimately consumed by a Collector, which is a logical function responsible for receiving, correlating, and analyzing HTS measurement results. The Collector may be co‑located with the HTS Egress node or may reside on a remote system such as a controller, analytics engine, or network management system. The Collector is not part of the HTS domain and does not process HTS packets directly.

An Example of an HTS Reference Topology |Intermediate | ---> | Egress | +---------+ +-------------+ +---------+ | | | | HTS Trigger | HTS Trigger | HTS Trigger | HTS Follow-up | HTS Follow-up | HTS Follow-up | (originates) | (adds telemetry) | (terminates) ----------------------------------------------------- | | Telemetry Export v +-------------+ | Collector | +-------------+ ]]> The HTS Egress node terminates HTS Follow‑up packets, validates the HTS shim and Telemetry Data TLVs, and exports the resulting measurement information to the Collector. HTS Error Codes, when present, are generated only by the HTS Egress node and delivered to the Collector as part of the telemetry export. Intermediate nodes do not generate HTS Error Codes. The HTS method consists of two phases:

Performing a measurement and/or obtaining network operational state information on a network element. HTS Trigger is a data or test packet instrumented to Trigger the collection of telemetry information on a network element. specifies operation of an HTS Trigger packet.
Collecting and transporting the measurement and/or the telemetry information. HTS Follow-up is a packet constructed to transport telemetry information that includes operational state and performance measurements originated on the nodes along the path traversed by the HTS Trigger. specifies operation of an HTS Follow-up packet.

HTS Packets HTS uses two packet types: HTS Trigger packets, which initiate telemetry collection, and HTS Follow-up packets, which carry the collected telemetry. The HTS Trigger and HTS Follow-up packets MUST traverse the same forwarding path within the HTS domain to ensure correct correlation of telemetry information. Characteristic Information, in context of HTS, refers to the set of fields carried in an HTS Trigger and HTS Follow-up packet that uniquely identifies the monitored flow and ensures that both packets traverse the same forwarding path within the HTS domain. For a given monitored flow, Characteristic Information MUST be invariant and MUST be identical in the HTS Trigger and all HTS Follow-up packets associated with that Trigger. HTS nodes MUST use this invariant information to correlate HTS Follow-up packets with the corresponding HTS Trigger packets. Characteristic Information MUST preserve the forwarding behavior of the HTS Trigger packet and MUST NOT be modified by HTS nodes. Below are some examples of Characteristic Information:

IOAM: Namespace-ID, IOAM-Trace-Type, Flow ID, Sequence Number
Segment Routing: SID list.
NSH: Base Header and Service Path Header
ECMP environments: reusing the same entropy label or IPv6 Flow Label.
Any other fields used by the forwarding plane to select the path.

The exact composition of Characteristic Information is transport‑specific and outside the scope of this document, but all HTS nodes MUST treat it as immutable and use it to correlate HTS Trigger and Follow-up packets.

HTS Trigger Packets An HTS Trigger is a user data packet or specially constructed test packet instrumented to cause network elements to collect telemetry information. The Trigger packet includes characteristic information that allows HTS nodes to associate the Trigger with the monitored flow. The specific format of the Trigger depends on the telemetry framework used in the HTS domain.

HTS Trigger in In-Situ OAM When HTS is used in an IOAM domain, the Trigger packet carries an IOAM Option-Type set to the "IOAM Hybrid Two-Step Option-Type" value (TBA1) allocated by IANA (). The HTS Trace IOAM Header is shown in .

Hybrid Two-Step Trace IOAM Header The HTS Trigger includes HTS IOAM Header (shown in ) consists of:

IOAM Namespace-ID - as defined in Section 5.3 ;
Flags - as defined in Section 3.2 ;
Extension-Flags - as defined in Section 3.2 ;
IOAM-Trace-Type - as defined in Section 5.4 ;
Flow ID (Optional) - as defined in Section 3.2 ;
Sequence Number (Optional) - as defined in Section 3.2 .

These fields provide the characteristic information needed for HTS nodes to associate the HTS Trigger packet with the monitored flow.

HTS Trigger in the Alternate-Marking Method In an Alternate‑Marking domain (, ), any packet in a flow that carries the marking information MAY serve as an HTS Trigger. The exact transport encapsulation of the HTS Trigger packet is specific to the Alternate‑Marking deployment and is outside the scope of this document. Regardless of the encapsulation, the HTS Trigger packet MUST carry characteristic information sufficient for HTS nodes to associate it with the monitored flow.

HTS Follow-up Packets An HTS Follow-up packet carries telemetry information collected by HTS nodes along the path traversed by the HTS Trigger packet. The ingress network element () that originates the HTS Trigger packet also originates the corresponding HTS Follow-up packet. In upstream collection mode, an HTS Follow-up packet MAY be generated without a preceding Trigger, provided that the Follow-up packet includes Characteristic Information sufficient to identify the monitored flow. An HTS Follow-up packet MUST carry the same Characteristic Information as the associated HTS Trigger packet. HTS nodes MUST treat this information as immutable and MUST use it to ensure that the HTS Follow-up packet traverses the same forwarding path as the HTS Trigger packet. The Follow-up packet consists of the transport encapsulation determined by the Characteristic Information, followed by the HTS shim and one or more Telemetry Data TLVs. The HTS shim indicates the maximum allowed size of the Follow-up packet and the telemetry profile requested by the ingress node. Only one outstanding Follow-up packet MAY exist per path at a node. If a node receives a new HTS Trigger packet while an HTS Follow-up packet for the same flow is still pending, the node MUST first originate the Follow-up packet for the earlier HTS Trigger before processing the new HTS Trigger packet. The following sections describe the operation of HTS nodes in the downstream mode of collecting the telemetry information. In the upstream mode, the behavior of HTS nodes, in general, identical with the exception that the HTS Trigger packet does not precede the HTS Follow-up packet.

Operation of the HTS Ingress Node The HTS ingress node is the network element that originates HTS Trigger packets and the corresponding HTS Follow-up packets. The ingress node is responsible for constructing the HTS Trigger packet, initiating telemetry collection, and generating HTS Follow-up packets that carry telemetry collected along the forwarding path.

Responsibilities of the HTS Ingress Node Upon originating an HTS Trigger, the ingress node MUST:

Construct the HTS Trigger packet with the appropriate transport encapsulation and Characteristic Information ().
Ensure that the HTS Trigger packet traverses the intended forwarding path, consistent with the monitored flow.
Record the Characteristic Information so that the HTS Follow-up packet can be constructed with identical values.
Transmit the Trigger packet into the HTS domain.

Originating the HTS Follow-up Packet For each HTS Trigger packet, the HTS ingress node MUST originate a corresponding HTS Follow-up packet unless operating in upstream collection mode. The ingress node constructs the HTS Follow-up packet using the format shown in . The Follow-up packet consists of the transport encapsulation determined by the Characteristic Information, followed by the HTS shim and one or more Telemetry Data TLVs. The ingress node MUST populate the HTS shim fields as described below and MUST ensure that the Follow-up packet carries the same Characteristic Information as the associated HTS Trigger. The HTS ingress node MUST:

Reuse the same Characteristic Information as the Trigger packet.
- This ensures that the Follow-up packet traverses the same forwarding path.
- The ingress node MUST NOT modify Characteristic Information once selected.
Construct the HTS shim () including:
- Version
- HTS Shim Length
- Flags
- Sequence Number
- HTS Max Length
- Telemetry Data Profile
Initialize the Sequence Number to zero for the first Follow-up packet associated with a Trigger.
Include one or more Telemetry Data TLVs, if local telemetry is collected at the ingress node.
Transmit the HTS Follow-up packet after sending the corresponding HTS Trigger packet.

Follow-up Packet Format Fields of the HTS shim are as follows:

Version (Ver) is the two-bits long field. It specifies the version of the HTS shim format. This document defines the format for the 0b00 value of the field.
HTS Shim Length is the six bits-long field. It defines the length of the HTS shim in octets. The minimal value of the field is eight octets.
Flags Field Format
Flags is eight-bits long. The format of the Flags field displayed in .
- Full (F) flag MUST be set to zero by the node originating the HTS Follow-up packet and MUST be set to one by the node that does not add its telemetry data to avoid exceeding MTU size.
- The node originating the Follow-up packet MUST zero the Reserved field and ignore it on the receipt.
Sequence Number is one octet-long field. The zero-based value of the field reflects the place of the HTS Follow-up packet in the sequence of the HTS Follow-up packets that originated in response to the same HTS Trigger. The ingress node MUST set the value of the field to zero.
Error Code is one octet-long field. It MUST be zeroed on transmission and ignored on receipt. An HTS Egress node sets the value of the Error Code based on processing an HTS Follow-up packet before delivering telemetry information to a Collector. IANA is requested to assign values according to .
HTS Max Length is four octet-long field. The value of the HTS Max Length field indicates the maximum length of the HTS Follow-up packet in octets. An operator MUST be able to configure the HTS Max Length field's value. The value SHOULD be set equal to the path MTU.
Telemetry Data Profile is the optional variable-length field of bit-size flags. Each flag indicates the requested type of telemetry data to be collected at each HTS node. The increment of the field is four bytes with a minimum length of zero. For example, IOAM-Trace-Type information defined in , Sequence Number and/or Flow ID () can be used in the Telemetry Data Profile field.
Telemetry Data TLV Format
Telemetry Data TLV is a variable-length field. Multiple TLVs MAY be placed in an HTS packet. Additional TLVs may be enclosed within a given TLV, subject to the semantics of the (outer) TLV in question. presents the format of a Telemetry Data TLV, where fields are defined as the following:
- Type - a one-octet-long field that characterizes the interpretation of the Value field.
- Reserved - one-octet-long field.
- Length - two-octet-long field equal to the length of the Value field in octets.
- Value - a variable-length field. The value of the Type field determines its interpretation and encoding. IOAM data fields, defined in , MAY be carried in the Value field.

All multibyte fields defined in this specification are in network byte order.

Operation of HTS Intermediate Nodes An HTS intermediate node is a network element that receives and forwards HTS Trigger packets and HTS Follow-up packets. Intermediate nodes collect telemetry data and contribute it to the Follow-up packet while ensuring that the packet remains within the MTU constraints of the path. Intermediate nodes MUST NOT modify Characteristic Information and MUST use it to correlate Trigger and Follow-up packets.

Processing an HTS Trigger Packet Upon receiving an HTS Trigger packet, an intermediate node MUST:

Verify Characteristic Information
- The node MUST verify that the Trigger contains valid Characteristic Information. If validation fails, the node MUST forward the packet unchanged.
Copy transport encapsulation
- The node MUST record the transport encapsulation and Characteristic Information for use when constructing a Follow-up packet.
Start the HTS Follow-up Timer
- The node MUST start a timer associated with the monitored flow. The timer ensures that a Follow-up packet will be originated even if no Follow-up packet is received from downstream nodes.
Forward the Trigger packet
- The node MUST forward the Trigger packet according to the forwarding behavior determined by the Characteristic Information.

Processing an HTS Follow-up Packet Upon receiving an HTS Follow-up packet, an intermediate node MUST:

Validate the Follow-up packet
- Verify that Characteristic Information matches a known Trigger.
- Verify that the Full flag is not set. If validation fails, the node MUST forward the packet unchanged and MUST NOT add telemetry.
Stop the HTS Follow-up Timer
- The node MUST stop the timer associated with the flow.
- The node MUST collect telemetry data as requested by the Telemetry Data Profile or by local HTS policy.
Determine whether telemetry fits within HTS Max Length
- If adding telemetry does not exceed HTS Max Length:
- - Append telemetry as a new Telemetry Data TLV
  - Forward the Follow-up packet
- If adding telemetry would exceed HTS Max Length:
- - Set the Full flag in the received Follow-up packet
  - Forward the received Follow-up packet unchanged
  - Originate a new Follow-up packet using the stored transport encapsulation and Characteristic Information
  - Increment the Sequence Number
  - Insert collected telemetry into the first Telemetry Data TLV
  - Transmit the new Follow-up packet

HTS Follow-up Timer Expiration If the HTS Follow-up Timer expires before a Follow-up packet is received, the intermediate node MUST:

Originate a new Follow-up packet
- Using the stored transport encapsulation and Characteristic Information.
Initialize the HTS shim
- Version = 0b00
- Sequence Number = 0
- HTS Shim Length and Telemetry Data Profile MAY be set according to local policy.
Insert telemetry data
- The node MUST insert locally collected telemetry into the first Telemetry Data TLV.
Transmit the Follow-up packet

Operation of the HTS Egress Node The HTS Egress node terminates synthetic HTS Trigger packets and forwards data packets used as HTS Trigger packets according to normal forwarding behavior. The Egress node is responsible for correlating synthetic HTS Trigger packets with the corresponding HTS Follow‑up packets, validating the HTS shim and Telemetry Data TLVs contained in HTS Follow‑up packets, generating HTS Error Codes when processing errors are detected, and exporting the resulting telemetry to the Collector. The Collector is a logical function that receives, correlates, and analyzes HTS measurement results. The Collector may be co‑located with the Egress node or may reside on a remote system such as a controller, analytics engine, or network management system. The Collector is not part of the HTS domain and does not process HTS packets directly. HTS Error Codes are generated only by the HTS Egress node and are delivered to the Collector as part of the telemetry export. HTS Trigger packets do not carry HTS shim or Telemetry Data TLVs and cannot be malformed in an HTS‑specific sense.

Processing an HTS Trigger Packet When an HTS Trigger packet (synthetic or data‑packet) arrives at the Egress node, the node performs the following steps:

Record Trigger context.
- The Egress node extracts the Sequence Number and Characteristic Information and records this measurement context. No correlation between Trigger and Follow‑up packets is required; the Sequence Number in the HTS Follow‑up packet identifies the relative location of the HTS Follow-up packet in the sequence of the Follow-up packets generated due to the same HTS Trigger packet.
Start the HTS Collection Timer.
- The Egress node starts the HTS Collection Timer associated with the Trigger. The timer is used to detect loss or excessive delay of the corresponding HTS Follow‑up packet.
Apply Trigger termination behavior.
- If the Trigger packet is synthetic, the Egress node MUST terminate the packet.
- If the Trigger packet is a data packet used as an HTS Trigger, the Egress node MUST forward the packet according to normal forwarding behavior.

Synthetic Trigger packets and data‑packet Triggers do not carry HTS shim or Telemetry Data TLVs and cannot be malformed in an HTS‑specific sense.

Processing an HTS Follow‑up Packet HTS Follow‑up packets may arrive at the Egress node as a sequence of packets generated in response to a single HTS Trigger packet. Each Follow‑up packet contains the HTS shim, Telemetry Data TLVs, and optionally the HTS HMAC TLV. HTS Error Codes generated during HTS Follow‑up packet validation are defined in . The conditions under which each Error Code is generated are specified in this section. The Egress node processes each Follow‑up packet in the sequence as follows:

Clear the HTS Collection Timer.
- Upon receiving any valid Follow‑up packet, the Egress node MUST clear the HTS Collection Timer associated with the monitored flow. Set Error Code to "No Error" value.
Validate the HTS HMAC TLV (if present).
- The Egress node performs HMAC verification before processing any telemetry data. If validation fails, set Error Code to "HMAC Failure" value. Go to step 13.
Validate the HTS shim.
- The Egress node verifies the Sequence Number and Full flag in each Follow‑up packet. Set Error Code to "Invalid HTS Shim" value. Go to step 13.
Validate Telemetry Data TLVs.
- The Egress node verifies the structure, length, and consistency of all Telemetry Data TLVs in each packet. If validation fails, set Error Code to "Invalid TLV" value. Go to step 13.
Restart the HTS Collection Timer.
- If the Full flag is set, indicating that additional Follow‑up packets are expected, the Egress node MUST restart the HTS Collection Timer.
Accumulate telemetry fragments.
- The Egress node stores the telemetry information from each Follow‑up packet in the sequence. If the Full flag is not set, the packet is the final Follow‑up packet for the Trigger.
Export telemetry to the Collector.
- When the final Follow‑up packet in the sequence is received (Full flag = 0), the Egress node aggregates all telemetry fragments collected and exports the complete telemetry record, including any HTS Error Codes, to the Collector using an out‑of‑band mechanism. The specific telemetry export protocol is outside the scope of this document.

HTS Collection Timer Expiration The HTS Collection Timer is started when the Egress node receives an HTS Trigger packet (synthetic or data‑packet). The timer is used to detect loss or excessive delay of HTS Follow‑up packets generated in response to that Trigger. Because multiple HTS Follow‑up packets may be generated for a single Trigger, the Egress node maintains the timer across the entire sequence of Follow‑up packets. When an HTS Follow‑up packet is received, the Egress node MUST clear the HTS Collection Timer associated with the monitored flow identified by a locally unique Characteristic Information. If the Full flag in the Follow‑up packet is set, indicating that additional Follow‑up packets are expected, he Egress node MUST restart the HTS Collection Timer. If the Full flag is not set, the Follow‑up packet is the final packet in the sequence, and the Egress node MUST NOT restart the timer. If the HTS Collection Timer expires before the final HTS Follow‑up packet in the sequence is received, the Egress node performs the following steps:

Generate a synthetic HTS Follow‑up packet.
- The Egress node originates a synthetic HTS Follow‑up packet for Sequence Number associated with the expired timer. This packet represents the Egress node’s best available telemetry for the Trigger's Characteristic Information.
Populate the HTS shim.
- The synthetic Follow‑up packet includes the Sequence Number and sets the Full flag to zero, indicating that no additional Follow‑up packets will be sent.
Insert an HTS Error Code.
- The Egress node MUST set Error Code value to "Follow‑up Not Received" (), indicating that one or more expected HTS Follow‑up packets were not received before the timer expired.
Export telemetry to the Collector.
- The Egress node aggregates all telemetry fragments collected for the Trigger’s Sequence Number, including any fragments received before timer expiration, and exports the resulting telemetry record to the Collector using an out‑of‑band mechanism. The specific telemetry export protocol is outside the scope of this document.

Operational Considerations This section describes operational aspects of deploying HTS in a network. These considerations are informational and do not impose additional protocol requirements beyond those specified in .

Path Congruence HTS relies on the Trigger and Follow‑up packets traversing the same forwarding path. Operators MUST ensure that the Characteristic Information used in the Trigger and Follow‑up packets is sufficient to preserve forwarding behavior across the HTS domain. Examples of fields that influence forwarding include:

Segment Routing SID lists
NSH Service Path Headers
IPv6 Flow Labels
Entropy Labels
IOAM Namespace-ID and IOAM-Trace-Type

Operators SHOULD verify that these fields are stable and predictable across the HTS domain. If forwarding behavior changes due to ECMP rebalancing, policy routing, or service‑function steering, HTS telemetry may be incomplete or inconsistent.

MTU and Packet Size Considerations HTS Follow‑up packets grow as telemetry is added by intermediate nodes. Operators SHOULD ensure that:

the HTS Max Length value is configured to avoid fragmentation,
the underlying transport encapsulation supports the expected packet size, and
intermediate nodes have sufficient headroom to append telemetry.

If an intermediate node cannot append telemetry without exceeding HTS Max Length, it will set the Full flag and originate a new Follow‑up packet. Operators SHOULD monitor this condition, as frequent Full‑flag events may indicate insufficient MTU headroom or excessive telemetry volume.

Timer Configuration HTS uses two types of timers:

HTS Follow‑up Timer (intermediate nodes)
HTS Collection Timer (egress nodes)

Operators SHOULD configure timer values based on:

expected path latency,
jitter characteristics,
telemetry volume, and
the number of intermediate nodes.

Timers that are too short may cause premature origination of Follow‑up packets. Timers that are too long may delay telemetry delivery.

Telemetry Volume and Profile Selection The Telemetry Data Profile determines which telemetry elements are collected by HTS nodes. Operators SHOULD select a profile that balances:

operational visibility,
packet size constraints,
processing overhead, and
the number of nodes in the HTS domain.

Profiles that request high‑volume telemetry (e.g., queue depth, buffer occupancy, interface load) may require larger HTS Max Length values or more frequent Full‑flag events.

Handling Mis‑injected or Malformed Packets HTS nodes MUST validate Trigger and Follow‑up packets as described in . Mis‑injected or malformed packets may occur due to:

incorrect configuration,
software defects,
malicious injection, or
inconsistent forwarding behavior.

Operators SHOULD monitor for validation failures and investigate persistent anomalies. Egress nodes MUST discard invalid Follow‑up packets and SHOULD log such events for operational visibility.

Upstream and Downstream Collection Modes HTS supports both upstream and downstream collection modes:

Downstream mode: The ingress node originates the Follow‑up packet.
Upstream mode: The egress node originates the Follow‑up packet if no Follow‑up is received before the HTS Collection Timer expires.

Operators SHOULD select the mode based on:

operational goals,
telemetry volume,
path length, and
processing capabilities of ingress and egress nodes.

Upstream mode may reduce packet size on long paths, while downstream mode provides earlier visibility into telemetry collected by intermediate nodes.

Deploying HTS in a Multicast Network HTS can be deployed in multicast networks, but operators should be aware that Trigger and Follow‑up packets behave differently in multicast forwarding trees.

Trigger Packet Behavior An HTS Trigger packet follows the multicast distribution tree determined by the transport encapsulation and Characteristic Information. As a result:

The Trigger fans out to all downstream branches.
Each downstream branch may traverse a different number of intermediate nodes.
Each leaf node (multicast receiver or egress HTS node) receives a copy of the Trigger.

HTS does not modify multicast forwarding behavior; the Trigger is replicated exactly as a normal multicast packet would be.

Follow‑up Packet Behavior HTS Follow‑up packets do not follow multicast replication semantics. A Follow‑up packet is always unicast toward the HTS collector or toward the node that originated the Trigger, depending on the deployment mode. As a result:

Each branch of the multicast tree produces its own Follow‑up packet.
Follow‑up packets from different branches may arrive at the collector at different times.
The collector MUST correlate Follow‑up packets using Characteristic Information and Sequence Number.

Network analytics should expect one train of Follow‑up packets per multicast branch, not one per Trigger.

Telemetry Volume Considerations Because each multicast branch produces a separate train of Follow‑up packets:

Telemetry volume increases with the number of multicast receivers.
HTS Max Length should be configured with awareness of the smallest MTU of all branches.
Frequent Full‑flag events on long branches may indicate insufficient MTU headroom.

Timer Considerations Each egress node in the multicast tree maintains its own HTS Collection Timer. Timer expirations may occur independently on different branches, especially when:

branch lengths differ,
congestion affects only part of the tree, or
Follow‑up packets are lost on specific branches.

Operators should expect asynchronous Follow‑up arrivals from different multicast receivers.

Collector Considerations The HTS collector MUST be prepared to receive:

multiple Follow‑up packets per Trigger,
each corresponding to a different multicast branch,
each containing telemetry from a distinct path.

Collectors SHOULD:

correlate Follow‑up packets using both the Characteristic Information and the HTS shim.
treat each Follow‑up as representing a unique multicast path,
aggregate results per Trigger if required by operational policy.

Operational Monitoring and Diagnostics Operators SHOULD monitor:

ratio of HTS Trigger to Follow‑up
timer expirations
validation failures.

Persistent issues may indicate:

misattributed Characteristic Information,
misconfiguration or overloading of telemetry profiles,
software defects.

Authentication in HTS Telemetry information may be used to drive network operation, closing the control loop for self-driving, self-healing networks. Thus, it is critical to provide a mechanism to protect the telemetry information collected using the HTS method.

HMAC Sub-TLV This document defines an optional HTS HMAC TLV that provides integrity protection for HTS shim fields and Telemetry Data TLVs. The format of the HMAC sub-TLV is displayed in .

HMAC Sub-TLV where fields are defined as follows:

Authentication Type - is a one-octet-long field, value 1 is allocated by IANA .
Length - two-octet-long field, set equal to the length of the Digest field in octets.
HMAC Type - is a one-octet-long field that identifies the type of the HMAC and the length of the digest and the length of the digest according to the HTS HMAC Type sub-registry (see ).
Digest - is a variable-length field that carries HMAC digest of the text that includes the encompassing TLV.

This specification defines the use of HMAC-SHA-256 truncated to 128 bits () in HTS. Future specifications may define the use in HTS of more advanced cryptographic algorithms or the use of digest of a different length. HMAC is calculated as defined in over text as the concatenation of the Sequence Number field of the Follow-up packet (see ) and the preceding data collected in the Telemetry Data TLV. The digest then MUST be truncated to 128 bits and written into the Digest field. Distribution and management of shared keys are outside the scope of this document. In the HTS authenticated mode, the Authentication sub-TLV MUST be present in each Telemetry Data TLV. HMAC MUST be verified before using any data in the included Telemetry Data TLV. If HMAC verification fails, the system MUST stop processing corresponding Telemetry Data TLV and notify an operator. Specification of the notification mechanism is outside the scope of this document.

Authentication Mechanisms in HTS To authenticate the Follow-up packets, operators MAY use authentication mechanisms available in the transport encapsulation used to carry HTS. Examples include:

IPsec AH ()/ESP () for IPv4 and IPv6
MACsec for Ethernet ()
DTLS for UDP‑based encapsulations ()
SRv6 HMAC TLV ()
NSH MD Type‑1 authentication metadata ()

Operators SHOULD enable authentication when HTS packets traverse untrusted or multi‑tenant environments.

Integrity of Characteristic Information Characteristic Information determines the forwarding behavior of HTS Trigger and Follow‑up packets. To prevent mis‑routing or mis‑injection, operators should use integrity mechanisms provided by the underlying transport encapsulation (e.g., IPsec AH/ESP, MACsec, NSH MD‑Type 1 authentication metadata). These mechanisms help ensure that Characteristic Information is not modified in transit. HTS nodes perform validation of Characteristic Information as part of normal HTS packet processing (). Integrity protection at the transport layer complements this validation by preventing unauthorized modification of fields that influence forwarding behavior.

Integrity of Telemetry Data HTS does not define new integrity mechanisms for telemetry data. Telemetry Data TLVs MAY be protected by existing OAM integrity mechanisms, such as:

IPsec AH ()/ESP () for IPv4 and IPv6
MACsec for Ethernet ()
DTLS for UDP‑based encapsulations ()
SRv6 HMAC TLV ()
NSH metadata integrity ()

Authentication of HTS Nodes HTS nodes SHOULD be authenticated using mechanisms appropriate to the deployment environment, such as:

control‑plane authentication (e.g., TLS, SSH, NETCONF/RESTCONF security)
routing‑protocol authentication (e.g., OSPFv3 authentication, BGP TTL security)
domain‑level trust models (e.g., SR domain trust, NSH SFC domain trust)

Authentication of HTS nodes ensures that Trigger and Follow‑up packets originate only from authorized devices.

IANA Considerations This document requests IANA to make several allocations and create registries associated with HTS. HTS also defines a new IOAM Option-Type, which must be allocated from the existing IOAM Option Types registry established by . HTS does not modify any other IOAM registries.

IOAM Option-Type for HTS This document defines a new IOAM HTS Option-Type, which instructs IOAM-capable nodes to perform HTS processing when operating within an IOAM domain. The IOAM HTS Option-Type is an alternative IOAM behavior, similar in nature to IOAM‑Trace, IOAM‑DEX, and IOAM Proof‑of‑Transit. IANA is requested to allocate a new value from the “IOAM Option Types” registry established by . IANA is requested to allocate a new code point as listed in . IOAM Option-Type for HTS

Value	Name	Description	Reference
TBA1	IOAM Hybrid Two-Step (HTS) Option-Type	Instructs IOAM nodes to apply HTS processing to the packet within the IOAM domain	This document

HTS Registry Group and HTS Registries IANA is requested to create "Hybrid Two-Step" registry group.

HTS TLV Registry IANA is requested to create the HTS TLV Type registry in the "Hybrid Two-Step" registry group. All code points in the range 1 through 175 in this registry shall be allocated according to the "IETF Review" procedure specified in . Code points in the range 176 through 239 in this registry shall be allocated according to the "First Come First Served" procedure specified in . The remaining code points are allocated according to : HTS TLV Type Registry

Value	Description	Reference
0	Reserved	This document
1 - 175	Unassigned	This document
176 - 239	Unassigned	This document
240 - 251	Experimental	This document
252 - 254	Private Use	This document
255	Reserved	This document

HTS Sub-TLV Type Sub-registry IANA is requested to create the HTS sub-TLV Type sub-registry as part of the HTS TLV Type registry. All code points in the range 1 through 175 in this registry shall be allocated according to the "IETF Review" procedure specified in . Code points in the range 176 through 239 in this registry shall be allocated according to the "First Come First Served" procedure specified in . The remaining code points are allocated according to : HTS Sub-TLV Type Sub-registry

Value	Description	TLV Used	Reference
0	Reserved	None	This document
1	HMAC	Any	This document
2 - 175	Unassigned		This document
176 - 239	Unassigned		This document
240 - 251	Experimental		This document
252 - 254	Private Use		This document
255	Reserved	None	This document

HMAC Type Sub-registry IANA is requested to create the HMAC Type sub-registry as part of the HTS TLV Type registry. All code points in the range 1 through 127 in this registry shall be allocated according to the "IETF Review" procedure specified in . Code points in the range 128 through 239 in this registry shall be allocated according to the "First Come First Served" procedure specified in . The remaining code points are allocated according to : HMAC Type Sub-registry

Value	Description	Reference
0	Reserved	This document
1	HMAC-SHA-256 16 octets long	This document
2 - 127	Unassigned	This document
128 - 239	Unassigned	This document
240 - 249	Experimental	This document
250 - 254	Private Use	This document
255	Reserved	This document

HTS Error Codes Registry IANA is requested to create in the HTS registry group the new Error Codes registry as follows:

Registry Name: Error Codes
Assignment Policy:
0 - 247 - IETF Review
248 - 251 - Experimental Use (Reserved, not to be assigned)
252 - 255 - Private Use (Reserved, not to be assigned)

Error Codes

Value	Name	Description	Reference
0	No Error	No error detected	This document
1	Invalid HTS Shim	HTS shim validation failed	This document
2	Invalid Telemetry Data TLV	TLV malformed, inconsistent, or unsupported	This document
3	HMAC Verification Failure	HMAC TLV present but authentication failed	This document
4	Follow‑up Not Received	HTS Collection Timer expired before final Follow‑up packet	This document
5	Local Processing Error	Internal Egress processing error	This document
6 - 247		Unassigned	This document
248-251		Reserved for Experimental Use	This document
252-255		Reserved for Private Use	This document

Security Considerations HTS introduces new packet types (Trigger and Follow‑up) and new metadata (HTS shim, Telemetry Data TLVs, and optional HTS HMAC TLV). This section describes the security considerations associated with processing these packets and metadata. HTS does not modify the security properties of the underlying transport encapsulation.

Secuiring the Integrity of Characteristic Information Characteristic Information determines the forwarding behavior of HTS Trigger and Follow‑up packets. If Characteristic Information is modified, mis‑routed, or mis‑injected, HTS packets may:

traverse unintended paths,
fail to correlate at the collector,
produce incomplete or misleading telemetry.

HTS nodes MUST validate Characteristic Information as described in . Packets whose Characteristic Information fails validation MUST be discarded. Operators SHOULD protect Characteristic Information using authentication or integrity mechanisms provided by the underlying transport encapsulation (e.g., IPsec AH/ESP, MACsec, NSH MD‑Type 1 authentication metadata).

Integrity of HTS Shim and Telemetry Data HTS Follow‑up packets carry telemetry data contributed by intermediate nodes. An attacker who modifies telemetry data or HTS shim fields could:

falsify performance measurements,
hide congestion or failures,
inject misleading operational information,
disrupt correlation at the collector.

Operators SHOULD enable integrity protection when telemetry is used for security‑sensitive applications. HTS defines an optional HTS HMAC TLV () that provides integrity protection for HTS shim fields and Telemetry Data TLVs. When present, HTS nodes MUST validate the HMAC TLV before processing the packet. Packets whose HMAC validation fails MUST be discarded.

Confidentiality of Telemetry Data HTS telemetry may include information about:

node performance,
queue depth,
buffer occupancy,
interface load,
processing delays.

This information may be sensitive in multi‑tenant or shared environments. Operators SHOULD protect telemetry export channels (e.g., IPFIX, gRPC, NETCONF/RESTCONF) using authentication and encryption mechanisms appropriate to the deployment environment. HTS does not define confidentiality mechanisms for telemetry data carried in Follow‑up packets. Operators SHOULD rely on transport‑layer confidentiality mechanisms (e.g., IPsec ESP, MACsec, DTLS) when confidentiality is required.

Replay Protection Replay of HTS Follow‑up packets could cause:

duplicate telemetry records,
misleading performance measurements,
incorrect correlation at collectors.

HTS shim Sequence Numbers provide basic replay detection. Egress HTS nodes and Collectors SHOULD discard Follow‑up packets whose Sequence Numbers are stale or inconsistent with previously received packets. When the HTS HMAC TLV is used, replayed packets will fail HMAC validation if the HMAC includes a nonce or timestamp.

Multicast Security Considerations In multicast deployments:

Trigger packets fan out across multiple branches,
Follow‑up packets are unicast back to the collector,
each branch produces its own Follow‑up packet.

Attackers could attempt to inject false Follow‑up packets from unauthorized multicast receivers. Egress HTS nodes MUST validate Characteristic Information and HTS shim fields and MUST validate the HTS HMAC TLV when present before processing the Follow-up packets. Operators SHOULD ensure that multicast receivers are authenticated and authorized to participate in HTS measurement.

Denial‑of‑Service Considerations HTS introduces additional packet processing at ingress, intermediate, and egress nodes. An attacker could attempt to overload HTS nodes by:

sending excessive Trigger packets,
injecting malformed Follow‑up packets,
forcing repeated timer expirations,
causing excessive Full‑flag events.

HTS ingress nodes MUST provide a mechanism to rate‑limit HTS Trigger packets and SHOULD apply local policies to prevent excessive resource consumption. Collectors SHOULD detect abnormal HTS packet rates and alert operators.

Acknowledgments Authors express their gratitude and appreciation to Joel Halpern for the most helpful and insightful discussion on the applicability of HTS in a Service Function Chaining domain. Also, the authors thank Bjørn Ivar Teigen for the discussion about ensuring proper correlation between generated telemetry information and an HTS Follow-up packet. And a special thank you to Xiao Min for thorough review and thoughtful suggestions that helped in improving the document.