Internet-Draft IPv6 HBH and DO Forwarding In Routers March 2024
Ouellette Expires 5 September 2024 [Page]
Workgroup:
IPv6 Operations
Internet-Draft:
draft-ouellette-v6ops-eh-router-forwarding-00
Published:
Intended Status:
Informational
Expires:
Author:
K. Ouellette
UNH-IOL

IPv6 Hop-by-hop and Destination Options Forwarding In Routers

Abstract

It has become accepted that packets containing IPv6 Extension Headers, especially Hop-by-hop Options, are frequently dropped on the Internet. However, the question still remains as to why they get dropped and what type of devices may be dropping them. This document describes research conducted to isolate routers in a simple topology, with minimal configuration, and shows that router implementations alone are likely not the cause of packets containing IPv6 Extension Headers being dropped on the Internet.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 5 September 2024.

Table of Contents

1. Introduction

It has become well known that packets containing IPv6 [RFC8200] Extension Headers (EHs), especially Hop-by-Hop Options, are often dropped when traversing the Internet. However, the question still remains as to what are the primary contributors to these packet drops and under what circumstances are the packets dropped. Some of the potential culprits include ACLs, firewalls, routers, load balancers, and more. It also is not known whether these packets are dropped intentionally (e.g., ACLs) or unintentionally (e.g., firmware bugs).

This document describes the research conducted and results of isolating routers to better understand what role they may play, if any, in the dropping of EHs. Specifically, this research is aiming to understand whether or not router implementations may be the cause of dropped packets containing EHs.

2. Layer 3 Topology

The layer 3 topology used for these experiments can be seen below in Figure 1. Layer 3 is specifically noted because layer 2 switches exist in the topology as well as some components being virtualized, however, this is not expected to have an effect on the traversal of packets containing EHs.

+--------+     +--------+     +--------+
| Client |-----| Router |-----| Server |
+--------+     +--------+     +--------+
Figure 1: Simple Two Network Topology

Both client and server are virtual machines running Ubuntu 22.04 and are configured to inject EHs into all egress packets. Additionally, the server has an out-of-the-box installation of an apache2 web server. Details of the routers used can be found below in Section 3.

3. Routers Under Test (RUTs)

Six routers in total were tested and are comprised of three major router manufacturers. Of the six routers, five are physical and the sixth is virtual. Due to confidentiality requirements, the specific manufacturers and models of the routers tested cannot be disclosed, therefore, the routers will be referred to with a numerical value from here on.

3.1. Configuration

For these experiments, all RUTs have very minimal configurations. No routing protocols are enabled and no ACLs are configured. The routers are configured to simply forward IPv6 traffic between the two interfaces.

4. Experiments

For all experiments, both the client and server are configured to inject varying EHs into all egress packets. A single HTTP request is then made from the client to the server and it was observed that the client received an HTTP reply containing the expected EHs. This indicates that the RUT properly forwarded all packets associated with both the HTTP request and reply that contained EHs without dropping the packets or stripping the EHs from them [I-D.herbert-eh-inflight-removal]. To execute the HTTP request, the curl utility was used.

4.1. Performance and Diagnostics Metrics (PDM) Destination Option

[RFC8250] defines the PDM destination option (DO), an option used for collecting performance metrics such as round-trip delay and server delay. For this experiment, a PDM implementation leveraging Extended Berkley Packet Filter (eBPF) was used [I-D.elkins-v6ops-bpf-pdm-ebpf].

The PDM DO was chosen to test with as it is a proposed standard and access to an existing implementation was provided for the use of this research.

4.2. Variable Length Hop-by-hop and Destination Options

The second type of experiment run was testing how the RUTs processed both HBH and DO of varying header sizes. For both HBH and DO, sizes of 8, 16, 32, 64, 128, 256, and 512 bytes were tested. In addition to this, tests with an EH chain of both HBH and DO for all enumerations of the 8, 16, 32, 64, 126, and 256 header sizes were also run. Overall, there were 48 unique scenarios tested.

To achieve this, the IPv6 Extension Headers Injection with eBPF project was used which allows for injecting many different types of extension headers with control over the size of each header.

5. Results and Findings

The following section reports on the findings of these experiments. Tables in this section will denote whether the router properly forwarded packets containing the EH(s) or not. Scenarios in which the router forwards the packets will be denoted with a check mark (✓) and scenarios where the router does not forward the packets will be denoted with an X (X).

5.1. Performance and Diagnostics Metrics (PDM) Destination Option

As can be seen in Table 1, all six of the routers tested properly forwarded packets containing the PDM DO.

Table 1: PDM Experiment Results
Router 1 Router 2 Router 3 Router 4 Router 5 Router 6

5.2. Variable Length Hop-by-hop and Destination Options

5.2.1. Single Extension Header

Table 2 shows the results of a single variable sized HBH or DO EH. Seen below, all but one router forwards the EHs in every scenario.

Table 2: Single EH Experiment Results
Scenario Router 1 Router 2 Router 3 Router 4 Router 5 Router 6
HBH 8
HBH 16
HBH 32
HBH 64
HBH 128
HBH 256 X
HBH 512 ? ? ? ? ?
DO 8
DO 16
DO 32
DO 64
DO 128
DO 256 X
DO 512 X

There are two observations to note regarding these results. The first is that five of the six routers have a question mark (?) as their result for the 512-byte HBH scenario. This is because this scenario was unable to be tested for these five routers. The reasoning for this can be found in Section 5.3. The other observation to make is that out of the six routers, Router 5 was the only router unable to forward packets for each scenario. Based on these results, it suggests that Router 5 has a limitation on the maximum size of an EH in a packet. If a packet contains an EH larger than this ceiling, it is dropped. In fact, further testing has shown that a 176-byte HBH or DO is the largest Router 5 is able to properly forward. When increasing to 184-bytes, the initial TCP SYN packet is dropped and therefore the TCP connection is not established and the HTTP request cannot be made.

5.2.2. Multiple Extension Headers

The following section includes a matrix for each RUT indicating whether or not it forwarded the HTTP traffic when the packets include both a HBH and DO of varying lengths.

Table 3: Router 1 HBH and DO Results
DO HBH 8 16 32 64 128 256
8  
16  
32  
64  
128  
256  
Table 4: Router 2 HBH and DO Results
DO HBH 8 16 32 64 128 256
8  
16  
32  
64  
128  
256  
Table 5: Router 3 HBH and DO Results
DO HBH 8 16 32 64 128 256
8  
16  
32  
64  
128  
256  
Table 6: Router 4 HBH and DO Results
DO HBH 8 16 32 64 128 256
8  
16  
32  
64  
128  
256  
Table 7: Router 5 HBH and DO Results
DO HBH 8 16 32 64 128 256
8   X
16   X
32   X
64   X X
128   X X X
256   X X X X X X
Table 8: Router 6 HBH and DO Results
DO HBH 8 16 32 64 128 256
8  
16  
32  
64  
128  
256  

Of the six routers, Router 5 (Table 7) was the only router that did not properly forward packets for all scenarios. As was seen in Section 5.2.1, Router 5 did not forward packets for the 256-byte and 512-byte scenarios which suggests that Router 5 only processes packets containing EHs up to a certain length. However, in conjunction with these results, it seems as if Router 5 may have limitations based on the total EH chain length rather than individual EHs. It does however properly process smaller EH configurations.

5.3. Notable Findings

In these results, there are two findings to note. The first is that as seen in Section 5.2.1 and Section 5.2.2, a router was discovered (i.e., Router 5) that dropped packets in certain scenarios. However, the exact cause of this is still unknown as additional testing not described in this document has shown that whether or not packets are dropped is more nuanced than simply just being based on the length of the EH chain.

The second notable finding is that as seen in Table 2, the 512-byte HBH scenario was unable to be performed for five out of the six routers. The exact cause is still being investigated, however, packets specifically with a 512-byte HBH option were dropped by the layer 2 switching infrastructure. Notably, both 504-byte and 520-byte HBH options traversed the infrastructure without a problem, and only 512-byte HBH options were affected. Although not recorded in the tables above, both 504-byte and 520-byte HBH scenarios were tested on Routers 1, 2, 3, 4, and 6 and they all forwarded the traffic as expected. Because of this, it is believed this is due to a firmware bug in one of the switches and was an unexpected finding. Router 3 was the only router not affected by this due to it having a different layer 2 path and not traversing the problematic switch.

6. Conclusion

Even though this research was conducted on a relatively small sample size of manufacturers and routers, the results indicate that router implementations in general are likely not the culprits for dropping packets containing HBH or DO EHs. One router did face limitations when processing larger EHs, however, EH chains of that size may not be practical when used in a real-world network and therefore may not be a concern. The most surprising finding was that a layer 2 switch was the cause of dropped packets containing 512-byte HBH options.

7. Future Work

7.1. Determine Causes of Failed Scenarios

Currently it is not known what the limitation in Router 5 is caused by and it is also not known why a switch in the layer 2 infrastructure is dropping packets specifically with 512-byte HBH options. Next steps will include trying to determine why these behaviors are occurring and making the manufacturers aware of these limitations with the hopes that they address them if possible.

7.2. Additional Routers

A logical next step would be to test more routers, especially across a wider breadth of manufacturers. Additionally, more testing could be performed with varying connectors and link speeds.

7.3. Additional Extension Headers and Chains

This research investigated how routers handle varying sizes of single HBH and DO EHs as well as chains of the two. However, more tests could be run with other standalone EHs as well as in various chain configurations.

7.4. Testing Under Load

As was mentioned in Section 4, these experiments were run with a single HTTP request and therefore were not run under load which may be the case for routers on the open internet. It is possible that a router's EH processing varies based on whether it is under load or not, especially in the context of processing packets on the fast vs slow path. Because of this, running these same experiments at higher throughputs may be interesting to explore.

8. Security Considerations

This document has no security considerations.

9. Privacy Considerations

This document has no privacy considerations.

10. IANA Considerations

This document has no IANA actions.

11. References

11.1. Normative References

[RFC8200]
Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, , <https://www.rfc-editor.org/rfc/rfc8200>.
[RFC8250]
Elkins, N., Hamilton, R., and M. Ackermann, "IPv6 Performance and Diagnostic Metrics (PDM) Destination Option", RFC 8250, DOI 10.17487/RFC8250, , <https://www.rfc-editor.org/rfc/rfc8250>.

11.2. Informative References

[I-D.elkins-v6ops-bpf-pdm-ebpf]
Elkins, N., Sharma, C., Umesh, A., V, B., and M. P. Tahiliani, "Implementation and Performance Evaluation of PDM using eBPF", Work in Progress, Internet-Draft, draft-elkins-v6ops-bpf-pdm-ebpf-00, , <https://datatracker.ietf.org/doc/html/draft-elkins-v6ops-bpf-pdm-ebpf-00>.
[I-D.herbert-eh-inflight-removal]
Herbert, T., "Infight Removal of IPv6 Hop-by-Hop and Routing Headers", Work in Progress, Internet-Draft, draft-herbert-eh-inflight-removal-04, , <https://datatracker.ietf.org/doc/html/draft-herbert-eh-inflight-removal-04>.

Acknowledgments

The author would like to acknowledge and thank the following individuals. Nalini Elkins and Michael Ackermann for their guidance and support throughout this research. Balajinaidu V, Amogh Umesh, and Chinmaya Sharma for developing and providing access to their PDM DO implementation. Justin Iurman for developing and providing access to their IPv6 Extension Headers Injection with eBPF project. Michayla Newcombe and Ben Patton for their review and feedback of this document.

Author's Address

Kyle Ouellette
University of New Hampshire InterOperability Laboratory
United States of America