306-ipv6-happy-eyeballs.txt

Filename: 306-ipv6-happy-eyeballs.txt
Title: A Tor Implementation of IPv6 Happy Eyeballs
Author: Neel Chauhan
Created: 25-Jun-2019
Supercedes: 299
Status: Open
Ticket: https://trac.torproject.org/projects/tor/ticket/29801

1. Introduction

   As IPv4 address space becomes scarce, ISPs and organizations will deploy
   IPv6 in their networks. Right now, Tor clients connect to entry nodes using
   IPv4 connectivity by default.

   When networks first transition to IPv6, both IPv4 and IPv6 will be enabled
   on most networks in a so-called "dual-stack" configuration. This is to not
   break existing IPv4-only applications while enabling IPv6 connectivity.
   However, IPv6 connectivity may be unreliable and clients should be able
   to connect to the entry node using the most reliable technology, whether
   IPv4 or IPv6.

   In ticket #27490, we introduced the option ClientAutoIPv6ORPort which
   lets a client randomly choose between IPv4 or IPv6. However, this
   random decision does not take into account unreliable connectivity
   or falling back to the alternate IP version should one be unreliable
   or unavailable.

   One way to select between IPv4 and IPv6 on a dual-stack network is a
   so-called "Happy Eyeballs" algorithm as per RFC 8305. In one, a client
   attempts the preferred IP family, whether IPv4 or IPv6. Should it work,
   the client sticks with the preferred IP family. Otherwise, the client
   attempts the alternate version. This means if a dual-stack client has
   both IPv4 and IPv6, and IPv6 is unreliable, preferred or not, the
   client uses IPv4, and vice versa. However, if IPv4 and IPv6 are both
   equally reliable, and IPv6 is preferred, we use IPv6.

   In Proposal 299, we have attempted a IP fallback mechanism using failure
   counters and preferring IPv4 and IPv6 based on the state of the counters.
   However, Prop299 was not standard Happy Eyeballs and an alternative,
   standards-compliant proposal was requested in [P299-TRAC] to avoid issues
   from complexity caused by randomness.

   This proposal describes a Tor implementation of Happy Eyeballs and is
   intended as a successor to Proposal 299.

2. Address/Relay Selection

   This section describes the necessary changes for address selection to
   implement Prop306.

2.1. Address Handling Changes

   To be able to handle Happy Eyeballs in Tor, we will need to modify the
   data structures used for connections to entry nodes, namely the extend info
   structure.

   Entry nodes are usually guards, but some clients don't use guards:

     * Bootstrapping clients can connect to fallback directory mirrors or
       authorities

     * v3 single onion services can use IPv4 or IPv6 addresses to connect
       to introduction and rendezvous points, and

     * Clients can be configured to disable entry guards

   Bridges are out of scope for this proposal, because Tor does not support
   multiple IP addresses in a single bridge line.

   The extend info structure should contain both an IPv4 and an IPv6 address.
   This will allow us to try IPv4 and the IPv6 addresses should both be
   available on a relay and the client is dual-stack.

   When processing:
     * relay descriptors,
     * hard-coded authority and fallback directory lists,
     * onion service descriptors, or
     * onion service introduce cells,
   and filling in the extend info data structure, we need to fill in both the
   IPv4 and IPv6 address if both are available. If only one family is
   available for a relay (IPv4 or IPv6), we should leave the other family null.

2.2 Bootstrap Changes

   Tor's hard-coded authority and fallback directory mirror lists contain
   some entries with IPv6 ORPorts. As of January 2020, 56% of authorities and
   47% of fallback directories have IPv6.

   During bootstrapping, we should have an option for the maximum number of
   IPv4-only nodes, before the next node must have an IPv6 ORPort. The
   parameter is as follows:

     * MaxNumIPv4BootstrapAttempts NUM

   During bootstrap, the minimum fraction of nodes with IPv6 ORPorts will be
   1/(1 + MaxNumIPv4BootstrapAttempts). And the average fraction will be
   larger than both minimum fraction, and the actual proportion of IPv6
   ORPorts in the fallback directory list. (Clients mainly use fallback
   directories for bootstrapping.)

   Since this option is used during bootstrapping, it can not have a
   corresponding consensus parameter.

   The default value for MaxNumIPv4BootstrapAttempts should be 2. This
   means that every third bootstrap node must have an IPv6 ORPort. And on
   average, just over half of bootstrap nodes chosen by clients will have an
   IPv6 ORPort. This change won't have much impact on load-balancing, because
   almost half the fallback directory mirrors have IPv6 ORPorts.

   The minimum value of MaxNumIPv4BootstrapAttempts is 0. (Every bootstrap
   node must have an IPv6 ORPort. This setting is equivalent to
   ClientPreferIPv6ORPort 1.)

   The maximum value of MaxNumIPv4BootstrapAttempts should be 100. (Since
   most clients only make a few bootstrap connections, bootstrap nodes will
   be chosen at random, regardless of their IPv6 ORPorts.)

2.3. Guard Selection Changes

   When we select guard candidates, we should have an option for the number of
   primary IPv6 entry guards. The parameter is as follows:

     * NumIPv6Guards NUM

   If UseEntryGuards is set to 1, we will select exactly this number of IPv6
   relays for our primary guard list, which is the set of relays we strongly
   prefer when connecting to the Tor network. (This number should also apply
   to all of Tor's other guard lists, scaled up based on the relative size of
   the list.)

   If NUM is -1, we try to learn the number from the NumIPv6Guards
   consensus parameter. If the consensus parameter isn't set, we should
   default to 1.

   The default value for NumIPv6Guards should be -1. (Use the consensus
   parameter, or the underlying default value of 1.)

   As of September 2019, approximately 20% of Tor's guards supported IPv6,
   by consensus weight. (Excluding exits that are also guards, because
   clients avoid choosing exits in their guard lists.)

   If all Tor clients implement NumIPv6Guards, then these 20% of guards will
   handle approximately 33% of Tor's traffic. (Because the default value of
   NumPrimaryGuards is 3.) This may have a significant impact on Tor's
   load-balancing. Therefore, we should deploy this feature gradually, and try
   to increase the number of relays that support IPv6 to at least 33%.

   To minimise the impact on load-balancing, IPv6 support should only be
   required for exactly NumIPv6Guards during guard list selection. All other
   guards should be IPv4-only guards. Once approximately 50% of guards support
   IPv6, NumIPv6Guards can become a minimum requirement, rather than an exact
   requirement.

   The minimum configurable value of NumIPv6Guards is -1. (Use the consensus
   parameter, or the underlying default.)

   The minimum resulting value of NumIPv6Guards is 0. (Guards will be chosen
   at random, regardless of their IPv6 ORPorts.)

   The maximum value of NumIPv6Guards should be the configured value of
   NumPrimaryGuards. (Every guard must have an IPv6 ORPort. This setting is
   equivalent to ClientPreferIPv6ORPort 1.)

3. Relay Connections

   If there is an existing authenticated connection, we must use it similar
   to how we used it pre-Prop306.

   If there is no existing authenticated connection for an entry node, tor
   currently attempts to connect using the first available, allowed, and
   preferred address. (Determined using the existing Client IPv4 and IPv6
   options.)

   We should also allow falling back to the alternate address. For this,
   a design will be given in Section 3.1.

3.1. TCP Connection to Preferred Address On First TCP Success

   In this design, we will connect via TCP to the first preferred address.
   On a failure or after a 250 msec delay, we attempt to connect via TCP to
   the alternate address. On a success, Tor attempts to authenticate and
   closes the other connection.

   This design is close to RFC 8305 and is similar to how Happy Eyeballs
   is implemented in a web browser.

3.2. Handling Connection Successes And Failures

   Should a connection to a entry node succeed and is authenticated via TLS,
   we can then use the connection. In this case, we should cancel all other
   connection timers and in-progress connections. Cancelling the timers is
   necessary so we don't attempt new unnecessary connections when our
   existing connection is successful, preventing denial-of-service risks.

   However, if we fail all available and allowed connections, we should tell
   the rest of Tor that the connection has failed. This is so we can attempt
   another entry node.

3.3. Connection Attempt Delays

   As mentioned in [TEOR-P306-REP], initially, clients should prefer IPv4
   by default. The Connection Attempt Delay, or delay between IPv4 and IPv6
   connections should be 250 msec. This is to avoid the overhead from tunneled
   IPv6 connections.

   The Connection Attempt Delay should not be dynamically adjusted, as it adds
   privacy risks. This value should be fixed, and could be manually adjusted
   using this torrc option or consensus parameter:

     * ConnectionAttemptDelay N [msec|second]

   The Minimum and Maximum Connection Attempt Delays should also not be
   dynamically adjusted for privacy reasons. The Minimum should be fixed at
   10 msec as per RFC 8305. But the maximum should be higher than the RFC 8305
   recommendation of 2 seconds. For Tor, we should make this timeout value 30
   seconds to match Tor's existing timeout.

   We need to make it possible for users to set the Maximum Connection Attempt
   Delay value higher for slower and higher-latency networks such as dial-up
   and satellite.

4. Option Changes

   As we enable IPv6-enabled clients to connect out of the box, we should
   adjust the default options to enable IPv6 while not breaking IPv4-only
   clients.

   The new default options should be:

    * ClientUseIPv4 1 (to enable IPv4)

    * ClientUseIPv6 1 (to enable IPv6)

    * ClientPreferIPv6ORPort 0 (for load-balancing reasons so we don't
      overload IPv6-only guards)

    * ConnectionAttemptDelay 250 msec (the recommended delay between IPv4
      and IPv6, as per RFC 8305)

   One thing to note is that clients should be able to connect with the above
   options on IPv4-only, dual-stack, and IPv6-only networks, and they should
   also work if ClientPreferIPv6ORPort is 1. But we shouldn't expect
   IPv4 or IPv6 to work if ClientUseIPv4 or ClientUseIPv6 is set to 0.

   When the majority of clients and relay are IPv6-capable, we could set the
   default value of ClientPreferIPv6ORPort to 1, in order to take advantage
   of IPv6. We could add a ClientPreferIPv6ORPort consensus parameter, so we
   can make this change network-wide.

5. Relay Statistics

   Entry nodes could measure the following statistics for both IPv4 and IPv6:

     * Number of successful connections

     * Number of extra Prop306 connections (unsuccessful or cancelled)
       * Client closes the connection before completing TLS
       * Client closes the connection before sending any circuit or data cells

     * Number of client and relay connections
       * We can distinguish between authenticated (relay, authority
         reachability) and unauthenticated (client, bridge) connections

   Should we implement Section 5:

     * We can send this information to the directory authorities using relay
       extra-info descriptors

     * We should consider the privacy implications of these statistics, and
       how much noise we need to add to them

     * We can include these statistics in the Heartbeat logs

6. Initial Feasibility Testing

   We should test this proposal with the following scenarios:

    * Different combinations of values for the options ClientUseIPv4,
      ClientUseIPv6, and ClientPreferIPv6ORPort on IPv4-only, IPv6-only,
      and dual-stack connections

    * Dual-stack connections of different technologies, including
      high-bandwidth and low-latency (e.g. FTTH), moderate-bandwidth and
      moderate-latency (e.g. DSL, LTE), and high-latency and low-bandwidth
      (e.g. satellite, dial-up) to see if Prop306 is reliable and feasible

7. Minimum Viable Prop306 Product

   The mimumum viable product for Prop306 must include the following:

    * The address handling, bootstrap, and entry guard changes described in
      Section 2. (Single Onion Services are optional, Bridge Clients are out
      of scope. The consensus parameter and torrc options are optional.)

    * The alternative address retry algorithm in Section 3.1.

    * The Connection Success/Failure mechanism in Section 3.2.

    * The Connection Delay mechanism in Section 3.3. (The
      ConnectionAttemptDelay torrc option and consensus parameter are
      optional.)

    * A default setup capable of both IPv4 and IPv6 connections with the
      options described in Section 4. (The ClientPreferIPv6ORPort consensus
      parameter is optional.)

8. Optional Features

   Some features which are optional include:

    * Single Onion services: extend info address changes for onion service
      descriptors and introduce cells. (Section 2.1.)

    * Bridge clients are out of scope: they would require bridge line format
      changes, internal bridge data structure changes, and extend info address
      changes. (Section 2.1.)

    * MaxNumIPv4BootstrapAttempts torrc option. We may need this option if
      the proposed default doesn't work for some clients. (Section 2.2.)

    * NumIPv6Guards torrc option and consensus parameter. We may need this
      option if the proposed default doesn't work for some clients.
      (Section 2.3.)

    * ConnectionAttemptDelay torrc option and consensus parameter. We will need
      this option if the Connection Attempt Delay needs to be manually
      adjusted, for instance, if clients often fail IPv6 connections.
      (Section 3.3.)

    * ClientPreferIPv6ORPort consensus parameter. (Section 4.)

    * IPv4, IPv6, client, relay, and extra Prop306 connection statistics.
      While optional, these statistics may be useful for debugging and
      reliability testing, and metrics on IPv4 vs IPv6. (Section 5.)

9. Acknowledgments

   Thank you so much to teor for your discussion on this happy eyeballs
   proposal. I wouldn't have been able to do this has it not been for
   your help.

10. Refrences

   [P299-TRAC]: https://trac.torproject.org/projects/tor/ticket/29801

   [TEOR-P306-REP]: https://lists.torproject.org/pipermail/tor-dev/2019-July/013919.html