264-subprotocol-versions.txt

Filename: 264-subprotocol-versions.txt
Title: Putting version numbers on the Tor subprotocols
Author: Nick Mathewson
Created: 6 Jan 2016
Status: Closed
Implemented-In: 0.2.9.4-alpha


1. Introduction

   At various points in Tor's history, we've needed to migrate from one
   protocol to another.  In the past, we've mostly done this by allowing
   relays to advertise support for various features.  We've done this in
   an ad-hoc way, though. In some cases, we've done it entirely based on
   the relays' advertised Tor version.

   That's a pattern we shouldn't continue.  We'd like to support more
   live Tor relay implementations, and that means that tying "features"
   to "tor version" won't work going forwards.

   This proposal describes an alternative method that we can use to
   simplify the advertisement and discovery of features, and the
   transition from one set of features to another.

1.1. History: "Protocols" vs version-based gating.

   For ages, we've specified a "protocols" line in relay descriptors,
   with a list of supported versions for "relay" and "link" protocols.
   But we've never actually looked at it, and instead we've relied on
   tor version numbers to determine which features we could rely upon.
   We haven't kept the relay and link protocols listed there up-to-date
   either.

   Clients have used version checks for three purposes historically:
   checking relays for bugs, checking relays for features, and
   implementing bug-workarounds on their own state files.

   In this design, feature checks are now performed directly with
   subprotocol versions. We only need to keep using Tor versions
   specifically for bug workarounds.

2. Design: Advertising protocols.

   We revive the "Protocols" design above, in a new form.

   "proto" SP Entries NL

     Entries =
     Entries = Entry
     Entries = Entry SP Entries

     Entry = Keyword "=" Values

     Values = Value
     Values = Value "," Values

     Value = Int
     Value = Int "-" Int

     Int = NON_ZERO_DIGIT
     Int = Int DIGIT


   Each 'Entry' in the "proto" line indicates that the Tor relay
   supports one or more versions of the protocol in question.  Entries
   should be sorted by keyword.  Values should be numerically ascending
   within each entry.  (This implies that there should be no overlapping
   ranges.)  Ranges should be represented as compactly as possible. Ints
   must be no more than 2^32 - 1.

   The semantics for each keyword must be defined in a Tor
   specification.  Extension keywords are allowed if they begin with
   "x-" or "X-".  Keywords are case-sensitive.

   During voting, authorities copy these lines immediately below the "v"
   lines, using "pr" as the keyword instead of "proto".
   When a descriptor does not contain a "proto" entry, the
   authorities should reconstruct it using the approach described below
   in section A.1.  They are included in the consensus using the same
   rules as currently used for "v" lines, if a sufficiently late
   consensus method is in use.

2.1. An alternative: Moving 'v' lines into microdescriptors.

   [[[[[
   Here's an alternative: we could put "v" and "proto" lines into
   microdescriptors.

   When building microdescriptors, authorities could copy all valid
   "proto" entries verbatim if a sufficiently late consensus method is
   in use.  When a descriptor does not contain a "proto" entry, the
   authorities should reconstruct it using the approach described below
   in section A.1.

   Tor clients that want to use "v" lines should prefer those in
   microdescriptors if present, and ignore those in the consensus.

   (Existing maintained client versions can be adapted to never look at
   "v" lines at all; the only versions that they still check for are
   ones not allowed on the network.  The "v" line can be dropped
   from the consensus entirely when current clients have upgraded.)
   ]]]]]

   [I am rejecting this alternative for now, since proto lines should
   compress very well, given that the proto line is completely
   inferrable from the v line.  Removing all the v lines from the
   current consensus would save only 1.7% after gzip compression.]

3. Using "proto"/"pr" and "v" lines

   Whenever possible, clients and relays should use the list of
   advertised protocols instead of version numbers.  Version numbers
   should only be used when implementing bug-workarounds for specific
   Tor versions.

   Every new feature in tor-spec.txt, dir-spec.txt, and rend-spec.txt
   should be gated on a particular protocol version.

4. Required protocols

   The consensus may contain four lines:
      "recommended-relay-protocols",
      "required-relay-protocols",
      "recommended-client-protocols", and
      "required-client-protocols".

   Each has the same format as the "proto" line.  To vote on these
   entries, a protocol/version combination is included only if it is
   listed by a majority of the voters.

   When a relay lacks a protocol listed in recommended-relay-protocols, it
   should warn its operator that the relay is obsolete.

   When a relay lacks a protocol listed in required-relay-protocols, it
   must not attempt to join the network.

   When a client lacks a protocol listed in recommended-client-protocols,
   it should warn the user that the client is obsolete.

   When a client lacks a protocol listed in required-client-protocols, it
   must not connect to the network.  This implements a "safe
   forward shutdown" mechanism for zombie clients.

   If a client or relay has a cached consensus telling it that a given
   protocol is required, and it does not implement that protocol, it
   SHOULD NOT try to fetch a newer consensus.

   [[XXX I propose we remove this idea:

    The above features should be backported to 0.2.4 and later, or all the
    versions we expect to continue supporting.]]

   These lines should be voted on.  A majority of votes is sufficient to
   make a protocol un-supported.

   and should require a supermajority of
   authorities (2/3) to make a protocol required.  The required protocols
   should not be torrc-configurable, but rather should be hardwired in the
   Tor code.


5. Current protocols

   (See "6. Maintaining the protocol list" below for information about
   how I got these, and why version 0.2.4.19 comes up so often.)

5.1. "Link"

   The "link" protocols are those used by clients and relays to initiate
   and receive OR connections and to handle cells on OR connections.
   The "link" protocol versions correspond 1:1 to those versions.

   Two Tor instances can make a connection to each other only if they
   have at least one link protocol in common.

   The current "link" versions are: "1" through "4"; see tor-spec.txt
   for more information.  All current Tor versions support "1-3";
   version from 0.2.4.11-alpha and on support "1-4".  Eventually we
   will drop "1" and "2".

5.2. "LinkAuth"

   LinkAuth protocols correspond to varieties of Authenticate cells used
   for the v3+ link protocools.

   The current version is "1".

   "2" is unused, and reserved by proposal 244.

   "3" is the ed25519 link handshake of proposal 220.

5.3. "Relay"

   The "relay" protocols are those used to handle CREATE cells, and
   those that handle the various RELAY cell types received after a
   CREATE cell.  (Except, relay cells used to manage introduction and
   rendezvous points are managed with the "HSIntro" and "HSRend" protocols
   respectively.)

      "1" -- supports the TAP key exchange, with all features in Tor
         0.2.3.  Support for CREATE and CREATED and CREATE_FAST and
         CREATED_FAST and EXTEND and EXTENDED.

      "2" -- supports the ntor key exchange, and all features in Tor
         0.2.4.19.  Includes support for CREATE2 and CREATED2 and
         EXTEND2 and EXTENDED2.

5.4. "HSIntro"

   The "HSIntro" protocol handles introduction points.

      "3" -- supports authentication as of proposal 121 in Tor
             0.2.1.6-alpha.

5.5. "HSRend"

   The "HSRend" protocol handles rendezvous points.

      "1" -- supports all features in Tor 0.0.6.

      "2" -- supports RENDEZVOUS2 cells of arbitrary length as long as they
             have 20 bytes of cookie in Tor 0.2.9.1-alpha.

5.6. "HSDir"

   The HSDir protocols are the set of hidden service document types
   that can be uploaded to, understood by, and downloaded from a tor
   relay, and the set of URLs available to fetch them.

      "1" -- supports all features in Tor 0.2.0.10-alpha.

5.7. "DirCache"

   The "DirCache" protocols are the set of documents available for
   download from a directory cache via BEGIN_DIR, and the set of URLs
   available to fetch them.  (This excludes URLs for hidden service
   objects.)

      "1" -- supports all features in Tor 0.2.4.19.

5.8. "Desc"

   Describes features present or absent in descriptors.

   Most features in descriptors don't require a "Desc" update -- only
   those that need to someday be required.  For example, someday clients
   will need to understand ed25519 identities.

      "1" -- supports all features in Tor 0.2.4.19.

      "2" -- cross-signing with onion-keys, signing with ed25519
             identities.

5.9. "Microdesc"

   Describes features present or absent in microdescriptors.

   Most features in descriptors don't require a "MicroDesc" update --
   only those that need to someday be required.
   These correspond more or less with consensus methods.

      "1" -- consensus methods 9 through 20.

      "2" -- consensus method 21 (adds ed25519 keys to microdescs).

5.10. "Cons"

   Describes features present or absent in consensus documents.

   Most features in consensus documents don't require a "Cons" update --
   only those that need to someday be required.

   These correspond more or less with consensus methods.

      "1" -- consensus methods 9 through 20.

      "2" -- consensus method 21 (adds ed25519 keys to microdescs).


6. Maintaining the protocol lists

   What makes a good fit for a "protocol" type?  Generally, it's a set
   of communications functionality that tends to upgrade in tandem, and
   in isolation from other parts of the Tor protocols.  It tends to be
   functionality where it doesn't make sense to implement only part of
   it -- though omitting the whole thing might make sense.

   (Looking through our suite of protocols, you might make a case for
   splitting DirCache into sub-protocols.)

   We shouldn't add protocols for features where others can remain
   oblivious to their presence or absence.  For example, if some
   directory caches start supporting a new header, and clients can
   safely send that header without knowing whether the directory cache
   will understand it, then a new protocol version is not required.

   Because all relays currently on the network are 0.2.4.19 or later, we
   can require 0.2.4.19, and use 0.2.4.19 as the minimal version so we
   we don't need to do code archaeology to determine how many
   no-longer-relevant versions of each protocol once existed.

   Adding new protocol types is pretty cheap, given compression.

A.1.  Inferring missing proto lines.

   The directory authorities no longer allow versions of Tor before
   0.2.4.18-rc.  But right now, there is no version of Tor in the
   consensus before 0.2.4.19.  Therefore, we should disallow versions of
   Tor earlier than 0.2.4.19, so that we can have the protocol list for
   all current Tor versions include:

     Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4
     LinkAuth=1 Microdesc=1-2 Relay=1-2

   For Desc, Tor versions before 0.2.7.stable should be taken to have
   Desc=1 and versions 0.2.7.stable or later should have Desc=1-2.

   For Microdesc and Cons, Tor versions before 0.2.7.stable should be
   taken to support version 1; 0.2.7.stable and later should have
   1-2.

A.2. Initial required protocols

   For clients we will Recommend and Require these.

        Cons=1-2 Desc=1-2 DirCache=1 HSDir=2 HSIntro=3 HSRend=1 Link=4
        LinkAuth=1 Microdesc=1-2 Relay=2

   For relays we will Require:

        Cons=1 Desc=1 DirCache=1 HSDir=2 HSIntro=3 HSRend=1 Link=3-4
        LinkAuth=1 Microdesc=1 Relay=1-2

   For relays, we will additionally Recommend all protocols which we
   recommend for clients.

A.3. Example integration with other open proposals

   In this appendix, I try to show that this proposal is viable by
   showing how it can integrate with other open proposals to avoid
   version-gating.  I'm looking at open/draft/accepted proposals only.

    140  Provide diffs between consensuses

         This proposal doesn't affect interoperability, though we could
         add a DirCache protocol version for it if we think we might
         want to require it someday.

    164  Reporting the status of server votes

         Interoperability not affected; no new protocol.

    165  Easy migration for voting authority sets

         Authority-only; no new protocol.

    168  Reduce default circuit window

         Interoperability slightly affected; could be a new Relay
         protocol.

    172  GETINFO controller option for circuit information
    173  GETINFO Option Expansion

         Client/Relay interop not affected; no new protocol.

    177  Abstaining from votes on individual flags

         Authority-only; no new protocol.

    182  Credit Bucket

         No new protocol.

    188  Bridge Guards and other anti-enumeration defenses

         No new protocol.

    189  AUTHORIZE and AUTHORIZED cells

         This would be a new protocol, either a Link protocol or a new
         LinkAuth protocol.

    191  Bridge Detection Resistance against MITM-capable Adversaries

         No new protocol.

    192  Automatically retrieve and store information about bridges

         No new protocol.

    195  TLS certificate normalization for Tor 0.2.4.x

         Interop not affected; no new protocol.

    201  Make bridges report statistics on daily v3 network status
         requests

         No new protocol.

    202  Two improved relay encryption protocols for Tor cells

         This would be a new Relay protocol.

    203  Avoiding censorship by impersonating an HTTPS server

         Bridge/PT only; no new protocol.

    209  Tuning the Parameters for the Path Bias Defense

         Client behavior only; no new protocol.

    210  Faster Headless Consensus Bootstrapping

         Client behavior only; no new protocol.

    212  Increase Acceptable Consensus Age

         Possibly add a new DirCache protocol version to describe the
         "will hold older descriptors" property.

    219  Support for full DNS and DNSSEC resolution in Tor

         New relay protocol, or new protocol class (DNS=2?)

    220  Migrate server identity keys to Ed25519

         Once link authentication is supported, that's a new LinkAuth
         protocol version.

         No new protocol version is required for circuit extension,
         since it's a backward-compatible change.

    224  Next-Generation Hidden Services in Tor

         Adds new HSDir and HSIntro and HSRend protocols.

    226 "Scalability and Stability Improvements to BridgeDB: Switching
         to a Distributed Database System and RDBMS"

         No new protocol.

    229  Further SOCKS5 extensions

         Client-only; no new protocol.

    233  Making Tor2Web mode faster

         No new protocol.

    234  Adding remittance field to directory specification

         Could be a new protocol; or not.

    237  All relays are directory servers

         No new protocol.

    239  Consensus Hash Chaining

         No new protocol.

    242  Better performance and usability for the MyFamily option

         New Desc protocol.

    244  Use RFC5705 Key Exporting in our AUTHENTICATE calls

         Part of prop220.  Also adds another LinkAuth protocol version.

    245  Deprecating and removing the TAP circuit extension protocol

         Removes Linkauth protocol 1.

         Removes a Desc protocol.

    246  Merging Hidden Service Directories and Introduction Points

         Possibly adds a new HSIntro or HSDir protocol.

    247  Defending Against Guard Discovery Attacks using Vanguards

         No new protocol.

    248  Remove all RSA identity keys

         Adds a new Desc protocol version and a new Cons protocol
         version; eventually removes a version of each.

    249  Allow CREATE cells with >505 bytes of handshake data

         Adds a new Link protocol version for CREATE2V.

         Adds a new Relay protocol version for new EXTEND2 semantics.

    250  Random Number Generation  During Tor Voting

         No new protocol.

    251  Padding for netflow record resolution reduction

         No new protocol.

    252  Single Onion Services

         No new protocol.

    253  Out of Band Circuit HMACs

         New Relay protocol.

    254  Padding Negotiation

         New Link protocol, new Relay protocol.

    255  Controller features to allow for load-balancing hidden services

         No new protocol.

    256  Key revocation for relays and authorities

         New Desc protocol.

    257  Refactoring authorities and taking parts offline

         No new protocol.

    258  Denial-of-service resistance for directory authorities

         No new protocol.

    259  New Guard Selection Behaviour

         No new protocol

    260  Rendezvous Single Onion Services

         No new protocol

    261  AEZ for relay cryptography

         New Relay protocol version.

    262  Re-keying live circuits with new cryptographic material

         New Relay protocol version

    263  Request to change key exchange protocol for handshake

         New Relay protocol version.