- tor-dev - lists.torproject.org

Proposal XXX: FlashFlow: A Secure Speed Test for Tor (Parent Proposal)
by Matt Traudt 02 Jun '20

02 Jun '20

Filename: xxx-flashflow.txt Title: FlashFlow: A Secure Speed Test for Tor (Parent Proposal) Author: Matthew Traudt, Aaron Johnson, Rob Jansen, Mike Perry Created: 23 April 2020 Status: Draft 1. Introduction FlashFlow is a new distributed bandwidth measurement system for Tor that consists of a single authority node ("coordinator") instructing one or more measurement nodes ("measurers") when and how to measure Tor relays. A measurement consists of the following steps: 1. The measurement nodes demonstrate to the target relay permission to perform measurements. 2. The measurement nodes open many TCP connections to the target relay and create a one-hop circuit to the target relay on each one. 3. For 30 seconds the measurement nodes send measurement cells to the target relay and verify that the cells echoed back match the ones sent. During this time the relay caps the amount of background traffic it transfers. Background and measurement traffic are handled separately at the relay. Measurement traffic counts towards all the standard existing relay statistics. 4. For every second during the measurement, the measurement nodes report to the authority node how much traffic was echoed back. The target relay also reports the amount of per-second background (non-measurement) traffic. 5. The authority node sums the per-second reported throughputs into 30 sums (one for each second) and calculates the median. This is the estimated capacity of the relay. FlashFlow performs a measurement of every relay according to a schedule described later in this document. Periodically it produces relay capacity estimates in the form of a v3bw file, which is suitable for direct consumption by a Tor directory authority. Alternatively an existing load balancing system such as Simple Bandwidth Scanner could be modified to use FlashFlow's v3bw file as input. It is envisioned that each directory authority that wants to use FlashFlow will run their own FlashFlow deployment consisting of a coordinator that they run and one or more measurers that they trust (e.g. because they run them themselves), similar to how each runs their own Torflow/sbws. Section 5.2 of this proposal describes long term plans involving multiple FlashFlow deployments. FlashFlow is more performant than Torflow: FlashFlow takes 5 hours to measure the entire existing Tor network from scratch (with 3 Gbit/s measurer capacity) while Torflow takes 2 days; FlashFlow measures relays it hasn't seen recently as soon as it learns about them (i.e. every new consensus) while Torflow can take a day or more; and FlashFlow accurately measures new high-capacity relays the first time and every time while Torflow takes days/weeks to assign them their full fair share of bandwidth (especially for non-exits). FlashFlow is more secure than Torflow: FlashFlow allows a relay to inflate its measured capacity by up to 1.33x (configured by a parameter) while Torflow allows weight inflation by a factor of 89x [0] or even 177x [1]. After an overview in section 2 of the planned deployment stages, section 3, 4, and 5 discuss the short, medium, and long term deployment plans in more detail. 2. Deployment Stages FlashFlow's deployment shall be broken up into three stages. In the short term we will implement a working FlashFlow measurement system. This requires code changes in little-t tor and an external FlashFlow codebase. The majority of the implementation work will be done in the short term, and the product is a complete FlashFlow measurement system. Remaining pieces (e.g. better authentication) are added later for enhanced security and network performance. In the medium term we will begin collecting data with a FlashFlow deployment. The intermediate results and v3bw files produced will be made available (semi?) publicly for study. In the long term experiments will be performed to study ways of using FF v3bw files to improve load balancing. Two examples: (1) using FF v3bw files instead of sbws's (and eventually phasing out torflow/sbws), and (2) continuing to run sbws but use FF's results as a better estimate of relay capacity than observed bandwidth. Authentication and other FlashFlow features necessary to make it completely ready for full production deployment will be worked on during this long term phase. 3. FlashFlow measurement system: Short term The core measurement mechanics will be implemented in little-t tor, but a separate codebase for the FlashFlow side of the measurement system will also be created. This section is divided into three parts: first a discussion of changes/additions that logically reside entirely within tor (essentially: relay-side modifications), second a discussion of the separate FlashFlow code that also requires some amount of tor changes (essentially: measurer-side and coordinator-side modifications), and third a security discussion. 3.1 Little-T Tor Components The primary additions/changes that entirely reside within tor on the relay side: - New torrc options/consensus parameters. - New cell commands. - Pre-measurement handshaking (with a simplified authentication scheme). - Measurement mode, during which the relay will echo traffic with measurers, set a cap on the amount of background traffic it transfers, and report the amount of transferred background traffic. 3.1.1 Parameters FlashFlow will require some consensus parameters/torrc options. Each has some default value if nothing is specified; the consensus parameter overrides this default value; the torrc option overrides both. FFMeasurementsAllowed: A global toggle on whether or not to allow measurements. Even if all other settings would allow a measurement, if this is turned off, then no measurement is allowed. Possible values: 0, 1. Default: 0 (disallowed). FFAllowedCoordinators: The list of coordinator TLS certificate fingerprints that are allowed to start measurements. Relays check their torrc when they receive a connection from a FlashFlow coordinator to see if it's on the list. If they have no list, they check the consensus parameter. If nether exist, then no FlashFlow deployment is allowed to measure this relay. Default: empty list. FFMeasurementPeriod: A relay should expect on average, to be measured by each FlashFlow deployment once each measurement period. A relay will not allow itself to be measured more than twice by a FlashFlow deployment in any time window of this length. Relays should not change this option unless they really know what they're doing. Changing it at the relay will not change how often FlashFlow will attempt to measure the relay. Possible values are in the range [1 hour, 1 month] inclusive. Default: 1 day. FFBackgroundTrafficPercent: The maximum amount of regular non-measurement traffic a relay should handle while being measured, as a percent of total traffic (measurement + non-measurement). This parameter is a trade off between having to limit background traffic and limiting how much a relay can inflate its result by handling no background traffic but reporting that it has done so. Possible values are in the range [0, 99] inclusive. Default: 25 (a maximum inflation factor of 1.33). FFMaxMeasurementDuration: The maximum amount of time, in seconds, that is allowed to pass from the moment the relay is notified that a measurement will begin soon and the end of the measurement. If this amount of time passes, the relay shall close all measurement connections and exit its measurement mode. Note this duration includes handshake time, thus it necessarily is larger than the expected actual measurement duration. Possible values are in the range [10, 120] inclusive. Default: 45. 3.1.2 New Cell Types FlashFlow will introduce a new cell command MEASURE. The payload of each MEASURE cell consists of: Measure command [1 byte] Length [2 bytes] Data [Length-3 bytes] The measure commands are: 0 -- MSM_PARAMS [forward] 1 -- MSM_PARAMS_OK [backward] 2 -- MSM_ECHO [forward and backward] 3 -- MSM_BG [backward] 4 -- MSM_ERR [forward and backward] Forward cells are sent from the measurer/coordinator to the relay. Backward cells are sent from the relay to the measurer/coordinator. MSM_PARAMS and MSM_PARAMS_OK are used during the pre-measurement stage to tell the target what to expect and for the relay to positively acknowledge the message. MSM_ECHO cells are the measurement traffic; the measurer generates them, sends them to the target, and the target echos them back. The target send a MSM_BG cell once per second to report the amount of background traffic it is handling. MSM_ERR cells are used to signal to the other party that there has been some sort of problem and that the measurement should be aborted. These measure commands are described in more detail in the next section. The only cell that sometimes undergoes cell encryption is MSM_ECHO; no other cell ever gets cell encrypted. (All cells are transmitted on a regular TLS-wrapped OR connection; that encryption still exists.) The relay "decrypts" MSM_ECHO cells before sending them back to the measurer; this mirrors the way relays decrypt/encrypt RELAY_DATA cells in order to induce realistic cryptographic CPU load. The measurer usually skips encrypting MSM_ECHO cells to reduce its own CPU load; however, to verify the relay is actually correctly decrypting all cells, the measurer will choose random outgoing cells, encrypt them, remember the ciphertext, and verify the corresponding incoming cell matches. 3.1.3 Pre-Measurement Handshaking/Starting a Measurement The coordinator connects to the target relay and sends it a MSM_PARAMS cell. If the target is unwilling to be measured at this time or if the coordinator didn't use a TLS certificate that the target trusts, it responds with an error cell and closes the connection. Otherwise it checks that the parameters of the measurement are acceptable (e.g. the version is acceptable, the duration isn't too long, etc.). If the target is happy, it sends a MSM_PARAMS_OK, otherwise it sends a MSM_ERR and closes the connection. Upon learning the IP addresses of the measurers from the coordinator in the MSM_PARAMS cell, the target whitelists their IPs in its DoS detection subsystem until the measurement ends (successfully or otherwise), at which point the whitelist is cleared. Upon receiving a MSM_PARAMS_OK from the target, the coordinator will instruct the measurers to open their TCP connections with the target. If the coordinator or any measurer receives a MSM_ERR, it reports the error to the coordinator and considers the measurement a failure. It is also a failure if any measurer is unable to open at least half of its TCP connections with the target. The payload of MSM_PARAMS cells [XXX more may need to be added]: - version [1 byte] - msm_duration [1 byte] - num_measurers [1 byte] - measurer_info [num_measurers times] - ipv4_addr [4 bytes] - num_conns [2 bytes] version dictates how this MSM_PARAMS cell shall be parsed. msm_duration is the duration, in seconds, that the actual measurement will last. num_measurers is how many measurer_info structs follow. For each measurer, the ipv4_addr it will use when connecting to the target is provided, as is num_conns, the number of TCP connections that measurer will open with the target. Future versions of FlashFlow and MSM_PARAMS will use TLS certificates instead of IP addresses. MSM_PARAMS_OK has no payload: it's just padding bytes to make the cell 514 bytes long. The payload of MSM_ECHO cells: - arbitrary bytes [max to fill up 514 byte cell] The payload of MSM_BG cells: - second [1 byte] - sent_bg_bytes [4 bytes] - recv_bg_bytes [4 bytes] second is the number of seconds since the measurement began. MSM_BG cells are sent once per second from the relay to the FlashFlow coordinator. The first cell will have this set to 1, and each subsequent cell will increment it by one. sent_bg_bytes is the number of background traffic bytes sent in the last second (since the last MSM_BG cell). recv_bg_bytes is the same but for received bytes. The payload of MSM_ERR cells: - err_code [1 byte] - err_str [possibly zero-len null-terminated string] The error code is one of: [... XXX TODO ...] 255 -- OTHER The error string is optional in all cases. It isn't present if the first byte of err_str is null, otherwise it is present. It ends at the first null byte or the end of the cell, whichever comes first. 3.1.4 Measurement Mode The relay considers the measurement to have started the moment it receives the first MSM_ECHO cell from any measurer. At this point, the relay - Starts a repeating 1s timer on which it will report the amount of background traffic to the coordinator over the coordinator's connection. - Enters "measurement mode" and limits the amount of background traffic it handles according to the torrc option/consensus parameter. The relay decrypts and echos back all MSM_ECHO cells it receives on measurement connections until it has reported its amount of background traffic the same number of times as there are seconds in the measurement (e.g. 30 per-second reports for a 30 second measurement). After sending the last MSM_BG cell, the relay drops all buffered MSM_ECHO cells, closes all measurement connections, and exits measurement mode. During the measurement the relay targets a ratio of background traffic to measurement traffic as specified by a consensus parameter/torrc option. For a given ratio r, if the relay has handled x cells of measurement traffic recently, Tor then limits itself to y = xr/(1-r) cells of non-measurement traffic this scheduling round. The target will enforce that a minimum of 10 Mbit/s of measurement traffic is recorded since the last background traffic scheduling round to ensure it always allows some minimum amount of background traffic. 3.2 FlashFlow Components The FF coordinator and measurer code will reside in a FlashFlow repository separate from little-t tor. There are three notable parameters for which a FF deployment must choose values. They are: - The number of sockets, s, the measurers should open, in aggregate, with the target relay. We suggest s=160 based on the FF paper. - The bandwidth multiplier, m. Given an existing capacity estimate for a relay, z, the coordinator will instruct the measurers to, in aggregate, send m*z Mbit/s to the target relay. We recommend m=2.25. - The measurement duration, d. Based on the FF paper, we recommend d=30 seconds. The rest of this section first discusses notable functions of the FlashFlow coordinator, then goes on to discuss FF measurer code that will require supporting tor code. 3.2.1 FlashFlow Coordinator The coordinator is responsible for scheduling measurements, aggregating results, and producing v3bw files. It needs continuous access to new consensus files, which it can obtain by running an accompanying Tor process in client mode. The coordinator has the following functions, which will be described in this section: - result aggregation. - schedule measurements. - v3bw file generation. 3.2.1.1 Aggregating Results Every second during a measurement, the measurers send the amount of verified measurement traffic they have received back from the relay. Additionally, the relay sends a MSM_BG cell each second to the coordinator with amount of non-measurement background traffic it is sending and receiving. For each second's reports, the coordinator sums the measurer's reports. The coordinator takes the minimum of the relay's reported sent and received background traffic. If, when compared to the measurer's reports for this second, the relay's claimed background traffic is more than what's allowed by the background/measurement traffic ratio, then the coordinator further clamps the relay's report down. The coordinator adds this final adjusted amount of background traffic to the sum of the measurer's reports. Once the coordinator has done the above for each second in the measurement (e.g. 30 times for a 30 second measurement), the coordinator takes the median of the 30 per-second throughputs and records it as the estimated capacity of the target relay. 3.2.1.2 Measurement Schedule The short term implementation of measurement scheduling will be simpler than the long term one due to (1) there only being one FlashFlow deployment, and (2) there being very few relays that support being measured by FlashFlow. In fact the FF coordinator will maintain a list of the relays that have updated to support being measured and have opted in to being measured, and it will only measure them. The coordinator divides time into a series of 24 hour periods, commonly referred to as days. Each period has measurement slots that are longer than a measurement lasts (30s), say 60s, to account for pre- and post-measurement work. Thus with 60s slots there's 1,440 slots in a day. At the start of each day the coordinator considers the list of relays that have opted in to being measured. From this list of relays, it repeatedly takes the relay with the largest existing capacity estimate. It selects a random slot. If the slot has existing relays assigned to it, the coordinator makes sure there is enough additional measurer capacity to handle this relay. If so, it assigns this relay to this slot. If not, it keeps picking new random slots until one has sufficient additional measurer capacity. Relays without existing capacity estimates are assumed to have the 75th percentile capacity of the current network. If a relay is not online when it's scheduled to be measured, it doesn't get measured that day. 3.2.1.2.1 Example Assume the FF deployment has 1 Gbit/s of measurer capacity. Assume the chosen multiplier m=2. Assume there are only 5 slots in a measurement period. Consider a set of relays with the following existing capacity estimates and that have opted in to being measured by FlashFlow. - 500 Mbit/s - 300 Mbit/s - 250 Mbit/s - 200 Mbit/s - 100 Mbit/s - 50 Mbit/s The coordinator takes the largest relay, 500 Mbit/s, and picks a random slot for it. It picks slot 3. The coordinator takes the next largest, 300, and randomly picks slot 2. The slots are now: 0 | 1 | 2 | 3 | 4 -------|-------|-------|-------|------- | | 300 | 500 | | | | | The coordinator takes the next largest, 250, and randomly picks slot 2. Slot 2 already has 600 Mbit/s of measurer capacity reserved (300*m); given just 1000 Mbit/s of total measurer capacity, there is just 400 Mbit/s of spare capacity while this relay requires 500 Mbit/s. There is not enough room in slot 2 for this relay. The coordinator picks a new random slot, 0. 0 | 1 | 2 | 3 | 4 -------|-------|-------|-------|------- 250 | | 300 | 500 | | | | | The next largest is 200 and the coordinator randomly picks slot 2 again (wow!). As there is just enough spare capacity, the coordinator assigns this relay to slot 2. 0 | 1 | 2 | 3 | 4 -------|-------|-------|-------|------- 250 | | 300 | 500 | | | 200 | | The coordinator randomly picks slot 4 for the last remaining relays, in that order. 0 | 1 | 2 | 3 | 4 -------|-------|-------|-------|------- 250 | | 300 | 500 | 100 | | 200 | | 50 3.2.1.3 Generating V3BW files Every hour the FF coordinator produces a v3bw file in which it stores the latest capacity estimate for every relay it has measured in the last week. The coordinator will create this file on the host's local file system. Previously-generated v3bw files will not be deleted by the coordinator. A symbolic link at a static path will always point to the latest v3bw file. $ ls -l v3bw -> v3bw.2020-03-01-05-00-00 v3bw.2020-03-01-00-00-00 v3bw.2020-03-01-01-00-00 v3bw.2020-03-01-02-00-00 v3bw.2020-03-01-03-00-00 v3bw.2020-03-01-04-00-00 v3bw.2020-03-01-05-00-00 3.2.2 FlashFlow Measurer The measurers take commands from the coordinator, connect to target relays with many sockets, send them traffic, and verify the received traffic is the same as what was sent. Measurers need access to a lot of internal tor functionality. One strategy is to house as much logic as possible inside an compile-time-optional control port module that calls into other parts of tor. Alternatively FlashFlow could link against tor and call internal tor functions directly. [XXX for now I'll assume that an optional little-t tor control port module housing a lot of this code is the best idea.] Notable new things that internal tor code will need to do on the measurer (client) side: 1. Open many TLS+TCP connections to the same relay on purpose. 2. Verify echo cells. 3.2.2.1 Open many connections FlashFlow prototypes needed to "hack in" a flag in the open-a-connection-with-this-relay function call chain that indicated whether or not we wanted to force a new connection to be created. Most of Tor doesn't care if it reuses an existing connection, but FF does want to create many different connections. The cleanest way to accomplish this will be investigated. On the relay side, these measurer connections do not count towards DoS detection algorithms. 3.2.2.2 Verify echo cells A parameter will exist to tell the measurers with what frequency they shall verify that cells echoed back to them match what was sent. This parameter does not need to exist outside of the FF deployment (e.g. it doesn't need to be a consensus parameter). The parameter instructs the measurers to check 1 out of every N cells. The measurer keeps a count of how many measurement cells it has sent. It also logically splits its output stream of cells into buckets of size N. At the start of each bucket (when num_sent % N == 0), the measurer chooses a random index in the bucket. Upon sending the cell at that index (num_sent % N == chosen_index), the measurer records the cell. The measurer also counts cells that it receives. When it receives a cell at an index that was recorded, it verifies that the received cell matches the recorded sent cell. If they match, no special action is taken. If they don't match, the measurer indicates failure to the coordinator and target relay and closes all connections, ending the measurement. 3.2.2.2.1 Example Consider bucket_size is 1000. For the moment ignore cell encryption. We start at idx=0 and pick an idx in [0, 1000) to record, say 640. At idx=640 we record the cell. At idx=1000 we choose a new idx in [1000, 2000) to record, say 1236. At idx=1236 we record the cell. At idx=2000 we choose a new idx in [2000, 3000). Etc. There's 2000+ cells in flight and the measurer has recorded two items: - (640, contents_of_cellA) - (1236, contents_of_cellB) Consider the receive side now. It counts the cells it receives. At receive idx=640, it checks the received cell matches the saved cell from before. At receive idx=1236, it again checks the received cell matches. Etc. 3.2.2.2.2 Motivation A malicious relay may want to skip decryption of measurement cells to save CPU cycles and obtain a higher capacity estimate. More generally, it could generate fake measurement cells locally, ignore the measurement traffic it is receiving, and flood the measurer with more traffic that it (the measurer) is even sending. The security of echo cell verification is discussed in section 3.3.1. 3.3 Security In this section we discuss the security of various aspects of FlashFlow and the tor changes it requires. 3.3.1 Echo Cell Verification: Bucket Size A smaller bucket size means more cells are checked and FF is more likely to detect a malicious target. It also means more bookkeeping overhead (CPU/RAM). An adversary that knows bucket_size and cheats on one item out of every bucket_size items will have a 1/bucket_size chance of getting caught in the first bucket. This is the worst case adversary. While cheating on just a single item per bucket yields very little advantage, cheating on more items per bucket increases the likelihood the adversary gets caught. Thus only the worst case is considered here. In general, the odds the adversary can successfully cheat in a single bucket are (bucket_size-1)/bucket_size Thus the odds the adversary can cheat in X consecutive buckets are [(bucket_size-1)/bucket_size]^X In our case, X will be highly varied: Slow relays won't see very many buckets, but fast relays will. The damage to the network a very slow relay can do by faking being only slightly faster is limited. Nonetheless, for now we motivate the selection of bucket_size with a slow relay: - Assume a very slow relay of 1 Mbit/s capacity that will cheat 1 cell in each bucket. Assume a 30 second measurement. - The relay will handle 1*30 = 30 Mbit of traffic during the measurement, or 3.75 MB, or 3.75 million bytes. - Cells are 514 bytes. Approximately (e.g. ignoring TLS) 7300 cells will be sent/recv over the course of the measurement. - A bucket_size of 50 results in about 146 buckets over the course of the 30s measurement. - Therefore, the odds of the adversary cheating successfully as (49/50)^(146), or about 5.2%. This sounds high, but a relay capable of double the bandwidth (2 Mbit/s) will have (49/50)^(2*146) or 0.2% odds of success, which is quite low. Wanting a <1% chance that a 10 Mbit/s relay can successfully cheat results in a bucket size of approximately 125: - 10*30 = 300 Mbit of traffic during 30s measurement. 37.5 million bytes. - 37,500,000 bytes / 514 bytes/cell = ~73,000 cells - bucket_size of 125 cells means 73,000 / 125 = 584 buckets - (124/125)^(584) = 0.918% chance of successfully cheating Slower relays can cheat more easily but the amount of extra weight they can obtain is insignificant in absolute terms. Faster relays are essentially unable to cheat. 3.3.2 Weight Inflation Target relays are an active part of the measurement process; they know they are getting measured. While a relay cannot fake the measurement traffic, it can trivially stop transferring client background traffic for the duration of the measurement yet claim it carried some. More generally, there is no verification of the claimed amount of background traffic during the measurement. The relay can claim whatever it wants, but it will not be trusted above the ratio the FlashFlow deployment is configured to know. This places an easy to understand, firm, and (if set as we suggest) low cap on how much a relay can inflate its measured capacity. Consider a background/measurement ratio of 1/4, or 25%. Assume the relay in question has a hard limit on capacity (e.g. from its NIC) of 100 Mbit/s. The relay is supposed to use up to 25% of its capacity for background traffic and the remaining 75%+ capacity for measurement traffic. Instead the relay ceases carrying background traffic, uses all 100 Mbit/s of capacity to handle measurement traffic, and reports ~33 Mbit/s of background traffic (33/133 = ~25%). FlashFlow would trust this and consider the relay capable of 133 Mbit/s. (If the relay were to report more than ~33 Mbit/s, FlashFlow limits it to just ~33 Mbit/s.) With r=25%, FlashFlow only allows 1.33x weight inflation. Prior work shows that Torflow allows weight inflation by a factor of 89x [0] or even 177x [1]. The ratio chosen is a trade-off between impact on background traffic and security: r=50% allows a relay to double its weight but won't impact client traffic for relays with steady state throughput below 50%, while r=10% allows a very low inflation factor but will cause throttling of client traffic at far more relays. We suggest r=25% (and thus 1/(1-0.25)=1.33x inflation) for a reasonable trade-off between performance and security. It may be possible to catch relays performing this attack, especially if they literally drop all background traffic during the measurement: have the measurer (or some party on its behalf) create a regular stream through the relay and measure the throughput on the stream before/during/after the measurement. This can be explored longer term. 3.3.3 Incomplete Authentication The short term FlashFlow implementation has the relay set two torrc options if they would like to allow themselves to be measured: a flag allowing measurement, and the list of coordinator TLS certificate that are allowed to start a measurement. The relay drops MSM_PARAMS cells from coordinators it does not trust, and immediately closes the connection after that. A FF coordinator cannot convince a relay to enter measurement mode unless the relay trusts its TLS certificate. A trusted coordinator specifies in the MSM_PARAMS cell the IP addresses of the measurers the relay shall expect to connect to it shortly. The target adds the measurer IP addresses to a whitelist in the DoS connection limit system, exempting them from any configured connection limit. If a measurer is behind a NAT, an adversary behind the same NAT can DoS the relay's available sockets until the end of the measurement. The adversary could also pretend to be the measurer. Such an adversary could induce measurement failures and inaccuracies. (Note: the whitelist is cleared after the measurement is over.) 4. FlashFlow measurement system: Medium term The medium term deployment stage begins after FlashFlow has been implemented and relays are starting to update to a version of Tor that supports it. We plan to host a FlashFlow deployment consisting of a FF coordinator and a single FF measurer on a single 1 Gbit/s machine. Data produced by this deployment will be made available (semi?) publicly, including both v3bw files and intermediate results. Any development changes needed during this time would go through separate proposals. 5. FlashFlow measurement system: Long term In the long term, finishing-touch development work will be done, including adding better authentication and measurement scheduling, and experiments will be run to determine the best way to integrate FlashFlow into the Tor ecosystem. Any development changes needed during this time would go through separate proposals. 5.1 Authentication to Target Relay Short term deployment already had FlashFlow coordinators using TLS certificates when connecting to relays, but in the long term, directory authorities will vote on the consensus parameter for which coordinators should be allowed to perform measurements. The voting is done in the same way they currently vote on recommended tor versions. FlashFlow measurers will be updated to use TLS certificates when connecting to relays too. FlashFlow coordinators will update the contents of MSM_PARAMS cells to contain measurer TLS certificates instead of IP addresses, and relays will update to expect this change. 5.2 Measurement Scheduling Short term deployment only has one FF deployment running. Long term this may no longer be the case because, for example, more than one directory authority decides to adopt it and they each want to run their own deployment. FF deployments will need to coordinate between themselves to not measure the same relay at the same time, and to handle new relays as they join during the middle of a measurement period (during the day). The following is quoted from Section 4.3 of the FlashFlow paper. To measure all relays in the network, the BWAuths periodically determine the measurement schedule. The schedule determines when and by whom a relay should be measured. We assume that the BWAuths have sufficiently synchronized clocks to facilitate coordinating their schedules. A measurement schedule is created for each measurement period, the length p of which determines how often a relay is measured. We use a measurement period of p = 24 hours. To help avoid active denial-of-service attacks on targeted relays, the measurement schedule is randomized and known only to the BWAuths. Before the next measurement period starts, the BWAuths collectively generate a random seed (e.g. using Tor’s secure-randomness protocol). Each BWAuth can then locally determine the shared schedule using pseudorandom bits extracted from that seed. The algorithm to create the schedule considers each measurement period to be divided into a sequence of t-second measurement slots. For each old relay, slots for each BWAuth to measure it are selected uniformly at random without replacement from all slots in the period that have sufficient unallocated measurement capacity to accommodate the measurement. When a new relay appears, it is measured separately by each BWAuth in the first slots with sufficient unallocated capacity. Note that this design ensures that old relays will continue to be measured, with new relays given secondary priority in the order they arrive. 5.3 Experiments [XXX todo] 5.4 Other Changes/Investigations/Ideas - How can FlashFlow data be used in a way that doesn't lead to poor load balancing given the following items that lead to non-uniform client behavior: - Guards that high-traffic HSs choose (for 3 months at a time) - Guard vs middle flag allocation issues - New Guard nodes (Guardfraction) - Exit policies other than default/all - Directory activity - Total onion service activity - Super long-lived circuits - Add a cell that the target relay sends to the coordinator indicating its CPU and memory usage, whether it has a shortage of sockets, how much bandwidth load it has been experiencing lately, etc. Use this information to lower a relays weight, never increase. - If FlashFlow and sbws work together (as opposed to FlashFlow replacing sbws), consider logic for how much sbws can increase/decrease FF results - Coordination of multiple FlashFlow deployments: scheduling of measurements, seeding schedule with shared random value. - Other background/measurement traffic ratios. Dynamic? (known slow relay => more allowed bg traffic?) - Catching relays inflating their measured capacity by dropping background traffic. - What to do about co-located relays. Can they be detected reliably? Should we just add a torrc option a la MyFamily for co-located relays? - What is the explanation for dennis.jackson's scary graphs in this [2] ticket? Was it because of the speed test? Why? Will FlashFlow produce the same behavior? 6. Citations [0] F. Thill. Hidden Service Tracking Detection and Bandwidth Cheating in Tor Anonymity Network. Master’s thesis, Univ. Luxembourg, 2014. [1] A. Johnson, R. Jansen, N. Hopper, A. Segal, and P. Syverson. PeerFlow: Secure Load Balancing in Tor. Proceedings on Privacy Enhancing Technologies (PoPETs), 2017(2), April 2017. [2] Mike Perry: Graph onionperf and consensus information from Rob's experiments https://trac.torproject.org/projects/tor/ticket/33076

5 8

Proposal 320: Removing TAP usage from v2 onion services
by Nick Mathewson 29 May '20

29 May '20

``` Filename: 320-tap-out-again.md Title: Removing TAP usage from v2 onion services Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` (This proposal is part of the Walking Onions spec project. It updates proposal 245.) # Removing TAP from v2 onion services As we implement walking onions, we're faced with a problem: what to do with TAP keys? They are bulky and insecure, and not used for anything besides v2 onion services. Keeping them in SNIPs would consume bandwidth, and keeping them indirectly would consume complexity. It would be nicer to remove TAP keys entirely. But although v2 onion services are obsolescent and their cryptographic parameters are disturbing, we do not want to drop support for them as part of the Walking Onions migration. If we did so, then we would force some users to choose between Walking Onions and v2 onion services, which we do not want to do. Instead, we describe here a phased plan to replace TAP in v2 onion services with ntor. This change improves the forward secrecy of _some_ of the session keys used with v2 onion services, but does not improve their authentication, which is strongly tied to truncated SHA1 hashes of RSA1024 keys. Implementing this change is more complex than similar changes elsewhere in the Tor protocol, since we do not want clients or services to leak whether they have support for this proposal, until support is widespread enough that revealing it is no longer a privacy risk. ## Ntor keys, link specifiers, and SNIPs in v2 descriptors. We define these entries that may appear in v2 onion service descriptors, once per introduction point. "identity-ed25519" "ntor-onion-key" [at most once each per intro point.] These values are in the same format as and follow the same rules as their equivalents in router descriptors. "link-specifiers" [at most once per introduction point] This value is the same as the link specifiers in a v3 onion service descriptor, and follows the same rules. Services should not include any of these fields unless a new network parameter, "hsv2-intro-updated" is set to 1. Clients should not parse these fields or use them unless "hsv2-use-intro-updated" is set to 1. We define a new field that can be used for hsv2 descriptors with walking onions: "snip" [at most once] This value is the same as the snip field introduced to a v3 onion service descriptor by proposal (XXX) and follows the same rules. Services should not include this field unless a new network parameter, "hsv2-intro-snip" is set to 1. Clients should not parse this field or use it unless the parameter "hsv2-use-intro-snip" is set to 1. Additionally, relays SHOULD omit the following legacy intro point parameters when a new network parameter, "hsv2-intro-legacy" is set to 0: "ip-address", "onion-port", and "onion-key". Clients should treat them as optional when "hsv2-tolerate-no-legacy" is set to 1. ## INTRODUCE cells, RENDEZVOUS cells, and ntor. We allow clients to specify the rendezvous point's ntor key in the INTRODUCE2 cell instead of the TAP key. To do this, the client simply sets KLEN to 32, and includes the ntor key for the relay. Clients should only use ntor keys in this way if the network parameter "hsv2-client-rend-ntor" is set to 1, and if the entry "allow-rend-ntor" is present in the onion service descriptor. Services should only advertise "allow-rend-ntor" in this way if the network parameter "hsv2-service-rend-ntor" is set to 1. ## Migration steps First, we implement all of the above, but set it to be disabled by default. We use torrc fields to selectively enable them for testing purposes, to make sure they work. Once all non-LTS versions of Tor without support for this proposal are obsolete, we can safely enable "hsv2-client-rend-ntor", "hsv2-service-rend-ntor", "hsv2-intro-updated", and "hsv2-use-intro-updated". Once all non-LTS versions of Tor without support for walking onions are obsolete, we can safely enable "hsv2-intro-snip", "hsv2-use-intro-snip", and "hsv2-tolerate-no-legacy". Once all non-LTS versions of Tor without support for _both_ of the above implementations are finally obsolete, we can finally set "hsv2-intro-legacy" to 0. ## Future work There is a final TAP-like protocol used for v2 hidden services: the client uses RSA1024 and DH1024 to send information about the rendezvous point and to start negotiating the session key to be used for end-to-end encryption. In theory we could get a benefit to forward secrecy by using ntor instead of TAP here, but we would get not corresponding benefit for authentication, since authentication is still ultimately tied to HSv2's scary RSA1024-plus-truncated-SHA1 combination. Given that, it might be just as good to allow the client to insert a curve25519 key in place of their DH1024 key, and use that for the DH handshake instead. That would be a separate proposal, though: this proposal is enough to allow all relays to drop TAP support.

8 11

Proposal 322: Extending link specifiers to include the directory port
by Nick Mathewson 27 May '20

27 May '20

``` Filename: 322-dirport-linkspec.md Title: Extending link specifiers to include the directory port Author: Nick Mathewson Created: 27 May 2020 Status: Open ``` ## Motivation Directory ports remain the only way to contact a (non-bridge) Tor relay that isn't expressible as a Link Specifier. We haven't specified a link specifier of this kind so far, since it isn't a way to contact a relay to create a channel. But authorities still expose directory ports, and encourage relays to use them preferentially for uploading and downloading. And with Walking Onions, it would be convenient to try to make every kind of "address" a link specifier -- we'd like want authorities to be able to specify a list of link specifiers that can be used to contact them for uploads and downloads. > It is possible that after revision, Walking Onions won't need a way > to specify this information. If so, this proposal should be moved > to "Reserve" status as generally unuseful. ## Proposal We reserve a new link specifier type "dir-url", for use with the directory system. This is a variable-length link specifier, containing a URL prefix. The only currently supported URL schema is "http://". Implementations SHOULD ignore unrecognized schemas. IPv4 and IPv6 addresses MAY be used directory; hostnames are also allowed. Implementations MAY ignore hostnames and only use raw addresses. The URL prefix includes everything through the string "tor" in the directory hierarchy. A dir-url link specifier SHOULD NOT appear in an EXTEND cell; implementations SHOULD reject them if they do appear.

1 0

Proposal 321: Better performance and usability for the MyFamily option (v2)
by Nick Mathewson 27 May '20

27 May '20

``` Filename: 321-happy-families.md Title: Better performance and usability for the MyFamily option (v2) Author: Nick Mathewson Created: 27 May 2020 Status: Open ``` ## Problem statement. The current family mechanism allows well-behaved relays to identify that they all belong to the same 'family', and should not be used in the same circuits. Right now, families work by having every family member list every other family member in its server descriptor. This winds up using O(n^2) space in microdescriptors and server descriptors. (For RAM, we can de-duplicate families which sometimes helps.) Adding or removing a server from the family requires all the other servers to change their torrc settings. The is growth in size is not just a theoretical problem. Family declarations currently make up a little over 55% of the microdescriptors in the directory--around 24% after compression. The largest family has around 270 members. With Walking Onions, 270 members times a 160-bit hashed identifier leads to over 5 kilobytes per SNIP, which is much greater than we'd want to use. This is an updated version of proposal 242. It differs by clarifying requirements and providing a more detailed migration plan. ## Design overview. In this design, every family has a master ed25519 "family key". A node is in the family iff its server descriptor includes a certificate of its ed25519 identity key with the family key. The certificate format the one in the tor-certs.txt spec; we would allocate a new certificate type for this usage. These certificates would need to include the signing key in the appropriate extension. Note that because server descriptors are signed with the node's ed25519 signing key, this creates a bidirectional relationship between the two keys, so that nodes can't be put in families without their consent. ## Changes to router descriptors We add a new entry to server descriptors: "family-cert" NL "-----BEGIN FAMILY CERT-----" NL cert "-----END FAMILY CERT-----". This entry contains a base64-encoded certificate as described above. It may appear any number of times; authorities MAY reject descriptors that include it more than three times. ## Changes to microdescriptors We add a new entry to microdescriptors: `family-keys`. This line contains one or more space-separated strings describing families to which the node belongs. These strings MUST be sorted in lexicographic order. Clients MUST NOT depend on any particular property of these strings. ## Changes to voting algorithm We allocate a new consensus method number for voting on these keys. When generating microdescriptors using a suitable consensus method, the authorities include a "family-keys" line if the underlying server descriptor contains any valid family-cert lines. For each valid family-cert in the server descriptor, they add a base-64-encoded string of that family-cert's signing key. > See also "deriving family lines from family-keys?" below for an > interesting but more difficult extension mechanism that I would > not recommend. ## Relay configuration There are several ways that we could configure relays to let them include family certificates in their descriptors. The easiest would be putting the private family key on each relay, so that the relays could generate their own certificates. This is easy to configure, but slightly risky: if the private key is compromised on any relay, anybody can claim membership in the family. That isn't so very bad, however -- all the relays would need to do in this event would be to move to a new private family key. A more orthodox method would be to keep the private key somewhere offline, and using it to generate a certificate for each relay in the family as needed. These certificates should be made with long-enough lifetimes, and relays should warn when they are going to expire soon. ## Changes to relay behavior Each relay should track which other relays they have seen using the same family-key as itself. When generating a router descriptor, each relay should list all of these relays on the legacy 'family' line. This keeps the "family" lines up-to-date with "family-keys" lines for compliant relays. Relays should continue listing relays in their family lines if they have seen a relay with that identity using the same family-key at any time in the last 7 days. The presence of this line should be configured by a network parameter, `derive-family-line`. Relays whose family lines do not stay at least mostly in sync with their family keys should be marked invalid by the authorities. ## Client behavior Clients should treat node A and node B as belonging to the same family if ANY of these is true: * The client has descriptors for A and B, and A's descriptor lists B in its family line, and B's descriptor lists A in its family line. * Client A has descriptors for A and B, and they both contain the same entry in their family-keys or family-cert. ## Migration For some time, existing relays and clients will not support family certificates. Because of this, we try to make sure above the well-behaved relays will list the same entries in both places. Once enough clients have migrated to using family certificates, authorities SHOULD disable `derive-family-line`. ## Security Listing families remains as voluntary in this design as in today's Tor, though bad-relay hunters can continue to look for families that have not adopted a family key. A hostile relay family could list a "family" line that did not match its "family-certs" values. However, the only reason to do so would be in order to launch a client partitioning attack, which is probably less valuable than the kinds of attacks that they could run by simply not listing families at all. ## Appendix: deriving family lines from family-keys? As an alternative, we might declare that _authorities_ should keep family lines in sync with family-certs. Here is a design sketch of how we might do that, but I don't think it's actually a good idea, since it would require major changes to the data flow of the voting system. In this design, authorties would include a "family-keys" line in each router section in their votes corresponding to a relay with any family-cert. When generating final microdescriptors using this method, the authorities would use these lines to add entries to the microdescriptors' family lines: 1. For every relay appearing in a routerstatus's family-keys, the relays calculate a consensus family-keys value by listing including all those keys that are listed by a majority of those voters listing the same router with the same descriptor. (This is the algorithm we use for voting on other values derived from the descriptor.) 2. The authorities then compute a set of "expanded families": one for each family key. Each "expanded family" is a set containing every router in the consensus associated with that key in its consensus family-keys value. 3. The authorities discard all "expanded families" of size 1 or smaller. 4. Every router listed for the "expanded family" has every other router added to the "family" line in its microdescriptor. (The "family" line is then re-canonicalized according to the rules of proposal 298 to remove its ) 5. Note that the final microdescriptor consensus will include the digest of the derived microdescriptor in step 4, rather than the digest of the microdescriptor listed in the original votes. (This calculation is deterministic.) The problem with this approach is that authorities would have to s to fetch microdescriptors they do not have in order to replace their family lines. Currently, voting never requires an authority to fetch a microdescriptor from another authority. If we implement vote compression and diffs as in the Walking Onions proposal, however, we might suppose that votes could include microdescriptors directly. Still, this is likely more complexity than we want for a transition mechanism. ## Appendix: Deriving family-keys from families?? We might also imagine that authorities could infer which families exist from the graph of family relationships, and then include synthetic "family-keys" entries for routers that belong to the same family. This has two challenges: first, to compute these synthetic family keys, the authorities would need to have the same graph of family relationships to begin with, which once again would require them to include the complete list of families in their votes. Secondly, finding all the families is equivalent to finding all maximal cliques in a graph. This problem is NP-hard in its general case. Although polynomial solutions exist for nice well-behaved graphs, we'd still need to worry about hostile relays including strange family relationships in order to drive the algorithm into its exponential cases. ## Appendix: New assigned values We need a new assigned value for the certificate type used for family signing keys. We need a new consensus method for placing family-keys lines in microdescriptors. ## Appendix: New network parameters * `derive-family-line`: If 1, relays should derive family lines from observed family-keys. If 0, they do not. Min: 0, Max: 1. Default: 1.

1 0

So Windows is Adding a Package Manager.....
by Keifer Bly 24 May '20

24 May '20

2 1

tor-dev Digest, Vol 112, Issue 21
by Support Team 20 May '20

20 May '20

Thank you for the insights. On Wed, May 20 2020, at 12:00 PM, <tor-dev(a)lists.torproject.org> Send tor-dev mailing list submissions to tor-dev(a)lists.torproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev or, via email, send a message with subject or body 'help' to tor-dev-request(a)lists.torproject.org You can reach the person managing the list at tor-dev-owner(a)lists.torproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of tor-dev digest..." Today's Topics: 1. Re: Proposal 319: RELAY_FRAGMENT cells (Nick Mathewson) 2. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 3. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 4. Re: Proposal 320: Removing TAP usage from v2 onion services (Sebastian Hahn) ---------------------------------------------------------------------- Message: 1 Date: Tue, 19 May 2020 13:29:54 -0400 From: Nick Mathewson To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal 319: RELAY_FRAGMENT cells Message-ID: Content-Type: text/plain; charset="UTF-8" On Thu, May 14, 2020 at 3:15 PM David Goulet wrote: > > On 11 May (16:47:24), Nick Mathewson wrote: [...] > > # Onion service concerns. > > > > We allocate a new extension for use in the ESTABLISH_INTRO by onion services, > > to indicate that they can receive a wide INTRODUCE2 cell. This extension > > contains: > > > > struct wide_intro2_ok { > > u16 max_len; > > } > > > > We allocate a new extension for use in the 'ESTABLISH_RENDEZVOUS' > > cell, to indicate acceptance of wide 'RENDEZVOUS2' cells. This > > extension contains: > > > > struct wide_rend2_ok { > > u16 max_len; > > } > > > > (Note that 'ESTABLISH_RENDEZVOUS' cells do not currently have a an > > extension mechanism. They should be extended to use the same > > extension format as 'ESTABLISH_INTRO' cells, with extensions placed > > after the rendezvous cookie.) > > Why would a client need to announce wide cells in the ESTABLISH phase as > opposed to using protover "Relay=N" ? This is not for announcing support of wide cells -- this is for reporting a setting for how wide fragmented cells should be. > The maximum length of a fragmented cell is capped to 2^16 (u16) so we don't > really need the establish process to inform us of the maximum expected length > but rather use the max_len in the first fragment? This all comes back to an earlier part of the proposal: Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. In other words, I'm imagining that there is a maximum length for each cell type that is much shorter than 65535, even though we're using two bytes for the length field. The extension in the establish_intro cell is to tell the intro point the longest introduce1 cell that it should accept; this extension in the establish_rend cell is to tell the rendezvous point the longest rendezvous1 cell that it should accept. Another way we could do this would be with a set of network parameters to describe the maximum length of each fragmented cell. Do you think that would be simpler? (I can't quite remember why I specified it this way in the first place.) > Furthermore, ESTABLISH_INTRO has extensions (only 1 as of today) so they could > also be fragments themselves and thus I'm not sure I see the point of having > two different ways of "expecting" fragments for the ESTABLISH_* cells and the > INTRO/RENDEZVOUS cells? The difference thing here is that everybody can tell which protocols that a relay supports, but there is no automatic way to tell which protocols an onion service or client supports. Since INTRODUCE2/RENDEZVOUS2 cells are handled by these clients, they need to get opted into by the relays. (I'm not sure I understood the question completely.) > > # Compatibility > > > > This proposal will require the allocation of a new 'Relay' protocol version, > > to indicate understanding of the RELAY_FRAGMENTED command. > > Here is a thought about a DoS vector. Here goes: > > As an upper limit of 65KB total fragment size, it represents ~126 cells in > total so I could basically send *125* cells and then stop which will put in > memory a bit more than 64KB and it will stay there until the last fragment is > received. > > And then I do that on 1000 different circuits bringing the total count in > memory to 64GB. All stuck there, all "waiting" for the last fragment. > > Our OOM would kick in killing circuits but it just seems to me a very easy way > to continously kick the OOM of a _service_ which is pretty bad side channel. A few responses here: First, we shouldn't allow 65535-byte fragmented cells. The actual maximum length should be something more like 1024 or 4096 bytes. Second, we should make sure that when we are reassembling cells, we use the same buf_t buffers that we use for other stuff. Our buffers are timestamped, so we can tell which buffer has had data stalling for the longest, and we should use that to make sure we're killing off the right circuits preferentially. Third, fragments should only be allowed at an onion service for INTRODUCE2, and those should only come one at a time from each introduction point, so the number that it's reassembling at the time will be limited by the number of intro circuits it has open. It'll be the the intro points that have to be keeping a bunch of cells in assembly at once, and be ready to kill off circuits that dawdle too long. Does this make more sense? If so I'll try to clarify it in the proposal.

1 0

Proposal 319: RELAY_FRAGMENT cells
by Nick Mathewson 19 May '20

19 May '20

``` Filename: 319-wide-everything.md Title: RELAY_FRAGMENT cells Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` (This proposal is part of the Walking Onions spec project.) # Introduction Proposal 249 described a system for `CREATE` cells to become wider, in order to accommodate hybrid crypto. And in order to send those cell bodies across circuits, it described a way to split `CREATE` cells into multiple `EXTEND` cells. But there are other cell types that can need to be wider too. For example, `INTRODUCE` and `RENDEZVOUS` cells also contain key material used for a handshake: if handshakes need to grow larger, then so do these cells. This proposal describes an encoding for arbitrary "wide" relay cells, that can be used to send a wide variant of anything. To be clear, although this proposal describes a way that all relay cells can become "wide", I do not propose that wide cells should actually be _allowed_ for all relay cell types. # Proposal We add a new relay cell type: `RELAY_FRAGMENT`. This cell type contains part of another relay cell. A `RELAY_FRAGEMENT` cell can either introduce a new fragmented cell, or can continue one that is already in progress. The format of a RELAY_FRAGMENT body is one of the following: // First body in a series struct fragment_begin { // What relay_command is in use for the underlying cell? u8 relay_command; // What will the total length of the cell be once it is reassembled? u16 total_len; // Bytes for the cell body u8 body[]; } // all other cells. struct fragment_continued { // More bytes for the cell body. u8 body[]; } To send a fragmented cell, first a party sends a RELAY_FRAGMENT cell containing a "fragment_begin" payload. This payload describes the total length of the cell, the relay command Fragmented cells other than the last one in sequence MUST be sent full of as much data as possible. Parties SHOULD close a circuit if they receive a non-full fragmented cell that is not the last fragment in a sequence. Fragmented cells MUST NOT be interleaved with other relay cells on a circuit, other than cells used for flow control. (Currently, this is only SENDME cells.) If any party receives any cell on a circuit, other than a flow control cell or a RELAY_FRAGEMENT cell, before the fragmented cell is complete, than it SHOULD close the circuit. Parties MUST NOT send extra data in fragmented cells beyond the amount given in the first 'total_len' field. Not every relay command may be sent in a fragmented cell. In this proposal, we allow the following cell types to be fragmented: EXTEND2, EXTENDED2, INTRODUCE1, INTRODUCE2, RENDEZVOUS. Any party receiving a command that they believe should not be fragmented should close the circuit. Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. All `RELAY_FRAGMENT` cells for the fragmented cell must have the same Stream ID. (For those cells allowed above, the Stream ID is always zero.) Implementations SHOULD close a circuit if they receive fragments with mismatched Stream ID. # Onion service concerns. We allocate a new extension for use in the ESTABLISH_INTRO by onion services, to indicate that they can receive a wide INTRODUCE2 cell. This extension contains: struct wide_intro2_ok { u16 max_len; } We allocate a new extension for use in the `ESTABLISH_RENDEZVOUS` cell, to indicate acceptance of wide `RENDEZVOUS2` cells. This extension contains: struct wide_rend2_ok { u16 max_len; } (Note that `ESTABLISH_RENDEZVOUS` cells do not currently have a an extension mechanism. They should be extended to use the same extension format as `ESTABLISH_INTRO` cells, with extensions placed after the rendezvous cookie.) # Handling RELAY_EARLY The first fragment of each EXTEND cell should be tagged with `RELAY_EARLY`. The remaining fragments should not. Relays should accept `EXTEND` cells if and only if their _first_ fragment is tagged with `RELAY_EARLY`. > Rationale: We could allow any fragment to be tagged, but that would give > hostile guards an opportunity to move RELAY_EARLY tags around and build a > covert channel. But if we later move to a relay encryption method that > lets us authenticate RELAY_EARLY, we could then require only that _any_ > fragment has RELAY_EARLY set. # Compatibility This proposal will require the allocation of a new 'Relay' protocol version, to indicate understanding of the RELAY_FRAGMENTED command.

4 3

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Fri, 15 May 2020 11:20:54 +1000 teor <teor(a)riseup.net> wrote: > You could try deleting functions, and then running tor's > "make test-network-all". If that test still passes, then the code is probably > unused (in practice). > > Chutney's CI does a few more tests for different tor versions, but I'm pretty > sure they all use the same code. This is what I will end up doing. I planned on it, but humans tend to know the purpose of the code better, and computers just run what is given to them :) so I figured it was best to check here first, with people more experienced with the codebase. Thanks, Caitlin

1 0

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Thu, 14 May 2020 19:58:45 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > isInExpectedDirInfoDocs looks like it might once have done something useful. My impression is to remove it unless we anticipate needing it in the near future. > configure and restart and wait_for_bootstrap are all in use; they are > invoked by name, from the command line, at the end of runConfigFile, > where it says `return getattr(network, verb)()`. Ah, I see now, thank you. I think it would be wise to comment the functions as such to indicate that they are used on the command line. > I think the rest are likely to be unused. I did grep most of these and found no signs of use, so this is my impression as well. For now I will make a mental note of all this while I work on more pertinent improvements to the code, but I do believe it would be nice to deal with truly unused code sooner rather than later. Caitlin

1 0

Deprecating Tor Protocol Versions
by teor 15 May '20

15 May '20

Hi all, Nick and I were talking about how we remove legacy features in tor, and their corresponding subprotocol versions. Here is a list of the current subprotocol versions: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n2049 Here's a recent protocol version proposal, which deals with recommending and requiring new features: https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-remov… But we don't have a similar proposal for removing support for older protocol versions from the tor codebase. For an example of what that proposal could look like, see our proposal for deprecating consensus methods: https://gitweb.torproject.org/torspec.git/tree/proposals/290-deprecate-cons… Here's the original conversation Nick and I had: https://github.com/torproject/tor/pull/1874#discussion_r423713579 But after reading our consensus methods deprecation proposal, I've changed my mind. I think that we should check for "protocol N, and any later version" by default. That's what we do for consensus methods, and it seems to work well. We can drop the earliest consensus methods, and recent tor versions just keep working. If we need an incompatible change, we can make it another protocol version, and recommend then require support for it. So here's an edited version of my notes on that ticket: There are a few instances of ">=" and "=" confusion in protocol versions. We should try to fix them all. It only matters when we remove protocol versions. We haven't really specified, tested, or exercised this functionality in practice. And so our reviewers lack experience. (And when we did discover a need for it with NSS and LinkAuth, it was more complicated than we expected.) I'd like to see a proposal that tells us how to check future protocol versions as they are added. Along with a migration plan for disabling protocol versions. So let's also open a ticket to check for "any future version". We should replace all "=" checks with ">=". Let's make sure we check all the places where we use protocol versions, even if they don't have a summary flag. Overall, I think it would be helpful if future protocol versions were orthogonal. Or if they depend on earlier features, that dependency should be clearly documented. (For example, Relay=3 IPv6 extends depends on Relay=2 EXTEND2 cells. So if we were checking EXTEND2 cell support, it would be Relay=2 or Relay=3.) T -- teor ----------------------------------------------------------------------

2 2