- tor-dev - lists.torproject.org

Tor Bandwidth Measurements Document Format
by juga 17 Apr '18

17 Apr '18

Hi, as commented with teor and pastly, i send in-line a draft specification for the document format that the bandwidth scanner implementations should produce. I've left my own questions/notes in square brackets. Thanks, juga. ======================================= Tor Bandwidth Measurements Document Format [juga: which name should we give to this document?] 1. Scope and preliminaries This document describes the format of Tor's bandwidth measurements document, version X.X.X [juga: which version should be this?] and later. Since Tor version X.X.X [juga: which tor version?] the directory authorities use the bandwidth measurements document called "V3BandwidthsFile" and produced by Torflow [1] (format described in README.spec.txt [2]). The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 1.2. Acknowledgements The original bandwidth measurement scanner (Torflow) and format was created by mike. Teor suggested to write this specification while contributing on pastly's new bandwidth scanner implementation. This specification was revised after feedback from: XXX 1.3 Outline The bandwidth measurements mentioned in sections 3.4.1 and 3.4.2 of dir-spec.txt [3] are obtained by bandwidth authorities, which are either directory authorities or other servers running bandwidth measurement scanners and sending the results to the former. [juga: it seems that bandwidth authorities have not been formally before] 2. Format details Bandwidth measurements MUST contain the following sections: - Header (exactly once) - Relays measurements (zero or more times) Each section (or entry) ends with a separator. 2.1. Nonterminals The following nonterminals are defined in the Onionoo details document specification [4]: fingerprint nickname In the bandwidth measurement documents nickname is optional. The following nonterminals are defined in the in dir-spec.txt: NL (newline) SP (space) "bw" = INT, the aggregated measured bandwidth of this relay, in kilobytes per second. We introduce the following nonterminals: [juga: this should probably be defined more formally and should probably link to other documents, which ones?] "version" = The name and the version of the bandwidth scannner software, such as "sbws 0.1.0". The name of the software, if absent, is assumed to be "torflow". [juga: which should be the version if absent?] "timestamp" = INT, the Unix Epoch time when the file was created. 2.2. Header format It MUST consists of: "timestamp" timestamp NL "version" version NL 2.3. Relay measurements format Relays measurements MUST consist of the following items. "node_id" fingerprint SP "bw" bandwidth SP When there are no more items, the "bw" item ends with NL instead of SP. 2.4. Optional extra items Different implementations of the bandwidth measurements scanners MAY include other items per relay. For instance, sbws includes: "rtt" = INT, Round Trip Time (to obtain 1B) Every relay measurement in sbws consists of: "node_id" fingerprint SP "bw" bandwidth SP "nick=" nickname SP "rtt=" rtt SP "time=" timestamp NL Every relay measurement in Torflow consists of: "node_id" fingerprint SP "bw" bandwidth SP "nick=" nickname SP "measured_at=" slice timestamp NL The "measured_at" does not correspond to the "time" in sbws. [juga: actually, if bwauths use "measured_at", then the code on them or sbws should be changed]. Torflow includes other items that are out of the scope of this document. References: 1. https://gitweb.torproject.org/torflow.git 2. https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/… 3. https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt 4. https://metrics.torproject.org/onionoo.html#details A. Sample data A.1. Torflow 1523911758 node_id=$68A483E05A2ABDCA6DA5A3EF8DB5177638A27F80 bw=392760 nick=Test measured_at=1523911725 updated_at=1523911725 pid_error=4.11374090719 pid_error_sum=4.11374090719 pid_bw=57136645 pid_delta=2.12168374577 circ_fail=0.2 scanner=/filepath A.2. sbws 1523911758 version=0.1.0 node_id=$68A483E05A2ABDCA6DA5A3EF8DB5177638A27F80 bw=392760 nick=Test rtt=380 time=1523911725

2 1

Hogging of second_elapsed_callback()
by Linus Nordberg 15 Apr '18

15 Apr '18

Hi, How long time can we spend signing status documents before tor gets sad? I ask because I'm planning on putting dirauth signing keys on a sloooooow HSM and would like to understand if I'd have to make format_networkstatus_vote() and networkstatus_compute_consensus() (and their callers) be "restartable", much like dirvote_act() is "restartable". Current measurements indicate a worst case figure of 6.5 seconds for a hashing and signing operation. This may even increase some over time. (It may also shrink considerably with some optimisations but this is where we are today.)

2 2

Notes from 12 April 2018 Simple Bandwidth Scanner Meeting
by Matt Traudt 13 Apr '18

13 Apr '18

See below for the pad notes. Next meeting is scheduled 19 April 2018 at 2200 UTC in #tor-meeting. (This one was held in #tor-dev, but we should use meetbot). ------------- Simple Bandwidth Scanner meeting 12 April 2018 #### Updates/Status messages #### pastly: What's on my plate? <- doesn't have to be all in your plate :P - Test coverage getting closer to 100% - Immediate future: switch to standard python logging module, which is quite good - Improving documentation - Checking results against torflow - Monitor CPU of sbws client/server - +1 on considering asyncio - See how chutney generates random strings - Run testnet authority - Reach out to current auths about running sbws/torflow and adding me as an auth juga: - open/close PRs/issues about things to improve in doc, refactor code, etc..., but not changing functionality - re. doc: - thought to update sbws spec (or create other) to doc differences with Torflow, not sure it's useful - i'd document further some of the classes/functions (as measure_relay) - code doc vs spec (see below) - find box to run other sbws, bwauth also in testnet? ## Topic: what is still missing for milestone 1? (aka 1st release, v1.0.0) - could we create all tickets needed to achive it? - maybe previous list is enough? Missing: - A consensus parameter stating the scaling factor - sbws config option to set fallback if no consensus param - `sbws generate` code to use the consensus param - https://stem.torproject.org/api/descriptor/networkstatus.html#stem.descript… - Correlation coefficient on comparision graphs ## Topic: comparing to torflow tah - Can we make the test sbws deployment a little bigger? - What else needs to be compared? teor: actually running it in a voting network, to check the feedback loop (if any) the scaling - Conclusions after comparing? - what we could think to change/improve after comparing? Graphs pastly can explain: - sbws vs moria, sorted by sbws: https://share.riseup.net/#-W_zqcv-08AX4SnOgTatUw - sorted by moria: https://share.riseup.net/#URXp6NccZHEhOPFJQcfO4w teor: the correlation seems good here If we're going to use these charts to compare, please compare two existing bwauths See: https://share.riseup.net/#lPGcIrgHp3ftnvTHUKqOKg (but ignore the sbws-scaled line, it's wrong wrong wrong) ## Topic: convincing people to run sbws juga: maybe something to do when 1st sbws release? pastly: yes, probalby. unless we need to convince testnet people <- ah, right i was thinking on the Tor net ## Topic: status of open sourcing sbws - No real update. Time is still passing. ## Topic: specifications torflow/BwAuthority: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/…, https://ohmygodel.com/publications/peerflow-popets2017.pdf has a section that also makes a nice summary sbws: https://github.com/pastly/simple-bw-scanner/blob/master/docs/source/specifi… (ask Pastly for access) bwscanner: no spec, but reading https://github.com/TheTorProject/bwscanner/blob/develop/bwscanner/circuit.p… it looks like a Torflow clone <- almost :) We need a spec for the v3bw file that tor reads (in torspec/dir-spec.txt) We need a spec for bwauth migration, including acceptance criteria for new bwauth implementations Scanners should have their own detailed design documents

3 5

Consensus-health single-relay data
by Tom Ritter 07 Apr '18

07 Apr '18

teor suggested the other day that it'd be really useful to be able to see the vote data for a single relay; since the _entire_ detailed page is huge and unwieldy. I've been pondering how I could support this without complicating the server, which results in a few constraints: a) I really don't want to create one file per-relay b) I really don't want to make any server-side logic, I'd like to keep it as static HTML and client-side javascript. (With the client-side javascript optional for most of the information.) I think I can make something work using AJAX and Range requests though. Before I fly forward with that though, I wanted to understand a bit more about the use cases where one wants to see per-relay vote data. The biggest question I have for you teor (and anyone else who wants to chime in) is how do you know what relay you want the information for? Do you know the fingerprint? If so, I could try adding a text box at the bottom of the index page saying "Supply a fingerprint, and I will populate vote data for this relay inline". -tom

3 5

[release] [protocol] Onionoo 5.2-1.12.0
by iwakeh 06 Apr '18

06 Apr '18

Hi there! Onionoo's protocol was extended and has a minor version jump to 5.2. Reminder:there will be a major *upcoming version* jump 6.0 after April 14, 2018. Take note of the field "next_major_version_scheduled" in documents. Download available at: https://dist.torproject.org/onionoo/5.2-1.12.0/ Protocol changes (also summarized in [0]): There is now a 'version_status' field, which is the status of the Tor software version of this relay based on the versions recommended by the directory authorities [1]. Operational changes: In order to always provide 'fresh' data Onionoo now fetches descriptors from both CollecTor instances and won't be affected as much anymore when one CollecTor instance is not reachable. For more please take a look at the change log [3]. The changes are already deployed on all onionoo.torproject.org instances. Please direct comments and questions to the metrics-team mailing list [3]. Cheers, iwakeh [0] https://metrics.torproject.org/onionoo.html#versions_5_2 [1] https://metrics.torproject.org/onionoo.html#details_relay_version_status https://metrics.torproject.org/onionoo.html#details_bridge_version_status [2] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md [3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

The case for Tor-over-QUIC
by Mike Perry 02 Apr '18

02 Apr '18

In Rome, I held a session about network protocol upgrades. My intent was to cover the switch to two guards, conflux, datagram transports, and QUIC. We ended up touching only briefly on everything but QUIC, but we went into enough depth on QUIC itself that it was a worthwhile and very productive session. Our notes are here: https://trac.torproject.org/projects/tor/wiki/org/meetings/2018Rome/Notes/F… I wanted to give a bit of background and some historical perspective about datagram transports for Tor, as well as explain those notes in long form, to get everybody on the same page about this idea. With the forthcoming IETF standard ratification of QUIC along with several solid reference implementations (canonical list: https://github.com/quicwg/base-drafts/wiki/Implementations) I believe we are close to the point where we can finally put together a plan (and a funding proposal) to undertake this work. Consider this mail a pre-proposal to temperature check and solicit early feedback about a Tor-over-QUIC deployment, before we invest the effort to deep dive into the framing, protocol, and implementation details that will be necessary for a full proposal. # Historical Background So, first: Tor has been considering UDP transports for a long time. There have been a series of research papers on the topic, dating back to Proposal 100 -- the very first Tor proposal: https://gitweb.torproject.org/torspec.git/tree/proposals/100-tor-spec-udp.t… That proposal only examines adding an unreliable DTLS link between relays, to enable transmission of unreliable (UDP) traffic end-to-end. Despite being marked as Dead, it is a good conceptual starting point, because it examines the relay-to-relay link crypto properties we need from a datagram layer and has some good comments at the bottom. However, adding the ability to simply carry UDP is not sufficient. What we have always really wanted is to transition the network to some form of end-to-end drop-based reliable congestion and flow control. There are numerous benefits from such a transition. Drop-signaled end-to-end congestion control means that instead of having to maintain arbitrary queues at relays (to make room for Tor's fixed SENDME window sizes for every single Tor circuit -- which as we have seen does not scale and is vulnerabe to DoS and OOM attacks), we can instead use a global fixed-length queue at each relay, and have each relay simply drop packets when this queue is full. This behavior mirrors how the internet works, and will not only put an upper bound on the end-to-end queuing latency of the Tor network, but it will also allow us to better ensure fairness, eliminate OOM conditions, and mitigate DoS and congestion attacks. QUIC's stream management and byte-acked teardown mechanisms will also eliminate a class of sidechannel issues in Tor where an adversary gets to send ignored data on invalid and/or recently closed streams: https://trac.torproject.org/projects/tor/ticket/25573 Furthermore, the switch to a secure datagram link protocol will eliminate RST spamming attacks, where an adversary spoofs RSTs to/from relay IP addresses to tear down our TLS links to gain information from the resulting circuit failures and retries across the network. Prior to the advent of QUIC, we studied many options for achieving end-to-end congestion control, but have found all of them to be lacking in one way or another. In 2011, Steven Murdoch did a literature review of these approaches: https://research.torproject.org/techreports/datagram-comparison-2011-11-07.… Section 4 of that review is the most useful thing to skim -- it explains why we found each of the end-to-end transport modes available at the time to be either insufficient for Tor's use, or encumbered by licensing or engineering incompatibilities. # QUIC Overview QUIC, however, is different. It basically serves as a complete drop-in replacement for Tor's n:1 streams-on-circuit connection model. QUIC has a connection layer, which corresponds to Tor's circuit concept, and a framing layer, which supports multiplexing for multiple reliable streams inside a single connection (corresponding to Tor's streams). Each QUIC connection provides an outer layer of drop-based congestion control. Unlike latency-based congestion control like bittorrent's utp, which was designed to compensate for poor queuing management at edge routers (as well as yield to TCP flows), QUIC's drop-based congestion control is designed to find the common bottleneck anywhere on the connection's path, and use packet drops from that queue to signal congestion and ensure fairness among connections at that bottleneck point. QUIC also provides optimizations to handle higher latency and drop-heavy links (SACK, SACK ranges, early retransmit, reordering tolerance, RTT estimates): https://quicwg.github.io/base-drafts/draft-ietf-quic-recovery.html Additionally, QUIC provides flow control at the connection level, as well as at the individual stream level. This is analogous to Tor's flow control model, but unlike Tor, QUIC implementations can adjust these window sizes automatically based on RTT measurements and congestion feedback: https://datatracker.ietf.org/doc/html/draft-ietf-quic-transport#section-3.4 https://datatracker.ietf.org/doc/html/draft-ietf-quic-transport#section-11 https://docs.google.com/document/d/1F2YfdDXKpy20WVKJueEf4abn_LVZHhMUMS5gX6P… It is perhaps no small coincidence that QUIC's architecture fits so perfectly into Tor's circuit model. While at an IETF meeting years ago, an attendee quietly whispered in my ear, "You know, we're designing IETF QUIC with Tor-like networks in mind." I always love me a good prospiracy (https://www.urbandictionary.com/define.php?term=prospiracy ;). # Using QUIC in Tor While QUIC does indeed solve a great many problems for Tor out of the box, the path to Tor-over-QUIC requires that we make a few design and engineering decisions specific to Tor. There are two research implementations that study using Tor-over-QUIC: Quux and QuTor. Quux was implemented by wrapping Chromium's C++ QUIC implementation in C bindings: https://www.benthamsgaze.org/2016/09/29/quux-a-quic-un-multiplexing-of-the-… https://www.benthamsgaze.org/wp-content/uploads/2016/09/393617_Alastair_Cla… QuTor is unfortunately paywalled at researchgate, so it is hard for me to tell what they did: https://www.researchgate.net/profile/Raik_Aissaoui/publication/292392094_QU… Unfortunately, the Quux implementation appears to use QUIC at a suboptimal position -- they replace Tor's TLS connections with QUIC, and use QUIC streams in place of Tor's circuit ID -- but only between relays. This means that it does not benefit from QUIC's end-to-end congestion control for the entire path of the circuit. Such a design will not solve the queuing and associated OOM problems at Tor relays, since relays would be unable to drop cells to signal backpressure to endpoints. Drops will instead block every circuit on a connection between relays, and even then, due to end-to-end reliability, relays will still need to queue without bound, subject to Tor's current (and inadequate) flow control. The optimal design is to use QUIC end-to-end. (Based on the researchgate abstract, it looks like this might be what QuTor did, but it is hard to tell.) Instead of replacing TLS with QUIC, we should use an unreliable link encryption protocol between relays (such as DTLS), and replace Tor's circuits with QUIC connections. In this design, Tor applications connect to the local SOCKS port using TCP, and the tor client converts it to a QUIC stream on a QUIC connection-circuit that runs all the way to the exit or onion service, which then will convert the stream back to TCP. The exit/service keeps buffering to a minimum by using QUIC pushback on the Tor side, and TCP pushback on the server side. While IETF QUIC provides its own crypto layer based on TLS 1.3, we won't really need this encryption. Instead, we will layer QUIC inside of Tor's layered onion encryption, with an ntor handshake to each hop in the path. # QUIC Deployment Considerations Past the above high-level decision of how to use QUIC, a handful of deployment considerations remain. ## Implementation As mentioned previously, there are a growing number of QUIC implementations out there, in several languages, with varying degrees of IETF compatibility: https://github.com/quicwg/base-drafts/wiki/Implementations For our purposes, the most interesting ones are ngtcp2 (native C implementation, BoringSSL, MIT licensed), picoquic (C, MIT license), and mozquic (C++ with C bindings, NSS, MPL licensed). The QUIC in Chromium is not IETF compliant, and is C++ with no C bindings. Sadly Rust implementations of QUIC are not complete. Google does also maintain a Go QUIC, which is complete. This suggests that experimental designs using one of the golang Tor implementations might be worthwhile. I'm told that the IETF hopes to finalize the QUIC standard within the year. ## Framing QUIC actually differentiates between connection-level packet sizes and frame sizes. If we want to preserve Tor's wire behavior, we will want to enforce a one-frame-per-packet policy from our QUIC library, and use QUIC's padding frames to make packet sizes uniform. ## Upgrade Path The upgrade path to QUIC will require some thorough thinking -- if all relays on a path don't yet support datagram links, should we embed QUIC inside an opaque RELAY cell type, or should wait for a flag day to switch over once sufficient relays have upgraded? Embedding QUIC in opaque RELAY cells seems like the natural choice, but will require additional engineering considerations. Additionally, a small and declining percentage of networks currently block all UDP other than DNS. At last measurement, the rate of QUIC failure was below 5%. However, like Google, we will likely want to retain the ability to fall back to TLS over TCP for these situations. ## Signaling Because we need to transition to unreliable relay links to get the most out of QUIC, we will need to figure out how to handle circuit setup -- there is currently no mechanism to ensure reliability for onionskins in Tor, since this is provided by TLS today. The simplest solution is to retain TLS between relays to transmit circuit extend cells out-of-band. Padding could be used to obscure timing information of this signaling. The alternative would be to implement some onionskin-level acking between relays, so that the datagram transport could be used for all traffic. However, since our deployment path will likely involve migration from TCP anyway, and since a small (and shrinking) percentage of networks simply block all non-DNS traffic, we will likely want to retain TCP connections for some time, making out-of-band signaling a natural first choice. ## Native UDP Support Unfortunately, during the standardization process, the IETF QUIC decided to drop support for unreliable datagram streams inside the QUIC connection, to simplify the layering of their implementation. However, because we will still need to layer QUIC inside of an onion layer, we can use that layer to carry either QUIC or UDP (at the expense of an additional field to differentiate this). # QUIC Research Questions A couple areas of open research into how QUIC will behave in a Tor deployment remain. It is my belief that while these areas are worth further study, they do not represent blockers to QUIC, as a naive deployment of QUIC will substantially outperform our current (lack of) congestion and poor flow control. ## End-to-end Congestion Control Study Unfortunately, it appears that the Quux QUIC paper studied QUIC at the wrong position - between relays, and the QuTor implementation is unclear. This means that it may still be an open question as to if QUIC's congestion control algorithms will behave optimally in a Tor-like network under heavy load. However, Google has conducted extensive internet-scale research into the congestion control algorithm, which very likely covers enough networks with Tor-like latency and drop characteristics. Because of this, I do not expect us to have to do a lot of tuning of their congestion control here, but it is worth investigating. ## Queue Management Fairness among flows in drop-based congestion control is heavily influenced by how the queue is managed, and particularly by the decision of which packet to drop when the queue is full. Modern favored algorithms involve probabilistic early dropping from the middle of the queue. The most popular approaches are the adaptive variants of RED and Blue: https://en.wikipedia.org/wiki/Random_early_detection https://en.wikipedia.org/wiki/Blue_(queue_management_algorithm) The simplest possible queue management design would be to set Tor's queues to 0, and allow the OS kernel or upstream router to apply its drop strategy. While this will be sufficient to ensure backpressure under congestion, it is not optimal for fairness. To enforce fairness, Tor would want to take information about QUIC connection IDs into account when dropping packets, similar to what is done with circuit ids by EWMA currently. This information will be unavailable to the kernel and upstream routers, due to the DTLS link encryption that will conceal the multiplexing of QUIC connections on connections to relays. Moreover, ensuring that sufficient queuing happens inside the Tor daemon (as opposed to the kernel) will require an aggressive socket-reading strategy. Tor's single-threaded nature will make this difficult -- to ensure that queuing happens inside the Tor daemon, packets must be continuously read from the kernel with minimal processing delay or delay from other events. Even in this case, optimality will still depend upon the bottleneck being Tor, and not in the kernel or in an upstream internet router. Thus, optimal queue management in Tor for QUIC remains an open research and engineering problem. (However, this is not really a deployment blocker. The deployment of QUIC on the internet has similar shortcomings -- the vast majority of internet routers also do not take QUIC connection ID into account when making UDP drop decisions, and yet the protocol still outperforms TCP in link utilization and drop recovery). ## Anonymity Since a Tor with bounded-length queues will have more predictable latency characteristics, a faster Tor based on QUIC may be more vulnerable to inferences about location that can be made by connection latency. Particularly, middle nodes may be able to tell how far the client is from the guard by measuring RTT. One solution to this may be adding delay in cases where RTT could be inferred. Another solution is to have two middle nodes for four hop paths, so that it is no longer clear which middle node you are in a QUIC circuit. There may be other attacks that come out of using end-to-end congestion control, but I am personally having a hard time coming up with cases where such side channels and resource starvation conditions could be any worse than those that already exist, or are unique to the OOM and DoS conditions currently possible due to Tor's lack of congestion control. -- Mike Perry

3 4

Is there strictly a one-to-one BW scanner to BW auth relationship?
by Rob Jansen 02 Apr '18

02 Apr '18

Hi, I understand that the current bandwidth measurement system is far from ideal, but I am wondering about how the current system works. Does each bandwidth authority also run a bandwidth scanner? Or is it possible that the results from a bandwidth scanner is supplied to multiple authorities? Thanks! Rob

4 6

Network team date change (this week only): 3 April, 1700 UTC
by Nick Mathewson 30 Mar '18

30 Mar '18

Hi! Lots of people are away for Easter Monday (it's a holiday in some countries!) so we're going to have our usual network-team meeting on Tuesday instead of Monday, for this week only. The time will be 1700 UTC as usual. Also as usual, you don't need to be a network-team member to attend! The meetings take place on the #tor-meeting IRC channel on OFTC. best wishes, -- Nick

1 0

Scaling bandwidth scanner results
by Matt Traudt 29 Mar '18

29 Mar '18

I've made some good progress on a bare bones, doesn't-do-more-than-it- has-to bandwidth scanner. It can generate output just like torflow[0]. We need to decide on how to scale results that come from different measurement systems. The simple, don't-make-it-harder-than-it-has-to-be idea is (quote [1], see [2]): > Express each weight as a proportion of the total, and multiply by some agreed total (e.g. for the current network it would have to be the total of the consensus weight, but within some limited range to avoid unbounded growth). So we need to: 1. Decide on a large total. I suggest 50 million to start the conversation (bike shedding) based on that being close to the current total consensus weight so relay operators won't see a large (though inconsequential) change. 2. Have all the torflow operators switch to this new method. Ouch. I wouldn't mind being told I'm wrong about this step being necessary. 3. Find all the places that hint at consensus weight being directly comparable to bandwidth (such as [3]) and change the wording. Matt [0]: https://paste.debian.net/1015409/ [1]: https://trac.torproject.org/projects/tor/wiki/org/meetings/2018Rom e/Notes/BandwidthAuthorityRequirements#Scaling [2]: https://trac.torproject.org/projects/tor/ticket/25459 [3]: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2290

3 2

Connections failed to default obfs4 bridges
by Rob Jansen 28 Mar '18

28 Mar '18

Hi, In a recent connectivity test to the default obfs4 bridges [0], we found that we are unable to connect to 10 or so of them (from open networks, i.e., no local filtering). Is this a feature, like some of them only respond to users in certain parts of the world? Or is this a bug, like the default list of bridges refers to old bridges that are no longer available? Or am I misunderstanding functionality here? Thanks! Rob [0] https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/Bundle-D…

3 3

Setting NumEntryGuards=2
by Mike Perry 28 Mar '18

28 Mar '18

Back in 2014, Tor moved from three guard nodes to one guard node: https://blog.torproject.org/improving-tors-anonymity-changing-guard-paramet… https://trac.torproject.org/projects/tor/ticket/12206 We made this change primarily to limit points of observability of entry into the Tor network for clients and onion services, as well as to reduce the ability of an adversary to track clients as they move from one internet connection to another by their choice of guards. At the time, I was in favor of two entry guards but did not have a strong preference, and we ended up choosing one guard. After seeing various consequences of using only one entry guard, I think a much stronger case can now be made for bumping back up to two. Roger suggested that I enumerate the pros and cons of this increase on this mailing list, so we can discuss and consider this switch. So here is my attempt at that list. Let's start with a more in-depth recap of the one-guard arguments, along with some recent observations that change things. Arguments for staying with just one guard: 1. One guard means less observability. As Roger put it in the above blog post: "I think the analysis of the network-level adversary in Aaron's paper is the strongest argument for restricting the variety of Internet paths that traffic takes between the Tor client and the Tor network." http://freehaven.net/anonbib/#ccs2013-usersrouted Unfortunately, we have since learned that Tor's path selection has the effect of giving the adversary the ability to generate at least one additional observation path. We first became aware of this in https://trac.torproject.org/projects/tor/ticket/14917, where the change to one guard allowed an adversary to discover your guard by choosing it as a rendezvous point and observing the circuit failure. After the fix for #14917, the onion service will build a connection to a second guard that it keeps in reserve. By using this attack (as well as a similar but more involved attack with unique exit policies and carefully chosen /16 exit node subnets), the adversary can force clients to be observed over two paths whenever they like. So while we may get benefit for moving from three guards to two guards, we don't get much (or any) benefit from moving to two guards to one guard against an active adversary that either connects to onion services, or serves content to clients and runs exits. 2. Guard fingerprintability is lower with one guard An adversary who is watching netflow connection records for an entire area is able to track users as they move from internet connection to internet connection through the degree of uniqueness of their guard choice. There is much less information in two guards than three, but still significantly more than with one guard: https://trac.torproject.org/projects/tor/ticket/9273#comment:3 But, even with one guard, if there are not very many Tor users in your area, you still may be trackable. "Guard bucket" designs are discussed on the blog post and in related tickets, but they are complicated and involve tricky tradeoffs (see https://trac.torproject.org/projects/tor/ticket/9273#comment:4) The best solution that I see to this is to make Tor maintain separate guard choices depending on the current SSID, BSSID, or default gateway router MAC from ARP. The default gateway ARP MAC is probably easiest for us to implement cross-platform and stable across wifi to ethernet. Arguments in favor of switching to two entry guards: 1. One guard allows course-grained netflow confirmation attacks The counterargument based on #14917 above also has an additional wrinkle: an adversary watching a suspect list of clients can easily observe the "temporarily use a second guard" activity using just connection-level ISP/AS netflow logs. To a large-scale netflow adversary, the use of a second guard only when the guard is chosen as the RP confirms not only guard choice, but the IP address of the onion service itself. Again, similar attacks may be possible against exit activity via cleverly crafted exit policies and /16 subnet choice. (Aside: if we allow routers to join any family they like without reciprocation, this attack becomes worse.) Having a dedicated second guard that is always in use will prevent the creation and exclusive use of a new TLS connection just for this request. Instead, the two TLS connections will both remain open and in regular use as long as the client is active. 2. More than one guard allows us to deploy conflux. As mentioned in the blog post, moving to one guard prevents us from deploying conflux -- a research design that improves Tor performance by combining two or more circuits together at an exit node or rendezvous point and load balancing between them: https://www.cypherpunks.ca/~iang/pubs/conflux-pets.pdf Conflux works by giving circuits a 256bit cookie that is send to the exit/RP, and circuits that are then built to the same exit/RP with the same cookie can then be fused together. Conflux was originally studied exclusively for performance, but the technique has other uses as well. If our conflux implementation includes packet acking, then circuits can still survive the loss of one guard node due to DoS, OOM, or churn because the second half of the path will remain open and usable. Furthermore, if exits remember this cookie for a short period of time after the last circuit is closed, the technique can be used to protect against DoS/OOM/guard downtime conditions that take down both guard nodes or destroy many circuits to confirm both guard node choices. In these cases, circuits could be rebuilt along an alternate path and resumed without end-to-end circuit connectivity loss. This same technique will also make things like ephemeral bridges (ie Snowflake/Flashproxy) more usable, because bridge uptime will no longer be so crucial to usability. It will also improve mobile usability by allowing us to resume connections after mobile Tor apps are briefly suspended, or if the user switches between cell and wifi networks. Furthermore, I believe that conflux will also be useful against traffic analysis and congestion attacks. Since the load balancing is dynamic and hard to predict by an external observer, traffic correlation and website traffic fingerprinting attacks will become harder, because the adversary can no longer be sure what percentage of the traffic they have seen (depending on their position and other potential concurrent activity). Similarly, it should also help dampen congestion attacks, since traffic will automatically shift away from a congested guard. So those are the major two arguments on each side. There are some additional minor arguments in favor of two guards (Circuit build timeout works better with more guards because it can avoid the slower guard, and a second guard will also prevent information leaks like https://trac.torproject.org/projects/tor/ticket/24487) but for brevity I decided to mention those only in passing. In terms of deployment, the switch to two guards is an extremely simple one. We should only need to set the NumEntryGuards=2 consensus parameter, and all recent clients will switch over immediately. What do we think? -- Mike Perry

6 13

Is OutboundBindAddress respected during ORPort IP auto detection?
by nusenu 25 Mar '18

25 Mar '18

Hi, I got an interesting bugreport for ansible-relayor [1], that leads me to the following question: Is OutboundBindAddress used during ORPort IP auto detection? Imagine the following setup: Relay with two public IPs: 1.1.1.1 2.2.2.2 Two tor instances (one per IP): 1.1.1.1:9000 2.2.2.2:9000 torrc for 2.2.2.2:9000 contains: [...] OutboundBindAddress 2.2.2.2 ORPort 2.2.2.2:9000 SyslogIdentityTag 2.2.2.2_9000 [...] when starting 2.2.2.2_9000 instance the log contains: Tor-2.2.2.2_9000[586]: Your server (1.1.1.1:9000) has not managed to confirm that its ORPort is reachable. Relays do not publish descriptors until their ORPort and DirPort are reachable. Please check your firewalls, ports, address, /etc/hosts file, etc. Note: Reachability tests appear to try to contact the first IP for the second instance. No NAT is involved. The solution was to add "Address 2.2.2.2", but that was not necessary previously(?) My assumption here: During auto detection the OutboundBindAddress configuration directive is not relevant. Is that the case? Or why does tor auto-detect IP 1.1.1.1 for instance on 2.2.2.2 even though OutboundBindAddress is used? thanks, nusenu this is on tor: 0.3.2.10 [1] https://github.com/nusenu/ansible-relayor/issues/153 -- https://mastodon.social/@nusenu twitter: @nusenu_

2 2

Bandwidth Scanner
by Aaron Gibson 19 Mar '18

19 Mar '18

Hi all, per reading [1], I see that there was significant discussion at tor-dev around the topic of bandwidth measurements. It was unclear to me what direction has been agreed on, or if anyone is leading this effort, so I wanted to add that: I have been working with Donncha, Juga, Teor, and Linus to find and fix issues in the BwScanner codebase. As George K. indicates, I am time limited and interrupt driven so I've made little progress in resolving the "last few bugs" [2] I do want to continue resolving these issues, and any help testing, reviewing pull requests or contributing patches would be awesome. I think it would also be very helpful for TPI to support this project with project management resources to keep things on track. BwScanner is written in python using twisted, uses travis-ci for unit tests and integration testing with chutney. Development has been using the github tools to facilitate code reviews and testing, and has OK testing coverage and documentation. --Aaron [1] https://trac.torproject.org/projects/tor/wiki/org/meetings/2018Rome/Notes/B… [2] https://github.com/thetorproject/bwscanner/issues

1 0

[release] [protocol] Onionoo 5.1-1.11.0
by iwakeh 15 Mar '18

15 Mar '18

Hi there! Onionoo's protocol was extended and has a minor version jump to 5.1. In addition, this release *prepares* a major *upcoming version* jump 6.0 after April 14, 2018. Take note of the field "next_major_version_scheduled" in documents. Download available at: https://dist.torproject.org/onionoo/5.1-1.11.0/ Protocol changes (also summarized in [0]): The field "n" is now always added in summary docs using "Unnamed" for relays/bridges that don't supply a nickname [1]. Relays are now always part of their own "effective_family" [2]. For more please the change log [3] The changes are already deployed on all onionoo.torproject.org instances. Please direct comments and questions to the metrics-team mailing list [4]. Cheers, iwakeh [0] https://metrics.torproject.org/onionoo.html#versions_5_1 [1] https://metrics.torproject.org/onionoo.html#summary_relay_n https://metrics.torproject.org/onionoo.html#summary_bridge_n [2] https://metrics.torproject.org/onionoo.html#details_relay_effective_family [3] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md [4] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Re: [tor-dev] (no subject)
by Sambuddho Chakravarty 15 Mar '18

15 Mar '18

Hi All We (myself and my student, cc'ed here) have some questions regarding Tor relay bandwidth. We created a small testbed (private Tor network) involving relays, clients and servers over virtual machines. The client uses the relays to create two hop circtuits to communicate to the server. The link bandwidths between the virtual machines is otherwise ~ 10 Gbps (tested via end-to-end measurement tools like iperf and via downloading large files of sizes ~2 GB through https downloads). When not using Tor, the client achieve a download rate of approximately 8 Gbit/s while downloading a file of approximately 2 Gbytes. The same process, when repeated over Tor circuit (through relays of the private Tor network, that used virtual machines), p*rovides a maxmium download rate **of about 700 Mbit/s for the client (while the CPU utilization was over 80%).* This is much less compared to the expected throughput of 8Gbps achieved when client communicates directly to the server (without using Tor circuits). We tinkered around with the Tor config and discovered that when changed the config fields of BandwidthBurst and BandwidthRate to 100 MBytes, we achieved end-to-end throughput of about 100 Mbit/s (while the CPU Utilization was less than 20%) (while the underlying network link capacity is 1 Gbit/s). The moment the BandwidthBurst and BandwidthRate values are increased to 1 GB (which should effectively be 8 Gbit/s), the achieved throughput is about 650-700 Mbps (which is still 8 times slower than expected). Further, removing all bandwidth restrictions, the inter VM bandwidth (for the relays in the private Tor network) is around 8 Gbit/s (measured both via Iperf and through downloading a file > 2 GBytes using encrypted HTTP traffic). While this is as per the expected behavior of TCP, to utilize all the available bandwidth, the same does not happen for Tor which continues to ramp upto a maximum of 700 Mbit/s We are confused as to why Tor's link bandwidth utilization is not higher (Even on a private Tor network), even when using a high capacity VMs in the private Tor network -- each one having about 4 cores and 8 GBytes of RAM. I understand after speaking from some Tor developers and maintainers that Tor does not utilize all the available CPU cores. Our VMs however do not have any other process running either. We run the Tor processes over Debian servers with little no other programs. More strangely, while download via HTTPS achieves a very large fraction of the link capacity (>80%), Tor traffic happens to achieve a throughput of around 700 Mbit/s (which is around 90% lower than the max link bandwidth). We tried the same with 2-hop Tor circuits, hoping to see some improvement. But to no avail. We continue to observe the same poor bandwidth utilization. It would be great if you shed some light as to where we may be going wrong. Regards Sambuddho

2 1

Tor: unspecified is the default Core Tor milestone
by teor 15 Mar '18

15 Mar '18

Hi, The network team maintains Core Tor, the network daemon that runs the tor network. We have planned our roadmap for the next 6 months. We only chose essential and sponsored tasks, because we want to finish on time. We moved all the other tickets out of the 0.3.4 and 0.3.5 milestones. If you want to add a ticket to Core Tor, please add it to the "Tor: unspecified" milestone. We will discuss essential tickets each week. We will only add tickets to the release milestone if we agree we have time to complete them. I have moved the "Tor: unspecified" milestone to the top of the milestones list, to make it easier to find. T

1 0

[Win32] crash in test/bench.exe
by Gisle Vanem 14 Mar '18

14 Mar '18

There is a crash in bench.exe. Running 'cdb -c g bench.exe': ntdll!RtlpWaitOnCriticalSection+0x6b: 77a9cfd6 ff4014 inc dword ptr [eax+14h] ds:002b:00000014=???????? ChildEBP RetAddr 0137f750 77aba38a ntdll!RtlpWaitOnCriticalSection+0x6b 0137f770 77aba259 ntdll!RtlpEnterCriticalSectionContended+0x12a *** WARNING: Unable to verify checksum for bench.exe 0137f77c 002150f7 ntdll!RtlEnterCriticalSection+0x49 0137f784 0023136c bench!tor_mutex_acquire(struct tor_mutex_t * m = 0x0039559c)+0x37 0137f794 000fc85f bench!atomic_counter_exchange(struct atomic_counter_t * counter = 0x0039559c, unsigned int newval = 6) +0xc (Inline) -------- bench!set_protocol_warning_severity_level+0xb 0137f7bc 0010563f bench!options_act(struct or_options_t * old_options = 0x00000000)+0x2af 0137f7d4 000f385b bench!set_options(struct or_options_t * new_val = 0x0f335730, char ** msg = 0x0137f80c)+0x6f 0137f80c 002f3707 bench!main(int argc = 0n1, char ** argv = 0x036b1e38)+0x24b ... Seems bench.c uses some mutex which is not initialised with 'tor_mutex_init()'. I fail to see which that should be. FYI. '&m->mutex' is 0 (= eax). But adding: tor_assert(&m->mutex); tor_assert(&m->mutex.OwningThread); to 'tor_mutex_acquire()' didn't help reveal the problem. -- --gv

2 3

3.2.10 compiler warnings
by grarpamp 11 Mar '18

11 Mar '18

Test from an older system before it was updated, unlikely to be prevalent, so don't bother fixing unless obvious. src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/test/vote_descriptors.inc:93: warning: string length '4135' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_hs_descriptor.inc:224: warning: string length '14104' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_microdesc.c:648: warning: string length '5844' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_descriptors.inc:305: warning: string length '10571' is greater than the length '4095' ISO C99 compilers are required to support

2 2

Proposal Waterfilling
by Florentin Rochet 09 Mar '18

09 Mar '18

Hello everyone, In order that our paper does not fall into the list of "yet another seems-to-be-cool-feature is that never going to be discussed because researchers moved on another topic", here is an attached proposal in which we summarize our work published in the last PETS event and how it can be implemented. And, if you find this interesting, I would be glad to submit a patch :) Any kind of feedbacks is more than welcome! Cheers! Florentin

8 29

Proposal 290: Continuously update consensus methods
by Nick Mathewson 03 Mar '18

03 Mar '18

Filename: 290-deprecate-consensus-methods.txt Title: Continuously update consensus methods Author: Nick Mathewson Created: 2018-02-21 Status: Open 1. Background Directory authorities use the "consensus method" mechanism to achieve forward compatibility during voting. When each authority publishes its vote, it includes a list of numbered consensus methods that it supports. Each authority chooses to calculate the consensus according to the highest consensus method it knows supported by more than 2/3 of the voting authorities. So long as all the authorities have a method in common, they will all reach the same consensus. Consensus method 1 was first introduced in the Tor 0.2.0 series around 2008. But by 2012, we realized that we had a problem: we were stuck documenting and supporting old consensus methods indefinitely. With proposal 215, we deprecated and removed support for all consensus methods before method 13. That was good as far as it went, but it didn't solve the problem going forward: the latest consensus methods is now 28. This proposal describes a policy for removing older consensus methods going forward, so we won't have to keep supporting them forever. 2. Proposal I propose that from time to time, old consensus methods should be deprecated. Specifically, I propose that we deprecate all methods older than the highest method supported in first stable release of the oldest LTS (long-term support) release series. For example, the current oldest LTS series is 0.2.5.x. The first stable release in that series was 0.2.5.10. The highest consensus method listed by 0.2.5.10 is 18. Therefore, we should currently consider ourselves free to deprecate all methods before 18. Once 0.2.5.x is deprecated, 0.2.9.x will become the oldest LTS series. The first stable release in that series was 0.2.9.8. The highest consensus method listed by 0.2.9.8 is 25. Therefore, once 0.2.5.x is deprecated (in May 2018), we may deprecate all methods before 25. When a consensus method is deprecated, it should no longer be listed or implemented by the latest Tor releases. (It's okay for older authorities to keep advertising it.) Most consensus methods add a feature that is used in "method M or later". Deprecating method M-1 means that the feature is used in all supported consensus methods. Therefore, we can remove any code that makes the feature conditional on a consensus method, and any code for previous implementations of the feature. Some consensus methods remove a feature that was used up to method N. Deprecating method M means that the feature is no longer used by any supported consensus methods. Therefore, we can remove any code that implements the feature. A. Acknowledgments Thanks to isis and teor for the discussion that led to this proposal. I believe that teor first suggested the policy described in section 2 above. B. Client and relay compatibility notes Dear reader: you may be worrying that this proposal will cause old clients or relays to stop working prematurely. That is not the case. Consensus methods determine how the authorities behave, but they do not represent backward-incompatible changes in how they generate their consensuses.

3 2

New link on Tor Metrics: download data as CSV
by Karsten Loesing 28 Feb '18

28 Feb '18

Hi tor-dev! Today we added a small, but potentially quite useful feature to Tor Metrics. The data that is displayed in a graph can now be downloaded as CSV file by clicking on "download data as CSV". This is different from before, where we provided the larger CSV file that a graph is based on. Getting the relevant data out of that would still require some data wrangling. This has become a lot easier now. Want to try it out? Go to any graph on Tor Metrics, for example: https://metrics.torproject.org/relays-ipv6.html?start=2013-02-28&end=2018-0… Click on "download data as CSV" and you'll get just the values that are plotted in the graph. Want to learn more and maybe help us make this feature more useful? There's more about it in tickets #25282 (closed), #25383, and #25387. All the best, Karsten

1 0

DocTor Check for <3 bw auth votes available
by nusenu 28 Feb '18

28 Feb '18

Hi Damian, would be great to have a check in DocTor that sends out an email for "there are less than 3 bw auth voting auths available" thanks, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

3 6

[release] CollecTor 1.5.0, metrics-lib 2.2.0
by iwakeh 26 Feb '18

26 Feb '18

Hi there! two new releases are available: https://dist.torproject.org/collector/1.5.0/ https://dist.torproject.org/metrics-lib/2.2.0/ These releases will be deployed on tp.o instances after several weeks of testing, and provide Tor's web server access logs that are currently available on [0] through CollecTor. All possibly privacy impacting data is removed from the logs prior publishing. What exactly is done to ensure privacy can be found in the specification doc [1]. The additions to metrics-lib's API can be read in its Javadoc pages [2]. Other details can be found in the respective change logs [3] and [4]. Please direct comments and questions to the metrics-team mailing list [5]. Cheers, iwakeh [0] https://webstats.torproject.org/ [1] https://metrics.torproject.org/web-server-logs.html [2] https://metrics.torproject.org/metrics-lib/ https://metrics.torproject.org/metrics-lib/index.html?org/torproject/descri… https://metrics.torproject.org/metrics-lib/index.html?org/torproject/descri… [3] https://gitweb.torproject.org/collector.git/tree/CHANGELOG.md?h=collector-1… [4] https://gitweb.torproject.org/metrics-lib.git/tree/CHANGELOG.md?h=metrics-l… [5] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Creating custom Tor Browser settings profile
by procmem 26 Feb '18

26 Feb '18

Hi I was wondering if Tor Broswer supports environment variables to customize about:config settings. The idea is to ship an explicitly labelled, custom profile one day to allow users seamless access to localhost daemons (like zeronet) by changing network.proxy.no_proxies_on to 0 - so they don't have to jump thru hoops or install foreign code like foxyproxy. At the same time, a separate browser profile affords regular users protection by keeping these fingerprinatble, non default settings contained and only used when the user needs them. Any way to do this?

2 1

txtorcon 0.20.0
by meejah 23 Feb '18

23 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm very happy to release txtorcon 0.20.0. This includes a few minor new features and an important bug fix if you're using a Tor new enough to have all the *PortLines changes (anything after 0.3.0.6; see Tor issue #20956). Full list of changes: * doc fixes (from hotelzululima) * Issue 246: fix endpoints so .connect on them works properly more than once (from Brian Warner) * allow a CertificateOptions to be passed as tls= to endpoints * PR 252: add method txtorcon.Tor.is_ready * PR 252: add method txtorcon.Tor.become_ready * PR 253: fix handling of certain defaults (*PortLines and friends) * fix last router (usually) missing with (new) `MicroDescriptorParser` * use OnionOO via Onion service tgel7v4rpcllsrk2.onion for txtorcon.Router.get_onionoo_details * fix parsing of Router started-times * Issue 255: removed routers always deleted following NEWCONSENSUS * Issue 279: remember proxy endpoint if it was Deferred You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.20.0 https://github.com/meejah/txtorcon/releases/tag/v0.20.0 Releases are also available from the hidden service: http://timaq4ygg2iegci7.onion/txtorcon-0.20.0.tar.gz http://timaq4ygg2iegci7.onion/txtorcon-0.20.0.tar.gz.asc You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the files downloaded: cat <<EOF | sha256sum --check dc80cb76b3ddacef6d671c0a088cb1a45274c0858554c32ce55d0f41421c740e txtorcon-0.20.0.tar.gz a957b3dc10f0b2b882ef6ad46d2932fa7731d5eeee6d52ba9d736d2bc4f1385f txtorcon-0.20.0-py2.py3-none-any.whl EOF thanks, meejah -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJaj2AIAAoJEMJgKAMSgGmnnGUIANXkNIF8c98nwCa4Q9Xws3Hv 9zAJSWsDst/58lHka6bE5d1JWYz78fTubTC2ofOFH0MOkjeSyEtgTO58xlKY9wTC w7ohePwcV+md1wPOpciHVZMBVhqUhKGqNhMYehTXjcCEwppgXhiINgGFLaF0XSUM aJzY7laEvg2E/1p+gH0e4tkJuSVKVOK+5MWBali8YlOZY28R6w/X0Q0n8su1ntl2 bKsxpTrv+p8J+K7WvpeaXIuZnC/fRwVFZ0vtwzKtrTKimAE60yPfOi5hLUyW2PGx FkpH6B0bgl5FMiY+Y0PV8rP2DXikCibpwsipBh4AHpTYGSdMkSogjiRupkJxeeQ= =GsVt -----END PGP SIGNATURE-----

1 0