- tor-dev - lists.torproject.org

Scaling bandwidth scanner results
by Matt Traudt 29 Mar '18

29 Mar '18

I've made some good progress on a bare bones, doesn't-do-more-than-it- has-to bandwidth scanner. It can generate output just like torflow[0]. We need to decide on how to scale results that come from different measurement systems. The simple, don't-make-it-harder-than-it-has-to-be idea is (quote [1], see [2]): > Express each weight as a proportion of the total, and multiply by some agreed total (e.g. for the current network it would have to be the total of the consensus weight, but within some limited range to avoid unbounded growth). So we need to: 1. Decide on a large total. I suggest 50 million to start the conversation (bike shedding) based on that being close to the current total consensus weight so relay operators won't see a large (though inconsequential) change. 2. Have all the torflow operators switch to this new method. Ouch. I wouldn't mind being told I'm wrong about this step being necessary. 3. Find all the places that hint at consensus weight being directly comparable to bandwidth (such as [3]) and change the wording. Matt [0]: https://paste.debian.net/1015409/ [1]: https://trac.torproject.org/projects/tor/wiki/org/meetings/2018Rom e/Notes/BandwidthAuthorityRequirements#Scaling [2]: https://trac.torproject.org/projects/tor/ticket/25459 [3]: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2290

3 2

Connections failed to default obfs4 bridges
by Rob Jansen 28 Mar '18

28 Mar '18

Hi, In a recent connectivity test to the default obfs4 bridges [0], we found that we are unable to connect to 10 or so of them (from open networks, i.e., no local filtering). Is this a feature, like some of them only respond to users in certain parts of the world? Or is this a bug, like the default list of bridges refers to old bridges that are no longer available? Or am I misunderstanding functionality here? Thanks! Rob [0] https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/Bundle-D…

3 3

Setting NumEntryGuards=2
by Mike Perry 28 Mar '18

28 Mar '18

Back in 2014, Tor moved from three guard nodes to one guard node: https://blog.torproject.org/improving-tors-anonymity-changing-guard-paramet… https://trac.torproject.org/projects/tor/ticket/12206 We made this change primarily to limit points of observability of entry into the Tor network for clients and onion services, as well as to reduce the ability of an adversary to track clients as they move from one internet connection to another by their choice of guards. At the time, I was in favor of two entry guards but did not have a strong preference, and we ended up choosing one guard. After seeing various consequences of using only one entry guard, I think a much stronger case can now be made for bumping back up to two. Roger suggested that I enumerate the pros and cons of this increase on this mailing list, so we can discuss and consider this switch. So here is my attempt at that list. Let's start with a more in-depth recap of the one-guard arguments, along with some recent observations that change things. Arguments for staying with just one guard: 1. One guard means less observability. As Roger put it in the above blog post: "I think the analysis of the network-level adversary in Aaron's paper is the strongest argument for restricting the variety of Internet paths that traffic takes between the Tor client and the Tor network." http://freehaven.net/anonbib/#ccs2013-usersrouted Unfortunately, we have since learned that Tor's path selection has the effect of giving the adversary the ability to generate at least one additional observation path. We first became aware of this in https://trac.torproject.org/projects/tor/ticket/14917, where the change to one guard allowed an adversary to discover your guard by choosing it as a rendezvous point and observing the circuit failure. After the fix for #14917, the onion service will build a connection to a second guard that it keeps in reserve. By using this attack (as well as a similar but more involved attack with unique exit policies and carefully chosen /16 exit node subnets), the adversary can force clients to be observed over two paths whenever they like. So while we may get benefit for moving from three guards to two guards, we don't get much (or any) benefit from moving to two guards to one guard against an active adversary that either connects to onion services, or serves content to clients and runs exits. 2. Guard fingerprintability is lower with one guard An adversary who is watching netflow connection records for an entire area is able to track users as they move from internet connection to internet connection through the degree of uniqueness of their guard choice. There is much less information in two guards than three, but still significantly more than with one guard: https://trac.torproject.org/projects/tor/ticket/9273#comment:3 But, even with one guard, if there are not very many Tor users in your area, you still may be trackable. "Guard bucket" designs are discussed on the blog post and in related tickets, but they are complicated and involve tricky tradeoffs (see https://trac.torproject.org/projects/tor/ticket/9273#comment:4) The best solution that I see to this is to make Tor maintain separate guard choices depending on the current SSID, BSSID, or default gateway router MAC from ARP. The default gateway ARP MAC is probably easiest for us to implement cross-platform and stable across wifi to ethernet. Arguments in favor of switching to two entry guards: 1. One guard allows course-grained netflow confirmation attacks The counterargument based on #14917 above also has an additional wrinkle: an adversary watching a suspect list of clients can easily observe the "temporarily use a second guard" activity using just connection-level ISP/AS netflow logs. To a large-scale netflow adversary, the use of a second guard only when the guard is chosen as the RP confirms not only guard choice, but the IP address of the onion service itself. Again, similar attacks may be possible against exit activity via cleverly crafted exit policies and /16 subnet choice. (Aside: if we allow routers to join any family they like without reciprocation, this attack becomes worse.) Having a dedicated second guard that is always in use will prevent the creation and exclusive use of a new TLS connection just for this request. Instead, the two TLS connections will both remain open and in regular use as long as the client is active. 2. More than one guard allows us to deploy conflux. As mentioned in the blog post, moving to one guard prevents us from deploying conflux -- a research design that improves Tor performance by combining two or more circuits together at an exit node or rendezvous point and load balancing between them: https://www.cypherpunks.ca/~iang/pubs/conflux-pets.pdf Conflux works by giving circuits a 256bit cookie that is send to the exit/RP, and circuits that are then built to the same exit/RP with the same cookie can then be fused together. Conflux was originally studied exclusively for performance, but the technique has other uses as well. If our conflux implementation includes packet acking, then circuits can still survive the loss of one guard node due to DoS, OOM, or churn because the second half of the path will remain open and usable. Furthermore, if exits remember this cookie for a short period of time after the last circuit is closed, the technique can be used to protect against DoS/OOM/guard downtime conditions that take down both guard nodes or destroy many circuits to confirm both guard node choices. In these cases, circuits could be rebuilt along an alternate path and resumed without end-to-end circuit connectivity loss. This same technique will also make things like ephemeral bridges (ie Snowflake/Flashproxy) more usable, because bridge uptime will no longer be so crucial to usability. It will also improve mobile usability by allowing us to resume connections after mobile Tor apps are briefly suspended, or if the user switches between cell and wifi networks. Furthermore, I believe that conflux will also be useful against traffic analysis and congestion attacks. Since the load balancing is dynamic and hard to predict by an external observer, traffic correlation and website traffic fingerprinting attacks will become harder, because the adversary can no longer be sure what percentage of the traffic they have seen (depending on their position and other potential concurrent activity). Similarly, it should also help dampen congestion attacks, since traffic will automatically shift away from a congested guard. So those are the major two arguments on each side. There are some additional minor arguments in favor of two guards (Circuit build timeout works better with more guards because it can avoid the slower guard, and a second guard will also prevent information leaks like https://trac.torproject.org/projects/tor/ticket/24487) but for brevity I decided to mention those only in passing. In terms of deployment, the switch to two guards is an extremely simple one. We should only need to set the NumEntryGuards=2 consensus parameter, and all recent clients will switch over immediately. What do we think? -- Mike Perry

6 13

Is OutboundBindAddress respected during ORPort IP auto detection?
by nusenu 25 Mar '18

25 Mar '18

Hi, I got an interesting bugreport for ansible-relayor [1], that leads me to the following question: Is OutboundBindAddress used during ORPort IP auto detection? Imagine the following setup: Relay with two public IPs: 1.1.1.1 2.2.2.2 Two tor instances (one per IP): 1.1.1.1:9000 2.2.2.2:9000 torrc for 2.2.2.2:9000 contains: [...] OutboundBindAddress 2.2.2.2 ORPort 2.2.2.2:9000 SyslogIdentityTag 2.2.2.2_9000 [...] when starting 2.2.2.2_9000 instance the log contains: Tor-2.2.2.2_9000[586]: Your server (1.1.1.1:9000) has not managed to confirm that its ORPort is reachable. Relays do not publish descriptors until their ORPort and DirPort are reachable. Please check your firewalls, ports, address, /etc/hosts file, etc. Note: Reachability tests appear to try to contact the first IP for the second instance. No NAT is involved. The solution was to add "Address 2.2.2.2", but that was not necessary previously(?) My assumption here: During auto detection the OutboundBindAddress configuration directive is not relevant. Is that the case? Or why does tor auto-detect IP 1.1.1.1 for instance on 2.2.2.2 even though OutboundBindAddress is used? thanks, nusenu this is on tor: 0.3.2.10 [1] https://github.com/nusenu/ansible-relayor/issues/153 -- https://mastodon.social/@nusenu twitter: @nusenu_

2 2

Bandwidth Scanner
by Aaron Gibson 19 Mar '18

19 Mar '18

Hi all, per reading [1], I see that there was significant discussion at tor-dev around the topic of bandwidth measurements. It was unclear to me what direction has been agreed on, or if anyone is leading this effort, so I wanted to add that: I have been working with Donncha, Juga, Teor, and Linus to find and fix issues in the BwScanner codebase. As George K. indicates, I am time limited and interrupt driven so I've made little progress in resolving the "last few bugs" [2] I do want to continue resolving these issues, and any help testing, reviewing pull requests or contributing patches would be awesome. I think it would also be very helpful for TPI to support this project with project management resources to keep things on track. BwScanner is written in python using twisted, uses travis-ci for unit tests and integration testing with chutney. Development has been using the github tools to facilitate code reviews and testing, and has OK testing coverage and documentation. --Aaron [1] https://trac.torproject.org/projects/tor/wiki/org/meetings/2018Rome/Notes/B… [2] https://github.com/thetorproject/bwscanner/issues

1 0

[release] [protocol] Onionoo 5.1-1.11.0
by iwakeh 15 Mar '18

15 Mar '18

Hi there! Onionoo's protocol was extended and has a minor version jump to 5.1. In addition, this release *prepares* a major *upcoming version* jump 6.0 after April 14, 2018. Take note of the field "next_major_version_scheduled" in documents. Download available at: https://dist.torproject.org/onionoo/5.1-1.11.0/ Protocol changes (also summarized in [0]): The field "n" is now always added in summary docs using "Unnamed" for relays/bridges that don't supply a nickname [1]. Relays are now always part of their own "effective_family" [2]. For more please the change log [3] The changes are already deployed on all onionoo.torproject.org instances. Please direct comments and questions to the metrics-team mailing list [4]. Cheers, iwakeh [0] https://metrics.torproject.org/onionoo.html#versions_5_1 [1] https://metrics.torproject.org/onionoo.html#summary_relay_n https://metrics.torproject.org/onionoo.html#summary_bridge_n [2] https://metrics.torproject.org/onionoo.html#details_relay_effective_family [3] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md [4] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Re: [tor-dev] (no subject)
by Sambuddho Chakravarty 15 Mar '18

15 Mar '18

Hi All We (myself and my student, cc'ed here) have some questions regarding Tor relay bandwidth. We created a small testbed (private Tor network) involving relays, clients and servers over virtual machines. The client uses the relays to create two hop circtuits to communicate to the server. The link bandwidths between the virtual machines is otherwise ~ 10 Gbps (tested via end-to-end measurement tools like iperf and via downloading large files of sizes ~2 GB through https downloads). When not using Tor, the client achieve a download rate of approximately 8 Gbit/s while downloading a file of approximately 2 Gbytes. The same process, when repeated over Tor circuit (through relays of the private Tor network, that used virtual machines), p*rovides a maxmium download rate **of about 700 Mbit/s for the client (while the CPU utilization was over 80%).* This is much less compared to the expected throughput of 8Gbps achieved when client communicates directly to the server (without using Tor circuits). We tinkered around with the Tor config and discovered that when changed the config fields of BandwidthBurst and BandwidthRate to 100 MBytes, we achieved end-to-end throughput of about 100 Mbit/s (while the CPU Utilization was less than 20%) (while the underlying network link capacity is 1 Gbit/s). The moment the BandwidthBurst and BandwidthRate values are increased to 1 GB (which should effectively be 8 Gbit/s), the achieved throughput is about 650-700 Mbps (which is still 8 times slower than expected). Further, removing all bandwidth restrictions, the inter VM bandwidth (for the relays in the private Tor network) is around 8 Gbit/s (measured both via Iperf and through downloading a file > 2 GBytes using encrypted HTTP traffic). While this is as per the expected behavior of TCP, to utilize all the available bandwidth, the same does not happen for Tor which continues to ramp upto a maximum of 700 Mbit/s We are confused as to why Tor's link bandwidth utilization is not higher (Even on a private Tor network), even when using a high capacity VMs in the private Tor network -- each one having about 4 cores and 8 GBytes of RAM. I understand after speaking from some Tor developers and maintainers that Tor does not utilize all the available CPU cores. Our VMs however do not have any other process running either. We run the Tor processes over Debian servers with little no other programs. More strangely, while download via HTTPS achieves a very large fraction of the link capacity (>80%), Tor traffic happens to achieve a throughput of around 700 Mbit/s (which is around 90% lower than the max link bandwidth). We tried the same with 2-hop Tor circuits, hoping to see some improvement. But to no avail. We continue to observe the same poor bandwidth utilization. It would be great if you shed some light as to where we may be going wrong. Regards Sambuddho

2 1

Tor: unspecified is the default Core Tor milestone
by teor 15 Mar '18

15 Mar '18

Hi, The network team maintains Core Tor, the network daemon that runs the tor network. We have planned our roadmap for the next 6 months. We only chose essential and sponsored tasks, because we want to finish on time. We moved all the other tickets out of the 0.3.4 and 0.3.5 milestones. If you want to add a ticket to Core Tor, please add it to the "Tor: unspecified" milestone. We will discuss essential tickets each week. We will only add tickets to the release milestone if we agree we have time to complete them. I have moved the "Tor: unspecified" milestone to the top of the milestones list, to make it easier to find. T

1 0

[Win32] crash in test/bench.exe
by Gisle Vanem 14 Mar '18

14 Mar '18

There is a crash in bench.exe. Running 'cdb -c g bench.exe': ntdll!RtlpWaitOnCriticalSection+0x6b: 77a9cfd6 ff4014 inc dword ptr [eax+14h] ds:002b:00000014=???????? ChildEBP RetAddr 0137f750 77aba38a ntdll!RtlpWaitOnCriticalSection+0x6b 0137f770 77aba259 ntdll!RtlpEnterCriticalSectionContended+0x12a *** WARNING: Unable to verify checksum for bench.exe 0137f77c 002150f7 ntdll!RtlEnterCriticalSection+0x49 0137f784 0023136c bench!tor_mutex_acquire(struct tor_mutex_t * m = 0x0039559c)+0x37 0137f794 000fc85f bench!atomic_counter_exchange(struct atomic_counter_t * counter = 0x0039559c, unsigned int newval = 6) +0xc (Inline) -------- bench!set_protocol_warning_severity_level+0xb 0137f7bc 0010563f bench!options_act(struct or_options_t * old_options = 0x00000000)+0x2af 0137f7d4 000f385b bench!set_options(struct or_options_t * new_val = 0x0f335730, char ** msg = 0x0137f80c)+0x6f 0137f80c 002f3707 bench!main(int argc = 0n1, char ** argv = 0x036b1e38)+0x24b ... Seems bench.c uses some mutex which is not initialised with 'tor_mutex_init()'. I fail to see which that should be. FYI. '&m->mutex' is 0 (= eax). But adding: tor_assert(&m->mutex); tor_assert(&m->mutex.OwningThread); to 'tor_mutex_acquire()' didn't help reveal the problem. -- --gv

2 3

3.2.10 compiler warnings
by grarpamp 11 Mar '18

11 Mar '18

Test from an older system before it was updated, unlikely to be prevalent, so don't bother fixing unless obvious. src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/test/vote_descriptors.inc:93: warning: string length '4135' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_hs_descriptor.inc:224: warning: string length '14104' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_microdesc.c:648: warning: string length '5844' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_descriptors.inc:305: warning: string length '10571' is greater than the length '4095' ISO C99 compilers are required to support

2 2