- tor-dev - lists.torproject.org

Re: [tor-dev] (no subject)
by Sambuddho Chakravarty 15 Mar '18

15 Mar '18

Hi All We (myself and my student, cc'ed here) have some questions regarding Tor relay bandwidth. We created a small testbed (private Tor network) involving relays, clients and servers over virtual machines. The client uses the relays to create two hop circtuits to communicate to the server. The link bandwidths between the virtual machines is otherwise ~ 10 Gbps (tested via end-to-end measurement tools like iperf and via downloading large files of sizes ~2 GB through https downloads). When not using Tor, the client achieve a download rate of approximately 8 Gbit/s while downloading a file of approximately 2 Gbytes. The same process, when repeated over Tor circuit (through relays of the private Tor network, that used virtual machines), p*rovides a maxmium download rate **of about 700 Mbit/s for the client (while the CPU utilization was over 80%).* This is much less compared to the expected throughput of 8Gbps achieved when client communicates directly to the server (without using Tor circuits). We tinkered around with the Tor config and discovered that when changed the config fields of BandwidthBurst and BandwidthRate to 100 MBytes, we achieved end-to-end throughput of about 100 Mbit/s (while the CPU Utilization was less than 20%) (while the underlying network link capacity is 1 Gbit/s). The moment the BandwidthBurst and BandwidthRate values are increased to 1 GB (which should effectively be 8 Gbit/s), the achieved throughput is about 650-700 Mbps (which is still 8 times slower than expected). Further, removing all bandwidth restrictions, the inter VM bandwidth (for the relays in the private Tor network) is around 8 Gbit/s (measured both via Iperf and through downloading a file > 2 GBytes using encrypted HTTP traffic). While this is as per the expected behavior of TCP, to utilize all the available bandwidth, the same does not happen for Tor which continues to ramp upto a maximum of 700 Mbit/s We are confused as to why Tor's link bandwidth utilization is not higher (Even on a private Tor network), even when using a high capacity VMs in the private Tor network -- each one having about 4 cores and 8 GBytes of RAM. I understand after speaking from some Tor developers and maintainers that Tor does not utilize all the available CPU cores. Our VMs however do not have any other process running either. We run the Tor processes over Debian servers with little no other programs. More strangely, while download via HTTPS achieves a very large fraction of the link capacity (>80%), Tor traffic happens to achieve a throughput of around 700 Mbit/s (which is around 90% lower than the max link bandwidth). We tried the same with 2-hop Tor circuits, hoping to see some improvement. But to no avail. We continue to observe the same poor bandwidth utilization. It would be great if you shed some light as to where we may be going wrong. Regards Sambuddho

2 1

Tor: unspecified is the default Core Tor milestone
by teor 15 Mar '18

15 Mar '18

Hi, The network team maintains Core Tor, the network daemon that runs the tor network. We have planned our roadmap for the next 6 months. We only chose essential and sponsored tasks, because we want to finish on time. We moved all the other tickets out of the 0.3.4 and 0.3.5 milestones. If you want to add a ticket to Core Tor, please add it to the "Tor: unspecified" milestone. We will discuss essential tickets each week. We will only add tickets to the release milestone if we agree we have time to complete them. I have moved the "Tor: unspecified" milestone to the top of the milestones list, to make it easier to find. T

1 0

[Win32] crash in test/bench.exe
by Gisle Vanem 14 Mar '18

14 Mar '18

There is a crash in bench.exe. Running 'cdb -c g bench.exe': ntdll!RtlpWaitOnCriticalSection+0x6b: 77a9cfd6 ff4014 inc dword ptr [eax+14h] ds:002b:00000014=???????? ChildEBP RetAddr 0137f750 77aba38a ntdll!RtlpWaitOnCriticalSection+0x6b 0137f770 77aba259 ntdll!RtlpEnterCriticalSectionContended+0x12a *** WARNING: Unable to verify checksum for bench.exe 0137f77c 002150f7 ntdll!RtlEnterCriticalSection+0x49 0137f784 0023136c bench!tor_mutex_acquire(struct tor_mutex_t * m = 0x0039559c)+0x37 0137f794 000fc85f bench!atomic_counter_exchange(struct atomic_counter_t * counter = 0x0039559c, unsigned int newval = 6) +0xc (Inline) -------- bench!set_protocol_warning_severity_level+0xb 0137f7bc 0010563f bench!options_act(struct or_options_t * old_options = 0x00000000)+0x2af 0137f7d4 000f385b bench!set_options(struct or_options_t * new_val = 0x0f335730, char ** msg = 0x0137f80c)+0x6f 0137f80c 002f3707 bench!main(int argc = 0n1, char ** argv = 0x036b1e38)+0x24b ... Seems bench.c uses some mutex which is not initialised with 'tor_mutex_init()'. I fail to see which that should be. FYI. '&m->mutex' is 0 (= eax). But adding: tor_assert(&m->mutex); tor_assert(&m->mutex.OwningThread); to 'tor_mutex_acquire()' didn't help reveal the problem. -- --gv

2 3

3.2.10 compiler warnings
by grarpamp 11 Mar '18

11 Mar '18

Test from an older system before it was updated, unlikely to be prevalent, so don't bother fixing unless obvious. src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/common/confline.c:135: warning: 'include_list' may be used uninitialized in this function src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/or/connection.c:1883: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/conscache.c:426: warning: passing argument 2 of 'consensus_cache_entry_map' discards qualifiers from pointer target type src/or/control.c:6815: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/scheduler.c:577: warning: format '%ld' expects type 'long int', but argument 5 has type 'time_t' src/test/vote_descriptors.inc:93: warning: string length '4135' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_hs_descriptor.inc:224: warning: string length '14104' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_microdesc.c:648: warning: string length '5844' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_descriptors.inc:305: warning: string length '10571' is greater than the length '4095' ISO C99 compilers are required to support

2 2

Proposal Waterfilling
by Florentin Rochet 09 Mar '18

09 Mar '18

Hello everyone, In order that our paper does not fall into the list of "yet another seems-to-be-cool-feature is that never going to be discussed because researchers moved on another topic", here is an attached proposal in which we summarize our work published in the last PETS event and how it can be implemented. And, if you find this interesting, I would be glad to submit a patch :) Any kind of feedbacks is more than welcome! Cheers! Florentin

8 29

Proposal 290: Continuously update consensus methods
by Nick Mathewson 03 Mar '18

03 Mar '18

Filename: 290-deprecate-consensus-methods.txt Title: Continuously update consensus methods Author: Nick Mathewson Created: 2018-02-21 Status: Open 1. Background Directory authorities use the "consensus method" mechanism to achieve forward compatibility during voting. When each authority publishes its vote, it includes a list of numbered consensus methods that it supports. Each authority chooses to calculate the consensus according to the highest consensus method it knows supported by more than 2/3 of the voting authorities. So long as all the authorities have a method in common, they will all reach the same consensus. Consensus method 1 was first introduced in the Tor 0.2.0 series around 2008. But by 2012, we realized that we had a problem: we were stuck documenting and supporting old consensus methods indefinitely. With proposal 215, we deprecated and removed support for all consensus methods before method 13. That was good as far as it went, but it didn't solve the problem going forward: the latest consensus methods is now 28. This proposal describes a policy for removing older consensus methods going forward, so we won't have to keep supporting them forever. 2. Proposal I propose that from time to time, old consensus methods should be deprecated. Specifically, I propose that we deprecate all methods older than the highest method supported in first stable release of the oldest LTS (long-term support) release series. For example, the current oldest LTS series is 0.2.5.x. The first stable release in that series was 0.2.5.10. The highest consensus method listed by 0.2.5.10 is 18. Therefore, we should currently consider ourselves free to deprecate all methods before 18. Once 0.2.5.x is deprecated, 0.2.9.x will become the oldest LTS series. The first stable release in that series was 0.2.9.8. The highest consensus method listed by 0.2.9.8 is 25. Therefore, once 0.2.5.x is deprecated (in May 2018), we may deprecate all methods before 25. When a consensus method is deprecated, it should no longer be listed or implemented by the latest Tor releases. (It's okay for older authorities to keep advertising it.) Most consensus methods add a feature that is used in "method M or later". Deprecating method M-1 means that the feature is used in all supported consensus methods. Therefore, we can remove any code that makes the feature conditional on a consensus method, and any code for previous implementations of the feature. Some consensus methods remove a feature that was used up to method N. Deprecating method M means that the feature is no longer used by any supported consensus methods. Therefore, we can remove any code that implements the feature. A. Acknowledgments Thanks to isis and teor for the discussion that led to this proposal. I believe that teor first suggested the policy described in section 2 above. B. Client and relay compatibility notes Dear reader: you may be worrying that this proposal will cause old clients or relays to stop working prematurely. That is not the case. Consensus methods determine how the authorities behave, but they do not represent backward-incompatible changes in how they generate their consensuses.

3 2

New link on Tor Metrics: download data as CSV
by Karsten Loesing 28 Feb '18

28 Feb '18

Hi tor-dev! Today we added a small, but potentially quite useful feature to Tor Metrics. The data that is displayed in a graph can now be downloaded as CSV file by clicking on "download data as CSV". This is different from before, where we provided the larger CSV file that a graph is based on. Getting the relevant data out of that would still require some data wrangling. This has become a lot easier now. Want to try it out? Go to any graph on Tor Metrics, for example: https://metrics.torproject.org/relays-ipv6.html?start=2013-02-28&end=2018-0… Click on "download data as CSV" and you'll get just the values that are plotted in the graph. Want to learn more and maybe help us make this feature more useful? There's more about it in tickets #25282 (closed), #25383, and #25387. All the best, Karsten

1 0

DocTor Check for <3 bw auth votes available
by nusenu 28 Feb '18

28 Feb '18

Hi Damian, would be great to have a check in DocTor that sends out an email for "there are less than 3 bw auth voting auths available" thanks, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

3 6

[release] CollecTor 1.5.0, metrics-lib 2.2.0
by iwakeh 26 Feb '18

26 Feb '18

Hi there! two new releases are available: https://dist.torproject.org/collector/1.5.0/ https://dist.torproject.org/metrics-lib/2.2.0/ These releases will be deployed on tp.o instances after several weeks of testing, and provide Tor's web server access logs that are currently available on [0] through CollecTor. All possibly privacy impacting data is removed from the logs prior publishing. What exactly is done to ensure privacy can be found in the specification doc [1]. The additions to metrics-lib's API can be read in its Javadoc pages [2]. Other details can be found in the respective change logs [3] and [4]. Please direct comments and questions to the metrics-team mailing list [5]. Cheers, iwakeh [0] https://webstats.torproject.org/ [1] https://metrics.torproject.org/web-server-logs.html [2] https://metrics.torproject.org/metrics-lib/ https://metrics.torproject.org/metrics-lib/index.html?org/torproject/descri… https://metrics.torproject.org/metrics-lib/index.html?org/torproject/descri… [3] https://gitweb.torproject.org/collector.git/tree/CHANGELOG.md?h=collector-1… [4] https://gitweb.torproject.org/metrics-lib.git/tree/CHANGELOG.md?h=metrics-l… [5] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Creating custom Tor Browser settings profile
by procmem 26 Feb '18

26 Feb '18

Hi I was wondering if Tor Broswer supports environment variables to customize about:config settings. The idea is to ship an explicitly labelled, custom profile one day to allow users seamless access to localhost daemons (like zeronet) by changing network.proxy.no_proxies_on to 0 - so they don't have to jump thru hoops or install foreign code like foxyproxy. At the same time, a separate browser profile affords regular users protection by keeping these fingerprinatble, non default settings contained and only used when the user needs them. Any way to do this?

2 1