- tor-dev - lists.torproject.org

Memory usage on 0.3.4.8
by r1610091651 24 Sep '18

24 Sep '18

Hi devs After upgrade from 3.3.7 -> 3.4.8, I've noticed that memory usage is erratic. Please consult the image below, showing memory available in green and memory used in blue on a tor dedicate system. [image: image.png] The upgrade to 3.4.8 took place on Saturday in week 37. The connection count / load was relativity stable: [image: image.png] (green inbound, blue outbound) I was wondering if there were any changes from 3.3.7 to 3.4.8 which might impact memory behaviour on a middle-relay. I'm also wander what is the "source" of this usage? Potentially relevant config entries (remainder is port/ip, accounting & exit(no)): MaxConsensusAgeForDiffs 4 hours DirCache 0 MaxMemInQueues 64 MB (but "defaulting" to 256mb now) Thanks in advance Seb

2 1

Strong Flow Correlation Attacks on Tor Using Deep Learning
by procmem 21 Sep '18

21 Sep '18

Hi I wanted to bring your attention to this recent Tor attack paper published at the CCS conference: https://arxiv.org/pdf/1808.07285.pdf > We show that with moderate learning, DeepCorr can correlate Tor connections (and therefore break its anonymity) with accuracies significantly higher than existing algorithms, and using substantially shorter lengths of flow observations. For instance, by collecting only about 900 packets of each target Tor flow (roughly 900KB of Tor data), DeepCorr provides a flow correlation accuracy of 96% compared to 4% by the state-of-the-art system of RAPTOR using the same exact setting. > > We hope that our work demonstrates the escalating threat of flow correlation attacks on Tor given recent advances in learning algorithms, calling for the timely deployment of effective countermeasures by the Tor community. It has an extremely high true positive rate under real world conditions. The two main suggested countermeasures are padding and AS-aware path selection. I think you are working on the former but no practical implementation of the latter is ready?

1 0

Proposal 297: Relaxing the protover-based shutdown rules
by Nick Mathewson 21 Sep '18

21 Sep '18

Filename: 297-safer-protover-shutdowns.txt Title: Relaxing the protover-based shutdown rules Author: Nick Mathewson Created: 19-Sep-2018 Status: Open Target: 0.3.5.x 1. Introduction In proposal 264 (now implemented) we introduced the subprotocol versioning mechanism to better handle forward-compatibility in the Tor network. Included was a mechanism for safely disabling obsolete versions of Tor that no longer ran any supported protocols. If a version of Tor receives a consensus that lists as "required" any protocol version that it cannot speak, Tor will not start--even if the consensus is in its cache. The intended use case for this is that once some protocol has been provided by all supported versions for a long time, the authorities can mark it as "required". We had thought about the "adding a requirement" case mostly. This past weekend, though, we found an unwanted side-effect: it is hard to safely *un*-require a currently required protocol. Here's what happened: - Long ago, we created the LinkAuth=1 protocol, which required direct access to the ClientRandom and ServerRandom fields. (0.2.3.6-alpha) - Later, once we implemented Ed25519 identity keys, we added an improved LinkAuth=3 protocol, which uses the RFC5705 "key export" mechanism. (0.3.0.1-alpha) - When we added the subprotocols mechanism, we listed LinkAuth=1 as required. (backported to 0.2.9.x) - While porting Tor to NSS, we found that LinkAuth=1 couldn't be supported, because NSS wisely declines to expose the TLS fields it uses. So we removed "LinkAuth=1" from the required list (backported to 0.3.2.x), and got a bunch of authorities to upgrade. - In 0.3.5.1-alpha, once enough authorities had upgraded, we removed "LinkAuth=1" from the supported subprotocols list when Tor is running with NSS. [*] - We found, however, that this change could cause a bug when Tor+NSS started with a cached consensus that was created before LinkAuth=1 was removed from the requirements. Tor would decline to start, because the (old) consensus told it that LinkAuth=1 was required. This proposal discusses two alternatives for making it safe to remove required subprotocol versions in the future. [*] There was actually a bug here where OpenSSL removed LinkAuth=1 too, but that's mostly beside the point for this timeline, other than the fact it would have made things waaay worse if people hadn't caught it. 2. Recommended change: consider the consensus date. I propose that when deciding whether to shut down because of subprotocol requirements, a Tor implementation should only shut down if the consensus is dated to some time after the implementation's release date. With this change, an old cached consensus cannot cause the implementation to shut down, but a newer one can. This makes it safe to put out a release that does not support a formerly required protocol, so long as the authorities have upgraded to stop requiring that protocol. (It is safe to use the *scheduled* release date for the implementation, plus a few months -- just so long as we don't plan to start requiring a subprotocol that's not supported by the latest version of Tor.) 3. Not-recommended change: ignore the cached consensus. Was it a mistake to have Tor consider a cached consensus when deciding whether to shut down? The rationale for considering the cached consensus was that when a Tor implementation is obsolete, we don't want it hammering on the network, probing for new consensuses, and possibly reconnecting aggressively as its handshakes fail. That still seems compelling to me, though it's possible that if we find some problem with the methodology from section 2 above, we'll need to find some other way to achieve this goal.

5 6

Back to the action!
by Fernando Fernandez Mancera 18 Sep '18

18 Sep '18

Greetings! I have been out this summer but I am back and I want to continue contributing. Starting next week I will attend the network-team meetings if you don't mind. If there is any task you could think it fits with my experience, please let me know. If there isn't, I am going to search in the ticket list. I am preparing my environment right now, and by tomorrow I will be ready to work! Thank you and see you soon at #tor-dev, ffmancera.

1 0

State of the HA proxy onion patch
by George Kadianakis 17 Sep '18

17 Sep '18

Hello Mahrud, I wanted to ask if you've been using the #4700 branch and how is it going? We've been planning to include #4700 in the upcoming 0.3.5 release if possible, and we remember that you had some pending patches to it. Do you think you can publish those somewhere if they are to be included upstream? There are also some further mods that need to happen that I'm not sure if you've performed in your local branch (torrc option & restricting the feature only to onion connections, as per #4700). Let us know how it's working for you and whether you have any patches that we should have in mind, so that we can see if we can fit it in the 035 release. Thank you! :)

2 4

Tor master exits with a required protocol error
by teor 17 Sep '18

17 Sep '18

(CC'd tor-relays, please direct replies to tor-dev.) Hi, If you are running Tor master, your Tor may exit with a required protocol error. The issue was introduced in commit 991bec67ee (about 8 hours ago). The following conditions will make Tor exit: * Tor will exit on startup if it has a cached consensus from the 14th of September (or earlier). It does not matter how old the consensus is. * If certain directory authorities go down, the remaining authorities may generate a consensus that will cause your Tor to exit. These consensuses require the LinkAuth=1 protocol, which Tor master believes it does not support. (The change was intended for NSS Tors, but mistakenly applies to all Tors.) Here's how you can mitigate this issue: * downgrade to 0.3.4.8 or earlier, or * delete the cached consensuses in your data directory. These consensuses may be called cached-consensus, cached-microdesc-consensus, unverified-consensus, and unverified-microdesc-consensus. Some may not be present. We should resolve this issue in master in the next 24 hours. Then it may take another 24 hours for packages to be rebuilt. The master ticket for this change is: https://trac.torproject.org/projects/tor/ticket/27288 T -- teor Please reply @torproject.org New subkeys 1 July 2018 PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ----------------------------------------------------------------------

1 0

Upcoming changes to CSV files provided by Tor Metrics
by Karsten Loesing 14 Sep '18

14 Sep '18

Hello tor-dev@, today we finally published specifications [0] for the per-graph CSV files that we added to Tor Metrics earlier this year. At the same time we're announcing two upcoming changes to CSV files provided by Tor Metrics: - August 15: We're going to make the first batch of changes to per-graph CSV files marked as "Suggested change" on the linked page. If you're currently working with per-graph CSV files, you should watch out for those changes. We're tracking these changes on #26998 [1]. - September 15: We're going to remove pre-aggregated CSV files after providing working alternatives with the per-graph CSV files for all use cases we thought of or that were brought to our attention. If you're currently working with pre-aggregated CSV files, you should consider switching after August 15 but before September 15. The relevant ticket for this change is #27000 [2]. Please direct any questions or comments to the metrics-team@ mailing list [3] or comment on either of the two tickets mentioned above. All the best, Karsten [0] https://metrics.torproject.org/stats.html [1] https://bugs.torproject.org/26998 [2] https://bugs.torproject.org/27000 [3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 1

[release] Onionoo 7.0-1.18.1
by Iain Learmonth 11 Sep '18

11 Sep '18

Hi, Onionoo's protocol was extended and has a major version bump to 7.0. Download available at: https://dist.torproject.org/onionoo/7.0-1.18.1/ Protocol changes (also summarized in [0]): Extended the "version" parameter to support lists and ranges Removed redundant "1_week" and "1_month" graphs from clients documents Changed "3_months" graphs to "6_months" graphs in all documents containing history objects Removed the "fingerprint" parameter Removed the previously deprecated "as_number" field from details documents Software changes are summarized in the changelog [1]. The changes are already deployed on all onionoo.torproject.org instances. Note that the 7.0-1.18.0 release suffered from a bug that did not surface until deployment and the deployment was rolled back until the 7.0-1.18.1 release could be made. This issue only affected the web server, not the updater or state files. Please direct comments and questions to the metrics-team mailing list [2]. Thanks, Iain -- on behalf of the Metrics Team. [0] https://metrics.torproject.org/onionoo.html#versions_6_1 [1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md?h=onionoo-6.2-… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Tor Volunteer
by ESAUL PARRA GARCIA 09 Sep '18

09 Sep '18

Hello everyone, I want to volunteer my time for the tor project. I have a background in statistics, python, R, and web development(react, nodejs, monbodb, aws, dynamob). Let me know what I can do to help. Email me back so I can send you my resume. Best, Esaul Parra Garcia -- University of California, Los Angeles B.S. Statistics | 2017 Email: epg323(a)ucla.edu | Mobile: 323.984.1204

5 10

PDF versions of Marionette Documents
by John Helmsen 04 Sep '18

04 Sep '18

These are the PDF versions of the Marionette Documents for easier reading. The markdown versions of them can be found on the github distribution at https://github.com/redjack/marionette/doc -- John Helmsen john.helmsen(a)redjack.com C: (301) 273-4372

3 2