- tor-dev - lists.torproject.org

Damian's Status Report - December 2014
by Damian Johnson 27 Dec '14

27 Dec '14

Hi all! Tis that time of year coal appears in all my sox so guess it's time for a status report. This month I set arm aside to polish improvements that accumulated in the Stem 1.3 release... https://blog.torproject.org/blog/stem-release-13 There were a lot of small tweaks, but the highlights for this month were... -------------------------------------------------------------------------------- Hidden Service Tutorial -------------------------------------------------------------------------------- By a bit of serendipity I came across a nice article by Jordan Wright for spinning up hidden services with Stem... https://jordan-wright.github.io/blog/2014/10/06/creating-tor-hidden-service… Federico Ceratto and Patrick O'Doherty have been improving our hidden service support, so I spruced up their contributions and wrote a tutorial of our own that demos it... https://stem.torproject.org/tutorials/over_the_river.html -------------------------------------------------------------------------------- Gentoo Support -------------------------------------------------------------------------------- Thanks to Anthony Basile Stem is now packaged for Gentoo. Installing is as simple as... % emerge stem I've been delighted at how many Gentoo people have jumped out of the woodwork to help. toralf worked with me on ticket #13904 to address testing issues, and Manuel took on Stem 1.3 packaging the same day as its release - thanks guys! -------------------------------------------------------------------------------- Couple other quick things of note is that I updated our OpenHub repository listing so it includes most of our newer projects... https://www.openhub.net/p/tor ... and trimmed membership of our internal lists. Cheers! -Damian

1 0

Drop or merge the projects wiki?
by Damian Johnson 24 Dec '14

24 Dec '14

Hi all. Quick question, any objections to us dropping the projects wiki? https://trac.torproject.org/projects/tor/wiki/org/projects It's years out of date and redundant with the volunteer page, which I continue to update on occasion... https://www.torproject.org/getinvolved/volunteer.html.en#Projects I'm asking here in case there's any strong objections, or folks want us to keep something from it. Cheers! -Damian

3 5

Stem Release 1.3
by Damian Johnson 22 Dec '14

22 Dec '14

Greetings wonderful people of the world! After months down in the engine room I'm delighted to announce the 1.3.0 release of Stem. For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to arm and Vidalia. https://stem.torproject.org/ So what's new in this release? ------------------------------------------------------------------------------ Better Hidden Service Support ------------------------------------------------------------------------------ Now it's easier than ever to spin up hidden services! Thanks to contributions from Federico Ceratto and Patrick O'Doherty we now have a set of methods specifically for working with hidden services. Check it out in our new tutorial... https://stem.torproject.org/tutorials/over_the_river.html ------------------------------------------------------------------------------ Faster Descriptor Parsing ------------------------------------------------------------------------------ This release dramatically improves the speed at which Stem can parse decriptors. Thanks to optimizations from Nick Mathewson and Ossi Herrala we can now read descriptors 40% faster! ------------------------------------------------------------------------------ This is just the tip of the iceberg. For a full rundown on the myriad of improvements and fixes in this release see... https://stem.torproject.org/change_log.html#version-1-3 Cheers! -Damian

1 0

Thinking of raising bandwidth stats interval from 15 minutes to 4 hours
by Nick Mathewson 22 Dec '14

22 Dec '14

See https://trac.torproject.org/projects/tor/ticket/13988 . Karsten suggests that I should announce this on tor-dev before merging/deploying it, and he's probably right. Will this break anything you know about? -- Nick

2 1

Tor dev IRC meetings for the rest of this month
by Nick Mathewson 22 Dec '14

22 Dec '14

Hi, all! The regular tor patch workshop will happen on schedule, at 1800 UTC on the #tor-dev IRC channel on Tuesday the 23nd and the Tuesday 29th. I'm also planning to do a Tor dev meeting at the regular time of 1330 UTC on Wednesday the 24th and Wednesday the 31st, though of course it's likely that some folks won't make it at one of those times. (I strongly encourage taking Christmas Eve or New Years eve off, if that's your kind of thing.) Best wishes & happy holidays, -- Nick

1 0

Reminder for weekly OONI dev meeting Monday 2014-12-22 18:00 UTC (19:00 CET)
by Arturo Filastò 22 Dec '14

22 Dec '14

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

KIST somewhat implemented
by Nick Mathewson 20 Dec '14

20 Dec '14

Hi! So, Rob came up with this clever way to use help from an OS kernel to improve Tor's performance: http://www.robgjansen.com/publications/kist-sec2014.pdf I've got a sketchy implementation together now. See my comments on the end of ticket 12890. I have a branch which I believe implements something like the KIST algorithm for Linux as I discussed with Rob. I made a list of not-yet-implemented issues. Rob, could you please let me know which of the issues I listed need to be fixed before this is reasonably testable? cheers, -- Nick

2 1

N reasons why the spooks love Tribler (Number N' will surprise you)
by Yawning Angel 20 Dec '14

20 Dec '14

Hello all, Recently a decentralized onion-routing based Bittorrent client known as Tribler has made rounds through the clickbait garbage^w^wtech journalism sites. Since protocol design is a research interest of mine, I did some casual analysis based off the documentation and publicly available source code. Disclaimer: Analysis was casual and not massively in-depth though I believe it to be correct. Known issues are highlighted as such, though there is a decent amount of new material. References to how Tor approaches certain design problems are provided as appropriate. TLDR: Do not use this if your threat model includes active attackers, adversaries versed in cryptography, those with lots of money, or if you wish to be anonymous. Material covered: * The protocol documentation located at https://github.com/Tribler/tribler/wiki/Anonymous-Downloading-and-Streaming… * The source code of the "devel" branch https://github.com/Tribler/tribler/tree/devel/Tribler as of commit a98db38f60550dd304d1e329e16a41a683666c15 Protocol flaws: * There is no built in safeguard against creating circuits of arbitrary lengths. The subset of the Tor protocol the Tribler designers implemented is lacking a RELAY_EARLY type construct. Active adversaries can exploit this flaw to create substantial load on the Tribler network, and mount certain kinds of attacks. See: * Proposal 110 " Avoiding infinite length circuits" (https://gitweb.torproject.org/torspec.git/tree/proposals/110-avoid-infinite…) * Evans, S., Dingledine, R., Grothoff, C. "A Practical Congestion Attack on Tor Using Long Paths" (http://grothoff.org/christian/tor.pdf) * The symmetric encryption is ECB-AES128, without authentication. The former is a documented flaw in the specification and is claimed that it is "for testing", with what looks like a skeletal attempt to use OFB-AES128 present in the code. ECB should not go on the wire ever and is only useful as a building block for other modes. Even as a "for testing" this is rather sloppy, and is something that should have been addressed way before real users start using the protocol. The skeletal OFB-AES128 implementation included in dead code is not any better as it reuses a hard coded initialization vector for every single encryption operation. IV reuse within a given key across encryption operations breaks the confidentiality of the mode. The lack of authentication is flat out inexcusable for reasons that should be obvious. It is worth noting that this is one of the kludgier bits of the Tor wire protocol currently (see the 'digest' and 'recognized' fields in each RELAY cell, and proposals/ideas/xxx-new-crypto-sketch.txt), so improvements over "what Tor does" are possible and recommended. Implementation flaws: (As an aside, it's somewhat difficult to sort through which parts of the cryptography code are actually used and which are not.) * When gmpy is not present, Diffie-Hellman keys are generated with python's random.randint(). This is issue #874, and it should be evident why this is incorrect, and utterly terrible. The fact that developer comments regarding this bug note that this fallback is present only because gmpy did not work on Android should raise massive red flags in light of the next issue. * When gmpy *is* installed, Diffie-Hellman keys are generated with gmpy's "rand()". while dh_secret >= DIFFIE_HELLMAN_MODULUS or dh_secret < 2: dh_secret = rand("next", DIFFIE_HELLMAN_MODULUS) As far as I can tell, gmpy's rand() is either unseeded or is seeded via the following idiom that is scattered throughout the code (I suspect it is the latter, due to the Paillier cryptosystem code being cross referenced from other modules, but it's hard to tell): # The definition of StrongRandom changes depending on the file. # But every file that uses the construct illustrated here pulls it # in from optional_crypto, which looks like: # try: # raise ImportError() # # Dead code, this would be slightly less horrific, maybe. # from Crypto.Random.random import StrongRandom # except ImportError: # # ... Words can not describe how sad I am. # from random import Random as StrongRandom from optional_crypto import StrongRandom from sys import maxint rand('init', 128) rand('seed', StrongRandom().randint(0, maxint)) The unseeded case results in a LCG of the form: x = 1 // Yes, a hard coded seed. x = (29CF535 * x + 1) % (2^32) It should be obvious as to why this is totally broken. The seeded case results in a LCG of the form: x = (48A74F367FA7B5C8ACBB36901308FA85 * x + 1) % (2^128) This is fatally broken and insecure on several fronts. a) "StrongRandom" is Mersenne Twister seeded once via os.urandom(). What little "StrongRandom" actually resembles something that is "Strong" and "Random" is due to the Python developers, though in practice it is neither. b) The GMP LCG is seeded with 32/64 bits of "entropy", where "entropy" is Mersenne Twister output. c) GMP's LCG is used for key generation. * How not to do Diffie-Hellman: key = pow(dh_received, dh_secret, DIFFIE_HELLMAN_MODULUS) This is relatively minor since recovering the secret key is trivial via PRNG attacks, so the fact that the modular exponentiation is not constant time matters less. * How not to do RSA: def rsa_encrypt(key, element): assert isinstance(element, (long, int)), type(element) _element = mpz(element) return long(pow(_element, key.e, key.n)) def rsa_decrypt(key, cipher): assert isinstance(cipher, long), type(cipher) _cipher = mpz(cipher) return long(pow(_cipher, key.d, key.n)) RSA being implemented without blinding, using GMP's modular exponentiation, and without OAEP is something I pray that I never see again. Things not covered: * How the intro point/rendezvous system works, including peer discovery and etc (Assuming it is trivial to obtain lots of ECElgamal keys/IP address:Port combinations of endpoints, it appears that it is possible to use the CREATE/CREATED response to mount an UDP amplification attack. But I did not check how easy this would be to mount in practice.) * The bittorrent bits. * The "dispersy" component beyond "ok, at least the ECElgamal key generation uses M2Crypto, assuming the horribly broken keygen code (community.privatesemantic.crypto.ecelgamal.ecelgamal_init()) is dead like I think it is". Recommendations: * For users, "don't". Cursory analysis found enough fundamental flaws, and secure protocol design/implementation errors that I would be reluctant to consider this secure, even if the known issues were fixed. It may be worth revisiting in several years when the designers obtain more experience, and a thorough third party audit of the improved code and design has been done. * For the developers: * Use a CSPRNG when doing key generation. Neither GMP's LCG nor Mersenne Twister are CSPRNGs, even if they are seeded from "StrongRandom" that actually is a "StrongRandom" (rather than a horrible misnomer). * Add a construct similar to RELAY_EARLY. * Add authenticated encryption to cell payload. This will break wire compatibility. * Use constant time modular exponentiation. As a matter of fact, don't include your own implementations of ECElGamal, RSA, and Diffie-Hellman. Use well tested/known secure library code instead. * Blind RSA operations, use padding as appropriate. * Fix the symmetric encryption somehow, after obtaining a in-depth understanding of what things break security of the mode you wish to use in various ways (Eg: IV/Nonce reuse). See: http://web.cs.ucdavis.edu/~rogaway/papers/modes.pdf * Instead of a DH + ElGamal handshake, consider Ntor. Even with GMPY, doing modular exponentiation is relatively CPU intensive, and there is the threat of CPU exhaustion attacks here. * Clean out all the dead code, after printing out copies to use to scare small children. Eg: Tribler/community/privatesemantic/crypto/elgamal.py Most of the fixes require major revisions to the wire protocol. As it appears that there is no versioning, how that will be done is left as an exercise for the student. Alternatively, rebase the system on I2P. Regards, -- Yawning Angel

2 2

Re: [tor-dev] N reasons why the spooks love Tribler (Number N' will surprise you)
by str4d 20 Dec '14

20 Dec '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Yawning Angel wrote: > Hello all, > > Recently a decentralized onion-routing based Bittorrent client > known as Tribler has made rounds through the clickbait > garbage^w^wtech journalism sites. Since protocol design is a > research interest of mine, I did some casual analysis based off the > documentation and publicly available source code. Thank you for taking the time to write this; I'm sure I speak for many when I say that your contributions are greatly appreciated :) If this is "casual analysis", I shudder to think what you would find during office hours :-P Perhaps it could be posted on the Cryptography Coding Standard wiki [0] as an example of what not to do? > Most of the fixes require major revisions to the wire protocol. > As it appears that there is no versioning, how that will be done > is left as an exercise for the student. > > Alternatively, rebase the system on I2P. If they did choose to rebase on I2P, they would be able to integrate with the existing I2P torrent swarms. In case the Tribler developers are following this ML, the protocol details common to all BitTorrent clients on I2P are documented in [1]. str4d [0] https://cryptocoding.net/ [1] https://geti2p.net/en/docs/applications/bittorrent -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJUlV4AAAoJEIA97kkaNHPnZFoP/iSsUzDBUsgApq6XC0o3TMlt SpjTEvYkuK4xq2wLEcggu+eX7ZOTQFLPjIocIfSkkPuj4DDT0/Esyg9/Fo55BsmQ xHN6u6CaaDNCe2VdnCm1T93MFUL9hTIvVV+hXaF78QIrbp2+g5swOpuBl8ePnMhX +YcBQ/T3mPMhKSBbDAQgRMl5vYovpu2ET2dYb0hhJcwtZ0OVSKlRSH0NJCrCH/KB UpJCI4gSj731POUhuKVUE6oYc6ZlUJt6262sRFAyLOuej+gHlcYtXrsNjX1zOKTV /vs0X6DeyYJlLD6gfNpZuDVZNA/a0zRJTuoK0imHwFIUQDuIXqPq3yTs8xcFcCtS 5PWjQKXgrOfN/ifwTqRaQHKm9UXaOFAkNDCfjGui+cftncZ0VquJBnsQFW6KbngK N+1+QetXk8RQ36IyBq9H7j6uqSMTfPQiBVUFWec9GcqWaruLRRcLj5IGFTHgiAGt KYd6bYgMFwUUTVVOTj0dgeeCqfFBdzR12rSzb3SzlXq/i8UgFcuIn5cw5p3TfWYm LN6twG5i8tK6F2U6TNaakbmih6xMVRMUBbV/EleBrIYdQeTYG8FvAsGQQ70r0kzz nBCiuAWYRN7EWNtmKClNJGj0GTe4gwN+0et6g0HmVqMerw+4tPAJ8exWvuJNT0e8 0EYzbEfMRcG61zVgNIPI =xhxt -----END PGP SIGNATURE-----

1 0

Hidden service performance
by David Goulet 19 Dec '14

19 Dec '14

Hello tor-dev, This email is to detail the work being done on a framework to do performance measurements for hidden service. I'm looking to get some feedback on the whole idea, improvement and help on possible analysis. If this work is not thrown out the window, I'll definitely do a Wiki page for a detail documentation but for now this is still a bit too early I think. *** PLEASE NOTE THAT THIS FRAMEWORK IS _NOT_ INTENDED TO BE USED ON THE REAL TOR NETWORK, ONLY TEST NETWORK! *** (I'll even add a check to make sure TestingTorNetwork is set to 1 else tor should not run.) The whole idea of this effort is to have a way to do very precise measurements of hidden service so we can come up with a performance baseline and have a reliable way to measure our progress on improving HS. We want to run this on large private network such as Shadow in order to get a better understanding of this whole HS subsystem. Finally, this can be extended to all sorts of component in tor in the future to help us analyze the beast that tor has become. :) This work uses tracing[0] for measurements. For that, I've instrumented the Tor code base with "tracepoints" that allows us to hook a tracer to those so we can collect data and do analysis afterward. The following branch contains a bunch of tracepoints mostly for HS and uses the LTTng[1] user space tracer. It's based on tor 0.2.5 version because we wanted to measure regression or improvement between 0.2.5 and upcoming 0.2.6 (especially with Andrea's new scheduler work). https://gitweb.torproject.org/user/dgoulet/tor.git [branch: hs-lttng-025] To compile this branch you need to install lttng-ust (tracer). It's pretty easy to install, you can find out how here[2]. (You will need version 2.4.0 or above). ./configure --enable-lttng-tracing In "src/or/trace/events.h", you can find all tracepoints that have been added with the data they collect. It is a bit too tidious to explain them all to you in this email but every "tor_trace_*" line is a tracepoint that you can grep in the code to see their location thus have a clearer picture of what they record. First, if you see any tracepoints that record something but are not at the right place in the code, you should definitely let me know. Second, telling me where tracepoint should be added also is very useful! No need to understand the full tracing system, just "that function should have a tracepoint to record this", I'll do the rest. Now to use tracing with Tor and analyze the data, you can find useful information and examples in this repository. (Pretty sure it's missing some documentation so please point it out!) https://gitorious.org/tor-hs-stats/tor-hs-stats Here are examples of graph that we can do with the collected data from the traces. (Introcuction Points Ready Time) https://people.torproject.org/~dgoulet/hs/ip-025.png or (Rendezvous Point Ready Time) https://people.torproject.org/~dgoulet/hs/rp-025.png Now to make analysis, I've build what is called in the tracing world a "state system". See that as a way to represent different objects in an application that are populated by the data that has been collected with tracing (trace). For this, please take a look at "lttng-analyze.py" in the gitorious repository. That script parses the trace event by event (event is what a tracepoint generated), populates the state system which is analyzed after the trace is fully parsed. It's pretty straight forward and here is the power of the state system. Once build from a trace, we can run *ALL* sorts of analysis on it to measure timings, find bad behaviors, etc.. If that interests you, I encourage you to play with all this and see for yourself what can be achieved thus opening your eyes on the possibilities. :) Ok, so there is a lot to explain to make things clear. I'm pretty sure I'm forgetting some here. Understanding the benifits of tracing is not that trivial at first glance so I'll be really happy to explain it to anyone asking. Any feedback is positive feedback! I'm happy to answer any questions. I'll most probably follow up this with a post about some results at some point. Cheers! David [0] https://en.wikipedia.org/wiki/Tracing_%28software%29 [1] https://lttng.org [2] https://lttng.org/download/

1 1