August 2013 - tor-dev - lists.torproject.org

[GSOC] Status report - Tor capabilities
by Cristian-Matei Toader 09 Aug '13

09 Aug '13

Hello tor-dev, Here goes the status report for the past 2 weeks: - still working at [1] in public branch [2] - rewrote how the filters work, such that a filter currently is an array of function pointers, each function being responsible of setting up the parameter filter for a syscall; also added support such that in the future it is a struct containing the array of function pointers and a linked list of parameters associated with the filter; - I have updated the filters which proved to be a bit more challenging than expected mainly due to the multiple worker thread processes created with clone, which are harder to follow with strace; - as of today, the latest version I have pushed has a more restrictive parameter filter which works for both a normal and ORPort launches. TODO: - fix open syscall problem for file names due to external library opens - investigate socketcall syscalls' parameters failure.. - finish support for multiple filters (beginning of stage 3) - fine-tune filters a bit (end of stage 2) More details may be found here [1]. Looking forward to some feedback, if you happen to have any! References: [1] https://trac.torproject.org/projects/tor/ticket/9249 [2] https://github.com/cristiantoader/tor-gsoc-capabilities/tree/gsoc-cap-stage2

1 0

[GSoC] Status Report - EvilGenius
by Johannes Fürmann 09 Aug '13

09 Aug '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi *, Quite a few things happened in the last few weeks. GSoC midterm, i thought to myself last week, is a great time for some refactoring, so I looked over the code a bit and was able to improve the overall code quality quite a bit. Also I improved the network handling, The network addresses are now actually being set on the VMs and everything is patched together quite nicely, including the connections from the router to the censorship provider. Additionally, I replaced the simple "print" commands with the python logging mechanism and added a dryrun mode that - instead of pulling the whole network up and tearing it down, just prints the vagrantfile, which facilitates testing a lot. Since every machine now has it's resources directory mounted as '/scripts', one can also access shell scripts from the machine's yaml definition. After this weekend, I am going to get started with adding a few modules and implement routing. Have a nice weekend, everyone! Johannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSBThsAAoJEAgie5o3Q/JpbdUP/RobudUibBGhLHYKYWho5x8h XGBaLhFzVxHfCQhxoIrNP5CbW1OiCb+HQClIFnk4QSnlD189qoK+LzLg/2SuvyVH EnYICDy+GE82F50cRIoZb5yIITKOohQR051ienXIJr8GT1rna50YezOyl4WY5DI+ LIoQ50MR7BdTdT4/BATRK6U3sAs7fVI/VElSHWcCErmNn6PF652B84mQk83+A02c XpkYyEgLZ55tjzq1V+CbcMzYBxnJhPJH8O993mE18AfhaPfnoQvF090xSTsYrm80 GUuRn2Xzw6CHHsbZofdvrS/pzQqHIGreyhKU0lTZxMhH7Lx3zAJ57nO1gpmpH83+ QTNeufwJGUL+vwm5Ci/WT8FGylvoAejvpYjJsyASvAjFMpi5TCs5zBSEYDKQAmFY XMiuAe3R4T/uczpsq0g/T2Y6vH8XpMQ4eadh9IVRjjZI7nRIbjI9hYYWukEsG9/r WSwmsS7ADscbeR6Z5EqLdgdhCgrcBG/jyX8PBTT0hMtoZf03VbPwRXJsFQrhEful KEcoHLCVbG5WUodDZPW9plupni0jefoZek46H0pAzejRzadtmYr9KvseutFC4hcq hArOzsztWqmDJTfoVtQpLgycBnhhz/0ajUpCyxGB5x/OyWj5gO2rVKD++ib7y9/E yR9Wt4fPBrS8X6a1P2JA =2ap3 -----END PGP SIGNATURE-----

1 0

Fwd: Brendan Eich floats idea of Firefox implementing Tor as FF plug-in?
by Bernard Tyers - ei8fdb 08 Aug '13

08 Aug '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Dave, I have forwarded your mail to the tor-dev list as someone there might be interested/able to help with the boot2gecko (FF OS) work. All the best, Bernard Begin forwarded message: > From: Dave Huseby <dhuseby(a)mozilla.com> > Date: 8 August 2013 19:48:35 GMT+01:00 > To: dev-privacy(a)lists.mozilla.org > Subject: Re: Brendan Eich floats idea of Firefox implementing Tor as FF plug-in? > > So, the tor project maintains their own fork of firefox in the tor > browser bundle. I know that there have been efforts to try to merge > their changes into our repos. AFAIK, the main reason we haven't done so > is because some of their changes modify the default behavior of firefox. > > As a side note, I have a working b2g build of libevent, openssl + > engines, and the tor daemon (all static). I'm in the process of setting > up the a good manifest so that others can recreate it. It won't be > buildable out of the box if you just clone my repos until I set up the > manifest correctly. But I've completed all of the work making > Android.mk files so that the three pieces build as b2g/external/*. If > anybody wants to take a look/help, my repos are here: github.com/dhuseby > All of my changes are in b2g branches in the libevent, openssl, and tor > forks. > > -dave > > On 08/08/2013 09:48 AM, Sid Stamm wrote: >> On 8/8/13 8:52 AM, Bernard Tyers - ei8fdb wrote: >>> "Maybe we should just adopt, support, and bundle Tor in Firefox..." >>> [1] >>> >>> It's early days, and just a comment on Twitter, but I wonder what >>> others have to say. >>> >>> Good or bad idea? I would say (without too much thinking): turned-off >>> by default, but available. Exciting possibilities. >> >> Let me turn it around: what do you think? Why would you like to see this? >> >> I think it's interesting, and at the very least we could work more >> closely with the Tor engineers to make each others software better. >> >> -Sid >> _______________________________________________ >> dev-privacy mailing list >> dev-privacy(a)lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-privacy >> > > _______________________________________________ > dev-privacy mailing list > dev-privacy(a)lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-privacy - -------------------------------------- Bernard / bluboxthief / ei8fdb IO91XM / www.ei8fdb.org -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJSA/jZAAoJENsz1IO7MIrrsfIIAIsylP6Q59wGh8JB4ThGIn4b wnddEG6KPCz8vBNgJ1cu6luVXiPLc5ByGRBMAIC0fuNwoP2G5x3JW0HvT9LPztSI giudGcdwocRKWXbfzgY7Pwy5yGwM+SxHAVdlYNTWdz+6CSdzy9kMwpd75u0+TYM1 bWCHY1Z3xhjrrBCwqTLzg3bbU3IWbPImCqGNiDJ5HB475TG7AGRYivM1Ls0+jRRQ 87glawpVvodSTydhg4K8uSwWfJBCEodH82WYBaipiGkaKDX/X/Ha2N7oBNPiQ4IS GX3b6fCjoO9CeRTJa8jRIHcBxbX3jO65jpTUbAb1ITVnPWpRLUr/UbpEfEIqlSM= =eLJC -----END PGP SIGNATURE-----

1 0

Fallback metrics service operator
by Karsten Loesing 08 Aug '13

08 Aug '13

Hi Sebastian, we discussed that you could become the fallback metrics service operator and take over operation whenever I have to go on a four-week round-the-world cruise or in the unlikely case that my application for the one-way trip to Mars [0] goes through. This is awesome! Thanks a lot! Moving this discussion to tor-dev@ in case this is interesting to others. So, you asked what the actual tasks are, and how much time I'd think they'll be taking. Let's start by taking a look at this overview [1] of the various tasks that are performed by metrics services. Going through the various tool names stated in brackets, starting with the ones that I think should have highest priority: - metrics-db-* is where we collect all the fine Tor network data. The cronjob behind metrics-db-R has to run once per hour, or our archives will be missing consensuses and votes published in that hour. The other metrics-db-*'s can handle short downtimes better than metrics-db-R. Beware that this is not the best code you ever read. - metrics-web-* is the metrics website, including the necessary PostgreSQL and Tomcat parts. I'm not spending much time on this codebase anymore, because we should really do something like Thomas does in his Visionion project. But until that's there, metrics-web is all we have. And I think it has quite a few visitors. So, if it breaks, we should fix it ASAP. - task-6498, task-8462, and task-2718 are additions to metrics-web which should be properly integrated into metrics-web.git. I should do this, maybe together with you. - Onionoo powers Atlas, Compass, and a few other tools. It has become quite complex over its 1.5 years of existence. It consists of two parts, a Java backend and a Tomcat frontend. It's actually one of the few codebases I maintain that has pretty high test coverage. I'm trying hard to keep this service running 24/7, and I'm impressed how fast people report if it breaks for just half an hour. Those are the highest priority ones. Looking at the lower priorities: - tor produces descriptors and statistics in its extra-info descriptors that we collect. The only thing I do is nag directory authority operators and the bridge authority operator if something's wrong, so that our descriptor archives will be as complete as possible. - Torperf is our performance measurement tool. You know it from a few years ago when you wrote a Python controller for it. It hasn't changed much since then. I started rewriting it in Twisted, but that's still work in proress. I'm afraid if Torperf breaks, we'll have to fix it. People care about the performance data we gather. - TorDNSEL produces exit lists that we download and archive. If TorDNSEL breaks, metrics-db starts complaining, so we learn that it needs to be fixed. It's always unclear who's going to fix it, but in the end there's always somebody who does. - BridgeDB exports bridge pool assignments that we sanitize and archive. There's currently a bug in BridgeDB making these files not as useful as they could be (#9264). Bugging the BridgeDB maintainer(s) about these things is part of the metrics service operator job, because we care about archives being complete. And finally, here are the services that I'd say are not part of the fallback metrics service operator job: - Atlas is already co-maintained by Philipp since the dev meeting, and he does a great job there so far. - Compass isn't really something I operate. I don't know enough about the web part behind it to fix it if it breaks horribly. I just deploy patches after doing a quick plausibility check, and if they break, I revert. I'd say if it breaks and I'm not around, leave it broken until somebody else fixes it. - DocTor may soon be replaced by a new tool that Damian writes and that will use the new descriptor fetching stuff in Stem. Ideally, we'll shut down DocTor in two months from now, so that's nothing to worry about from a metrics service operation point of view. What did I miss? So, how do we estimate how much time it will take you to get started? Depends on how certain your want to be that you'll be able to handle things if they break. In a perfect world, I'd say that you should set up each service on a different machine and run it for a while. And when we get new hardware in a month or two, we should move services together. I'm aware that this requires a lot of work on both your and my side to understand the code, make it easier to install, and document everything better. But this would also improve the tools a lot. In a maybe more realistic world, I'd say that you should take a look at the codebases and at the service directories on yatei, and then we discuss why things are set up as they are. This should also put you into a fine position to rescue broken services, though you might have to do more research once that happens. Maybe there are more approaches? What do you prefer? Oh, and to be honest, I didn't apply for the Mars thing. ;) All the best, Karsten [0] http://newsfeed.time.com/2013/05/09/78000-people-apply-for-one-way-trip-to-… [1] https://metrics.torproject.org/tools.html

1 0

Cookies to third parties?
by joost＠plaintext.ath.cx 08 Aug '13

08 Aug '13

Hi List, Thanks for this nice software. Using it for quite a while now, supporting it via exit-nodes and bridges. But now I came across one strange thing I would like to ask you. I have just tried the orbot/orweb v2 on Android (4.2), which works. But if you go to the website of the jondonyn-project, http://ip-check.info/ and do the test there, it says that third-parties may set cookies? Is this really true? (I even disabled cookies alltogether. But the test yields the same "problem") Is the test on the website wrong? Thanks Joost

1 0

Re: [tor-dev] visionion: defaults for import data
by Karsten Loesing 08 Aug '13

08 Aug '13

(Moving this discussion to tor-dev@ with Thomas' permission.) On 8/7/13 6:04 PM, thomas lörtsch wrote: > hi karsten, > > to reduce complexity in the mapReduce script it would be helpful if every field of import data always had at least a default value. > > for importRelays that would be: > addd "" > node "" > span 0 > date "" > nick "" > role [] > flag [] > bwa 0 > bwc 0 > tsv "" > osv "" > pbr 0 > pbg 0 > pbm 0 > pbe 0 > pex [] > as 0 > cc "" > > > for importBridges: > addd "" > node "" > span 0 > date "" > nick "" > bwa 0 > bwc 0 > tsv "" > osv "" > brp "" > bre 0 > brt [] > > > and for importClients: > addd "" > span 0 > date "" > cb 0 > cbcc [] > cr 0 > crcc [] > cpt { obfs2: 0, obfs3: 0, OR: 0, Unknown: 0 } > cip { v4: 0, v6: 0 } > > > I hope I got that right. if you've got better ideas for default values of arrays and objects please tell me. Done, except that I put in empty objects {} for cbcc, crcc, cpt, and cip as default values, as discussed on IRC. Please find the updated .json files here: https://people.torproject.org/~karsten/volatile/bridges-2013-08-08.json.gz https://people.torproject.org/~karsten/volatile/clients-2013-08-08.json.gz https://people.torproject.org/~karsten/volatile/relays-2013-08-08.json.gz Best, Karsten

1 0

How the pluggable transports factory works (or: How to deploy your very own pluggable transport)
by George Kadianakis 08 Aug '13

08 Aug '13

--How the pluggable transports factory works-- Pluggable transports make Tor harder to detect. They are "pluggable" and Tor should be able to support a variety of transports [0]. For this reason, it should be easy to write new pluggable transports. Writing a pluggable transport involves understanding how Tor bridges work and how the Tor pluggable transport interface works [1] . Software like obfsproxy and pyptlib (both in Python) might help to abstract away the details of the pluggable transport interface [2]. If you plan to write a pluggable transport, here are a few things that you might want to think about: - Before you start coding. If you have a good pluggable transport idea, it would be wise to contact us before spending weeks hacking on it. We might know someone who is already working on a similar concept, or we might be able to help you with the design/implementation of your transport. Send an email to tor-dev(a)lists.torproject.org (note that tor-dev is a public list. Use tor-assistants(a)torproject.org if it's your sekrit research project). Even before coding, you should have a clear threat model in mind for your transport. You should also compose a clean and readable protocol specification document. See [3] for some specs and threat models of already existing pluggable transports. It's up to you to decide on a programming language for your pluggable transport. We have currently deployed transports written in Python, C and Go. We don't really care which language you end up using, as long as it's multiplatform (works on Windows) and you can write secure and comprehensive code with it. A pluggable transport should not make users less safe. Note that this means that if you write your transport in C or C++, we will spend much more time nitpicking on your code so that it feels secure. Worst-case-scenario we might ask you to rewrite your transport in a more confidence-inspiring language. Finally, check out how already existing transports have been deployed. See how their torrc looks like, how Tor communicates with them, etc. Tor expects a certain interface from pluggable transports, and your transport should be able to follow it. If your idea can't work as a Tor pluggable transport, then contact us and we can think of ways to bend it into shape. - While you are coding While you are writing your pluggable transport, try to be conservative on the libraries you use. Are you coding in Python and you want to import PySketchCrypto? Make sure that the library is widely used and its code is clean and secure. Feel responsible for the libraries you choose. Try to think of the end goal while you are writing your pluggable transports. Make sure that the networking architecture you are using is compatible with the one Tor expects from pluggable transports. Don't forget to actually test your pluggable transport with Tor (in the same way that it would get deployed) before you conclude that "it works" (see the ClientTransportPlugin/Bridge torrc options). - After you finish coding When you have finished writing your pluggable transport, put it in a git repository. Don't forget to include the threat model and the protocol specification of your transport in your repository. Send an email to the tor-dev mailing list and let us know of your creation. - After you go public The final step is to get your transport included in the Pluggable Transport Browser Bundle. Before this happens we will nitpick on your idea, your threat model and your specification, review your code, request fixes, etc. This might be a frustrating procedure, but hopefully it will result in better and more maintainable software (most likely we will be the ones fixing any bugs in your code after it gets deployed). After your transort has reached a certain level of legitimacy it will be time to include it in the Pluggable Transport Browser Bundle. This procedure usually involves taking the latest Pluggable Transport Tor Browser Bundle, compiling your transport and your libs, adding them in the bundle, and finally signing it and releasing it to the world. Refs: [0]: https://www.torproject.org/docs/pluggable-transports.html.en [1]: https://gitweb.torproject.org/torspec.git/blob/HEAD:/pt-spec.txt https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/196-transpor… [2]: https://gitweb.torproject.org/pluggable-transports/obfsproxy.git https://gitweb.torproject.org/pluggable-transports/pyptlib.git http://pyptlib.readthedocs.org/en/latest/ [3]: https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/blob/HEAD:… https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/blob/HEAD:… https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/blob/HEAD:… https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/blob/HEAD:… https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/design.txt

1 0

Why not use The Update Framework? (TUF)
by adrelanos 08 Aug '13

08 Aug '13

> For TBB 3.0, we should use the Firefox updater. We should audit the Firefox updater for issues, and triage which of Thandy's features we should merge to it. (For example, we might want to sign the metadata file if it isn't signed; timestamp it if it isn't timestamped, add multiple-signature support, and so on.) [1] That sounds like reinventing the wheel. > Thandy was a good research platform, not a long-term piece of software we want to support. [1] Why not use its predecessor, TUF? [2] [3] TUF is written in python, and after all those years, TUF developers are still maintaining it and actively developing it. I think in future TUF will become a mature and widespread solution. Also work is being done to let pip (the python library installer) internally use TUF. So it can't be so bad after all? If you have discussed this and reasons for rejecting, fine. Just wanted to throw it in, because I think basing this feature on another active project (TUF) works better than reinventing the wheel. [1] Commenting on: https://trac.torproject.org/projects/tor/wiki/org/meetings/2013SummerDevMee… [2] https://www.updateframework.com/ [3] https://github.com/theupdateframework/tuf

2 2

augmenting RSA identities/signatures with ECC and beyond
by Jacob Appelbaum 06 Aug '13

06 Aug '13

Hi, Linus and I had an interesting discussion at IETF 87 this past week in Berlin. We're both concerned about long term Directory Authority identity keys as well consensus signing with RSA keys. We've agreed that we're interested in writing a proposal whereby we add additional identity keys for authorities. Thus, we'll have whatever security may be provided by RSA and the security that should be provided by ECC signatures. The work on ntor should directly assist us in having almost all the required crypto we'll need for such augmentation. I tend to think that every directory authority should generate an additional and new long term ECC identity key. This will require that tor-gencert is extended to understand both ECC and RSA. We'll want to add these fingerprints to src/or/config.c for each respective DA. We'll want each directory authority to sign with both RSA and ECC. We'll also want to extend the consensus format to handle publication of such signatures. Older clients should be able to parse the consensus without worry and they will check RSA signatures as always. Newer clients should check both and report a mismatch into the logs at a high level. When combined with ntor, I believe that we will have significantly improved the cryptography in Tor. It would be nice to be able to add other signature schemes - specifically for pq crypto related undertakings. In an ideal world, I'd like to be able to sign the consensus from my directory authority with RSA, ECC and some kind of djb approved, tanja tested post-quantum computer signature construct. What do you think we should consider as we draft this proposal? All the best, Jacob

2 3

Damian's Status Report - July 2013
by Damian Johnson 03 Aug '13

03 Aug '13

Hi all. This month was mostly spent on non-tor work including a server migration, bad service outage at work, and a full week of cleaning my apartment. Still, plenty of spiffy news in stem land... ====================================================================== Remote Descriptor Fetching ====================================================================== Major feature for this month was the addition of a module to remotely fetch tor descriptors... https://stem.torproject.org/api/descriptor/remote.html https://lists.torproject.org/pipermail/tor-dev/2013-July/005156.html This works much like tor itself does, downloading descriptor content from directory authorities and mirrors. With it we can now easily script against the present state of the tor network without piggybacking on a live tor instance. Curious what you can use it for? See our present monitors for some ideas... https://lists.torproject.org/pipermail/tor-dev/2013-July/005209.html https://gitweb.torproject.org/atagar/tor-utils.git/tree This also included a little work with Nick on the spec and tor side... * Dropped the unimplemented microdescriptor query from the spec. (#9271) * Noting the max queryable fingerprints/hashes in spec. (#9282) * We're getting a high failure rate on the downloads we make. A little more investigation is needed on my part to help narrow this down. (#9379) ====================================================================== Other news includes... * Revised the appearance of stem's frontpage. The blue buttons were pretty jarring, so switched to something that matches the rest of the page... http://www.atagar.com/transfer/stem_frontpage/before.png http://www.atagar.com/transfer/stem_frontpage/after.png * Added Slackware to our download page. Many thanks to Markus for adding us to SlackBuilds! http://slackbuilds.org/repository/14.0/python/stem/ * Worked with Sreenatha to port Tor Weather to stem. Unfortunately Weather does not presently have an active maintainer so I'm not sure how we will proceed on this front. (#8264) https://lists.torproject.org/pipermail/tor-dev/2013-July/005111.html * The Munich dev meeting has attracted quite a few potential volunteers. After discussing prospective projects with seros I tidied up our volunteer page. Changes included... * Added Christian's Globe project https://lists.torproject.org/pipermail/tor-commits/2013-August/060124.html * Dropped Thandy https://lists.torproject.org/pipermail/tor-commits/2013-July/059830.html * Merged TorStatus into the entry for Atlas https://lists.torproject.org/pipermail/tor-commits/2013-July/060003.html * General page corrections https://lists.torproject.org/pipermail/tor-commits/2013-August/060125.html * Our automated Jenkins test runs ran into another regression in tor that caused it to segfault. (#9298) * STREAM events mishandled IPv6 addresses. (caught and patched by soult, #9181) * Thread with Pierre about TorPylle which turned into a discussion with Nick regarding future language direction for the tor codebase. I'm looking forward to seeing where it goes! https://lists.torproject.org/pipermail/tor-dev/2013-July/005161.html * We just finished with midterms for Google Summer of Code. Chang unfortunately did not pass, but the other projects are going well. * Sorted out travel arrangements for the GSoC mentor summit. Nick and I will be going, and Moritz is presently on the waitlist. * Code reviewed ra's rttprober. He provided some great feedback for which I still owe him a reply. https://lists.torproject.org/pipermail/tor-dev/2013-July/005183.html Cheers! -Damian

1 0