- tor-dev - lists.torproject.org

[GSoC 2016] Orfox - Report 3
by Amogh Pradeep 30 Jun '16

30 Jun '16

Hey guys, These are the things that I’ve done in the past couple of weeks. 1) Finish the network audit. I’ve managed to comb through the Firefox for android code to find all the network code in the java layer. [0] The good news here is that the Java layer has now been reduced to using 2 network libraries instead of 3. This is a result of one of our patches going upstream. [1] 2) Stable build with ESR45. I’ve finally managed to get Orfox building fine on ESR45. APK [2] There are still a few quirks that we need to work out, after which we should be able to get an official build out. This APK is highly experimental at the moment, please use it just for testing purposes only. 3) Conversation with Mozilla team. I had a nice chat with mozilla about the java network code in firefox. [3] To try to find what might be useful for them as well, this way, we could send more code upstream. And it also helps with judging what we must do next. 4) Orfox Logo update. Thanks to Nathan, we have a new logo for Orfox :D [4] We had to replace our old one as it looked too much like Mozilla’s fox. Best, Amogh Pradeep amoghbl1(a)gmail.com [0] https://trac.torproject.org/projects/tor/ticket/19519 <https://trac.torproject.org/projects/tor/ticket/19519> [1] https://bugzilla.mozilla.org/1169421 <https://bugzilla.mozilla.org/1169421> [2] http://people.torproject.org/~amoghbl1/fennec-45.2.0.en-US.android-arm.apk <http://people.torproject.org/~amoghbl1/fennec-45.2.0.en-US.android-arm.apk> [3] http://logs.glob.uno/?c=mozilla%23mobile&s=28+Jun+2016&e=29+Jun+2016 <http://logs.glob.uno/?c=mozilla#mobile&s=28+Jun+2016&e=29+Jun+2016> [4] http://imgur.com/Ys502Bx <http://imgur.com/Ys502Bx>

2 1

[GSOC16] Fingerprint Central - Cookies and localStorage
by Pierre Laperdrix 30 Jun '16

30 Jun '16

Hello everyone, I know the next status report is not due until next week but I wanted to get some feedback on how I use cookies and localStorage on FP Central. Right now, due the very short lifespan of cookies in the Tor browser, I don't use cookies as an identification mechanism but as an expiration/anti-pollution mechanism. Here is how it works: - For users without JavaScript, I put a cookie the first time a fingerprint is sent. This means that in the same session, no new fingerprint from the same user will be stored if he or she wants to see his or her own fingerprint again. If the user generates a new identity, he will be able to store a new fingerprint since no cookie will be present. - For users with JavaScript, I add a cookie when the test suite is run and I use localStorage to store data generated during the collection process. When the user sees his complete fingerprint with the percentage for each attribute, all the values are stored in localStorage. If the user gets back to the FP page page, it won't have to contact the server and the fingerprint will appear instantaneously by getting the stored copy from localStorage. The presence of data in localStorage prevents the user from sending his fingerprint a second time. The presence of the cookie acts as an expiration mechanism. If the cookie is still present, I consider the data in localStorage to be valid and the user cannot perform the collection process again. If the cookie has expired, I remove data present in localStorage and allow the user to execute the whole process again. I don't know if my approach is a good one but I wanted to use what is available to me in the browser to prevent too many fingerprints from being stored and to lessen as much as possible the server load. If anyone has suggestions on my use of cookies and localStorage, I'll gladly take them. Now, what I'm not really sure is: what about users who want to play with their browser and see the modifications done to their fingerprint? The way the site is built now prevents that. There are no buttons to force the collection process again. I see two alternatives here: add a "Force fingerprinting" button even though the database could be polluted or add a sort of "playground" webpage that is not connected to the database where the user can rerun the test suite as many times as he wants. Hopefully, if everything goes well on my end, I'll be able to launch a beta version of the website next week. We could see from there if everything is working well and if the limitations imposed on users are not too important. Have a great week-end, Pierre

3 2

HSFETCH fails on basic auth services
by Razvan Dragomirescu 30 Jun '16

30 Jun '16

Hello everyone, I seem to have found an issue (bug?) with the controller HSFETCH command - I can't seem to be able to fetch hidden service descriptors for services that use basic authentication. Tor appears to want to decrypt the introduction points for some reason and also fails to look at the HidServAuth directive. Connections (via SOCKS proxy for instance) to said service work fine, so Tor is configured correctly, but HSFETCH fails and Tor outputs this in the logs: *Jun 29 20:08:53.000 [warn] Failed to parse introduction points. Either the service has published a corrupt descriptor or you have provided invalid authorization data.* *Jun 29 20:08:53.000 [warn] Fetching v2 rendezvous descriptor failed. Retrying at another directory.* Is this a known issue? Is there another way to fetch the descriptor of a hidden service? I really don't want it to be published since I'm rewriting it anyway, but I need to fetch it somehow. I can use "PublishHidServDescriptors 0" to stop it from publishing the service at all but I have no idea how to fetch it from the local cache. Any controller commands for that? To summarize - HSFETCH appears to fail for hidden services with basic auth and I couldn't find a way to obtain the hidden service descriptor from the hidden service machine itself before publishing. Any advice would be appreciated. Thank you, Razvan -- Razvan Dragomirescu Chief Technology Officer Cayenne Graphics SRL

2 3

is the consensus document unpredictable / unique?
by Razvan Dragomirescu 29 Jun '16

29 Jun '16

Hello everyone, I couldn't find a detailed description of the Tor consensus, so I'm checking that my understanding of it is correct. Basically, would it be correct to assume that the consensus document (or a hash thereof) for a date in the future is an unpredictable value that will also be unique to all nodes inquiring about it at that time? I'm thinking of using a hash of the consensus document - like http://171.25.193.9:443/tor/status-vote/current/consensus - as a descriptor cookie in a hidden service. This way, an attacker cannot generate or publish a hidden service descriptor for the future (one with a correct cookie). A client can fetch the consensus at the time it wants to connect, hash it, then use that as the descriptor cookie to determine the correct descriptor id and decrypt the introduction point list. Does anyone see any issues with this? In my project, the hidden service private key is on a smartcard, so it can't be copied, you can only ask the smartcard to sign something with it for you - and I'm trying to prevent an attacker from generating hidden service descriptors in advance,to be used without the smartcard. If future descriptors depend on an unpredictable future value (the hash of the consensus at that time), an attacker can only generate descriptors for past and current time periods. Thank you, Razvan -- Razvan Dragomirescu Chief Technology Officer Cayenne Graphics SRL

5 14

[GSoC '16] Status Report #2 - Exitmap Improvements Project
by Mridul Malpotra 27 Jun '16

27 Jun '16

Hey everyone. This is my second status report for the Exitmap improvements project. These 2 weeks, I was building up on the browser automation aspect to allow for browser interactions on Exitmap scans. These will better help disguise scanner activity against malicious exit nodes that might not act like an adversary when they detect scanner-like behavior in the client. Work and details: ----------------------- 1. I am using the Selenium WebDriver Python bindings to help in simulating browser actions. Have implemented some simple actions in Selenium and have also taken inspiration from the Tor Browser Selenium library suggested by the community. 2. Have included actions like opening a random number of links on the webpage in the same page, new tab or another window. Tried to get clickable elements of a webpage but facing some issues with dynamic pages where buttons do not enabled instantly. 3. Added browser_simulation.py that allows for a module to import the functions use the simulate() function with the intended URL to performs these operations. Some problems in these functions with some websites. Could be due to dynamic content. Needs to be fixed. Problems faced: ---------------------- 1. Major issue that I am facing is with the multiple invocations of the module for every exit node. The asynchronous calls are throttled due to the more serial Selenium calls to the TBB driver. I have not found a working solution for this as of now, which makes the tests running with browser automation to be much slower. 2. For some operations like opening tabs, I could not find defined library calls, leading to more hackier ways to complete these operations. This often leads to unexpected behavior and needs to be fixed. Future plans: ----------------- As continuous scanning is more important toward the project, I will be focusing on that for the moment and revisiting browser automation after I complete the former. My next report is due on 1st July. I also have submitted the report a week late due to unexpected complete disconnect from internet in the area I was at that moment for 6 days. This caused my work to stop for some time and I apologize for the irregularity. Immediately having shifted to a metropolitan city, I will ensure my reports will now be on time and work will progress without hindrance. Regards, -- Mridul ============================= PGP keyID: 0xb716e33ab6d0a653

1 0

TorBirdy 0.2.0: Sixth Beta Release
by Sukhbir Singh 27 Jun '16

27 Jun '16

We are pleased to announce the sixth beta release of TorBirdy and the first in the 0.2 series: TorBirdy 0.2.0. All users are encouraged to upgrade as this release fixes numerous security and privacy issues. Notable changes include fixing local timestamp disclosure in the date and the message-ID headers, as detailed in tickets #6314 [0] and #6315 [1]. The patch for sanitizing the date header is shipped [2] with TorBirdy. The patch for the message-ID header was submitted upstream [3] to Mozilla and merged in Thunderbird 45, and it is therefore recommended that you upgrade to Thunderbird 45 if possible. There are currently no known leaks in TorBirdy but please note that we are still in beta, so the usual caveats apply. If you are using TorBirdy for the first time, visit the wiki [4] to get started. Other changes in this release include: 0.2.0, 27 Jun 2016 * Bug #6314 [5]: Prevent local timestamp disclosure via Date header * Bug #6315 [6]: Prevent local timestamp disclosure via Message-ID header * Bug #13721 [7]: Fix usage of wrong locale * Bug #17426 [8]: Allow configuration of default email protocol * Bug #15459 [9]: Add support for deterministic XPI generation * Bug #11387 [10], #13006 [11]: Fix non-standard EHLO argument * Bug #17118 [12]: Allow manual account configuration for Gmail with OAuth2 * Bug #19031 [13]: Add and audit support for RSS reader * Bug #7847 [14]: Audit and update support for NNTP * Bug #10683 [15]: Update Thunderbird UI to reflect TorBirdy's state * Bug #19330 [16]: Set secure defaults for outgoing mail servers * Removed compatibility for older versions of Thunderbird and added support for Thunderbird 37+ * Added support for automatic configuration of Riseup email accounts * Updated various privacy and security settings (see commit 2bdeffbb [17] for a list of the changes) * Update translations for current languages Many thanks to Arthur Edelstein and the Tails Developers for this release! We offer two ways of installing TorBirdy -- either by visiting our website [18] (GPG signature [19]; signed by 0xB01C8B006DA77FAA) or by visiting the Mozilla Add-ons page [20] for TorBirdy. Please note that there may be a delay -- which can range from a few hours to days -- before the extension is reviewed by Mozilla and updated on the Add-ons page. (Packages for Debian GNU/Linux will be created and uploaded shortly.) [0] - https://bugs.torproject.org/6314 [1] - https://bugs.torproject.org/6315 [2] - https://gitweb.torproject.org/torbirdy.git/commit/?id=bfa822de6071e894dad3d… [3] - https://bugzilla.mozilla.org/show_bug.cgi?id=902580 [4] - https://trac.torproject.org/projects/tor/wiki/torbirdy [5] - https://bugs.torproject.org/6314 [6] - https://bugs.torproject.org/6315 [7] - https://bugs.torproject.org/13721 [8] - https://bugs.torproject.org/17426 [9] - https://bugs.torproject.org/15459 [10] - https://bugs.torproject.org/11387 [11] - https://bugs.torproject.org/13006 [12] - https://bugs.torproject.org/17118 [13] - https://bugs.torproject.org/19031 [14] - https://bugs.torproject.org/7847 [15] - https://bugs.torproject.org/10683 [16] - https://bugs.torproject.org/19330 [17] - https://gitweb.torproject.org/torbirdy.git/commit/?id=2bdeffbb [18] - https://dist.torproject.org/torbirdy/torbirdy-current.xpi [19] - https://dist.torproject.org/torbirdy/torbirdy-current.xpi.asc [20] - https://addons.mozilla.org/en-us/thunderbird/addon/torbirdy/ [Also on https://blog.torproject.org/blog/torbirdy-020-sixth-beta-release] -- Sukhbir

1 0

Help complete the list of official & community projects
by Nima Fatemi 27 Jun '16

27 Jun '16

Hi there, I'm working with some folks to compile a list of most active official and community projects we have. There are some projects listed on our volunteer page[1] but I'm not confident if the list is complete or even up to date. Please have a look at that page and let me know if your project is listed and it's not up to date, and definitely let me know if it's not listed at all. So if you've recently wrote a Tor library in a language only 5 people on this planet know about, this is a good chance for you to let us know of your project. :) Here's the information I need from you: Name of the project, along with a paragraph explaining the significance of the project and why it matters. Link to the project, and name or the handle of the maintainer. Examples: ====================================================================== ## Stem Stem is a Python controller library for Tor. With it you can use Tor's control protocol to script against the Tor process. https://stem.torproject.org/ maintainer: atagar ## OONI A free software, global observation network for detecting censorship, surveillance and traffic manipulation on the internet. https://ooni.torproject.org maintainer: ooni-team ====================================================================== [1] https://www.torproject.org/getinvolved/volunteer.html.en#Projects Alternatively we can create a new wiki page just for this projects, until #16576 is resolved. Except I'm only looking for active projects at this point. Happy Friday! -- Nima 0X58C4B928A3E218F6 | @mrphs "I disapprove of what you say, but I will defend to the death your right to say it" --Evelyn Beatrice Hall

8 7

RB Trees
by U+039b 25 Jun '16

25 Jun '16

Hello! I had a discussion with David Goulet about RB trees [0]. More particularly about the usage of RB trees for some Tor operations like [1] or HSDir cache. David said that Nick wants to use a FreeBSD implementation - is this one[2]? If it is, I think it could be hard for new contributors like me to maintain/write/debug code using it. Since inline functions are as fast as macros [3], is it conceivable to use functional implementation instead of macro implementation? If it is, I can propose a generic C implementation of RB tree. It seems that Tor uses some kind of hashmap [4]. What about using hashmaps with RB tree as bucket and thus ensure bounded time complexity? In some cases, it could be interesting to store RB tree nodes in hashmap to ensure lookup in O(1) while keeping entries sorted. Regards, L. [0] https://en.wikipedia.org/wiki/Red%E2%80%93black_tree [1] https://trac.torproject.org/projects/tor/ticket/13739 [2] https://github.com/freebsd/freebsd/blob/master/sys/sys/tree.h [3] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [4] https://gitweb.torproject.org/tor.git/tree/src/common/container.h

1 0

Freenet + Onioncat: Is the traffic welcome?
by konstant＠mail2tor.com 25 Jun '16

25 Jun '16

I posted steps on how to connect Freenet nodes over Onioncat and Garlicat for Tor/I2P. I am looking to scale it into an Opennet inside Tor with a lot of peers: https://emu.freenetproject.org/pipermail/devl/2016-June/039056.html https://emu.freenetproject.org/pipermail/devl/2016-June/039059.html Is the extra traffic desirable in Tor? Reading asn's comment, I was under the impression that you are interested in adding higher latency traffic such as Freenet or mixnets for better anonymity: https://blog.torproject.org/blog/crowdfunding-future-hidden-services Using both projects in tandem can finally realize the vision of FreeHaven. You are the best at firewall circumvention, performance and accessing the web, Freenet supplies users with censorship resistant publishing and p2p services. There is a HotPETs 16 paper co-authored by George Danezis on renewing interest in anonymous storage networks: http://www0.cs.ucl.ac.uk/staff/M.Isaakidis/p/isaakidis-p2pstorageservices-h…

4 8

We need a new RPM maintainer
by Nick Mathewson 24 Jun '16

24 Jun '16

Hi, all! Ondrej Mikle, who has built our RPMs for a while, is no longer able to do so. (He isn't using RPM distros any more, doesn't have as much free time, and etc.) First: Thanks, Ondrej, for all your hard work! Here's the instructions he sent on how to do it (quoted with permission): {{{ Basically every part of the process is documented here: https://gitweb.torproject.org/user/hiviah/rpm-build-scripts.git/tree/README It's maybe not the best README, because the actual merge-and-build part is in the middle (but I always use it, otherwise I'd just forget all the necessary steps). Nevertheless, the introduction maybe necessary for someone not familiar with the process. The process in the short is: 0. have the environment ready (has to be done just once) 1. clone my own RPM fork which has RPM changes (https://gitweb.torproject.org/user/hiviah/rpm-tor.git) 2. Merge and build according to "MERGING REPO USED FOR RPM BUILDING FROM ORIGIN" 3. Download, review, sign the built RPMs (I used feddei.tpo VM for this, but you can have your own VM as I had last time since feddei.tpo was down) 4. upload to palmeri.tpo 5. distribute to everyone else via "static-update-component deb.torproject.org" on palmeri.tpo I could walk somebody through this, as I said, this is not rocket surgery, mostly attention to detail :) If there ever was a critical need for a special build due to some unforseen bug, I could still do it, but this is for exceptional case only. }}} The easiest way for RPMs to happen would be if somebody who already has a Tor LDAP account wants to step forward and volunteer for a while. It would be harder if this were somebody that we didn't know who volunteered: until our RPM process creates reproducible builds, we need to have some amount of trust in the person who creates them. There may be other issues here too. I am no packaging expert. :) best wishes, -- Nick Mathewson

6 7