- tor-dev - lists.torproject.org

Reminder: tor development meetings this (and every) Wednesday, 19:00 UTC
by Nick Mathewson 10 Jun '14

10 Jun '14

Here's your regular reminder for the weekly IRC meeting for people working on the program "tor". (This won't cover all the other programs developed under the Tor umbrella.) The next meeting time will be: Wednesday June 11, 19:00 UTC. (That's 3pm EDT and 12:00 noon PDT.) As usual, we'll do it on the #tor-dev IRC channel. I've seen a drop-off in the number of people coming since I stopped sending these out weekly, so I'm sending them again for a while. Among other stuff, I expect we'll be wrapping up (or planning to wrap up) 0.2.5.5-alpha and maybe doing some initial strategizing on 0.2.6. cheers, -- Nick

1 0

GSoC: BridgeDB Twitter Distributor report
by Kostas Jakeliunas 10 Jun '14

10 Jun '14

Hey all, in the past weeks I've been working on understanding what can be done using Twitter APIs and its media support in its CDN (for a later captcha implementation), as well as on improving my existing Twitter bridge distributor bot PoC. I've written some broken code, but it's alright. More details below. Distributor bot improvements included working on adding a churn rate control mechanism which securely stores Twitter user IDs (with code and design ideas from BridgeDB's HMAC approach to remembering e.g. email addresses in the EmailDistributor), and implementing a (mostly) bogus text-based challenge-response system (this is mostly so that we have a generic design for doing challenge-responses in this distributor - we'll be able to later on replace it with a decent CAPTCHA, for example. It's just nice to have a generic system and a thing for testing out the bot, etc.) I've also looked into using isis' new and shiny BridgeRequest objects to process user (well) 'bridge requests' in a non-hacky way; this should also eventually result in a bridge request syntax compatible with (a subset of) GetTor commands. But I still need to figure out the best way to use BridgeRequests, so nothing interesting to show yet. TODO * (still yet to) summarize a nice meeting i've had with sysrqb and isis. No definite conclusions were made, but there were (iirc) some nice ideas about a generic BridgeDB API that could be used by third party components, etc. (i.e. it might be worth pursuing it even if the Social Distributor is to be implemented at some later point.) * clean up my mess, test new code not to fail, and push new things onto https://github.com/wfn/twidibot/ (current (old) code there does work, if anyone's curious to run it) * figure out BridgeRequests, the new IRequestBridges (ha!) interface, and use these in the twitter bot * be able to 'serve' the bot fake bridge data so it could process it in a way that may be compatible with a future BridgeDB API (i.e., hopefully this bot will be able to run as a third-party-thing, separate from core bridgedb. This is hopefully how future distributors will/should work.) This way the bot will be more/actually 'realistic' in the way it serves current bogus bridge lines to users. (I thought I'd have this by now, but I don't. Hrm.) * continue looking into captcha systems modulo what can be used in the twitter context * look into bridgedb buckets and what I can help re: them, so the bridgedb API could happen sooner than later. (Old todo list item, did not yet touch it.) All in all, need to write more non-broken code, fewer words, and just continue with the current bot. Have a nice day/night/thing! -- Kostas. 0x0e5dce45 @ pgp.mit.edu

1 0

[GSoC 2014] Revamp GetTor: Send HTTP links for downloading TBB
by Israel Leiva 10 Jun '14

10 Jun '14

Hi! I've been discussing with my mentors (Sukhbir and Nima) the option of sending HTTP links to download TBB via GetTor. The main reason for this would be that in some censor countries is very difficult to download a "big file" (>= 10 Mb) over SSL, so HTTP links would be needed. Obviously, the user would be warned about the lack of SSL and that he/she *must* verify the integrity of the bundle. On the other side, bundles downloaded over HTTPS should also be verified if you need maximum security, since we can't trust in the cloud services neither. (e.g. Dropbox). Since this is not an easy decision, we've decided to discuss it publicly. Any thoughts on this? What's your experience regarding this matter? All feedback is welcomed. In the meanwhile we'll keep considering HTTPS links only. Regards. -- israel

2 2

GSoC: Multicore daemon status report
by Белоус Михаил 10 Jun '14

10 Jun '14

Hi all, As start of my project, I made patch to work queue, making thread pool more efficient and wrote some benchmarks. Future work 1.Rewrite architecture of tor circuit processing, to use thread pool 2.Write benchmarks showing, that multi core implementation efficient, or not? Most of my work can be found at https://github.com/towelenee/tor. Please feel free to provide feedback of any sort. Cheers, Mikhail

1 0

[announce] Tails HackFest, July 5-6, Paris
by intrigeri 08 Jun '14

08 Jun '14

Hi, I'm looking forward to see quite a few of you there! Join us at the Tails HackFest, 2014! July 5-6, 2014 -- Paris, France Description and goals ===================== Join us to make online anonymity and digital privacy usable by the masses! Whether you're a writer, a software developer, a designer, a system administrator or just plain interested, come learn about the challenges faced by Tails, and how you can be part of the solution. The Tails HackFest will bring together anyone interested in making Tails more usable and more secure. This open event will be an intense mix of teaching, drawing, coding, sharing, learning and celebrating. Logistics ========= * Where: the venue for the event is IRILL, Paris, France (https://www.irill.org/about/information-for-guests) * Dates: Saturday, July 5, 2014 - Sunday, July 6, 2014 * Time: 10 AM - 10 PM * Registration: if you want to attend, please consider dropping us a note about it. This is optional, but would help organizing this event. * Contact: <tails-hackfest-2014(a)boum.org>, #tails-hackfest on irc.oftc.net * Details, scheduling and updates: https://tails.boum.org/blueprint/HackFest_2014_Paris/ Host and sponsors ================= Many thanks to Debian, IRILL, Mozilla and the Tor project for supporting this event! Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc

1 0

Re: [tor-dev] Introducing CollecTor
by Karsten Loesing 08 Jun '14

08 Jun '14

[Replying on list with Damian's permission.] On 06/06/14 17:21, Damian Johnson wrote: >> I'm currently extending metrics-lib to fetch descriptors from CollecTor. >> Maybe Damian extends Stem to do the same (or accepts patches for that >> purpose). > > Probably obvious, but what's the advantage of this over fetching new > descriptors from authorities and mirrors? What kind of CollecTor > methods did you have in mind? If you're only interested in new descriptors, then the authorities and mirrors are the place to go. But if you care about having the full history of descriptors, then you should go to CollecTor. Imagine that your tool might break for a few hours. There's no way to get the missing data from the directory authorities or mirrors anymore. For applications like ExoneraTor, where you need to store every single consensus, this can make the difference. The same holds for Onionoo and the metrics website, though it's less critical to miss a few hours. Also, if you only care about relay descriptors, then you should be fine asking directory authorities and mirrors. But sanitized bridge descriptors, exit lists, bridge pool assignments, and Torperf measurements are only available from CollecTor. Note that there's also a disadvantage of fetching from CollecTor: it's a single point of failure. Now, we could add mirrors, but it would be even better to set up a second CollecTor instance (ideally run by another person) in parallel to the current first instance, and have the two exchange data. Future work, on my list. I guess the above should be added to the CollecTor homepage. Do you have any suggestions where you would have expected to read about this, or do you have further points to add? I'll add something next week. All the best, Karsten

1 0

[GSoC] Consensus Diffs - First report
by Daniel Martí 08 Jun '14

08 Jun '14

Hello everyone, This is my first status report regarding my GSoC project, Implementing consensus diffs. We weekly meetings with Nick and Sebastian, my mentors, every Wednesday at 14h UTC, if you want to take a closer look at the project's progress or want to participate. In the application timeline I specified that I would spend the first weeks coding the server side part, which is mostly just the generation of diffs. And that's what I've been doing thus far. I'm a bit behind on schedule due to exams, but I'm not too far off. I first implemented a dynamic algorithm, quadratic in both time and space, which seemed to work fine. Of course it didn't work for consensuses, since that would mean an allocation of nearly 3GB. So the next thing I did was switching to Hirschberg's algorithm, which still takes quadratic time but only takes linear space due to a divide-and-conquer approach. But that still leaves us at a horrible 16 seconds of runtime to generate a regular microdescriptor consensus diff, whereas 'diff -d' only takes a fifth of a second. That is largely due to the fact that we're not taking advantage of the consensus properties yet. So what I will do next is improve the algorithm, which should make it very close to linear in time. The idea is to navigate the two consensuses, looking at the digests that will already be sorted for us. That's part of the 'r' line for descriptor consensuses, and the 'm' line for microdescriptor consensuses. This should take linear time. Now, the part of Hirschberg's algorithm that makes it quadratic in time is finding 'k', the column at which the problem can be split in half wihtout losing precision. This is well explained here: http://wordaligned.org/articles/longest-common-subsequence But once we are navigating the nodes ordered by digests, we can avoid having to calculate 'k'. Given that we are at any pair of matching digests, we can assume that this is already a point 'k' because all the nodes that are before the digest on the first consensus can only be found before the digest on the second consensus, and the same goes for later digests. So the final algorithm should travel both consensuses in linear time, and use our quadratic time algorithm on the small chunks between these matching digests. The quadratic time algorithm would at worst be used for maybe a couple hundred lines, a size at which it still performs fairly well. You may be wondering why I went with an O(MN) algorithm when I could have gone with the more common O(ND) algorithm, which I understand is the one used in the GNU diffutils. The main reason is the use-case. O(ND) performs worse than O(MN) when the differences are large, because then D=2N. In our case, when we are running the algorithm it's because nodes disappeared, appeared or simply to find changes in nodes that are still there. So it will be very common that we find diffs D that are very close in size to N or M. Plus, as far as I understand, the divide-and-conquer algorithm is pretty clean and easy to understand while the O(ND) algorithm by Myers would probably be harder to implement in a clean way. I'm in no way an expert in diff/lcs algorithms, so suggestions and comments are very welcome. -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

1 1

DetecTor Review
by Damian Johnson 07 Jun '14

07 Jun '14

Hi Kai. I just took a quick peek at your DetecTor project (http://detector.io/) It looks like the project has been inactive for a while so just some quick thoughts in case it sparks back up. > log_setting = "notice file " + data_dir + os.sep + "tor.log" The following might be nicer... log_setting = "notice file %s%stor.log" % (data_dir, os.sep) Same goes for most instances of string concatenation, actually. > 'StrictExitNodes': str(1), Interesting, never seen this before. Should be the following instead. 'StrictExitNodes': '1', > def stop_tor(control_port): > tor_pid = stem.util.system.get_pid_by_port(control_port) > if tor_pid: > os.kill(tor_pid, signal.SIGTERM) Certainly a fine approach. Would you like to avoid having Tor outlive this python process? If so then that's as easy as including 'take_ownership = True' in your launch_tor_with_config() call. If not then you can use the process object returned by launch_tor_with_config() to kill it as well... tor_process = stem.process.launch_tor_with_config() ... do stuff... tor_process.kill() Also this method has tabs mixed with spaces. I would highly recommend strictly using spaces - python is whitespace dependent, and occasional tabs can make things a lot more confusing. > def is_tor_running(control_port): > return bool(stem.util.system.get_pid_by_port(control_port)) I believe you can use the poll() method of the process object to make this a little less platform dependent. > if os.path.isdir(dir_name) != True: You never need to compare with booleans in an if statement. This is equivalent to... if not os.path.isdir(dir_name): > config.readfp(open('sphere_control.config')) Minor thing but this doesn't properly close the file handle. It's probably done on its own during GC, but it would be more proper to do... with open('sphere_control.config') as config_file: config.readfp(config_file) Honestly I'm really not sure what the following code is doing. I suspect there's a lot of low hanging fruit for improving this (for instance using "', '.join(countries_list[i])" instead of tracking a 'countries_string') but first step would be some comments explaining what this is trying to achieve. Cheers! -Damian

1 0

GSoC : Rewrite Tor Weather (Status Update 1)
by Sreenatha Bhatlapenumarthi 07 Jun '14

07 Jun '14

Hi all, So here is my first update on the work regarding weather rewrite. 1. (#12085) Develop python wrapper for Onionoo services. Made an initial python wrapper to be used for fetching Onionoo documents. It currently supports request-response caching to decrease the latency(picked up from OnionPy). 2. (#11956) Wrote a simple script for searching for relays matching user's query. This functionality is to be implemented in the rewritten weather as well. Modified it to use the new Onionoo wrapper instead of 'urllib' module for fetching response from Onionoo. 3. (#9889) Rewrote the previous tshirt script, making use of the new Onionoo wrapper. Future work 1. Add and expand the features provided by the Onionoo wrapper. Currently in the process of adding utility methods for backend scripts that use Onionoo data. 2. Test the Onionoo wrapper 3. Write the remaining backend scripts(hourly, daily) etc described in the wiki at https://trac.torproject.org/projects/tor/wiki/doc/weather-in-2014 Most of my work can be found at https://github.com/lucyd/tor-weather. Please feel free to provide feedback of any sort. Cheers, Sreenatha

1 0

Pluggable Transport Combiner Status Report 1
by quinn jarrell 07 Jun '14

07 Jun '14

Hi everyone, I'm the GSoC student working on the the Pluggable Transport Combiner. So here's what I've been working on: - I added in a configuration class and file reader to obfs-flashproxy client. - The obfs-flashproxy client can now launch any number of transports in a row. - It chains together the pluggable transports with SOCKS servers What I'll be working on next: - Documentation on the design - Making the combiner intercept the first tor connection instead of recieving the first pt's output as it does now - Testing authenticated transports. If you've got any questions or comments shoot me an email! Quinn Jarrell

2 1