- tor-dev - lists.torproject.org

[GSoC '13] Status report - Searchable metrics archive
by Kostas Jakeliunas 23 Sep '13

23 Sep '13

Hi all, TL;DR of TL;DR: all good as far as GSoC scope goes; work will continue; the live torsearch backend has been updated; will update API doc today. TL;DR: - updated database / imported data up until current date - hourly cronjob continuously rsync's latest archival data, imports to DB (+ tested and deployed): * had some trouble ensuring no duplication but also no excessive DB querying (the intermediary 'fingerprint' table (that basically solves the DB bottleneck problem) was previously updated in semi-manual-mode); solved with a kind of an 'upsert'; works well; * made switching from archival data import process to the rsync'ed ::metrics-recent import seamless (data can overlap, importer will take care of it; import speed is good) - Onionoo API improvements: * date range queries for all three document types currently provided * the network status entry document was expanded/improved, as per previous discussion and Karsten's feedback * yet to finish off IP address and nickname summarization - for now, providing two document versions - original and 'condensed' (valid-after ranges) * updating Onionoo API doc soon (sorry - had meant to update until now.) - caching: * tried some things, settled for a semi-vanilla python-level-caching solution for the time being, some testing was done * have yet to still work out some snags, not deploying for now * more (pure json response dumping to disk, etc.) can be done later on - other things: * wrote a working stub / minimal version for a DB-maintenance-from-within-torsearch component (e.g. for a full VACUUM with no DB access) * other small things I may have forgotten to mention General picture: the metrics archive / torsearch is in decent shape and an appropriate (to gsoc) stage of development. Development will continue as we're not done here yet, but the archive is in a functioning state, delivering the minimum functionality required. I will finish cleaning up before the hard pencils down date; and then continue development (at whatever pace). :) The things that were blocking before are no longer blocking, so ironically, I think I'll now be able to write more code i.r.l. to torsearch than over the summer. Anyway - I've been working on things falling within the GSoC scope, finishing the most urgent matters that were left re: gsoc milestone - quoting from my previous status report re: what to do next (see the bullet point list above for a concise no-ramble-mode version of all this): > update the database to the latest consensuses and descriptors Done. However, > import 2008 year data, etc. I had parts of 2009 data in previously while working on DB bottlenecks, but presently, the statuses still start at 2010; this got pushed down the priority lane and takes a lot of (passive) time. Will be able to batch-import now and confirm. > turn on the cronjob for rsync and import of newest archives Done. [0, 1] What kept me a bit busy was integrating the third helper/mediating 'fingerprint' table (we query this one before extracting things from the massive 'statusentry' table (which contains all network status entries)) into the batch_import_consensuses process used for import. <details> I had some problems making sure data in the fingerprint table is not duplicated and contains latest info (last valid-after etc), while minimizing the amount of work and queries needed to test for row/entry existence, etc. Solved with an actually simple hybrid 'upsert' approach [2], all is well (did some testing, OK, but not extensive.) Improved the archival importer for it to be able to process duplicate consensus documents from different directories (hence they're bypassing Stem's persistence file check - so need to take care of duplicated processing ourselves) - this might happen in production when, after massively importing data from the downloaded archives, we switch to rsync's ::metrics-recent/relay-descriptors folder for import. Consensuses may overlap (they overlapped when I did the switch.) When passed down a consensus document by Stem (so after checking for duplication in Stem's persistence file), we simply check whether we already store this consensus, using a separate 'consensus' table, which only stores consensus document validity ranges (this is fast and works nicely). [2] </details> > [from an older report] Onionoo date range queries Done. [4] Works for all three document types (details, summary, statuses); can be used together with offset+limit, etc. Smart/decent datetime parsing - can pass '2011-07-28 12:00:00', or '2011-07', or '2011', etc. Example: http://ts.mkj.lt:5555/details?search=200.&from=2010&to=2012 > caching Tried all kinds of options, and since I wanted to have something decent and stable working for now, opted for a very simplistic stock python-level caching approach (no onionoo-like 'JSON documents to disk' thing yet.) But I've yet to work out some things, delayed deploying code and live version. > hopefully later integration with the bandwidth and weights documents in Onionoo proper Out of current scope; hopefully will be able to work on this next / soon. > [from an older report] Documentation, specification/implementation document Not finished yet - not very good. I should try more of this "publish early and often" thing. > expand the list of fields contained in the three [...] documents Tried things, but e.g. providing assigned flag data was pushed to later. Current live backend not changed in regards to this. > [from an older report] rewrite/improve the network status entry document / improve the 'statuses' API point Done [7] (deployed changes, will update API doc - this is not a nice thing to do (i.e. doc being not updated), but since it's not production yet, deployed early) For now, we are providing two network status document versions - the original one (+ 'relays_published'), and a condensed one (?condensed=true). The latter basically zips up all the valid-after values into ranges, as per Karsten's suggestion, basically telling the client where any gaps in consensuses were present (which may turn out to be a rather useful thing, by the way.) This works well together with from..to date ranges, offset, and limit. I'm yet to finish off IP address and nickname summarization - for now, when in 'condensed' presentation mode, each range contains its last addresses and nickname. Example: http://ts.mkj.lt:5555/statuses?lookup=F397038ADC51336135E7B80BD99CA38443602… Also, as torsearch might one day be deployed on a machine such that the maintainer won't have direct access to the DB server, we'll need to do VACUUM FULL et al. from within torsearch. It is advisable to do a full VACUUM or even a REINDEX now and then, especially after a ton of entries is added (say, after a massive/bulk data batch-import.) Wrote a working stub for a 'torsearch/DB maintenance' component (for vacuum full, for now.) [8] Other maintenance things may be added to this separate file/component later on. I think that's all for now. Hopefully I'm past the 'experiment with all the things!' stage (I've still been trying and tinkering with stuff / different approaches to problems; but for what it's worth, the current codebase and the live backend are in decent shape.) Comments / suggestions / barbaric yawps? [0]: https://github.com/wfn/torsearch/commit/b9a31fdabc52bca1baa00064359c6f691af… [1]: see e.g. http://ts.mkj.lt:5555/details [2]: https://github.com/wfn/torsearch/commit/74cb2b19e1a5b47fa95b62df8d89e0c9fb2… # there is no 3. It's.. gone! [4]: https://github.com/wfn/torsearch/commit/5f862930b81ad34e04d6315c6c57bfd3cb4… # more numbers have mysteriously disappeared! [API doc commit would go here] [7]: https://github.com/wfn/torsearch/commit/fa2183bb9fb58bff5bc1469fa947367ecd0… [8]: https://github.com/wfn/torsearch/commit/0f24fee4fb73a12fd088f0d1e271dbdb056… -- Kostas (wfn on #tor-dev) 0x0e5dce45 @ pgp.mit.edu

1 1

Tor on the Beaglebone Black
by Joshua Datko 22 Sep '13

22 Sep '13

I've been running a relay on a Beaglebone black for about two weeks now and I'm wondering where to take this thing. The Beaglebone black is a 1GHz armhf System on a Chip, with 512 MB RAM. At the moment, it's a dedicated relay running Ubuntu (raring) but Debian is equally as easy to setup. I have a post with very basic installations: http://datko.net/2013/08/24/beagleboneblacktor/ So, what next? I looked at the torrouter project and I think I would have to probably have to redo the roadmap to suit the Beaglebone. Also, there is only one onboard NIC, so I'm not sure if that's a show stopper. I was going to do a fresh install of Debian, install Tor and then build an image so that it could be booting (or flashed to the eMMC) from a SD card, but wether that's better than installing tor oneself, I don't know. Anyway, just throwing an idea out there. Thanks everybody for the work on tor and I'm happy to be running a relay now. Josh

2 1

GSoC '13 Status Report - EvilGenius
by Johannes Fürmann 20 Sep '13

20 Sep '13

Hello tor-dev, Since the last report on EvilGenius, I was able to make a few last changes to EvilGenius which is now more or less feature-complete: * Internal routing issues are now solved packets are flowing happily. * Integrated OONI backend and probe * Wrote some user documentation I guess what's left to do for me is to fix some bugs, clean up the code a bit and get the developer documentation up to speed. Additionally, I will proceed to implement some actual censorship providers based on the ones I already had implemented in earlier versions. Have a nice weekend! Johannes

1 0

Pluggable Transport *Biweekly* Meeting, hence no meeting today
by vmonmoonshine＠gmail.com 20 Sep '13

20 Sep '13

Thanks for everybody who participated last week. The summary of the meeting can be found here: https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports/meeti… We decided that we are going to have these meetings biweekly till we see if we need more or less frequent meetings. So the next one will be next week. See you all there. Best, Vmon

1 0

entry guards and linkability
by Leif Ryge 19 Sep '13

19 Sep '13

Is the following statement correct? When a user connects to Tor from multiple locations where the network is monitored by the same adversary, their persistent use of the same set of entry guards uniquely identifies them and reveals their location to the adversary. Assuming this is an accurate assessment, wouldn't it make sense to maintain separate sets of entry guards for each network that the user connects from? ~leif

2 3

What to read to analyze Tor's use of NTor?
by Roger Dingledine 19 Sep '13

19 Sep '13

Hi Nick, Ian, I've been pointing people to "Section 6 of http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.228.6223" when they ask what NTor is. But then I realized that that's not the best (single) place to send cryptographers when I ask them to analyze whether we've designed or built it right. Then I found https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/216-ntor-han… which looks great but then I also found https://gitweb.torproject.org/torspec.git/blob/HEAD:/tor-spec.txt#l852 which looks crisper. So the questions: A) Which combination and order of these three resources should I point people at? That is, does the tor-spec stanza replace proposal 216 completely, or is there still some use to looking at the proposal too, or is the proposal wrong now because you fixed stuff since then but didn't change the proposal, etc? Did I miss any good resources? B) What are the sketchiest parts -- the parts of the design or the implementation that you most want review on, or that you think would be most fruitful for finding issues? C) What else should I be asking you, in terms of how to get this thing reviewed the mostest and the bestest? We rolled out NTor quicker than we rolled out TAP, relatively speaking, and now it would count as breaking a widely deployed system so I bet we can get some more people evaluating it. Thanks! --Roger

2 1

Using TorPS to estimate chances of all-TAP paths
by Roger Dingledine 19 Sep '13

19 Sep '13

Hi Aaron, Sathya, I have another use for your fine path simulator. :) See https://trac.torproject.org/projects/tor/ticket/9777 The first question is: for a recent consensus, what's the chance that your middle hop and your exit hop will both be running 0.2.4.7-alpha or earlier? The extra credit for question one is to generate the answers to question one for each of the past n consensuses (where n is, say, the past month) so somebody can graph them. The second question is: if we change path selection to try again if we choose such a path, what does that do to our anonymity, for various definitions of anonymity like the ones in the upcoming CCS paper? The extra credit for question two, if you haven't guessed it already, is to get some data points for question two from the past as well. I'm guessing getting 24*30 of them will be no fun, so whatever isn't no fun is fine. What can I do to help make it happen? If the answers are "actually not all that high" and "oh hey those curves aren't all that different" respectively, I want to put #9777 into 0.2.4 and enable it. If those aren't the answers, we'll need to sort out a better plan. Thanks! --Roger

1 0

OnionMail First Test
by Liste 19 Sep '13

19 Sep '13

Work complete for the version 0.0.0Beta http://anopticon.ws/13c03c09/OnionMail-0.0.0.B.zip to create user: telnet 127.0.0.1 9100 C = client S = Server S: +OK ControlPort 1.0 sfl‚4£$£"f C: server alice hackme <- default password S: +OK KESIA [example] `example` C: addusr test <-- local part S: +OK New user created S: Mail=test(a)your.onion S: User=test S: SMTPPasswd=lunc6){><uFXSpaoWnJHl1px S: POP3Passwd=kfzb};VW,NKwLz"JM2Hmrwot S: Server=your.onion S: MsgSize=2000000 S: MaxMsg=128 S: MsgOld=40 S: Inet= S: QFDN=your.onion S: . C: quit S: +OK Closing smtp server (tls if available) 127.0.0.1 25 user test etc...etc... pop3 port 110 (tls if available) 127.0.0.1

5 6

BSD 2/3-Clause License (was: Torperf)
by Karsten Loesing 18 Sep '13

18 Sep '13

On 9/16/13 2:51 PM, Sathyanarayanan Gunasekaran wrote: > I'm currently using BSD 3-Clause License (used by Tor) for torperf2. > Kevin ported ooni's setup script and made it work with torperf2, but > ooni uses a BSD 2-Clause License (both are copyrighted to The Tor > Project). Kevin and I think they are compatible, but I wanted to check > in with you before I merged the changes. Thoughts? I have, honestly, no idea. Cc'ing tor-dev@ in the hope that someone else can answer this question. All the best, Karsten

3 2

Liberte-Linux cable-communication
by josef.winger＠email.de 17 Sep '13

17 Sep '13

Was there a discussion about the 'cables-communication' approach from Liberte-Linux in the mailing list? What's the general oppinion here? Is it trustable/secure/relyable? best /jo

1 0

Message to obfsproxy maintainers: New obfsproxy/pyptlib releases
by George Kadianakis 16 Sep '13

16 Sep '13

Hey people, some days ago I pushed new releases for obfsproxy and pyptlib. Latest releases are obfsproxy-0.2.3 and pyptlib-0.0.4: https://gitweb.torproject.org/pluggable-transports/pyptlib.git https://gitweb.torproject.org/pluggable-transports/obfsproxy.git Please upgrade the packages if you are interested :) The new releases add some nice features and bugfixes (see ChangeLog files). Also, obfsproxy-0.2.3 depends on pyptlib-0.0.4 so it would be great if you could upgrade both packages. (There is probably going to be another release in the next two weeks. I will keep you posted when it happens.) Thanks! PS: I have CCed the obfsproxy maintainers for Debian, FreeBSD, Arch and Gentoo. Sorry if I forgot someone else.

1 0

Status report - Stream-RTT
by ra 14 Sep '13

14 Sep '13

tl;dr When building a circuit, measuring the RTT a single time could provide better latency and anonymity while not affecting throughput. Multiple measurements could be used for running real-time applications like VoIP or optimizing throughput. Despite the fact that the Tor network is currently in an unusual state so to say, I have been spending the last weeks looking into stream-RTT data of circuits. I gathered the data shortly before and at the beginning of the huge botnet usage. This is what I have found out: As assumed stream-RTT measurements of a single circuit are not at a fixed value but distributed since they are subject to multiple influences. After comparing stream-RTT distributions of multiple circuits, I found lots of different shapes and I realized that no single distribution fits them all. The Time-To-First-Byte (TTFB) for fetching a small website over HTTP is used to approximate the latency of a certain circuit. I used different methods to check the correlation between the RTT of a circuit and its TTFB - all indicating a very high correlation. Hence, stream-RTTs of a circuit make a good estimator for its TTFB and therefor its latency. In terms of latency, using a single stream-RTT measurement ("First-RTT") performs better than the currently used method CBT. So far I haven't done any testing/calculations on the other metrics: bandwidth and anonymity. I would assume the former to be unaffected by First-RTT. Latter could probably be slightly increased, if the percentage of discarded circuits would be reduced from 20% with CBT to 10% or 15% with First-RTT - while still achieving a minor improvement in latency. Nevertheless I would not recommend using First-RTT as method for providing low latency circuits to applications, because it only gives a small hint about the quality of a circuit and cannot make sure that some latency properties hold for a certain circuit. Nevertheless First-RTT works pretty well comparing to the minimum effort it takes. Additionally I played around a lot with methods to provide a better estimator for latency properties of a certain circuit. But they all need far more than a single measurement and are therefor out of scope for the common case. Besides they cannot protect against suddenly changing circuit conditions. But they could be used to fulfill a application specific maximum RTT for real-time applications like VoIP. With the use of similar techniques it should be possible to detect circuits that include a node that's within its bandwidth limit. This could be used for providing high bandwidth circuits for applications like BitTorrent. Best, Robert

1 0

Re: [tor-dev] Help with rebasing arma's n23-5 tor branch to current master
by Charlie Belmer 14 Sep '13

14 Sep '13

Hey Karsten & Dev team, I didn't mean to take this off channel, so my apologies for that. I ran a few more tests this week (and my apologies on the slowness of some of this work - life has been hectic this past month), but I am not finding much of use. I could not find any errors which are not also present in when N23 is disabled. If you want to browse the logs for yourself, I uploaded a partial log from a run to http://www.charliebelmer.com/scallion.n23.log(730MB). The most telling entries I found so far are below, but I see the same errors whether UseN23 is set to 1 or not, so it is likely not the root cause. I'll keep digging. Error 1 (Seems to occur when opening channel): 0:0:49:406270 [thread-2] 0:9:41:348157618 [scallion-info] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] bw_weights_error_t networkstatus_check_weights(int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int)(): Bw weight mismatch 1. G=214 M=12174 E=165 D=27654 T=40207 Wmd=4769 Wme=-724848 Wmg=0 Wed=461 Wee=734848 Wgd=4769 Wgg=10000 Wme=-724848 Wmg=0 0:0:49:407692 [thread-2] 0:9:41:348157618 [scallion-info] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] bw_weights_error_t networkstatus_check_weights(int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int)(): Bw weight mismatch 1. G=214 M=12174 E=165 D=27654 T=40207 Wmd=4769 Wme=-724848 Wmg=0 Wed=461 Wee=734848 Wgd=4769 Wgg=10000 Wme=-724848 Wmg=0 Error 2: 0:0:54:227263 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void conn_read_callback(int, short, void *)(): socket 1000264 wants to read. 0:0:54:227281 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_read_to_buf(connection_t *, ssize_t *, int *)(): 1000264: starting, inbuf_datalen 0 (0 pending in tls object). at_most 16448. 0:0:54:227302 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [_tcp_endOfFileSignalled] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: signaling close to user, socket no longer usable 0:0:54:227321 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] TLS error: unexpected close while reading (SSL negotiation finished successfully) 0:0:54:227338 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int tor_tls_read(tor_tls_t *, char *, size_t)(): read returned r=0, err=-8 0:0:54:227354 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_read_to_buf(connection_t *, ssize_t *, int *)(): TLS connection closed on read. Closing. (Nickname nonexit10, address 91.1.0.0) 0:0:54:227373 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_close_for_error(channel_t *)(): Closing channel 0x7f546d03ce40 due to lower-layer error 0:0:54:227389 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_change_state(channel_t *, channel_state_t)(): Changing state of channel 0x7f546d03ce40 (global ID 21) from "open" to "closing" 0:0:54:227420 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_remove_from_digest_map(channel_t *)(): Removed channel 0x7f546d03ce40 (global ID 21) from identity map in state closing (4) with digest BBF25A366EA5376B0026CE6EDE3DF536211429BE 0:0:54:227440 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [tcp_close] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: user closed connection 0:0:54:227452 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [_tcp_setState] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: moved from TCP state 'TCPS_CLOSEWAIT' to 'TCPS_LASTACK' 0:0:54:227473 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void connection_mark_for_close_internal_(connection_t *, int, const char *)(): Calling connection_mark_for_close_internal_() on an OR conn at /home/coffee/shadow/shadow/build/tor/src/or/connection.c:2932 0:0:54:227491 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int conn_close_if_marked(int)(): Cleaning up connection (fd -1). 0:0:54:227509 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void circuit_n_chan_done(channel_t *, int)(): chan to NULL/91.1.0.0:9111, status=0 0:0:54:227546 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_change_state(channel_t *, channel_state_t)(): Changing state of channel 0x7f546d03ce40 (global ID 21) from "closing" to "channel error" 0:0:54:227565 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_remove(connection_t *)(): removing socket -1 (type OR), n_conns now 26 On Tue, Sep 3, 2013 at 10:47 AM, Karsten Loesing <karsten(a)torproject.org>wrote: > On 9/3/13 4:32 PM, Charlie Belmer wrote: > > Hi Karsten, > > > > I have run several tests, but I haven't been able to satisfy myself yet. > I > > attached the results from my latest round of tests for your review. I am > > comparing the n23 branch (UseN23 1 set in each torrc) with the initial > head > > I merged n23 with. It seems that for some reason, the n23 branch > completes > > far fewer downloads, though I haven't yet found the cause, or found > > anything interesting in the logs. > > > > $ grep "fg-download-complete" m1.large-tor/data/scallion.log | wc -l > > 8827 > > $ grep "fg-download-complete" m1.large-n23/scallion.log | wc -l > > 308 > > > > I would definitely appreciate any thoughts, insights, or advice! > > Interesting! Can you re-run the n23 simulation on info or even debug > log level? That might reveal some more information. You'll do that by > running "scallion --log-level=INFO". A tiny network should be > sufficient for that. Watch out for filling disks. > > If you want me to look at the scallion.log file, feel free to post it > somewhere, and I'll see if something looks unusual to me. > > Thanks, > Karsten > > > > On Tue, Sep 3, 2013 at 8:58 AM, Karsten Loesing <karsten(a)torproject.org > >wrote: > > > >> Hi Charlie, > >> > >> did you have the chance to simulate your rebased n23-5 branch using > Shadow? > >> > >> If not, maybe I can help by simulating it once and writing down more > >> specific instructions for you? > >> > >> Thanks! > >> Karsten > >> > >> > >> On 8/8/13 10:36 AM, Karsten Loesing wrote: > >>> (Moving this discussion back to tor-dev@ with Charlie's permission.) > >>> > >>> On 8/6/13 4:10 AM, Charlie Belmer wrote: > >>>> Hey Karsten, > >>>> > >>>> I've been reading up on Shadow & Scallion to do some performance > >> testing on > >>>> N23. Before I go too far down that road, I wanted to check in with > you. > >>>> > >>>> Are there existing blueprints or best practices I should be following? > >>> > >>> Thanks for looking into this! > >>> > >>> So, I wouldn't call the following notes blueprints or best practices, > >>> but they are a description of what I would do to run Shadow simulations > >>> of your branch. > >>> > >>> First of all, be sure to read the Shadow wiki which has contains lots > of > >>> interesting stuff: https://github.com/shadow/shadow/wiki > >>> > >>> The next step would be to get a vanilla Tor master (that your branch is > >>> based on) running in a minimal or tiny Shadow network. It's probably > >>> easiest to use a local Ubuntu VM for this. Or you can start an > m1.large > >>> EC2 instance for the tiny network, or an even smaller instance for the > >>> minimal network. > >>> > >>> Once that simulation succeeds, you'll want to run another simulation > >>> using your branch, extract performance metrics using Shadow's analyze > >>> script, produce a PDF with graphs also using the analyze script, and > >>> then wonder if things are faster than before. > >>> > >>> The minimal or tiny networks are mostly there to learn that things are > >>> working as expected. You'll probably want larger networks for real > >>> performance results, but starting small to get everything working is > >>> probably easier. > >>> > >>> Sounds doable, I guess. However, here's where things start to get > >>> tricky: in order to simulate Tor master (or a branch based on it) in > >>> Shadow, you'll have to perform quite a few extra steps like compiling > >>> your own CMake and Clang/LLVM. Not really hard, but you need to get > all > >>> pieces together before Shadow does what you want. > >>> > >>> I'm pasting detailed instructions that I used to simulate Steven's uTP > >>> branch in Shadow below as an example. Not everything applies to your > >>> branch, but it should give you the general idea. > >>> > >>> If something remains unclear, please feel free to ask! > >>> > >>>> Depending how it goes, I'd also look at re-trying adaptive 23 or > tuning > >> the > >>>> existing implementation. > >>> > >>> Neat! We should probably include Mashael in that discussion once you > >>> have a working and simulated n23-5 branch. Maybe she has ideas what > >>> parts need tweaking. I'm cc'ing her in this mail so she knows what > >>> you're working on. > >>> > >>> In any case, having a working rebased n23-5 branch is already a big > step > >>> forward. > >>> > >>> Thanks! > >>> Karsten > >>> > >>> > >>> === START OF INSTRUCTIONS === > >>> > >>> Launch new EC2 instance running Ubuntu Server 13.04 64 bit > >>> > >>> Set ulimit -n to 25000, see Shadow wiki for details > >>> > >> > https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs… > >>> > >>> $ sudo apt-get update > >>> $ sudo apt-get upgrade > >>> > >>> Clone and test-compile modified libutp > >>> > >>> $ sudo apt-get install build-essential git automake libssl-dev > >> libevent-dev > >>> $ mkdir src > >>> $ cd src/ > >>> $ git clone https://github.com/sjmurdoch/libutp > >>> $ cd libutp/ > >>> $ make > >>> $ cd ../ > >>> > >>> Clone and test-compile Tor branch utp > >>> > >>> $ git clone https://git.torproject.org/tor.git > >>> $ cd tor/ > >>> $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git > >>> $ git fetch sjm217 > >>> $ git checkout -b utp sjm217/utp > >>> $ ./autogen.sh > >>> $ LDFLAGS="-L/home/ubuntu/src/libutp" > CFLAGS="-I/home/ubuntu/src/libutp" > >>> LIBS="-lutp -lrt" ./configure --disable-asciidoc > >>> $ make > >>> $ make distclean > >>> $ cd ../ > >>> > >>> Modify Tor to use uTP for all links (or "0 &&" for not using uTP for > any > >>> link) > >>> > >>> diff --git a/src/or/channeltls.c b/src/or/channeltls.c > >>> index 0551b73..b7b36e1 100644 > >>> --- a/src/or/channeltls.c > >>> +++ b/src/or/channeltls.c > >>> @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, > uint16_t > >>> port, > >>> /* Create a uTP connection */ > >>> tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, > sizeof(sin)); > >>> tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0); > >>> - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { > >>> + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { > >>> log_info(LD_CHANNEL, > >>> "Trying uTP connection to %s", addr_str); > >>> tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const > struct > >>> sockaddr*)&sin, > >>> > >>> Install CMake 2.8.10 > >>> > >>> $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz > >>> $ tar xf cmake-2.8.10.2.tar.gz > >>> $ cd cmake-2.8.10.2/ > >>> $ ./configure > >>> $ make > >>> $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH > >>> $ cd ../ > >>> $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake > >>> > >>> Build Clang and LLVM from source > >>> > >>> $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz > >>> $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz > >>> $ tar xaf llvm-3.2.src.tar.gz > >>> $ tar xaf clang-3.2.src.tar.gz > >>> $ cp -R clang-3.2.src llvm-3.2.src/tools/clang > >>> $ cd llvm-3.2.src/ > >>> $ mkdir build > >>> $ cd build/ > >>> $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. > >>> $ make > >>> $ make install > >>> $ export PATH=~/.local/bin/:$PATH > >>> $ cd ../../ > >>> $ which clang # result: /home/ubuntu/.local/bin//clang > >>> > >>> Install Shadow > >>> > >>> $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy > >>> libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 > >>> python-matplotlib python-numpy python-scipy > >>> $ git clone https://github.com/shadow/shadow.git > >>> $ cd shadow > >>> $ ./setup dependencies > >>> $ ./setup build -g -i /home/ubuntu/src/libutp -i > >>> /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp > >>> --tor-prefix /home/ubuntu/src/tor --tor-lib utp > >>> $ ./setup install > >>> $ export PATH=~/.shadow/bin/:$PATH > >>> > >>> Simulate tiny network in Shadow > >>> > >>> $ cd resource/examples/scallion/ > >>> $ tar xf tiny-4GB-m1.large.tar.xz > >>> $ cd tiny-4GB-m1.large/ > >>> $ scallion > >>> > >>> Graph simulation results > >>> > >>> Add one line to beginning of analyze.py: > >>> > >>> import matplotlib; matplotlib.use('Agg') > >>> > >>> For additional traffic simulations: > >>> > >>> $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log > >>> > >>> $ cd ~/src/ > >>> $ mkdir results > >>> $ cd results/ > >>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py parse --output vanilla > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp > >>> branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla > >>> 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for > >>> none of the links" > >>> > >>> === END OF INSTRUCTIONS === > >>> > >>> > >> > >> > > > >

2 2

Hidden Service location referrals in DNS/web
by Jeroen Massar 13 Sep '13

13 Sep '13

Hi, I don't know what the current status/discussion for this is, hence this email, a reply to a ticket # or mail thread would be fine too ;) Is there currently a plan or way to state for a site: "you can find us as hidden service X" ? In DNS we could use something like: www NAPTR 10 50 "s" "TOR+HTTPS" "" _https._tcp _https._tor. SRV 0 0 443 <fingerprint>.onion. Of course, there is a latency hit there, DNSSEC should be used for authentication and if there is somebody watching DNS they know that you have something that looks at Tor records, thus it might not be ideal for all situations. Having a Torified DNS resolver would resolve that though. Using _https._tor. allows other services to also use this scheme. When 'datagram tor' comes out we can add a "_dtor" or something to support datagram variants of the connections. Another way, which does not show that you are looking for Tor would be to use the "alternate" link ability of HTML, eg for RSS there is: <link rel="alternate" type="application/rss+xml" title="RSS version" href="/rss/" /> Thus we could have something like: <link rel="alternate" type="application/tor+hiddenservice" title="Tor hidden service version" href="https://<fingerprint>.onion" /> on a webpage. If the webpage you get that on is https you have assurance that the hiddenservice is truly there. That would be webpage only though, but that is quite a large portion of the web... Greets, Jeroen

1 0

Thoughts on Proposal 203 [Avoiding censorship by impersonating a HTTPS server]
by Kevin Butler 13 Sep '13

13 Sep '13

(This email got way out of hand from a basic 'I'll bounce an idea here', here's to hoping I haven't made some huge oversight.) I've been thinking about the https frontend after reading the basic problem when I started looking into Tor dev but never took the time to read the actual proposal. When I got some basic idea around how to solve the core problem, I took the time to take a read and it turns out the proposal actually has 90% of what I could think of, but I'm glad I took some time to think from a (I hope) fresh perspective. So anyway, on point. I think Designs #2&3 are the best ideas for Proposal 203 (probably leaning toward #2 more). They're basically the same concept anyway. I came to the same conclusion that we definitely need a shared key to be distributed per bridge address for this to work in any fashion, ideally these keys could be rotated frequently. I also totally agree with the server being a key implementation detail, ideally we want something drop in that could go alongside an existing website. As for content I think mock corporate login pages are a neat idea, while mock private forums are not. Regarding authentication and distinguishability, I don't agree with trying to distinguish Tor clients from non-Tor based on anything the client initially sends, as any sort of computation that isn't webserver-y could be a timing attack or otherwise. I have some specific ideas around how we can implement this to address the issues/concerns outlined in the current proposal. I think the best course of action is to use a webserver's core functionalities to our advantage. I have not made much consideration for client implementation. But here are some thoughts on how we could potentially achieve our goals: - Shared secrets are shared with users whenever bridge IPs are exchanged, it is necessary for these to be large random values and not user-like passwords. (As one of the Authorize proposals also mentions) This exchange would ideally give a domain name for the bridge so as we're not trying to connect to an IP, but to reduce user error the domain and key should be concatenated and base64d so it's a single copy/paste for the user without them trying to navigate to a url thinking it's a Tor enabled link or something. - The users Tor client (assuming they added the bridge), connects to the server over https(tls) to the root domain. It should also downloads all the resources attached to the main page, emulating a web browser for the initial document. - The server should reply with it's normal root page. This page can be dynamically generated, no requirement for it to be static, the only requirement is that one of the linked documents (css, js, img) be served with a header that allows decent caching (>1hr). The far future file could be the document itself but it doesn't have to be. - (This part is probably way too trashy to the server performance, I'm winging it as I can think of it.) - For all files included in the main document, whichever has the furthest future cache header, we'll call that file F. - If we have precomputed the required values (see below) for F, then we are ok to move to the next step (see next bullet point), otherwise, serve all of the files with cache headers under one hour. - If F doesn't have the pre-computations ready, this is the time to spin off a subprocess to start calculating stuff (at lowish cpu priority probably). - The subprocess should calculate an intensive function (e.g. scrypt) of hash(contents of F xored with the shared key) for (X...Y) iterations, inclusive. X and Y should be chosen so that X is on the magnitude of seconds to compute while Y is a couple of thousand iterations above it. Store a 'map' of numberOfIterations => { result, hmac(result + tls cert identifier) }. The hmac should be keyed with the shared secret. The tls cert identifier should probably be its public key or signature? It should store these results in fast cache (hopefully in memory). - So we have our file F, and a precomputed value Z which was the function applied Y times and has a hmac H. We set a cookie on the client base64("Y || random padding || H") - The server should remember which IPs which were given this Y value. This cookie should pretty much look like any session cookie that comes out of rails, drupal, asp, anyone who's doing cookie sessions correctly. Once the cookie is added to the headers, just serve the document as usual. Essentially this should all be possible in an apache/nginx module as the page content shouldn't matter. - Here's a core idea: the server has a handler setup for each of the Z values, hex encoded is probably best (longer!) e.g. /FFFEF421516AB3B2E42... - The webserver should be setup to accept secure websocket upgrades to these urls and route the connection to the local Tor socket. - If the iteration value for the given url is not the same as the one given to the ip trying the path, or the iteration value doesn't match Y, the connection should be dropped/rejected. (This can be legitimate) - If the connection is accepted, the current Y value should be decremented. If Y < X for the current F then we should rotate our keys. (This is a bit of a question, we could manipulate one of the files, but that interferes with the website and could cause distinguishability). - Basically after Y-X Tor clients (not related to how many https users are served), we should be rotating our keys incase the keys leaked, or changing handlers to stop old handlers being used. - When rotating keys we should be sure to not accept requests on the old handlers, by either removing them(404) or by 403ing them, whatever. The decrementing of Y is to try to make replay attacks less feasible, although that would mean tls was broken if they were able to get the initial value, but fuck, who knows with breach & crime et cetera. - (Best read the rest before reading this part: To reduce key churn, or allow long term guard-like functionality, the old handlers could be saved and remain unique to a single ip, by sending cookies from client to server that are a unique id accepted from that ip, the server could know to use an old shared key or something so the client wouldn't blacklist them. Or the client could know not to blacklist previously successful bridges by remembering their tls cert or something. I haven't really thought much about this, but it's probably manageable.) - The idea here is that the webserver (apache/nginx) is working EXACTLY as a normal webserver should, unless someone hits these exact urls which they should have a negligable chance of doing unless they have the current shared secret. There might be a timing attack here, but in that case we can just add a million other handlers that all lead to a 403? (But either way, if someones spamming thousands of requests then you should be able to ip block, but rotating keys should help reduce the feasability of timing attacks or brute forcing?) - So, how does the client figure out the url to use for wss://? Using the cache headers, the client should be able to determine which file is F. If all files are served with a cache header under one hour, then we wait a time period T. Realistically, if the Tor client knows this is a bridge, the only reason this wait should happen is if precomputings happening, so it should just choose another bridge to use... or wait minutes and notify the user that it's for good reason. - Assuming we get a valid F, we look at our cookies. For all cookies, if they're base64, convert to binary, then, try treating the first K bytes (we should have an upper bound for Y, lets say it's probably an 8 byte unsigned long) as a number I. We replicate the computations that the server would have done to get our Zc(lient). - Using this Zc, and the cert provided by the server, we can compute our local Hc. If Hc doesn't match the last (Length of hmac used) bytes in the cookie then try the next cookie. - If no cookie matches, then we either have an old key or we're being MITMd (the computation was ok but the cert didn't match). In these cases, we should fake some user navigation for a couple of pages then close the connection and blacklist the bridge (run for the hills and don't blow the bridge!). - If we get a match, then we know Zc, so we upgrade the connection to wss://domain/Zc which should be a valid secure websocket connection (usable as tcp) unless another ip was already accepted on this iteration value then the server should reject us. If we get rejected at this stage, we know the server had good reason (trying to stop replays) so we just retry from the start and cross our digital fingers. (If bridges are sufficiently private then this should be a non-issue as it will likely only happen with 2 Tor clients connecting within the same second or so) - At this point there should be an encrypted tcp tunnel between the Tor client and the bridge's apache/nginx, and an unencrypted connection between the webserver and the bridge's Tor socket. Should be able to just talk Tor protocol now and get on with things. So to summarise, - Using general web tools to negotiate, secret paths, headers and cookies - Proof of work-ish system using the shared key to establish a unique url - Checking for a MITM and allowing key rotation by using our shared key with a hmac to determine: - If the provided certificate matches what the server thought it gave us - If our result is correct with our current key - Assuming it's sound, I think the serverside could be implemented as an apache module that could be a relatively easy drop in. Concerns: - Distinguishability of client https & websockets implementation. - Content for servers - Everything above as I'm sure theres obviously critical flaw I'm overlooking! - Amount of work it would take on client side :( Rym

2 3

RFC: Lightweight Obfuscated Datagram Protocol (LODP)
by Yawning Angel 13 Sep '13

13 Sep '13

Hello all, Inspired by a conversation with arma, I have been working on designing a UDP based protocol to serve as the bulk data transport for something along the lines of "obfs3, but over UDP". The spec document is approaching the point where I would like to have increased public review before I go and implement it. People who sit in #tor-dev probably have seen various revisions of this draft already. Some notes: * This is heavily inspired by SSU from the i2p project, and Dust by Brandon Wiley. * Ensuring that data gets delivered reliably and in order is outside the scope of this document. Such things are a well researched problem so my reference implementation will likely draw upon an existing protocol instead of reinventing the wheel here. * As far as existing work in this area, the only other protocols that attempt to do similar things are i2p's SSU and Dust. * LODP assumes that there is a shared secret between peers to attempt to thwart active probing attacks. * As part of my evil plans, "secure fingerprinting resistant UDP" is something useful to have. * Open questions/things that are subject to change are tagged with XXX. Questions, comments, feedback appreciated. -- Yawning Angel

1 1

Pluggable transport weekly meeting - Friday Sept 13
by vmonmoonshine＠gmail.com 13 Sep '13

13 Sep '13

Hello pluggable transport devs, As there was no objection to the proposed time, we are going to have our first one today/tomorrow: Friday - Sept 13 CEST: 18:00 BST (Summer GMT): 17:00 UTC: 16:00 EST: 12:00 MNT: 10:00 PST: 9:00 @ #tor-dev on irc.oftc.net And stick to it in future. The format will be scrum-esque. At the beginning we'll ask who likes to participate. Then we ask each participant in some order to give a short update on what they have done last week, what will be doing in the coming week and if they have any questions/problem they want to ask from/share with other developers. We hope to be able to keep it under 30min, so if you find your peer for deeper detailed discussion then you might want to keep the discussion for after the meeting. Thanks! Hopefully See you all, Vmon

1 0

Pluggable transport weekly meeting
by Vmon 13 Sep '13

13 Sep '13

I sent this email quite a while ago and I was surprised that nobody was interested/replied. Today I found out that I had sent it to a wrong address. But here we are, so I'm sending it again. So please reply so we can kick this off soon. Thanks, Vmon ---------- Forwarded message ---------- From: <vmonmoonshine(a)gmail.com> Date: Wed, Aug 21, 2013 at 6:46 AM Subject: Pluggable transport weekly meeting To: tor-dev-request(a)lists.torproject.org Hello Tor devs, Following up on what we came up with in the dev summit ( https://trac.torproject.org/projects/tor/wiki/org/meetings/2013SummerDevMee… ), we are going to have weekly 30-minute IRC meeting focusing on pluggable transports. The format (I think) will be scrum-esque that every developer who is working on a pluggable transport will update everybody else about the work they did/are doing on their transport during the week and ask questions if they have any, for example if they got stuck somewhere and they think somebody can help. Preliminary, we decided to have the meeting on Fridays, cause why not, but if you have serious problem with Fridays then we might be able to pick a better day. For the time of the meeting, considering the geographical positions of the current transport developers, we'll probably end up having a CEST evening and PST morning meeting. Having that in mind I suggest: CEST: 18:00 BST (Summer GMT): 17:00 EST: 12:00 MNT: 10:00 PST: 9:00 So if this doesn't work for you, please reply to this email with your alternative proposal. Thanks, Vmon

7 9

Pluggable Transports and rate limiting
by George Kadianakis 11 Sep '13

11 Sep '13

Hey Yawning (and tor-dev), a topic that we will soon need to consider seriously is rate limiting of pluggable transports. For example, Obfsproxy at the moment does not understand rate limiting and will happily read and write as many bytes as needed. After some discussions in IRC and #3587, we decided to add rate-limiting commands to the TransportControlPort -- that's the (unimplemented) real-time sister of ExtendedORPort. You can see the details in the relevant proposal: https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/196-transpor… (see RATE_LIMIT command etc.) The idea here was that Tor would monitor its bandwidth usage and order PTs to slow down appropriately. If an accounting limit was hit, Tor would order PTs to stop pushing traffic completely. Does this look like a reasonable way to enforce accounting/rate-limiting policies? How could it be improved? For example, I'm kind of sad about this approach, because it requires little-t-tor to do all the hard work of monitoring bandwidth usage and giving out intelligent rate-limiting orders. For example, how do we rate-limit N pluggable transports to 100 kb/s? Do we allow each pluggable transport to push 100/N kb/s? I don't like that this logic must be implemented in little-t-tor. Also, do the TransportControlPort rate-limiting commands look reasonable? Is the payload of RATE_LIMITED (two integers: bandwidth rate and bandwidth burst) properly designed, or does it need more information? Also also, will this work nicely with transport combos (#7167)?

3 3

Tor Cloud is broken - please help debug?
by Runa A. Sandvik 09 Sep '13

09 Sep '13

Hi everyone, It seems something is wrong with the Tor Cloud [1] instances [2]. My best guess is that the instances are failing to auto-update packages and are left in a state where tor is not running, new packages are not being installed, and the instances are not doing anything useful. I am not able to look into this issue until next week. Does anyone want to poke at it sooner? The best ways to debug this would be: * Set up a Tor Cloud instance (open all the right ports and make sure you have a set of SSH keys for it), connect to it and look through the logs. I'm guessing the package update process is waiting for user input or something similar. * Set up an Ubuntu cloud instance and go through the Tor Cloud setup [3] manually to see when/if something breaks. [1]: https://cloud.torproject.org/ [2]: https://metrics.torproject.org/network.html#cloudbridges [3]: https://gitweb.torproject.org/tor-cloud.git/blob_plain/HEAD:/ec2-prep.sh -- Runa A. Sandvik

1 0

dynamic changes to ORPort set (via controller)
by Zack Weinberg 09 Sep '13

09 Sep '13

Suppose I have a relay (not a client) which starts out with one open ORPort and a local controller. 1) Is there a better way (read "supported in STEM") than using SETCONF to add ORPort lines, to direct this relay to open additional ORPorts? 2) Will adding or removing ORPorts at runtime affect established connections to other ORPorts (those which are open before and after the change)? thanks zw

2 2

Quickly testing TOR using Chutney and Fluxcapacitor
by Marek Majkowski 09 Sep '13

09 Sep '13

Tor ships with some testing facilities (tor/src/test/test), but they are not comprehensive. (I'm aware there's an ongoing effort on improving these tests). To test anything non trivial it is necessary to run a tor node as a part of a tor network. There is another tool called chutney [1] that can be used to set up a small testing Tor network: it generates plenty of torrc configs and spawns a number of Tor servers locally. This is very handy, but it can take a while before the network starts working - the Auth servers need to establish the consensus and do all the things I have no clue about. Here's a simple shell script [2]: - It uses chutney to start a testing tor network. - It waits for it to work by trying to establish a connection to localhost:22. - Finally it tears down the network. Normally it takes around 5 minutes for the network to converge: $ time ./go.sh real 4m8.330s user 0m2.064s sys 0m0.392s Of course, you don't have to set a completely fresh Tor network for every single test, but that's what I often do. I'd be eager to hear how people are reusing chutney networks. In past I wrote this thing called fluxcapacitor [3], it's a tool that speeds up tests. After a few fixes I was able to run chutney on it: $ time /tmp/fluxcapacitor/fluxcapacitor ./go.sh real 0m11.450s user 0m2.340s sys 0m2.120s Running stuff under fluxcapacitor is not deterministic, sometimes it takes 8 seconds, sometimes 15 but it generally works and should go pretty quick. That's it. I thought someone may find it useful. Cheers, Marek [1] https://gitweb.torproject.org/chutney.git [2] https://github.com/majek/dump/blob/master/tor/go.sh [3] https://github.com/majek/fluxcapacitor.git

3 4

[GSOC] Status report - Tor capabilities
by Cristian-Matei Toader 09 Sep '13

09 Sep '13

Hello tor-dev, Not much has happened since the last status report. I have been fixing small issues [1] with the seccomp2 filter and have been working on the change requests from nickm's code review for merging stage 2 of tor sandboxing [2]. I am currently waiting for feedback, which is very likely to be final, and a merge of my work in tor's master branch. After stage 2 is completed, I will move on with stage 3 by dropping privileges in different sections of the tor program, as they no longer become required, in order to create a more restrictive sandbox. [1] https://github.com/cristiantoader/tor-gsoc-capabilities/commits/gsoc-cap-st… [2] https://trac.torproject.org/projects/tor/ticket/9249

1 0

[GSOC 2013] Steganography Browser Extension Status Report
by Hareesan 09 Sep '13

09 Sep '13

Hi all, First of all, let me apologize for being late in sending this report. In last two weeks, I was mostly engaged with code updating and fine tuning the code. I spent time in changing the public key cryptography libraries [1] to support Firefox supported imports. And I'm not completed with this part because I need to spend time in understanding the libraries. Since I'm having exams from this week, I won't be able to work on the project for coming weeks and this will be my last status report regarding my GSOC project. Once my exams over, I will continue working in the project from where I'm pausing now. Up to now I have created the extension for Firefox with the following features, 1. Upload contents from physical files to encryption / decryption. Up to now the contents will be popped as byte data format once the user click on encrypt/decrypt button in the upload menu. 2. Get web content from from web page for encryption/decryption purpose. Once user right click on a web content such as image, html video, html audio and select on encrypt or decrypt option and click on encrypt/decrypt button. the byte data of selected content will be popped. All these byte data popped will be used to further message encryption need in the future development of the project. 3. The code to encrypt message using random number also added and the methods to access the libraries will be displayed in the documentation. 4. In options page, the filter creation and relevant sqlite database access codes are added. It can be tested using the Filters tab in options page. All the codes up to now are reviewed and tested by Sukhbir Singh, and changes are made according to mentor's suggestion and possible bugs are got fixed time to time. Time to time, many suggestions and feedbacks were given by Moritz Bartl as well. Future plans for the continuity of the project. 1. There is a need to add public cryptography libraries to the extension. Up to now, the libraries are selected and added to the repository. But there is a need to change libraries to support Firefox supported import export mechanisms. 2. After fixing these all, the primary part of the project to add Steganography libraries exists. You can find the full source code in public repository[2]. Within the coming weeks, I will complete the documentation in detail. Thank you. [1] http://ats.oka.nu/titaniumcore/js/crypto/readme.txt [2] https://github.com/rharishan/Steganography-Browser -- Hareesan

1 0

Testing flash proxy infrastructure
by David Fifield 08 Sep '13

08 Sep '13

George asked me for tips on testing flash proxy infrastructure. You will need to run a websocket-server (https://gitweb.torproject.org/flashproxy.git/tree/HEAD:/websocket-transport) in front of a relay: ServerTransportPlugin websocket exec /usr/local/bin/websocket-server --port 9901 Or you can just use the public one at 173.255.221.44:9901. Let's say the websocket-server is listening at 127.0.0.1:9901. You follow these instructions to set up your own facilitator. https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/facilitator-how… It's fairly involved, and at the end you have an Apache serving a CGI, plus some local daemons. For your own local testing, you can use a self-signed certificate or even plain HTTP, and you probably don't want to do the email poller because that's a process on its own. In /etc/init.d/facilitator, edit the RELAY variable to point to your chosen websocket-server. Say your facilitator CGI is running at https://facilitator:8000/. You point flashproxy-client to the faciltiator using the -f option. ./flashproxy-client -f https://facilitator:8000/ --external --register-methods=http --unsafe-logging To run flashproxy-client in managed mode from torrc, remove --external. The two rendezvous methods that don't require setting up a lot of stuff are http and url. You can add a client registration like so: ./flashproxy-reg-http -f https://facilitator:8000/ --unsafe-logging ./flashproxy-reg-url -f https://facilitator:8000/ --facilitator-pubkey=reg-daemon.pub 127.0.0.1:9000 With flashproxy-reg-url, you will have to paste the URL it prints out into a browser. flashproxy-client will automatically pass the -f option to its registration helpers. To run a local flash proxy, open a non-Tor browser to embed.html and give it the custom facilitator URL: file:///home/user/flashproxy/proxy/embed.html?debug&facilitator=https://facilitator:8000/&initial_facilitator_poll_interval=0 To bypass the facilitator and connect directly to a client and relay, do this: file:///home/user/flashproxy/proxy/embed.html?debug&client=127.0.0.1:9000&relay=127.0.0.1:9901 David Fifield

1 0