- tor-dev - lists.torproject.org

Code Review: or-applet
by Damian Johnson 27 May '14

27 May '14

Hi Yawning, I just took a quick peek at your or-applet (https://github.com/Yawning/or-applet) Nice work! Added it to Stem's examples page. Just a few thoughts. Nothing very substantial... > def _format_status(status): > if status == CircStatus.LAUNCHED: > return 'LAUNCHED (circuit ID assigned to new circuit)' > elif status == CircStatus.BUILT: > return 'BUILT (all hops finished, can now accept streams)' > elif status == CircStatus.EXTENDED: > return 'EXTENDED (one more hop has been completed)' > elif status == CircStatus.FAILED: > return 'FAILED (circuit closed (was not built))' > elif status == CircStatus.CLOSED: > return 'CLOSED (circuit closed (was built))' > return str(status) Very minor thing but I'd probably opt for the following... FORMAT_STATUSES = { CircStatus.LAUNCHED: 'LAUNCHED (circuit ID assigned to new circuit)', CircStatus.BUILT: 'BUILT (all hops finished, can now accept streams)', CircStatus.EXTENDED: 'EXTENDED (one more hop has been completed)', CircStatus.FAILED: 'FAILED (circuit closed (was not built))', CircStatus.CLOSED: 'CLOSED (circuit closed (was built))', } def _format_status(status): return FORMAT_STATUSES.get(status, str(status)) Ditto for the rest of the status_icon.py functions. > def _build_dynamic_menu(self): > circuits = self._ctl.get_circuits() > if circuits == None: > item = Gtk.MenuItem('No circuits established') > item.set_sensitive(False) > self._menu.append(item) > return > streams = self._ctl.get_streams() The get_circuits() method will never return None as you're using it here. It'll either return a list or raise a stem.ControllerError. I think what you actually want is... circuits = self._ctl.get_circuits(None) That way it'll return None rather than raise an exception if it fails. > our_streams.append('[' + stream.id + ']: ' + stream.target) In python string concatenation is discouraged unless it gives you readability benefits. It's slightly more performant to use format() or the following... our_streams.append('[%s]: %s' % (stream.id, stream.target)) That said, really doesn't matter. :) > if len(our_streams) == 0: > ... The boolean value of an empty list is False, so this is usually done as... if our_streams: ... > if self._control == None: Minor thing but None comparison is the one time it's suggested that you use the 'is' keyword (ie, 'if self._control is None:'). > class PopupMenu: It's generally a good idea to always extend object... class PopupMenu(object): > def is_newnym_available(self): > if self._control == None: > return False > return self._control.is_newnym_available() Hmmm, a goal I had for stem Controller objects was that they could be gracefully disconnected and reconnected. The Controller's is_newnym_available() already checks is_alive() and returns False if you're disconnected from tor. Ideally self._control should never be None. However, if you've never had an *initial* tor connection then I'm not sure that it's possible to instantiate a Controller. If so then that's a stem issue I'll need to think about... Cheers! -Damian

1 0

[GSoC Stegotorus] server segfault
by vmon＠riseup.net 27 May '14

27 May '14

Hey Noah, Thanks for the progress. T:7.9593: ckt 1 <ntp 10 outq 0>: send 8 <d=0 p=32 f=DAT> 7.9593 [debug] send_targeted: <1> efficiency: 151.000000 7.9594 [debug] conn_send_eof: <1.10> flushing out 5496 bytes 7.9594 [debug] conn_do_flush: <1.10> flushing 5496 bytes to peer [enabled=6] 7.9595 [debug] send: <1.4> must send 7.9595 [debug] transmit_room: <1.4> checking available capacity for type 3 7.9596 [debug] send_targeted: <1.4> requests 75 bytes (http) [error] Segmentation fault at 0000000000000000 ./stegotorus[0x40a0b9] /lib/x86_64-linux-gnu/libc.so.6(+0x36ff0)[0x7fd322381ff0] It says 7.9595 [debug] transmit_room: <1.4> checking available capacity for type 3 so it was trying to send (probably for the first time) swf and it crashed. If you are lucky and swf doesn't get chosen by random then you won't get crash for a while. I very much think it is the problem with that extra extract_approperiate_response_body that you wrote in the encode that causes the crashes. It returns negative value because there is no http response header there. Take those off and see if it passes or not. If the error persists, I'll try to locate it by going step by step. "0.0005 [debug] ApachePayloadServer: payload database does not exists. 0.0006 [debug] ApachePayloadServer: scarping payloads to create the database... 0.0066 [warn] scrape: error opening the payload database file: No such file or directory 0.0070 [error] ApachePayloadServer: Cannot open payload info file. " For http_apache you either need to run apache on your server or give another http server as the cover_server option in the command line. Let's not go there yet. If http steg works, we should be happy with this step. Thanks for the progress and the commit, vmon

1 0

How to distribute Tor with other software
by Micah Lee 27 May '14

27 May '14

I just wrote a little program called OnionShare, that makes it simple to share a file securely using a password-protected Tor hidden service: https://github.com/micahflee/onionshare Right now it only runs in Tails. I'd like to make a versin that runs in OSX and Windows, too. How can I make software for OSX and Windows that relies on Tor like this, but without having to bundle Tor with it? I would really like to not be in the business of distributing Tor myself. When you run onionshare.py, it modifies /etc/tor/torrc and reloads the Tor config, and when it's done it restores the original torrc and reloads again. I could require people to open Tor Browser and have it just use the Tor that comes with that, except the fact that it needs to modify torrc makes that complicated. -- Micah Lee

4 4

Mbox sandbox
by Nicolas Vigier 27 May '14

27 May '14

On Fri, 16 May 2014, isis wrote: > Nicolas Vigier transcribed 2.6K bytes: > > > > - Looked at Mbox[2]: a sandboxing tool based on ptrace and seccomp/BPF. > > This can be used in the test suite to get the list of files modified > > by the browser after running a test, to check that it did not create > > or modify files in unexpected places. This can also be used to log > > all network connections, to check that everything goes through tor. > > I was previously thinking about doing that using Docker, but now it > > seems more simple with Mbox. > > Mbox is neat! It looks like it's git based, right? Or at least includes some > sort of CVS system. Either way, great idea, testing for connections not > matching " -> 127.0.0.1" should be easy. :) > > > [2]: http://pdos.csail.mit.edu/mbox/ Yes, it's nice! It's not git or CVS based. But it stores all new and modified files in a separate directory, doing copy-on-write when opening files with write permissions, by hijacking arguments of system calls which access files. And after running the program, asks you which copy of the files you want to keep (or you can manually copy the files from the sandbox directory). It can also be used to log in a file all network connections opened. Initially the filesystem sandoxing part didn't work with Tor Browser because of some bugs in Mbox, but it's now fixed so we'll be able to use it in the TBB test suite to monitor the files modified, created and removed by Tor Browser, and network connections made. Yesterday I also made a patch that allows filtering which connections can be made: https://github.com/tsgates/mbox/commit/6dd0e49202795564e627e9eeba664fc685b1… It could be used for instance to make sure a program will not connect anywhere without using tor. This can be done like this: $ cat tor.profile [fs] direct: / [network] block: 0.0.0.0 allow: 127.0.0.1:9050 $ mbox -p ./tor.profile -- curl -I http://www.google.com/ curl: (6) Could not resolve host: www.google.com $ mbox -p ./tor.profile -- curl --socks5-hostname 127.0.0.1:9050 -I http://www.google.com/ HTTP/1.1 302 Found Cache-Control: private Content-Type: text/html; charset=UTF-8 Location: http://www.google.co.in/?gfe_rd=cr&ei=GoiDU-76DcSU-wbdy4HgDg Content-Length: 261 Date: Mon, 26 May 2014 18:29:46 GMT Server: GFE/2.0 Alternate-Protocol: 80:quic $ mbox -o /dev/null -p ./tor.profile -- nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:32 CEST Nmap scan report for localhost (127.0.0.1) Host is up (0.0022s latency). Not shown: 999 closed ports PORT STATE SERVICE 9050/tcp open tor-socks Nmap done: 1 IP address (1 host up) scanned in 0.18 seconds $ nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:32 CEST Nmap scan report for localhost (127.0.0.1) Host is up (0.00048s latency). Not shown: 996 closed ports PORT STATE SERVICE 25/tcp open smtp 111/tcp open rpcbind 631/tcp open ipp 9050/tcp open tor-socks Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds It's also possible to kill a process which does unallowed connections, rather than just blocking those connections: $ cat tor.profile [fs] direct: / [network] kill: 0.0.0.0 allow: 127.0.0.1:9050 $ mbox -o /dev/null -p ./tor.profile -- nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:37 CEST Stop executing pid=5298: Connect to 127.0.0.1 port 80 However, a warning if some people want to use this: it's still alpha quality software, so probably not a good idea to use it for something where security is important without more review.

1 0

New Feature: Tor Interpreter
by Damian Johnson 26 May '14

26 May '14

Hi all, after a couple months down in the engine room I'm delighted to announce a new Stem feature for advanced Tor users and developers! https://stem.torproject.org/tutorials/down_the_rabbit_hole.html The control interpreter is a new method for interacting with Tor's control interface that combines an interactive python interpreter with raw access similar to telnet. This adds several usability features, such as... * Irc-style commands like '/help'. * Tab completion for Tor's controller commands. * History scrollback by pressing up/down. * Transparently handles Tor authentication at startup. * Colorized output for improved readability. This is the last major feature going into Stem's 1.2.0 release, which is coming out later this month. Until then you can easily give it a whirl with... % git clone https://git.torproject.org/stem.git % cd stem % ./tor-prompt Running into an issue? Got a feature request? As always feedback appreciated! -Damian

1 0

Question to Java developers: what are good frameworks for Tor's server-side Java applications
by Karsten Loesing 25 May '14

25 May '14

Hello devs, I'm seeking advice from people with experience in writing server-side Java applications. Let me give you some background about this request: for the past five years, I have been developing server-side Java applications which all process large amounts of Tor directory data and provide their output via a web interface. Examples: - The metrics data processor (metrics-db) fetches Tor descriptors from the Tor directory authorities, the bridge authority, etc., performs some sanity-checks, and provides descriptors by type as tarballs. We're talking about roughly 7 GiB new bzip2-compressed data per month. - The metrics website (metrics-web) uses the output from the metrics data processor, stuffs everything into a database, computes aggregates, and presents results in graphs and .csv files. - The Onionoo service processes the same data from the metrics data processor, but provides statistics per Tor relay, not for the Tor network as a whole. The processing is done every two hours and may take 30 minutes to 1.5 hours, depending on how overloaded the server is. - The ExoneraTor service, again, uses the same data and puts it in a database to answer whether a certain IP address has been a Tor relay at some point in the past. That's what is done. And here's how it's done under the surface: - There's one or more cronjobs, each of which starts an ant task to process data. Some of these tasks import data into the database, others store results in the file system. - Each application uses a web application deployed in Tomcat to provide results to web users. Most things are written in servlets, some use JSPs. My problem is that this approach is rather fragile and difficult to setup for new volunteers. I'm aware of that, and I'd like to improve it. My question is: what Java frameworks should I be looking at for the applications described above? Bonus points if something is in Debian stable. Note that "switch to $some_other_programming_language" is not a very useful answer to me, at least not for the larger applications. There's just too much existing code and not enough developer time to port it. Thanks in advance! All the best, Karsten

3 3

RFC: obfs4 (Name not final)
by Yawning Angel 23 May '14

23 May '14

Hello, The people that have been following Pluggable Transport development may know that I have been working on something tentatively called "obfs4" recently. It's rapidly approaching the point where I would like to open it up for review and feedback, hence the e-mail. A quick and dirty description would be: obfs4 is ScrambleSuit with djb crypto. Instead of obfs3 style UniformDH and CTR-AES256/HMAC-SHA256, obfs4 uses a combination of Curve25519, Elligator2, HMAC-SHA256, XSalsa20/Poly1305 and SipHash-2-4. The feature set offered by obfs4 is comparable to ScrambleSuit with the following differences (implementation specific changes are noted as such): * The key exchange is based on the ntor handshake, and thus authenticates the server to the client. Both obfs3 and ScrambleSuit depend on the encapsulated protocol to handle this on it's own. * obfs4 always does a full handshake. ScrambleSuit style session ticket handshakes are not supported. Even with Elligator2 mapping taken into account, the obfs4 handshake is significantly faster, so there is less of a need for this. * (Impl) obfs4proxy currently does not offer Inter-Arrival Time obfuscation, but this could easily be added if it is something that a lot of people want. It is worth noting that while obfsproxy and obfsclient both support IAT obfuscation, support for it is currently disabled by default as a performance tradeoff. * (Impl) obfs4proxy is currently written in golang. Neither a clear plus or minus, it does allow people to run bridges on reserved ports significantly easier, is probably faster (native code, can use multiple cores for most things), but is more annoying to build and results in rather large binaries (the go runtime is statically linked). * (Impl) obfs4 supports a lot of the protocol improvements made to ScrambleSuit that are pending merge (See #11271, #11203). This difference is temporary. The code and a draft spec is at: https://github.com/Yawning/obfs4 Open questions: * Is the base design and my implementation sane? * Should obfs4proxy also have (disabled) IAT obfuscation? Adding it later will not require wire protocol changes. * The handshake length mimics ScrambleSuit in terms of maximum padding (< 1500 bytes). Should this be increased to be similar to obfs3 (~8k of maximum padding)? The server side cost for this shouldn't be that high. * Is this different enough from ScrambleSuit to be worth deploying? I would be ok with this ending up as "just a research project" and shelving it if the consensus is otherwise. * Should I have named it ScrambleSuit 2? It's a direct descendant of ScrambleSuit and is an obfs derivative only in name at this point. Future plans: * Implement suggestions/improve the code/fix bugs/write more unit tests. * Improve goptlib. * If this is deployed in meaningful amounts, support it in obfsclient. For the extremely brave, I am running a test bridge with the most recent snapshot of the code: UseBridges 1 ClientTransportPlugin obfs4 exec /path/to/obfs4proxy # Bridge line of doom, all one line. Bridge obfs4 178.209.52.110:52810 67E72FF33D7D41BF11C569646A0A7B4B188340DF node-id=Z+cv8z19Qb8RxWlkagp7SxiDQN8= public-key=vm+w9k57aMIX/o+A9ef5C7o9xKEkhr7kRBj3N4etDn8= As with any PT that requires per-bridge arguments, this also requires tor-0.2.5.x. The node-id in this case happens to be my bridge's fingerprint, the public key is the long term key used to validate my bridge's identity and generate M_[C,S]/MAC in the handshake (The bridge line is a bit of a UI/UX disaster unless people are copy/pasting it). Getting either of them wrong will result in handshake failures and tor not bootstrapping. A few warnings: * The spec assumes familiarity with ntor, ScrambleSuit and NaCl's crypto_secretbox. * The wire protocol is not final and I *will* push builds to the bridge as I break backward compatibility without notifying people (It's been 0 days since a wire protocol change.). The test bridge will randomly be broken/rebooted/etc as I also use it for other things. * Development was done with go1.2.x, older versions of the runtime are not supported. * It would be a terrible idea to use obfs4proxy as anything other than a client at this point. Questions, comments, feedback all appreciated. -- Yawning Angel PS: I also wrote https://github.com/yawning/or-applet recently. It's even less supported/tested than obfs4 is (it is written only for my laptop), but some people may find it cute.

4 5

Pluggable transports meeting today (Friday 23rd of May 2014)
by George Kadianakis 23 May '14

23 May '14

Hello friends, just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur today at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Please note that we recently *changed* the time from 17:00 to 16:00 UTC. Thanks!

1 0

GSoC: Revamp GetTor - new design
by Israel Leiva 22 May '14

22 May '14

Hi all. One of the decision I've had to take during the community bonding was if rewrite GetTor or fix the current code. IMO the best approach is to rewrite it, and I've been working on a proposal design [1]. I'm currently discussing this new design with my mentors, and I'd really appreciate any other feedback from the dev community. Obviously, several parts of the current code will be reused; what will change is how is organized. Also, there are some things that are not yet covered and need further discussion, such as packages checksums and syncronization. [1] https://github.com/ileiva/gettor/tree/master/spec Regards. -- israel

1 0

A Child's Garden of Pluggable Transports
by David Fifield 21 May '14

21 May '14

Have you ever wondered what makes the Tor protocol fingerprintable, and makes pluggable transports necessary? Have you wondered how obfs3 obscures byte patterns in Tor? What a flash proxy WebSocket connection actually looks like, and why it defeats IP blocking but not DPI? Then I have the wiki page for you: https://trac.torproject.org/projects/tor/wiki/doc/AChildsGardenOfPluggableT… It's a visualization of different pluggable transports, meant to be an aid to understanding. At the top is an ordinary Tor handshake, with some fingerprintable data fields highlighted. The following sections, one for each transport, show how those fields are hidden--or not. I tried to demonstrate aspects of different transports that I think are hard to intuit, such as what flash proxy rendezvous looks like, and how transports look under the encrypted layer that is visible to a censor. There are sections for obfs3, ScrambleSuit, FTE, flash proxy, meek, and Bananaphone. The page is missing a few more from https://www.torproject.org/docs/pluggable-transports. If you know how to run any of those transports, and you know an effective way to visualize it, please add it to the page. David Fifield

2 1