- tor-dev - lists.torproject.org

Improving Unit Tests Coverage with Sample Patch
by Clark Wood 31 May '14

31 May '14

Hello, I'm interested in volunteering some to help improve unit test coverage. Please find attached a small patch with unit tests for $TOR/src/or/cpuworker.c. I intend to do some more work for cpuworker's functions, but for now am curious about whether the "if (1)" block spanning lines 671-720 is needed. --Clark

1 0

wfpadtools: comments about primitives
by Marc Juarez 30 May '14

30 May '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I am a GSoC student working in a new PT for the development of future Website Fingerprinting countermeasures in Tor. The PT is not targeting any specific defense, but to link padding defenses in general. The idea is to implement a set of primitives that any link padding-based defense would benefit of. In this email I provide a more detailed description of these primitives and give a short update about their state. I forked the obfsproxy repo and made it publicly available in bitbucket: https://bitbucket.org/mjuarezm/obfsproxy-wfpadtools. I would appreciate comments from the Tor development community. I'm specially looking for advices that help me generalizing the padding module (padutils) and comments about the primitives I describe below. The envisaged primitives are: 1. A general padding class that provides methods to pad the link This part is almost finished. For that I reused the Scramblesuit's probdist and fifobuffer modules to buffer and flush data according to a given probability distribution. Using this class I have implemented a simple version of BuFLO (which is also included in the padutils module). 2. Create a framing layer for the PT This layer would allows to signal padding messages so that they can be filtered out. I have to implement the protocol's message scheme so I can parse the flags in the header and get rid of the padding messages. 3. Implement the state machine of the basic padding protocol. BuFLO assumes that there exists a mechanism to detect the beginning and end visits. I would implement this using a new state in the protocol's state machine (ST_VISITING), but I still need to find a way to signal the beginning and the end of each visit. 4. Implement the protocol's handshake. I took a look to the Scramblesuit's handshake.The handshake of this protocol would boil down to the negotiation of the parameters (e.g., probability distributions) for the padding. 5. Implement a flow control protocol This would adjust the padding parameters while running. The fifobuffer we are currently using helps queuing the messages but we will need an extra logic to detect congestion. 6. Padding operations We will implement padding operations that might be handy for the future countermeasure. For example, one possible operation could be to specify the number of cells to send in response to a padding cell request. 7. A module to test protection against Website Fingerprinting This module would leverage the Peekaboo's framework by Dyer et al. (we may consider to extend it to include the newest Website Fingerprinting attacks). Best, - -- marc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTiKxFAAoJEGfJ5xfgazlxw70H/1EbzYOuhLwDe5xdLiz/Pn3z UrXV+GCPG+Wa3SiQOFqCqSKn9D/E0dwLLZKcG/zE9F8wZeSbtjlDIBn4NNUwMzz3 XMfCHcohxLHtKFwhTncHkTvEn6ZI1IzAUn1wCHdNWVGoTfVVuY5RtNKF0F0IxBBf IE1mEFoM1wKzmQIFvB/MkPlLo65Ilhwv20GHgqGwDpKXYwBROBHR7CLn29oL/GB8 lZWVm0ZeDB0wtqjdPqxh9g2nEoxQR83BzJHeos+fue1onnZw+L23gbzfIvznjuL5 47Yed3T4l1q1k1i56G96/CoBIFlBboKfz5WhJ0dnDz2p4s996BNNIazxNybenjU= =5LGv -----END PGP SIGNATURE-----

4 7

Re: [tor-dev] tor-dev Digest, Vol 40, Issue 37
by 8106501726＠txt.att.net 30 May '14

30 May '14

Stop -----Original Message----- From: <tor-dev(a)lists.torproject.org> Sent: Fri, 30 May 2014 12:00:02 +0000 To: 8106501726(a)txt.att.net Subject: tor-dev Digest, Vol 40, Issue 37 >Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > >To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailma ================================================================== This mobile text message is brought to you by AT&T

1 0

Reminder: tor development meetings this (and every) Wednesday, 19:00 UTC
by Nick Mathewson 28 May '14

28 May '14

Hi, all! Here's your regular reminder for the weekly IRC meeting for people working on the program "tor". (This won't cover all the other programs developed under the Tor umbrella.) The next meeting time will be: Wednesday May 28, 19:00 UTC. (That's 3pm EDT and 12:00 noon PDT.) As usual, we'll do it on the #tor-dev IRC channel. I've seen a drop-off in the number of people coming since I stopped sending these out weekly, so I'm sending them again for a while. -- Nick

1 0

Code Review: or-applet
by Damian Johnson 27 May '14

27 May '14

Hi Yawning, I just took a quick peek at your or-applet (https://github.com/Yawning/or-applet) Nice work! Added it to Stem's examples page. Just a few thoughts. Nothing very substantial... > def _format_status(status): > if status == CircStatus.LAUNCHED: > return 'LAUNCHED (circuit ID assigned to new circuit)' > elif status == CircStatus.BUILT: > return 'BUILT (all hops finished, can now accept streams)' > elif status == CircStatus.EXTENDED: > return 'EXTENDED (one more hop has been completed)' > elif status == CircStatus.FAILED: > return 'FAILED (circuit closed (was not built))' > elif status == CircStatus.CLOSED: > return 'CLOSED (circuit closed (was built))' > return str(status) Very minor thing but I'd probably opt for the following... FORMAT_STATUSES = { CircStatus.LAUNCHED: 'LAUNCHED (circuit ID assigned to new circuit)', CircStatus.BUILT: 'BUILT (all hops finished, can now accept streams)', CircStatus.EXTENDED: 'EXTENDED (one more hop has been completed)', CircStatus.FAILED: 'FAILED (circuit closed (was not built))', CircStatus.CLOSED: 'CLOSED (circuit closed (was built))', } def _format_status(status): return FORMAT_STATUSES.get(status, str(status)) Ditto for the rest of the status_icon.py functions. > def _build_dynamic_menu(self): > circuits = self._ctl.get_circuits() > if circuits == None: > item = Gtk.MenuItem('No circuits established') > item.set_sensitive(False) > self._menu.append(item) > return > streams = self._ctl.get_streams() The get_circuits() method will never return None as you're using it here. It'll either return a list or raise a stem.ControllerError. I think what you actually want is... circuits = self._ctl.get_circuits(None) That way it'll return None rather than raise an exception if it fails. > our_streams.append('[' + stream.id + ']: ' + stream.target) In python string concatenation is discouraged unless it gives you readability benefits. It's slightly more performant to use format() or the following... our_streams.append('[%s]: %s' % (stream.id, stream.target)) That said, really doesn't matter. :) > if len(our_streams) == 0: > ... The boolean value of an empty list is False, so this is usually done as... if our_streams: ... > if self._control == None: Minor thing but None comparison is the one time it's suggested that you use the 'is' keyword (ie, 'if self._control is None:'). > class PopupMenu: It's generally a good idea to always extend object... class PopupMenu(object): > def is_newnym_available(self): > if self._control == None: > return False > return self._control.is_newnym_available() Hmmm, a goal I had for stem Controller objects was that they could be gracefully disconnected and reconnected. The Controller's is_newnym_available() already checks is_alive() and returns False if you're disconnected from tor. Ideally self._control should never be None. However, if you've never had an *initial* tor connection then I'm not sure that it's possible to instantiate a Controller. If so then that's a stem issue I'll need to think about... Cheers! -Damian

1 0

[GSoC Stegotorus] server segfault
by vmon＠riseup.net 27 May '14

27 May '14

Hey Noah, Thanks for the progress. T:7.9593: ckt 1 <ntp 10 outq 0>: send 8 <d=0 p=32 f=DAT> 7.9593 [debug] send_targeted: <1> efficiency: 151.000000 7.9594 [debug] conn_send_eof: <1.10> flushing out 5496 bytes 7.9594 [debug] conn_do_flush: <1.10> flushing 5496 bytes to peer [enabled=6] 7.9595 [debug] send: <1.4> must send 7.9595 [debug] transmit_room: <1.4> checking available capacity for type 3 7.9596 [debug] send_targeted: <1.4> requests 75 bytes (http) [error] Segmentation fault at 0000000000000000 ./stegotorus[0x40a0b9] /lib/x86_64-linux-gnu/libc.so.6(+0x36ff0)[0x7fd322381ff0] It says 7.9595 [debug] transmit_room: <1.4> checking available capacity for type 3 so it was trying to send (probably for the first time) swf and it crashed. If you are lucky and swf doesn't get chosen by random then you won't get crash for a while. I very much think it is the problem with that extra extract_approperiate_response_body that you wrote in the encode that causes the crashes. It returns negative value because there is no http response header there. Take those off and see if it passes or not. If the error persists, I'll try to locate it by going step by step. "0.0005 [debug] ApachePayloadServer: payload database does not exists. 0.0006 [debug] ApachePayloadServer: scarping payloads to create the database... 0.0066 [warn] scrape: error opening the payload database file: No such file or directory 0.0070 [error] ApachePayloadServer: Cannot open payload info file. " For http_apache you either need to run apache on your server or give another http server as the cover_server option in the command line. Let's not go there yet. If http steg works, we should be happy with this step. Thanks for the progress and the commit, vmon

1 0

How to distribute Tor with other software
by Micah Lee 27 May '14

27 May '14

I just wrote a little program called OnionShare, that makes it simple to share a file securely using a password-protected Tor hidden service: https://github.com/micahflee/onionshare Right now it only runs in Tails. I'd like to make a versin that runs in OSX and Windows, too. How can I make software for OSX and Windows that relies on Tor like this, but without having to bundle Tor with it? I would really like to not be in the business of distributing Tor myself. When you run onionshare.py, it modifies /etc/tor/torrc and reloads the Tor config, and when it's done it restores the original torrc and reloads again. I could require people to open Tor Browser and have it just use the Tor that comes with that, except the fact that it needs to modify torrc makes that complicated. -- Micah Lee

4 4

Mbox sandbox
by Nicolas Vigier 26 May '14

26 May '14

On Fri, 16 May 2014, isis wrote: > Nicolas Vigier transcribed 2.6K bytes: > > > > - Looked at Mbox[2]: a sandboxing tool based on ptrace and seccomp/BPF. > > This can be used in the test suite to get the list of files modified > > by the browser after running a test, to check that it did not create > > or modify files in unexpected places. This can also be used to log > > all network connections, to check that everything goes through tor. > > I was previously thinking about doing that using Docker, but now it > > seems more simple with Mbox. > > Mbox is neat! It looks like it's git based, right? Or at least includes some > sort of CVS system. Either way, great idea, testing for connections not > matching " -> 127.0.0.1" should be easy. :) > > > [2]: http://pdos.csail.mit.edu/mbox/ Yes, it's nice! It's not git or CVS based. But it stores all new and modified files in a separate directory, doing copy-on-write when opening files with write permissions, by hijacking arguments of system calls which access files. And after running the program, asks you which copy of the files you want to keep (or you can manually copy the files from the sandbox directory). It can also be used to log in a file all network connections opened. Initially the filesystem sandoxing part didn't work with Tor Browser because of some bugs in Mbox, but it's now fixed so we'll be able to use it in the TBB test suite to monitor the files modified, created and removed by Tor Browser, and network connections made. Yesterday I also made a patch that allows filtering which connections can be made: https://github.com/tsgates/mbox/commit/6dd0e49202795564e627e9eeba664fc685b1… It could be used for instance to make sure a program will not connect anywhere without using tor. This can be done like this: $ cat tor.profile [fs] direct: / [network] block: 0.0.0.0 allow: 127.0.0.1:9050 $ mbox -p ./tor.profile -- curl -I http://www.google.com/ curl: (6) Could not resolve host: www.google.com $ mbox -p ./tor.profile -- curl --socks5-hostname 127.0.0.1:9050 -I http://www.google.com/ HTTP/1.1 302 Found Cache-Control: private Content-Type: text/html; charset=UTF-8 Location: http://www.google.co.in/?gfe_rd=cr&ei=GoiDU-76DcSU-wbdy4HgDg Content-Length: 261 Date: Mon, 26 May 2014 18:29:46 GMT Server: GFE/2.0 Alternate-Protocol: 80:quic $ mbox -o /dev/null -p ./tor.profile -- nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:32 CEST Nmap scan report for localhost (127.0.0.1) Host is up (0.0022s latency). Not shown: 999 closed ports PORT STATE SERVICE 9050/tcp open tor-socks Nmap done: 1 IP address (1 host up) scanned in 0.18 seconds $ nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:32 CEST Nmap scan report for localhost (127.0.0.1) Host is up (0.00048s latency). Not shown: 996 closed ports PORT STATE SERVICE 25/tcp open smtp 111/tcp open rpcbind 631/tcp open ipp 9050/tcp open tor-socks Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds It's also possible to kill a process which does unallowed connections, rather than just blocking those connections: $ cat tor.profile [fs] direct: / [network] kill: 0.0.0.0 allow: 127.0.0.1:9050 $ mbox -o /dev/null -p ./tor.profile -- nmap localhost Starting Nmap 6.45 ( http://nmap.org ) at 2014-05-26 20:37 CEST Stop executing pid=5298: Connect to 127.0.0.1 port 80 However, a warning if some people want to use this: it's still alpha quality software, so probably not a good idea to use it for something where security is important without more review.

1 0

New Feature: Tor Interpreter
by Damian Johnson 26 May '14

26 May '14

Hi all, after a couple months down in the engine room I'm delighted to announce a new Stem feature for advanced Tor users and developers! https://stem.torproject.org/tutorials/down_the_rabbit_hole.html The control interpreter is a new method for interacting with Tor's control interface that combines an interactive python interpreter with raw access similar to telnet. This adds several usability features, such as... * Irc-style commands like '/help'. * Tab completion for Tor's controller commands. * History scrollback by pressing up/down. * Transparently handles Tor authentication at startup. * Colorized output for improved readability. This is the last major feature going into Stem's 1.2.0 release, which is coming out later this month. Until then you can easily give it a whirl with... % git clone https://git.torproject.org/stem.git % cd stem % ./tor-prompt Running into an issue? Got a feature request? As always feedback appreciated! -Damian

1 0

Question to Java developers: what are good frameworks for Tor's server-side Java applications
by Karsten Loesing 25 May '14

25 May '14

Hello devs, I'm seeking advice from people with experience in writing server-side Java applications. Let me give you some background about this request: for the past five years, I have been developing server-side Java applications which all process large amounts of Tor directory data and provide their output via a web interface. Examples: - The metrics data processor (metrics-db) fetches Tor descriptors from the Tor directory authorities, the bridge authority, etc., performs some sanity-checks, and provides descriptors by type as tarballs. We're talking about roughly 7 GiB new bzip2-compressed data per month. - The metrics website (metrics-web) uses the output from the metrics data processor, stuffs everything into a database, computes aggregates, and presents results in graphs and .csv files. - The Onionoo service processes the same data from the metrics data processor, but provides statistics per Tor relay, not for the Tor network as a whole. The processing is done every two hours and may take 30 minutes to 1.5 hours, depending on how overloaded the server is. - The ExoneraTor service, again, uses the same data and puts it in a database to answer whether a certain IP address has been a Tor relay at some point in the past. That's what is done. And here's how it's done under the surface: - There's one or more cronjobs, each of which starts an ant task to process data. Some of these tasks import data into the database, others store results in the file system. - Each application uses a web application deployed in Tomcat to provide results to web users. Most things are written in servlets, some use JSPs. My problem is that this approach is rather fragile and difficult to setup for new volunteers. I'm aware of that, and I'd like to improve it. My question is: what Java frameworks should I be looking at for the applications described above? Bonus points if something is in Debian stable. Note that "switch to $some_other_programming_language" is not a very useful answer to me, at least not for the larger applications. There's just too much existing code and not enough developer time to port it. Thanks in advance! All the best, Karsten

3 3