- tor-dev - lists.torproject.org

Deployability of Python software.
by Arturo Filastò 10 Mar '12

10 Mar '12

We were discussing last night with George about deployability of python application on multiple platforms. In particular how it would work out if there were to be a python port of obfsproxy and we wanted to have it deployed inside of the Tor Browser bundle. The issues that he said were raised in other discussions with Nick and Roger are mainly the following: - How do we get a good Windows binary of the Software? - How do we keep the size down to an acceptable level? - What kind of performance drawbacks would we be experiencing? - Is it even secure to do crypto in python? I will try and address these issues as they are something that I ran into also while designing AWAF (Anonymous Web Application Framework): http://wiki.globaleaks.org/index.php/Awaf and https://piratenpad.de/p/AnonymousWebApplicationFramework For packaging python software on Windows and OSX, what is generally done is shipped a precompiled python interpreter and bundle everything up with a nice bow. This technique is already quite tested in real world applications: an example that I particularly like is Tucan Manager (http://www.tucaneando.com/development.html) This application is basically a download manager written in python and gtk. The final size of the packaged software is 20MB. If you remove gtk this size goes down to around 10MB. What they are using to bundle up the application for Windows is py2exe and py2app for OSX. Another very widely used solution for packaging python applications in PyInstaller and that is probably the solution I would recommend. Quite a few open source software uses it already: http://www.pyinstaller.org/wiki/ProjectsUsingPyInstaller George also mentioned to me pypy, though I don't think pypy is ready for building shippable application just yet. The issue of size is something that we should come to an agreement on what is acceptable. What is the maximum size that we are comfortable with shipping? We are already shipping a TBB that has 25 MB of QT libraries in it, I don't think a 13 MB Python interpreter is going to be killer. With respect to performance I don't think it is particularly an issue. Python is pretty fast and if it is not fast enough for what needs to be done you can always rewrite the code in C and integrate that piece of application logic as a python binding. By talking to some of the core python developers my understanding is that there is a way of securely storing keys in memory and wiping that memory region in python. It involves using bytearray. We you override a cell in a byte array you are not simply dereferencing the pointer to the python struct, you are actually overwriting that portion of memory. I think I might write a blog post about this and illustrate what other python crypto software is using to solve this problem (PyCrypto etc.). In conclusion having a python interpreter shipped as part of Tor would allow developers of anonymity related software to integrate their "Tor add-ons" into a Tor bundle easily. I am thinking of for example making a Tor IRCD bundle, a Tor HTTPD bundle, etc. What do you think? - Art.

5 5

New IP-address for directory authority maatuska
by Linus Nordberg 29 Feb '12

29 Feb '12

Hi, The provider where maatuska, my directory authority, has been hosted for the last two years or so has had some trouble with its switches lately. I was unable to keep maatuska on the network enough to be a good participant of making a consensus. I decided to move the machine to a place where I have physical access 24x7 and can run it on a network under my control. Other directory authorities need to change its idea of where maatuska lives in order to make it part of the consensus making. Clients benefit from the same change, for downloading the consensus. Nick, Roger, can you please apply the following patch. --8<---------------cut here---------------start------------->8--- diff --git a/src/or/config.c b/src/or/config.c index d341d63..3a7bf65 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -928,7 +928,7 @@ add_default_trusted_dir_authorities(dirinfo_type_t type) "208.83.223.34:443 0AD3 FA88 4D18 F89E EA2D 89C0 1937 9E0E 7FD9 4417", "maatuska orport=80 no-v2 " "v3ident=49015F787433103580E3B66A1707A00E60F2D15B " - "213.115.239.118:443 BD6A 8292 55CB 08E6 6FBE 7D37 4836 3586 E46B 3810", + "171.25.193.9:443 BD6A 8292 55CB 08E6 6FBE 7D37 4836 3586 E46B 3810", NULL }; for (i=0; dirservers[i]; i++) { --8<---------------cut here---------------end--------------->8--- Thanks. -- Linus

2 1

Pluggable Transport through SOCKS proxy
by Arturo Filastò 29 Feb '12

29 Feb '12

Following up on the discussion at the Tor dev meeting with Nick this is the proposal for the Pluggable Transport through SOCKS proxies. You can also find it committed on github: https://github.com/hellais/torspec/commit/f0a89c8ded1971199fec6ffd0fe7e4056… Filename: xxx-pluggable-transports-through-proxy.txt Title: Pluggable Transport through SOCKS proxy Author: Arturo Filastò Created: 28 February 2012 Status: Draft Overview Tor introduced Pluggable Transports in proposal 180 Pluggable Transports for circumvention. The problem is that Tor currently cannot use a Pluggable Transport proxy and a SOCKS proxy at the same time. This has been noticed by users in #5195, where Tor would be failing saying "Unacceptable option value: You have configured more than one proxy type". Trivia This comes from a discussion that came up with Nick and I promised to write a proposal for it if I wanted to hear what he had to say. Nick spoke and I am writing this proposal. Acknowledgements Most of the credit goes to Nick Mathewson for the main idea and the rest of it goes to George Kadianakis for helping me out in writing it. Motivation After looking at some options we decided to go for this solution solution since it guarantees backwards compatibility and is not particularly costly to implement. Design overview When Tor is configured to use both a Pluggable Transport proxy and SOCKS proxy it should delegate the proxying to the pluggable transport proxy. This can be achieved by setting the environment variables for the SOCKS proxy to that specified inside of the torrc. When the pluggable transport proxy starts it will first read the environment variables and if it detects that it should be using a SOCKS proxy make all it's traffic go through it. Once the pluggable transport proxy has successfully established a connection to the SOCKS proxy it should notify Tor of it's success or failure. When both the SOCKS and the PluggableTransport directives are set Tor should set the environemnt variable start the pluggabletransport proxy and wait for it to report back on the SOCKS proxy status. If the pluggable transport reports back a failure or it does not report back at all (maybe because it is an outdated version), Tor should notify the user of the failure and exit with an error. The environment variables can also contain the credentials for accessing the proxy. Specifications: Tor Pluggable Transport communication When Tor detects a SOCKS proxy directive and a Pluggable Transport proxy directive it sets the environment variable: "TOR_PT_PROXY" -- This is the address of the proxy to be used by the pluggable transport proxy. It is in the format: <proxy_type>://<user_name?>:<password?>@<ip>:<port> ex. socks5://tor:test1234@198.51.100.1:8000 socks4a://198.51.100.2:8001 If the pluggable transport proxy detects that the TOR_PT_PROXY environment variable is set it attempts connecting to it. On successs it will write to stdout (as specified in 180-pluggable-transport.txt) PROXY true. On failure it should write PROXY-ERROR <errormessage>. If Tor does not read any PROXY line or it reads a PROXY-ERROR line and it is configured to use both SOCKS and PluggableTransport it should exit with error.

2 3

Tor and DNS
by Linus Nordberg 26 Feb '12

26 Feb '12

Hi, After some interesting discussions irl last week with knowledgeable DNS and security people (hi Jakob) I'd like to hear from people involved with DNS in Tor what current status is and what needs to be done. More specifically, what's the status of ttdnsd and TorDNSd? Are they being used? Any thoughts about having a local validating resolver? I know there's been some discussions (4zm, are you here?) about using libunbound (which could be interesting for DNSSEC support). Did that evolve into anything useful? I'm by no means a DNS expert but would love to see some discussion about this, partly because future IPv6 work will depend on changes to our DNS system. Thanks, Linus

9 30

Windows Alternative of torsocks/tsocks ?
by Catalin Patulea 18 Feb '12

18 Feb '12

[resending after subscribing to list] Hi Fabio, > On 1/22/12 11:28 AM, Roger Dingledine wrote: >> See also TorCap2: http://www.virtualventures.ca/~cat/ >> but I've never succeeded at getting a license statement out of the author. Roger, I'm sorry if I seemed uncooperative when you asked about the licensing terms of TorCap2. I can see a reference to a discussion with you back in 2006, but I no longer have copies of the actual correspondence, so I don't recall what happened. > Can you confirm whether the TorCap2 is licensed under BSD, GPL or other > opensource license? I have released TorCap2 under the LGPL 3.0 (I have updated the readme and included a license.txt in the zip files). You can use TorCap2 as-is in an application of any license. Any modifications to TorCap2 must be published under the LGPL. > at some Tor related Project we would like to use TorCap2 as a part of an > OpenSource Project whose concept is described at > https://piratenpad.de/p/AnonymousWebApplicationFramework . On a technical note, I doubt that TorCap2 is the right approach here. If you can limit yourself to support Python-only applications to start, then a SOCKS client in Python that implements the same interface as the 'socket' module seems like a much cleaner solution to me. This is now platform-independent, less complex (easier to debug), and gives the Python code the *option* between native sockets and Tor-ified sockets, under the same interface. (maybe you want to check connectivity.. or find the hosts's external IP by fetching a URL.. or talk to hosts on the local network.. etc.) Porting existing Python code would be a matter of search-replacing 'socket' with 'torsocket' (or even just 'import torsocket as socket'). Of course, with this design you would leak network traffic if you spawned a separate process, or if you used a Python native library that directly accesses the OS's socket API. But maybe it's good to start with supporting Python-only applications initially, given the complexity of Win32 API hooking.. Just my $0.02.. Catalin

3 3

Unit testing blog posts
by Esteban Manchado Velázquez 18 Feb '12

18 Feb '12

I ended up publishing it in two parts :-) Here are the URLs: http://hcoder.org/2012/02/13/unit-testing-advice-for-seasoned-hackers-12/ http://hcoder.org/2012/02/15/unit-testing-advice-for-seasoned-hackers-22/ Now I'll have a look at the coverage and see what I can do for the unit tests before I move on to Stem/chutney. -- Esteban

3 3

Fwd: Plan of attack for part (c)
by Watson Ladd 17 Feb '12

17 Feb '12

I'm forwarding the following 2 messages about work that I'm thinking about doing to the Tor code to pave the way towards multiple cryptographic methods for route extension. Feel free to comment critically on the proposed design, especially if it impacts areas you are working on or know about. If you have alternative ideas that's also good. Sincerely, Watson Ladd ---------- Forwarded message ---------- From: Robert Ransom <rransom.8774(a)gmail.com> Date: Thu, Feb 16, 2012 at 8:03 PM Subject: Fwd: Plan of attack for part (c) To: Nick Mathewson <nickm(a)torproject.org>, George Kadianakis <desnacked(a)riseup.net> Cc: Watson Ladd <watsonbladd(a)gmail.com> Forwarding to Nick and George (who has looked at the crypto code before), because I'm not going to be able to think about this much today. This sort of discussion should probably be on the tor-dev mailing list. ---------- Forwarded message ---------- From: Watson Ladd <watsonbladd(a)gmail.com> Date: Thu, 16 Feb 2012 13:15:58 -0600 Subject: Plan of attack for part (c) To: "rransom.8774" <rransom.8774(a)gmail.com> Dear Robert, I've come up with the following sketch of a plan to handle the obstacle of crypt_path_t to multiple negotiation methods. It isn't actually the problem. The problem is the crypto_dh_t * dh_handshake_state. I propose defining an enum handshake_methods_t and a handshake_state that looks like typedef struct { enum method; union { crypto_dh_t *dh_handshake; } u; } handshake_state; The extend_info_t structure will need to be modified to point to a list of similar handshake_key structures, probably in the form of a linked list. This involves modifying the functions that allocate extend_info_t's, of which thankfully there are few. Once this is done we can genericize the onion_skin functions by making new functions which essentially consist of a switch statement on the value of method. One unresolved issue is selecting the method to be used when multiple methods are available: I'll probably just take the first handshake_key and complain if it doesn't match what the handshake_state should be when processing the response. Servers are a trickier issue, but if they have a list of handshake_keys we can sort through it to find the key that matches a particular method, and rely on the new CREATE2 cell mechanism to tell us what the method is. Are there requirements for a fix that this idea doesn't meet? Any hard decisions that I've made the wrong call on? And thank you very much for taking the time to help me through working with the tor code. Sincerely, Watson Ladd -- "Those who would give up Essential Liberty to purchase a little Temporary Safety deserve neither Liberty nor Safety." -- Benjamin Franklin

1 0

Talk on flash proxies Feb. 17
by David Fifield 13 Feb '12

13 Feb '12

If anyone is interested in the bay area, I'll be giving a short tech talk on flash proxies 6:30–7:00 PM on February 17. Gates Computer Science building room 104 353 Serra Street Stanford, CA 94305 http://infolab.stanford.edu/pub/keller/gates-map.html David Fifield

1 0

Obfsproxy client for Android
by Nathan Freitas 12 Feb '12

12 Feb '12

Thoughts on attempting to port and ship Obfsproxy client functionality to Android? We have a good number of Iranian users it seems, and I think we can pull it off in a few days, if it isn't insanely complex. Where should we begin? Any details on exactly what was done for the new TBB? +n

2 4

First test "report"
by Esteban Manchado Velázquez 12 Feb '12

12 Feb '12

Hi there, I'm done with the first batch of work on the test side. You have the (rebased just now) work here: https://github.com/emanchado/tor/commits/master. The rest of this e-mail summarises what I've done and explains my plan for further work. The content of the commits -------------------------- I have reviewed the whole test_util.c and made three kinds of changes: 1. Added more tests. Some of them failed, and after checking with other people on IRC the conclusion seems to be that they should pass (ie. they reproduce actual bugs in the code). To keep the test suite from failing, they are inside "#if 0" blocks. So someone should look for "#if 0" inside test_util.c and fix the code that makes those fail. Or maybe I should file bugs for those? 2. In comparison assertions, the general convention seems to be to place the expected value first ("test_eq(0, functioncall(...))" rather than "test_eq(functioncall(...), 0)"). I have modified the assertions not following that convention, so they all look the same. 3. General clean up, small code reorganisations, fix typos and such. Eg, I have turned all the "tt_int_op(a, ==, b)" into "test_eq(a, b)". I was thinking of blogging about what I saw (esp. related to point 1). I think there are valuable lessons to be learned, which will help other people writing tests (both for Tor and outside of Tor). I'm not sure if there's enough content for a blog post, but if I do it after all, should I post the link here? Plan for the future ------------------- As I haven't explored coverage yet, I'll start with that for the code in src/common/ and see what I can do. My gut feeling (without having looked at other parts of the code yet!) is that tackling unit tests for code in src/or/ will be hard for me, and it seems better to explain what kind of things I have done in src/common/ so others, more familiar with that code, can fix/improve the unit tests for src/or/*. In other words, it seems much more effective for me to support someone who knows the code in src/or/*, rather than the other way around. After I'm done with what I think is easy enough for me to do with the unit tests, I'll have a look at Stem (Damian's suggestion) and chutney (Nick's suggestion), and decide what I do next. -- Esteban

3 3