- tor-dev - lists.torproject.org

Damian's Status Report - March 2013
by Damian Johnson 01 Apr '13

01 Apr '13

Hi all. Even without counting the Boston dev meeting March was a highly productive month. Noteworthy things include... * Stem Tutorials Stem's tutorials got an overhaul, including: * A much friendlier layout. No more intimidating wall of text - the tutorials have been rewritten and broken into subsections. * "To Russia With Love" tutorial, exemplifying client usage and programmatically managing a tor process. * "Tortoise and the Hare" tutorial, demoing tor event handling through a curses bandwidth graph. * "Double Double Toil and Trouble", which ends the tutorials with a page of scripts and applications that use stem. Feedback welcome! The shiny new tutorials are available at... https://stem.torproject.org/tutorials.html * Stem Packaging Thanks to a half dozen package maintainers stem is now available on several platforms, with more in the works... https://stem.torproject.org/download.html The most recent was the Python Package Index (PyPI), which can make stem installation as simple as 'pip install stem'... https://pypi.python.org/pypi/stem/ * Google Summer of Code The timing of this year's winter developer meeting couldn't have been better. During it I begged, bribed, and poked people with pointy sticks until they 'volunteered' to mentor something in this year's GSoC. Thanks to them our project ideas page is no longer overly sparse. Google will be announcing the selected orgs on April 8th. * Stem Release 1.0 Last, but certainly not least I wrote numerous finishing touches for stem and made its long overdue initial release! https://blog.torproject.org/blog/stem-release-10 Cheers! -Damian

1 0

Stem Release 1.0
by Damian Johnson 31 Mar '13

31 Mar '13

Hi all. After eighteen months of work and a number of delays rivaling that of the Big Dig I'm pleased to announce the initial release of stem! For those who aren't familiar with it, stem is a python controller library for tor. With it you can write scripts and applications that interact with your tor client or relay. For some examples of what you can do see the tutorials on... https://stem.torproject.org/ Stem is compatible with python 2.6 and higher (including the 3.x series), and is a near complete implementation of tor's control and directory specifications. It has relatively high test coverage (~80% for most modules) and integration tests to check its continued interoperability with new releases of tor. As always, if you encounter issues or have feature requests then please let me know! Also, if you write something that uses stem then please tell me, both so we can continue to improve our API and expand the tutorial's list of examples. Many thanks to everyone that helped make this initial release of stem possible, both with its development and packaging! -Damian

1 0

Re: [tor-dev] Using Tor as a library
by wac 29 Mar '13

29 Mar '13

Hi Fabio Pietrosanti: Thanks, I appreciate your help. I will be working on this. I actually was considering to maybe provide it to the community so people could pinpoint bugs and contribute patches. I will be studying onionbrowser. But seems they are running Tor as a thread inside the browser executing tor_main and then have the browser connect to itself using socks 5 which is somehow ugly. This looked interesting: https://lists.torproject.org/pipermail/tor-talk/2011-September/021528.html but seems Tor just evolved. > That's the future of Tor, to be integrated as a library just like an > encryption library into application. Same feeling here. Thanks for your comments Christopher Schmidt. I do appreciate to hear different points of view and know beforehand feelings of people against. Yet I disagree and I would like to argument why. >Embedding a Tor client in another application cripples >auditability, configurability, updateability etc. of Tor. Not necessarily. If it is open source would be auditable. I could dynamically link the library to let you update it on your own if you want. Also the application can be updated. But even if statically linked and not updated as the Tor network evolves the application would become unusable if the standard changes therefore killing the insecure application eventually. Regarding configurability I could proxy your configuration to the Tor library. Since it is open you can check I am being honest or if there are bugs even if such application is not open source. (you could even override received configuration from the application) I could let you configure the application to use embedded Tor or not, then you could use your own tor client if you want. And this is not entirely clear to me but I believe Tor could be routed trough Tor again if you are forced to use embedded library and you are worried about it being insecure. And in any case well you can opt in or out of the application. Yet as a library I believe there would be increase of useability. That means Tor will be more popular. If it becomes more popular it means more support. I mean I probably wouldn't be donating a relay to the network If I weren't using Tor. > Even worse, an application trying to outsmart >the user by controlling Tor on its own poses a severe security risk. Well an application using Tor as it is and properly configured could create a covert channel and totally do away with your anonymity. So I am supposing the application is not intentionally evil. >From another standpoint every security chain is as strong as its weakest link. Usually the weak link is the user miss-configuring things and then comes the false sense of security. As Fabio pointed earlier the DNS leak or the direct connection for example. In any case if you want to provide arguments against I would like to hear them. I tell you what. I don't have intentions to change Tor. I basically want to do this for me. But since I notice now it could be useful to the community and the community could be providing feedback I can make it public under the same license. So this is what I am going to do. Fork Tor under the same license and change it until I get a working library. If somebody wants to join well everybody invited. I think more hands working on it will be great. And if you want to merge it with Tor well you all can decide that later. And if you want we can cooperate directly. After all both projects would be sharing a lot of sourcecode. Many fixes and optimizations will be able to come back to Tor and the library could receive updates from the Tor project. I am thinking to name it libtor. What do you say? I think a good approach could be to start with the ugly hack creating a clean API that would get extended over time and then change bit by bit until we get something clean. Regards Waldo

2 1

Using Tor as a library
by wac 28 Mar '13

28 Mar '13

Hi Folks: I am looking to use Tor as a library for my program. I managed to build it from the sources including the required libraries. But now I am looking for hints on how to replace the SOCKS 4/5 or the transparent proxy with direct calls from my application to link them. I'm hoping somebody here can point me in the right direction. That is turning off all listening for incoming connections. Just my application going through tor circuits. I am looking not only to send/receive data over tor (looks I can do this touching connection_edge.c) but also to get callbacks reporting connection status (looks I have to change control_event_circuit_status and control_event_circuit_status_minor) and pass configuration data directly instead of loading from a configuration file (seems I have to call set_options at config.c with prefilled or_options_t structure). Just guessing. Regards Waldo

5 5

Re: [tor-dev] [tor-talk] Mobile Anonymity and Circumvention seems not working
by Nathan Freitas 28 Mar '13

28 Mar '13

Does the "check.torproject.org" site return a connected message in the default browser? chandra mohan <rkchandramohan(a)gmail.com> wrote: >Hi, > > I tried Trasparent proying of Orbot as mentioned >https://guardianproject.info/apps/orbot/->OTHER >*APPS* and observed all web (traffic built-in Browser, Gmail, YouTube, >Maps and any other application >that uses standard web traffic)and all DNS requests from my Android >mobile >are not following Tor >encryprtion. > > Observed with wireshark and above mentioned application traffics >are having DNS request and DNS >response packets and followed by large number of HTTP packets. > > My Mobile is running on Android 2.2 Kernel which is already root >and I also gave super user >permission for Tor installation. > > what is possibe issue here? > >Regards, >Chandramohan >_______________________________________________ >tor-talk mailing list >tor-talk(a)lists.torproject.org >https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

1 0

Tor Exit Images
by Griffin Boyce 24 Mar '13

24 Mar '13

Hey all, After talking to Wendy Seltzer, I decided to bring this up on the list. I frequently talk to people who would like to run an exit node, but who aren't as good a sysadmin as they'd like to be. It would be great if there were server images that could be fairly easily installed and then configured. All of these people so far have had the means to spend $150ish a month on the required hosting, they just felt that getting it running was a stumbling block. Thoughts? ~Griffin -- "What do you think Indians are supposed to look like? What's the real difference between an eagle feather fan and a pink necktie? Not much." ~Sherman Alexie PGP Key etc: https://www.noisebridge.net/wiki/User:Fontaine

3 5

Basics of Developing #newbie
by Ashish Patil 24 Mar '13

24 Mar '13

Hello! I am new to Tor-Dev & am looking forward to contributing as much as I can. My question is, after resolving any bug in my local copy of the source code, what are the next steps? Basically, do I have to build again? Run tests? And other similar tasks in what order? Hope you got my question. I went through this in the wiki & FAQs but did not find anything detailed. (Hopefully I'll write one when I know it all) --Ashish

1 0

[Stegotorus] Stegotorus stuff
by vmonmoonshine＠gmail.com 24 Mar '13

24 Mar '13

Hello TorDev, Here, I'm reporting my recent progress on Stegotorus as well as answering some questions from past. During past month, I mainly worked in "retransmit = false" mode trying to restore reliable browsing. I have resolved quite few bugs. Some of them where due to incomplete implementation of "retransmit = false" mode after implementing retransmission. But the major one seems to be the flush_interval implementation: if server had transmitted data to the client, the client opens a new connection, "probably" in near future, to retrieve more data. The probability follows geometric distribution to simulate user browsing behavior. This is OK most of the time. But when you transmit 1000s of packets, inevitably it happens that it samples delays such 40-60 seconds. Unfortunately, Firefox isn't patient enough to wait so long to receive the rest of the data and judge the connection as dead. I changed this now that if it real data came from the server on previous connection, the client opens a new connection 100msec which I know isn't the point but I wanted to restore reliability. The better way might be to schedule opening few connections at a time (instead of one). In this way it is very improbable that all of them sample unacceptable delays. I am going to debug retransmission in the next step. Now few questions, that I was putting off partialy cause I was tired talking and I wanted to do something practical before talking more: From: Roger Dingledine <arma(a)mit.edu> on Sun, 13 Jan 2013 13:04:23 -0500: > 3) What's the overhead of putting your Tor traffic through each of the > steg modules? Now the chopper is measuring the efficiency of the steg modules based on the following measure for each circuit: (no of byte transmitted from upstream)/(no of bytes transmitted downstream ino order to transmit upstream bytes) The summary of the results measured using speedtest.net: |-------------+----------+---------+------+------| | | Mb/s | Mb/s | Efficiency | |-------------+----------+---------+------+------| | Steg module | Download | Upload | S->C | C->S | |-------------+----------+---------+------+------| | Direct | 19.52 | 2.08 | 1 | 1 | | nosteg | 12.98 | 2.26(!) | 0.98 | 0.98 | | http | 0.28 | 0.31 | 0.03 | 0.07 | | http_apache | 1.43 | 0.47 | 0.65 | 0.52 | |-------------+----------+---------+------+------| (!) The faster upload of nosteg than the direct connection with Stegotorus is persistent and I can't figure out why. it might be a bug with speedtest.net algorithm or a their weird notion of speed. If you are downloading, the server->client efficiency is like this: |-------------+-----------------| | Steg module | S->C Efficiency | |-------------+-----------------| | Direct | 1 | | nosteg | 0.98 | | http | 0.10 | | http_apache | 0.65 | |-------------+-----------------| The efficiency of course varies depending on the cover texts. These are measured for http module using the fake cover generated by zack and for http_apcahe using random javascripts that I have downloaded plus the pdf files I had on my computer. The advantage of http_apache over http steg module is due to http_apache always chooses the most efficient cover possible verses http module which randomly chooses 10 candidates and then chooses the best one among them. The http_apach approach, however, isn't security-wise plausible. The best solution seems to be hybrid of the two with user having a say how efficient/secure wants it to be. With all my cat and mouse with curl, it still manages to steal data from libevent and I lose 1 in 100 packets or so. In presence of retransmission this is not a big deal but I have to solve it in more fundamental way. From: George Kadianakis <desnacked(a)riseup.net> on Wed, 30 Jan 2013 14:09:35 +0200: > The code monkey inside me doesn't see any code-quality-related tickets > in that URL. stegotorus boasts 12k LoC of C++ and some of that code > was written in haste. You can divide the ST code into three major parts. Network, chopper and the steg modules. The network is basically, obfsproxy. For the chopper, beside Zack's obsession with overloading functions with the same name by changing type/no of parameters, I think the rest is a pretty good standard code and well commented (I suspect it is very similar to a typical TCP implementation). It lacks is a flow chart that explain what exactly happens when data is received or sent. The ugly code are the steg modules, (partially because it is in C rather than C++), but then the steg modules are not innate part of Stegtorus and are meant to be re-implemented by interested party. Having said that, Zack has mentioned that hardening the code is number 1 priority. Just because he mentioned it later, I haven't included it in the track list yet. > Looking at this from a deployment point of view, > I feel much more relaxed and easygoing with deploying programs written > in a high-level language (like Flashproxy or pyobfsproxy) than with > deploying an unaudited C++ program with many bad code moments. I suspect that python inefficiency will break Stegotorus completely. Now when I run Stegotorus it eats all CPU time. I know this is partially due to heavy logging and partially because I compile it with zero optimization and full debug but I can only imagine how it would look like if it was written in python. There are lots of exact timing there in scope of msec. These are not important when you are running nosteg module which is basically obfsproxy with variable packet size. But when you turn to simulating http with multi-connections and constant negotiation of chopper with steg modules, I doubt if python is an option. Beside, the steg algorithms can get quite complicated. > I think that a trac ticket with a bit of research on how painful would > be to write a deployable <high-level language>-port of stegotorus > would be a good idea. I know that Zack is also interested in exploring > this avenue. I haven't looked into pyobfsproxy, but translating the chopper into python over pyobfsproxy doesn't seems to be an impossible task (you are looking at around 3000 LoC). But then you still need some steg module to get something more than pyobfsproxy (if we close our eyes on the preformance doubts I raised). Maybe, having a cython/c interface to steg module be a middle way compromise. Wow I talk too much! I'll spare everybody for now. Vmon

1 0

Does anyone need python 2.5 support in stem?
by Damian Johnson 22 Mar '13

22 Mar '13

Hi all. Will anyone be sad if I drop python 2.5 support from stem (just supporting 2.6 and higher, including the 3.x series)? If all goes well stem will be having its first release this next weekend and my thinking is this... * Python itself considers python 2.5 to be obsolete, having dropped docs for everything below 2.6. * Every OS I can think of except maybe CentOS has at least 2.6, and more likely 2.7. * I have a dozen dirty hacks throughout the codebase just for 2.5 compatability and I'd love to get rid of them. I'm presently working on a bug that needs the io module which was introduced in 2.6, so I'll need another gross hack for 2.5 support after this too... :( * Testing with 2.5 is a pita for me, so in general I don't (only when a 2.5 bug is reported or prior to a release). * I've never heard of someone actually using stem with python 2.5, so maintaining backward compatability this far back is likely a waste of time. Cheers! -Damian

2 1

Stem Tutorials
by Damian Johnson 19 Mar '13

19 Mar '13

Hi all. Stem is having its initial release later this month, and in anticipation of it I've been expanding its tutorials... https://stem.torproject.org/tutorial.html Are there any rough spots in these tutorials? Other categories that we should cover? Suggestions welcome! -Damian

3 3