- tor-dev - lists.torproject.org

use sphinx on stem
by Beck Chen 05 Jun '12

05 Jun '12

Hi Damian, I explored sphinx and tried it on stem these days, and it seemed pretty handy. I think a good way to produce nice documentation for stem is to create a page for each module manually, write an introduction by hand, and use autoclass / autofunction / autoexception in the autodoc extension to create API docs. automodule seems too general and does not work well with module's docstring in stem since it is not written in reST format, so I don't like it. What do you suggest? And how's your progress now? Cheers, Beck

2 3

txtorcon, async Tor controller
by meejah 05 Jun '12

05 Jun '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Erring on the side of "release early, release often" I have put my Twisted-based (asynchronous, Python) Tor control protocol implementation online: http://readthedocs.org/docs/txtorcon/en/latest/ https://github.com/meejah/txtorcon It is MIT licensed (to match Twisted). I would certainly not consider it "done", and I made it to learn more about Twisted and Python -- criticisms, comments appreciated. Currently it has the following features (see the above-linked documentation for more, and examples): . TorControlProtocol implements the control protocol . TorState tracks the state of Tor (streams, circuits, routers, address-map), listening for updates . TorConfig provides read/write configuration access , with HS abstraction (still needs some work) . IStringAttacher, a stream-to-circuit attacher interface for new streams . launch_tor can launch slave Tor processes . integrates into Twisted's endpoints with TCPHiddenServiceEndpoint The main code is about 1600 LOC, ~4000 with tests and 25% comments (according to ohcount). There is currently 98% test coverage, if one believes code-coverage is a good metric. In the short-term, be aware that I'm planning to re-organize where things are in files. If you "import txtorcon" and use the classes like "txtorcon.TorConfig" it will all still work. Thanks for your attention, mike -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJPczCxAAoJEMJgKAMSgGmnTrEH/RG1TLbEsqALWyh5WSm1azYU 7QHx9eup+/NKUE8C6WLPGQyprTkL/snIRZZGYDdkz5grkxcsYaWaVNNtDdUTdctN KCi2E3rbzdUYHV0aN/VdoNvJdpa8H3J2dpyx4/kFmZ2Z04+VLZOqeX6ANMdYZbYv FXv37j0dnl15h+t57+65Cf5c8BVbSW50vqXUx/eHWS73BISq3LP30OV4Ut8k3Xbg IXVf1S/EFeoXxRoGfn9i4i4txeQNyQxCOX0k+fynvIGP+lFuYciSGgGJydYBIkhE 87TMJ//c1tPq41jn5prdbWRTE4mPWA5U03w35wUGrhUWSNUb+OhM6fV4vdRwq30= =ilGQ -----END PGP SIGNATURE-----

3 6

GSoC Intro: Stegotorus
by vmon 04 Jun '12

04 Jun '12

Hello Tor-devers, It is a beautiful spring/summer day and this is vmon. I m going spend my summer with you guys. You might think that I m a bit off schedule with such a late GSoCc intro, but till 2 days ago I didn't know what I was going to do this summer for tor, so I couldn't write much before. I've been using tor for a long time. I don't remember exactly when was the first time I encountered Tor but I think it was somehow through JonDo. Being behind censor, you always desperately google the word "proxy", and "Java Anonymous Proxy" came up in once. Reading more about it, lead me to Tor. So I was a Tor user for a while, when I heard of gsoc. I talked to a friend of mine who had a gsoc experience before. He told me that Tor/EFF have some slots. It was a very interesting for us as we were both frustrated with our government behavior and we wanted to do something about it. We kept submitting our idea for a while (OK, twice) and kept getting rejected. So finally, I decided to deal with our concerns in a more "tor-friendly" way. One of our concerns (which is getting more serious year after year) was that what if the government decides to close down any protocol that they don't like, and only let few common protocols, e.g. http, pass through. How useful would Tor be in such a situation? Around the same time, obfsproxy came out of no where, so I thought, one way to overcome this problem was to use obfsproxy to send tor over http. After submitting my idea, I found out that at the end of the day, it wasn't *that* original. Zack/zwol had worked on it for a year. It is called Stegotorus. However, considering, the arm-race nature of the problem and the state of development of Stegotorus, tor people (specifically asn) suggested me to polish, improve and strengthen Stegotorus, instead of re-inventing the wheel. The final obstacle, that Stegotorus wasn't open, was resolved last week and I am very excited to deliver you a strong more robust Stegotorus by the end of the summer. Zack is my primary mentor, followed by Steven (sjmurdoc1) and Roger (armadev) (according to Tor blog). I'll be sticking around #tor-dev (vmon) as much as possible and I'm happy to know, talk and learn from everyone of you. See you on #tor-dev Cheers, vmon P.S. Dear GSoC Admin, I know I'm late on my first bi-weekly update but I thought my updates makes much more sense to people, If they are proceeded by an intro. I'll send my updates shortly.

4 3

TorStatus
by Felix Ker 04 Jun '12

04 Jun '12

I'm trying to use TorStatus and it took me almost 2 hours to set everything up and able to run tns_update.pl smoothly with no error. However, I am facing a problem that Google cannot help me. [0] starting... [0] mirror? [0] connecting to Tor [0] authenticating with Tor [0] starting descriptions *The TorStatus database was not updated properly. An error has occured. I will continue to try to update, however.* It says an error has occured but I am unable to find any error logs. Anyone? ___ Best Regards, Felix Ker Mobile +65-91456635 Email felix(a)ker.sg *IMPORTANT* Find out the best ways to get my attention: http://protocol.by/felixker/

3 5

[GSoC] Vidalia - Status Report
by Feroze Naina 02 Jun '12

02 Jun '12

I have completed the prototype for Torrc and I've tested it writing a plugin. So far value(), setValue(), TorrcPath() functions are confirmed working. I've still not been able to get clear() working and the execution simple stops. I'm not sure if there is an error in the plugin or the prototype without the debugger. I'm planning on integrating QScriptEngineDebugger to help writing and testing plugins. I didn't work in the first week during exams. My commits - https://github.com/feroze/vidalia/commits/hiddenservice

1 1

[GSoC] Pluggable Transports in Python Status Report
by Brandon Wiley 01 Jun '12

01 Jun '12

Deliverable #1 for this summer’s project is this: “A library for parsing pluggable transport configuration options This will be a python library that authors of SOCKS proxies can use to integrate their proxies with Tor.” A first pass at this is available from github and pypi: http://github.com/blanu/pyptlib http://pypi.python.org/pypi/pyptlib/0.1 I tried to follow the spec very closely in developing the API. Next steps for the library are testing, improvement documentation including pydoc strings, and error checking for higher level protocol conformance, for instance the order in which proxy reply lines are output. After that, it will be time to use the library in building the pluggable transport framework. Status updates can also be read on my project blog: http://stepthreeprivacy.org/

1 0

Damian's Status Report - May 2012
by Damian Johnson 01 Jun '12

01 Jun '12

Hi all. As you may have noticed from the elevated traffic on this list a few of us are trying to do more development discussions in public. To that end I'm bucking tradition and sending my monthly status report here too. They've always been public... http://www.atagar.com/arm/log.php ... but since their initial emails have been on a closed list I doubt that most people knew that they existed. On that note, if anyone wants to hack on fun tor related python stuff then let me know! Stem is always looking for new people that want to get involved. ======================================== Hi all. Spring is in the air, and with it many lovely things including the UW Street Fair, Folklife, and an iSec Open Forum. It also included a week of oncall but that doesn't count in the 'lovely things' column. On the upside though, this time it was pleasantly light (first time I've gone a whole week without being woken up by a pager at 3am!). As a GSoC admin I don't have much to report besides a blog posting at the start of the month [1]. However, as a mentor some neat things are in the works... * Ravi is adding SafeCookie support in stem [2]. I've code reviewed the first couple iterations and it's looking great. Given some tests and revising to fit with recent refactoring it should soon be ready to merge. After that he'll start working on the general controller. This last weekend I refactored how response classes are organized and implemented GETINFO as an example, so this will hopefully be an easy project for him to start hacking on. * Beck added descriptor validation to check that the signing key's hash matches its fingerprint [3]. This is the first part of descriptor integrity validation that Karsten suggested, however this project has gotten stuck so he's moving on to other stem tasks. * Investigated a couple issues brought up by Sathyanarayanan [4][5]. * Two Wesleyan students will start working on stem soon! [6] Needless to say helping these projects has occupies much of my time, and will take even more once the Wesleyan students get started. However, after years of trying in vain to attract developers to my projects I wouldn't have it any other way. :) Other stem tasks I finished this month includes... * ExtraInfo descriptor parsing. This took me a couple weeks since it contains so many attributes, but I'm glad it's finished. Now all that remains before we can port Onionoo are the consensus' network status entries. That's now at the top of my todo list for descriptor work, but I'm setting that list aside for now in favor of Sphinx and helping Ravi and Beck with controller work. * Improved the launch_tor() function, making the test instance easily configurable via a temporary torrc (similar to what Vidalia does) and adding integ tests. I also added a workaround so it'll work on Windows [7][8]. * Updated the stem development wiki [9] so it'll be easier for new volunteers to find tasks that interest them. * Discussed descriptor type annotations with Karsten and implemented stem's side of it [10]. We also discussed some changes to bridge descriptor sanatization which led to some minor stem changes too. Cheers! -Damian [1] https://blog.torproject.org/blog/gsoc-2012-projects [2] https://trac.torproject.org/5262 [3] https://trac.torproject.org/5810 [4] https://trac.torproject.org/5917 [5] https://trac.torproject.org/5918 [6] https://lists.torproject.org/pipermail/tor-dev/2012-May/003547.html [7] https://trac.torproject.org/5783 [8] https://trac.torproject.org/5493 [9] https://trac.torproject.org/projects/tor/wiki/doc/stem [10] https://trac.torproject.org/5651

1 0

Fwd: Re: Can we stop sanitizing nicknames in bridge descriptors?
by Karsten Loesing 29 May '12

29 May '12

Forwarding my original answer to Sebastian here. -------- Original Message -------- Subject: Re: [tor-dev] Can we stop sanitizing nicknames in bridge descriptors? Date: Mon, 21 May 2012 19:56:34 +0200 From: Karsten Loesing <karsten(a)torproject.org> To: Sebastian G. <bastik.tor> <bastik.tor(a)googlemail.com> Hi Sebastian, On 5/21/12 7:08 PM, Sebastian G. <bastik.tor> wrote: (Did you intend to send this mail only to me, not to tor-dev? Feel free to move the discussion back to tor-dev if you want.) > Karsten Loesing, 21.05.2012 11:05: >>> >>> Here we go with the similarities of bridge and relay nicknames. >> >> Thanks for spending this much time on the analysis! > > I could have done far worse, but also a lot better in terms of time > spend on extracting the data that I wanted or at least considered that > they'd might be useful. > > Sometimes I'm just slow at things, e.g. writing this reply. > >> Here's what I did with your findings.txt: >> >> - extract unique fingerprint pairs of relays and bridges that you found >> as having similar nicknames, >> >> - look through descriptor archives to see if relay and bridge were >> running in the same /24 at any time in May 2008, and >> >> - determine the absolute and relative number of bridges in a given >> network status that could have been located via nickname similarity. >> >> Results are that 24 of your 81 guesses (30%) were correct in the sense >> that a bridge was at least once running in the same /24 as the relay >> with similar nickname. At any time in May 2008, you'd have located >> between 1 and 6 bridges (2.5% to 18%) with 3 bridges (10%) in the mean >> via nickname similarity. > > Not too bad. I agree. :) >> I think it's acceptable to publish more recent bridge descriptors with >> nicknames in a week from now. Results may look quite different with >> 1000 bridges instead of 30. > > May 2008 was the first month with bridges. I expected lot's of relay > operators that tested a bridge with the same name. Things may have > changed over time. I assume that further comparisons won't have such a > "high" hit ratio. That would be my guess, too. In May 2008, only a few early adopters were running bridges, and most of those probably ran relays at the same time, too. Plus, they were enthusiastic and put some energy in finding cool nicknames. It might be that this has changed since then. To be honest, I didn't look at 2012 tarballs yet. >> Again, thanks for running this analysis! Maybe you're interested in >> automating your comparison and re-running it for a 2012 tarball? > > My claim was you got the data, so you can check. (Not with May 2008) > > To be honest, my first impression was that I wouldn't do anything useful > and did not intend to do that. I guessed it wouldn't turn out that it > doesn't hurt since at least 2011, so I wouldn't find anything good. > > Then you asked and I agreed, but already thought "I couldn't keep my > mouth shut!". I mean I replied to this topic. I surely could have said > no there. I didn't. > > After and while I was doing what I did. I would have said no to the > question if I'm going to do this again. That's valid for up to Sunday > night. Today I'm agreeing again. > > That's a pretty long way to say: Yes! Hah, great! :) I'm going to make the 2012 tarballs available next Wednesday (May 30), assuming that my poor Linux box doesn't run out of $resource. I'll let you know. > Thank you,it's an 2012 tarball. The number of bridges is scary. > > I'm going to upload some files somewhere and explain what I did. Step by > step (somewhat around that). So anyone can check and reproduce what I > did. It would be nice to hear feedback and ways to improve the way I did > what I did. > > Maybe you can tell me if the findings.txt was alright. Yes, the file format was fine. > Unless one objects or you disagree I'm going to upload the files I > created and explain how and maybe I can say even why. No objections at all. Open discussion is good. > I created a Blog, just because I wanted it some when in the past, but > found it silly. That's the channel I planed to use. Maybe it's OK to put > it on a Tor-List as well, but maybe it's considered as noise. I wonder if the Tor wiki would be a better place to collect ideas for reversing the bridge descriptor sanitizing process. Feel free to grab a new page in doc/ and start describing what you did. Best, Karsten

2 4

[GSoC] Update - Safe cookie support in Stem
by Ravi Chandra Padmala 28 May '12

28 May '12

Hello I'm reporting my progress over the last week. I have been working on implementing support for the safe cookie authentication method for Stem[1]. It is in a usable state[2] right now, but, it isn't fully tested. In the process, Stem also helped uncover a bug in the AUTHCHALLENGE error responses[3]. Now, I'll be writing the tests for the safe cookie implementation and getting it merged. Next, I'll begin work on the general controller class. Damien has already implemented an the GETINFO response parser. Beck also wants to help implement the controller class. I/We will be working on the deliverables scheduled for week two in my proposal, i.e. Implementing the wrapper functions and response parsing for the following control commands, GETCONF, SETCONF, LOADCONF, RESETCONF, SAVECONF, SIGNAL, TAKEOWNERSHIP, USEFEATURE and QUIT. 1. https://trac.torproject.org/projects/tor/ticket/5262 2. http://repo.or.cz/w/stem/neena.git/shortlog/refs/heads/safe-cookie-alt 3. https://trac.torproject.org/projects/tor/ticket/5760 -- Regards, neena

1 0

Sanitized bridge descriptor format 1.0
by Karsten Loesing 24 May '12

24 May '12

Hi Damian, I plan to make a few changes to the bridge descriptor sanitizer to implement changes discussed on this list and in various Trac tickets. The result will be format version 1.0. Here's what will change compared to the current (unversioned) format. Can you take a look whether stem would be happy with these descriptors and if there are other tweaks I should do to make it even happier? - Bridge network statuses contain a "published" line containing the publication timestamp, so that parsers don't have to learn that timestamp from the file name anymore. - Bridge network status entries are ordered by hex-encoded fingerprint, not by base64-encoded fingerprint, which is mostly a cosmetic change. - Server descriptors and extra-info descriptors are stored under the SHA1 hashes of the descriptor identifiers of their non-scrubbed forms. Previously, descriptors were (supposed to be; see #5607) stored under the digests of their scrubbed forms. The reason for hashing digests is to prevent looking up an existing descriptor from the bridge authority by its non-scrubbed descriptor digest. With this change, we don't have to repair references between statuses, server descriptors, and extra-info descriptors anymore which turned out to be error-prone (#5608). Server descriptors and extra-info descriptors contain a new "router-digest" line with the hex-formatted descriptor identifier. These lines are necessary, because we cannot calculate the identifier anymore and because we don't want to rely on the file name. - Bridge nicknames (#5684) in all descriptor types and dirreq-* statistics lines (#5807) in extra-info descriptors are not sanitized anymore. - All sanitized bridge descriptors contain @type annotations (#5651). Please let me know what you think about these changes. I plan to start sanitizing descriptors with the described changes tomorrow or the day after and make them available on May 30 (or later if the sanitizing/compressing/uploading takes much longer than expected). Thanks, Karsten

2 5