Twisted-based Tor client performance measurement tool
Hi everyone, you probably heard of Torperf [0], the tool that produces our Tor client performance graphs [1]. Torperf is mostly a bunch of scripts and lengthy HOWTOs, so setting it up and keeping it happy is not exactly trivial. The same applies to extending it, e.g., to make downloads using Selenium/Firefox rather than using its own C SOCKS client. I'm pondering a rewrite of Torperf in Twisted. The idea is to have a single Twisted application that is trivial to install and that does the following things: 1) set up local Tor clients, configure them, and register for events; 2) run a local web server to download files from or upload files to; 3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service, 3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830; 4) store request timestamps and Tor controller events to SQLite; 5) provide results via a RESTful API over its web server. That's a lot, and to make things even more fun, there's a sponsor deadline to have more realistic Torperf measurements by February 28. So, I want to start with 3.4 and minimal versions of 4 and 5. But I want to make sure that the remaining parts can be added later without redesigning everything again. My questions: - Is Twisted the right framework for this? What are the alternatives? - Is there existing code that I should look at before writing new code? - Does anybody want to help out? :) Thanks! Karsten [0] https://gitweb.torproject.org/torperf.git [1] https://metrics.torproject.org/performance.html
Hi Karsten, Over a year ago I wrote a small measuring proxy called Monitor In The Middle. This proxy sits between the browser and Tor and examines all HTTP traffic. Results of the measurements like response times, timeouts etc can be viewed using an internal webserver (at http://mitm.proxy). My code uses Twisted, but does not use the Tor control interface. Wrote an article about it at: http://freedomboxblog.nl/mitm-for-tor/ You can download the source-code at: http://freedomboxblog.nl/wp-content/uploads/2011/09/mitm-0.9.tar.gz Maybe this can be adapted to your needs? Rob. http://freedomboxblog.nl On Mon, 2013-01-21 at 21:06 +0100, Karsten Loesing wrote:
Hi everyone,
you probably heard of Torperf [0], the tool that produces our Tor client performance graphs [1]. Torperf is mostly a bunch of scripts and lengthy HOWTOs, so setting it up and keeping it happy is not exactly trivial. The same applies to extending it, e.g., to make downloads using Selenium/Firefox rather than using its own C SOCKS client.
I'm pondering a rewrite of Torperf in Twisted. The idea is to have a single Twisted application that is trivial to install and that does the following things:
1) set up local Tor clients, configure them, and register for events; 2) run a local web server to download files from or upload files to; 3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service, 3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830; 4) store request timestamps and Tor controller events to SQLite; 5) provide results via a RESTful API over its web server.
That's a lot, and to make things even more fun, there's a sponsor deadline to have more realistic Torperf measurements by February 28. So, I want to start with 3.4 and minimal versions of 4 and 5. But I want to make sure that the remaining parts can be added later without redesigning everything again.
My questions: - Is Twisted the right framework for this? What are the alternatives? - Is there existing code that I should look at before writing new code? - Does anybody want to help out? :)
Thanks! Karsten
[0] https://gitweb.torproject.org/torperf.git
[1] https://metrics.torproject.org/performance.html _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On 1/21/13 10:28 PM, Rob van der Hoeven wrote:
Hi Karsten,
Over a year ago I wrote a small measuring proxy called Monitor In The Middle. This proxy sits between the browser and Tor and examines all HTTP traffic. Results of the measurements like response times, timeouts etc can be viewed using an internal webserver (at http://mitm.proxy).
My code uses Twisted, but does not use the Tor control interface. Wrote an article about it at:
http://freedomboxblog.nl/mitm-for-tor/
You can download the source-code at:
http://freedomboxblog.nl/wp-content/uploads/2011/09/mitm-0.9.tar.gz
Maybe this can be adapted to your needs?
Hi Rob, that's pretty cool! I was wondering if we'd need such a measuring proxy. So far, we only have some timestamps from Firefox's Navigation Timing implementation: https://github.com/willscott/pagespeed/blob/master/measurement/harness.selen... I'll have a look at your code once I have a clearer picture of the overall design of this Twisted-based Torperf re-implementation. Thanks, Karsten
Hi Karsten, txtorcon has examples that do some of these things, and Twisted includes a Web client (and Web server). I wouldn't mind helping out here and there, but I can't commit specific time for the next little while as I have a day job, plus am pretty busy for the next couple months with some home improvements. If you did want to use txtorcon, I'd be very interested to hear about (and add) any features it lacks (using the "Issues" interface on github.com/meejah/txtorcon would be the best place to request one).
1) set up local Tor clients, configure them, and register for events; 2) run a local web server to download files from or upload files to;
A super-simple version of this is examples/launch_tor_with_hidden_service.py (the web server then being on a hidden service, too). Adding timing code would be easy.
3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service,
You'd need https://github.com/ln5/twisted-socks for a SOCKS client for this (looks like V4 only?). There are some other ones floating around out there, too, but nothing in core Twisted (as far as I recall). Ah, like https://twistedmatrix.com/trac/ticket/1330
3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830;
In the "exit_scanner" branch is some more-complicated code to do something kind-of like this (and more), and there's also examples/circuit_failure_rates.py -- which both sound similar but not quite the same as what you want (but probably decent examples). The failures rates one is way simpler (passively analyizes failures rates per-guard). I'd guess all you'd need for the selenium stuff is event-based subprocess launching, which Twisted does have (txtorcon uses it to launch slave Tors).
4) store request timestamps and Tor controller events to SQLite; 5) provide results via a RESTful API over its web server.
My questions: - Is Twisted the right framework for this? What are the alternatives?
If you like event-based code, it's almost certainly the best choice. If threaded is fine (or preferred), then there are a lot more options (and of course, you'd then want Stem as the controller library) but I don't know them that well. Probably it would mean combining a Python web-client library with some suitable web-server thing (dmaybe the built-in SimpleHTTPServer? Lukas Lueg contributed an example for doing this with txtorcon). Depending on how much work you're doing, beware that Python doesn't play very nicely with threads. Also, since you're talking about a Web service, it might be worth looking at Cyclone for that piece (which is Tornado, the friendfeed thing, but with their custom async-stuff taken out and Twisted's put in). Twisted's Web server will work well for simple things (e.g. the speed tests), but if you want a "modern" AJAX-y sort of thing, Cyclone is likely a better choice. -- meejah
meejah <meejah@meejah.ca> wrote Tue, 22 Jan 2013 11:32:39 +0400: | You'd need https://github.com/ln5/twisted-socks for a SOCKS client for | this (looks like V4 only?). There are some other ones floating around | out there, too, but nothing in core Twisted (as far as I | recall). Ah, like https://twistedmatrix.com/trac/ticket/1330 twisted-socks needs more love. I should try to understand and then merge [a0cb2383] to begin with (sorry Arturo) if you decide to use it. [a0cb2383] https://github.com/hellais/twisted-socks/commit/a0cb2383ed4409e0dd840455ec89...
On 1/22/13 9:32 AM, Linus Nordberg wrote:
meejah <meejah@meejah.ca> wrote Tue, 22 Jan 2013 11:32:39 +0400:
| You'd need https://github.com/ln5/twisted-socks for a SOCKS client for | this (looks like V4 only?). There are some other ones floating around | out there, too, but nothing in core Twisted (as far as I | recall). Ah, like https://twistedmatrix.com/trac/ticket/1330
twisted-socks needs more love. I should try to understand and then merge [a0cb2383] to begin with (sorry Arturo) if you decide to use it.
[a0cb2383] https://github.com/hellais/twisted-socks/commit/a0cb2383ed4409e0dd840455ec89...
I can now confirm that it works if you apply this trivial patch: https://github.com/kloesing/twisted-socks/commit/1d73b6f6bbc490b88ebe9fa0213... Best, Karsten
Karsten Loesing <karsten@torproject.org> wrote Tue, 22 Jan 2013 21:56:47 +0100: | > | You'd need https://github.com/ln5/twisted-socks for a SOCKS client for | > | this (looks like V4 only?). There are some other ones floating around | > | out there, too, but nothing in core Twisted (as far as I | > | recall). Ah, like https://twistedmatrix.com/trac/ticket/1330> | > twisted-socks needs more love. I should try to understand and then merge | > [a0cb2383] to begin with (sorry Arturo) if you decide to use it. | > | > [a0cb2383] https://github.com/hellais/twisted-socks/commit/a0cb2383ed4409e0dd840455ec89... | | I can now confirm that it works if you apply this trivial patch: | | https://github.com/kloesing/twisted-socks/commit/1d73b6f6bbc490b88ebe9fa0213... Thanks! I just realised that I had already merged Arturos fix, plus your fix, back in September. It was cleverly disguised as a1c4f02c * Add license info to all source files. though, so it wasn't easy to spot. *blushes* Karsten, can you confirm that 17723a8c in my repo works for you?
On 1/22/13 10:35 PM, Linus Nordberg wrote:
Karsten Loesing <karsten@torproject.org> wrote Tue, 22 Jan 2013 21:56:47 +0100:
| > | You'd need https://github.com/ln5/twisted-socks for a SOCKS client for | > | this (looks like V4 only?). There are some other ones floating around | > | out there, too, but nothing in core Twisted (as far as I | > | recall). Ah, like https://twistedmatrix.com/trac/ticket/1330> | > twisted-socks needs more love. I should try to understand and then merge | > [a0cb2383] to begin with (sorry Arturo) if you decide to use it. | > | > [a0cb2383] https://github.com/hellais/twisted-socks/commit/a0cb2383ed4409e0dd840455ec89... | | I can now confirm that it works if you apply this trivial patch: | | https://github.com/kloesing/twisted-socks/commit/1d73b6f6bbc490b88ebe9fa0213...
Thanks!
I just realised that I had already merged Arturos fix, plus your fix, back in September. It was cleverly disguised as
a1c4f02c * Add license info to all source files.
though, so it wasn't easy to spot. *blushes*
Karsten, can you confirm that 17723a8c in my repo works for you?
Works like a charm! Best, Karsten
On Jan 22, 2013, at 8:32 AM, meejah <meejah@meejah.ca> wrote:
3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service,
You'd need https://github.com/ln5/twisted-socks for a SOCKS client for this (looks like V4 only?). There are some other ones floating around out there, too, but nothing in core Twisted (as far as I recall). Ah, like https://twistedmatrix.com/trac/ticket/1330
What I am currently using as a SOCKS Client these days is this one: https://github.com/hellais/txsocksx. It's got decent unittest coverage and some examples on how to use it. It supports v4 and v5. We've been using it quite heavily in ooniprobe and it appears to be doing it's job well. ~ A.
Arturo Filastò <art@torproject.org> writes:
What I am currently using as a SOCKS Client these days is this one: https://github.com/hellais/txsocksx.
Ah, that's one of the "implementations out there" I was thinking of. Don't know why I didn't find it again, sorry. +1 for tests :) -- meejah
On 1/22/13 8:32 AM, meejah wrote:
Hi Karsten,
Hi meejah,
txtorcon has examples that do some of these things, and Twisted includes a Web client (and Web server). I wouldn't mind helping out here and there, but I can't commit specific time for the next little while as I have a day job, plus am pretty busy for the next couple months with some home improvements.
If you did want to use txtorcon, I'd be very interested to hear about (and add) any features it lacks (using the "Issues" interface on github.com/meejah/txtorcon would be the best place to request one).
Sounds great! I avoided talking to Tor's control port, but I'll sure have a look at txtorcon when doing that.
1) set up local Tor clients, configure them, and register for events; 2) run a local web server to download files from or upload files to;
A super-simple version of this is examples/launch_tor_with_hidden_service.py (the web server then being on a hidden service, too). Adding timing code would be easy.
Neat, somebody already did this. :) Hidden services are pretty low on my priority list, but I'll give that a look once it's at the top.
3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service,
You'd need https://github.com/ln5/twisted-socks for a SOCKS client for this (looks like V4 only?). There are some other ones floating around out there, too, but nothing in core Twisted (as far as I recall). Ah, like https://twistedmatrix.com/trac/ticket/1330
I used Linus' twisted-socks, and after a tiny tweak it works quite nicely. The next step would be add more timestamps to it, but I first wanted to take a closer look at the code that Arturo suggested later on.
3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830;
In the "exit_scanner" branch is some more-complicated code to do something kind-of like this (and more), and there's also examples/circuit_failure_rates.py -- which both sound similar but not quite the same as what you want (but probably decent examples). The failures rates one is way simpler (passively analyizes failures rates per-guard). I'd guess all you'd need for the selenium stuff is event-based subprocess launching, which Twisted does have (txtorcon uses it to launch slave Tors).
Thanks for the pointers!
4) store request timestamps and Tor controller events to SQLite; 5) provide results via a RESTful API over its web server.
My questions: - Is Twisted the right framework for this? What are the alternatives?
If you like event-based code, it's almost certainly the best choice. If threaded is fine (or preferred), then there are a lot more options (and of course, you'd then want Stem as the controller library) but I don't know them that well. Probably it would mean combining a Python web-client library with some suitable web-server thing (dmaybe the built-in SimpleHTTPServer? Lukas Lueg contributed an example for doing this with txtorcon). Depending on how much work you're doing, beware that Python doesn't play very nicely with threads.
I don't mind much about event-based vs. threaded. Concurrency is always hard to get right. But it seems Twisted comes with a lot of infrastructure out of the box, which may lead to fewer lines of code.
Also, since you're talking about a Web service, it might be worth looking at Cyclone for that piece (which is Tornado, the friendfeed thing, but with their custom async-stuff taken out and Twisted's put in). Twisted's Web server will work well for simple things (e.g. the speed tests), but if you want a "modern" AJAX-y sort of thing, Cyclone is likely a better choice.
Cyclone might be overkill, and last I checked it wasn't in Debian's stable repositories. If the fancy RESTful API for results is the only reason for using Cyclone, I think I'd rather write a simpler results interface. Thanks for your suggestions! Best, Karsten
Karsten Loesing <karsten@torproject.org> wrote Mon, 21 Jan 2013 21:06:38 +0100: | That's a lot, and to make things even more fun, there's a sponsor | deadline to have more realistic Torperf measurements by February 28. This is what stops me from going "Wooha! o/". I'd love to help out with this later -- not only is this interesting but I'm also a user of the thing. Thanks for getting this going, Karsten.
On 1/22/13 9:35 AM, Linus Nordberg wrote:
Karsten Loesing <karsten@torproject.org> wrote Mon, 21 Jan 2013 21:06:38 +0100:
| That's a lot, and to make things even more fun, there's a sponsor | deadline to have more realistic Torperf measurements by February 28.
This is what stops me from going "Wooha! o/".
Which part? ;)
I'd love to help out with this later -- not only is this interesting but I'm also a user of the thing.
Thanks for getting this going, Karsten.
Sure thing. There's a proof-of-concept prototype in the perfd branch of my public torperf repository: git clone https://git.torproject.org/karsten/torperf.git cd torperf/ git checkout -b perfd origin/perfd This prototype sets up a web server and runs periodic web requests using your SOCKS client code. See README.perfd for details. Feedback much appreciated! Best, Karsten
Karsten Loesing <karsten@torproject.org> wrote Tue, 22 Jan 2013 22:08:39 +0100: | > | That's a lot, and to make things even more fun, there's a sponsor | > | deadline to have more realistic Torperf measurements by February 28. | > | > This is what stops me from going "Wooha! o/". | | Which part? ;) The deadline part. | Feedback much appreciated! Local testing on Debian 7.0 works fine. Unable to test with a globally reachable address on that machine atm. All testing on FreeBSD 8.3, Python 2.7.3, Twisted 12.1.0 fail with an exception that doesn't make much sense to me: --8<---------------cut here---------------start------------->8--- 2013-01-23 09:41:58+0100 [-] Unhandled error in Deferred: 2013-01-23 09:41:58+0100 [-] Unhandled Error Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/twisted/internet/task.py", line 215, in __call__ d = defer.maybeDeferred(self.f, *self.a, **self.kw) File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 134, in maybeDeferred result = f(*args, **kw) File "perfd.py", line 150, in makeRequest self.file_size) File "perfd.py", line 104, in __init__ deferred = wrapper.connect(factory) --- <exception caught here> --- File "/home/linus/usr/src/torperf/socksclient.py", line 108, in connect wf = _WrappingFactory(f, _canceller) exceptions.TypeError: __init__() takes exactly 2 arguments (3 given) --8<---------------cut here---------------end--------------->8---
Linus Nordberg <linus@torproject.org> wrote Wed, 23 Jan 2013 09:47:37 +0100: | All testing on FreeBSD 8.3, Python 2.7.3, Twisted 12.1.0 fail with an | exception that doesn't make much sense to me: That's now been fixed in twisted-socks (thanks meejah) and perfd works as expected both directly and over Tor. Karsten, let me know if you want me to test this more heavily somehow.
On 1/23/13 10:19 PM, Linus Nordberg wrote:
Linus Nordberg <linus@torproject.org> wrote Wed, 23 Jan 2013 09:47:37 +0100:
| All testing on FreeBSD 8.3, Python 2.7.3, Twisted 12.1.0 fail with an | exception that doesn't make much sense to me:
That's now been fixed in twisted-socks (thanks meejah) and perfd works as expected both directly and over Tor.
Cool!
Karsten, let me know if you want me to test this more heavily somehow.
Thanks, but I think it's too early for more testing. What this prototype needs is more code to reach feature parity with current Torperf. That means implementing the Tor controller connection using txtorcon and adding more timestamps to twisted-socks (or a local copy of it). Once we have that we can test it and maybe throw out most of the existing Torperf code. Will let you know once I make more progress there. Thanks! Karsten
On 1/21/13 9:06 PM, Karsten Loesing wrote:
Hi everyone,
you probably heard of Torperf [0], the tool that produces our Tor client performance graphs [1]. Torperf is mostly a bunch of scripts and lengthy HOWTOs, so setting it up and keeping it happy is not exactly trivial. I am wondering if this should not use and leverage the existing OONI project that does "more or less" what functionally TorPerf has to do?
Fabio
On 1/24/13 10:31 AM, Fabio Pietrosanti (naif) wrote:
On 1/21/13 9:06 PM, Karsten Loesing wrote:
Hi everyone,
you probably heard of Torperf [0], the tool that produces our Tor client performance graphs [1]. Torperf is mostly a bunch of scripts and lengthy HOWTOs, so setting it up and keeping it happy is not exactly trivial. I am wondering if this should not use and leverage the existing OONI project that does "more or less" what functionally TorPerf has to do?
Which of Torperf's planned functionality overlaps with OONI?
1) set up local Tor clients, configure them, and register for events; 2) run a local web server to download files from or upload files to; 3) periodically run one or more tests which can be: 3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service, 3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830; 4) store request timestamps and Tor controller events to SQLite; 5) provide results via a RESTful API over its web server.
For the moment, I think we're better off keeping the two projects separate, mostly because of Torperf's February 28 deadline. But I wouldn't mind integrating Torperf's functionality into OONI afterwards. Arturo, maybe you can comment on this? For everyone else following this thread, I updated the README to list next steps and incorporate all feedback I got so far: https://gitweb.torproject.org/karsten/torperf.git/blob/perfd:/README.perfd Best, Karsten
participants (7)
-
Arturo Filastò -
Fabio Pietrosanti (naif) -
Karsten Loesing -
Linus Nordberg -
Linus Nordberg -
meejah -
Rob van der Hoeven