Hi Karsten,
txtorcon has examples that do some of these things, and Twisted includes a Web client (and Web server). I wouldn't mind helping out here and there, but I can't commit specific time for the next little while as I have a day job, plus am pretty busy for the next couple months with some home improvements.
If you did want to use txtorcon, I'd be very interested to hear about (and add) any features it lacks (using the "Issues" interface on github.com/meejah/txtorcon would be the best place to request one).
- set up local Tor clients, configure them, and register for events;
- run a local web server to download files from or upload files to;
A super-simple version of this is examples/launch_tor_with_hidden_service.py (the web server then being on a hidden service, too). Adding timing code would be easy.
- periodically run one or more tests which can be:
3.1) an HTTP GET request over Tor to its own web server, 3.2) an HTTP POST request to measure upload speed, 3.3) a GET or POST request to a locally running hidden service,
You'd need https://github.com/ln5/twisted-socks for a SOCKS client for this (looks like V4 only?). There are some other ones floating around out there, too, but nothing in core Twisted (as far as I recall). Ah, like https://twistedmatrix.com/trac/ticket/1330
3.4) a series of fetches of top 50 Alexa domains using Selenium/Firefox; 3.5) a series of requests to track stream/circ allocations for #5830;
In the "exit_scanner" branch is some more-complicated code to do something kind-of like this (and more), and there's also examples/circuit_failure_rates.py -- which both sound similar but not quite the same as what you want (but probably decent examples). The failures rates one is way simpler (passively analyizes failures rates per-guard). I'd guess all you'd need for the selenium stuff is event-based subprocess launching, which Twisted does have (txtorcon uses it to launch slave Tors).
- store request timestamps and Tor controller events to SQLite;
- provide results via a RESTful API over its web server.
My questions:
- Is Twisted the right framework for this? What are the alternatives?
If you like event-based code, it's almost certainly the best choice. If threaded is fine (or preferred), then there are a lot more options (and of course, you'd then want Stem as the controller library) but I don't know them that well. Probably it would mean combining a Python web-client library with some suitable web-server thing (dmaybe the built-in SimpleHTTPServer? Lukas Lueg contributed an example for doing this with txtorcon). Depending on how much work you're doing, beware that Python doesn't play very nicely with threads.
Also, since you're talking about a Web service, it might be worth looking at Cyclone for that piece (which is Tornado, the friendfeed thing, but with their custom async-stuff taken out and Twisted's put in). Twisted's Web server will work well for simple things (e.g. the speed tests), but if you want a "modern" AJAX-y sort of thing, Cyclone is likely a better choice.