- tor-dev - lists.torproject.org

Re: [tor-dev] Proposal: Controller events to better understand connection/circuit usage
by Damian Johnson 20 Feb '13

20 Feb '13

> AppWard and ExitWard indicate the direction of cell flow through the > circuit. Previous cell events aggregated statistics for both directions and > therefore did not capture information which I believe may be important. > These are much more informative than N and P. Do you have suggestions for > improving this further? Ahhh, gotcha. Personally I'd go for 'inbound' and 'outbound' since they better connotate directionality. AppWard and ExitWard sound weird, but they make sense now too and would be fine if CELL_STATS got a quick description saying what they mean.

2 6

[Patch] or/onion_ntor.c
by Gisle Vanem 19 Feb '13

19 Feb '13

I suggest that the whole body of this file gets included inside an "#ifdef CURVE25519_ENABLED" in-case someone (like me!) inadvertently tries to compile it w/o this define set. Patch: --- Git-latest\src\or\onion_ntor.c Tue Feb 12 18:40:04 2013 +++ or\onion_ntor.c Tue Feb 12 23:46:51 2013 @@ -9,6 +9,8 @@ #include "torlog.h" #include "util.h" +#ifdef CURVE25519_ENABLED /* Rest of file */ + /** Free storage held in an ntor handshake state. */ void ntor_handshake_state_free(ntor_handshake_state_t *state) @@ -292,4 +294,4 @@ memwipe(&s, 0, sizeof(s)); return bad ? -1 : 0; } - +#endif /* CURVE25519_ENABLED */ --gv

2 1

changes to vidalia / browser launch in TBB?
by Greg Norcie 14 Feb '13

14 Feb '13

Hi all, I'm working on a followup to my usability paper from HotPETS... (http://petsymposium.org/2012/papers/hotpets12-1-usability.pdf) and doing some fact checking for the write up. I'm trying to find out if the developers tweaked Vidalia since v2.2.35-7.1. In my tests it seems like there is much less of a lag between Vidalia launching and the TBB window opening. I was hoping to find out if this was an intentional code change on the developers part, since it elliminates one of the isssues mentioned in my paper. -- -- Greg Norcie (greg(a)norcie.com) GPG key: 0x1B873635

1 0

Proposal: Controller events to better understand connection/circuit usage
by Karsten Loesing 12 Feb '13

12 Feb '13

Hi everyone, here's a proposal that defines three new controller events for the TestingTorNetwork mode that shall help us better understand connection and circuit usage in private Tor networks with the goal to make simulations more accurate. And there's also code: https://gitweb.torproject.org/karsten/tor.git/shortlog/refs/heads/morestats2 Feedback, to both proposal and code, much appreciated! Thanks, Karsten Filename: xxx-usage-controller-events.txt Title: Controller events to better understand connection/circuit usage Author: Rob Jansen, Karsten Loesing Created: 2013-02-06 Status: Open Target: 0.2.5.x 1. Overview This proposal defines three new controller events that shall help understand connection and circuit usage. These events are designed to be emitted in private Tor networks only. This proposal also defines a tweak to an existing event for the same purpose. 2. Motivation We need to better understand connection and circuit usage in order to better simulate Tor networks. Existing controller events are a fine start, but we need more detailed information about per-connection bandwidth, processed cells by circuit, and token bucket refills. This proposal defines controller events containing the desired information. Most of these usage data are too sensitive to be captured in the public network, unless aggregated sufficiently. That is why we're focusing on private Tor networks first, that is, relays that have TestingTorNetwork set. The new controller events described in this proposal shall all be restricted to private Tor networks. In the next step we might define aggregate statistics to be gathered by public relays, but that will require a new proposal. 3. Design The proposed new event types use Tor's asynchronous event mechanism where a controller registers for events by type and processes events received from the Tor process. Tor controllers can register for any of the new event types, but events will only be emitted if the Tor process is running in TestingTorNetwork mode. 4. Security implications There should be no security implications from the new event types, because they are only emitted in private Tor networks. 5. Specification 5.1. Adding an ID field to ORCONN events The new syntax for ORCONN events is: "650" SP "ORCONN" SP (LongName / Target) SP ORStatus [ SP "ID=" ConnId ] [ SP "REASON=" Reason ] [ SP "NCIRCS=" NumCircuits ] CRLF ConnID is the connection ID which is locally unique among all connection types and which is only included in TestingTorNetwork mode. The remaining specification of that event type stays unchanged. 5.2. Bandwidth used on an OR or DIR or EXIT connection The syntax is: "650" SP "CONN_BW" SP ConnID SP ConnType SP BytesRead SP BytesWritten CRLF BytesRead = 1*DIGIT BytesWritten = 1*DIGIT ConnID is the connection ID which is locally unique among all connection types. ConnType is the connection type, which can be "OR" or "DIR" or "EXIT". BytesWritten and BytesRead are the number of bytes written and read by Tor since the last CONN_BW event on this connection. These events are generated about once per second per connection; no events are generated for connections that have not read or written. These events are only generated if TestingTorNetwork is set. 5.3. Per-circuit cell stats The syntax is: "650" SP "CELL_STATS" SP PCircID SP PConnID SP PAdded SP PRemoved SP PTime SP NCircID SP NConnID SP NAdded SP NRemoved SP NTime CRLF PCircID and NCircID are the locally unique IDs of the app-ward (PCircID) and exit-ward (NCircID) circuit. PConnID and NConnID are the locally unique IDs of the app-ward (PConnID) and exit-ward (NConnID) OR connection. PAdded and NAdded are the total number of cells added to the app-ward (PAdded) and exit-ward (NAdded) queues of this circuit. PRemoved and NRemoved are the total number of cells processed from the app-ward (PRemoved) and exit-ward (NRemoved) queues of this circuit. PTime and NTime are the total waiting times in milliseconds of all processed cells in the app-ward (PTime) and exit-ward (NTime) queues of this circuit. PAdded, NAdded, PRemoved, NRemoved, PTime, and NTime are semicolon-separated key-value lists with keys being lower-case cell types and values being cell numbers or waiting times. These events are generated about once per second per circuit; no events are generated for circuits that have not added or processed any cell. These events are only generated if TestingTorNetwork is set. 5.4. Token buckets refilled The syntax is: "650" SP "TB_EMPTY" SP ["GLOBAL" || "RELAY" || "ORCONN" SP ConnID] SP ReadBucketEmpty SP WriteBucketEmpty SP LastRefill CRLF This event is generated when refilling a previously empty token bucket. The "GLOBAL" and "RELAY" keywords are used for the global or relay token buckets, the "ORCONN" keyword together with a ConnID is used for the token buckets of an OR connection. If both global and relay buckets and/or the buckets of one or more OR connections run out of tokens at the same time, multiple separate events are generated. ReadBucketEmpty (WriteBucketEmpty) is the time in millis that the read (write) bucket was empty. LastRefill is the time in millis since the last refill. ReadBucketEmpty or WriteBucketEmpty are capped at LastRefill in order not to report empty times more than once. These events are only generated if TestingTorNetwork is set. 6. Compatibility There should not be any compatibility issues with other Tor versions. 7. Implementation Most of the implementation should be straight-forward. There's one exception: we pondered adding a unique circuit ID to CELL_STATS events, but so far, only origin circuits have a unique ID. We could move that field from origin_circuit_t to circuit_t and update all references in the code. But this may have undesired side effects which we're not yet aware of. We don't have a good answer yet if we need this ID or not. 8. Performance and scalability notes Most of the new code won't be executed in normal Tor mode. Wherever we needed new fields in existing structs, we tried hard to keep them as small as possible. Still, we should make sure that memory requirements won't grow significantly on busy relays.

5 14

[Stegotorus] Dropping proxy test result
by vmonmoonshine＠gmail.com 10 Feb '13

10 Feb '13

Hi Zack, I added --drop-rate option to the tester-proxy and committed it to f--bad-proxy-ack-tester branch. I confirm that Stegotorus fails webpage-tester test when the drop-rate >= %10. Now, I'll go and look at the problem in more depth. Generally speaking, firefox is less successful in fetching web pages than webpage_tester, i.e. curl. I don't know why but I hope that problem will go away, when the problem with ack is solved. And on this: > Speaking of, vmon, I know you had trouble getting libcurl and libevent > to play nice -- did you ever look at libevhtp? First glance, I don't think it's helpful. It's an http server, isn't it? I need a http client which send the GET Request but let libevent to read the replay (only write in the socket). Hopefully I'll have more useful information soon. Cheers, Vmon

1 0

Documenting the translation process
by Runa A. Sandvik 07 Feb '13

07 Feb '13

Hi everyone, I started documenting how the translation process works for translators, developers, and those who manage it all: https://trac.torproject.org/projects/tor/wiki/doc/translation/Introduction. Any feedback will be greatly appreciated. Thanks, -- Runa A. Sandvik

1 0

Re: [tor-dev] [tor-talk] Exit Node Scanning Status
by Damian Johnson 03 Feb '13

03 Feb '13

> Actually, I might be a good fit. Feel free to ping me off-list to discuss. Unless there's a reason that you'd like to make this private it's better to have development discussions in the open (though tor-dev@ is probably more appropriate). Imho writing a SoaT counterpart based on either stem [1] or txtorcon [2] would be a great project if you're interested in improving our bad-exit detection infrastructure. Cheers! -Damian [1] https://stem.torproject.org/ [2] https://txtorcon.readthedocs.org/

4 5

Damian's Status Report - January 2013
by Damian Johnson 03 Feb '13

03 Feb '13

Hi all. This January I've been integrating feedback from first time stem users and implementing their feature requests. Projects included... * Python 3.x Support Stem now works under both the python 2.x and 3.x series. To use stem with python 3 simply use it when you install... python3 setup.py install This turned out to be a larger project than I had anticipated, taking almost half the month. But it was well worth the effort. https://gitweb.torproject.org/stem.git/commit/3930f1f https://trac.torproject.org/7843 * PEP8 Compliance Ditched most of my odd coding preferences in favor of the standard python style guide. I've integrated both pep8 and pyflakes with our tests to prevent regression. Hopefully this'll make it easier for others to contribute to stem. https://gitweb.torproject.org/stem.git/commit/d29ec4d https://trac.torproject.org/7631 * Arm Codebase Refactoring Prior to being shanghaied into the projects above I sunk quite a bit of time into overhauling arm. Thus far I've dropped around a third of the codebase in favor of similar (but tested!) capabilities in stem. * Descriptor Improvements Got lots of input concerning stem's descriptor module (thanks Karsten!), leading to quite a few improvements... * used feedback from Aaron Johnson to make the descriptor API less confusing * we now support bridge network status documents (https://trac.torproject.org/7938) * added support for '-legacy' authorities (https://trac.torproject.org/7866) * parsing error if network status documents lacked a 'directory-footer' line (https://trac.torproject.org/7932) * empty 'bridge-ip-versions' lines caused problems (https://trac.torproject.org/7967) * we didn't recognize the @type annotation for key certificates (https://trac.torproject.org/7987) * we weren't parsing the new ntor-onion-key lines (caught by sonu, https://trac.torproject.org/7868) * error when ran with pypy (caught by peer) Sean helped quite a bit at the start of the month, but has since been busy with other things. * added a Controller.get_streams() method (https://trac.torproject.org/7859) * tests for the close_stream() method (https://trac.torproject.org/7687) * expanded the Controller's unit tests (https://trac.torproject.org/7874) * fixed type checks (https://trac.torproject.org/7873) * fixed test_reattaching_listeners (https://trac.torproject.org/7872) Cheers! -Damian

1 0

extension works in FF, fails in TBB
by Greg Norcie 01 Feb '13

01 Feb '13

Dear Tor Devs, Hello. My name is Greg Norcie, I'm a PhD student at Indiana University, working on a project focusing on improving Tor usability. This is a follow up to a paper looking at usability issues with the Tor Browser Bundle. (http://petsymposium.org/2012/papers/hotpets12-1-usability.pdf) The research group I'm leading is currently working on tweaking the Tor Browser Bundle to test solutions to the issues we noted in a formal lab study. (Our previous paper simply suggested changes, but did not scientifically validate our suggestions) One issue we noted was people feeling Tor was slow. Now, this is a function of how Tor works - traffic passed through a series of nodes will always have more latency than a direct connection. So we are experimenting with a custom extension that pops up a message to TBB users when a delay is detected, to see if reframing the delay makes users tolerate the latency better. However, we have hit a snag. While in a normal, run of the mill Firefox install w/ the same extensions as the TBB, our extension functions - delays are detected and a pop up box appears. However, when we install our custom extension in the Tor Browser Bundle, it fails to work. We initially thought this might be because our extension was utilizing Javascript, but disabling NoScript doesn't fix thes issue. We're at a bit of a loss as to how to solve this issue, and are reaching out to the Tor developer community to see if anyone might be able to offer some insight as to what could be causing this failure. Our lead developer recently left the project, so we're struggling to get things on track in time for PETS) The extension in question is available at the following URL (password is "cryptoparty" - our uni's MegaUpload clone requires a password for all files): https://www.slashtmp.iu.edu/files/download?FILE=gnorcie%2F1700EHKJOv Any help anyone can provide would be greatly appreciated. (And of course, it goes without saying we will be making our code available to the Tor community) Thanks for reading, have a great day. Sincerely, Greg Norcie PhD Student - Security Informatics Indiana University

1 0

torsocks release pending: towards a release candidate
by Oscar Koeroo 01 Feb '13

01 Feb '13

Hi, I've created trac ticket[1] #8063. The torsocks script has an option "on" which did't work and in the current state I'm curious on which shell it did work :-) The patch is included and only affects the src/torsocks.in file with a oneliner. More details are included on the ticket. Oscar [1]: https://trac.torproject.org/projects/tor/ticket/8063

6 8