commit f529758e08f66332e9f48d1f97baa240d5906f62 Author: Nick Mathewson <nickm@torproject.org> Date: Thu Feb 7 11:44:40 2013 -0500 Add proposal 218: Controller events to better understand connection/circuit usage --- proposals/000-index.txt | 2 + proposals/218-usage-controller-events.txt | 156 +++++++++++++++++++++++++++++ 2 files changed, 158 insertions(+), 0 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 7677925..3a04148 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -138,6 +138,7 @@ Proposals by number: 215 Let the minimum consensus method change with time [OPEN] 216 Improved circuit-creation key exchange [FINISHED] 217 Tor Extended ORPort Authentication [OPEN] +218 Controller events to better understand connection/circuit usage [OPEN] Proposals by status: @@ -185,6 +186,7 @@ Proposals by status: 214 Allow 4-byte circuit IDs in a new link protocol 215 Let the minimum consensus method change with time 217 Tor Extended ORPort Authentication [for 0.2.5.x] + 218 Controller events to better understand connection/circuit usage [for 0.2.5.x] ACCEPTED: 140 Provide diffs between consensuses 147 Eliminate the need for v2 directories in generating v3 directories [for 0.2.4.x] diff --git a/proposals/218-usage-controller-events.txt b/proposals/218-usage-controller-events.txt new file mode 100644 index 0000000..0268df5 --- /dev/null +++ b/proposals/218-usage-controller-events.txt @@ -0,0 +1,156 @@ +Filename: 218-usage-controller-events.txt +Title: Controller events to better understand connection/circuit usage +Author: Rob Jansen, Karsten Loesing +Created: 2013-02-06 +Status: Open +Target: 0.2.5.x + +1. Overview + + This proposal defines three new controller events that shall help + understand connection and circuit usage. These events are designed + to be emitted in private Tor networks only. This proposal also + defines a tweak to an existing event for the same purpose. + +2. Motivation + + We need to better understand connection and circuit usage in order to + better simulate Tor networks. Existing controller events are a fine + start, but we need more detailed information about per-connection + bandwidth, processed cells by circuit, and token bucket refills. This + proposal defines controller events containing the desired information. + + Most of these usage data are too sensitive to be captured in the + public network, unless aggregated sufficiently. That is why we're + focusing on private Tor networks first, that is, relays that have + TestingTorNetwork set. The new controller events described in this + proposal shall all be restricted to private Tor networks. In the next + step we might define aggregate statistics to be gathered by public + relays, but that will require a new proposal. + +3. Design + + The proposed new event types use Tor's asynchronous event mechanism + where a controller registers for events by type and processes events + received from the Tor process. + + Tor controllers can register for any of the new event types, but + events will only be emitted if the Tor process is running in + TestingTorNetwork mode. + +4. Security implications + + There should be no security implications from the new event types, + because they are only emitted in private Tor networks. + +5. Specification + +5.1. Adding an ID field to ORCONN events + + The new syntax for ORCONN events is: + + "650" SP "ORCONN" SP (LongName / Target) SP ORStatus + [ SP "ID=" ConnId ] [ SP "REASON=" Reason ] + [ SP "NCIRCS=" NumCircuits ] CRLF + + ConnID is the connection ID which is locally unique among all + connection types and which is only included in TestingTorNetwork + mode. + + The remaining specification of that event type stays unchanged. + +5.2. Bandwidth used on an OR or DIR or EXIT connection + + The syntax is: + "650" SP "CONN_BW" SP ConnID SP ConnType SP BytesRead SP + BytesWritten CRLF + BytesRead = 1*DIGIT + BytesWritten = 1*DIGIT + + ConnID is the connection ID which is locally unique among all + connection types. + + ConnType is the connection type, which can be "OR" or "DIR" or "EXIT". + + BytesWritten and BytesRead are the number of bytes written and read + by Tor since the last CONN_BW event on this connection. + + These events are generated about once per second per connection; no + events are generated for connections that have not read or written. + These events are only generated if TestingTorNetwork is set. + +5.3. Per-circuit cell stats + + The syntax is: + "650" SP "CELL_STATS" SP + PCircID SP PConnID SP PAdded SP PRemoved SP PTime SP + NCircID SP NConnID SP NAdded SP NRemoved SP NTime CRLF + + PCircID and NCircID are the locally unique IDs of the app-ward + (PCircID) and exit-ward (NCircID) circuit. + + PConnID and NConnID are the locally unique IDs of the app-ward + (PConnID) and exit-ward (NConnID) OR connection. + + PAdded and NAdded are the total number of cells added to the app-ward + (PAdded) and exit-ward (NAdded) queues of this circuit. + + PRemoved and NRemoved are the total number of cells processed from the + app-ward (PRemoved) and exit-ward (NRemoved) queues of this circuit. + + PTime and NTime are the total waiting times in milliseconds of all + processed cells in the app-ward (PTime) and exit-ward (NTime) queues + of this circuit. + + PAdded, NAdded, PRemoved, NRemoved, PTime, and NTime are + semicolon-separated key-value lists with keys being lower-case cell + types and values being cell numbers or waiting times. + + These events are generated about once per second per circuit; no + events are generated for circuits that have not added or processed any + cell. These events are only generated if TestingTorNetwork is set. + +5.4. Token buckets refilled + + The syntax is: + "650" SP "TB_EMPTY" SP ["GLOBAL" || "RELAY" || "ORCONN" SP ConnID] + SP ReadBucketEmpty SP WriteBucketEmpty SP LastRefill CRLF + + This event is generated when refilling a previously empty token + bucket. The "GLOBAL" and "RELAY" keywords are used for the global or + relay token buckets, the "ORCONN" keyword together with a ConnID is + used for the token buckets of an OR connection. + + If both global and relay buckets and/or the buckets of one or more OR + connections run out of tokens at the same time, multiple separate + events are generated. + + ReadBucketEmpty (WriteBucketEmpty) is the time in millis that the read + (write) bucket was empty. LastRefill is the time in millis since the + last refill. ReadBucketEmpty or WriteBucketEmpty are capped at + LastRefill in order not to report empty times more than once. + + These events are only generated if TestingTorNetwork is set. + +6. Compatibility + + There should not be any compatibility issues with other Tor versions. + +7. Implementation + + Most of the implementation should be straight-forward. + + There's one exception: we pondered adding a unique circuit ID to + CELL_STATS events, but so far, only origin circuits have a unique ID. + We could move that field from origin_circuit_t to circuit_t and + update all references in the code. But this may have undesired side + effects which we're not yet aware of. We don't have a good answer + yet if we need this ID or not. + +8. Performance and scalability notes + + Most of the new code won't be executed in normal Tor mode. Wherever + we needed new fields in existing structs, we tried hard to keep them + as small as possible. Still, we should make sure that memory + requirements won't grow significantly on busy relays. +