[tor-dev] Proposal: Controller events to better understand connection/circuit usage

[Karsten Loesing]

On 2/23/13 11:20 PM, Rob Jansen wrote:
> On Feb 23, 2013 4:22 PM, "Karsten Loesing" <karsten at torproject.org> wrote:
>> Your understanding of n_circ_id and p_circ_id matches mine, but are you
>> sure there's a UID for circuits other than origin circuits?  I think you
>> mean origin_circuit_t->global_identifier.  But there's no such field for
>> or_circuit_t or circuit_t.  Or do you mean something else?
>>
> Ok. Though I thought my original patch moved the global Id to the base
> circuit struct. Perhaps I didn't. Anyway, I'm not sure it matters...

Your original patch did move the ID to circuit_t, but I thought we
wanted to avoid numbering non-origin circuits (mostly because that
affects relays in non-TestingTorNetwork mode and could lead to busy
relays running out of IDs at some point), which is why I took this
change out.

Also, even if we moved the field to circuit_t, the new CELL_STATS events
would be the only ones using these UIDs, because all other events are
for origin circuits only.  I don't see how these IDs would help us.  We
never learn what UID the next or previous node in a circuit picks for a
given circuit.

>> tl;dr: I _think_ it's possible to reconstruct circuits from ORCONN and
>> CELL_STATS events as they are currently specified in proposal 218.
>>
> Great, but do we really expect every Tor controller parser to get this
> right? It seems complicated enough that there should be an easier way.
> Maybe it's just wishful thinking on my part.

I agree that reconstructing circuits from ORCONN and CELL_STATS events
is far from trivial.  I don't really see how to make it simpler though.

>From an earlier mail in this thread:
>> Finally, Rob, should I look into CIRC_BW events you suggested a while
>> ago?  If yes, what did you have in mind how that event would look like,
>> and when/by whom would it be emitted?
>
> If we want to do this, it would likely be an aggregation of all STREAM_BW
> events for a circuit, but only during the time when those streams belonged
> to that circuit. I don't think it makes sense to emit it for every
> STREAM_BW event though, so what if we aggregate and emit it once per
> second? A format similar to the STREAM_BW format should work fine.

Done.  I specified and implemented such a CIRC_BW event.


Here's the updated proposal 218 (Nick, please don't merge this yet):

https://gitweb.torproject.org/user/karsten/torspec.git/blob/refs/heads/proposal218:/proposals/218-usage-controller-events.txt

Here's the tor branch:

https://gitweb.torproject.org/karsten/tor.git/shortlog/refs/heads/morestats3

Here's a Shadow log file containing all new events:

https://people.torproject.org/~karsten/volatile/scallion.log.gz

Here's the Java program that I used to parse the Shadow log file:

https://people.torproject.org/~karsten/volatile/ParseProposal218Events.java

And finally, here's the output, which should be easier to understand now:

https://people.torproject.org/~karsten/volatile/scallion.txt

Search for "Circuit [fileclient-60.1.0.0]:14" to find the circuit I
mentioned earlier in this thread.


Can you review the proposal changes and tell me if they make sense to you?

Best,
Karsten