[tor-bugs] #5334 [Analysis]: Make simulator authors aware of "sparse exit traffic" and "varying directory load" wishlist items

[Tor Bug Tracker & Wiki]

#5334: Make simulator authors aware of "sparse exit traffic" and "varying
directory load" wishlist items
----------------------+-----------------------------------------------------
 Reporter:  arma      |          Owner:       
     Type:  task      |         Status:  new  
 Priority:  normal    |      Milestone:       
Component:  Analysis  |        Version:       
 Keywords:            |         Parent:  #4682
   Points:            |   Actualpoints:       
----------------------+-----------------------------------------------------
 Proposal 182 points out that we let the read bucket go negative (by
 reading the rest of the TLS record, which was already actually read by
 openssl so we might as well use what it says), but then we don't let the
 corresponding write occur, so we trap the cells inside Tor until the write
 bucket can catch up. The suggested fix is to allow writes based on how
 much we read, to avoid trapping cells.

 But what situations occur where we end up with fewer write tokens in our
 token bucket than read tokens? Two big ones come to mind:

 - Sparse streams at exits, such that we read a few bytes and package them
 into a whole cell. Fetching hundreds of tiny web bug images is a good
 example here, if they're all on different streams. Another example would
 be an irc connection, though most users probably don't know what irc is.
 (An instant message conversation may or may not be a good example, since
 if the conversation is balanced, the exit would be reading cells from the
 Tor network and writing only a few bytes, thus offsetting the scarcity of
 write tokens.) The "sparse exit traffic" issue could be solved by having
 some fraction of the clients fetching that sort of traffic, to see what
 sort of an impact it would have if the behavior became popular.

 - Directory fetches produce wild imbalances between reads and writes. The
 population of users in the simulations seem to fetch a little bit of
 network information once at the beginning, and then never need it after
 that. Great, but it means our simulated environment misses out on this
 aspect of the real network. There are some hackish heuristics in
 global_write_bucket_low() in src/or/connection.c to decide whether a given
 relay should decline to answer a directory request, based on rate limits,
 size of request, current bucket levels, whether we ran out last second,
 etc. Whatever we do for solving proposal 182 should tune these heuristics
 to a) avoid answering if it would hurt our write bucket too much, yet b)
 make sure enough places still answer that everything still works smoothly.
 These contradictory goals seem like they need a realistic simulation
 framework, and I think neither of the current simulation frameworks handle
 the topic well?

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/5334>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online