[tor-bugs] #9321 [Tor]: Load balance right when we have higher guard rotation periods

[Tor Bug Tracker & Wiki]

#9321: Load balance right when we have higher guard rotation periods
-------------------------+-------------------------------------------------
     Reporter:  arma     |      Owner:
         Type:  project  |     Status:  new
     Priority:  normal   |  Milestone:  Tor: 0.2.6.x-final
    Component:  Tor      |    Version:
   Resolution:           |   Keywords:  needs-proposal, tor-auth, tor-
Actual Points:           |  client, 026-triaged-1
       Points:           |  Parent ID:  #11480
-------------------------+-------------------------------------------------

Comment (by asn):

 Replying to [comment:19 asn]:
 > - We keep a summary file for every consensus.
 >
 >   The idea here is that our summary files will be much faster to parse
 >   than full fledged consensuses. Ideally, the parsing time would be
 >   reduced from 1.5 hours to a few seconds/minutes, but I'm not sure if
 >   that's realistic.
 >
 >   To get an idea of the data size, for a 9 months period, we are
 >   talking about 6480 summary files (consensuses) and about 2500 guards
 >   per summary file.
 >
 >   This seems like the most easy to implement and understand scheme,
 >   but I'm not sure if it will be efficient enough.
 >

 I experimented a bit with this idea to see what's the speed improvement.

 FWIW, my initial summary files map 1-1 to consensuses, and they look like
 this:
 {{{
 <summary file header>
 <consensus valid-after time>
 <guard fpr 1>
 <guard fpr 2>
 ...
 }}}
 this is trivial to parse in Python.

 So, for the measurements, I parsed 10 consensuses. When parsed normally,
 the time was:
 {{{
 real    0m13.193s
 user    0m12.960s
 sys     0m0.072s
 }}}
 When I preprocessed them to create summary files for each consensus, and
 then parsed the summary files, the time was:
 {{{
 real    0m0.776s
 user    0m0.708s
 sys     0m0.064s
 }}}

 This is a 17x improvement in time. This means that 9 months of consesuses
 that would originally be parsed in 2 hours, will take about 7 minutes when
 summary files are used.

 Of course, there will also be the preprocessing time, but this can be done
 once for each new consensus...

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9321#comment:20>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online