[tor-bugs] #13339 [Tor]: Merge GSoC project - Consensus Diffs

[Tor Bug Tracker & Wiki]

#13339: Merge GSoC project - Consensus Diffs
-----------------------------+-------------------------------------------
     Reporter:  mvdan        |      Owner:
         Type:  enhancement  |     Status:  needs_revision
     Priority:  major        |  Milestone:  Tor: 0.2.7.x-final
    Component:  Tor          |    Version:
   Resolution:               |   Keywords:  gsoc merge tor-client prop140
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-------------------------------------------

Comment (by mvdan):

 After a quick chat with Nick, what is left to do is clear to me:

 * Clients will never keep more than one consensus, clarify that in
 prop140. Make servers give more than one base consensus in the headers
 when requesting a consensus diff.

 * Figure out the best use of the SaveConsensuses config option. Options
 (by my personal order of preference, will probably go with the first one):
  - Maximum storage to use on disk (oldest removed first)
  - Maximum number to keep on disk (oldest removed first)
  - Maximum amount of time to keep them for

 * Figure out the write_bytes_to_outbuf stuff - both for code duplication
 and for the proper functioning of my code.

 * Some other directory.c and dirserv.c improvements suggested by nickm

 I also noticed that you guys bumped the standard from c89 to c99, so I did
 a style/format rewrite adapting to c99. This makes the code a tad more
 readable, especially some chunks of code in consdiff.c:
 https://github.com/mvdan/tor/commit/d0a9f85ead92895c524203e13c6ff9af8f9282e7

 Apart from those, I only need more eyes on consdiff.c. The ed diff
 generation stuff is not simple when you first dive into it, so feel free
 to ping me with any questions you have. Do let me know if any chunks of
 code are missing comments too.

 And one last thing, there are two items on my TODO list for which I need
 further info:

 > * I'd like to see fewer copies of strings done here. There's an easy way
 to do that, I think.

 nickm, what do you mean by 'here'?

 > * How expensive is dirserv_update_consensus_diffs? It seems kind of
 pricey. Maybe it needs to happen in the background?

 If we do that, we need some kind of mechanism to not serve any consensus
 diffs until they are all updated on disk and mmapped correctly. We need a
 thread-safe way to lock that out until it's complete. It could well get
 pricey, especially if the user sets a large SaveConsensuses value.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/13339#comment:21>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online