[tor-bugs] #13339 [Tor]: Merge GSoC project - Consensus Diffs

[Tor Bug Tracker & Wiki]

#13339: Merge GSoC project - Consensus Diffs
-------------------------+-------------------------------------------------
     Reporter:  mvdan    |      Owner:
         Type:           |     Status:  needs_revision
  enhancement            |  Milestone:  Tor: 0.2.7.x-final
     Priority:  major    |    Version:  Tor: 0.2.7
    Component:  Tor      |   Keywords:  gsoc, merge, tor-client, prop140,
   Resolution:           |  027-triaged-1-in,TorCoreTeam201508
Actual Points:           |  Parent ID:
       Points:  medium   |
-------------------------+-------------------------------------------------

Comment (by mvdan):

 Replying to [comment:28 nickm]:
 > Replying to [comment:25 mvdan]:
 > > I've addressed most of the comments:
 https://github.com/mvdan/tor/commits/consdiff
 > >
 > > The last commit adds four TODOs, which I think are the only things
 left to deal with.
 > >
 > > > // TODO: change bool with max size to take up in disk
 > >
 > > I'll do this one.
 >
 > Great.

 I'll do it on top of my current consdiff branch. Will you work on a branch
 of yours? Feel free to cherry pick my future commits.

 > > > // TODO: check that the stored consensus is intact
 > >
 > > Not sure how to do this one yet.
 >
 > Hm.  Maybe we should store it along with a digest, or try to parse it
 before we compute the diff?

 What if the digest is corrupt? :)

 Trying to parse it sounds better. Although that just checks whether it is
 valid, not whether it is correct.

 I can't think of a simple solution to all of this right now. Surely there
 must be other files that may be corrupt in the filesystem when Tor starts
 up. What does it do for the rest? Sounds to me like we would be better off
 with a generic mechanism to check whether the tor data/config dir hasn't
 been tinkered with.

 > > > // TODO: duplicate code from add_networkstatus_bytes_to_outbuf,
 works but is
 > > > // probably buggy.
 > >
 > > All of this code is confusing to me, so help from someone who
 initially wrote it would be great.
 > >
 > > > // TODO: might want to do this in the background and have a lock to
 not
 > > > // serve consensus diffs until they are updated
 > >
 > > What do we want to do about this one after all?
 >
 > I can do these last two.
 >
 > The final one will depend on the runtime of the code.  How long does it
 take to recompute all the diffs?  If it's a few milliseconds, that's fine.
 If it's a few '''hundred''' milliseconds or worse, we probably need to
 stick it in another thread.

 Back when I ran tests on my algorithm code, I did compute times for single
 test cases. I never tested the speed of the whole diff generation process
 at startup.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/13339#comment:29>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online