[tor-bugs] #12538 [Tor]: Make all relays automatically be dir caches

[Tor Bug Tracker & Wiki]

#12538: Make all relays automatically be dir caches
-------------------------------------------------+-------------------------
 Reporter:  cypherpunks                          |          Owner:
     Type:  task                                 |         Status:
 Priority:  High                                 |  needs_review
Component:  Tor                                  |      Milestone:  Tor:
 Severity:  Normal                               |  0.2.8.x-final
 Keywords:  tor-guard, tor-relay, prop237,       |        Version:  Tor:
  026-triaged-1, sebastian-review,               |  unspecified
  027-triaged-1-out, 028-triage, 028-triaged,    |     Resolution:
  mike-can, pre028-patch                         |  Actual Points:
Parent ID:                                       |         Points:
  Sponsor:                                       |  medium/large
-------------------------------------------------+-------------------------

Comment (by teor):

 Replying to [comment:73 sysrqb]:
 > Replying to [comment:68 nickm]:
 > > `feature12538_rebased_7` in my public repository is rebased on master,
 with the NotDir/BelieveNotDir stuff removed, and the bugs mentioned above
 fixed.
 > >
 > > My only remaining issue is:
 > > > * I worry about the definition of dir_server_mode(). Possibly, if we
 have advertised that we're a dirserver, we should actually be a dirserver?
 >
 > Alternatively,
 >
 > 1) if we have the V2Dir flag then dir_server_mode() returns 1, else the
 function is unchanged.
 > 2) when options->DirCache is disabled, dir_server_mode() only uses
 options->DirCache after the next consensus is published. Serve the
 documents we currently have without caching new docs until then.
 > 3) split dir_server_mode() into dir_server_mode() and dir_cache_mode(),
 where dir_server_mode() looks at our is_v2_dir flag rather than
 options->DirCache shortcircuit; and dir_cache_mode() is:
 > {{{ return options->DirCache && dir_server_mode(); }}}. Then
 directory_permits_begindir_requests() depends on dir_server_mode() and we
 continue serving clients until we notice we lost the V2Dir flag.
 >
 > I worry because each of these is slightly surprising for relay operators
 in different ways. Tweaking the man page's description of DirCache so it
 says the relay stops caching new directory documents and stops accepting
 client dir connections after it loses its V2Dir flag (if it earned it), is
 easy. Or, we differentiate between caching and serving, always serving if
 we have V2Dir but only cache if DirCache is set. This could be sad for
 client's though, depending on how often they see cache misses, but maybe
 this is a general improvement and not something to be overly concerned
 about.

 I think if an operator sets DirCache 0, caching *and* serving should stop.
 This is analogous with setting DirPort 0, and obeys the "principle of
 least surprise".
 Otherwise, what can an operator do to stop serving directory documents
 right away?
 (For example, what if they are close to quota for the month?)

 If DirCache is 1 but we don't yet have the V2Dir flag:
 To be consistent with DirPort directories, we could serve as soon as
 DirCache is 1.
 (This also allows self-testing. Do we do that for DirCache directories in
 this patch?)
 Just like DirPort directories, we could trust clients to obey the
 consensus and not contact us until we get the V2Dir flag. (Is there any
 harm here?)

 Clients are already pretty tolerant of directory misses.
 #4483 will make clients much more tolerant of directory misses during
 initial bootstrap.
 (Clients have very little server information during bootstrap, so misses
 on the few servers they have are more of an issue.)

 > > Also we should test this on chutney to verify that everything works
 correctly for servers without DirPort support.
 >
 >
 > Replying to [comment:69 teor]:
 > > It's important that these relays work in mixed dirport / no dirport
 networks, which probably means creating a new torrc template and a new
 network, then testing that network as part of make test-network-all in
 tor.
 > When I ran previous branches in chutney with mixed versions and mixed
 dirport, I modified them manually. By this I mean I created 100 nodes,
 then disabled the dirport on a handful of them, then I started the test
 network. When all nodes were running, I stopped a few of the dirport-
 enabled instances and restarted them with a binary of a different tor
 version. It wasn't the most elegant, but it worked.

 Yes, chutney works well with manual modification - I do it all the time to
 test new features.

 But this feature is so important I want to make sure we test it regularly.
 If we integrate it into chutney's test suite, and then into tor's "make
 test-network-all", it will get tested on a regular basis.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/12538#comment:74>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online