[tor-bugs] #21345 [Core Tor/Tor]: Do relays count dir reqs as completed before they're complete?

[Tor Bug Tracker & Wiki]

#21345: Do relays count dir reqs as completed before they're complete?
--------------------------+------------------------------------
 Reporter:  arma          |          Owner:
     Type:  task          |         Status:  new
 Priority:  Medium        |      Milestone:  Tor: 0.3.1.x-final
Component:  Core Tor/Tor  |        Version:
 Severity:  Normal        |     Resolution:
 Keywords:                |  Actual Points:
Parent ID:                |         Points:
 Reviewer:                |        Sponsor:
--------------------------+------------------------------------

Comment (by arma):

 Here was the original write-up that I did on my airplane this weekened. I
 have since come to question whether it's actually true, i.e. whether
 there's actually a bug.

 Directory mirrors include a dirreq-v3-reqs line in their extrainfo
 descriptors, including a count of how many consensus fetches were
 successfully delivered to users.

 We decide it's completed when geoip_change_dirreq_state() gets called and
 either of these is true:
 {{{
   if ((type == DIRREQ_DIRECT &&
          new_state == DIRREQ_FLUSHING_DIR_CONN_FINISHED) ||
       (type == DIRREQ_TUNNELED &&
          new_state == DIRREQ_CHANNEL_BUFFER_FLUSHED)) {
 }}}

 That first one (direct) happens from connection_dir_finished_flushing()
 which is called from connection_finished_flushing() which is called from
 connection_handle_write_impl() which only calls finished_flushing if
 there's still nothing to flush after it called flushed_some, and it's
 complicated, but I think
 connection_dirserv_add_networkstatus_bytes_to_outbuf() will keep adding
 more bytes to the outbuf until there aren't any more to add. So long story
 short, for the DIRREQ_DIRECT case, I think we don't mark it completed
 until the last byte gets flushed from the directory conn. That's good.

 That second one (tunneled) is where the problem is. It gets called from
 channel_notify_flushed(), which gets called from
 connection_handle_write_impl() whenever buf_datalen(outbuf) is 0 after
 flush_buf_tls() finishes. So, the directory request gets marked as
 completed as soon as the first 16384 (DIRSERV_BUFFER_MIN) bytes get
 flushed onto the linked TLS conn. And on Linux, the kernel is happy to lie
 to you about the first several hundred KB of data, telling you it flushed
 when it hasn't done anything but get moved to kernel buffers.

 tl;dr, we mark a tunneled networkstatus directory response as completed as
 soon as we start answering it.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21345#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online