[tor-bugs] #16585 [Tor]: relay stops forwarding all traffic when client function cannot establish circuits -- scheduler_run() starvation

[Tor Bug Tracker & Wiki]

#16585: relay stops forwarding all traffic when client function cannot establish
circuits -- scheduler_run() starvation
---------------------------+---------------------------
     Reporter:  starlight  |      Owner:
         Type:  defect     |     Status:  new
     Priority:  major      |  Milestone:
    Component:  Tor        |    Version:  Tor: 0.2.6.10
   Resolution:             |   Keywords:
Actual Points:             |  Parent ID:
       Points:             |
---------------------------+---------------------------

Comment (by starlight):

 Looking at it more, it seems the biggest delays are
 associated with searching the database of relays
 for new circuit paths.

 Seems to me this massive transaction should be
 handled as an independent event queued off to
 a worker thread, but that it is presently
 handled in the main relay worker code path.

 Probably back when only 2000 relays existed
 the delays were barely noticeable, but with
 the population 7000 and rising the problem
 will only become worse with time.

 The system where this ran has two cores but
 the second worker thread/core never hits more
 than 1% utilization.  Have run `top` while
 provoking the issue and is clear that the
 problem is a CPU-bound main `tor` thread.

 NOTE:  In the attached "interesting" log
 extraction, all [debug] lines are removed
 from the first section above the line-break.
 Below the line-break, only selected [debug]
 entries are included, and `channel_write_packed_cell()`
 lines are collapsed to show just the number
 of invocations.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16585#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online