[tor-bugs] #29427 [Core Tor/Tor]: kist: Poor performance with a small amount of sockets

[Tor Bug Tracker & Wiki]

#29427: kist: Poor performance with a small amount of sockets
-----------------------------+------------------------------------
 Reporter:  dgoulet          |          Owner:  (none)
     Type:  defect           |         Status:  new
 Priority:  Medium           |      Milestone:  Tor: 0.4.1.x-final
Component:  Core Tor/Tor     |        Version:  Tor: 0.3.2.1-alpha
 Severity:  Major            |     Resolution:
 Keywords:  tor-sched, kist  |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------------
Changes (by robgjansen):

 * cc: robgjansen (added)


Comment:

 # More intuition about the problem:

 KIST tries not to overload the kernel when there are many sockets, and so
 it only runs every 10 msec. On high performance relays with lots of
 sockets, this is a good thing.

 But on a relay with only 1 active socket, it is possible that you fill the
 kernel socket buffer, and the NIC only takes e.g. 2 msec to send it, but
 then KIST doesn't run again for another 8 msec. So the NIC is sitting
 there idle for those 8 msec.

 This is only an issue when the sum of bytes in all kernel-level socket
 buffers is less than the number of bytes the NIC could send in 10 msec.
 This can happen when there are only a few sockets, or on a really really
 fast NIC.

 -----

 # What to do:

 ## Clients:

 KIST was designed for relays. Clients don't need to prioritize traffic the
 same way relays do, so they don't really need KIST. Clients can simply run
 the vanilla scheduler so that they read/write ASAP (rather than deferring
 I/O like KIST does). Or clients can run KIST with a 1 msec scheduling
 frequency.

 ## Relays:

 For relays, we could guess how long it would take the kernel to send out
 all of the notsent bytes sitting in kernel buffers plus all outbuf bytes
 sitting in Tor outbufs. If the time we guess is less than 10 msec, then we
 could run KIST sooner. This guess would probably involve knowing or
 estimating the NIC speed.

 Slightly more formally, if we know we can write b bytes per KIST
 scheduling run, i.e. b bytes per m milliseconds, but we only actually
 wrote w bytes where w < b, then we know that we need to call back sooner
 than m milliseconds. We can dynamically compute the time it will take us
 to write the w bytes as:

 new_callback_time = m * (w/b)

 Then we check back again in new_callback_time milliseconds (the time when
 the kernel will be empty) instead of m milliseconds.

 Then also, KIST will never let itself write more than b bytes across all
 sockets, because it knows that its network card can't possibly write more
 than b bytes.

 -----

 # Issues:

 I don't know how each relay can reliably compute the value of b. Maybe we
 start with the "observed bandwidth" as an estimate? But then we need to
 allow b to grow in case the relay suddenly got faster, or for new relays?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29427#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online