[tor-relays] Unused Tor exit nodes capacity

Sun Dec 13 23:06:02 UTC 2015

> On 14 Dec 2015, at 07:18, Dirk Eschbach <tor-relay.dirk at o.banes.ch> wrote:
...
> The big question now is:
> Why do the machines do not have more throughput ?
> Is the reason for this the way the distribution through the Tor network
> works.
> Moritz hinted it might have to do with the way the tor "bandwidth
> scanners" measure the ability of a server to handle traffic.

No, this is not the issue, your relay's own self-measured throughput is the issue. See below.

> Can you explain me / point me to documentation where this process is
> described and how this can be optimized.
> What are the criteria for tor exit node server traffic distribution ?

By consensus weight, which is determined by the bandwidth authorities.
https://blog.torproject.org/blog/lifecycle-of-a-new-relay <https://blog.torproject.org/blog/lifecycle-of-a-new-relay>
https://stem.torproject.org/tutorials/examples/votes_by_bandwidth_authorities.html <https://stem.torproject.org/tutorials/examples/votes_by_bandwidth_authorities.html>

> How do the clients choose the exit ?

From those servers they believe exit to the address and port they want, randomly, weighted by the server's consensus weight.

The details of bandwidth weight selection are in section 3.4.2 of the Directory Specification:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt <https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt>

"The bandwidth in a "w" line should be taken as the best estimate
   of the router's actual capacity that the authority has.  For now,
   this should be the lesser of the observed bandwidth and bandwidth
   rate limit from the server descriptor.  It is given in kilobytes
   per second, and capped at some arbitrary value (currently 10 MB/s).

   The Measured= keyword on a "w" line vote is currently computed
   by multiplying the previous published consensus bandwidth by the
   ratio of the measured average node stream capacity to the network
   average. If 3 or more authorities provide a Measured= keyword for
   a router, the authorities produce a consensus containing a "w"
   Bandwidth= keyword equal to the median of the Measured= votes."

When I look at the bandwidth authority votes for DigiGesTor1e1, they say:
w Bandwidth=9586 Measured=24200
w Bandwidth=9586 Measured=15900
w Bandwidth=9586 Measured=43500
w Bandwidth=9586 Measured=19700
w Bandwidth=9586 Measured=19200

Those votes are in these large files:
http://171.25.193.9:443/tor/status-vote/current/authority <http://171.25.193.9:443/tor/status-vote/current/authority>
http://199.254.238.53/tor/status-vote/current/authority <http://199.254.238.53/tor/status-vote/current/authority>
http://131.188.40.189/tor/status-vote/current/authority <http://131.188.40.189/tor/status-vote/current/authority>
http://128.31.0.34:9131/tor/status-vote/current/authority <http://128.31.0.34:9131/tor/status-vote/current/authority>
http://154.35.175.225/tor/status-vote/current/authority <http://154.35.175.225/tor/status-vote/current/authority>

So the bandwidth authorities have having no trouble measuring your relay, they think it should be 2x - 4x as fast.
Your relay itself has't observed itself sustaining that performance over a 10-second interval, so it won't allow the directory authorities to assign it more bandwidth.

Section 2.1.1 of the Directory Specification:

""bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL

       [Exactly once]

       Estimated bandwidth for this router, in bytes per second.  The
       "average" bandwidth is the volume per second that the OR is willing to
       sustain over long periods; the "burst" bandwidth is the volume that
       the OR is willing to sustain in very short intervals.  The "observed"
       value is an estimate of the capacity this relay can handle.  The
       relay remembers the max bandwidth sustained output over any ten
       second period in the past day, and another sustained input.  The
       "observed" value is the lesser of these two numbers."

Please improve the throughput your relay can sustain over a 10-second period.
Try some performance tuning steps, testing your relay with Tor client after each one.
(See below.)

Have you set a limit on MaxAdvertisedBandwidth in the torrc files?

> Konsole output
> top - 09:57:49 up 16 days, 11:57,  1 user,  load average: 1.14, 0.91, 0.81
> Tasks:244 total,  3 running,241 sleeping,  0 stopped,  0 zombie
> %Cpu0  :12.5 us, 3.4 sy, 0.0 ni,79.7 id, 0.0 wa, 0.0 hi, 4.4 si, 0.0 st
> %Cpu1  :15.9 us, 5.4 sy, 0.0 ni,74.3 id, 0.3 wa, 0.0 hi, 4.1 si, 0.0 st
> %Cpu2  :14.3 us, 2.7 sy, 0.0 ni,77.0 id, 0.0 wa, 0.0 hi, 6.0 si, 0.0 st
> %Cpu3  : 9.5 us, 3.4 sy, 0.0 ni,80.3 id, 0.0 wa, 0.0 hi, 6.8 si, 0.0 st
> KiB Mem:   3877624 total, 2739880 used, 1137744 free,    1288 buffers
> KiB Swap: 4026364 total,  364264 used, 3662100 free.   10752 cached Mem
> 
> Looking at the network connection it is without any problem possible to
> start big downloads without reducing TOR throughput.

Have you tried doing a large download through your exit via a Tor client and seeing how fast that is?
ExitNodes <fingerprint od your exit>
StrictNodes 1

> The servers are connected with 1 Gbit/s each.

It looks like your Tor processes are neither CPU nor network-bound.

Does your network connection drop packets or have large latency?

How many file descriptors are the tor processes allowed to open?
How many connections does each tor process have open at once?
(There should be thousands per process on a busy relay.)

https://gitweb.torproject.org/tor.git/tree/doc/TUNING <https://gitweb.torproject.org/tor.git/tree/doc/TUNING>

Are there any performance-related messages in the Tor logs?

Is your hardware / kernel / firewall / etc. capable of handling many connections?

Does your provider rate-limit any kinds of traffic?
Does your provider limit the number of open connections?

Is your DNS resolver keeping up with the requests?
Do you have a local caching DNS resolver on each machine?

Tim

Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP 968F094B

teor at blah dot im
OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20151214/65fca641/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20151214/65fca641/attachment-0001.sig>