- tor-relays - lists.torproject.org

Re: [tor-relays] max / burst speed
by Rick Huebner 29 Sep '11

29 Sep '11

On 9/27/2011 1:37 PM, "Steve Snyder" <swsnyder(a)snydernet.net> wrote: > Either there is simply not enough traffic to saturate all available middle nodes or Tor's node selection algorithm is, um, sub-optimal. I just started my relay a month ago, so I've done some research, and it seems to be pretty complicated. Please excuse what turned into an over-long post, but I think this is info that new relay operators don't generally know, and should. At its base, the traffic allocation scheme is elegantly simple: each client randomly selects circuit participants based on the sum of all the relays' bandwidth values from the most recent consensus (not counting complications like starting with a guard node, ending with an exit, etc.). So ideally, a relay with twice the bandwidth of another will (on average) be selected for circuits twice as often, and (again, on average) end up processing twice as much traffic. And also, if the total bandwidth demand from all clients is enough to consume, say, 60% of all Tor relays' available bandwidth, each relay will (on average) be kept operating at about 60% of capacity. But... the bandwidth figure in the consensus that clients use to select relays is not a simple figure. Apparently, it used to be based on the rates reported by each relay in their uploaded server descriptors, but people tried to game the system and fed it bogus data, so that was replaced by the authoritative servers going out and measuring the actual bandwidth of each relay by downloading a test file through it to see how fast it really went. So the values your relay uploads in the descriptor are ignored now, and your set Rate and Burst speeds aren't relevant for traffic allocation except in as much as they affect the speed that the official bandwidth scanner can download the test file through you. The only bandwidth figure that matters for driving traffic to your relay is the one reported in the consensus, which you can see at https://metrics.torproject.org/networkstatus.html, or from your own relay's DirPort at http://<relay>:<dirport>/tor/status-vote/current/consensus if you're a directory mirror. Note that these are not the values from any of the TorStatus servers, they're displaying something else entirely (and not always the same thing as each other, either). So anyway, don't expect a change in your torrc file to immediately bring you more traffic. Your relay has to get rescanned first, which probably only happens once a day. OK, fair enough. But those bandwidth numbers aren't just the simple speed which was seen when the scanner last downloaded a test file through your relay. To smooth out random speed changes as your relay is remeasured day to day, the bandwidth scanners use some kind of fairly slow exponential moving average of your download speeds. So it takes considerable time for changes in your relay's speed to slowly seep into the bandwidth number in the consensus. And for new relays, it seems that the initial value which the new measurements are slowly averaged into starts out pretty low, so that your reported bandwidth also starts out pretty low, and then gradually rises over time to somewhere around your actual Rate limit. And by slow, I mean weeks; my new relay has been running for a month, and over that time, I've seen the reported bandwidth slowly, with many fits and starts and temporary setbacks, go from about 20-30 to ~200 (about half my actual Rate limit), and it's still rising. Which kind of has the effect of putting new relays on probation, and slowly feeding them more and more traffic over time to see how they do, which is not a bad thing at all. But new relay operators are usually excited and anxious to see stuff happen, and need to be aware of this slow starting ramp up period and not get too discouraged or give up because they're not seeing much traffic at first. OK, so the bandwidth rating that matters is measured and slowly averaged... but there's yet another layer. To improve the overall performance of the Tor network, and to help clients generally create faster circuits, there's another bias factor thrown in. I don't know the details, but faster relays have their measured bandwidth figures artificially boosted to drive even more traffic through them than their high bandwidth would naturally attract, and/or slower relays have their bandwidth figures artificially lowered to drive less traffic through them (not sure which or both, but the effect is the same regardless). So if the overall client bandwidth demand is, again, 60% of the total Tor network bandwidth available, instead of each relay being at ~60% capacity, the fastest relays will be more fully utilized, and, unavoidably, that means that the slower relays will be correspondingly less utilized. This might dismay slower relay operators who feel that they're being prevented from contributing as much as they'd like, but objectively, it's generally better for a Tor client to have a 300 KB/s circuit hop than a 30 KB/s one. The faster relays are just nicer for clients to use, and it's better overall for the Tor network to make sure they get used as much as possible. And if those fast relays are getting more than their prorated "fair" share of usage based on their actual speeds, that unavoidably means that slower relays are getting less usage than their speed would normally merit. But that doesn't mean the slower relays are useless! Simply by existing, those extra relays greatly increase the difficulty of various attacks on Tor, just because they *might* have been used for any given circuit. Also, the whole guard node system for making certain nasty attacks infeasible relies on having lots of potential guard nodes to choose between, even relatively slow ones. And of course, all exit nodes are especially precious, almost regardless of speed. And finally, even if they're not used all that much while client demand on the Tor network is low to moderate, they provide an important spare reserve of bandwidth to make sure that some relay somewhere will always be ready to handle a new circuit even if the network becomes very busy and manages to max out the high speed "backbone" relays, or Tor is subjected to some kind of DOS attack. So anyway, for all the relay operators asking "why isn't my relay being used more?", there's your infodump. If it's a new relay, or you recently upgraded/raised your speed limits, keep an eye on the official bandwidth figure for your relay in the consensus, especially the nice graph displayed in the Router Detail page that you reach by clicking your router link on the https://metrics.torproject.org/networkstatus.html page. If that graph shows any upwards trend, the effects of your change are still slowly percolating into your official bandwidth figure, and more traffic will appear as it rises. If it's plateaued out, you're getting your share of the overall Tor traffic based on your relay's overall performance and the total client demand on the Tor network. For slow nodes who've limited their overall Rate to avoid hitting bandwidth caps, you might consider using AccountingMax to cap the usage to a safe level, and increase your speeds; you may find it more rewarding to relay significant traffic for 6 hours per day and then hibernate for 18 than to stay on "inactive reserve" status all the time. From the overall viewpoint of the network, is it better to have 1000 new relays at good speeds up 1/4 of the time (effectively adding 250 new fast relays), or to have them at slow speeds all of the time, not being used much? I'm not really sure, but I've noticed that the AccountingMax hibernation feature is hardly used at all from what I see on TorStatus, and I wonder why. OK, enough already, this turned out way longer than I was expecting. Hope it helps.

4 4

(no subject)
by Tim Myers 28 Sep '11

28 Sep '11

Stop Sent via BlackBerry by AT&T

3 2

Re: [tor-relays] max / burst speed
by Rick Huebner 28 Sep '11

28 Sep '11

On 9/27/2011 1:37 PM, "Steve Snyder" <swsnyder(a)snydernet.net> wrote: > Also, I have come to distrust TorStatus. Why? Because difference sites will have different numbers while supposedly monitoring the same network. One of my nodes is currently showing an Observed bandwidth of 10KB (yes, ten) at torstatus.blutmagie.de while the same node is showing an Observed 83KB at torstatus.all.de. Don't take the reported values as gospel. They seem to be measuring different things. As far as I can tell, blutmagie.de appears to be calculating the average bandwidth actually used overall per day, i.e. the average of the rates shown in the Recent Read/Write History graphs, which in turn appear to be faithful copies of the node's reported extra-info data. all.de reports the node's own observed bandwidth figure from its most recently uploaded server descriptor, and I have no idea what it's showing in its Read/Write History graphs, it's not the extra-info history data, and I can't relate it to anything. So blutmagie.de shows how much traffic the different routers are actually processing on average per second, and all.de shows the peak speed reached by each router in a short sprint (10 seconds?).

1 0

Re: [tor-relays] max / burst speed
by Rick Huebner 27 Sep '11

27 Sep '11

On 9/27/2011 1:37 PM, tor-relays-request(a)lists.torproject.org wrote: > No idea what shaping algorithm Tor uses, nor any clue on recommended > burst ratios under said algorithm. Anyone???. I believe Tor uses a token bucket algorithm (https://secure.wikimedia.org/wikipedia/en/wiki/Token_bucket) for traffic shaping. Which works sort of like a fuel tank with a steady rate of fuel being replenished each second (RelayBandwidthRate) and a max capacity of RelayBandwidthBurst bytes. If the tank is full, up to RelayBandwidthBurst bytes can be consumed, but only RelayBandwidthRate bytes will be replenished each second. So there's a sort of reserve available of Burst - Rate bytes, and consumption > Rate bytes/second depletes the reserve, and consumption < Rate refills the reserve, up to the max capacity. In general I think the higher the Burst you can allow, the better; it helps fast clients with small requests to zip through your relay quickly without clamping them to your overall Rate limit, and doesn't cost you anything since your long-term average is still enforced (bursts over your limit are balanced with slack periods under your limit). But I'm not clear on what the highest effective value would be. I could even see a point to having the Burst higher than your physical connection's actual limit, e.g. setting it to 10 times your connection's actual limit would allow bursts to max out your connection for 10 seconds (occasionally, to be balanced again by slack periods under your rate). But really, I doubt that values more than 2 or 3 times your Rate limit will actually matter, since Tor traffic doesn't seem to often come in those kinds of isolated high-multiple bursts (probably the natural averaging effect of having many clients routing dozens/hundreds? of circuits though your relay at once). A simple pragmatic test might be to run the Vidalia bandwidth meter for a day or two, and if the peak rate isn't hitting your Burst, it's high enough not to matter. One interesting side effect of a high Burst might be its impact on the way your relay's bandwidth is measured by the official bandwidth scanners, though. If the bandwidth scanner's test download file is small enough (less than many second's worth of your overall Rate limit bandwidth), a high Burst might allow the test file to be downloaded much more quickly than with a low Burst, raising your official measured bandwidth rating (and drawing more traffic to your relay). Larger test download files would mitigate this, but increase the scanner's workload and time between each relay's re-scans. Just a thought, I don't know how much effect this actually has.

1 0

max / burst speed
by Sebastian Urbach 27 Sep '11

27 Sep '11

Hi all, I doubled my max / burst speed values for my relay a few days ago. Unfortunately the new values are displayes in the statistic but not used at all. Does anyone have a suggestion for me how to get more work :-) ? Thanks and cheers -- Mit freundlichen Grüßen / Yours sincerely Sebastian Urbach -------------------------------------------------------- Religion is something left over from the infancy of our intelligence, it will fade away as we adopt reason and science as our guidelines. -------------------------------------------------------- Bertrand Arthur William Russell (1872-1970), British philosopher, logician, mathematician, historian, and social critic.

7 12

Fwd: Arm 1.4.4 Release
by Damian Johnson 26 Sep '11

26 Sep '11

(for the rare person that isn't on both tor-talk@ and tor-relays@...) ---------- Forwarded message ---------- From: Damian Johnson <atagar1(a)gmail.com> Date: Sun, Sep 25, 2011 at 8:21 PM Subject: Arm 1.4.4 Release To: tor-talk(a)lists.torproject.org Hi all. A new release of arm (http://www.atagar.com/arm/) is now available. Besides the normal batch of bug fixes and minor features this includes an interactive interpretor for raw control port access... http://www.atagar.com/arm/images/screenshot_interpretor_full.png http://www.atagar.com/arm/releaseNotes.php#1.4.4 This is intended to be a tool for developers, highly knowledgeable operators, and anyone that would like to learn about Tor's control protocol. It provides usability improvements like tab completion and history scroll-back, along with IRC style interpretor commands... * /help - provides usage information for all of the tor/interpretor commands and tor's configuration options * /info - queries relay information via fingerprint, nickname, or IP address * /find - searches the backlog for the given regex * /events - displays any events that we've listened for * /write - dumps the interpretor backlog to a file This can both be used via a new page in the curses interface and as a standalone prompt by running "arm --prompt"... http://www.atagar.com/arm/images/screenshot_interpretorPrompt.png Hopefully I'm not the only person that finds this to be useful! -Damian PS. Jesse, Spider.007, and pyllyukko: Tor is now hosting a mirror of the arm tarballs at... https://archive.torproject.org/arm/ This should, of course, be a little preferable for Gentoo, Arch Linux, and Slackware since it provides SSL using their cert. Thanks again for maintaining arm on those platforms!

1 0

Tor relays
by Mahesh Arunachalam 24 Sep '11

24 Sep '11

Please send me some tor relays.

2 1

Re: [tor-relays] Upgrading to 0.2.3.4 on Debian
by auto63557492＠hushmail.com 17 Sep '11

17 Sep '11

Updated my sources.list to have just this for tor: "deb http://deb.torproject.org/torproject.org/ wheezy main" When I run sudo apt-get update, system returns: "W: Failed to fetch http://deb.torproject.org/torproject.org/dists/wheezy/Release Unable to find expected entry main/binary-powerpc/Packages in Meta- index file (malformed Release file?)" Should there be a "binary-powerpc" directory under wheezy/main (or testing/main, stable/main, squeeze/main or any other for that matter)? Or am I missing something else? Again, thanks in advance, Bill - - - - Date: Thu, 15 Sep 2011 18:44:53 +0200 From: Moritz Bartl <moritz(a)torservers.net> Subject: Re: [tor-relays] Upgrading to 0.2.3.4 on Debian To: tor-relays(a)lists.torproject.org Message-ID: <4E722B85.9050006(a)torservers.net> Content-Type: text/plain; charset=ISO-8859-1 On 15.09.2011 06:41, auto63557492(a)hushmail.com wrote: > I run aptitude regularly, and my sources.list includes "deb > http://deb.torproject.org/torprojects.org squeeze main" but there > are no updates coming through for tor beyond 0.2.1.29-1 on my ppc > platform. > Should there be, or should I be doing something else to get to > 0.2.3.4? Am I even asking the right question? add deb http://deb.torproject.org/torproject.org experimental-squeeze main if you want unstable releases. -- Moritz Bartl https://www.torservers.net/

1 0

Upgrading to 0.2.3.4 on Debian
by auto63557492＠hushmail.com 15 Sep '11

15 Sep '11

I setup a relay to support the cause, but also as a means to learn about Linux - still a long way to got on the latter, and so this question: I run aptitude regularly, and my sources.list includes "deb http://deb.torproject.org/torprojects.org squeeze main" but there are no updates coming through for tor beyond 0.2.1.29-1 on my ppc platform. Should there be, or should I be doing something else to get to 0.2.3.4? Am I even asking the right question? Thanks...

4 3

increase txqueuelen: reasonable or not?
by Steve Snyder 26 Aug '11

26 Aug '11

The config page at torservers.net advises increasing the TX Queue Length from the default 1,000 to 20,000. For a differing opinion see the assertion that this increases latencies and hampers network congestion recovery ("bufferbloat"): http://www.cyberciti.biz/faq/gentoo-centos-rhel-debian-fedora-increasing-tx… I'm running an exit node on a 100Mbps network (multi-core CPU on RHEL5), and typically have 600 - 800 connections at any given time. TorStatus usually reports my throughput at 700KB to 800KB per second. I suspect that the 20,000 recommendation pertains to Gbit Ethernet. Still, though, is the default of 1,000 adequate for my Tor traffic? More generally, I haven't tweaked my networking (system network buffer sizes, etc.) because it is unclear to me what recommendations are appropriate for exit nodes running contemporary versions (v0.2.2.30+) of Tor. Reliable benchmarking can't be done because the system load varies with traffic at any given time. Is there a way to prove empirically that this or that adjustment to the system configuration really has an effect on the performance of a Tor relay node? Thanks.

4 3