- tor-commits - lists.torproject.org

[torspec/master] changes to 186 based on comments from Karsten
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit e80e9974fa38eeff7ae63a03dd6f8c2a0df858b6 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Sep 22 17:21:24 2011 -0400 changes to 186 based on comments from Karsten --- proposals/186-multiple-orports.txt | 27 +++++++++++++++------------ 1 files changed, 15 insertions(+), 12 deletions(-) diff --git a/proposals/186-multiple-orports.txt b/proposals/186-multiple-orports.txt index 5b59c77..56a76c7 100644 --- a/proposals/186-multiple-orports.txt +++ b/proposals/186-multiple-orports.txt @@ -24,14 +24,14 @@ Configuring additional addresses and ports: In consonance with our changes to the (Socks|Trans|NATD|DNS)Port options made in 0.2.3.x for proposal 171, I make a corresponding - change to allow multiple SocksPort options and deprecate - SocksListenAddress. + change to allow multiple ORPort options and deprecate + ORListenAddress. The new syntax will be: - "SocksPort" PortDescription Options? + "ORPort" PortDescription Option* - Options = "NoAdvertise" | "NoListen" | "AllAddrs" | "IPV4Only" + Option = "NoAdvertise" | "NoListen" | "AllAddrs" | "IPV4Only" | "IPV6Only" PortDescription = PORTLIST | @@ -41,7 +41,7 @@ Configuring additional addresses and ports: (PORTLIST and ADDRESS are defined below.) The 'NoAdvertise' option performs the function of the old - SocksListenAddress option. If it is set, we bind a port, but + ORListenAddress option. If it is set, we bind a port, but don't put it in our descriptor. The 'NoListen' option tells Tor to advertise an address, but not @@ -60,7 +60,7 @@ Configuring additional addresses and ports: As with the client *Port options, only the old format or the new format are allowed: either a single numeric socksport and zero or more sockslistenaddress options, or a set of one or more - SocksPorts in the new extended format. + ORPorts in the new extended format. In current operating systems (unless we get into crazy nonportable tricks) we need to use one socket for every address:port that Tor @@ -71,21 +71,21 @@ Configuring additional addresses and ports: Example: We want to bind on 0.0.0.0:9001 - SocksPort 9001 + ORPort 9001 Example: Our firewall is redirecting ports 80, 443, and 7000-8000 on all hosts in x.244.2.0/24 onto our port 2929. - SocksPort 2929 no-advertise - SocksPort x.244.2.0/24:80,443,7000-8000 no-listen + ORPort 2929 noadvertise + ORPort x.244.2.0/24:80,443,7000-8000 nolisten Example: We have a dynamic DNS provider that maps tornode.example.com to our current external IPv4 and IPv6 addresses. Our firewall forwards port 443 on those address to our port 1337. - SocksPort 1337 no-advertise alladdrs - SocksPort tornode.example.com:443 no-bind alladdrs + ORPort 1337 noadvertise alladdrs + ORPort tornode.example.com:443 nobind alladdrs Self-testing: @@ -93,7 +93,7 @@ Self-testing: before they declare themselves reachable. If a Tor has a lot of advertised ports, that could be prohibitive. Instead, it should try a sample of ports for each address. It should - not advertise any given SocksPort line until it has tried + not advertise any given ORPort line until it has tried extending to or connecting to a sample of the address/port combinations. @@ -129,6 +129,9 @@ New descriptor syntax: A node must not list more than 8 or-address lines. + A PORTLIST must have no more than 16 PORTSPEC entries, and its entries must + be disjoint. + (Q: Any reason to allow more than 2? Multiple interfaces, I guess.) New authority behavior:

1 0

[torspec/master] Make 185 a little more explicit on who sets dir-cache in their routerdesc
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit d1fbdaf64ce9d58530a606bf7f44409d9f9d894f Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Sep 22 16:19:00 2011 -0400 Make 185 a little more explicit on who sets dir-cache in their routerdesc --- proposals/185-dir-without-dirport.txt | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/proposals/185-dir-without-dirport.txt b/proposals/185-dir-without-dirport.txt index c84ec35..838a58a 100644 --- a/proposals/185-dir-without-dirport.txt +++ b/proposals/185-dir-without-dirport.txt @@ -30,8 +30,12 @@ Configuration: Advertising cache status: Nodes which are running as a directory cache but which do not have - a DirPort set should set the entry "dir-cache 1" in their router - descriptors. + a DirPort set, or which do not have a working DirPort, should set + the entry "dir-cache 1" in their router descriptors and give their + directory port as 0 in their router lines. (Nodes which have a + working directory port advertise it as usual, and do not include + a "dir-cache" line. Implementations should accept and ignore + dir-cache lines with values other than "dir-cache 1".) Consensus:

1 0

[tor/master] New torrc option to allow bucket refill intervals of less than 1 sec
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 6b1d8321aef15a948ec32333a98217450bd02f78 Author: Florian Tschorsch <tschorsch(a)informatik.uni-wuerzburg.de> Date: Wed Sep 7 20:21:53 2011 -0400 New torrc option to allow bucket refill intervals of less than 1 sec Implements bug3630. --- doc/tor.1.txt | 8 ++++ src/or/config.c | 8 ++++ src/or/connection.c | 64 +++++++++++++++++------------- src/or/main.c | 110 +++++++++++++++++++++++++++++++++++++++----------- src/or/or.h | 3 + 5 files changed, 141 insertions(+), 52 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 4edee80..05f52f8 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -737,6 +737,14 @@ The following options are useful only for clients (that is, if unattached waiting for an appropriate circuit, before we fail it. (Default: 2 minutes.) +**TokenBucketRefillInterval** __NUM__:: + Set the refill interval of Tor's token bucket to NUM milliseconds. + NUM must be positive and either a divisor or a multiple of 1 second. + Note that this option retains the configured bandwidth limits and refills + token buckets only in ratio to the interval. This option will be ignored + when Tor was built with Libevent's bufferevents enabled. (Default: 1 second) + + **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent connections to hosts that match this value and attempt to reuse the same diff --git a/src/or/config.c b/src/or/config.c index bc77b3a..b04cee4 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -386,6 +386,7 @@ static config_var_t _option_vars[] = { OBSOLETE("SysLog"), V(TestSocks, BOOL, "0"), OBSOLETE("TestVia"), + V(TokenBucketRefillInterval, MSEC_INTERVAL, "10 msec"), V(TrackHostExits, CSV, NULL), V(TrackHostExitsExpire, INTERVAL, "30 minutes"), OBSOLETE("TrafficShaping"), @@ -1382,6 +1383,13 @@ options_act(const or_options_t *old_options) if (accounting_is_enabled(options)) configure_accounting(time(NULL)); + if (options->TokenBucketRefillInterval < 0 + || options->TokenBucketRefillInterval > 1000) { + log_warn(LD_CONFIG, "Token bucket refill interval must be in the range " + "of [0:1000]"); + return -1; + } + #ifdef USE_BUFFEREVENTS /* If we're using the bufferevents implementation and our rate limits * changed, we need to tell the rate-limiting system about it. */ diff --git a/src/or/connection.c b/src/or/connection.c index 0092030..42d7e2f 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2389,20 +2389,21 @@ connection_bucket_init(void) } /** Refill a single bucket called name with bandwidth rate - * rate and bandwidth burst burst, assuming that - * seconds_elapsed seconds have passed since the last call. - **/ + * per millisecond rate and bandwidth burst per refill interval + * burst, assuming that milliseconds_elapsed milliseconds + * have passed since the last call. */ static void connection_bucket_refill_helper(int *bucket, int rate, int burst, - int seconds_elapsed, const char *name) + int milliseconds_elapsed, + const char *name) { int starting_bucket = *bucket; - if (starting_bucket < burst && seconds_elapsed) { - if (((burst - starting_bucket)/seconds_elapsed) < rate) { + if (starting_bucket < burst && milliseconds_elapsed) { + if (((burst - starting_bucket)/milliseconds_elapsed) < rate) { *bucket = burst; /* We would overflow the bucket; just set it to * the maximum. */ } else { - int incr = rate*seconds_elapsed; + int incr = rate*milliseconds_elapsed; *bucket += incr; if (*bucket > burst || *bucket < starting_bucket) { /* If we overflow the burst, or underflow our starting bucket, @@ -2416,41 +2417,46 @@ connection_bucket_refill_helper(int *bucket, int rate, int burst, } } -/** A second has rolled over; increment buckets appropriately. */ +/** Time has passed; increment buckets appropriately. */ void -connection_bucket_refill(int seconds_elapsed, time_t now) +connection_bucket_refill(int milliseconds_elapsed, time_t now) { const or_options_t *options = get_options(); smartlist_t *conns = get_connection_array(); - int relayrate, relayburst; + int bandwidthrate, bandwidthburst, relayrate, relayburst; + + bandwidthrate = (int)options->BandwidthRate / 1000; + bandwidthburst = (int)options->BandwidthBurst; if (options->RelayBandwidthRate) { - relayrate = (int)options->RelayBandwidthRate; + relayrate = (int)options->RelayBandwidthRate / 1000; relayburst = (int)options->RelayBandwidthBurst; } else { - relayrate = (int)options->BandwidthRate; - relayburst = (int)options->BandwidthBurst; + relayrate = bandwidthrate; + relayburst = bandwidthburst; } - tor_assert(seconds_elapsed >= 0); + tor_assert(milliseconds_elapsed >= 0); write_buckets_empty_last_second = global_relayed_write_bucket <= 0 || global_write_bucket <= 0; /* refill the global buckets */ connection_bucket_refill_helper(&global_read_bucket, - (int)options->BandwidthRate, - (int)options->BandwidthBurst, - seconds_elapsed, "global_read_bucket"); + bandwidthrate, bandwidthburst, + milliseconds_elapsed, + "global_read_bucket"); connection_bucket_refill_helper(&global_write_bucket, - (int)options->BandwidthRate, - (int)options->BandwidthBurst, - seconds_elapsed, "global_write_bucket"); + bandwidthrate, bandwidthburst, + milliseconds_elapsed, + "global_read_bucket"); connection_bucket_refill_helper(&global_relayed_read_bucket, - relayrate, relayburst, seconds_elapsed, + relayrate, relayburst, + milliseconds_elapsed, "global_relayed_read_bucket"); connection_bucket_refill_helper(&global_relayed_write_bucket, - relayrate, relayburst, seconds_elapsed, + relayrate, relayburst, + milliseconds_elapsed, "global_relayed_write_bucket"); /* refill the per-connection buckets */ @@ -2458,18 +2464,20 @@ connection_bucket_refill(int seconds_elapsed, time_t now) { if (connection_speaks_cells(conn)) { or_connection_t *or_conn = TO_OR_CONN(conn); + int orbandwidthrate = or_conn->bandwidthrate / 1000; + int orbandwidthburst = or_conn->bandwidthburst; if (connection_bucket_should_increase(or_conn->read_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->read_bucket, - or_conn->bandwidthrate, - or_conn->bandwidthburst, - seconds_elapsed, + orbandwidthrate, + orbandwidthburst, + milliseconds_elapsed, "or_conn->read_bucket"); } if (connection_bucket_should_increase(or_conn->write_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->write_bucket, - or_conn->bandwidthrate, - or_conn->bandwidthburst, - seconds_elapsed, + orbandwidthrate, + orbandwidthburst, + milliseconds_elapsed, "or_conn->write_bucket"); } } diff --git a/src/or/main.c b/src/or/main.c index ad5558e..1a95e31 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -91,10 +91,10 @@ static int stats_prev_global_read_bucket; /** What was the write bucket before the last second_elapsed_callback() call? * (used to determine how many bytes we've written). */ static int stats_prev_global_write_bucket; -#else +#endif + static uint64_t stats_prev_n_read = 0; static uint64_t stats_prev_n_written = 0; -#endif /* XXX we might want to keep stats about global_relayed_*_bucket too. Or not.*/ /** How many bytes have we read since we started the process? */ @@ -1507,9 +1507,6 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) size_t bytes_written; size_t bytes_read; int seconds_elapsed; -#ifdef USE_BUFFEREVENTS - uint64_t cur_read,cur_written; -#endif const or_options_t *options = get_options(); (void)timer; (void)arg; @@ -1523,30 +1520,28 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) /* the second has rolled over. check more stuff. */ seconds_elapsed = current_second ? (int)(now - current_second) : 0; #ifdef USE_BUFFEREVENTS - connection_get_rate_limit_totals(&cur_read, &cur_written); - bytes_written = (size_t)(cur_written - stats_prev_n_written); - bytes_read = (size_t)(cur_read - stats_prev_n_read); + { + uint64_t cur_read,cur_written; + connection_get_rate_limit_totals(&cur_read, &cur_written); + bytes_written = (size_t)(cur_written - stats_prev_n_written); + bytes_read = (size_t)(cur_read - stats_prev_n_read); + stats_n_bytes_read += bytes_read; + stats_n_bytes_written += bytes_written; + if (accounting_is_enabled(options) && seconds_elapsed >= 0) + accounting_add_bytes(bytes_read, bytes_written, seconds_elapsed); + stats_prev_n_written = cur_written; + stats_prev_n_read = cur_read; + } #else - bytes_written = stats_prev_global_write_bucket - global_write_bucket; - bytes_read = stats_prev_global_read_bucket - global_read_bucket; + bytes_read = (size_t)(stats_n_bytes_read - stats_prev_n_read); + bytes_written = (size_t)(stats_n_bytes_written - stats_prev_n_written); + stats_prev_n_read = stats_n_bytes_read; + stats_prev_n_written = stats_n_bytes_written; #endif - stats_n_bytes_read += bytes_read; - stats_n_bytes_written += bytes_written; - if (accounting_is_enabled(options) && seconds_elapsed >= 0) - accounting_add_bytes(bytes_read, bytes_written, seconds_elapsed); + control_event_bandwidth_used((uint32_t)bytes_read,(uint32_t)bytes_written); control_event_stream_bandwidth_used(); - if (seconds_elapsed > 0) - connection_bucket_refill(seconds_elapsed, now); -#ifdef USE_BUFFEREVENTS - stats_prev_n_written = cur_written; - stats_prev_n_read = cur_read; -#else - stats_prev_global_read_bucket = global_read_bucket; - stats_prev_global_write_bucket = global_write_bucket; -#endif - if (server_mode(options) && !we_are_hibernating() && seconds_elapsed > 0 && @@ -1594,6 +1589,57 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) current_second = now; /* remember which second it is, for next time */ } +#ifndef USE_BUFFEREVENTS +/** Timer: used to invoke refill_callback(). */ +static periodic_timer_t *refill_timer = NULL; + +/** Libevent callback: invoked periodically to refill token buckets + * and count r/w bytes. It is only used when bufferevents are disabled. */ +static void +refill_callback(periodic_timer_t *timer, void *arg) +{ + static struct timeval current_millisecond; + struct timeval now; + + size_t bytes_written; + size_t bytes_read; + int milliseconds_elapsed = 0; + int seconds_rolled_over = 0; + + const or_options_t *options = get_options(); + + (void)timer; + (void)arg; + + tor_gettimeofday(&now); + + /* If this is our first time, no time has passed. */ + if (current_millisecond.tv_sec) { + long mdiff = tv_mdiff(&current_millisecond, &now); + if (mdiff > INT_MAX) + mdiff = INT_MAX; + milliseconds_elapsed = (int)mdiff; + seconds_rolled_over = (int)(now.tv_sec - current_millisecond.tv_sec); + } + + bytes_written = stats_prev_global_write_bucket - global_write_bucket; + bytes_read = stats_prev_global_read_bucket - global_read_bucket; + + stats_n_bytes_read += bytes_read; + stats_n_bytes_written += bytes_written; + if (accounting_is_enabled(options) && milliseconds_elapsed >= 0) + accounting_add_bytes(bytes_read, bytes_written, seconds_rolled_over); + + if (milliseconds_elapsed > 0) + connection_bucket_refill(milliseconds_elapsed, now.tv_sec); + + stats_prev_global_read_bucket = global_read_bucket; + stats_prev_global_write_bucket = global_write_bucket; + + current_millisecond = now; /* remember what time it is, for next time */ +} +#endif + #ifndef MS_WINDOWS /** Called when a possibly ignorable libevent error occurs; ensures that we * don't get into an infinite loop by ignoring too many errors from @@ -1791,6 +1837,22 @@ do_main_loop(void) tor_assert(second_timer); } +#ifndef USE_BUFFEREVENTS + if (!refill_timer) { + struct timeval refill_interval; + int msecs = get_options()->TokenBucketRefillInterval; + + refill_interval.tv_sec = msecs/1000; + refill_interval.tv_usec = (msecs%1000)*1000; + + refill_timer = periodic_timer_new(tor_libevent_get_base(), + &refill_interval, + refill_callback, + NULL); + tor_assert(refill_timer); + } +#endif + for (;;) { if (nt_service_is_stopping()) return 0; diff --git a/src/or/or.h b/src/or/or.h index 56f34fc..37eea1c 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -3107,6 +3107,9 @@ typedef struct { * log whether it was DNS-leaking or not? */ int HardwareAccel; /**< Boolean: Should we enable OpenSSL hardware * acceleration where available? */ + /** Token Bucket Refill resolution in milliseconds (ignored when + * bufferevents are enabled) */ + int TokenBucketRefillInterval; char *AccelName; /**< Optional hardware acceleration engine name. */ char *AccelDir; /**< Optional hardware acceleration engine search dir. */ int UseEntryGuards; /**< Boolean: Do we try to enter from a smallish number

1 0

[tor/master] Move around check for TokenBucketRefillInterval; disallow changes to it
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 0721abbf1b89602d4be4dd97a7bd3a5849d6ecae Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Sep 7 21:02:49 2011 -0400 Move around check for TokenBucketRefillInterval; disallow changes to it --- src/or/config.c | 18 +++++++++++------- 1 files changed, 11 insertions(+), 7 deletions(-) diff --git a/src/or/config.c b/src/or/config.c index b04cee4..5a7a590 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -1383,13 +1383,6 @@ options_act(const or_options_t *old_options) if (accounting_is_enabled(options)) configure_accounting(time(NULL)); - if (options->TokenBucketRefillInterval < 0 - || options->TokenBucketRefillInterval > 1000) { - log_warn(LD_CONFIG, "Token bucket refill interval must be in the range " - "of [0:1000]"); - return -1; - } - #ifdef USE_BUFFEREVENTS /* If we're using the bufferevents implementation and our rate limits * changed, we need to tell the rate-limiting system about it. */ @@ -3173,6 +3166,11 @@ options_validate(or_options_t *old_options, or_options_t *options, REJECT("TransPort and TransListenAddress are disabled in this build."); #endif + if (options->TokenBucketRefillInterval <= 0 + || options->TokenBucketRefillInterval > 1000) { + REJECT("TokenBucketRefillInterval must be between 1 and 1000 inclusive."); + } + if (options->AccountingMax && (is_listening_on_low_port(options->ORPort, options->ORListenAddress) || is_listening_on_low_port(options->DirPort, options->DirListenAddress))) @@ -3975,6 +3973,12 @@ options_transition_allowed(const or_options_t *old, return -1; } + if (old->TokenBucketRefillInterval != new_val->TokenBucketRefillInterval) { + *msg = tor_strdup("While Tor is running, changing TokenBucketRefill" + "Interval is not allowed"); + return -1; + } + if (old->DisableIOCP != new_val->DisableIOCP) { *msg = tor_strdup("While Tor is running, changing DisableIOCP " "is not allowed.");

1 0

[tor/master] Make documentation for TokenBucketRefillInterval match its behavior
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 1e611846280b3198445e4e6d8fb56c426e452b7b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Sep 7 21:03:08 2011 -0400 Make documentation for TokenBucketRefillInterval match its behavior --- doc/tor.1.txt | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 05f52f8..15f5a21 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -737,13 +737,13 @@ The following options are useful only for clients (that is, if unattached waiting for an appropriate circuit, before we fail it. (Default: 2 minutes.) -**TokenBucketRefillInterval** __NUM__:: +**TokenBucketRefillInterval** __NUM__ [**msec**|**second**]:: Set the refill interval of Tor's token bucket to NUM milliseconds. - NUM must be positive and either a divisor or a multiple of 1 second. - Note that this option retains the configured bandwidth limits and refills - token buckets only in ratio to the interval. This option will be ignored - when Tor was built with Libevent's bufferevents enabled. (Default: 1 second) - + NUM must be between 1 and 1000, inclusive. Note that the configured + bandwidth limits are still expressed in bytes per second: this + option only affects the frequency with which Tor checks to see whether + previously exhausted connections may read again. This option is + if Tor was built with Libevent's bufferevents enabled. (Default: 10 msec.) **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent

1 0

[tor/master] Refactor connection_bucket_refill(_helper) to avoid roundoff
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 052b95e2f196211c97b33ce8c68960e1c2561ea0 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Sep 7 21:22:02 2011 -0400 Refactor connection_bucket_refill(_helper) to avoid roundoff We were doing "divide bandwidth by 1000, then multiply by msec", but that would lose accuracy: instead of getting your full bandwidth, you'd lose up to 999 bytes per sec. (Not a big deal, but every byte helps.) Instead, do the multiply first, then the division. This can easily overflow a 32-bit value, so make sure to do it as a 64-bit operation. --- src/or/connection.c | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 42d7e2f..cb93a81 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2389,7 +2389,7 @@ connection_bucket_init(void) } /** Refill a single bucket called name with bandwidth rate - * per millisecond rate and bandwidth burst per refill interval + * per second rate and bandwidth burst per refill interval * burst, assuming that milliseconds_elapsed milliseconds * have passed since the last call. */ static void @@ -2398,13 +2398,13 @@ connection_bucket_refill_helper(int *bucket, int rate, int burst, const char *name) { int starting_bucket = *bucket; - if (starting_bucket < burst && milliseconds_elapsed) { - if (((burst - starting_bucket)/milliseconds_elapsed) < rate) { + if (starting_bucket < burst && milliseconds_elapsed > 0) { + int64_t incr = (((int64_t)rate) * milliseconds_elapsed) / 1000; + if ((burst - starting_bucket) < incr) { *bucket = burst; /* We would overflow the bucket; just set it to * the maximum. */ } else { - int incr = rate*milliseconds_elapsed; - *bucket += incr; + *bucket += (int)incr; if (*bucket > burst || *bucket < starting_bucket) { /* If we overflow the burst, or underflow our starting bucket, * cap the bucket value to burst. */ @@ -2425,11 +2425,11 @@ connection_bucket_refill(int milliseconds_elapsed, time_t now) smartlist_t *conns = get_connection_array(); int bandwidthrate, bandwidthburst, relayrate, relayburst; - bandwidthrate = (int)options->BandwidthRate / 1000; + bandwidthrate = (int)options->BandwidthRate; bandwidthburst = (int)options->BandwidthBurst; if (options->RelayBandwidthRate) { - relayrate = (int)options->RelayBandwidthRate / 1000; + relayrate = (int)options->RelayBandwidthRate; relayburst = (int)options->RelayBandwidthBurst; } else { relayrate = bandwidthrate; @@ -2464,7 +2464,7 @@ connection_bucket_refill(int milliseconds_elapsed, time_t now) { if (connection_speaks_cells(conn)) { or_connection_t *or_conn = TO_OR_CONN(conn); - int orbandwidthrate = or_conn->bandwidthrate / 1000; + int orbandwidthrate = or_conn->bandwidthrate; int orbandwidthburst = or_conn->bandwidthburst; if (connection_bucket_should_increase(or_conn->read_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->read_bucket,

1 0

[tor/master] Make bufferevents work with TokenBucketRefillInterval
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 41dfc4c19c4e0ee37d092fa0ed7faf349e6ab467 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Sep 7 22:00:48 2011 -0400 Make bufferevents work with TokenBucketRefillInterval --- doc/tor.1.txt | 3 +-- src/common/compat_libevent.c | 34 +++++++++++++++++++++------------- src/common/compat_libevent.h | 2 +- src/or/config.c | 1 + src/or/connection.c | 5 ++++- src/or/connection_or.c | 7 ++++++- 6 files changed, 34 insertions(+), 18 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 15f5a21..4ae721e 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -742,8 +742,7 @@ The following options are useful only for clients (that is, if NUM must be between 1 and 1000, inclusive. Note that the configured bandwidth limits are still expressed in bytes per second: this option only affects the frequency with which Tor checks to see whether - previously exhausted connections may read again. This option is - if Tor was built with Libevent's bufferevents enabled. (Default: 10 msec.) + previously exhausted connections may read again. (Default: 10 msec.) **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c index beae950..3201738 100644 --- a/src/common/compat_libevent.c +++ b/src/common/compat_libevent.c @@ -169,6 +169,7 @@ struct event_base *the_event_base = NULL; #ifdef USE_BUFFEREVENTS static int using_iocp_bufferevents = 0; +static void tor_libevent_set_tick_timeout(int msec_per_tick); int tor_libevent_using_iocp_bufferevents(void) @@ -236,6 +237,10 @@ tor_libevent_initialize(tor_libevent_cfg *torcfg) "You have a *VERY* old version of libevent. It is likely to be buggy; " "please build Tor with a more recent version."); #endif + +#ifdef USE_BUFFEREVENTS + tor_libevent_set_tick_timeout(torcfg->msec_per_tick); +#endif } /** Return the current Libevent event base that we're set up to use. */ @@ -598,26 +603,29 @@ static const struct timeval *one_tick = NULL; /** * Return a special timeout to be passed whenever libevent's O(1) timeout * implementation should be used. Only use this when the timer is supposed - * to fire after 1 / TOR_LIBEVENT_TICKS_PER_SECOND seconds have passed. + * to fire after msec_per_tick ticks have elapsed. */ const struct timeval * tor_libevent_get_one_tick_timeout(void) { - if (PREDICT_UNLIKELY(one_tick == NULL)) { - struct event_base *base = tor_libevent_get_base(); - struct timeval tv; - if (TOR_LIBEVENT_TICKS_PER_SECOND == 1) { - tv.tv_sec = 1; - tv.tv_usec = 0; - } else { - tv.tv_sec = 0; - tv.tv_usec = 1000000 / TOR_LIBEVENT_TICKS_PER_SECOND; - } - one_tick = event_base_init_common_timeout(base, &tv); - } + tor_assert(one_tick); return one_tick; } +/** Initialize the common timeout that we'll use to refill the buckets every + * time a tick elapses. */ +static void +tor_libevent_set_tick_timeout(int msec_per_tick) +{ + struct event_base *base = tor_libevent_get_base(); + struct timeval tv; + + tor_assert(! one_tick); + tv.tv_sec = msec_per_tick / 1000; + tv.tv_usec = (msec_per_tick % 1000) * 1000; + one_tick = event_base_init_common_timeout(base, &tv); +} + static struct bufferevent * tor_get_root_bufferevent(struct bufferevent *bev) { diff --git a/src/common/compat_libevent.h b/src/common/compat_libevent.h index 15b0fc2..0247297 100644 --- a/src/common/compat_libevent.h +++ b/src/common/compat_libevent.h @@ -62,6 +62,7 @@ int tor_event_base_loopexit(struct event_base *base, struct timeval *tv); typedef struct tor_libevent_cfg { int disable_iocp; int num_cpus; + int msec_per_tick; } tor_libevent_cfg; void tor_libevent_initialize(tor_libevent_cfg *cfg); @@ -73,7 +74,6 @@ void tor_check_libevent_header_compatibility(void); const char *tor_libevent_get_version_str(void); #ifdef USE_BUFFEREVENTS -#define TOR_LIBEVENT_TICKS_PER_SECOND 3 const struct timeval *tor_libevent_get_one_tick_timeout(void); int tor_libevent_using_iocp_bufferevents(void); int tor_set_bufferevent_rate_limit(struct bufferevent *bev, diff --git a/src/or/config.c b/src/or/config.c index 5a7a590..e22f539 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -5645,6 +5645,7 @@ init_libevent(const or_options_t *options) memset(&cfg, 0, sizeof(cfg)); cfg.disable_iocp = options->DisableIOCP; cfg.num_cpus = get_num_cpus(options); + cfg.msec_per_tick = options->TokenBucketRefillInterval; tor_libevent_initialize(&cfg); diff --git a/src/or/connection.c b/src/or/connection.c index cb93a81..9dd9297 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2561,7 +2561,10 @@ connection_bucket_init(void) burst = options->BandwidthBurst; } - rate /= TOR_LIBEVENT_TICKS_PER_SECOND; + /* This can't overflow, since TokenBucketRefillInterval <= 1000, + * and rate started out less than INT32_MAX. */ + rate = (rate * options->TokenBucketRefillInterval) / 1000; + bucket_cfg = ev_token_bucket_cfg_new((uint32_t)rate, (uint32_t)burst, (uint32_t)rate, (uint32_t)burst, tick); diff --git a/src/or/connection_or.c b/src/or/connection_or.c index a75444e..29f0f8d 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -580,7 +580,12 @@ connection_or_update_token_buckets_helper(or_connection_t *conn, int reset, { const struct timeval *tick = tor_libevent_get_one_tick_timeout(); struct ev_token_bucket_cfg *cfg, *old_cfg; - int rate_per_tick = rate / TOR_LIBEVENT_TICKS_PER_SECOND; + int64_t rate64 = (((int64_t)rate) * options->TokenBucketRefillInterval) + / 1000; + /* This can't overflow, since TokenBucketRefillInterval <= 1000, + * and rate started out less than INT_MAX. */ + int rate_per_tick = (int) rate64; + cfg = ev_token_bucket_cfg_new(rate_per_tick, burst, rate_per_tick, burst, tick); old_cfg = conn->bucket_cfg;

1 0

[tor/master] Merge branch 'feature3630-rebased'
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit 5a8dcca8f7f8637792d63a9aec1408da26c7cbe6 Merge: 40288e1 fee094a Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Sep 22 15:54:40 2011 -0400 Merge branch 'feature3630-rebased' changes/feature3630 | 8 +++ doc/tor.1.txt | 7 +++ src/common/compat_libevent.c | 34 ++++++++----- src/common/compat_libevent.h | 2 +- src/or/config.c | 13 +++++ src/or/connection.c | 71 ++++++++++++++++----------- src/or/connection_or.c | 7 ++- src/or/main.c | 110 ++++++++++++++++++++++++++++++++--------- src/or/or.h | 2 + 9 files changed, 185 insertions(+), 69 deletions(-)

1 0

[tor/master] Fix issues in 3630 patch noted by Karsten
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit fee094afcdd8589f29be82190acd73dbb60ee08b Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Sep 22 15:07:01 2011 -0400 Fix issues in 3630 patch noted by Karsten --- src/or/connection.c | 10 +++++----- src/or/or.h | 3 +-- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 9dd9297..ad63a9d 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2388,10 +2388,10 @@ connection_bucket_init(void) } } -/** Refill a single bucket called name with bandwidth rate - * per second rate and bandwidth burst per refill interval - * burst, assuming that milliseconds_elapsed milliseconds - * have passed since the last call. */ +/** Refill a single bucket called name with bandwidth rate per + * second rate and bandwidth burst burst, assuming that + * milliseconds_elapsed milliseconds have passed since the last + * call. */ static void connection_bucket_refill_helper(int *bucket, int rate, int burst, int milliseconds_elapsed, @@ -2449,7 +2449,7 @@ connection_bucket_refill(int milliseconds_elapsed, time_t now) connection_bucket_refill_helper(&global_write_bucket, bandwidthrate, bandwidthburst, milliseconds_elapsed, - "global_read_bucket"); + "global_write_bucket"); connection_bucket_refill_helper(&global_relayed_read_bucket, relayrate, relayburst, milliseconds_elapsed, diff --git a/src/or/or.h b/src/or/or.h index 37eea1c..b05c767 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -3107,8 +3107,7 @@ typedef struct { * log whether it was DNS-leaking or not? */ int HardwareAccel; /**< Boolean: Should we enable OpenSSL hardware * acceleration where available? */ - /** Token Bucket Refill resolution in milliseconds (ignored when - * bufferevents are enabled) */ + /** Token Bucket Refill resolution in milliseconds. */ int TokenBucketRefillInterval; char *AccelName; /**< Optional hardware acceleration engine name. */ char *AccelDir; /**< Optional hardware acceleration engine search dir. */

1 0

[tor/master] Changes file for feature3630
by nickm＠torproject.org 23 Sep '11

23 Sep '11

commit a8297a301e118a9e77fec51fe8d9d4d0f3350980 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Sep 7 22:10:49 2011 -0400 Changes file for feature3630 --- changes/feature3630 | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/changes/feature3630 b/changes/feature3630 new file mode 100644 index 0000000..dab46c8 --- /dev/null +++ b/changes/feature3630 @@ -0,0 +1,8 @@ + o Major features (networking): + - Add a new TokenBucketRefillInterval option to refill token buckets + more frequently than once per second. This should improve network + performance, alleviate queueing problems, and make traffic less + bursty. Implements proposal 183; closes ticket 3630. Design by + Florian Tschorsch and Björn Scheuermann; implementation by + Florian Tschorsch. +

1 0