- tor-commits - lists.torproject.org

[tor/master] One more fix for bug 5049.
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 5b0977df3153a82c51fadbda68235805b5a5cd45 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jun 13 16:42:51 2012 -0400 One more fix for bug 5049. --- src/or/control.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/src/or/control.c b/src/or/control.c index dfa7d36..9465f89 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -4109,11 +4109,13 @@ control_event_buildtimeout_set(const circuit_build_times_t *cbt, buildtimeout_set_event_t type) { const char *type_string = NULL; - double qnt = circuit_build_times_quantile_cutoff(); + double qnt; if (!control_event_is_interesting(EVENT_BUILDTIMEOUT_SET)) return 0; + qnt = circuit_build_times_quantile_cutoff(); + switch (type) { case BUILDTIMEOUT_SET_EVENT_COMPUTED: type_string = "COMPUTED";

1 0

[tor/master] Satisfy make check-spaces
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit d98590d3b73be541cc6b3b59477bc6600212eb04 Author: Andrea Shepard <andrea(a)persephoneslair.org> Date: Tue Jun 12 19:38:28 2012 -0700 Satisfy make check-spaces --- src/or/circuitbuild.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 4f46022..fa99a49 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -593,7 +593,9 @@ circuit_build_times_init(circuit_build_times_t *cbt) * on or something. */ -void circuit_build_times_free_timeouts(circuit_build_times_t *cbt) { +void +circuit_build_times_free_timeouts(circuit_build_times_t *cbt) +{ if (!cbt) return; if (cbt->liveness.timeouts_after_firsthop) {

1 0

[tor/master] Merge branch 'trac-5049-squashed'
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit e3a6e01fe196fa1c1ae707065d515718795bc1d1 Merge: 7826eef 5b0977d Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jun 13 16:55:39 2012 -0400 Merge branch 'trac-5049-squashed' changes/bug5049 | 4 + src/or/circuitbuild.c | 293 +++++++++++++++++++++++++++++++++++++++---------- src/or/circuitbuild.h | 1 + src/or/circuituse.c | 4 +- src/or/control.c | 4 +- 5 files changed, 245 insertions(+), 61 deletions(-)

1 0

[tor/master] Only use -Qunused-arguments when building with clang. fix on 5210 fix.
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 7826eef93e8a3b1f90198596bfd19b589f58cc88 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jun 13 16:37:23 2012 -0400 Only use -Qunused-arguments when building with clang. fix on 5210 fix. --- configure.in | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diff --git a/configure.in b/configure.in index 18869a1..e690cb2 100644 --- a/configure.in +++ b/configure.in @@ -563,9 +563,16 @@ dnl use it with a build of a library. all_ldflags_for_check="$TOR_LDFLAGS_zlib $TOR_LDFLAGS_openssl $TOR_LDFLAGS_libevent" all_libs_for_check="$TOR_ZLIB_LIBS $TOR_LIB_MATH $TOR_LIBEVENT_LIBS $TOR_OPENSSL_LIBS $TOR_LIB_WS32 $TOR_LIB_GDI" +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ +#if !defined(__clang__) +#error +#endif])], have_clang=yes, have_clang=no) + if test x$enable_gcc_hardening != xno; then CFLAGS="$CFLAGS -D_FORTIFY_SOURCE=2" - TOR_CHECK_CFLAGS(-Qunused-arguments) + if test x$have_clang = xyes; then + TOR_CHECK_CFLAGS(-Qunused-arguments) + fi TOR_CHECK_CFLAGS(-fstack-protector-all) TOR_CHECK_CFLAGS(-Wstack-protector) TOR_CHECK_CFLAGS(-fwrapv) @@ -1243,11 +1250,6 @@ if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xy #error #endif])], have_gcc43=yes, have_gcc43=no) -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ -#if !defined(__clang__) -#error -#endif])], have_clang=yes, have_clang=no) - save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Wshorten-64-to-32" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], have_shorten64_flag=yes,

1 0

[tor/master] Merge branch 'bug5263_023'
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 54ef039ba527acba4cbcde158b17674e91ede904 Merge: 37ef4f1 9c58873 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jun 13 16:23:16 2012 -0400 Merge branch 'bug5263_023' changes/bug5263 | 5 +++++ src/or/main.c | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 0 deletions(-)

1 0

[tor/master] Fix busy Libevent loops (infinite loops in Shadow)
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 03b48352c6b8416faee50e769bf226b06958e253 Author: Rob G. Jansen <jansen(a)cs.umn.edu> Date: Tue Feb 28 18:19:49 2012 -0500 Fix busy Libevent loops (infinite loops in Shadow) There is a bug causing busy loops in Libevent and infinite loops in the Shadow simulator. A connection that is marked for close, wants to flush, is held open to flush, but is rate limited (the token bucket is empty) triggers the bug. This commit fixes the bug. Details are below. This currently happens on read and write callbacks when the active socket is marked for close. In this case, Tor doesn't actually try to complete the read or write (it returns from those methods when marked), but instead tries to clear the connection with conn_close_if_marked(). Tor will not close a marked connection that contains data: it must be flushed first. The bug occurs when this flush operation on the marked connection can not occur because the connection is rate-limited (its write token bucket is empty). The fix is to detect when rate limiting is preventing a marked connection from properly flushing. In this case, it should be flagged as read/write_blocked_on_bandwidth and the read/write events de-registered from Libevent. When the token bucket gets refilled, it will check the associated read/write_blocked_on_bandwidth flag, and add the read/write event back to Libevent, which will cause it to fire. This time, it will be properly flushed and closed. The reason that both read and write events are both de-registered when the marked connection can not flush is because both result in the same behavior. Both read/write events on marked connections will never again do any actual reads/writes, and are only useful to trigger the flush and close the connection. By setting the associated read/write_blocked_on_bandwidth flag, we ensure that the event will get added back to Libevent, properly flushed, and closed. Why is this important? Every Shadow event occurs at a discrete time instant. If Tor does not properly deregister Libevent events that fire but result in Tor essentially doing nothing, Libevent will repeatedly fire the event. In Shadow this means infinite loop, outside of Shadow this means wasted CPU cycles. --- src/or/main.c | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diff --git a/src/or/main.c b/src/or/main.c index 163dd5d..a5ad1d9 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -848,6 +848,20 @@ conn_close_if_marked(int i) "Holding conn (fd %d) open for more flushing.", (int)conn->s)); conn->timestamp_lastwritten = now; /* reset so we can flush more */ + } else if (sz == 0) { /* retval is also 0 */ + /* Connection must flush before closing, but it's being rate-limited. + Let's remove from Libevent, and mark it as blocked on bandwidth so it + will be re-added on next token bucket refill. Prevents busy Libevent + loops where we keep ending up here and returning 0 until we are no + longer blocked on bandwidth. */ + if (connection_is_reading(conn)) { + conn->read_blocked_on_bw = 1; + connection_stop_reading(conn); + } + if (connection_is_writing(conn)) { + conn->write_blocked_on_bw = 1; + connection_stop_writing(conn); + } } return 0; }

1 0

[tor/master] Add changes file for bug5263
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 9c588730598c8da6c8ad8f0829e443235699a37d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jun 5 12:11:08 2012 -0400 Add changes file for bug5263 --- changes/bug5263 | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/changes/bug5263 b/changes/bug5263 new file mode 100644 index 0000000..25793f1 --- /dev/null +++ b/changes/bug5263 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Disable writing on marked-for-close connections when they are + blocked on bandwidth, to prevent busy-looping in Libevent. Fixes + bug 5263; bugfix on 0.0.2pre13, where we first added a special + case for flushing marked connections.

1 0

[tor/master] Add rate-limited log message to bug5263 fix
by nickm＠torproject.org 13 Jun '12

13 Jun '12

commit 9282c88998f80a027b5260472bf21125693a15bc Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue May 15 10:22:17 2012 -0400 Add rate-limited log message to bug5263 fix Initially I said, "I claim that we shouldn't be reading and marked; let's see if I'm right." But Rob finds that it does. --- src/or/main.c | 18 ++++++++++++++---- 1 files changed, 14 insertions(+), 4 deletions(-) diff --git a/src/or/main.c b/src/or/main.c index a5ad1d9..3e913d9 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -854,14 +854,24 @@ conn_close_if_marked(int i) will be re-added on next token bucket refill. Prevents busy Libevent loops where we keep ending up here and returning 0 until we are no longer blocked on bandwidth. */ - if (connection_is_reading(conn)) { - conn->read_blocked_on_bw = 1; - connection_stop_reading(conn); - } if (connection_is_writing(conn)) { conn->write_blocked_on_bw = 1; connection_stop_writing(conn); } + if (connection_is_reading(conn)) { +#define MARKED_READING_RATE 180 + static ratelim_t marked_read_lim = RATELIM_INIT(MARKED_READING_RATE); + char *m; + if ((m = rate_limit_log(&marked_read_lim, now))) { + log_warn(LD_BUG, "Marked connection (fd %d, type %s, state %s) " + "is still reading; that shouldn't happen.%s", + (int)conn->s, conn_type_to_string(conn->type), + conn_state_to_string(conn->type, conn->state), m); + tor_free(m); + } + conn->read_blocked_on_bw = 1; + connection_stop_reading(conn); + } } return 0; }

1 0

[ooni-probe/master] Add txscapy to the list of used libraries and update makefile
by art＠torproject.org 13 Jun '12

13 Jun '12

commit 54c7193ffaf82473ca337b6540652002ca43adee Author: Arturo Filastò <art(a)baculo.org> Date: Wed Jun 13 19:28:16 2012 +0200 Add txscapy to the list of used libraries and update makefile --- ooni/lib/Makefile | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/ooni/lib/Makefile b/ooni/lib/Makefile index dfb2449..37abe05 100644 --- a/ooni/lib/Makefile +++ b/ooni/lib/Makefile @@ -1,4 +1,4 @@ -all: txtorcon txtraceroute +all: txtorcon txtraceroute txscapy txtraceroute: echo "Processing dependency txtraceroute..." @@ -12,3 +12,9 @@ txtorcon: mv txtorcon.git/txtorcon txtorcon rm -rf txtorcon.git +txscapy: + echo "Processing dependency txscapy" + git clone https://github.com/hellais/txscapy.git txscapy.git + mv txscapy.git/txscapy.py txscapy.py + rm -rf txscapy.git +

1 0

[ooni-probe/master] Remove dead code.
by art＠torproject.org 13 Jun '12

13 Jun '12

commit a6eebf219c45431721071baa2b295bf1bd1c77fd Author: Arturo Filastò <hellais(a)torproject.org> Date: Sat Jun 2 21:10:15 2012 +0200 Remove dead code. --- ooni/assets/tcpscan.txt | 1 + ooni/plugoo/__init__.py | 33 --------------------------------- 2 files changed, 1 insertions(+), 33 deletions(-) diff --git a/ooni/assets/tcpscan.txt b/ooni/assets/tcpscan.txt index f67dce3..4204d77 100644 --- a/ooni/assets/tcpscan.txt +++ b/ooni/assets/tcpscan.txt @@ -1 +1,2 @@ 127.0.0.1:9050 +127.0.0.1:6666 diff --git a/ooni/plugoo/__init__.py b/ooni/plugoo/__init__.py index 650cea0..f3a49e9 100644 --- a/ooni/plugoo/__init__.py +++ b/ooni/plugoo/__init__.py @@ -17,15 +17,8 @@ import os from datetime import datetime import yaml -try: - import socks -except: - "Error SocksiPy is not installed!" -import socket - import logging import itertools -import gevent def gen_headers(self, options="common"): """ @@ -52,29 +45,3 @@ def gen_headers(self, options="common"): return False return headers - - -class torify(object): - """This is the torify decorator. It should be used to - decorate functions that should use to for connecting to - the interwebz. The suggary syntax is the following: - @torify([urllib2]) - def myfunction(): - f = urllib2.urlopen('https://torproject.org/') - remember to set the proxyaddress in the config file. - """ - def __init__(self, f): - print f - self.f = f - - def __get__(self, instance, owner): - self.modules = instance.modules - def decorator(*args): - print instance.config.main.proxyaddress - host, port = instance.config.main.proxyaddress.split(":") - socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, host, int(port)) - # Wrap the modules into socks - for module in self.modules: - socks.wrapmodule(module) - return self.f(instance, *args) - return decorator diff --git a/oonib/oonibackend.py b/oonib/oonibackend.py old mode 100644 new mode 100755

1 0