October 2016 - tor-commits - lists.torproject.org

[tor/master] Module documentation for circuitmux_ewma.c
by nickm＠torproject.org 18 Oct '16

18 Oct '16

commit 84829e336c3f72e4926e90d6f2e58eed0a538bce Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 18 11:28:40 2016 -0400 Module documentation for circuitmux_ewma.c --- src/or/circuitmux_ewma.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/src/or/circuitmux_ewma.c b/src/or/circuitmux_ewma.c index 13836cd..5c2ebde 100644 --- a/src/or/circuitmux_ewma.c +++ b/src/or/circuitmux_ewma.c @@ -4,6 +4,28 @@ /** * \file circuitmux_ewma.c * \brief EWMA circuit selection as a circuitmux_t policy + * + * The "EWMA" in this module stands for the "exponentially weighted moving + * average" of the number of cells sent on each circuit. The goal is to + * prioritize cells on circuits that have been quiet recently, by looking at + * those that have sent few cells over time, prioritizing recent times + * more than older ones. + * + * Specifically, a cell sent at time "now" has weight 1, but a time X ticks + * before now has weight ewma_scale_factor ^ X , where ewma_scale_factor is + * between 0.0 and 1.0. + * + * For efficiency, we do not re-scale these averages every time we send a + * cell: that would be horribly inefficient. Instead, we we keep the cell + * count on all circuits on the same circuitmux scaled relative to a single + * tick. When we add a new cell, we scale its weight depending on the time + * that has elapsed since the tick. We do re-scale the circuits on the + * circuitmux periodically, so that we don't overflow double. + * + * + * This module should be used through the interfaces in circuitmux.c, which it + * implements. + * **/ #define TOR_CIRCUITMUX_EWMA_C_ @@ -28,9 +50,10 @@ /*** Some useful constant #defines ***/ -/*DOCDOC*/ +/** Any halflife smaller than this number of seconds is considered to be + * "disabled". */ #define EPSILON 0.00001 -/*DOCDOC*/ +/** The natural logarithm of 0.5. */ #define LOG_ONEHALF -0.69314718055994529 /*** EWMA structures ***/

1 0

[tor/master] changes file for module docs
by nickm＠torproject.org 18 Oct '16

18 Oct '16

commit 9a3adb07c49a522ffbefb009687c3bd9ee282d59 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 18 19:35:10 2016 -0400 changes file for module docs --- changes/more_module_docs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/changes/more_module_docs b/changes/more_module_docs new file mode 100644 index 0000000..0066ddf --- /dev/null +++ b/changes/more_module_docs @@ -0,0 +1,4 @@ + o Documentation: + - Module-level documentation for several more modules. Closes tickets + 19287 and + 19290.

1 0

[tor/master] Module-level docs for ext_orport and router.c
by nickm＠torproject.org 18 Oct '16

18 Oct '16

commit 54fda6b98ad79d3c2ce50858cea2226675ebfd4c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 18 19:32:49 2016 -0400 Module-level docs for ext_orport and router.c --- src/or/ext_orport.c | 12 +++++++++++- src/or/router.c | 19 +++++++++++++++++-- 2 files changed, 28 insertions(+), 3 deletions(-) diff --git a/src/or/ext_orport.c b/src/or/ext_orport.c index fb7add1..676adfd 100644 --- a/src/or/ext_orport.c +++ b/src/or/ext_orport.c @@ -4,7 +4,17 @@ /** * \file ext_orport.c * \brief Code implementing the Extended ORPort. -*/ + * + * The Extended ORPort interface is used by pluggable transports to + * communicate additional information to a Tor bridge, including + * address information. For more information on this interface, + * see pt-spec.txt in torspec.git. + * + * There is no separate structure for extended ORPort connections; they use + * or_connection_t objects, and share most of their implementation with + * connection_or.c. Once the handshake is done, an extended ORPort connection + * turns into a regular OR connection, using connection_ext_or_transition(). + */ #define EXT_ORPORT_PRIVATE #include "or.h" diff --git a/src/or/router.c b/src/or/router.c index 1ac8351..10498e8 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -37,8 +37,23 @@ /** * \file router.c - * \brief OR functionality, including key maintenance, generating - * and uploading server descriptors, retrying OR connections. + * \brief Miscellaneous relay functionality, including RSA key maintenance, + * generating and uploading server descriptors, picking an address to + * advertise, and so on. + * + * This module handles the job of deciding whether we are a Tor relay, and if + * so what kind. (Mostly through functions like server_mode() that inspect an + * or_options_t, but in some cases based on our own capabilities, such as when + * we are deciding whether to be a directory cache in + * router_has_bandwidth_to_be_dirserver().) + * + * Also in this module are the functions to generate our own routerinfo_t and + * extrainfo_t, and to encode those to signed strings for upload to the + * directory authorities. + * + * This module also handles key maintenance for RSA and Curve25519-ntor keys, + * and for our TLS context. (These functions should eventually move to + * routerkeys.c along with the code that handles Ed25519 keys now.) **/ /************************************************************/

1 0

[tor/master] Document connection_or.c and connection.c at module level
by nickm＠torproject.org 18 Oct '16

18 Oct '16

commit d1b45786b1f91d3a65e88cc99c8021ea9d25a106 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 18 12:25:55 2016 -0400 Document connection_or.c and connection.c at module level --- src/or/connection.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ src/or/connection_or.c | 11 +++++++++++ 2 files changed, 55 insertions(+) diff --git a/src/or/connection.c b/src/or/connection.c index 5ecd1ad..49cb78e 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -8,6 +8,50 @@ * \file connection.c * \brief General high-level functions to handle reading and writing * on connections. + * + * Each connection (ideally) represents a TLS connection, a TCP socket, a unix + * socket, or a UDP socket on which reads and writes can occur. (But see + * connection_edge.c for cases where connections can also represent streams + * that do not have a corresponding socket.) + * + * The module implements the abstract type, connection_t. The subtypes are: + * <ul> + * <li>listener_connection_t, implemented here in connection.c + * <li>dir_connection_t, implemented in directory.c + * <li>or_connection_t, implemented in connection_or.c + * <li>edge_connection_t, implemented in connection_edge.c, along with + * its subtype(s): + * <ul><li>entry_connection_t, also implemented in connection_edge.c + * </ul> + * <li>control_connection_t, implemented in control.c + * </ul> + * + * The base type implemented in this module is responsible for basic + * rate limiting, flow control, and marshalling bytes onto and off of the + * network (either directly or via TLS). + * + * Connections are registered with the main loop with connection_add(). As + * they become able to read or write register the fact with the event main + * loop by calling connection_watch_events(), connection_start_reading(), or + * connection_start_writing(). When they no longer want to read or write, + * they call connection_stop_reading() or connection_start_writing(). + * + * To queue data to be written on a connection, call + * connection_write_to_buf(). When data arrives, the + * connection_process_inbuf() callback is invoked, which dispatches to a + * type-specific function (such as connection_edge_process_inbuf() for + * example). Connection types that need notice of when data has been written + * receive notification via connection_flushed_some() and + * connection_finished_flushing(). These functions all delegate to + * type-specific implementations. + * + * Additionally, beyond the core of connection_t, this module also implements: + * <ul> + * <li>Listeners, which wait for incoming sockets and launch connections + * <li>Outgoing SOCKS proxy support + * <li>Outgoing HTTP proxy support + * <li>An out-of-sockets handler for dealing with socket exhaustion + * </ul> **/ #define CONNECTION_PRIVATE diff --git a/src/or/connection_or.c b/src/or/connection_or.c index 72d8e13..267c32d 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -8,6 +8,17 @@ * \file connection_or.c * \brief Functions to handle OR connections, TLS handshaking, and * cells on the network. + * + * An or_connection_t is a subtype of connection_t (as implemented in + * connection.c) that uses a TLS connection to send and receive cells on the + * Tor network. (By sending and receiving cells connection_or.c, it cooperates + * with channeltls.c to implement a the channel interface of channel.c.) + * + * Every OR connection has an underlying tortls_t object (as implemented in + * tortls.c) which it uses as its TLS stream. It is responsible for + * sending and receiving cells over that TLS. + * + * This module also implements the client side of the v3 Tor link handshake, **/ #include "or.h" #include "buffers.h"

1 0

[torsocks/master] Use xz for dist tarball now
by dgoulet＠torproject.org 18 Oct '16

18 Oct '16

commit d5d7eaab8fb82353079180e71884c37f6e2e882c Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Oct 18 12:39:02 2016 -0400 Use xz for dist tarball now Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- .gitignore | 1 + configure.ac | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 89ed3e9..6d80fe1 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,7 @@ Makefile.in *.loT *.info *.bz2 +*.xz *.tar aclocal.m4 autom4te.cache/ diff --git a/configure.ac b/configure.ac index a77a2de..d5e3200 100644 --- a/configure.ac +++ b/configure.ac @@ -17,7 +17,7 @@ AC_CONFIG_HEADER([include/config.h]) AC_USE_SYSTEM_EXTENSIONS # Automake initialization -AM_INIT_AUTOMAKE([foreign dist-bzip2 no-dist-gzip]) +AM_INIT_AUTOMAKE([foreign dist-xz no-dist-gzip]) # Silent compilation. Easier to spot errors! m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])

1 0

[torsocks/master] Update version to v2.2.0
by dgoulet＠torproject.org 18 Oct '16

18 Oct '16

commit e54d80bc9595beeceac637b03e5c5395c07e62f7 Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Oct 18 12:44:01 2016 -0400 Update version to v2.2.0 Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- ChangeLog | 17 +++++++++++++++++ configure.ac | 2 +- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 0890707..e90ea19 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,20 @@ +2016-10-18 torsocks 2.2.0 + * Use xz for dist tarball now + * Remove TODO as we use the bugtracker for those + * execve: only include xattr.h for Linux + * syscall: sched_getaffinity is only Linux + * close: Prefix debug messages with [close] + * Add check for Apple's System Integrity Protection. + * Quote the non-zero length check of $getcap. + * compat: Fix bad use of defined macro for OS X + * Use AC_USE_SYSTEM_EXTENSIONS to try to use POSIX extensions + * log: Fix whitespace in log.h + * syscall: OS X doesn't support sched_getaffinity() + * Fix memcpy buffer overrun in gethostbyaddr() + * Fix memcpy() buffer overrun in gethostbyname() + * Fix typo: catched -> caught + * syscall: Whitelist sched_getaffinity(2) + 2016-06-21 torsocks 2.2.0-rc1 * Return connection refused on deny by ruleset error * Fix wrong tor_address check for NULL diff --git a/configure.ac b/configure.ac index d5e3200..f5dd992 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ ############################################################################## # Process this file with autoconf to produce a configure script. -AC_INIT([torsocks], [2.2.0-rc1],[dgoulet@torproject.org],[],[https://torproject.org]) +AC_INIT([torsocks], [2.2.0],[dgoulet@torproject.org],[],[https://torproject.org]) AC_CONFIG_AUX_DIR([config]) AC_CANONICAL_TARGET # Get hostname and other information.

1 0

[torsocks/master] Remove TODO as we use the bugtracker for those
by dgoulet＠torproject.org 18 Oct '16

18 Oct '16

commit b92860d5979de1902707b638933d315e3313848f Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Oct 18 12:25:58 2016 -0400 Remove TODO as we use the bugtracker for those Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- TODO | 94 -------------------------------------------------------------------- 1 file changed, 94 deletions(-) diff --git a/TODO b/TODO deleted file mode 100644 index ab593f4..0000000 --- a/TODO +++ /dev/null @@ -1,94 +0,0 @@ -Version 2.0 ------------ -* Support res_* API -* Check recvmmsg() FD passing on Unix socket and for TCP socket, clean exit -* Support the complete list of dangerous syscall numbers with syscall() -* Clean configure.ac - DONE -* Create new updated torsocks shell script - DONE -* Rename torsocks to something more meaningful -* Add tests and a bunch load of them! - -Version 1.0 ------------ - -- is_local is fishy. investigate. -- Resolve race condition where hooks are called before dlsym'd. - DONE -- Need to ensure even locally destined dns requests are caught and handled - appropriately. - Do we need to detect TCP dns requests in connect() to ensure - they are rejected/re-routed? - Where do such requests come from? res_*? - How can we re-route them to Tor? DNSPort? -- kill ICMP as well. - -Version 1.1 ------------ -* What about the isc library calls like res_init and so on? How widely are - they use? Does this present a more elegant way to implement this stuff? - -* It should be possible to make this thread-safe with --enable-threads. - -* Add stdout and stderr messages that enable a parent program (such as TorK) - to monitor torsocks for errors. - -* Merge dsocks so we support BSD (http://monkey.org/~dugsong/dsocks/) - --------------------------------------------------------------------------- - -DONE: - -* In fact, one could modify tsocks further to play nicely with tor. For - example, you could prevent nonlocal UDP traffic from being sent at all, - and so on. - DONE - - - Addressed in connect(), sendto() and sendmsg(). - - Don't need to handle send(), since that is handled in connect(). - - ------------------------------------------------------------------------- -Original TODO: - -- Update FAQ to include information about the install location of the - tsocks library - -- Install the saveme and validateconf binaries, presumably with new names - - -TODO ----- - -* Think about how local names get resolved. Is it important? For example, - if I 'telnet localhost' which using tordns, what should happen? Should - we search through /etc/hosts ourselves or what? - -* What about the isc library calls like res_init and so on? How widely are - they use? Does this present a more elegant way to implement this stuff? - -* Perhaps intercept res_query etc and try to do something appropriate. - -* It should be possible to make this thread-safe with --enable-threads. - -* validateconf needs to be updated to understand (and show information about) - the tordns configuration options. - -* Perhaps intercept reverse name lookups? - DONE - getaddrbyhost (RH) - -* In fact, one could modify tsocks further to play nicely with tor. For - example, you could prevent nonlocal UDP traffic from being sent at all, - and so on. - -The 'tordns' feature for tsocks only works with applications using the -standard c library name lookup mechanisms. - -Some programs roll their own name lookup functions, for example, 'curl' -uses the 'adns' library, not the standard c library name lookup calls. -In this case, you'll still be leaking DNS requests. - -However, it seems that most programs which go to the trouble of implementing -asynchronous DNS requests or doing other tricky things have native SOCKS or -HTTP proxy support. - -Requests for reverse name lookups are not intercepted. - -Finally, the tsocks library is NOT thread-safe, with or without this patch. -You should not 'torify' multithreaded applications.

1 0

[torsocks/master] execve: only include xattr.h for Linux
by dgoulet＠torproject.org 18 Oct '16

18 Oct '16

commit d13977e5a4b5666eaae23deb4ffdfa313da7c078 Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Oct 18 10:38:28 2016 -0400 execve: only include xattr.h for Linux Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- src/lib/execve.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/execve.c b/src/lib/execve.c index ec3d12c..3504b9b 100644 --- a/src/lib/execve.c +++ b/src/lib/execve.c @@ -16,7 +16,6 @@ */ #include <sys/types.h> -#include <sys/xattr.h> #include <sys/stat.h> #include "torsocks.h" @@ -43,6 +42,7 @@ check_cap_suid(const char *filename) /* Capabilities as such are just on Linux. */ #ifdef __linux__ +#include <sys/xattr.h> static const char *sec_cap = "security.capability"; ssize_t len = getxattr(filename, sec_cap, NULL, 0); if (len > 0) {

1 0

[torsocks/master] syscall: sched_getaffinity is only Linux
by dgoulet＠torproject.org 18 Oct '16

18 Oct '16

commit b0174499cf17cf153d76b26dadf6b57e18d4a801 Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Oct 18 10:32:43 2016 -0400 syscall: sched_getaffinity is only Linux Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- src/common/compat.h | 3 --- src/lib/syscall.c | 5 ++--- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/src/common/compat.h b/src/common/compat.h index 0eb4fda..aa1da28 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -194,9 +194,6 @@ void tsocks_once(tsocks_once_t *o, void (*init_routine)(void)); #define TSOCKS_NR_GETPEERNAME SYS_getpeername #define TSOCKS_NR_LISTEN SYS_listen #define TSOCKS_NR_RECVMSG SYS_recvmsg -#if !defined(__darwin__) -#define TSOCKS_NR_SCHED_GETAFFINITY SYS_sched_getaffinity -#endif /* ! __darwin__ */ #endif /* __FreeBSD__, __FreeBSD_kernel__, __darwin__, __NetBSD__ */ diff --git a/src/lib/syscall.c b/src/lib/syscall.c index ff5a4da..789c380 100644 --- a/src/lib/syscall.c +++ b/src/lib/syscall.c @@ -164,7 +164,8 @@ static LIBC_RECVMSG_RET_TYPE handle_recvmsg(va_list args) return tsocks_recvmsg(sockfd, msg, flags); } -#if !defined(__darwin__) +#if defined(__linux__) + /* * Handle sched_getaffinity(2) syscall. * NOTE: ffmpeg is one of the application that needs this one on the @@ -183,9 +184,7 @@ static LIBC_SYSCALL_RET_TYPE handle_sched_getaffinity(va_list args) return tsocks_libc_syscall(TSOCKS_NR_SCHED_GETAFFINITY, pid, cpusetsize, mask); } -#endif /* ! __darwin__ */ -#if defined(__linux__) /* * Handle gettid(2) syscall. */

1 0

[webwml/master] Add my new GPG key to signing-keys.wml
by nickm＠torproject.org 18 Oct '16

18 Oct '16

commit 9cf47c85b3d9568ab39e82b98c6d4f3b81f8c49f Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 18 08:59:37 2016 -0400 Add my new GPG key to signing-keys.wml --- docs/en/signing-keys.wml | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/en/signing-keys.wml b/docs/en/signing-keys.wml index 343f266..4b8e6d4 100644 --- a/docs/en/signing-keys.wml +++ b/docs/en/signing-keys.wml @@ -19,8 +19,9 @@ Nicolas Vigier (0xD0220E4B) and Linus Nordberg (0x23291265) sign the Tor Browser releases.</li> <li>Roger Dingledine (0x28988BF5 and 0x19F78451) or Nick Mathewson - (0x165733EA, or its subkey 0x8D29319A) sign the Tor source code - tarballs.</li> + (0xFE43009C4607B1FB with signing key 0x6AFEE6D49E92B601) + sign the Tor source code tarballs. (Nick's old key was 0x165733EA + with signing key 0x8D29319A; it signed older tarballs.)</li> <li>Tor Project Archive (0x886DDD89) signs the deb.torproject.org repositories and archives.</li> <li>Damian Johnson (0x9ABBEEC6) signs Arm releases.</li> @@ -43,6 +44,15 @@ uid Roger Dingledine <arma(a)torproject.org> sub 4096R/B0E5067D 2015-06-10 [expires: 2016-06-09] + pub 4096R/FE43009C4607B1FB 2016-09-21 [expires: 2019-09-21] + Key fingerprint = 2133 BC60 0AB1 33E1 D826 D173 FE43 009C 4607 B1FB + uid Nick Mathewson <nickm(a)alum.mit.edu> + uid Nick Mathewson <nickm(a)wangafu.net> + uid Nick Mathewson <nickm(a)torproject.org> + uid Nick Mathewson <nickm(a)freehaven.net> + sub 4096R/6AFEE6D49E92B601 2016-09-23 [expires: 2018-09-23] + sub 4096R/91DDED0286AC8BFF 2016-09-23 [expires: 2018-09-23] + pub 3072R/165733EA 2004-07-03 Key fingerprint = B35B F85B F194 89D0 4E28 C33C 2119 4EBB 1657 33EA uid Nick Mathewson <nickm(a)alum.mit.edu>

1 0