[or-cvs] r11244: fixed branch (hopefully) (in tor/branches/114-dist-storage: . doc doc/spec doc/spec/proposals src/common src/or)
kloesing at seul.org
kloesing at seul.org
Tue Aug 21 08:37:14 UTC 2007
Author: kloesing
Date: 2007-08-21 04:37:13 -0400 (Tue, 21 Aug 2007)
New Revision: 11244
Modified:
tor/branches/114-dist-storage/ChangeLog
tor/branches/114-dist-storage/INSTALL
tor/branches/114-dist-storage/autogen.sh
tor/branches/114-dist-storage/configure.in
tor/branches/114-dist-storage/doc/HACKING
tor/branches/114-dist-storage/doc/TODO
tor/branches/114-dist-storage/doc/spec/control-spec.txt
tor/branches/114-dist-storage/doc/spec/dir-spec-v2.txt
tor/branches/114-dist-storage/doc/spec/path-spec.txt
tor/branches/114-dist-storage/doc/spec/proposals/000-index.txt
tor/branches/114-dist-storage/doc/spec/proposals/001-process.txt
tor/branches/114-dist-storage/doc/spec/proposals/106-less-tls-constraint.txt
tor/branches/114-dist-storage/doc/spec/proposals/107-uptime-sanity-checking.txt
tor/branches/114-dist-storage/doc/spec/proposals/108-mtbf-based-stability.txt
tor/branches/114-dist-storage/doc/spec/proposals/109-no-sharing-ips.txt
tor/branches/114-dist-storage/doc/spec/proposals/110-avoid-infinite-circuits.txt
tor/branches/114-dist-storage/doc/spec/proposals/111-local-traffic-priority.txt
tor/branches/114-dist-storage/doc/spec/proposals/112-bring-back-pathlencoinweight.txt
tor/branches/114-dist-storage/doc/spec/proposals/113-fast-authority-interface.txt
tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
tor/branches/114-dist-storage/doc/spec/rend-spec.txt
tor/branches/114-dist-storage/doc/tor-osx-dmg-creation.txt
tor/branches/114-dist-storage/doc/tor-rpm-creation.txt
tor/branches/114-dist-storage/doc/tor.1.in
tor/branches/114-dist-storage/src/common/aes.h
tor/branches/114-dist-storage/src/common/compat.c
tor/branches/114-dist-storage/src/common/compat.h
tor/branches/114-dist-storage/src/common/container.c
tor/branches/114-dist-storage/src/common/container.h
tor/branches/114-dist-storage/src/common/crypto.c
tor/branches/114-dist-storage/src/common/ht.h
tor/branches/114-dist-storage/src/common/log.c
tor/branches/114-dist-storage/src/common/log.h
tor/branches/114-dist-storage/src/common/test.h
tor/branches/114-dist-storage/src/common/torgzip.c
tor/branches/114-dist-storage/src/common/tortls.c
tor/branches/114-dist-storage/src/common/tortls.h
tor/branches/114-dist-storage/src/common/util.c
tor/branches/114-dist-storage/src/common/util.h
tor/branches/114-dist-storage/src/or/Makefile.am
tor/branches/114-dist-storage/src/or/buffers.c
tor/branches/114-dist-storage/src/or/circuitbuild.c
tor/branches/114-dist-storage/src/or/circuituse.c
tor/branches/114-dist-storage/src/or/command.c
tor/branches/114-dist-storage/src/or/config.c
tor/branches/114-dist-storage/src/or/connection.c
tor/branches/114-dist-storage/src/or/connection_edge.c
tor/branches/114-dist-storage/src/or/control.c
tor/branches/114-dist-storage/src/or/directory.c
tor/branches/114-dist-storage/src/or/dirserv.c
tor/branches/114-dist-storage/src/or/dirvote.c
tor/branches/114-dist-storage/src/or/dns.c
tor/branches/114-dist-storage/src/or/dnsserv.c
tor/branches/114-dist-storage/src/or/eventdns.c
tor/branches/114-dist-storage/src/or/eventdns_tor.h
tor/branches/114-dist-storage/src/or/hibernate.c
tor/branches/114-dist-storage/src/or/main.c
tor/branches/114-dist-storage/src/or/or.h
tor/branches/114-dist-storage/src/or/policies.c
tor/branches/114-dist-storage/src/or/relay.c
tor/branches/114-dist-storage/src/or/rendclient.c
tor/branches/114-dist-storage/src/or/rendcommon.c
tor/branches/114-dist-storage/src/or/rendservice.c
tor/branches/114-dist-storage/src/or/router.c
tor/branches/114-dist-storage/src/or/routerlist.c
tor/branches/114-dist-storage/src/or/routerparse.c
tor/branches/114-dist-storage/src/or/test.c
Log:
fixed branch (hopefully)
/home/or/svnrepo/hooks/commit-email.pl: `/usr/bin/svnlook diff /home/or/svnrepo -r 11244' failed with this output:
Modified: tor/branches/114-dist-storage/ChangeLog
===================================================================
--- tor/branches/114-dist-storage/ChangeLog 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/ChangeLog 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,76 +1,218 @@
-Changes in version 0.2.0.3-alpha - 2007-??-??
- o Minor features:
- - Create listener connections before we setuid to the configured User and
- Group. This way, you can choose port values under 1024, start Tor as
- root, and have Tor bind those ports before it changes to another UID.
- - tor-gencert creates all files as readable to the file creator only, and
- write-protects the authority identity key.
+Changes in version 0.2.0.5-alpha - 2007-??-??
+ o Major bugfixes (compilation):
+ - Try to fix win32 compilation again: Improve checking for ipv6 types.
+ - Fix compile on platforms without getaddrinfo: bug found by Li-Hui
+ Zhou.
- o Minor features (directory authority):
- - Fail quickly and (relatively) harmlessly if we generate a network
- status document that is somehow malformed.
+ o Minor featuers (security):
+ - Warn about unsafe ControlPort configurations.
+
+Changes in version 0.2.0.4-alpha - 2007-08-01
+ o Major security fixes:
+ - Close immediately after missing authentication on control port;
+ do not allow multiple authentication attempts.
+
+ o Major bugfixes (compilation):
+ - Fix win32 compilation: apparently IN_ADDR and IN6_ADDR are already
+ defined there.
+
+ o Minor features (performance):
+ - Be even more aggressive about releasing RAM from small
+ empty buffers. Thanks to our free-list code, this shouldn't be too
+ performance-intensive.
+ - Disable sentinel-based debugging for buffer code: we squashed all
+ the bugs that this was supposed to detect a long time ago, and
+ now its only effect is to change our buffer sizes from nice
+ powers of two (which platform mallocs tend to like) to values
+ siightly over powers of two (which make some platform mallocs sad).
+ - Log malloc statistics from mallinfo() on platforms where it
+ exists.
+
+
+Changes in version 0.2.0.3-alpha - 2007-07-29
+ o Major features:
+ - The first pieces of our "bridge" design for blocking-resistance
+ are implemented. People can run bridge directory authorities;
+ people can run bridges; and people can configure their Tor clients
+ with a set of bridges to use as the first hop into the Tor network.
+ See http://archives.seul.org/or/talk/Jul-2007/msg00249.html for
+ details.
+ - Create listener connections before we setuid to the configured
+ User and Group. Now non-Windows users can choose port values
+ under 1024, start Tor as root, and have Tor bind those ports
+ before it changes to another UID. (Windows users could already
+ pick these ports.)
+ - Added a new ConstrainedSockets config option to set SO_SNDBUF and
+ SO_RCVBUF on TCP sockets. Hopefully useful for Tor servers running
+ on "vserver" accounts. (Patch from coderman.)
+ - Be even more aggressive about separating local traffic from relayed
+ traffic when RelayBandwidthRate is set. (Refines proposal 111.)
+
+ o Major features (experimental):
+ - First cut of code for "v3 dir voting": directory authorities will
+ vote on a common network status document rather than each publishing
+ their own opinion. This code needs more testing and more corner-case
+ handling before it's ready for use.
+
+ o Security fixes:
+ - Directory authorities now call routers Fast if their bandwidth is
+ at least 100KB/s, and consider their bandwidth adequate to be a
+ Guard if it is at least 250KB/s, no matter the medians. This fix
+ complements proposal 107. [Bugfix on 0.1.2.x]
+ - Directory authorities now never mark more than 3 servers per IP as
+ Valid and Running. (Implements proposal 109, by Kevin Bauer and
+ Damon McCoy.)
+ - Minor change to organizationName and commonName generation
+ procedures in TLS certificates during Tor handshakes, to invalidate
+ some earlier censorware approaches. This is not a long-term
+ solution, but applying it will give us a bit of time to look into
+ the epidemiology of countermeasures as they spread.
+
+ o Major bugfixes (directory):
+ - Rewrite directory tokenization code to never run off the end of
+ a string. Fixes bug 455. Patch from croup. [Bugfix on 0.1.2.x]
+
o Minor features (controller):
- Add a SOURCE_ADDR field to STREAM NEW events so that controllers can
match requests to applications. (Patch from Robert Hogan.)
- Report address and port correctly on connections to DNSPort. (Patch
from Robert Hogan.)
+ - Add a RESOLVE command to launch hostname lookups. (Original patch
+ from Robert Hogan.)
+ - Add GETINFO status/enough-dir-info to let controllers tell whether
+ Tor has downloaded sufficient directory information. (Patch
+ from Tup.)
+ - You can now use the ControlSocket option to tell Tor to listen for
+ controller connections on Unix domain sockets on systems that
+ support them. (Patch from Peter Palfrader.)
+ - STREAM NEW events are generated for DNSPort requests and for
+ tunneled directory connections. (Patch from Robert Hogan.)
+ - New "GETINFO address-mappings/*" command to get address mappings
+ with expiry information. "addr-mappings/*" is now deprecated.
+ (Patch from Tup.)
- o Performance improvements (win32):
+ o Minor features (misc):
+ - Merge in some (as-yet-unused) IPv6 address manipulation code. (Patch
+ from croup.)
+ - The tor-gencert tool for v3 directory authorities now creates all
+ files as readable to the file creator only, and write-protects
+ the authority identity key.
+ - When dumping memory usage, list bytes used in buffer memory
+ free-lists.
+ - When running with dmalloc, dump more stats on hup and on exit.
+ - Directory authorities now fail quickly and (relatively) harmlessly
+ if they generate a network status document that is somehow
+ malformed.
+
+ o Traffic load balancing improvements:
+ - If exit bandwidth ever exceeds one third of total bandwidth, then
+ use the correct formula to weight exit nodes when choosing paths.
+ (Based on patch from Mike Perry.)
+ - Choose perfectly fairly among routers when choosing by bandwidth and
+ weighting by fraction of bandwidth provided by exits. Previously, we
+ would choose with only approximate fairness, and correct ourselves
+ if we ran off the end of the list. [Bugfix on 0.1.2.x]
+
+ o Performance improvements:
+ - Be more aggressive with freeing buffer RAM or putting it on the
+ memory free lists.
- Use Critical Sections rather than Mutexes for synchronizing threads
on win32; Mutexes are heavier-weight, and designed for synchronizing
between processes.
- o Deprecated features:
+ o Deprecated and removed features:
- RedirectExits is now deprecated.
+ - Stop allowing address masks that do not correspond to bit prefixes.
+ We have warned about these for a really long time; now it's time
+ to reject them. (Patch from croup.)
- o Minor features (controller):
- - You can now use the ControlSocket option to tell Tor to listen for
- controller connections on Unix domain sockets on systems that support
- them. (Patch from Peter Palfrader.)
- - STREAM NEW events are generated for DNSPort requests and for tunneled
- directory connections. (Patch from Robert Hogan.)
-
- o Major bugfixes (directory):
- - Fix a crash bug in directory authorities when we re-number the
- routerlist while inserting a new router. [Bugfix on 0.1.2.x]
-
o Minor bugfixes (directory):
- - Fix another crash bug related to extra-info caching. (Bug found by
+ - Fix another crash bug related to extra-info caching. (Bug found by
Peter Palfrader.) [Bugfix on 0.2.0.2-alpha]
- Directories no longer return a "304 not modified" when they don't
have the networkstatus the client asked for. Also fix a memory
leak when returning 304 not modified. [Bugfixes on 0.2.0.2-alpha]
+ - We had accidentally labelled 0.1.2.x directory servers as not
+ suitable for begin_dir requests, and had labelled no directory
+ servers as suitable for uploading extra-info documents. [Bugfix
+ on 0.2.0.1-alpha]
o Minor bugfixes (dns):
- Fix a crash when DNSPort is set more than once. (Patch from Robert
Hogan.) [Bugfix on 0.2.0.2-alpha]
- Add DNSPort connections to the global connection list, so that we
- can time them out correctly. (Bug found by mwenge) [Bugfix on
+ can time them out correctly. (Bug found by Robert Hogan.) [Bugfix
+ on 0.2.0.2-alpha]
+ - Fix a dangling reference that could lead to a crash when DNSPort is
+ changed or closed (Patch from Robert Hogan.) [Bugfix on
0.2.0.2-alpha]
+
+ o Minor bugfixes (controller):
+ - Provide DNS expiry times in GMT, not in local time. For backward
+ compatibility, ADDRMAP events only provide GMT expiry in an extended
+ field. "GETINFO address-mappings" always does the right thing.
+ - Use CRLF line endings properly in NS events.
+ - Terminate multi-line control events properly. (Original patch
+ from tup.) [Bugfix on 0.1.2.x-alpha]
+ - Do not include spaces in SOURCE_ADDR fields in STREAM
+ events. Resolves bug 472. [Bugfix on 0.2.0.x-alpha]
+
+
+Changes in version 0.1.2.15 - 2007-07-17
+ o Major bugfixes (compilation):
+ - Fix compile on FreeBSD/NetBSD/OpenBSD. Oops.
+
+ o Major bugfixes (crashes):
+ - Try even harder not to dereference the first character after
+ an mmap(). Reported by lodger.
+ - Fix a crash bug in directory authorities when we re-number the
+ routerlist while inserting a new router.
+ - When the cached-routers file is an even multiple of the page size,
+ don't run off the end and crash. (Fixes bug 455; based on idea
+ from croup.)
- Fix eventdns.c behavior on Solaris: It is critical to include
orconfig.h _before_ sys/types.h, so that we can get the expected
- definition of _FILE_OFFSET_BITS. [Bugfix on 0.1.2.x]
+ definition of _FILE_OFFSET_BITS.
+ o Major bugfixes (security):
+ - Fix a possible buffer overrun when using BSD natd support. Bug
+ found by croup.
+ - When sending destroy cells from a circuit's origin, don't include
+ the reason for tearing down the circuit. The spec says we didn't,
+ and now we actually don't. Reported by lodger.
+ - Keep streamids from different exits on a circuit separate. This
+ bug may have allowed other routers on a given circuit to inject
+ cells into streams. Reported by lodger; fixes bug 446.
+ - If there's a never-before-connected-to guard node in our list,
+ never choose any guards past it. This way we don't expand our
+ guard list unless we need to.
+
+ o Minor bugfixes (guard nodes):
+ - Weight guard selection by bandwidth, so that low-bandwidth nodes
+ don't get overused as guards.
+
+ o Minor bugfixes (directory):
+ - Correctly count the number of authorities that recommend each
+ version. Previously, we were under-counting by 1.
+ - Fix a potential crash bug when we load many server descriptors at
+ once and some of them make others of them obsolete. Fixes bug 458.
+
o Minor bugfixes (hidden services):
- Stop tearing down the whole circuit when the user asks for a
connection to a port that the hidden service didn't configure.
- Resolves bug 444. [Bugfix on 0.1.2.x]
+ Resolves bug 444.
- o Minor bugfixes (versioning):
- - Stop under-counting the number of authorities that recommend each
- version. [Bugfix on 0.1.2.x]
+ o Minor bugfixes (misc):
+ - On Windows, we were preventing other processes from reading
+ cached-routers while Tor was running. Reported by janbar.
+ - Fix a possible (but very unlikely) bug in picking routers by
+ bandwidth. Add a log message to confirm that it is in fact
+ unlikely. Patch from lodger.
+ - Backport a couple of memory leak fixes.
+ - Backport miscellaneous cosmetic bugfixes.
- o Minor bugfixes (guard nodes):
- - If there's a never-before-connected-to guard node in our list,
- never choose any guards past it. This way we don't expand our
- guard list unless we need to. [Bugfix in 0.1.2.x]
- o Security fixes (BSD natd support):
- - Fix a possible buffer overrun when using BSD natd support. Bug found
- by croup.
-
-
Changes in version 0.2.0.2-alpha - 2007-06-02
o Major bugfixes on 0.2.0.1-alpha:
- Fix an assertion failure related to servers without extra-info digests.
@@ -136,7 +278,7 @@
Add a standalone tool to generate key certificates. (Proposal 103.)
o Security fixes:
- - Directory authorities now call routers stable if they have an
+ - Directory authorities now call routers Stable if they have an
uptime of at least 30 days, even if that's not the median uptime
in the network. Implements proposal 107, suggested by Kevin Bauer
and Damon McCoy.
@@ -262,6 +404,8 @@
we restart.
- Add even more asserts to hunt down bug 417.
- Build without verbose warnings even on (not-yet-released) gcc 4.2.
+ - Fix a possible (but very unlikely) bug in picking routers by bandwidth.
+ Add a log message to confirm that it is in fact unlikely.
o Minor bugfixes (controller):
- Make 'getinfo fingerprint' return a 551 error if we're not a
Modified: tor/branches/114-dist-storage/INSTALL
===================================================================
--- tor/branches/114-dist-storage/INSTALL 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/INSTALL 2007-08-21 08:37:13 UTC (rev 11244)
@@ -2,9 +2,9 @@
Most users should simply follow the directions at
http://tor.eff.org/docs/tor-doc-unix
-If you got the source from cvs, run "./autogen.sh", which will run the
-various auto* programs and then run ./configure for you. From there,
-refer to the above instructions.
+If you got the source from Subversion, run "./autogen.sh", which will
+run the various auto* programs and then run ./configure for you. From
+there, refer to the above instructions.
If it doesn't build for you:
Modified: tor/branches/114-dist-storage/autogen.sh
===================================================================
--- tor/branches/114-dist-storage/autogen.sh 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/autogen.sh 2007-08-21 08:37:13 UTC (rev 11244)
@@ -4,4 +4,4 @@
autoheader && \
autoconf && \
automake --add-missing --copy && \
- if test x$NOCONF = x ; then ./configure; fi
+ if test x$NOCONF = x ; then ./configure "$@"; fi
Modified: tor/branches/114-dist-storage/configure.in
===================================================================
--- tor/branches/114-dist-storage/configure.in 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/configure.in 2007-08-21 08:37:13 UTC (rev 11244)
@@ -4,13 +4,13 @@
dnl See LICENSE for licensing information
AC_INIT
-AM_INIT_AUTOMAKE(tor, 0.2.0.2-alpha-dev)
+AM_INIT_AUTOMAKE(tor, 0.2.0.4-alpha-dev)
AM_CONFIG_HEADER(orconfig.h)
AC_CANONICAL_HOST
if test -f /etc/redhat-release ; then
- CFLAGS="$CFLAGS -I/usr/kerberos/include"
+ CPPFLAGS="$CPPFLAGS -I/usr/kerberos/include"
fi
AC_ARG_ENABLE(debug,
@@ -152,7 +152,7 @@
dnl Check for functions before libevent, since libevent-1.2 apparently
dnl exports strlcpy without defining it in a header.
-AC_CHECK_FUNCS(gettimeofday ftime socketpair uname inet_aton strptime getrlimit strlcat strlcpy strtoull ftello getaddrinfo localtime_r gmtime_r memmem strtok_r inet_pton inet_ntop)
+AC_CHECK_FUNCS(gettimeofday ftime socketpair uname inet_aton strptime getrlimit strlcat strlcpy strtoull ftello getaddrinfo localtime_r gmtime_r memmem strtok_r inet_pton inet_ntop mallinfo)
if test $enable_threads = "yes"; then
AC_CHECK_HEADERS(pthread.h)
@@ -244,7 +244,7 @@
dnl These headers are not essential
-AC_CHECK_HEADERS(stdint.h sys/types.h inttypes.h sys/param.h sys/wait.h limits.h sys/limits.h netinet/in.h arpa/inet.h machine/limits.h syslog.h sys/time.h sys/resource.h inttypes.h utime.h sys/utime.h sys/mman.h netintet/in.h netinet/in6.h)
+AC_CHECK_HEADERS(stdint.h sys/types.h inttypes.h sys/param.h sys/wait.h limits.h sys/limits.h netinet/in.h arpa/inet.h machine/limits.h syslog.h sys/time.h sys/resource.h inttypes.h utime.h sys/utime.h sys/mman.h netintet/in.h netinet/in6.h malloc.h)
AC_CHECK_HEADERS(net/if.h, net_if_found=1, net_if_found=0,
[#ifdef HAVE_SYS_TYPES_H
@@ -329,7 +329,7 @@
AC_CHECK_TYPES([uint, u_char])
-AC_CHECK_TYPES([struct in6_addr, struct sockaddr_storage], , ,
+AC_CHECK_TYPES([struct in6_addr, struct sockaddr_in6, struct sockaddr_storage, sa_family_t], , ,
[#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
#endif
@@ -341,7 +341,19 @@
#endif
#ifdef HAVE_SYS_SOCKET_H
#include <sys/socket.h>
-#endif])
+#endif
+#ifdef MS_WINDOWS
+#define WIN32_WINNT 0x400
+#define _WIN32_WINNT 0x400
+#define WIN32_LEAN_AND_MEAN
+#if defined(_MSC_VER) && (_MSC_VER < 1300)
+#include <winsock.h>
+#else
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#endif
+#endif
+])
if test -z "$CROSS_COMPILE"; then
AC_CACHE_CHECK([whether time_t is signed], tor_cv_time_t_signed, [
@@ -609,7 +621,7 @@
enable_gcc_warnings=no
fi
-# Add some more warnings which we use in the cvs version but not in the
+# Add some more warnings which we use in development but not in the
# released versions. (Some relevant gcc versions can't handle these.)
if test x$enable_gcc_warnings = xyes; then
@@ -623,7 +635,8 @@
#error
#endif]), have_gcc42=yes, have_gcc42=no)
- CFLAGS="$CFLAGS -W -Wfloat-equal -Wundef -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings -Wredundant-decls -Wchar-subscripts -Wcomment -Wformat=2 -Wwrite-strings -Waggregate-return -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wbad-function-cast -Wswitch-enum -Werror"
+ CFLAGS="$CFLAGS -W -Wfloat-equal -Wundef -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings -Wredundant-decls -Wchar-subscripts -Wcomment -Wformat=2 -Wwrite-strings -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wbad-function-cast -Wswitch-enum -Werror"
+ # Disabled, so we can use mallinfo(): -Waggregate-return
if test x$have_gcc4 = xyes ; then
# These warnings break gcc 3.3.5 and work on gcc 4.0.2
Modified: tor/branches/114-dist-storage/doc/HACKING
===================================================================
--- tor/branches/114-dist-storage/doc/HACKING 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/HACKING 2007-08-21 08:37:13 UTC (rev 11244)
@@ -3,6 +3,15 @@
http://tor-buildbot.freehaven.net:8010/
+0.1. Useful command-lines that are non-trivial to reproduce but can
+help with tracking bugs or leaks.
+
+dmalloc -l ~/dmalloc.log
+(run the commands it tells you)
+./configure --with-dmalloc
+
+valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor
+
1. Coding conventions
1.0. Whitespace and C conformance
Modified: tor/branches/114-dist-storage/doc/TODO
===================================================================
--- tor/branches/114-dist-storage/doc/TODO 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/TODO 2007-08-21 08:37:13 UTC (rev 11244)
@@ -15,10 +15,6 @@
Documentation and testing on 0.1.2.x-final series
- - Pending backports for 0.1.2.x:
- - r10148: Open cached-routers with FILE_SHARE_READ on win32.
- - r10493: Weight guard selection by bandwidth.
-
N - Test guard unreachable logic; make sure that we actually attempt to
connect to guards that we think are unreachable from time to time.
Make sure that we don't freak out when the network is down.
@@ -41,6 +37,24 @@
- In a good HOWTO.
Things we'd like to do in 0.2.0.x:
+ - Bug reports Roger has heard along that way that don't have enough
+ details/attention to solve them yet.
+ - tup said that when he set FetchUselessDescriptors, after
+ 24 or 48 hours he wasn't fetching any descriptors at all
+ anymore. This was in 0.2.0 but worked fine in 0.1.2.
+ - arma noticed that when his network went away and he tried
+ a new guard node and the connect() syscall failed to it,
+ the guard wasn't being marked as down. 0.2.0.x.
+ - after being without network for 12 hours, arma's tor decided
+ it couldn't fetch any network statuses, and never tried again
+ even when the network came back and arma clicked on things.
+ also 0.2.0.
+ - phobos says relaybandwidth* sometimes don't do what we expect.
+ http://interloper.org/tmp/2007-06-bw-usage.png
+ - this notion of authorities notifying servers that they're
+ unreachable is bunk -- it's leftover from the time when all
+ servers ran 24/7. now it triggers every time a server goes
+ away and then returns before the old descriptor has expired.
- Update dir-spec with decisions made on these issues:
o clients don't log as loudly when they receive them
o they don't count toward the 3-strikes rule
@@ -64,26 +78,47 @@
o Don't flip out with warnings when voting-related URLs are
uploaded/downloaded.
. Finalize proposal
- . Merge 101 and 103 and dir-spec.txt into a new dir-spec.txt; fork
+ o Merge 101 and 103 and dir-spec.txt into a new dir-spec.txt; fork
the existing one into dir-spec-v2.txt.
+ * Describe schedule in copious detail.
- Get authorities voting
- . Implement parsing for new document formats
+ o Implement parsing for new document formats
o Parse key certificates
o Parse votes and consensuses
- - Unit tests for above
+ o Unit tests for above
. Code to manage key certificates
o Generate certificates
o Authorities load certificates
o Clients cache certificates on disk
+ o Learn new ones when they show up in votes.
+ o Forget ones that are very old.
- Download as needed.
+ o Actually invoke trusted_dirs_flush_certs_to_disk()
- Serve list as needed.
o Avoid double-checking signatures every time we get a vote.
- Warn about expired stuff.
+ - Fix all XXXX020s in vote code
o Code to generate votes
o Code to generate consensus from a list of votes
- - Add a signature to a consensus.
+ * Detect whether votes are really all for the same period.
+ o Add a signature to a consensus.
+ * Unit tests for detached signatures and signature manipulation.
o Code to check signatures on a consensus
- Push/pull documents as appropriate.
+ o Push vote on voting
+ o Push vote
+ o Process vote when received
+ o Even if we get it before we start voting ourself.
+ o Push signature on forming consensus.
+ o Push signature
+ o Add signatures when received
+ o Queue received signatures before consensus is ready
+ o When consensus is ready, use queued signatures.
+ - Pull votes and signatures if we don't get them.
+ o Serve consensuses.
+ - Store consensuses
+ - Cache votes and signatures on disk.
+ o Discard votes in advance of next voting period.
o Have clients know which authorities are v3 authorities, and what
their keys are.
- While we're at it, let v3 authorities have fqdns lines.
@@ -106,7 +141,10 @@
- Drop bandwidth history from router-descriptors
- 105: Version negotiation for the Tor protocol
- 108: Base "Stable" Flag on Mean Time Between Failures
- - 109: No more than one server per IP address
+ - Track mtbf in rephist.c
+ - Record mtbf between invocations
+ - Base stable on mtbf.
+ o 109: No more than one server per IP address
o 103: Splitting identity key from regularly used signing key
o Merge with 101 into a new dir-spec.txt
- 113: Simplifying directory authority administration
@@ -115,11 +153,11 @@
on just like relay cells
- Refactoring:
- Make resolves no longer use edge_connection_t unless they are actually
- _on_ on a socks connection: have edge_connection_t and (say)
+ _on_ a socks connection: have edge_connection_t and (say)
dns_request_t both extend an edge_stream_t, and have p_streams and
n_streams both be linked lists of edge_stream_t.
. Make cells get buffered on circuit, not on the or_conn.
- O Implement cell queues
+ o Implement cell queues
o Keep doubly-linked list of active circuits on each or_conn.
o Put all relay data on the circuit cell queue, not on the outbuf.
o Don't move them into the target conn until there is space on the
@@ -206,11 +244,11 @@
o Make it handle .onion and .exit correctly.
- Document.
- Handle TCP DNS requests too?
- - Add a way to request DNS resolves from the controller.
+ o Add a way to request DNS resolves from the controller.
- A better UI for authority ops.
- Follow weasel's proposal, crossed with mixminion dir config format
- Write a proposal
- - Bridges users (rudimentary version)
+ . Bridges users (rudimentary version)
o Ability to specify bridges manually
o Config option 'UseBridges' that bridge users can turn on.
o uses bridges as first hop rather than entry guards.
@@ -221,6 +259,8 @@
like the ones you have, ask a new bridge for its server/authority.
. Ask all directory questions to bridge via BEGIN_DIR.
- use the bridges for dir fetches even when our dirport is open.
+R - drop 'authority' queries if they're to our own identity key; accept
+ them otherwise.
N - Design/implement the "local-status" or something like it, from the
"Descriptor purposes: how to tell them apart" section of
http://archives.seul.org/or/dev/May-2007/msg00008.html
@@ -228,16 +268,16 @@
manually listed in the torrc.
D and some mechanism for specifying that we want to stop using
a given bridge in this cache.
- - timeout and retry schedules for fetching bridge descriptors
+ o timeout and retry schedules for fetching bridge descriptors
- give extend_info_t a router_purpose again
o react faster to download networkstatuses after the first bridge
descriptor arrives
- - be more robust to bridges being marked as down and leaving us
+ o be more robust to bridges being marked as down and leaving us
stranded without any known "running" bridges.
- Bridges operators (rudimentary version)
- Ability to act as dir cache without a dir port.
o Bridges publish to bridge authorities
- - Fix BEGIN_DIR so that you connect to bridge of which you only
+ o Fix BEGIN_DIR so that you connect to bridge of which you only
know IP (and optionally fingerprint), and then use BEGIN_DIR to learn
more about it.
- look at server_mode() and decide if it always applies to bridges too.
@@ -248,7 +288,7 @@
D Should do reachability testing but only on the purpose==bridge
descriptors we have.
- Bridges
- - Clients can ask bridge authorities for updates on known bridges.
+ o Clients can ask bridge authorities for updates on known bridges.
- More TLS normalization work: make Tor less easily
fingerprinted.
- Directory system improvements
Modified: tor/branches/114-dist-storage/doc/spec/control-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/control-spec.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/control-spec.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -503,6 +503,7 @@
an empty string.
"status/circuit-established"
+ "status/enough-dir-info"
"status/..."
These provide the current internal Tor values for various Tor
states. See Section 4.1.10 for explanations. (Only a few of the
@@ -708,6 +709,20 @@
stable releases after 0.1.2.2-alpha, the release where it was first
available.)
+3.20. RESOLVE
+
+ The syntax is
+ "RESOLVE" *Option *Address CRLF
+ Option = "mode=reverse"
+ Address = a hostname or IPv4 address
+
+ This command launches a remote hostname lookup request for every specified
+ request (or reverse lookup if "mode=reverse" is specified). Note that the
+ request is done in the background: to see the answers, your controller will
+ need to listen for ADDRMAP events; see 4.1.7 below.
+
+ [Added in Tor 0.2.0.3-alpha]
+
4. Replies
Reply codes follow the same 3-character format as used by SMTP, with the
@@ -906,9 +921,9 @@
the new address as a response.
The "SOURCE_ADDR" field is included with NEW and NEWRESOLVE events if
- extended events are enabled. It indicates the address that requested
- the connection, and can be (e.g.) used to look up the requesting
- program.
+ extended events are enabled. It indicates the address and port
+ that requested the connection, and can be (e.g.) used to look up the
+ requesting program.
4.1.3. OR Connection status changed
@@ -979,8 +994,8 @@
left in for backward compatibility; new code should look at GMTExpiry
instead.
- [XXX We should rename this to ADDRESSMAP. -RD]
- [Why? Surely it can't be worth the compatibility issues. -NM]
+ These events are generated when a new address mapping is entered in the
+ cache, or when the answer for a RESOLVE command is found.
4.1.8. Descriptors uploaded to us in our role as authoritative dirserver
Modified: tor/branches/114-dist-storage/doc/spec/dir-spec-v2.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/dir-spec-v2.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/dir-spec-v2.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -482,6 +482,12 @@
Directory server administrators may label some servers or IPs as
blacklisted, and elect not to include them in their network-status lists.
+ Authorities SHOULD 'disable' any servers in excess of 3 on any single IP.
+ When there are more than 3 to choose from, authorities should first prefer
+ authorities to non-authorities, then prefer Running to non-Running, and
+ then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the
+ authority *should* advertise it without the Running or Valid flag.
+
Thus, the network-status list includes all non-blacklisted,
non-expired, non-superseded descriptors.
Modified: tor/branches/114-dist-storage/doc/spec/path-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/path-spec.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/path-spec.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -189,7 +189,11 @@
on the fraction of bandwidth available from non-Exit nodes. Call the
total clipped advertised bandwidth for Exit nodes under consideration E,
and the total clipped advertised bandwidth for non-Exit nodes under
- consideration N. If E<N/2, we do not consider Exit-flagged nodes.
+ and the total clipped advertised bandwidth for all nodes under
+ consideration T. If E<T/3, we do not consider Exit-flagged nodes.
+ Otherwise, we weight their bandwidth with the factor 1-T/(3E). This
+ ensures that bandwidth is evenly distributed over nodes in 3-hop paths.
+
Otherwise, we weight their bandwidth with the factor (E-N/2)/(N+E-N/2) ==
(2E - N)/(2E + N). This ensures that bandwidth is evenly distributed over
nodes in 3-hop paths.
Modified: tor/branches/114-dist-storage/doc/spec/proposals/000-index.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/000-index.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/000-index.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -33,4 +33,7 @@
112 Bring Back Pathlen Coin Weight [OPEN]
113 Simplifying directory authority administration [OPEN]
114 Distributed Storage for Tor Hidden Service Descriptors [OPEN]
+115 Two Hop Paths [OPEN]
+116 Two hop paths from entry guards [OPEN]
+117 IPv6 exits [OPEN]
Modified: tor/branches/114-dist-storage/doc/spec/proposals/001-process.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/001-process.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/001-process.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 001-process.txt
Title: The Tor Proposal Process
-Version: $Revision: 11537 $
-Last-Modified: $Date: 2007-01-26T19:04:29.998860Z $
+Version: $Revision: 10625 $
+Last-Modified: $Date: 2007-06-17 00:23:19 +0100 (So, 17 Jun 2007) $
Author: Nick Mathewson
Created: 30-Jan-2007
Status: Meta
Modified: tor/branches/114-dist-storage/doc/spec/proposals/106-less-tls-constraint.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/106-less-tls-constraint.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/106-less-tls-constraint.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 106-less-tls-constraint.txt
Title: Checking fewer things during TLS handshakes
-Version: $Revision: 12105 $
-Last-Modified: $Date: 2007-01-30T07:50:01.643717Z $
+Version: $Revision: 10625 $
+Last-Modified: $Date: 2007-06-17 00:23:19 +0100 (So, 17 Jun 2007) $
Author: Nick Mathewson
Created: 9-Feb-2007
Status: Closed
Modified: tor/branches/114-dist-storage/doc/spec/proposals/107-uptime-sanity-checking.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/107-uptime-sanity-checking.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/107-uptime-sanity-checking.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 107-uptime-sanity-checking.txt
Title: Uptime Sanity Checking
Version: $Revision: 10625 $
-Last-Modified: $Date: 2007-06-17 01:23:19 +0200 (So, 17 Jun 2007) $
+Last-Modified: $Date: 2007-06-17 00:23:19 +0100 (So, 17 Jun 2007) $
Author: Kevin Bauer & Damon McCoy
Created: 8-March-2007
Status: Closed
Modified: tor/branches/114-dist-storage/doc/spec/proposals/108-mtbf-based-stability.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/108-mtbf-based-stability.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/108-mtbf-based-stability.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 108-mtbf-based-stability.txt
Title: Base "Stable" Flag on Mean Time Between Failures
-Version: $Revision: 10684 $
-Last-Modified: $Date: 2007-06-26 23:48:42 +0200 (Di, 26 Jun 2007) $
+Version: $Revision: 10729 $
+Last-Modified: $Date: 2007-07-02 23:07:53 +0100 (Mo, 02 Jul 2007) $
Author: Nick Mathewson
Created: 10-Mar-2007
Status: Open
@@ -41,7 +41,8 @@
Rejected Alternative:
"A router's Stability shall be defined as the sum of $\alpha ^ d$ for every
- $d$ such that the router was not observed to be unavailable $d$ days ago."
+ $d$ such that the router was considered reachable for the entire day
+ $d$ days ago.
This allows a simpler implementation: every day, we multiply
yesterday's Stability by alpha, and if the router was observed to be
@@ -54,7 +55,7 @@
(By requiring that routers be up for an entire day to get their
stability increased, instead of counting fractions of a day, we
- capture the notion that stability is more like "probability of being
+ capture the notion that stability is more like "probability of
staying up for the next hour" than it is like "probability of being
up at some randomly chosen time over the next hour." The former
notion of stability is far more relevant for long-lived circuits.)
Modified: tor/branches/114-dist-storage/doc/spec/proposals/109-no-sharing-ips.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/109-no-sharing-ips.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/109-no-sharing-ips.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,10 +1,10 @@
Filename: 109-no-sharing-ips.txt
Title: No more than one server per IP address.
-Version: $Revision: 10625 $
-Last-Modified: $Date: 2007-06-17 01:23:19 +0200 (So, 17 Jun 2007) $
+Version: $Revision: 10968 $
+Last-Modified: $Date: 2007-07-29 06:20:31 +0100 (So, 29 Jul 2007) $
Author: Kevin Bauer & Damon McCoy
Created: 9-March-2007
-Status: Accepted
+Status: Closed
Overview:
This document describes a solution to a Sybil attack vulnerability in the
@@ -34,14 +34,19 @@
For each IP address, each directory authority tracks the number of routers
using that IP address, along with their total observed bandwidth. If there
are more than MAX_SERVERS_PER_IP servers at some IP, the authority should
- "disable" all but MAX_SERVERS_PER_IP servers. If the total observed
+ "disable" all but MAX_SERVERS_PER_IP servers. When choosing which servers
+ to disable, the authority should first disable non-Running servers in
+ increasing order of observed bandwidth, and then should disable Running
+ servers in increasing order of bandwidth.
+
+ [[ We don't actually do this part here. -NM
+
+ If the total observed
bandwidth of the remaining non-"disabled" servers exceeds MAX_BW_PER_IP,
the authority should "disable" some of the remaining servers until only one
server remains, or until the remaining observed bandwidth of non-"disabled"
- servers is under MAX_BW_PER_IP. When choosing which servers to disable,
- the authority should first disable non-Running servers in increasing order
- of observed bandwidth, and then should disable Running servers in
- increasing order of bandwidth.
+ servers is under MAX_BW_PER_IP.
+ ]]
Servers that are "disabled" MUST be marked as non-Valid and non-Running.
Modified: tor/branches/114-dist-storage/doc/spec/proposals/110-avoid-infinite-circuits.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/110-avoid-infinite-circuits.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/110-avoid-infinite-circuits.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
-Filename: 1xx-avoid-infinite-circuits.txt
+Filename: 110-avoid-infinite-circuits.txt
Title: Avoiding infinite length circuits
-Version: $Revision: 10625 $
-Last-Modified: $Date: 2007-06-17 01:23:19 +0200 (So, 17 Jun 2007) $
+Version: $Revision: 11054 $
+Last-Modified: $Date: 2007-08-07 23:09:09 +0100 (Di, 07 Aug 2007) $
Author: Roger Dingledine
Created: 13-Mar-2007
Status: Open
@@ -102,6 +102,8 @@
Acknowledgements:
- This design has been kicking around since Christian Grothoff and I
- came up with it at PET 2004.
+ This design has been kicking around since Christian Grothoff and I came
+ up with it at PET 2004. (Nathan Evans, Christian Grothoff's student,
+ is working on implementing a fix based on this design in the summer
+ 2007 timeframe.)
Modified: tor/branches/114-dist-storage/doc/spec/proposals/111-local-traffic-priority.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/111-local-traffic-priority.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/111-local-traffic-priority.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 111-local-traffic-priority.txt
Title: Prioritizing local traffic over relayed traffic
-Version: $Revision: 10625 $
-Last-Modified: $Date: 2007-06-17 01:23:19 +0200 (So, 17 Jun 2007) $
+Version: $Revision: 10974 $
+Last-Modified: $Date: 2007-07-29 23:13:44 +0100 (So, 29 Jul 2007) $
Author: Roger Dingledine
Created: 14-Mar-2007
Status: Open
@@ -118,20 +118,35 @@
(Gosh. How could UDP designs possibly be compatible with rate limiting
with multiple bucket sizes?)
- Option 4: ?
+ Option 4: put both classes of circuits over a single connection, and
+ keep track of the last time we read or wrote a high-priority cell. If
+ it's been less than N seconds, give the whole connection high priority,
+ else give the whole connection low priority.
+ Option 5: put both classes of circuits over a single connection, and
+ play a complex juggling game by periodically telling the remote side
+ what rate limits to set for that connection, so you end up giving
+ priority to the right connections but still stick to roughly your
+ intended bandwidthrate and relaybandwidthrate.
+
+ Option 6: ?
+
Prognosis:
- Of the above options, only option 2 can actually be built and achieve
- what we want. So that's it by default, unless we can come up with
- something better.
+ Nick really didn't like option 2 because of the partitioning questions.
- In terms of implementation, it will be easy: just add a bit to
- or_connection_t that specifies priority_traffic (used by the initiator
- of the connection to ignore that connection when relaying a create
- request), and another bit that specifies client_only (used by a
- receiving Tor server so it can ignore that connection when sending
- create requests).
+ I've put option 4 into place as of Tor 0.2.0.3-alpha.
-[Not writing the rest of the proposal until we sort out which option
-we'll take.]
+ In terms of implementation, it will be easy: just add a time_t to
+ or_connection_t that specifies client_used (used by the initiator
+ of the connection to rate limit it differently depending on how
+ recently the time_t was reset). We currently update client_used
+ in three places:
+ - command_process_relay_cell() when we receive a relay cell for
+ an origin circuit.
+ - relay_send_command_from_edge() when we send a relay cell for
+ an origin circuit.
+ - circuit_deliver_create_cell() when send a create cell.
+ We could probably remove the third case and it would still work,
+ but hey.
+
Modified: tor/branches/114-dist-storage/doc/spec/proposals/112-bring-back-pathlencoinweight.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/112-bring-back-pathlencoinweight.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/112-bring-back-pathlencoinweight.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 112-bring-back-pathlencoinweight.txt
Title: Bring Back Pathlen Coin Weight
Version: $Revision: 10625 $
-Last-Modified: $Date: 2007-06-17 01:23:19 +0200 (So, 17 Jun 2007) $
+Last-Modified: $Date: 2007-06-17 00:23:19 +0100 (So, 17 Jun 2007) $
Author: Mike Perry
Created:
Status: Superseded
Modified: tor/branches/114-dist-storage/doc/spec/proposals/113-fast-authority-interface.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/113-fast-authority-interface.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/113-fast-authority-interface.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 113-fast-authority-interface.txt
Title: Simplifying directory authority administration
-Version: $Revision: 12412 $
-Last-Modified: $Date: 2007-04-16T19:11:29.511998Z $
+Version: $Revision: 10625 $
+Last-Modified: $Date: 2007-06-17 00:23:19 +0100 (So, 17 Jun 2007) $
Author: Nick Mathewson
Created:
Status: Open
Modified: tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,7 +1,7 @@
Filename: 114-distributed-storage.txt
Title: Distributed Storage for Tor Hidden Service Descriptors
Version: $Revision: 10821 $
-Last-Modified: $Date: 2007-07-12 20:06:13 +0200 (Do, 12 Jul 2007) $
+Last-Modified: $Date: 2007-07-12 19:06:13 +0100 (Do, 12 Jul 2007) $
Author: Karsten Loesing
Created: 13-May-2007
Status: Open
@@ -9,12 +9,10 @@
Change history:
13-May-2007 Initial proposal
- 14-May-2007 Added changes suggested by Lasse Øverlier
+ 14-May-2007 Added changes suggested by Lasse Overlier
30-May-2007 Changed descriptor format, key length discussion, typos
09-Jul-2007 Incorporated suggestions by Roger, added status of specification
and implementation for upcoming GSoC mid-term evaluation
- 11-Aug-2007 Updated implementation statuses, included non-consecutive
- replication to descriptor format
Overview:
@@ -130,13 +128,8 @@
- routerlist.c: Changed router_get_routerlist() to initialize routing list.
- or.h: Added hs_dirs member to routerlist_t.
- - Changed routerlist_free() to free storage held by routing list.
- - Added UPDATE_HS_DIRS_INTERVAL.
- - Added update_hs_dir_routing_table().
- - Changed run_scheduled_events().
- - Added is_hs_dir member to routerstatus_t.
-
- [Aug 11: Specified and running.]
+ [July 9: Specified and running, though the routing list is compiled for
+ each request anew.]
/2/ Determine responsible hidden service directory
@@ -151,13 +144,11 @@
- rend-spec.txt, section 1.4: Added description of how to determine the
responsible node(s) for a given descriptor ID.
- - routerlist.c: Added get_responsible_hs_dirs() to determine the routers
- that are responsible for a given descriptor ID.
-
- - Added is_hs_dir member to routerstatus_t.
- - Added have_enough_hs_dirs().
- - Added next_hs_dir().
-
+ - routerlist.c: Added get_responsible_hs_dir() to determine the router that
+ is responsible for a given descriptor ID.
+ - container.h: Added prototype for smartlist_digest_next_circular().
+ - container.c: Added implementation for smartlist_digest_next_circular().
+
[July 9: Specified and running.]
Hidden service clients and providers:
@@ -229,19 +220,20 @@
- routerparse.c: Added 8 keywords to directory_keyword to parse v2 hidden
service descriptors.
- rendcommon.c: Added rend_cache_store_v2_dir() to allow a hidden service
- directory to parse a v2 descriptor and store it in the local cache under
- its descriptor ID instead of its service ID.
+ directory to store a v2 descriptor in the local cache under its
+ descriptor ID instead of its service ID.
+ - rendcommon.c: Moved the parsing part from rend_cache_store() to the new
+ function rend_cache_store_parse() to reuse it for v2 descriptors.
- or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
IDs are longer than v0/1 onion addresses.
- - Changed directory_handle_command_post().
-
- [Aug 11: Specified and running.]
+ [July 9: Base version specified and running; no checking of published
+ descriptors, tunneling over BEGIN_DIR cells not yet implemented.]
/7/ Accept v2 fetch requests
Same as /6/, but with fetch requests for hidden service descriptors.
- (requires /2/ and /4/)
+ (requires /4/)
- rend-spec.txt, section 3.3: Added the processing of v2 fetch requests.
@@ -251,9 +243,8 @@
- or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
IDs are longer than v0/1 onion addresses.
- - Changed directory_handle_command_get().
-
- [Aug 11: Specified and running.]
+ [July 9: Base version specified and running; tunneling over BEGIN_DIR
+ cells not yet implemented.]
/8/ Replicate descriptors with neighbors
@@ -270,19 +261,8 @@
- rend-spec.txt, section 3.3: Added the replication of v2 descriptors.
- - Added HS_DIR_REPLICATION_INTERVAL.
- - Added next_hs_dir and previous_hs_dir.
- - Changed directory_handle_command_get().
- - Changed run_scheduled_events.
- - Added hs_dir_perform_replication().
- - Added rend_cache_lookup_v2_replicas.
- - Added DIR_PURPOSE_REPLICATE_RENDDESC_V2.
- - Changed directory_initiate_command.
- - directory_send_command.
- - Changed connection_dir_client_reached_eof.
+ [July 9: To some extend specified, but not yet implemented.]
- [Aug 11: To some extend specified, running.]
-
Authoritative directory nodes:
/9/ Confirm a router's hidden service directory functionality
@@ -306,17 +286,16 @@
"hidden-service-directory" flag in router descriptors.
- routerparse.c: Added 1 keyword to directory_keyword to parse the
"hidden-service-dir" flag in router descriptors.
- - or.h: Added is_hs_dir and wants_to_be_hs_dir members to routerinfo_t.
+ - or.h: Added is_hs_dir member to routerinfo_t and to routerstatus_t.
- dirserv.c: Changed routerstatus_format_entry() to include the "HSDir"
flag in vote and consensus status documents.
- dirserv.c: Changed set_routerstatus_from_routerinfo() to set the "HSDir"
flag.
- - Added dirserv_thinks_router_is_hs_dir().
- - Added MIN_UPTIME_HS_DIR and HS_DIR_REACHABLE_TIMEOUT.
+ [July 9: Base version specified and running in which all nodes that have
+ the hidden-service-dir flag set in their router descriptor get the
+ HSDir flag, not only those which are running for at least 24 hours.]
- [Aug 11: Specified and running.]
-
Hidden service provider:
/10/ Configure v2 hidden service
@@ -360,8 +339,6 @@
service provider uses a freshly generated public key for every
introduction point.
- - TODO: Change in rend_encode_v2_descriptors.
-
[July 9: Specified, but not yet implemented.]
/12/ Encode v2 descriptors and send v2 publish requests
@@ -375,7 +352,7 @@
the next period. Publication is performed by sending the descriptor to all
hidden service directories that are responsible for keeping replicas for
the descriptor ID. This includes two non-consecutive replicas that are
- stored at 3 consecutive nodes each. (requires /1/, /2/, and /3/)
+ stored at 3 consecutive nodes each. (requires /1/ and /3/)
- rend-spec.txt, section 1.2: Added the new v2 hidden service descriptor
format.
@@ -388,20 +365,25 @@
- rendservice.c: Changed rend_consider_services_upload() to also initiate
the upload of v2 descriptors, if configured.
- rendservice.c: Extended rend_service_t by a member secret_cookie.
+ - rendcommon.c: Added rend_compute_v2_descriptor_fields() to prepare the
+ encoding of a v2 descriptor.
- rendcommon.c: Added rend_encode_v2_descriptor() to encode a v2
descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
- or.h: Added constant DIR_PURPOSE_UPLOAD_RENDDESC_V2.
- directory.c: Added directory_post_to_hs_dir().
- directory.c: Changed directory_initiate_command() to also recognize v2
publish requests.
- directory.c: Changed directory_send_command() to also prepare v2 publish
requests.
+ - directory.c: Changed directory_handle_command_post() to handle v2 publish
+ requests.
- crypto.c: Added implementation for crypto_cipher_encrypt_cbc().
- - Changed connection_dir_client_reached_eof().
+ [July 9: Base version specified and running; yet, replication is not
+ implemented, republication does not depend on publication periods, yet.]
- [Aug 11: Specified and running.]
-
Hidden service client:
/13/ Send v2 fetch requests
@@ -425,10 +407,10 @@
- rendcommon.c: Changed rend_cache_lookup_entry to enable it to also lookup
v2 descriptors.
- - rendcommon.c: Added rend_compute_v2_desc_id() to generate v2 descriptor IDs
+ - rendcommon.c: Added rend_compute_desc_id() to generate v2 descriptor IDs
from v2 onion addresses.
- rendcommon.c: Changed rend_valid_service_id() to also consider v2 onion
- addresses as valid and return the version number of the request (0 or 2).
+ addresses as valid and return the version number of the request (1 or 2).
- rendclient.c: Added rend_client_refetch_v2_renddesc() to fetch v2 service
descriptors using the secret cookie.
- rendclient.c: Changed rend_client_remove_intro_point() to copy the secret
@@ -443,14 +425,16 @@
fetch requests.
- directory.c: Changed directory_send_command() to also prepare v2 fetch
requests.
+ - directory.c: Changed directory_handle_command_get() to handle v2 fetch
+ requests.
- connection_edge.c: Changed connection_ap_handshake_rewrite_and_attach()
to fetch v2 service descriptors.
- connection_edge.c: Changed parse_extended_hostname() to accept both,
current and v2 onion addresses.
- config.c: Added config options FetchV2HidServDescriptors.
- [Aug 11: Base version specified and running, but no memory of failed
- hidden service directories, yet.]
+ [July 9: Base version specified and running in which only one node is
+ responsible for a specific descriptor ID.]
/14/ Process v2 fetch reply and parse v2 descriptors
@@ -470,14 +454,15 @@
introduction points of v2 hidden service descriptors.
- routerparse.c: Added desc_token_table[] to parse v2 hidden service
descriptors.
- - routerparse.c: Added 8 keywords to directory_keyword to parse v2 hidden
- service descriptors, and 5 to parse the decrypted list of introduction
- points.
+ - routerparse.c: Added 8 to directory_keyword to parse v2 hidden service
+ descriptors, and 5 to parse the decrypted list of introduction points.
- rendcommon.c: Added rend_cache_store_v2_client() to parse a v2 descriptor
and parse the encrypted list of introduction points.
- - or.h: Added rend_version and secret_cookie to edge_connection_t, to
- dir_connection_t, and to origin_circuit_t to be able to decrypt
- introduction points when receiving a v2 descriptor.
+ - or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
+ to origin_circuit_t to be able to decrypt introduction points when
+ receiving a v2 descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
- directory.c: Changed connection_dir_client_reached_eof() to also parse v2
fetch replies.
- crypto.c: Added implementation for crypto_cipher_decrypt_cbc().
@@ -507,6 +492,8 @@
- or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
to origin_circuit_t to be able to decrypt introduction points when
receiving a v2 descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
- circuitlist.c: Changed _circuit_mark_for_close() to pass the secret
cookie to rend_client_remove_intro_point() when an intro circ has failed.
- circuituse.c: Changed circuit_get_open_circ_or_launch() to fetch a v2
@@ -523,12 +510,12 @@
The new v2 hidden service descriptor format looks like this:
onion-address = h(public-key) + cookie
- descriptor-id = h(h(public-key) + h(time-period + cookie + relica))
+ descriptor-id = h(h(public-key) + h(time-period + cookie))
descriptor-content = {
descriptor-id,
version,
public-key,
- h(time-period + cookie + replica),
+ h(time-period + cookie),
timestamp,
protocol-versions,
{ introduction-points } encrypted with cookie
@@ -544,14 +531,13 @@
Therefore, "descriptor-id" is derived from the "public-key" of the hidden
service provider, the current "time-period" which changes every 24 hours,
- a secret "cookie" shared between hidden service provider and clients, and
- a "replica" denoting the number of this non-consecutive replica. (The
- "time-period" is constructed in a way that time periods do not change at
- the same moment for all descriptors by deriving a value between 0:00 and
- 23:59 hours from h(public-key) and making the descriptors of this hidden
+ and a secret "cookie" shared between hidden service provider and clients.
+ (The "time-period" is constructed in a way that time periods do not change
+ at the same moment for all descriptors by deriving a value between 0:00 and
+ 23:59 hours from "public-key" and making the descriptors of this hidden
service provider expire at that time of the day.) The "descriptor-id" is
defined to be 160 bits long. [extending the "descriptor-id" length
- suggested by LØ]
+ suggested by LO]
Only the hidden service provider and the clients are able to generate
future "descriptor-ID"s. Hence, the "onion-address" is extended from now
@@ -570,7 +556,7 @@
The "introduction-points" that are included in the descriptor are encrypted
using the same "cookie" that is shared between hidden service provider and
clients. [correction to use another key than h(time-period + cookie) as
- encryption key for introduction points made by LØ]
+ encryption key for introduction points made by LO]
A new text-based format is proposed for descriptors instead of an extension
of the existing binary format for reasons of future extensibility.
@@ -954,10 +940,4 @@
Added rend_decrypt_introduction_points() to decrypt and parse the list of
introduction points (/14/).
-Test:
-
- The changes were tested via test functions in test.c for separate,
- short-running functionality and using an automatic validation based on
- PuppeTor.
-
-
\ No newline at end of file
+
Modified: tor/branches/114-dist-storage/doc/spec/rend-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -198,9 +198,17 @@
KL is the length of PK, in octets.
TS is the number of seconds elapsed since Jan 1, 1970.
- The members of Ipt are identity key digests, encoded in hex, and
- prefixed with a '$'.
+ AUTHT specifies which authentication/authorization mechanism is
+ required by the hidden service or the introduction point. AUTHD
+ is arbitrary data that can be associated with an auth approach.
+ Currently only AUTHT of [00 00] is supported, with an AUTHL of 0.
+ See section 2 of this document for details on auth mechanisms.
+ The members of Ipt may be either (a) nicknames, or (b) identity key
+ digests, encoded in hex, and prefixed with a '$'. Clients must
+ accept both forms. Services must only generate the second form.
+ Once 0.0.9.x is obsoleted, we can drop the first form.
+
[It's ok for Bob to advertise 0 introduction points. He might want
to do that if he previously advertised some introduction points,
and now he doesn't have any. -RD]
@@ -209,7 +217,7 @@
descriptors. -NM]
Changes in version 2: Bob's OP generates v2 service descriptors in addition
- or as replacement to v1 service descriptors. The format of a v2 service
+ or as replacement to v0 service descriptors. The format of a v2 service
descriptor is as follows:
"rendezvous-service-descriptor" descriptor-id NL
@@ -297,14 +305,8 @@
"authentication" ...
+ [TODO114 include some means of authentication]
-[ AUTHT specifies which authentication/authorization mechanism is ]
-[ required by the hidden service or the introduction point. AUTHD ]
-[ is arbitrary data that can be associated with an auth approach. ]
-[ Currently only AUTHT of [00 00] is supported, with an AUTHL of 0. ]
-[ See section 2 of this document for details on auth mechanisms. ]
-
-
"introduction-points" NL encrypted-string
[Exactly once]
@@ -351,6 +353,7 @@
"authentication" ...
+ [TODO114 include some means of authentication]
"signature" NL signature-string
@@ -694,9 +697,9 @@
Every onion router that has its directory port open can decide whether it
wants to store and serve hidden service descriptors by setting a new config
- option "HSDir" 0|1 to 1. An onion router with this config option being set
- includes the flag "hidden-service-dir" in its router descriptors that it
- sends to directory authorities.
+ option "HidServDirectoryV2" 0|1 to 1. An onion router with this config
+ option being set includes the flag "hidden-service-dir" in its router
+ descriptors that it sends to directory authorities.
The directory authorities include a new flag "HSDir" for routers that
decided to provide storage for hidden service descriptors and that are
Modified: tor/branches/114-dist-storage/doc/tor-osx-dmg-creation.txt
===================================================================
--- tor/branches/114-dist-storage/doc/tor-osx-dmg-creation.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/tor-osx-dmg-creation.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -48,7 +48,7 @@
do a configure with these parameters:
CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \
--bindir=/Library/Tor --sysconfdir=/Library \
- --enable-static --disable-shared
+ --enable-eventdns --enable-static --disable-shared
3) In same top level dir, do a "make dist-osx". There now exists a
.dmg file in the same directory. Install from this dmg.
Modified: tor/branches/114-dist-storage/doc/tor-rpm-creation.txt
===================================================================
--- tor/branches/114-dist-storage/doc/tor-rpm-creation.txt 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/tor-rpm-creation.txt 2007-08-21 08:37:13 UTC (rev 11244)
@@ -53,3 +53,14 @@
This parameter controls the target operating system. Normally, this is
only "linux". If you wish to build rpms for a non-linux operating
system, you can replace "linux" with your operating system.
+
+The process used to create the distributed rpms is as follows:
+
+Download and Extract the latest tor source code from https://tor.eff.org/.
+In the Tor directory:
+./configure --enable-eventdns --enable-static --disable-shared
+make dist-rpm
+
+You should have at least two, maybe three, rpms. There should be the binary
+i386.rpm, a src.rpm, and on redhat/centos machines, a debuginfo.rpm.
+
Modified: tor/branches/114-dist-storage/doc/tor.1.in
===================================================================
--- tor/branches/114-dist-storage/doc/tor.1.in 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/doc/tor.1.in 2007-08-21 08:37:13 UTC (rev 11244)
@@ -101,6 +101,35 @@
Windows since that platform lacks getrlimit(). (Default: 1000)
.LP
.TP
+\fBConstrainedSockets \fR\fB0\fR|\fB1\fR\fP
+If set, Tor will tell the kernel to attempt to shrink the buffers for all
+sockets to the size specified in \fBConstrainedSockSize\fP. This is useful
+for virtual servers and other environments where system level TCP
+buffers may be limited. If you're on a virtual server, and you
+encounter the "Error creating network
+socket: No buffer space available" message, you are likely experiencing
+this problem.
+
+The preferred solution is to have the admin increase the buffer pool for
+the host itself via /proc/sys/net/ipv4/tcp_mem or equivalent facility; this
+configuration option is a second-resort.
+
+The DirPort option should also not be used if TCP buffers are scarce. The
+cached directory requests consume additional sockets which exacerbates the
+problem.
+
+You should \fBnot\fP enable this feature unless you encounter the "no buffer
+space available" issue. Reducing the TCP buffers affects window size for
+for the TCP stream and will reduce throughput in proportion to round trip
+time on long paths. (Default: 0.)
+.LP
+.TP
+\fBConstrainedSockSize \fR\fIN\fR \fBbytes\fR|\fBKB\fP
+When \fBConstrainedSockets\fP is enabled the receive and transmit buffers for
+all sockets will be set to this limit. Must be a value between 2048
+and 262144, in 1024 byte increments. Default of 8192 is recommended.
+.LP
+.TP
\fBControlPort \fR\fIPort\fP
If set, Tor will accept connections on this port and allow those
connections to control the Tor process using the Tor Control Protocol
@@ -923,7 +952,7 @@
accepts and serves hidden service descriptors. (Default: 0)
.LP
.TP
-\fBHSDir \fR\fB0\fR|\fB1\fR\fP
+\fBHidServDirectoryV2 \fR\fB0\fR|\fB1\fR\fP
When this option is set, Tor also accepts and serves v2 hidden service
descriptors. The Tor node does not need to be an authoritative directory for
this. (Default: 0)
Modified: tor/branches/114-dist-storage/src/common/aes.h
===================================================================
--- tor/branches/114-dist-storage/src/common/aes.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/aes.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -14,7 +14,7 @@
* \brief Headers for aes.c
*/
-#include "../common/torint.h"
+#include "torint.h"
struct aes_cnt_cipher;
typedef struct aes_cnt_cipher aes_cnt_cipher_t;
Modified: tor/branches/114-dist-storage/src/common/compat.c
===================================================================
--- tor/branches/114-dist-storage/src/common/compat.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/compat.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -101,6 +101,7 @@
#include "log.h"
#include "util.h"
+#include "container.h"
/* Inline the strl functions if the platform doesn't have them. */
#ifndef HAVE_STRLCPY
@@ -678,14 +679,14 @@
strerror(errno));
return -1;
}
- if (rlim.rlim_max < limit) {
+ if ((unsigned long)rlim.rlim_max < limit) {
log_warn(LD_CONFIG,"We need %lu file descriptors available, and we're "
"limited to %lu. Please change your ulimit -n.",
limit, (unsigned long)rlim.rlim_max);
return -1;
}
- most = (rlim.rlim_max > cap) ? cap : (unsigned) rlim.rlim_max;
- if (most > rlim.rlim_cur) {
+ most = ((unsigned long)rlim.rlim_max > cap) ? cap : (unsigned) rlim.rlim_max;
+ if (most > (unsigned long)rlim.rlim_cur) {
log_info(LD_NET,"Raising max file descriptors from %lu to %lu.",
(unsigned long)rlim.rlim_cur, most);
}
@@ -818,9 +819,6 @@
const char *
tor_inet_ntop(int af, const void *src, char *dst, size_t len)
{
-#ifdef HAVE_INET_NTOP
- return inet_ntop(af,src,dst,(socklen_t)len);
-#else
if (af == AF_INET) {
if (tor_inet_ntoa(src, dst, len) < 0)
return NULL;
@@ -836,7 +834,8 @@
words[i] = (((uint16_t)addr->s6_addr[2*i])<<8) + addr->s6_addr[2*i+1];
}
if (words[0] == 0 && words[1] == 0 && words[2] == 0 && words[3] == 0 &&
- words[4] == 0 && (words[5] == 0 || words[5] == 0xffff) && words[6]) {
+ words[4] == 0 && ((words[5] == 0 && words[6] && words[7]) ||
+ (words[5] == 0xffff))) {
/* This is an IPv4 address. */
if (words[5] == 0) {
tor_snprintf(buf, sizeof(buf), "::%d.%d.%d.%d",
@@ -895,7 +894,6 @@
} else {
return NULL;
}
-#endif
}
/** Given <b>af</b>==AF_INET or <b>af</b>==AF_INET6, and a string <b>src</b>
@@ -910,9 +908,6 @@
int
tor_inet_pton(int af, const char *src, void *dst)
{
-#ifdef HAVE_INET_PTON
- return inet_pton(af, src, dst);
-#else
if (af == AF_INET) {
return tor_inet_aton(src, dst);
} else if (af == AF_INET6) {
@@ -971,7 +966,7 @@
} else if (*src == ':' && i > 0 && gapPos==-1) {
gapPos = i;
++src;
- } else if (*src == ':' && i == 0 && src[1] == ':') {
+ } else if (*src == ':' && i == 0 && src[1] == ':' && gapPos==-1) {
gapPos = i;
src += 2;
} else {
@@ -979,7 +974,9 @@
}
}
- if (setWords > 8 || (setWords < 8 && gapPos == -1))
+ if (setWords > 8 ||
+ (setWords == 8 && gapPos != -1) ||
+ (setWords < 8 && gapPos == -1))
return 0;
if (gapPos >= 0) {
@@ -999,11 +996,10 @@
} else {
return -1;
}
-#endif
}
/** Similar behavior to Unix gethostbyname: resolve <b>name</b>, and set
- * *addr to the proper IP address, in network byte order. Returns 0
+ * *<b>addr</b> to the proper IP address, in host byte order. Returns 0
* on success, -1 on failure; 1 on transient failure.
*
* (This function exists because standard windows gethostbyname
@@ -1012,38 +1008,93 @@
int
tor_lookup_hostname(const char *name, uint32_t *addr)
{
+ tor_addr_t myaddr;
+ int ret;
+
+ if ((ret = tor_addr_lookup(name, AF_INET, &myaddr)))
+ return ret;
+
+ if (IN_FAMILY(&myaddr) == AF_INET) {
+ *addr = IPV4IPh(&myaddr);
+ return ret;
+ }
+
+ return -1;
+}
+
+/** Similar behavior to Unix gethostbyname: resolve <b>name</b>, and set
+ * *<b>addr</b> to the proper IP address and family. The <b>family</b>
+ * argument (which must be AF_INET, AF_INET6, or AF_UNSPEC) declares a
+ * <i>preferred</i> family, though another one may be returned if only one
+ * family is implemented for this address.
+ *
+ * Return 0 on success, -1 on failure; 1 on transient failure.
+ */
+int
+tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr)
+{
/* Perhaps eventually this should be replaced by a tor_getaddrinfo or
* something.
*/
struct in_addr iaddr;
+ struct in6_addr iaddr6;
tor_assert(name);
tor_assert(addr);
+ tor_assert(family == AF_INET || family == AF_UNSPEC);
+ memset(addr, 0, sizeof(addr)); /* Clear the extraneous fields. */
if (!*name) {
/* Empty address is an error. */
return -1;
- } else if (tor_inet_aton(name, &iaddr)) {
- /* It's an IP. */
- memcpy(addr, &iaddr.s_addr, 4);
+ } else if (tor_inet_pton(AF_INET, name, &iaddr)) {
+ /* It's an IPv4 IP. */
+ addr->sa.sin_family = AF_INET;
+ memcpy(&addr->sa.sin_addr, &iaddr, sizeof(struct in_addr));
return 0;
+ } else if (tor_inet_pton(AF_INET6, name, &iaddr6)) {
+ addr->sa6.sin6_family = AF_INET6;
+ memcpy(&addr->sa6.sin6_addr, &iaddr6, sizeof(struct in6_addr));
+ return 0;
} else {
#ifdef HAVE_GETADDRINFO
int err;
struct addrinfo *res=NULL, *res_p;
+ struct addrinfo *best=NULL;
struct addrinfo hints;
int result = -1;
memset(&hints, 0, sizeof(hints));
- hints.ai_family = PF_INET;
+ hints.ai_family = family;
hints.ai_socktype = SOCK_STREAM;
err = getaddrinfo(name, NULL, &hints, &res);
if (!err) {
+ best = NULL;
for (res_p = res; res_p; res_p = res_p->ai_next) {
- if (res_p->ai_family == AF_INET) {
- struct sockaddr_in *sin = (struct sockaddr_in *)res_p->ai_addr;
- memcpy(addr, &sin->sin_addr, 4);
- result = 0;
+ if (family == AF_UNSPEC) {
+ if (res_p->ai_family == AF_INET) {
+ best = res_p;
+ break;
+ } else if (res_p->ai_family == AF_INET6 && !best) {
+ best = res_p;
+ }
+ } else if (family == res_p->ai_family) {
+ best = res_p;
break;
}
}
+ if (!best)
+ best = res;
+ if (best->ai_family == AF_INET) {
+ addr->sa.sin_family = AF_INET;
+ memcpy(&addr->sa.sin_addr,
+ &((struct sockaddr_in*)best->ai_addr)->sin_addr,
+ sizeof(struct in_addr));
+ result = 0;
+ } else if (best->ai_family == AF_INET6) {
+ addr->sa6.sin6_family = AF_INET6;
+ memcpy(&addr->sa6.sin6_addr,
+ &((struct sockaddr_in6*)best->ai_addr)->sin6_addr,
+ sizeof(struct in6_addr));
+ result = 0;
+ }
freeaddrinfo(res);
return result;
}
@@ -1073,14 +1124,19 @@
#else
err = h_errno;
#endif
-#endif
+#endif /* endif HAVE_GETHOSTBYNAME_R_6_ARG. */
if (ent) {
- /* break to remind us if we move away from IPv4 */
- tor_assert(ent->h_length == 4);
- memcpy(addr, ent->h_addr, 4);
+ addr->sa.sin_family = ent->h_addrtype;
+ if (ent->h_addrtype == AF_INET) {
+ memcpy(&addr->sa.sin_addr, ent->h_addr, sizeof(struct in_addr));
+ } else if (ent->h_addrtype == AF_INET6) {
+ memcpy(&addr->sa6.sin6_addr, ent->h_addr, sizeof(struct in6_addr));
+ } else {
+ tor_assert(0); /* gethostbyname() returned a bizarre addrtype */
+ }
return 0;
}
- memset(addr, 0, 4);
+ memset(addr, 0, sizeof(tor_addr_t)); /* XXXX020 is this redundant? */
#ifdef MS_WINDOWS
return (err == WSATRY_AGAIN) ? 1 : -1;
#else
@@ -1439,7 +1495,7 @@
#endif
#endif
-#ifdef USE_WIN32_THREADS
+#if defined(USE_WIN32_THREADS) && 0
/** A generic lock structure for multithreaded builds. */
struct tor_mutex_t {
HANDLE handle;
@@ -1489,6 +1545,40 @@
{
return (unsigned long)GetCurrentThreadId();
}
+#elif defined(USE_WIN32_THREADS)
+/** A generic lock structure for multithreaded builds. */
+struct tor_mutex_t {
+ CRITICAL_SECTION mutex;
+};
+tor_mutex_t *
+tor_mutex_new(void)
+{
+ tor_mutex_t *m = tor_malloc_zero(sizeof(tor_mutex_t));
+ InitializeCriticalSection(&m->mutex);
+ return m;
+}
+void
+tor_mutex_free(tor_mutex_t *m)
+{
+ DeleteCriticalSection(&m->mutex);
+ tor_free(m);
+}
+void
+tor_mutex_acquire(tor_mutex_t *m)
+{
+ tor_assert(m);
+ EnterCriticalSection(&m->mutex);
+}
+void
+tor_mutex_release(tor_mutex_t *m)
+{
+ LeaveCriticalSection(&m->mutex);
+}
+unsigned long
+tor_get_thread_id(void)
+{
+ return (unsigned long)GetCurrentThreadId();
+}
#elif defined(USE_PTHREADS)
/** A generic lock structure for multithreaded builds. */
struct tor_mutex_t {
@@ -1562,6 +1652,139 @@
};
#endif
+/* Condition stuff. DOCDOC */
+#ifdef USE_PTHREADS
+struct tor_cond_t {
+ pthread_cond_t cond;
+};
+tor_cond_t *
+tor_cond_new(void)
+{
+ tor_cond_t *cond = tor_malloc_zero(sizeof(tor_cond_t));
+ if (pthread_cond_init(&cond->cond, NULL)) {
+ tor_free(cond);
+ return NULL;
+ }
+ return cond;
+}
+void
+tor_conf_free(tor_cond_t *cond)
+{
+ tor_assert(cond);
+ if (pthread_cond_destroy(&cond->cond)) {
+ log_warn(LD_GENERAL,"Error freeing condition: %s", strerror(errno));
+ return;
+ }
+ tor_free(cond);
+}
+int
+tor_cond_wait(tor_cond_t *cond, tor_mutex_t *mutex)
+{
+ return pthread_cond_wait(&cond->cond, &mutex->mutex) ? -1 : 0;
+}
+void
+tor_cond_signal_one(tor_cond_t *cond)
+{
+ pthread_cond_signal(&cond->cond);
+}
+void
+tor_cond_signal_all(tor_cond_t *cond)
+{
+ pthread_cond_broadcast(&cond->cond);
+}
+void
+tor_threads_init(void)
+{
+}
+#elif defined(USE_WIN32_THREADS)
+static DWORD cond_event_tls_index;
+struct tor_cond_t {
+ CRITICAL_SECTION mutex;
+ smartlist_t *events;
+};
+tor_cond_t *
+tor_cond_new(void)
+{
+ tor_cond_t *cond = tor_malloc_zero(sizeof(tor_cond_t));
+ InitializeCriticalSection(&cond->mutex);
+ cond->events = smartlist_create();
+ return cond;
+}
+void
+tor_cond_free(tor_cond_t *cond)
+{
+ tor_assert(cond);
+ DeleteCriticalSection(&cond->mutex);
+ /* XXXX020 notify? */
+ smartlist_free(cond->events);
+ tor_free(cond);
+}
+int
+tor_cond_wait(tor_cond_t *cond, tor_mutex_t *mutex)
+{
+ HANDLE event;
+ int r;
+ tor_assert(cond);
+ tor_assert(mutex);
+ event = TlsGetValue(cond_event_tls_index);
+ if (!event) {
+ event = CreateEvent(0, FALSE, FALSE, NULL);
+ TlsSetValue(cond_event_tls_index, event);
+ }
+ EnterCriticalSection(&cond->mutex);
+
+ tor_assert(WaitForSingleObject(event, 0) == WAIT_TIMEOUT);
+ tor_assert(!smartlist_isin(cond->events, event));
+ smartlist_add(cond->events, event);
+
+ LeaveCriticalSection(&cond->mutex);
+
+ tor_mutex_release(mutex);
+ r = WaitForSingleObject(event, INFINITE);
+ tor_mutex_acquire(mutex);
+
+ switch (r) {
+ case WAIT_OBJECT_0: /* we got the mutex normally. */
+ break;
+ case WAIT_ABANDONED: /* holding thread exited. */
+ case WAIT_TIMEOUT: /* Should never happen. */
+ tor_assert(0);
+ break;
+ case WAIT_FAILED:
+ log_warn(LD_GENERAL, "Failed to acquire mutex: %d",(int) GetLastError());
+ }
+ return 0;
+}
+void
+tor_cond_signal_one(tor_cond_t *cond)
+{
+ HANDLE event;
+ tor_assert(cond);
+
+ EnterCriticalSection(&cond->mutex);
+
+ if ((event = smartlist_pop_last(cond->events)))
+ SetEvent(event);
+
+ LeaveCriticalSection(&cond->mutex);
+}
+void
+tor_cond_signal_all(tor_cond_t *cond)
+{
+ tor_assert(cond);
+
+ EnterCriticalSection(&cond->mutex);
+ SMARTLIST_FOREACH(cond->events, HANDLE, event, SetEvent(event));
+ smartlist_clear(cond->events);
+ LeaveCriticalSection(&cond->mutex);
+}
+void
+tor_threads_init(void)
+{
+ cond_event_tls_index = TlsAlloc();
+}
+#endif
+
/**
* On Windows, WSAEWOULDBLOCK is not always correct: when you see it,
* you need to ask the socket for its actual errno. Also, you need to
Modified: tor/branches/114-dist-storage/src/common/compat.h
===================================================================
--- tor/branches/114-dist-storage/src/common/compat.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/compat.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -36,6 +36,15 @@
#include <ctype.h>
#endif
#include <stdarg.h>
+#ifdef HAVE_SYS_SOCKET_H
+#include <sys/socket.h>
+#endif
+#ifdef HAVE_NETINET_IN_H
+#include <netinet/in.h>
+#endif
+#ifdef HAVE_NETINET6_IN6_H
+#include <netinet6/in6.h>
+#endif
#ifndef NULL_REP_IS_ZERO_BYTES
#error "It seems your platform does not represent NULL as zero. We can't cope."
@@ -131,7 +140,7 @@
#define STMT_END } else STMT_NIL
#else
#define STMT_BEGIN do {
-#define STMT_END } while(0)
+#define STMT_END } while (0)
#endif
#endif
@@ -263,11 +272,104 @@
#if !defined(HAVE_STRUCT_IN6_ADDR) && !defined(MS_WINDOWS)
struct in6_addr
{
- uint8_t s6_addr[16];
+ union {
+ uint8_t u6_addr8[16];
+ uint16_t u6_addr16[8];
+ uint32_t u6_addr32[4];
+ } in6_u;
+#define s6_addr in6_u.u6_addr8
+#define s6_addr16 in6_u.u6_addr16
+#define s6_addr32 in6_u.u6_addr32
};
#endif
+#if defined(__APPLE__) || defined(__darwin__) || defined(__FreeBSD__) \
+ || defined(__NetBSD__) || defined(__OpenBSD__)
+/* Many BSD variants seem not to define these. */
+#ifndef s6_addr16
+#define s6_addr16 __u6_addr.__u6_addr16
+#endif
+#ifndef s6_addr32
+#define s6_addr32 __u6_addr.__u6_addr32
+#endif
+#endif
+
+#ifndef HAVE_SA_FAMILY_T
+typedef uint16_t sa_family_t;
+#endif
+
+/* XXXX020 detect sockaddr_in6 correctly on ms_windows; this is also a hack. */
+#if !defined(HAVE_STRUCT_SOCKADDR_IN6) && !defined(MS_WINDOWS)
+struct sockaddr_in6 {
+ sa_family_t sin6_family;
+ uint16_t sin6_port;
+ // uint32_t sin6_flowinfo;
+ struct in6_addr sin6_addr;
+ // uint32_t sin6_scope_id;
+};
+#endif
+
+typedef uint8_t maskbits_t;
struct in_addr;
+typedef union tor_addr_t
+{
+ /* XXXX020 There are extra fields in sockaddr_in and sockaddr_in6 that
+ * make this union waste space. Do we care? */
+ struct sockaddr_in sa;
+ struct sockaddr_in6 sa6;
+} tor_addr_t;
+
+/* XXXX020 rename these. */
+static INLINE uint32_t IPV4IP(const tor_addr_t *a);
+static INLINE uint32_t IPV4IPh(const tor_addr_t *a);
+static INLINE uint32_t IPV4MAPh(const tor_addr_t *a);
+static INLINE uint16_t IN_FAMILY(const tor_addr_t *a);
+static INLINE const struct in_addr *IN4_ADDRESS(const tor_addr_t *a);
+static INLINE const struct in6_addr *IN6_ADDRESS(const tor_addr_t *a);
+static INLINE uint16_t IN_PORT(const tor_addr_t *a);
+
+static INLINE uint32_t
+IPV4IP(const tor_addr_t *a)
+{
+ return a->sa.sin_addr.s_addr;
+}
+static INLINE uint32_t IPV4IPh(const tor_addr_t *a)
+{
+ /*XXXX020 remove this function */
+ return ntohl(IPV4IP(a));
+}
+static INLINE uint32_t
+IPV4MAPh(const tor_addr_t *a)
+{
+ return ntohl(a->sa6.sin6_addr.s6_addr32[3]);
+}
+static INLINE uint16_t
+IN_FAMILY(const tor_addr_t *a)
+{
+ return a->sa.sin_family;
+}
+static INLINE const struct in_addr *
+IN4_ADDRESS(const tor_addr_t *a)
+{
+ return &a->sa.sin_addr;
+}
+static INLINE const struct in6_addr *
+IN6_ADDRESS(const tor_addr_t *a)
+{
+ return &a->sa6.sin6_addr;
+}
+static INLINE uint16_t
+IN_PORT(const tor_addr_t *a)
+{
+ if (IN_FAMILY(a) == AF_INET)
+ return a->sa.sin_port;
+ else
+ return a->sa6.sin6_port;
+}
+
+#define INET_NTOA_BUF_LEN 16 /* 255.255.255.255 */
+#define TOR_ADDR_BUF_LEN 46 /* ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255 */
+
int tor_inet_aton(const char *cp, struct in_addr *addr) ATTR_NONNULL((1,2));
const char *tor_inet_ntop(int af, const void *src, char *dst, size_t len);
int tor_inet_pton(int af, const char *src, void *dst);
@@ -275,6 +377,9 @@
void set_socket_nonblocking(int socket);
int tor_socketpair(int family, int type, int protocol, int fd[2]);
int network_init(void);
+
+int tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr_out);
+
/* For stupid historical reasons, windows sockets have an independent
* set of errnos, and an independent way to get them. Also, you can't
* always believe WSAEWOULDBLOCK. Use the macros below to compare
@@ -360,6 +465,16 @@
#define tor_get_thread_id() (1UL)
#endif
+#ifdef TOR_IS_MULTITHREADED
+typedef struct tor_cond_t tor_cond_t;
+tor_cond_t *tor_cond_new(void);
+void tor_conf_free(tor_cond_t *cond);
+int tor_cond_wait(tor_cond_t *cond, tor_mutex_t *mutex);
+void tor_cond_signal_one(tor_cond_t *cond);
+void tor_cond_signal_all(tor_cond_t *cond);
+void tor_threads_init(void);
+#endif
+
/* Platform-specific helpers. */
#ifdef MS_WINDOWS
char *format_win32_error(DWORD err);
Modified: tor/branches/114-dist-storage/src/common/container.c
===================================================================
--- tor/branches/114-dist-storage/src/common/container.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/container.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -196,7 +196,8 @@
return 0;
}
-/** DOCDOC */
+/** If <b>element</b> is equal to an element of <b>sl</b>, return that
+ * element's index. Otherwise, return -1. */
int
smartlist_string_pos(const smartlist_t *sl, const char *element)
{
@@ -247,25 +248,6 @@
return 0;
}
-/** Assuming the members of <b>sl</b> are in ascending order, return the
- * first element that is greater than <b>id</b>; if all elements are smaller
- * than <b>id</b>, return the first element in <b>sl</b>; if <b>sl</b> is
- * NULL, or has no elements, return NULL.
- */
-const char *
-smartlist_digest_next_circular(const smartlist_t *sl, const char *id)
-{
- int i;
- if (!sl) return NULL;
- if (sl->num_used == 0) return NULL;
- for (i=0; i < sl->num_used; i++) {
- if (memcmp((const char*)sl->list[i],id,DIGEST_LEN) > 0) {
- return (const char*)sl->list[i];
- }
- }
- return (const char*)sl->list[0];
-}
-
/** Return true iff some element E of sl2 has smartlist_isin(sl1,E).
*/
int
@@ -787,8 +769,6 @@
{
#ifndef OPTIMIZED_DIGESTMAP_SET
digestmap_entry_t *resolve;
-#else
- digestmap_entry_t **resolve_ptr;
#endif
digestmap_entry_t search;
void *oldval;
@@ -810,31 +790,28 @@
return NULL;
}
#else
- /* XXXX020 We spend up to 5% of our time in this function, so the code
- * below is meant to optimize the check/alloc/set cycle by avoiding the
- * two trips to the hash table that we do in the unoptimized code above.
- * (Each of HT_INSERT and HT_FIND calls HT_SET_HASH and HT_FIND_P.)
- *
- * Unfortunately, doing this requires us to poke around inside hash-table
- * internals. It would be nice to avoid that. */
- if (!map->head.hth_table ||
- map->head.hth_n_entries >= map->head.hth_load_limit)
- digestmap_impl_HT_GROW((&map->head), map->head.hth_n_entries+1);
- _HT_SET_HASH(&search, node, digestmap_entry_hash);
- resolve_ptr = _digestmap_impl_HT_FIND_P(&map->head, &search);
- if (*resolve_ptr) {
- oldval = (*resolve_ptr)->val;
- (*resolve_ptr)->val = val;
- return oldval;
- } else {
- digestmap_entry_t *newent = tor_malloc_zero(sizeof(digestmap_entry_t));
- memcpy(newent->key, key, DIGEST_LEN);
- newent->val = val;
- newent->node.hte_hash = search.node.hte_hash;
- *resolve_ptr = newent;
- ++map->head.hth_n_entries;
- return NULL;
- }
+ /* We spend up to 5% of our time in this function, so the code below is
+ * meant to optimize the check/alloc/set cycle by avoiding the two trips to
+ * the hash table that we do in the unoptimized code above. (Each of
+ * HT_INSERT and HT_FIND calls HT_SET_HASH and HT_FIND_P.)
+ */
+ _HT_FIND_OR_INSERT(digestmap_impl, node, digestmap_entry_hash, &(map->head),
+ digestmap_entry_t, &search, ptr,
+ {
+ /* we found an entry. */
+ oldval = (*ptr)->val;
+ (*ptr)->val = val;
+ return oldval;
+ },
+ {
+ /* We didn't find the entry. */
+ digestmap_entry_t *newent =
+ tor_malloc_zero(sizeof(digestmap_entry_t));
+ memcpy(newent->key, key, DIGEST_LEN);
+ newent->val = val;
+ _HT_FOI_INSERT(node, &(map->head), &search, newent, ptr);
+ return NULL;
+ });
#endif
}
Modified: tor/branches/114-dist-storage/src/common/container.h
===================================================================
--- tor/branches/114-dist-storage/src/common/container.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/container.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -8,7 +8,6 @@
#define CONTAINER_H_ID \
"$Id$"
-#include "compat.h"
#include "util.h"
/** A resizeable list of pointers, with associated helpful functionality.
@@ -46,8 +45,6 @@
int smartlist_string_num_isin(const smartlist_t *sl, int num) ATTR_PURE;
int smartlist_digest_isin(const smartlist_t *sl, const char *element)
ATTR_PURE;
-const char *smartlist_digest_next_circular(const smartlist_t *sl,
- const char *id);
int smartlist_overlap(const smartlist_t *sl1, const smartlist_t *sl2)
ATTR_PURE;
void smartlist_intersect(smartlist_t *sl1, const smartlist_t *sl2);
@@ -269,5 +266,50 @@
return digestmap_iter_done((digestmap_iter_t*)iter); \
}
+#if SIZEOF_INT == 4
+#define BITARRAY_SHIFT 5
+#define BITARRAY_MASK 31
+#elif SIZEOF_INT == 8
+#define BITARRAY_SHIFT 6
+#define BITARRAY_MASK 63
+#else
+#error "int is neither 4 nor 8 bytes. I can't deal with that."
#endif
+/** A random-access array of one-bit-wide elements. */
+typedef unsigned int bitarray_t;
+/** Create a new bit array that can hold <b>n_bits</b> bits. */
+static INLINE bitarray_t *
+bitarray_init_zero(int n_bits)
+{
+ size_t sz = (n_bits+BITARRAY_MASK) / (1u << BITARRAY_SHIFT);
+ return tor_malloc_zero(sz*sizeof(unsigned int));
+}
+/** Free the bit array <b>ba</b>. */
+static INLINE void
+bitarray_free(bitarray_t *ba)
+{
+ tor_free(ba);
+}
+/** Set the <b>bit</b>th bit in <b>b</b> to 1. */
+static INLINE void
+bitarray_set(bitarray_t *b, int bit)
+{
+ b[bit >> BITARRAY_SHIFT] |= (1u << (bit & BITARRAY_MASK));
+}
+/** Set the <b>bit</b>th bit in <b>b</b> to 0. */
+static INLINE void
+bitarray_clear(bitarray_t *b, int bit)
+{
+ b[bit >> BITARRAY_SHIFT] &= ~ (1u << (bit & BITARRAY_MASK));
+}
+/** Return true iff <b>bit</b>th bit in <b>b</b> is nonzero. NOTE: does
+ * not necessarily return 1 on true. */
+static INLINE unsigned int
+bitarray_is_set(bitarray_t *b, int bit)
+{
+ return b[bit >> BITARRAY_SHIFT] & (1u << (bit & BITARRAY_MASK));
+}
+
+#endif
+
Modified: tor/branches/114-dist-storage/src/common/crypto.c
===================================================================
--- tor/branches/114-dist-storage/src/common/crypto.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/crypto.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1164,7 +1164,7 @@
tor_assert(from);
tor_assert(fromlen);
- /* initialize cipher contex */
+ /* initialize cipher context */
EVP_CIPHER_CTX_init(&ctx);
/* generate random initialization vector and write it to the first 16 bytes
Modified: tor/branches/114-dist-storage/src/common/ht.h
===================================================================
--- tor/branches/114-dist-storage/src/common/ht.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/ht.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -382,6 +382,31 @@
return 0; \
}
+/** Implements an over-optimized "find and insert if absent" block;
+ * not meant for direct usage by typical code, or usage outside the critical
+ * path.*/
+#define _HT_FIND_OR_INSERT(name, field, hashfn, head, eltype, elm, var, y, n) \
+ { \
+ struct name *_##var##_head = head; \
+ eltype **var; \
+ if (!_##var##_head->hth_table || \
+ _##var##_head->hth_n_entries >= _##var##_head->hth_load_limit) \
+ name##_HT_GROW(_##var##_head, _##var##_head->hth_n_entries+1); \
+ _HT_SET_HASH((elm), field, hashfn); \
+ var = _##name##_HT_FIND_P(_##var##_head, (elm)); \
+ if (*var) { \
+ y; \
+ } else { \
+ n; \
+ } \
+ }
+#define _HT_FOI_INSERT(field, head, elm, newent, var) \
+ { \
+ newent->field.hte_hash = (elm)->field.hte_hash; \
+ *var = newent; \
+ ++((head)->hth_n_entries); \
+ }
+
/*
* Copyright 2005, Nick Mathewson. Implementation logic is adapted from code
* by Cristopher Clark, retrofit to allow drop-in memory management, and to
Modified: tor/branches/114-dist-storage/src/common/log.c
===================================================================
--- tor/branches/114-dist-storage/src/common/log.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/log.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -361,9 +361,9 @@
}
#endif
-/** Close all open log files. */
+/** Close all open log files, and free other static memory. */
void
-close_logs(void)
+logs_free_all(void)
{
logfile_t *victim, *next;
next = logfiles;
@@ -375,6 +375,7 @@
tor_free(victim->filename);
tor_free(victim);
}
+ tor_free(appname);
}
/** Remove and free the log entry <b>victim</b> from the linked-list
Modified: tor/branches/114-dist-storage/src/common/log.h
===================================================================
--- tor/branches/114-dist-storage/src/common/log.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/log.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -13,7 +13,7 @@
#ifndef __LOG_H
#define LOG_H_ID "$Id$"
-#include "../common/compat.h"
+#include "compat.h"
#ifdef HAVE_SYSLOG_H
#include <syslog.h>
@@ -104,7 +104,7 @@
int add_callback_log(int loglevelMin, int loglevelMax, log_callback cb);
int get_min_log_level(void);
void switch_logs_debug(void);
-void close_logs(void);
+void logs_free_all(void);
void add_temp_log(void);
void close_temp_logs(void);
void rollback_log_changes(void);
Modified: tor/branches/114-dist-storage/src/common/test.h
===================================================================
--- tor/branches/114-dist-storage/src/common/test.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/test.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -12,8 +12,6 @@
* \brief Macros used by unit tests.
*/
-#include <string.h>
-#include <stdio.h>
#include "compat.h"
#ifdef __GNUC__
Modified: tor/branches/114-dist-storage/src/common/torgzip.c
===================================================================
--- tor/branches/114-dist-storage/src/common/torgzip.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/torgzip.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -136,6 +136,10 @@
}
done:
*out_len = stream->total_out;
+ if (stream->total_out > out_size + 4097) {
+ /* If we're wasting more than 4k, don't. */
+ tor_realloc(*out, stream->total_out + 1);
+ }
if (deflateEnd(stream)!=Z_OK) {
log_warn(LD_BUG, "Error freeing gzip structures");
goto err;
Modified: tor/branches/114-dist-storage/src/common/tortls.c
===================================================================
--- tor/branches/114-dist-storage/src/common/tortls.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/tortls.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -274,7 +274,7 @@
if ((nid = OBJ_txt2nid("organizationName")) == NID_undef)
goto error;
if (!(X509_NAME_add_entry_by_NID(name, nid, MBSTRING_ASC,
- (unsigned char*)"Tor", -1, -1, 0)))
+ (unsigned char*)"t o r", -1, -1, 0)))
goto error;
if ((nid = OBJ_txt2nid("commonName")) == NID_undef) goto error;
if (!(X509_NAME_add_entry_by_NID(name, nid, MBSTRING_ASC,
@@ -288,7 +288,7 @@
if ((nid = OBJ_txt2nid("organizationName")) == NID_undef)
goto error;
if (!(X509_NAME_add_entry_by_NID(name_issuer, nid, MBSTRING_ASC,
- (unsigned char*)"Tor", -1, -1, 0)))
+ (unsigned char*)"t o r", -1, -1, 0)))
goto error;
if ((nid = OBJ_txt2nid("commonName")) == NID_undef) goto error;
if (!(X509_NAME_add_entry_by_NID(name_issuer, nid, MBSTRING_ASC,
@@ -361,7 +361,7 @@
char nn2[128];
if (!nickname)
nickname = "null";
- tor_snprintf(nn2, sizeof(nn2), "%s <identity>", nickname);
+ tor_snprintf(nn2, sizeof(nn2), "%s <signing>", nickname);
tor_tls_init();
Modified: tor/branches/114-dist-storage/src/common/tortls.h
===================================================================
--- tor/branches/114-dist-storage/src/common/tortls.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/tortls.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -12,8 +12,8 @@
* \brief Headers for tortls.c
**/
-#include "../common/crypto.h"
-#include "../common/compat.h"
+#include "crypto.h"
+#include "compat.h"
/* Opaque structure to hold a TLS connection. */
typedef struct tor_tls_t tor_tls_t;
Modified: tor/branches/114-dist-storage/src/common/util.c
===================================================================
--- tor/branches/114-dist-storage/src/common/util.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/util.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -68,6 +68,9 @@
#ifdef HAVE_TIME_H
#include <time.h>
#endif
+#if defined(HAVE_MALLOC_H) && defined(HAVE_MALLINFO)
+#include <malloc.h>
+#endif
#ifndef O_BINARY
#define O_BINARY 0
@@ -213,6 +216,26 @@
tor_free(mem);
}
+/** DOCDOC */
+void
+tor_log_mallinfo(int severity)
+{
+#ifdef HAVE_MALLINFO
+ struct mallinfo mi;
+ memset(&mi, 0, sizeof(mi));
+ mi = mallinfo();
+ log(severity, LD_MM,
+ "mallinfo() said: arena=%d, ordblks=%d, smblks=%d, hblks=%d, "
+ "hblkhd=%d, usmblks=%d, fsmblks=%d, uordblks=%d, fordblks=%d, "
+ "keepcost=%d",
+ mi.arena, mi.ordblks, mi.smblks, mi.hblks,
+ mi.hblkhd, mi.usmblks, mi.fsmblks, mi.uordblks, mi.fordblks,
+ mi.keepcost);
+#else
+ (void)severity;
+#endif
+}
+
/* =====
* Math
* ===== */
@@ -403,6 +426,21 @@
return strncmp(s1, s2, n);
}
+/** Compare the s1_len-byte string <b>s1</b> with <b>s2</b>,
+ * without depending on a terminating nul in s1. Sorting order is first by
+ * length, then lexically; return values are as for strcmp.
+ */
+int
+strcmp_len(const char *s1, const char *s2, size_t s1_len)
+{
+ size_t s2_len = strlen(s2);
+ if (s1_len < s2_len)
+ return -1;
+ if (s1_len > s2_len)
+ return 1;
+ return memcmp(s1, s2, s2_len);
+}
+
/** Compares the first strlen(s2) characters of s1 with s2. Returns as for
* strcasecmp.
*/
@@ -505,6 +543,16 @@
return s;
}
+/** As eat_whitespace_no_nl, but stop at <b>eos</b> whether we have
+ * found a non-whitespace character or not. */
+const char *
+eat_whitespace_eos_no_nl(const char *s, const char *eos)
+{
+ while (s < eos && (*s == ' ' || *s == '\t'))
+ ++s;
+ return s;
+}
+
/** Return a pointer to the first char of s that is whitespace or <b>#</b>,
* or to the terminating NUL if no such character exists.
*/
@@ -528,6 +576,29 @@
}
}
+/** As find_whitespace, but stop at <b>eos</b> whether we have found a
+ * whitespace or not. */
+const char *
+find_whitespace_eos(const char *s, const char *eos)
+{
+ /* tor_assert(s); */
+ while (s < eos) {
+ switch (*s)
+ {
+ case '\0':
+ case '#':
+ case ' ':
+ case '\r':
+ case '\n':
+ case '\t':
+ return s;
+ default:
+ ++s;
+ }
+ }
+ return s;
+}
+
/** Return true iff the 'len' bytes at 'mem' are all zero. */
int
tor_mem_is_zero(const char *mem, size_t len)
@@ -1780,18 +1851,96 @@
int
is_internal_IP(uint32_t ip, int for_listening)
{
- if (for_listening && !ip) /* special case for binding to 0.0.0.0 */
+ tor_addr_t myaddr;
+ myaddr.sa.sin_family = AF_INET;
+ myaddr.sa.sin_addr.s_addr = htonl(ip);
+
+ return tor_addr_is_internal(&myaddr, for_listening);
+}
+
+/** Return true iff <b>ip</b> is an IP reserved to localhost or local networks
+ * in RFC1918 or RFC4193 or RFC4291. (fec0::/10, deprecated by RFC3879, is
+ * also treated as internal for now.)
+ */
+int
+tor_addr_is_internal(const tor_addr_t *addr, int for_listening)
+{
+ uint32_t iph4 = 0;
+ uint32_t iph6[4];
+ sa_family_t v_family;
+ v_family = IN_FAMILY(addr);
+
+ if (v_family == AF_INET) {
+ iph4 = IPV4IPh(addr);
+ } else if (v_family == AF_INET6) {
+ if (tor_addr_is_v4(addr)) { /* v4-mapped */
+ v_family = AF_INET;
+ iph4 = ntohl(IN6_ADDRESS(addr)->s6_addr32[3]);
+ }
+ }
+
+ if (v_family == AF_INET6) {
+ iph6[0] = ntohl(IN6_ADDRESS(addr)->s6_addr32[0]);
+ iph6[1] = ntohl(IN6_ADDRESS(addr)->s6_addr32[1]);
+ iph6[2] = ntohl(IN6_ADDRESS(addr)->s6_addr32[2]);
+ iph6[3] = ntohl(IN6_ADDRESS(addr)->s6_addr32[3]);
+ if (for_listening && !iph6[0] && !iph6[1] && !iph6[2] && !iph6[3]) /* :: */
+ return 0;
+
+ if (((iph6[0] & 0xfe000000) == 0xfc000000) || /* fc00/7 - RFC4193 */
+ ((iph6[0] & 0xffc00000) == 0xfe800000) || /* fe80/10 - RFC4291 */
+ ((iph6[0] & 0xffc00000) == 0xfec00000)) /* fec0/10 D- RFC3879 */
+ return 1;
+
+ if (!iph6[0] && !iph6[1] && !iph6[2] &&
+ ((iph6[3] & 0xfffffffe) == 0x00000000)) /* ::/127 */
+ return 1;
+
return 0;
- if (((ip & 0xff000000) == 0x0a000000) || /* 10/8 */
- ((ip & 0xff000000) == 0x00000000) || /* 0/8 */
- ((ip & 0xff000000) == 0x7f000000) || /* 127/8 */
- ((ip & 0xffff0000) == 0xa9fe0000) || /* 169.254/16 */
- ((ip & 0xfff00000) == 0xac100000) || /* 172.16/12 */
- ((ip & 0xffff0000) == 0xc0a80000)) /* 192.168/16 */
- return 1;
- return 0;
+ } else if (v_family == AF_INET) {
+ if (for_listening && !iph4) /* special case for binding to 0.0.0.0 */
+ return 0;
+ if (((iph4 & 0xff000000) == 0x0a000000) || /* 10/8 */
+ ((iph4 & 0xff000000) == 0x00000000) || /* 0/8 */
+ ((iph4 & 0xff000000) == 0x7f000000) || /* 127/8 */
+ ((iph4 & 0xffff0000) == 0xa9fe0000) || /* 169.254/16 */
+ ((iph4 & 0xfff00000) == 0xac100000) || /* 172.16/12 */
+ ((iph4 & 0xffff0000) == 0xc0a80000)) /* 192.168/16 */
+ return 1;
+ return 0;
+ }
+
+ /* unknown address family... assume it's not safe for external use */
+ /* rather than tor_assert(0) */
+ log_warn(LD_BUG, "tor_addr_is_internal() called with a non-IP address.");
+ return 1;
}
+#if 0
+/** Convert a tor_addr_t <b>addr</b> into a string, and store it in
+ * <b>dest</b> of size <b>len</b>. Returns a pointer to dest on success,
+ * or NULL on failure.
+ */
+void
+tor_addr_to_str(char *dest, const tor_addr_t *addr, int len)
+{
+ const char *ptr;
+ tor_assert(addr && dest);
+
+ switch (IN_FAMILY(addr)) {
+ case AF_INET:
+ ptr = tor_inet_ntop(AF_INET, &addr->sa.sin_addr, dest, len);
+ break;
+ case AF_INET6:
+ ptr = tor_inet_ntop(AF_INET6, &addr->sa6.sin6_addr, dest, len);
+ break;
+ default:
+ return NULL;
+ }
+ return ptr;
+}
+#endif
+
/** Parse a string of the form "host[:port]" from <b>addrport</b>. If
* <b>address</b> is provided, set *<b>address</b> to a copy of the
* host portion of the string. If <b>addr</b> is provided, try to
@@ -1841,7 +1990,6 @@
ok = 0;
*addr = 0;
}
- *addr = ntohl(*addr);
}
if (address && ok) {
@@ -1875,6 +2023,32 @@
return -1;
}
+/** Compare two addresses <b>a1</b> and <b>a2</b> for equality under a
+ * etmask of <b>mbits</b> bits. Return -1, 0, or 1.
+ *
+ * XXXX020Temporary function to allow masks as bitcounts everywhere. This
+ * will be replaced with an IPv6-aware version as soon as 32-bit addresses are
+ * no longer passed around.
+ */
+int
+addr_mask_cmp_bits(uint32_t a1, uint32_t a2, maskbits_t bits)
+{
+ if (bits > 32)
+ bits = 32;
+ else if (bits == 0)
+ return 0;
+
+ a1 >>= (32-bits);
+ a2 >>= (32-bits);
+
+ if (a1 < a2)
+ return -1;
+ else if (a1 > a2)
+ return 1;
+ else
+ return 0;
+}
+
/** Parse a string <b>s</b> in the format of (*|port(-maxport)?)?, setting the
* various *out pointers as appropriate. Return 0 on success, -1 on failure.
*/
@@ -1933,7 +2107,7 @@
*/
int
parse_addr_and_port_range(const char *s, uint32_t *addr_out,
- uint32_t *mask_out, uint16_t *port_min_out,
+ maskbits_t *maskbits_out, uint16_t *port_min_out,
uint16_t *port_max_out)
{
char *address;
@@ -1943,7 +2117,7 @@
tor_assert(s);
tor_assert(addr_out);
- tor_assert(mask_out);
+ tor_assert(maskbits_out);
tor_assert(port_min_out);
tor_assert(port_max_out);
@@ -1973,9 +2147,9 @@
if (!mask) {
if (strcmp(address,"*")==0)
- *mask_out = 0;
+ *maskbits_out = 0;
else
- *mask_out = 0xFFFFFFFFu;
+ *maskbits_out = 32;
} else {
endptr = NULL;
bits = (int) strtol(mask, &endptr, 10);
@@ -1986,9 +2160,16 @@
"Bad number of mask bits on address range; rejecting.");
goto err;
}
- *mask_out = ~((1u<<(32-bits))-1);
+ *maskbits_out = bits;
} else if (tor_inet_aton(mask, &in) != 0) {
- *mask_out = ntohl(in.s_addr);
+ bits = addr_mask_get_bits(ntohl(in.s_addr));
+ if (bits < 0) {
+ log_warn(LD_GENERAL,
+ "Mask %s on address range isn't a prefix; dropping",
+ escaped(mask));
+ goto err;
+ }
+ *maskbits_out = bits;
} else {
log_warn(LD_GENERAL,
"Malformed mask %s on address range; rejecting.",
@@ -2007,11 +2188,249 @@
return -1;
}
-/** Given an IPv4 address <b>in</b> (in network order, as usual),
- * write it as a string into the <b>buf_len</b>-byte buffer in
- * <b>buf</b>.
+/** Parse a string <b>s</b> containing an IPv4/IPv6 address, and possibly
+ * a mask and port or port range. Store the parsed address in
+ * <b>addr_out</b>, a mask (if any) in <b>mask_out</b>, and port(s) (if any)
+ * in <b>port_min_out</b> and <b>port_max_out</b>.
+ *
+ * The syntax is:
+ * Address OptMask OptPortRange
+ * Address ::= IPv4Address / "[" IPv6Address "]" / "*"
+ * OptMask ::= "/" Integer /
+ * OptPortRange ::= ":*" / ":" Integer / ":" Integer "-" Integer /
+ *
+ * - If mask, minport, or maxport are NULL, we do not want these
+ * options to be set; treat them as an error if present.
+ * - If the string has no mask, the mask is set to /32 (IPv4) or /128 (IPv6).
+ * - If the string has one port, it is placed in both min and max port
+ * variables.
+ * - If the string has no port(s), port_(min|max)_out are set to 1 and 65535.
+ *
+ * Return an address family on success, or -1 if an invalid address string is
+ * provided.
*/
int
+tor_addr_parse_mask_ports(const char *s, tor_addr_t *addr_out,
+ maskbits_t *maskbits_out,
+ uint16_t *port_min_out, uint16_t *port_max_out)
+{
+ char *base = NULL, *address, *mask = NULL, *port = NULL, *rbracket = NULL;
+ char *endptr;
+ int any_flag=0, v4map=0;
+
+ tor_assert(s);
+ tor_assert(addr_out);
+
+ /* IP, [], /mask, ports */
+#define MAX_ADDRESS_LENGTH (TOR_ADDR_BUF_LEN+2+(1+INET_NTOA_BUF_LEN)+12+1)
+
+ if (strlen(s) > MAX_ADDRESS_LENGTH) {
+ log_warn(LD_GENERAL, "Impossibly long IP %s; rejecting", escaped(s));
+ goto err;
+ }
+ base = tor_strdup(s);
+
+ /* Break 'base' into separate strings. */
+ address = base;
+ if (*address == '[') { /* Probably IPv6 */
+ address++;
+ rbracket = strchr(address, ']');
+ if (!rbracket) {
+ log_warn(LD_GENERAL,
+ "No closing IPv6 bracket in address pattern; rejecting.");
+ goto err;
+ }
+ }
+ mask = strchr((rbracket?rbracket:address),'/');
+ port = strchr((mask?mask:(rbracket?rbracket:address)), ':');
+ if (port)
+ *port++ = '\0';
+ if (mask)
+ *mask++ = '\0';
+ if (rbracket)
+ *rbracket = '\0';
+ if (port && mask)
+ tor_assert(port > mask);
+ if (mask && rbracket)
+ tor_assert(mask > rbracket);
+
+ /* Now "address" is the a.b.c.d|'*'|abcd::1 part...
+ * "mask" is the Mask|Maskbits part...
+ * and "port" is the *|port|min-max part.
+ */
+
+ /* Process the address portion */
+ memset(addr_out, 0, sizeof(tor_addr_t));
+
+ if (!strcmp(address, "*")) {
+ addr_out->sa.sin_family = AF_INET; /* AF_UNSPEC ???? XXXXX020 */
+ any_flag = 1;
+ } else if (tor_inet_pton(AF_INET6, address, &addr_out->sa6.sin6_addr) > 0) {
+ addr_out->sa6.sin6_family = AF_INET6;
+ } else if (tor_inet_pton(AF_INET, address, &addr_out->sa.sin_addr) > 0) {
+ addr_out->sa.sin_family = AF_INET;
+ } else {
+ log_warn(LD_GENERAL, "Malformed IP %s in address pattern; rejecting.",
+ escaped(address));
+ goto err;
+ }
+
+ v4map = tor_addr_is_v4(addr_out);
+
+/*
+#ifdef ALWAYS_V6_MAP
+ if (v_family == AF_INET) {
+ v_family = AF_INET6;
+ IN_ADDR6(addr_out).s6_addr32[3] = IN6_ADDRESS(addr_out).s_addr;
+ memset(&IN6_ADDRESS(addr_out), 0, 10);
+ IN_ADDR6(addr_out).s6_addr16[5] = 0xffff;
+ }
+#else
+ if (v_family == AF_INET6 && v4map) {
+ v_family = AF_INET;
+ IN4_ADDRESS((addr_out).s_addr = IN6_ADDRESS(addr_out).s6_addr32[3];
+ }
+#endif
+*/
+
+ /* Parse mask */
+ if (maskbits_out) {
+ int bits = 0;
+ struct in_addr v4mask;
+
+ if (mask) { /* the caller (tried to) specify a mask */
+ bits = (int) strtol(mask, &endptr, 10);
+ if (!*endptr) { /* strtol converted everything, so it was an integer */
+ if ((bits<0 || bits>128) ||
+ ((IN_FAMILY(addr_out) == AF_INET) && bits > 32)) {
+ log_warn(LD_GENERAL,
+ "Bad number of mask bits (%d) on address range; rejecting.",
+ bits);
+ goto err;
+ }
+ } else { /* mask might still be an address-style mask */
+ if (tor_inet_pton(AF_INET, mask, &v4mask) > 0) {
+ bits = addr_mask_get_bits(ntohl(v4mask.s_addr));
+ if (bits < 0) {
+ log_warn(LD_GENERAL,
+ "IPv4-style mask %s is not a prefix address; rejecting.",
+ escaped(mask));
+ goto err;
+ }
+ } else { /* Not IPv4; we don't do address-style IPv6 masks. */
+ log_warn(LD_GENERAL,
+ "Malformed mask on address range %s; rejecting.",
+ escaped(s));
+ goto err;
+ }
+ }
+ if (IN_FAMILY(addr_out) == AF_INET6 && v4map) {
+ if (bits > 32 && bits < 96) { /* Crazy */
+ log_warn(LD_GENERAL,
+ "Bad mask bits %i for V4-mapped V6 address; rejecting.",
+ bits);
+ goto err;
+ }
+ /* XXXX020 is this really what we want? */
+ bits = 96 + bits%32; /* map v4-mapped masks onto 96-128 bits */
+ }
+ } else { /* pick an appropriate mask, as none was given */
+ if (any_flag)
+ bits = 0; /* This is okay whether it's V6 or V4 (FIX V4-mapped V6!) */
+ else if (IN_FAMILY(addr_out) == AF_INET)
+ bits = 32;
+ else if (IN_FAMILY(addr_out) == AF_INET6)
+ bits = 128;
+ }
+ *maskbits_out = (maskbits_t) bits;
+ } else {
+ if (mask) {
+ log_warn(LD_GENERAL,
+ "Unexpected mask in addrss %s; rejecting", escaped(s));
+ goto err;
+ }
+ }
+
+ /* Parse port(s) */
+ if (port_min_out) {
+ uint16_t port2;
+ if (!port_max_out) /* caller specified one port; fake the second one */
+ port_max_out = &port2;
+
+ if (parse_port_range(port, port_min_out, port_max_out) < 0) {
+ goto err;
+ } else if ((*port_min_out != *port_max_out) && port_max_out == &port2) {
+ log_warn(LD_GENERAL,
+ "Wanted one port from address range, but there are two.");
+
+ port_max_out = NULL; /* caller specified one port, so set this back */
+ goto err;
+ }
+ } else {
+ if (port) {
+ log_warn(LD_GENERAL,
+ "Unexpected ports in addrss %s; rejecting", escaped(s));
+ goto err;
+ }
+ }
+
+ tor_free(base);
+ return IN_FAMILY(addr_out);
+ err:
+ tor_free(base);
+ return -1;
+}
+
+/** Determine whether an address is IPv4, either native or ipv4-mapped ipv6.
+ * Note that this is about representation only, as any decent stack will
+ * reject ipv4-mapped addresses received on the wire (and won't use them
+ * on the wire either).
+ */
+int
+tor_addr_is_v4(const tor_addr_t *addr)
+{
+ tor_assert(addr);
+
+ if (IN_FAMILY(addr) == AF_INET)
+ return 1;
+
+ if (IN_FAMILY(addr) == AF_INET6) { /* First two don't need to be ordered */
+ if ((IN6_ADDRESS(addr)->s6_addr32[0] == 0) &&
+ (IN6_ADDRESS(addr)->s6_addr32[1] == 0) &&
+ (ntohl(IN6_ADDRESS(addr)->s6_addr32[2]) == 0x0000ffffu))
+ return 1;
+ }
+
+ return 0; /* Not IPv4 - unknown family or a full-blood IPv6 address */
+}
+
+/** Determine whether an address <b>addr</b> is null, either all zeroes or
+ * belonging to family AF_UNSPEC.
+ */
+int
+tor_addr_is_null(const tor_addr_t *addr)
+{
+ tor_assert(addr);
+
+ switch (IN_FAMILY(addr)) {
+ case AF_INET6:
+ return (!IN6_ADDRESS(addr)->s6_addr32[0] &&
+ !IN6_ADDRESS(addr)->s6_addr32[1] &&
+ !IN6_ADDRESS(addr)->s6_addr32[2] &&
+ !IN6_ADDRESS(addr)->s6_addr32[3]);
+ case AF_INET:
+ return (!IN4_ADDRESS(addr)->s_addr);
+ default:
+ return 1;
+ }
+ //return 1;
+}
+
+/** Given an IPv4 in_addr struct *<b>in</b> (in network order, as usual),
+ * write it as a string into the <b>buf_len</b>-byte buffer in
+ * <b>buf</b>.
+ */
+int
tor_inet_ntoa(const struct in_addr *in, char *buf, size_t buf_len)
{
uint32_t a = ntohl(in->s_addr);
@@ -2022,37 +2441,207 @@
(int)(uint8_t)((a )&0xff));
}
-/** Given a host-order <b>addr</b>, call tor_inet_ntoa() on it
- * and return a strdup of the resulting address.
+/** Take a 32-bit host-order ipv4 address <b>v4addr</b> and store it in the
+ * tor_addr *<b>dest</b>.
+ *
+ * XXXX020 Temporary, for use while 32-bit int addresses are still being
+ * passed around.
*/
+void
+tor_addr_from_ipv4(tor_addr_t *dest, uint32_t v4addr)
+{
+ tor_assert(dest);
+ memset(dest, 0, sizeof(dest));
+ dest->sa.sin_family = AF_INET;
+ dest->sa.sin_addr.s_addr = htonl(v4addr);
+}
+
+/** Copy a tor_addr_t from <b>src</b> to <b>dest</b>.
+ */
+void
+tor_addr_copy(tor_addr_t *dest, const tor_addr_t *src)
+{
+ tor_assert(src && dest);
+ memcpy(dest, src, sizeof(tor_addr_t));
+}
+
+/** Given two addresses <b>addr1</b> and <b>addr2</b>, return 0 if the two
+ * addresses are equivalent under the mask mbits, less than 0 if addr1
+ * preceeds addr2, and greater than 0 otherwise.
+ *
+ * Different address families (IPv4 vs IPv6) are always considered unequal.
+ */
+int
+tor_addr_compare(const tor_addr_t *addr1, const tor_addr_t *addr2)
+{
+ return tor_addr_compare_masked(addr1, addr2, 128);
+}
+
+/** As tor_addr_compare(), but only looks at the first <b>mask</b> bits of
+ * the address.
+ *
+ * Reduce over-specific masks (>128 for ipv6, >32 for ipv4) to 128 or 32.
+ */
+int
+tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2,
+ maskbits_t mbits)
+{
+ uint32_t ip4a=0, ip4b=0;
+ sa_family_t v_family[2];
+ int idx;
+ uint32_t masked_a, masked_b;
+
+ tor_assert(addr1 && addr2);
+
+ /* XXXX020 this code doesn't handle mask bits right it's using v4-mapped v6
+ * addresses. If I ask whether ::ffff:1.2.3.4 and ::ffff:1.2.7.8 are the
+ * same in the first 16 bits, it will say "yes." That's not so intuitive.
+ */
+
+ v_family[0] = IN_FAMILY(addr1);
+ v_family[1] = IN_FAMILY(addr2);
+
+ if (v_family[0] == AF_INET) { /* If this is native IPv4, note the address */
+ ip4a = IPV4IPh(addr1); /* Later we risk overwriting a v4-mapped address */
+ } else if ((v_family[0] == AF_INET6) && tor_addr_is_v4(addr1)) {
+ v_family[0] = AF_INET;
+ ip4a = IPV4MAPh(addr1);
+ }
+
+ if (v_family[1] == AF_INET) { /* If this is native IPv4, note the address */
+ ip4b = IPV4IPh(addr2); /* Later we risk overwriting a v4-mapped address */
+ } else if ((v_family[1] == AF_INET6) && tor_addr_is_v4(addr2)) {
+ v_family[1] = AF_INET;
+ ip4b = IPV4MAPh(addr2);
+ }
+
+ if (v_family[0] > v_family[1]) /* Comparison of virtual families */
+ return 1;
+ else if (v_family[0] < v_family[1])
+ return -1;
+
+ if (mbits == 0) /* Under a complete wildcard mask, consider them equal */
+ return 0;
+
+ if (v_family[0] == AF_INET) { /* Real or mapped IPv4 */
+ if (mbits >= 32) {
+ masked_a = ip4a;
+ masked_b = ip4b;
+ } else if (mbits == 0) {
+ return 0;
+ } else {
+ masked_a = ip4a >> (32-mbits);
+ masked_b = ip4b >> (32-mbits);
+ }
+ if (masked_a < masked_b)
+ return -1;
+ else if (masked_a > masked_b)
+ return 1;
+ return 0;
+ } else if (v_family[0] == AF_INET6) { /* Real IPv6 */
+ const uint32_t *a1 = IN6_ADDRESS(addr1)->s6_addr32;
+ const uint32_t *a2 = IN6_ADDRESS(addr2)->s6_addr32;
+ for (idx = 0; idx < 4; ++idx) {
+ uint32_t masked_a = ntohl(a1[idx]);
+ uint32_t masked_b = ntohl(a2[idx]);
+ if (!mbits) {
+ return 0; /* Mask covers both addresses from here on */
+ } else if (mbits < 32) {
+ masked_a >>= (32-mbits);
+ masked_b >>= (32-mbits);
+ }
+
+ if (masked_a > masked_b)
+ return 1;
+ else if (masked_a < masked_b)
+ return -1;
+
+ if (mbits < 32)
+ return 0;
+ mbits -= 32;
+ }
+ return 0;
+ }
+
+ tor_assert(0); /* Unknown address family */
+ return -1; /* unknown address family, return unequal? */
+}
+
+/** Given a host-order <b>addr</b>, call tor_inet_ntop() on it
+ * and return a strdup of the resulting address.
+ */
char *
tor_dup_addr(uint32_t addr)
{
- char buf[INET_NTOA_BUF_LEN];
+ char buf[TOR_ADDR_BUF_LEN];
struct in_addr in;
in.s_addr = htonl(addr);
- tor_inet_ntoa(&in, buf, sizeof(buf));
+ tor_inet_ntop(AF_INET, &in, buf, sizeof(buf));
return tor_strdup(buf);
}
-/**
- * Set *<b>addr</b> to the host-order IPv4 address (if any) of whatever
- * interface connects to the internet. This address should only be used in
- * checking whether our address has changed. Return 0 on success, -1 on
- * failure.
+/** Convert the tor_addr_t *<b>addr</b> into string form and store it in
+ * <b>dest</b>, which can hold at least <b>len</b> bytes. Returns <b>dest</b>
+ * on success, NULL on failure.
*/
+const char *
+tor_addr_to_str(char *dest, const tor_addr_t *addr, int len)
+{
+ tor_assert(addr && dest);
+
+ if (IN_FAMILY(addr) == AF_INET) {
+ return tor_inet_ntop(AF_INET, IN4_ADDRESS(addr), dest, len);
+ } else if (IN_FAMILY(addr) == AF_INET6) {
+ return tor_inet_ntop(AF_INET6, IN6_ADDRESS(addr), dest, len);
+ } else {
+ return NULL;
+ }
+}
+
+/** Convert the string in <b>src</b> to a tor_addr_t <b>addr</b>.
+ */
int
-get_interface_address(int severity, uint32_t *addr)
+tor_addr_from_str(tor_addr_t *addr, const char *src)
{
+ tor_assert(addr && src);
+ return tor_addr_parse_mask_ports(src, addr, NULL, NULL, NULL);
+}
+
+/** Set *<b>addr</b> to the IP address (if any) of whatever interface
+ * connects to the internet. This address should only be used in checking
+ * whether our address has changed. Return 0 on success, -1 on failure.
+ */
+int
+get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr)
+{
int sock=-1, r=-1;
- struct sockaddr_in target_addr, my_addr;
- socklen_t my_addr_len = sizeof(my_addr);
+ struct sockaddr_storage my_addr, target_addr;
+ socklen_t my_addr_len;
tor_assert(addr);
- *addr = 0;
- sock = tor_open_socket(PF_INET,SOCK_DGRAM,IPPROTO_UDP);
+ memset(addr, 0, sizeof(tor_addr_t));
+ memset(&target_addr, 0, sizeof(target_addr));
+ my_addr_len = sizeof(my_addr);
+ ((struct sockaddr_in*)&target_addr)->sin_port = 9; /* DISGARD port */
+ /* Don't worry: no packets are sent. We just need to use a real address
+ * on the actual internet. */
+ if (family == AF_INET6) {
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6*)&target_addr;
+ sock = tor_open_socket(PF_INET6,SOCK_DGRAM,IPPROTO_UDP);
+ my_addr_len = sizeof(struct sockaddr_in6);
+ sin6->sin6_family = AF_INET6;
+ sin6->sin6_addr.s6_addr16[0] = htons(0x2002); /* 2002:: */
+ } else if (family == AF_INET) {
+ struct sockaddr_in *sin = (struct sockaddr_in*)&target_addr;
+ sock = tor_open_socket(PF_INET,SOCK_DGRAM,IPPROTO_UDP);
+ my_addr_len = sizeof(struct sockaddr_in);
+ sin->sin_family = AF_INET;
+ sin->sin_addr.s_addr = htonl(0x12000001); /* 18.0.0.1 */
+ } else {
+ return -1;
+ }
if (sock < 0) {
int e = tor_socket_errno(-1);
log_fn(severity, LD_NET, "unable to create socket: %s",
@@ -2060,29 +2649,20 @@
goto err;
}
- memset(&target_addr, 0, sizeof(target_addr));
- target_addr.sin_family = AF_INET;
- /* discard port */
- target_addr.sin_port = 9;
- /* 18.0.0.1 (Don't worry: no packets are sent. We just need a real address
- * on the internet.) */
- target_addr.sin_addr.s_addr = htonl(0x12000001);
-
if (connect(sock,(struct sockaddr *)&target_addr,sizeof(target_addr))<0) {
int e = tor_socket_errno(sock);
log_fn(severity, LD_NET, "connect() failed: %s", tor_socket_strerror(e));
goto err;
}
- if (getsockname(sock, (struct sockaddr*)&my_addr, &my_addr_len)) {
+ if (getsockname(sock,(struct sockaddr*)&my_addr, &my_addr_len)) {
int e = tor_socket_errno(sock);
log_fn(severity, LD_NET, "getsockname() to determine interface failed: %s",
tor_socket_strerror(e));
goto err;
}
- *addr = ntohl(my_addr.sin_addr.s_addr);
-
+ memcpy(addr, &my_addr, sizeof(tor_addr_t));
r=0;
err:
if (sock >= 0)
@@ -2090,6 +2670,24 @@
return r;
}
+/**
+ * Set *<b>addr</b> to the host-order IPv4 address (if any) of whatever
+ * interface connects to the internet. This address should only be used in
+ * checking whether our address has changed. Return 0 on success, -1 on
+ * failure.
+ */
+int
+get_interface_address(int severity, uint32_t *addr)
+{
+ tor_addr_t local_addr;
+ int r;
+
+ r = get_interface_address6(severity, AF_INET, &local_addr);
+ if (r>=0)
+ *addr = IPV4IPh(&local_addr);
+ return r;
+}
+
/* =====
* Process helpers
* ===== */
Modified: tor/branches/114-dist-storage/src/common/util.h
===================================================================
--- tor/branches/114-dist-storage/src/common/util.h 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/common/util.h 2007-08-21 08:37:13 UTC (rev 11244)
@@ -17,12 +17,6 @@
#include "compat.h"
#include <stdio.h>
#include <stdlib.h>
-#ifdef HAVE_SYS_TIME_H
-#include <sys/time.h>
-#endif
-#ifdef HAVE_TIME_H
-#include <time.h>
-#endif
/* Replace assert() with a variant that sends failures to the log before
* calling assert() normally.
@@ -105,6 +99,8 @@
#define tor_strndup(s, n) _tor_strndup(s, n DMALLOC_ARGS)
#define tor_memdup(s, n) _tor_memdup(s, n DMALLOC_ARGS)
+void tor_log_mallinfo(int severity);
+
/** Return the offset of <b>member</b> within the type <b>tp</b>, in bytes */
#if defined(__GNUC__) && __GNUC__ > 3
#define STRUCT_OFFSET(tp, member) __builtin_offsetof(tp, member)
@@ -155,6 +151,8 @@
int tor_strisprint(const char *s) ATTR_PURE ATTR_NONNULL((1));
int tor_strisnonupper(const char *s) ATTR_PURE ATTR_NONNULL((1));
int strcmpstart(const char *s1, const char *s2) ATTR_PURE ATTR_NONNULL((1,2));
+int strcmp_len(const char *s1, const char *s2, size_t len)
+ ATTR_PURE ATTR_NONNULL((1,2));
int strcasecmpstart(const char *s1, const char *s2)
ATTR_PURE ATTR_NONNULL((1,2));
int strcmpend(const char *s1, const char *s2) ATTR_PURE ATTR_NONNULL((1,2));
@@ -173,7 +171,9 @@
const char *eat_whitespace(const char *s) ATTR_PURE;
const char *eat_whitespace_eos(const char *s, const char *eos) ATTR_PURE;
const char *eat_whitespace_no_nl(const char *s) ATTR_PURE;
+const char *eat_whitespace_eos_no_nl(const char *s, const char *eos) ATTR_PURE;
const char *find_whitespace(const char *s) ATTR_PURE;
+const char *find_whitespace_eos(const char *s, const char *eos) ATTR_PURE;
int tor_mem_is_zero(const char *mem, size_t len) ATTR_PURE;
int tor_digest_is_zero(const char *digest) ATTR_PURE;
char *esc_for_log(const char *string) ATTR_MALLOC;
@@ -247,14 +247,29 @@
int parse_port_range(const char *port, uint16_t *port_min_out,
uint16_t *port_max_out);
int parse_addr_and_port_range(const char *s, uint32_t *addr_out,
- uint32_t *mask_out, uint16_t *port_min_out,
+ maskbits_t *maskbits_out, uint16_t *port_min_out,
uint16_t *port_max_out);
int addr_mask_get_bits(uint32_t mask);
-#define INET_NTOA_BUF_LEN 16
+int addr_mask_cmp_bits(uint32_t a1, uint32_t a2, maskbits_t bits);
int tor_inet_ntoa(const struct in_addr *in, char *buf, size_t buf_len);
char *tor_dup_addr(uint32_t addr) ATTR_MALLOC;
int get_interface_address(int severity, uint32_t *addr);
+int get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr);
+int tor_addr_compare(const tor_addr_t *addr1, const tor_addr_t *addr2);
+int tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2,
+ maskbits_t mask);
+int tor_addr_is_v4(const tor_addr_t *addr);
+int tor_addr_is_internal(const tor_addr_t *ip, int for_listening) ATTR_PURE;
+int tor_addr_parse_mask_ports(const char *s,
+ tor_addr_t *addr_out, maskbits_t *mask_out,
+ uint16_t *port_min_out, uint16_t *port_max_out);
+const char * tor_addr_to_str(char *dest, const tor_addr_t *addr, int len);
+int tor_addr_from_str(tor_addr_t *addr, const char *src);
+void tor_addr_copy(tor_addr_t *dest, const tor_addr_t *src);
+void tor_addr_from_ipv4(tor_addr_t *dest, uint32_t v4addr);
+int tor_addr_is_null(const tor_addr_t *addr);
+
/* Process helpers */
void start_daemon(void);
void finish_daemon(const char *desired_cwd);
Modified: tor/branches/114-dist-storage/src/or/Makefile.am
===================================================================
--- tor/branches/114-dist-storage/src/or/Makefile.am 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/Makefile.am 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1,3 +1,5 @@
+CPPFLAGS += -I../common
+
TESTS = test
noinst_PROGRAMS = test
@@ -40,7 +42,7 @@
tor_main.o: micro-revision.i
-micro-revision.i: FORCE
+micro-revision.i:
@svkdir=$$SVKROOT; \
if test "x$$svkdir" = x ; then \
svkdir=$$HOME/.svk; \
Modified: tor/branches/114-dist-storage/src/or/buffers.c
===================================================================
--- tor/branches/114-dist-storage/src/or/buffers.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/buffers.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -15,7 +15,7 @@
#include "or.h"
-#define SENTINELS
+#undef SENTINELS
#undef CHECK_AFTER_RESIZE
#undef PARANOIA
#undef NOINLINE
@@ -178,10 +178,11 @@
/** Freelists to hold 4k and 16k memory chunks. This seems to be what
* we use most. */
static free_mem_list_t free_mem_list_4k = { NULL, 0, 0, 4096, 16, INT_MAX };
-static free_mem_list_t free_mem_list_16k = { NULL, 0, 0, 16384, 4, 128 };
+static free_mem_list_t free_mem_list_8k = { NULL, 0, 0, 8192 , 8, 128 };
+static free_mem_list_t free_mem_list_16k = { NULL, 0, 0, 16384, 4, 64 };
/** Macro: True iff the size is one for which we keep a freelist. */
-#define IS_FREELIST_SIZE(sz) ((sz) == 4096 || (sz) == 16384)
+#define IS_FREELIST_SIZE(sz) ((sz) == 4096 || (sz) == 8192 || (sz) == 16384)
/** Return the proper freelist for chunks of size <b>sz</b>, or fail
* with an assertion. */
@@ -190,12 +191,33 @@
{
if (sz == 4096) {
return &free_mem_list_4k;
+ } else if (sz == 8192) {
+ return &free_mem_list_8k;
} else {
tor_assert(sz == 16384);
return &free_mem_list_16k;
}
}
+/** Write the sizes of the buffer freelists at log level <b>severity</b> */
+void
+buf_dump_freelist_sizes(int severity)
+{
+ size_t sz;
+ log(severity, LD_MM, "======= Buffer freelists.");
+ for (sz = 4096; sz <= 16384; sz *= 2) {
+ uint64_t total_size;
+ free_mem_list_t *lst;
+ if (!IS_FREELIST_SIZE(sz))
+ continue;
+ lst = get_free_mem_list(sz);
+ total_size = ((uint64_t)sz)*lst->len;
+ log(severity, LD_MM,
+ U64_FORMAT" bytes in %d %d-byte buffers. (low-water: %d)",
+ U64_PRINTF_ARG(total_size), lst->len, (int)sz, lst->lowwater);
+ }
+}
+
/** Throw the memory from <b>buf</b> onto the appropriate freelist.
* Return true if we added the memory, 0 if the freelist was full. */
static int
@@ -254,21 +276,27 @@
}
/** Remove elements from the freelists that haven't been needed since the
- * last call to this function. */
+ * last call to this function. If <b>free_all</b>, we're exiting and we
+ * should clear the whole lists. */
void
-buf_shrink_freelists(void)
+buf_shrink_freelists(int free_all)
{
- int j;
- for (j = 0; j < 2; ++j) {
- free_mem_list_t *list = j ? &free_mem_list_16k : &free_mem_list_4k;
- if (list->lowwater > list->slack) {
+ int list_elt_size;
+ for (list_elt_size = 4096; list_elt_size <= 16384; list_elt_size *= 2) {
+ free_mem_list_t *list = get_free_mem_list(list_elt_size);
+ if (list->lowwater > list->slack || free_all) {
int i, n_to_skip, n_to_free;
char **ptr;
- log_info(LD_GENERAL, "We haven't used %d/%d allocated %d-byte buffer "
+ if (free_all) { /* Free every one of them */
+ log_info(LD_GENERAL, "Freeing all %d elements from %d-byte freelist.",
+ list->len, (int)list->chunksize);
+ n_to_free = list->len;
+ } else { /* Skip over the slack and non-lowwater entries */
+ log_info(LD_GENERAL, "We haven't used %d/%d allocated %d-byte buffer "
"memory chunks since the last call; freeing all but %d of them",
list->lowwater, list->len, (int)list->chunksize, list->slack);
- /* Skip over the slack and non-lowwater entries */
- n_to_free = list->lowwater - list->slack;
+ n_to_free = list->lowwater - list->slack;
+ }
n_to_skip = list->len - n_to_free;
for (ptr = &list->list, i = 0; i < n_to_skip; ++i) {
char *mem = *ptr;
@@ -424,8 +452,16 @@
size_t new_len;
new_len = buf->len;
- if (buf->datalen == 0 && buf->highwater == 0 &&
+ /* Actually, we ignore highwater here if we're going to throw it on the
+ * freelist, since it's way cheaper to use the freelist than to use (some)
+ * platform mallocs.
+ *
+ * DOCDOC If it turns out to be a good idea, add it to the doxygen for this
+ * function.
+ */
+ if (buf->datalen == 0 && // buf->highwater == 0 &&
IS_FREELIST_SIZE(buf->len)) {
+ buf->highwater = 0;
if (add_buf_mem_to_freelist(buf))
return;
}
@@ -451,6 +487,12 @@
buf->cur = _wrap_ptr(buf, buf->cur+n);
} else {
buf->cur = buf->mem;
+ if (IS_FREELIST_SIZE(buf->len)) {
+ buf->highwater = 0;
+
+ if (add_buf_mem_to_freelist(buf))
+ return;
+ }
}
check();
}
Modified: tor/branches/114-dist-storage/src/or/circuitbuild.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuitbuild.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/circuitbuild.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -294,7 +294,7 @@
return circ;
}
-/** Build a new circuit for <b>purpose</b>. If <b>info</b>
+/** Build a new circuit for <b>purpose</b>. If <b>exit</b>
* is defined, then use that as your exit router, else choose a suitable
* exit node.
*
@@ -502,7 +502,7 @@
append_cell_to_circuit_queue(circ, circ->n_conn, &cell, CELL_DIRECTION_OUT);
/* mark it so it gets better rate limiting treatment. */
- circ->n_conn->client_used = 1;
+ circ->n_conn->client_used = time(NULL);
return 0;
}
@@ -1072,6 +1072,7 @@
int i;
for (i = 0; i < smartlist_len(source); ++i) {
+ /*XXXX020 some of these are leaked somewhere.. fix that. */
tmp = tor_malloc(sizeof(uint16_t));
memcpy(tmp, smartlist_get(source, i), sizeof(uint16_t));
smartlist_add(dest, tmp);
@@ -1583,8 +1584,7 @@
options->ExcludeNodes, excluded,
state->need_uptime, state->need_capacity, 0,
options->_AllowInvalid & ALLOW_INVALID_MIDDLE, 0, 0);
- if (preferred)
- tor_free(preferred);
+ tor_free(preferred);
smartlist_free(excluded);
return choice;
}
@@ -1885,8 +1885,8 @@
return changed;
}
-/** Return true iff enough time has passed since we last tried connect to the
- * unreachable guard <b>e</b> that we're willing to try again. */
+/** Return true iff enough time has passed since we last tried to connect
+ * to the unreachable guard <b>e</b> that we're willing to try again. */
static int
entry_is_time_to_retry(entry_guard_t *e, time_t now)
{
@@ -1913,7 +1913,7 @@
* if demanded by <b>need_uptime</b> or <b>need_capacity</b>;
* (This check is currently redundant with the Guard flag, but in
* the future that might change. Best to leave it in for now.)
- * - Allowed by our current ReachableAddresses config option; and
+ * - Allowed by our current ReachableORAddresses config option; and
* - Currently thought to be reachable by us (unless assume_reachable
* is true).
*/
@@ -1937,8 +1937,7 @@
return NULL;
if (router_is_unreliable(r, need_uptime, need_capacity, 0))
return NULL;
- if (firewall_is_fascist_or() &&
- !fascist_firewall_allows_address_or(r->addr,r->or_port))
+ if (!fascist_firewall_allows_address_or(r->addr,r->or_port))
return NULL;
return r;
}
@@ -2083,17 +2082,6 @@
entry_guards_changed();
}
-/** Release all storage held by the list of entry guards. */
-void
-entry_guards_free_all(void)
-{
- if (entry_guards) {
- SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, tor_free(e));
- smartlist_free(entry_guards);
- entry_guards = NULL;
- }
-}
-
/** How long (in seconds) do we allow an entry guard to be nonfunctional,
* unlisted, excluded, or otherwise nonusable before we give up on it? */
#define ENTRY_GUARD_REMOVE_AFTER (30*24*60*60)
@@ -2679,6 +2667,7 @@
uint32_t addr;
uint16_t port;
char identity[DIGEST_LEN];
+ download_status_t fetch_status;
} bridge_info_t;
/** A list of known bridges. */
@@ -2708,23 +2697,31 @@
}
#endif
-/** Return 1 if <b>ri</b> is one of our known bridges (either by
- * comparing keys if possible, else by comparing addr/port). */
-int
-routerinfo_is_a_bridge(routerinfo_t *ri)
+/** Return a bridge pointer if <b>ri</b> is one of our known bridges
+ * (either by comparing keys if possible, else by comparing addr/port).
+ * Else return NULL. */
+static bridge_info_t *
+routerinfo_get_configured_bridge(routerinfo_t *ri)
{
SMARTLIST_FOREACH(bridge_list, bridge_info_t *, bridge,
{
if (tor_digest_is_zero(bridge->identity) &&
bridge->addr == ri->addr && bridge->port == ri->or_port)
- return 1;
+ return bridge;
if (!memcmp(bridge->identity, ri->cache_info.identity_digest,
DIGEST_LEN))
- return 1;
+ return bridge;
});
- return 0;
+ return NULL;
}
+/** Return 1 if <b>ri</b> is one of our known bridges, else 0. */
+int
+routerinfo_is_a_configured_bridge(routerinfo_t *ri)
+{
+ return routerinfo_get_configured_bridge(ri) ? 1 : 0;
+}
+
/** Remember a new bridge at <b>addr</b>:<b>port</b>. If <b>digest</b>
* is set, it tells us the identity key too. */
void
@@ -2740,30 +2737,69 @@
smartlist_add(bridge_list, b);
}
+/** Schedule the next fetch for <b>bridge</b>, based on
+ * some retry schedule. */
+static void
+bridge_fetch_status_increment(bridge_info_t *bridge, time_t now)
+{
+ switch (bridge->fetch_status.n_download_failures) {
+ case 0: bridge->fetch_status.next_attempt_at = now+60*15; break;
+ case 1: bridge->fetch_status.next_attempt_at = now+60*15; break;
+ default: bridge->fetch_status.next_attempt_at = now+60*60; break;
+ }
+ if (bridge->fetch_status.n_download_failures < 10)
+ bridge->fetch_status.n_download_failures++;
+}
+
+/** We just got a new descriptor for <b>bridge</b>. Reschedule the
+ * next fetch for a long time from <b>now</b>. */
+static void
+bridge_fetch_status_arrived(bridge_info_t *bridge, time_t now)
+{
+ bridge->fetch_status.next_attempt_at = now+60*60;
+ bridge->fetch_status.n_download_failures = 0;
+}
+
/** For each bridge in our list for which we don't currently have a
* descriptor, fetch a new copy of its descriptor -- either directly
* from the bridge or via a bridge authority. */
void
-fetch_bridge_descriptors(void)
+fetch_bridge_descriptors(time_t now)
{
char address_buf[INET_NTOA_BUF_LEN+1];
struct in_addr in;
or_options_t *options = get_options();
int num_bridge_auths = get_n_authorities(BRIDGE_AUTHORITY);
+ int ask_bridge_directly;
if (!bridge_list)
return;
SMARTLIST_FOREACH(bridge_list, bridge_info_t *, bridge,
{
- if (router_get_by_digest(bridge->identity))
- continue; /* we've already got one. great. */
+ if (bridge->fetch_status.next_attempt_at > now)
+ continue; /* don't bother, no need to retry yet */
+
+ /* schedule another fetch as if this one will fail, in case it does */
+ bridge_fetch_status_increment(bridge, now);
+
in.s_addr = htonl(bridge->addr);
tor_inet_ntoa(&in, address_buf, sizeof(address_buf));
- if (tor_digest_is_zero(bridge->identity) ||
- !options->UpdateBridgesFromAuthority ||
- !num_bridge_auths) {
+ ask_bridge_directly = tor_digest_is_zero(bridge->identity) ||
+ !options->UpdateBridgesFromAuthority ||
+ !num_bridge_auths;
+
+ if (ask_bridge_directly &&
+ !fascist_firewall_allows_address_or(bridge->addr, bridge->port)) {
+ log_notice(LD_DIR, "Bridge at '%s:%d' isn't reachable by our "
+ "firewall policy. %s.", address_buf, bridge->port,
+ num_bridge_auths ? "Asking bridge authority instead" :
+ "Skipping");
+ ask_bridge_directly = 0;
+ }
+
+ if (ask_bridge_directly) {
if (!connection_get_by_type_addr_port_purpose(
CONN_TYPE_DIR, bridge->addr, bridge->port,
DIR_PURPOSE_FETCH_SERVERDESC)) {
@@ -2773,11 +2809,21 @@
1, bridge->identity,
DIR_PURPOSE_FETCH_SERVERDESC,
ROUTER_PURPOSE_BRIDGE,
- 0, "authority", NULL, 0);
+ 0, "authority.z", NULL, 0);
}
} else {
- /* we have a digest and we want to ask an authority. */
- // XXX
+ /* We have a digest and we want to ask an authority. We could
+ * combine all the requests into one, but that may give more
+ * hints to the bridge authority than we want to give. */
+ char resource[10 + HEX_DIGEST_LEN];
+ memcpy(resource, "fp/", 3);
+ base16_encode(resource+3, HEX_DIGEST_LEN+1,
+ bridge->identity, DIGEST_LEN);
+ memcpy(resource+3+HEX_DIGEST_LEN, ".z", 3);
+ log_info(LD_DIR, "Fetching bridge info '%s' from bridge authority.",
+ resource);
+ directory_get_from_dirserver(DIR_PURPOSE_FETCH_SERVERDESC,
+ ROUTER_PURPOSE_BRIDGE, resource, 1);
}
});
}
@@ -2791,16 +2837,22 @@
tor_assert(ri->purpose == ROUTER_PURPOSE_BRIDGE);
if (get_options()->UseBridges) {
int first = !any_bridge_descriptors_known();
+ bridge_info_t *bridge = routerinfo_get_configured_bridge(ri);
+ time_t now = time(NULL);
ri->is_running = 1;
+
+ /* it's here; schedule its re-fetch for a long time from now. */
+ bridge_fetch_status_arrived(bridge, now);
+
add_an_entry_guard(ri, 1);
log_notice(LD_DIR, "new bridge descriptor '%s'", ri->nickname);
if (first)
- routerlist_retry_directory_downloads(time(NULL));
+ routerlist_retry_directory_downloads(now);
}
}
/** Return 1 if any of our entry guards have descriptors that
- * are marked with purpose 'bridge'. Else return 0.
+ * are marked with purpose 'bridge' and are running. Else return 0.
*
* We use this function to decide if we're ready to start building
* circuits through our bridges, or if we need to wait until the
@@ -2808,18 +2860,65 @@
int
any_bridge_descriptors_known(void)
{
+ tor_assert(get_options()->UseBridges);
return choose_random_entry(NULL)!=NULL ? 1 : 0;
-#if 0
+}
+
+/** Return 1 if we have at least one descriptor for a bridge and
+ * all descriptors we know are down. Else return 0. If <b>act</b> is
+ * 1, then mark the down bridges up; else just observe and report. */
+static int
+bridges_retry_helper(int act)
+{
routerinfo_t *ri;
+ int any_known = 0;
+ int any_running = 0;
if (!entry_guards)
entry_guards = smartlist_create();
SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e,
{
ri = router_get_by_digest(e->identity);
- if (ri && ri->purpose == ROUTER_PURPOSE_BRIDGE)
- return 1;
+ if (ri && ri->purpose == ROUTER_PURPOSE_BRIDGE) {
+ any_known = 1;
+ if (ri->is_running)
+ any_running = 1; /* some bridge is both known and running */
+ else if (act) { /* mark it for retry */
+ ri->is_running = 1;
+ e->can_retry = 1;
+ e->bad_since = 0;
+ }
+ }
});
- return 0;
-#endif
+ return any_known && !any_running;
}
+/** Do we know any descriptors for our bridges, and are they all
+ * down? */
+int
+bridges_should_be_retried(void)
+{
+ return bridges_retry_helper(0);
+}
+
+/** Mark all down known bridges up. */
+void
+bridges_retry_all(void)
+{
+ bridges_retry_helper(1);
+}
+
+/** Release all storage held by the list of entry guards and related
+ * memory structs. */
+void
+entry_guards_free_all(void)
+{
+ if (entry_guards) {
+ SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, tor_free(e));
+ smartlist_free(entry_guards);
+ entry_guards = NULL;
+ }
+ clear_bridge_list();
+ smartlist_free(bridge_list);
+ bridge_list = NULL;
+}
+
Modified: tor/branches/114-dist-storage/src/or/circuituse.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuituse.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/circuituse.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -1012,7 +1012,8 @@
if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) {
/* need to pick an intro point */
- extend_info = rend_client_get_random_intro(conn->rend_query, conn->rend_version);
+ extend_info = rend_client_get_random_intro(conn->rend_query,
+ conn->rend_version);
if (!extend_info) {
log_info(LD_REND,
"No intro points for '%s': refetching service descriptor.",
Modified: tor/branches/114-dist-storage/src/or/command.c
===================================================================
--- tor/branches/114-dist-storage/src/or/command.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/command.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -322,6 +322,12 @@
return;
}
+ if (CIRCUIT_IS_ORIGIN(circ)) {
+ /* if we're a server and treating connections with recent local
+ * traffic better, then this is one of them. */
+ conn->client_used = time(NULL);
+ }
+
if (!CIRCUIT_IS_ORIGIN(circ) &&
cell->circ_id == TO_OR_CIRCUIT(circ)->p_circ_id)
direction = CELL_DIRECTION_OUT;
Modified: tor/branches/114-dist-storage/src/or/config.c
===================================================================
--- tor/branches/114-dist-storage/src/or/config.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/config.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -180,7 +180,7 @@
VAR("Group", STRING, Group, NULL),
VAR("HardwareAccel", BOOL, HardwareAccel, "0"),
VAR("HashedControlPassword",STRING, HashedControlPassword, NULL),
- VAR("HSDir", BOOL, HSDir, "0"),
+ VAR("HidServDirectoryV2", BOOL, HidServDirectoryV2, "0"),
VAR("HiddenServiceDir", LINELIST_S, RendConfigLines, NULL),
VAR("HiddenServiceExcludeNodes", LINELIST_S, RendConfigLines, NULL),
VAR("HiddenServiceNodes", LINELIST_S, RendConfigLines, NULL),
Modified: tor/branches/114-dist-storage/src/or/connection.c
===================================================================
--- tor/branches/114-dist-storage/src/or/connection.c 2007-08-21 07:24:31 UTC (rev 11243)
+++ tor/branches/114-dist-storage/src/or/connection.c 2007-08-21 08:37:13 UTC (rev 11244)
@@ -28,6 +28,7 @@
static int connection_read_to_buf(connection_t *conn, int *max_to_read);
static int connection_process_inbuf(connection_t *conn, int package_partial);
static void client_check_address_changed(int sock);
+static void set_constrained_socket_buffers(int sock, int size);
static uint32_t last_interface_ip = 0;
static smartlist_t *outgoing_addrs = NULL;
@@ -898,6 +899,8 @@
/* length of the remote address. Must be whatever accept() needs. */
socklen_t remotelen = 256;
char tmpbuf[INET_NTOA_BUF_LEN];
+ or_options_t *options = get_options();
+
tor_assert((size_t)remotelen >= sizeof(struct sockaddr_in));
memset(addrbuf, 0, sizeof(addrbuf));
@@ -923,6 +926,9 @@
set_socket_nonblocking(news);
+ if (options->ConstrainedSockets)
+ set_constrained_socket_buffers(news, (int)options->ConstrainedSockSize);
+
tor_assert(((struct sockaddr*)addrbuf)->sa_family == conn->socket_family);
if (conn->socket_family == AF_INET) {
@@ -1096,6 +1102,9 @@
set_socket_nonblocking(s);
+ if (options->ConstrainedSockets)
+ set_constrained_socket_buffers(s, (int)options->ConstrainedSockSize);
+
memset(&dest_addr,0,sizeof(dest_addr));
dest_addr.sin_family = AF_INET;
dest_addr.sin_port = htons(port);
@@ -1261,6 +1270,7 @@
if (listensockaddr) {
conn = connection_create_listener(listensockaddr, type, address);
+ tor_free(listensockaddr);
tor_free(address);
} else
conn = NULL;
@@ -1271,7 +1281,7 @@
if (new_conns)
smartlist_add(new_conns, conn);
}
- });
+ });
}
if (free_launch_elts) {
@@ -1362,14 +1372,19 @@
* tokens we just put in. */
static int write_buckets_empty_last_second = 0;
+/** How many seconds of no active local circuits will make the
+ * connection revert to the "relayed" bandwidth class? */
+#define CLIENT_IDLE_TIME_FOR_PRIORITY 30
+
/** Return 1 if <b>conn</b> should use tokens from the "relayed"
* bandwidth rates, else 0. Currently, only OR conns with bandwidth
* class 1, and directory conns that are serving data out, count.
*/
static int
-connection_counts_as_relayed_traffic(connection_t *conn)
+connection_counts_as_relayed_traffic(connection_t *conn, time_t now)
{
- if (conn->type == CONN_TYPE_OR && !TO_OR_CONN(conn)->client_used)
+ if (conn->type == CONN_TYPE_OR &&
+ TO_OR_CONN(conn)->client_used + CLIENT_IDLE_TIME_FOR_PRIORITY < now)
return 1;
if (conn->type == CONN_TYPE_DIR && DIR_CONN_IS_SERVER(conn))
return 1;
@@ -1412,7 +1427,7 @@
/** How many bytes at most can we read onto this connection? */
static int
-connection_bucket_read_limit(connection_t *conn)
+connection_bucket_read_limit(connection_t *conn, time_t now)
{
int base = connection_speaks_cells(conn) ?
CELL_NETWORK_SIZE : RELAY_PAYLOAD_SIZE;
@@ -1431,7 +1446,7 @@
return conn_bucket>=0 ? conn_bucket : 1<<14;
}
- if (connection_counts_as_relayed_traffic(conn) &&
+ if (connection_counts_as_relayed_traffic(conn, now) &&
global_relayed_read_bucket <= global_read_bucket)
global_bucket = global_relayed_read_bucket;
@@ -1441,7 +1456,7 @@
/** How many bytes at most can we write onto this connection? */
int
-connection_bucket_write_limit(connection_t *conn)
+connection_bucket_write_limit(connection_t *conn, time_t now)
{
int base = connection_speaks_cells(conn) ?
CELL_NETWORK_SIZE : RELAY_PAYLOAD_SIZE;
@@ -1453,7 +1468,7 @@
return conn->outbuf_flushlen;
}
- if (connection_counts_as_relayed_traffic(conn) &&
+ if (connection_counts_as_relayed_traffic(conn, now) &&
global_relayed_write_bucket <= global_write_bucket)
global_bucket = global_relayed_write_bucket;
@@ -1526,7 +1541,7 @@
if (num_written > 0)
rep_hist_note_bytes_written(num_written, now);
- if (connection_counts_as_relayed_traffic(conn)) {
+ if (connection_counts_as_relayed_traffic(conn, now)) {
global_relayed_read_bucket -= num_read;
global_relayed_write_bucket -= num_written;
}
@@ -1545,7 +1560,7 @@
if (global_read_bucket <= 0) {
reason = "global read bucket exhausted. Pausing.";
- } else if (connection_counts_as_relayed_traffic(conn) &&
+ } else if (connection_counts_as_relayed_traffic(conn, time(NULL)) &&
global_relayed_read_bucket <= 0) {
reason = "global relayed read bucket exhausted. Pausing.";
} else if (connection_speaks_cells(conn) &&
@@ -1569,7 +1584,7 @@
if (global_write_bucket <= 0) {
reason = "global write bucket exhausted. Pausing.";
- } else if (connection_counts_as_relayed_traffic(conn) &&
+ } else if (connection_counts_as_relayed_traffic(conn, time(NULL)) &&
global_relayed_write_bucket <= 0) {
reason = "global relayed write bucket exhausted. Pausing.";
#if 0
@@ -1617,7 +1632,7 @@
/** A second has rolled over; increment buckets appropriately. */
void
-connection_bucket_refill(int seconds_elapsed)
+connection_bucket_refill(int seconds_elapsed, time_t now)
{
or_options_t *options = get_options();
smartlist_t *conns = get_connection_array();
@@ -1668,7 +1683,7 @@
if (conn->read_blocked_on_bw == 1 /* marked to turn reading back on now */
&& global_read_bucket > 0 /* and we're allowed to read */
- && (!connection_counts_as_relayed_traffic(conn) ||
+ && (!connection_counts_as_relayed_traffic(conn, now) ||
global_relayed_read_bucket > 0) /* even if we're relayed traffic */
&& (!connection_speaks_cells(conn) ||
conn->state != OR_CONN_STATE_OPEN ||
@@ -1682,7 +1697,7 @@
if (conn->write_blocked_on_bw == 1
&& global_write_bucket > 0 /* and we're allowed to write */
- && (!connection_counts_as_relayed_traffic(conn) ||
+ && (!connection_counts_as_relayed_traffic(conn, now) ||
global_relayed_write_bucket > 0)) {
/* even if we're relayed traffic */
LOG_FN_CONN(conn, (LOG_DEBUG,LD_NET,
@@ -1831,7 +1846,8 @@
if (at_most == -1) { /* we need to initialize it */
/* how many bytes are we allowed to read? */
- at_most = connection_bucket_read_limit(conn);
+ /* XXXX020 too many calls to time(). Do they hurt? */
+ at_most = connection_bucket_read_limit(conn, time(NULL));
}
bytes_in_buf = buf_capacity(conn->inbuf) - buf_datalen(conn->inbuf);
@@ -2058,7 +2074,7 @@
}
max_to_write = force ? (int)conn->outbuf_flushlen
- : connection_bucket_write_limit(conn);
+ : connection_bucket_write_limit(conn, now);
if (connection_speaks_cells(conn) &&
conn->state > OR_CONN_STATE_PROXY_READING) {
@@ -2413,6 +2429,7 @@
if (conn->type == CONN_TYPE_OR_LISTENER ||
conn->type == CONN_TYPE_AP_LISTENER ||
conn->type == CONN_TYPE_AP_TRANS_LISTENER ||
+ conn->type == CONN_TYPE_AP_DNS_LISTENER ||
conn->type == CONN_TYPE_AP_NATD_LISTENER ||
conn->type == CONN_TYPE_DIR_LISTENER ||
conn->type == CONN_TYPE_CONTROL_LISTENER)
@@ -2539,6 +2556,29 @@
}
}
+/** Some systems have limited system buffers for recv and xmit on
+ * sockets allocated in a virtual server or similar environment. For a Tor
+ * server this can produce the "Error creating network socket: No buffer
+ * space available" error once all available TCP buffer space is consumed.
+ * This method will attempt to constrain the buffers allocated for the socket
+ * to the desired size to stay below system TCP buffer limits.
+ */
+static void
+set_constrained_socket_buffers(int sock, int size)
+{
+ void *sz = (void*)&size;
+ if (setsockopt(sock, SOL_SOCKET, SO_SNDBUF, sz, sizeof(size)) < 0) {
+ int e = tor_socket_errno(sock);
+ log_warn(LD_NET, "setsockopt() to constrain send "
+ "buffer to %d bytes failed: %s", size, tor_socket_strerror(e));
+ }
+ if (setsockopt(sock, SOL_SOCKET, SO_RCVBUF, sz, sizeof(size)) < 0) {
+ int e = tor_socket_errno(sock);
+ log_warn(LD_NET, "setsockopt() to constrain recv "
+ "buffer to %d bytes failed: %s", size, tor_socket_strerror(e));
+ }
+}
+
/** Process new bytes that have arrived on conn-\>inbuf.
*
* This function just passes conn to the connection-specific
Modified: tor/branches/114-dist-storage/src/or/connection_edge.c
===================================================================
More information about the tor-commits
mailing list