April 2011 - tor-commits - lists.torproject.org

[tor/release-0.2.2] Consider sending stream-level SENDME cells on partial flushes.
by arma＠torproject.org 01 Apr '11

01 Apr '11

commit 1d36a8e9ae766f67097f4c5e522c33ff0c4a693f Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 17:48:45 2011 -0400 Consider sending stream-level SENDME cells on partial flushes. Right now, we only consider sending stream-level SENDME cells when we have completely flushed a connection_edge's outbuf, or when it sends us a DATA cell. Neither of these is ideal for throughput. This patch changes the behavior so we now call connection_edge_consider_sending_sendme when we flush _some_ data from an edge outbuf. Fix for bug 2756; bugfix on svn r152. --- changes/bug2756 | 11 +++++++++++ src/or/connection.c | 2 ++ src/or/connection_edge.c | 17 +++++++++++++++++ src/or/connection_edge.h | 1 + 4 files changed, 31 insertions(+), 0 deletions(-) diff --git a/changes/bug2756 b/changes/bug2756 new file mode 100644 index 0000000..0cad515 --- /dev/null +++ b/changes/bug2756 @@ -0,0 +1,11 @@ + o Minor bugfixes (spec conformance, performance): + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data + (which it wasn't supposed to do if it had exhausted its + window!) or until we had completely flushed all our queued + data. Fixing this should improve throughput. Fixes bug 2756; + bugfix on the earliest released versions of Tor (svn commit + r152). + diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..86ed45e 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -3245,6 +3245,8 @@ connection_flushed_some(connection_t *conn) r = connection_dirserv_flushed_some(TO_DIR_CONN(conn)); } else if (conn->type == CONN_TYPE_OR) { r = connection_or_flushed_some(TO_OR_CONN(conn)); + } else if (CONN_IS_EDGE(conn)) { + r = connection_edge_flushed_some(TO_EDGE_CONN(conn)); } conn->in_flushed_some = 0; return r; diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index f02479f..85a5225 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -301,6 +301,23 @@ connection_edge_end_errno(edge_connection_t *conn) return connection_edge_end(conn, reason); } +/** We just wrote some data to <b>conn</b>; act appropriately. + * + * (That is, if it's open, consider sending a stream-level sendme cell if we + * have just flushed enough.) + */ +int +connection_edge_flushed_some(edge_connection_t *conn) +{ + switch (conn->_base.state) { + case AP_CONN_STATE_OPEN: + case EXIT_CONN_STATE_OPEN: + connection_edge_consider_sending_sendme(conn); + break; + } + return 0; +} + /** Connection <b>conn</b> has finished writing and has no bytes left on * its outbuf. * diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h index fa5f91c..0b08dd0 100644 --- a/src/or/connection_edge.h +++ b/src/or/connection_edge.h @@ -23,6 +23,7 @@ int connection_edge_process_inbuf(edge_connection_t *conn, int connection_edge_destroy(circid_t circ_id, edge_connection_t *conn); int connection_edge_end(edge_connection_t *conn, uint8_t reason); int connection_edge_end_errno(edge_connection_t *conn); +int connection_edge_flushed_some(edge_connection_t *conn); int connection_edge_finished_flushing(edge_connection_t *conn); int connection_edge_finished_connecting(edge_connection_t *conn);

1 0

[tor/release-0.2.2] Fix a log message typo
by arma＠torproject.org 01 Apr '11

01 Apr '11

commit 98abe1420d21cf67931229fb776117927c7ab1d6 Author: Robert Ransom <rransom.8774(a)gmail.com> Date: Tue Mar 15 09:48:26 2011 -0700 Fix a log message typo --- changes/log-typo-2011-03-15-01 | 3 +++ src/or/routerparse.c | 2 +- 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/changes/log-typo-2011-03-15-01 b/changes/log-typo-2011-03-15-01 new file mode 100644 index 0000000..3830df3 --- /dev/null +++ b/changes/log-typo-2011-03-15-01 @@ -0,0 +1,3 @@ + o Minor bugfixes + - Fix a minor typo in a log message. Bugfix on 0.2.2.6-alpha. + diff --git a/src/or/routerparse.c b/src/or/routerparse.c index 28ce97e..e0605dc 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -2822,7 +2822,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, ns->flavor = flav = flavor; } if (flav != FLAV_NS && ns_type != NS_TYPE_CONSENSUS) { - log_warn(LD_DIR, "Flavor found on non-consenus networkstatus."); + log_warn(LD_DIR, "Flavor found on non-consensus networkstatus."); goto err; }

1 0

[tor/release-0.2.2] Check newconn, not conn, before marking newconn for close
by arma＠torproject.org 01 Apr '11

01 Apr '11

commit 06f026de6624ea0591b8d08da9c5c69ffa44568c Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 18:52:15 2011 -0400 Check newconn, not conn, before marking newconn for close This fixes bug 2757. It's a bugfix on 85da676108f0d. --- changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/changes/bug2757 b/changes/bug2757 new file mode 100644 index 0000000..f947afa --- /dev/null +++ b/changes/bug2757 @@ -0,0 +1,6 @@ + - Minor bugfixes + o Avoid a double-mark-for-free warning when failing to attach a + transparent proxy connection. (We thought we had fixed this in + 0.2.2.23-alpha, but it turns out our fix was checking the wrong + connection.) Fixes bug 2757; bugfix on 0.1.2.1-alpha (the original + bug) and 0.2.2.23-alpha (the incorrect fix). diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..1894bc7 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1178,7 +1178,7 @@ connection_handle_listener_read(connection_t *conn, int new_type) } if (connection_init_accepted_conn(newconn, conn->type) < 0) { - if (! conn->marked_for_close) + if (! newconn->marked_for_close) connection_mark_for_close(newconn); return 0; }

1 0

[tor/release-0.2.2] Fix two compile warnings when using clang
by arma＠torproject.org 01 Apr '11

01 Apr '11

commit 473ff2656355a796761e9eb5e15b30cd853c58b6 Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Thu Mar 10 01:47:00 2011 +0100 Fix two compile warnings when using clang Issue found by Steven Murdoch --- src/or/geoip.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/src/or/geoip.c b/src/or/geoip.c index e5694b9..f3e0b72 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -278,6 +278,8 @@ geoip_is_loaded(void) return geoip_countries != NULL && geoip_entries != NULL; } +#define MAX_LAST_SEEN_IN_MINUTES 0x3FFFFFFFu + /** Entry in a map from IP address to the last time we've seen an incoming * connection from that IP address. Used by bridges only, to track which * countries have them blocked. */ @@ -413,12 +415,13 @@ geoip_note_client_seen(geoip_client_action_t action, lookup.ipaddr = addr; lookup.action = (int)action; ent = HT_FIND(clientmap, &client_history, &lookup); + tor_assert(now / 60 <= MAX_LAST_SEEN_IN_MINUTES); if (ent) { - ent->last_seen_in_minutes = now / 60; + ent->last_seen_in_minutes = (unsigned)(now/60); } else { ent = tor_malloc_zero(sizeof(clientmap_entry_t)); ent->ipaddr = addr; - ent->last_seen_in_minutes = now / 60; + ent->last_seen_in_minutes = (unsigned)(now/60); ent->action = (int)action; HT_INSERT(clientmap, &client_history, ent); }

1 0

[tor/release-0.2.2] Fix compilation under LLVM/clang with --enable-gcc-warnings
by arma＠torproject.org 01 Apr '11

01 Apr '11

commit 56bdc844ba68ac0911efc7ad3398f1eafeaaac76 Author: Steven Murdoch <Steven.Murdoch(a)cl.cam.ac.uk> Date: Wed Mar 9 19:05:51 2011 +0000 Fix compilation under LLVM/clang with --enable-gcc-warnings - When compiling using clang (2.9 or lower) do not enable -Wnormalized=id or -Woverride-init when --enable-gcc-warnings or --enable-gcc-warnings-advisory is set as these options are unsupported. --- configure.in | 17 +++++++++++++++-- 1 files changed, 15 insertions(+), 2 deletions(-) diff --git a/configure.in b/configure.in index f30402d..9999f60 100644 --- a/configure.in +++ b/configure.in @@ -894,6 +894,11 @@ if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xy #error #endif])], have_gcc43=yes, have_gcc43=no) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ +#if !defined(__clang__) || (__clang_major__ > 2) || (__clang_major__ == 2 && __clang_minor__ > 9) +#error +#endif])], have_clang29orlower=yes, have_clang29orlower=no) + save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Wshorten-64-to-32" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], have_shorten64_flag=yes, @@ -924,11 +929,19 @@ if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xy if test x$have_gcc42 = xyes ; then # These warnings break gcc 4.0.2 and work on gcc 4.2 # XXXX020 See if any of these work with earlier versions. - CFLAGS="$CFLAGS -Waddress -Wmissing-noreturn -Wnormalized=id -Woverride-init -Wstrict-overflow=1" + CFLAGS="$CFLAGS -Waddress -Wmissing-noreturn -Wstrict-overflow=1" + # We used to use -Wstrict-overflow=5, but that breaks us heavily under 4.3. fi - if test x$have_gcc43 = xyes ; then + if test x$have_gcc42 = xyes && test x$have_clang29orlower = xno; then + # These warnings break gcc 4.0.2 and clang, but work on gcc 4.2 + # We only disable these for clang 2.9 and lower, in case they are + # supported in later versions. + CFLAGS="$CFLAGS -Wnormalized=id -Woverride-init" + fi + + if test x$have_gcc43 = xyes ; then # These warnings break gcc 4.2 and work on gcc 4.3 # XXXX020 See if any of these work with earlier versions. CFLAGS="$CFLAGS -Wextra -Warray-bounds"

1 0

[metrics-web/master] Add v3 certificates tarball to data page.
by karsten＠torproject.org 01 Apr '11

01 Apr '11

commit d105d7e8e394515556219d03eb4dcf0f2e864866 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Apr 1 15:17:13 2011 +0200 Add v3 certificates tarball to data page. --- .../torproject/ernie/web/ResearchDataServlet.java | 7 +++++++ web/WEB-INF/data.jsp | 9 +++++++++ 2 files changed, 16 insertions(+), 0 deletions(-) diff --git a/src/org/torproject/ernie/web/ResearchDataServlet.java b/src/org/torproject/ernie/web/ResearchDataServlet.java index ca6bf7b..5186ca8 100644 --- a/src/org/torproject/ernie/web/ResearchDataServlet.java +++ b/src/org/torproject/ernie/web/ResearchDataServlet.java @@ -68,6 +68,7 @@ public class ResearchDataServlet extends HttpServlet { * arrays containing the file url and optional signature file. */ SortedMap<Date, Map<String, String[]>> relayDescriptors = new TreeMap<Date, Map<String, String[]>>(); + String[] certs = new String[2]; SortedMap<Date, String[]> bridgeDescriptors = new TreeMap<Date, String[]>(); SortedMap<String, Map<String, String[]>> relayStatistics = @@ -122,6 +123,11 @@ public class ResearchDataServlet extends HttpServlet { } relayDescriptors.get(month).get(type)[index] = url; + /* URL contains v3 certificates. */ + } else if (filename.startsWith("certs.tar")) { + int index = filename.endsWith(".asc") ? 1 : 0; + certs[index] = url; + /* URL contains bridge descriptors. */ } else if (filename.startsWith("bridge-descriptors-20")) { String yearMonth = filename.substring(filename.indexOf("20")); @@ -231,6 +237,7 @@ public class ResearchDataServlet extends HttpServlet { /* Add the maps to the request and forward it to the JSP to display * the page. */ request.setAttribute("relayDescriptors", relayDescriptors); + request.setAttribute("certs", certs); request.setAttribute("bridgeDescriptors", bridgeDescriptors); request.setAttribute("relayStatistics", relayStatistics); request.setAttribute("torperfData", torperfData); diff --git a/web/WEB-INF/data.jsp b/web/WEB-INF/data.jsp index 30d42f6..fdd5a69 100644 --- a/web/WEB-INF/data.jsp +++ b/web/WEB-INF/data.jsp @@ -97,6 +97,15 @@ </tr> </c:forEach> </table> + <c:if test="${certs[0] ne null}"> + <br> + <p>In order to verify the v3 votes and v3 consensuses, download + the tarball of <a href="${certs[0]}">v3 certificates</a> + <c:if test="${certs[1] ne null}"> + (<a href="${certs[1]}">sig</a>) + </c:if> + which is updated whenever new v3 certificates become available.</p> + </c:if> <br> <a name="bridgedesc"></a> <h3>Bridge descriptor archives</h3>

1 0

[metrics-db/master] Extract certificates from votes while parsing them.
by karsten＠torproject.org 01 Apr '11

01 Apr '11

commit 4d74615040dc9b55ecee9f7beb4326275d006bbd Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Apr 1 14:00:01 2011 +0200 Extract certificates from votes while parsing them. --- src/org/torproject/ernie/db/ArchiveWriter.java | 20 ++++++++++++-- .../torproject/ernie/db/RelayDescriptorParser.java | 28 +++++++++++++++++--- 2 files changed, 41 insertions(+), 7 deletions(-) diff --git a/src/org/torproject/ernie/db/ArchiveWriter.java b/src/org/torproject/ernie/db/ArchiveWriter.java index 68cdbaa..e30cd1b 100644 --- a/src/org/torproject/ernie/db/ArchiveWriter.java +++ b/src/org/torproject/ernie/db/ArchiveWriter.java @@ -12,7 +12,7 @@ import org.apache.commons.codec.binary.*; public class ArchiveWriter { private Logger logger; private File outputDirectory; - private int storedConsensuses = 0, storedVotes = 0, + private int storedConsensuses = 0, storedVotes = 0, storedCerts = 0, storedServerDescriptors = 0, storedExtraInfoDescriptors = 0; public ArchiveWriter(File outputDirectory) { @@ -68,6 +68,18 @@ public class ArchiveWriter { } } + public void storeCertificate(byte[] data, String fingerprint, + long published) { + SimpleDateFormat printFormat = new SimpleDateFormat( + "yyyy-MM-dd-HH-mm-ss"); + printFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + String filename = outputDirectory + "/certs/" + + fingerprint + "-" + printFormat.format(new Date(published)); + if (this.store(data, filename)) { + this.storedCerts++; + } + } + public void storeServerDescriptor(byte[] data, String digest, long published) { SimpleDateFormat printFormat = new SimpleDateFormat("yyyy/MM/"); @@ -99,11 +111,13 @@ public class ArchiveWriter { public void intermediateStats(String event) { intermediateStats.append("While " + event + ", we stored " + this.storedConsensuses + " consensus(es), " + this.storedVotes - + " vote(s), " + this.storedServerDescriptors - + " server descriptor(s), and " + this.storedExtraInfoDescriptors + + " vote(s), " + this.storedCerts + " certificate(s), " + + this.storedServerDescriptors + " server descriptor(s), and " + + this.storedExtraInfoDescriptors + " extra-info descriptor(s) to disk.\n"); this.storedConsensuses = 0; this.storedVotes = 0; + this.storedCerts = 0; this.storedServerDescriptors = 0; this.storedExtraInfoDescriptors = 0; } diff --git a/src/org/torproject/ernie/db/RelayDescriptorParser.java b/src/org/torproject/ernie/db/RelayDescriptorParser.java index bfd0033..089b743 100644 --- a/src/org/torproject/ernie/db/RelayDescriptorParser.java +++ b/src/org/torproject/ernie/db/RelayDescriptorParser.java @@ -77,11 +77,21 @@ public class RelayDescriptorParser { boolean isConsensus = true; String validAfterTime = null, fingerprint = null, dirSource = null; - long validAfter = -1L; + long validAfter = -1L, dirKeyPublished = -1L; SortedSet<String> dirSources = new TreeSet<String>(); SortedSet<String> serverDescriptors = new TreeSet<String>(); SortedSet<String> hashedRelayIdentities = new TreeSet<String>(); + StringBuilder certificateStringBuilder = null; + String certificateString = null; while ((line = br.readLine()) != null) { + if (certificateStringBuilder != null) { + if (line.startsWith("r ")) { + certificateString = certificateStringBuilder.toString(); + certificateStringBuilder = null; + } else { + certificateStringBuilder.append(line + "\n"); + } + } if (line.equals("vote-status vote")) { isConsensus = false; } else if (line.startsWith("valid-after ")) { @@ -91,8 +101,16 @@ public class RelayDescriptorParser { dirSource = line.split(" ")[2]; } else if (line.startsWith("vote-digest ")) { dirSources.add(dirSource); + } else if (line.startsWith("dir-key-certificate-version ")) { + certificateStringBuilder = new StringBuilder(); + certificateStringBuilder.append(line + "\n"); } else if (line.startsWith("fingerprint ")) { fingerprint = line.split(" ")[1]; + } else if (line.startsWith("dir-key-published ")) { + String dirKeyPublishedTime = line.substring( + "dir-key-published ".length()); + dirKeyPublished = parseFormat.parse(dirKeyPublishedTime). + getTime(); } else if (line.startsWith("r ")) { String[] parts = line.split(" "); if (parts.length < 9) { @@ -137,9 +155,11 @@ public class RelayDescriptorParser { byte[] forDigest = new byte[sig - start]; System.arraycopy(data, start, forDigest, 0, sig - start); String digest = DigestUtils.shaHex(forDigest).toUpperCase(); - if (this.aw != null) { - this.aw.storeVote(data, validAfter, dirSource, digest); - } + this.aw.storeVote(data, validAfter, dirSource, digest); + } + if (certificateString != null) { + this.aw.storeCertificate(certificateString.getBytes(), + dirSource, dirKeyPublished); } } }

1 0

r24525: {website} Unbreak the var tag (website/trunk/css)
by Robert Ransom 01 Apr '11

01 Apr '11

Author: rransom Date: 2011-04-01 04:46:33 +0000 (Fri, 01 Apr 2011) New Revision: 24525 Modified: website/trunk/css/typography.css Log: Unbreak the var tag Modified: website/trunk/css/typography.css =================================================================== --- website/trunk/css/typography.css 2011-04-01 02:55:51 UTC (rev 24524) +++ website/trunk/css/typography.css 2011-04-01 04:46:33 UTC (rev 24525) @@ -69,7 +69,7 @@ a:hover { color: #a07320; } strong { font-weight: bold; } -em { font-style: italic; } +em, var { font-style: italic; } blockquote { color: #454545;

1 0

r24524: {website} Warn users to not unpack or run TBB as root Everyone is *sup (website/trunk/projects/en)
by Robert Ransom 01 Apr '11

01 Apr '11

Author: rransom Date: 2011-04-01 02:55:51 +0000 (Fri, 01 Apr 2011) New Revision: 24524 Modified: website/trunk/projects/en/torbrowser.wml Log: Warn users to not unpack or run TBB as root Everyone is *supposed to* know that running big messy programs as root is a Bad Thing, but unpacking or running TBB as root currently makes it fail to run entirely. (See bug #2824.) Modified: website/trunk/projects/en/torbrowser.wml =================================================================== --- website/trunk/projects/en/torbrowser.wml 2011-04-01 02:23:15 UTC (rev 24523) +++ website/trunk/projects/en/torbrowser.wml 2011-04-01 02:55:51 UTC (rev 24524) @@ -227,7 +227,7 @@ <var>LANG</var> is the language listed in the filename), and either double click on the directory or <code>cd</code> into it, then execute the <strong>start-tor-browser</strong> script. This will launch Vidalia and once that connects to Tor, it will launch - Firefox.</p> + Firefox. <em>Do not unpack or run TBB as root.</em></p> <a id="Windows"></a> <h3><a class="anchor" href="#Windows">Windows instructions</a></h3>

1 0

r24523: {website} Finish formatting the Linux TBB instructions properly (website/trunk/projects/en)
by Robert Ransom 01 Apr '11

01 Apr '11

Author: rransom Date: 2011-04-01 02:23:15 +0000 (Fri, 01 Apr 2011) New Revision: 24523 Modified: website/trunk/projects/en/torbrowser.wml Log: Finish formatting the Linux TBB instructions properly Modified: website/trunk/projects/en/torbrowser.wml =================================================================== --- website/trunk/projects/en/torbrowser.wml 2011-04-01 02:19:19 UTC (rev 24522) +++ website/trunk/projects/en/torbrowser.wml 2011-04-01 02:23:15 UTC (rev 24523) @@ -224,7 +224,7 @@ run:<br> <code>tar -xvzf tor-browser-gnu-linux-i686-<version-torbrowserbundlelinux32>-dev-<var>LANG</var>.tar.gz</code> or <code>tor-browser-gnu-linux-x86_64-<version-torbrowserbundlelinux64>-dev-<var>LANG</var>.tar.gz</code> (where - LANG is the language listed in the filename), and either double click on the + <var>LANG</var> is the language listed in the filename), and either double click on the directory or <code>cd</code> into it, then execute the <strong>start-tor-browser</strong> script. This will launch Vidalia and once that connects to Tor, it will launch Firefox.</p>

1 0