June 2012 - tor-commits - lists.torproject.org

[tor/master] Remove validate_pluggable_transports_config(): redundant since 9d9b5ed0.
by nickm＠torproject.org 14 Jun '12

14 Jun '12

commit aa212b173c340536eb94b4978174b068ead89cb1 Author: George Kadianakis <desnacked(a)riseup.net> Date: Thu Jun 14 18:01:22 2012 +0300 Remove validate_pluggable_transports_config(): redundant since 9d9b5ed0. The warning message of validate_pluggable_transports_config() is superseded by the changes in the warning message of connection_or_connect() when the proxy credentials can't be found. --- changes/bug5589 | 3 +++ src/or/circuitbuild.c | 28 ---------------------------- src/or/circuitbuild.h | 2 -- src/or/connection_or.c | 7 ++++--- src/or/main.c | 9 --------- 5 files changed, 7 insertions(+), 42 deletions(-) diff --git a/changes/bug5589 b/changes/bug5589 new file mode 100644 index 0000000..cbcaaa7 --- /dev/null +++ b/changes/bug5589 @@ -0,0 +1,3 @@ + o Code simplifications and refactoring: + - Remove validate_pluggable_transports_config(): its warning + message is now handled by connection_or_connect(). diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 3ab72e4..807738b 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -4795,34 +4795,6 @@ transport_add_from_config(const tor_addr_t *addr, uint16_t port, } } -/** Warn the user of possible pluggable transport misconfiguration. - * Return 0 if the validation happened, -1 if we should postpone the - * validation. */ -int -validate_pluggable_transports_config(void) -{ - /* Don't validate if managed proxies are not yet fully configured. */ - if (!bridge_list || pt_proxies_configuration_pending()) - return -1; - - SMARTLIST_FOREACH_BEGIN(bridge_list, const bridge_info_t *, b) { - /* Skip bridges without transports. */ - if (!b->transport_name) - continue; - /* See if the user has Bridges that specify nonexistent - pluggable transports. We should warn the user in such case, - since it's probably misconfiguration. */ - if (!transport_get_by_name(b->transport_name)) - log_warn(LD_CONFIG, "We can't find a pluggable transport proxy " - "that supports '%s' for bridge '%s:%u'. This can happen " - "if you haven't provided a ClientTransportPlugin line, or " - "if your pluggable transport proxy stopped working.", - b->transport_name, fmt_addr(&b->addr), b->port); - } SMARTLIST_FOREACH_END(b); - - return 0; -} - /** Return a bridge pointer if ri is one of our known bridges * (either by comparing keys if possible, else by comparing addr/port). * Else return NULL. */ diff --git a/src/or/circuitbuild.h b/src/or/circuitbuild.h index 5b77399..72ce8ef 100644 --- a/src/or/circuitbuild.h +++ b/src/or/circuitbuild.h @@ -166,7 +166,5 @@ int find_transport_by_bridge_addrport(const tor_addr_t *addr, uint16_t port, const transport_t **transport); transport_t *transport_get_by_name(const char *name); -int validate_pluggable_transports_config(void); - #endif diff --git a/src/or/connection_or.c b/src/or/connection_or.c index cbb1110..467e169 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -1085,9 +1085,10 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, if (transport_name) { log_warn(LD_GENERAL, "We were supposed to connect to bridge '%s:%u' " - "using pluggable transport '%s', but it seems that we can't " - "find a pluggable transport proxy supporting '%s'. Please make " - "sure that your configuration file is valid.", + "using pluggable transport '%s', but we can't find a pluggable " + "transport proxy supporting '%s'. This can happen if you " + "haven't provided a ClientTransportPlugin line, or if " + "your pluggable transport proxy stopped running.", fmt_addr(&TO_CONN(conn)->addr), TO_CONN(conn)->port, transport_name, transport_name); } else { diff --git a/src/or/main.c b/src/or/main.c index 163dd5d..5ede4b8 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1118,7 +1118,6 @@ run_scheduled_events(time_t now) static int should_init_bridge_stats = 1; static time_t time_to_retry_dns_init = 0; static time_t time_to_next_heartbeat = 0; - static int has_validated_pt = 0; const or_options_t *options = get_options(); int is_server = server_mode(options); @@ -1526,14 +1525,6 @@ run_scheduled_events(time_t now) if (!net_is_disabled() && pt_proxies_configuration_pending()) pt_configure_remaining_proxies(); - /** 11c. validate pluggable transports configuration if we need to */ - if (!has_validated_pt && - (options->Bridges || options->ClientTransportPlugin)) { - if (validate_pluggable_transports_config() == 0) { - has_validated_pt = 1; - } - } - /** 12. write the heartbeat message */ if (options->HeartbeatPeriod && time_to_next_heartbeat <= now) {

1 0

[tor/master] Merge remote-tracking branch 'asn-mytor/bug5589_take2'
by nickm＠torproject.org 14 Jun '12

14 Jun '12

commit 4fdce6b091181f01295270eedb23ef7735904635 Merge: 7e57609 aa212b1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jun 14 13:05:16 2012 -0400 Merge remote-tracking branch 'asn-mytor/bug5589_take2' changes/bug5589 | 3 +++ src/or/circuitbuild.c | 28 ---------------------------- src/or/circuitbuild.h | 2 -- src/or/connection_or.c | 7 ++++--- src/or/main.c | 9 --------- 5 files changed, 7 insertions(+), 42 deletions(-)

1 0

[torspec/master] Move implemented parts of proposal 180 to /pt-spec.txt
by nickm＠torproject.org 14 Jun '12

14 Jun '12

commit 41f4fcc4816de39bb5f410a73fe800678a32340d Author: George Kadianakis <desnacked(a)riseup.net> Date: Thu Jun 14 18:44:28 2012 +0300 Move implemented parts of proposal 180 to /pt-spec.txt --- pt-spec.txt | 296 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 296 insertions(+), 0 deletions(-) diff --git a/pt-spec.txt b/pt-spec.txt new file mode 100644 index 0000000..c8f107f --- /dev/null +++ b/pt-spec.txt @@ -0,0 +1,296 @@ + + Pluggable Transport Specification + + Jacob Appelbaum + Nick Mathewson + + + +Overview + + This proposal describes a way to decouple protocol-level obfuscation + from the core Tor protocol in order to better resist client-bridge + censorship. Our approach is to specify a means to add pluggable + transport implementations to Tor clients and bridges so that they can + negotiate a superencipherment for the Tor protocol. + +Specifications: Client behavior + + We extend the bridge line format to allow you to say which method + to use to connect to a bridge. + + The new format is: + Bridge method address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v] + + To connect to such a bridge, the Tor program needs to know which + SOCKS proxy will support the transport called "method". It + then connects to this proxy, and asks it to connect to + address:port. If [id-fingerprint] is provided, Tor should expect + the public identity key on the TLS connection to match the digest + provided in [id-fingerprint]. If any [k=v] items are provided, + they are configuration parameters for the proxy: Tor should + separate them with semicolons and put them in the user and + password fields of the request, splitting them across the fields + as necessary. If a key or value value must contain a semicolon or + a backslash, it is escaped with a backslash. + + Method names must be C identifiers. + + For reference, the old bridge format was + Bridge address[:port] [id-fingerprint] + where port defaults to 443 and the id-fingerprint is optional. The + new format can be distinguished from the old one by checking if the + first argument has any non-C-identifier characters. (Looking for a + period should be a simple way.) Also, while the id-fingerprint could + optionally include whitespace in the old format, whitespace in the + id-fingerprint is not permitted in the new format. + + Example: if the bridge line is "bridge trebuchet www.example.com:3333 + keyid=09F911029D74E35BD84156C5635688C009F909F9 rocks=20 height=5.6m" + AND if the Tor client knows that the 'trebuchet' method is supported, + the client should connect to the proxy that provides the 'trebuchet' + method, ask it to connect to www.example.com, and provide the string + "rocks=20;height=5.6m" as the username, the password, or split + across the username and password. + + There are two ways to tell Tor clients about protocol proxies: + external proxies and managed proxies. An external proxy is configured + with + ClientTransportPlugin <method> socks4 <address:port> [auth=X] + or + ClientTransportPlugin <method> socks5 <address:port> [username=X] [password=Y] + as in + "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". + This example tells Tor that another program is already running to handle + 'trubuchet' connections, and Tor doesn't need to worry about it. + + A managed proxy is configured with + ClientTransportPlugin <methods> exec <path> [options] + as in + "ClientTransportPlugin trebuchet exec /usr/libexec/trebuchet --managed". + This example tells Tor to launch an external program to provide a + socks proxy for 'trebuchet' connections. The Tor client only + launches one instance of each external program with a given set of + options, even if the same executable and options are listed for + more than one method. + + In managed proxies, <methods> can be a comma-separated list of + pluggable transport method names, as in: + "ClientTransportPlugin pawn,bishop,rook exec /bin/ptproxy --managed". + + If instead of a transport method, the torrc lists "*" for a managed + proxy, Tor uses that proxy for all transport methods that the plugin + supports. So "ClientTransportPlugin * exec /usr/libexec/tor/foobar" + tells Tor that Tor should use the foobar plugin for every method that + the proxy supports. See the "Managed proxy interface" section below + for details on how Tor learns which methods a plugin supports. + + If two plugins support the same method, Tor should use whichever + one is listed first. + + The same program can implement a managed or an external proxy: it just + needs to take an argument saying which one to be. + +Server behavior + + Server proxies are configured similarly to client proxies. When + launching a proxy, the server must tell it what ORPort it has + configured, and what address (if any) it can listen on. The + server must tell the proxy which (if any) methods it should + provide if it can; the proxy needs to tell the server which + methods it is actually providing, and on what ports. + + When a client connects to the proxy, the proxy may need a way to + tell the server some identifier for the client address. It does + this in-band. + + As before, the server lists proxies in its torrc. These can be + external proxies that run on their own, or managed proxies that Tor + launches. + + An external server proxy is configured as + ServerTransportPlugin <method> proxy <address:port> <param=val> ... + as in + "ServerTransportPlugin trebuchet proxy 127.0.0.1:999 rocks=heavy". + The param=val pairs and the address are used to make the bridge + configuration information that we'll tell users. + + A managed proxy is configured as + ServerTransportPlugin <methods> exec </path/to/binary> [options] + or + ServerTransportPlugin * exec </path/to/binary> [options] + + When possible, Tor should launch only one binary of each binary/option + pair configured. So if the torrc contains + + ClientTransportPlugin foo exec /usr/bin/megaproxy --foo + ClientTransportPlugin bar exec /usr/bin/megaproxy --bar + ServerTransportPlugin * exec /usr/bin/megaproxy --foo + + then Tor will launch the megaproxy binary twice: once with the option + --foo and once with the option --bar. + +Managed proxy interface + + When the Tor client or relay launches a managed proxy, it communicates + via environment variables. At a minimum, it sets (in addition to the + normal environment variables inherited from Tor): + + {Client and server} + + "TOR_PT_STATE_LOCATION" -- A filesystem directory path where the + proxy should store state if it wants to. This directory is not + required to exist, but the proxy SHOULD be able to create it if + it doesn't. The proxy MUST NOT store state elsewhere. + Example: TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state/ + + "TOR_PT_MANAGED_TRANSPORT_VER" -- To tell the proxy which + versions of this configuration protocol Tor supports. Future + versions will give a comma-separated list. Clients MUST accept + comma-separated lists containing any version that they + recognize, and MUST work correctly even if some of the versions + they don't recognize are non-numeric. Valid version characters + are non-space, non-comma printing ASCII characters. + Example: TOR_PT_MANAGED_TRANSPORT_VER=1,1a,2,4B + + {Client only} + + "TOR_PT_CLIENT_TRANSPORTS" -- A comma-separated list of which + methods this client should enable, or * if all methods should + be enabled. The proxy SHOULD ignore methods that it doesn't + recognize. + Example: TOR_PT_CLIENT_TRANSPORTS=trebuchet,battering_ram,ballista + + {Server only} + + "TOR_PT_EXTENDED_SERVER_PORT" -- An <address>:<port> where tor + should be listening for connections speaking the extended + ORPort protocol (See the "The extended ORPort protocol" section + below). If tor does not support the extended ORPort protocol, + it MUST use the empty string as the value of this environment + variable. + Example: TOR_PT_EXTENDED_SERVER_PORT=127.0.0.1:4200 + + "TOR_PT_ORPORT" -- Our regular ORPort in a form suitable + for local connections, i.e. connections from the proxy to + the ORPort. + Example: TOR_PT_ORPORT=127.0.0.1:9001 + + "TOR_PT_SERVER_BINDADDR" -- A comma seperated list of + <key>-<value> pairs, where <key> is a transport name and + <value> is the adress:port on which it should listen for client + proxy connections. + The keys holding transport names must appear on the same order + as they appear on TOR_PT_SERVER_TRANSPORTS. + This might be the advertised address, or might be a local + address that Tor will forward ports to. It MUST be an address + that will work with bind(). + Example: + TOR_PT_SERVER_BINDADDR=trebuchet-127.0.0.1:1984,ballista-127.0.0.1:4891 + + "TOR_PT_SERVER_TRANSPORTS" -- A comma-separated list of server + methods that the proxy should support, or * if all methods + should be enabled. The proxy SHOULD ignore methods that it + doesn't recognize. + Example: TOR_PT_SERVER_TRANSPORTS=trebuchet,ballista + + The transport proxy replies by writing NL-terminated lines to + stdout. The line metaformat is + + <Line> ::= <Keyword> <OptArgs> <NL> + <Keyword> ::= <KeywordChar> | <Keyword> <KeywordChar> + <KeyWordChar> ::= <any US-ASCII alphanumeric, dash, and underscore> + <OptArgs> ::= <Args>* + <Args> ::= <SP> <ArgChar> | <Args> <ArgChar> + <ArgChar> ::= <any US-ASCII character but NUL or NL> + <SP> ::= <US-ASCII whitespace symbol (32)> + <NL> ::= <US-ASCII newline (line feed) character (10)> + + Tor MUST ignore lines with keywords that it doesn't recognize. + + First, if there's an error parsing the environment variables, the + proxy should write: + ENV-ERROR <errormessage> + and exit. + + If the environment variables were correctly formatted, the proxy + should write: + VERSION <configuration protocol version> + to say that it supports this configuration protocol version (example + "VERSION 1"). It must either pick a version that Tor told it about + in TOR_PT_MANAGED_TRANSPORT_VER, or pick no version at all, say: + VERSION-ERROR no-version + and exit. + + The proxy should then open its ports. If running as a client + proxy, it should not use fixed ports; instead it should autoselect + ports to avoid conflicts. A client proxy should by default only + listen on localhost for connections. + + A server proxy SHOULD try to listen at a consistent port, though it + SHOULD pick a different one if the port it last used is now allocated. + + A client or server proxy then should tell which methods it has + made available and how. It does this by printing zero or more + CMETHOD and SMETHOD lines to its stdout. These lines look like: + + CMETHOD <methodname> socks4/socks5 <address:port> [ARGS=arglist] \ + [OPT-ARGS=arglist] + + as in + + CMETHOD trebuchet socks5 127.0.0.1:19999 ARGS=rocks,height \ + OPT-ARGS=tensile-strength + + The ARGS field lists mandatory parameters that must appear in + every bridge line for this method. The OPT-ARGS field lists + optional parameters. If no ARGS or OPT-ARGS field is provided, + Tor should not check the parameters in bridge lines for this + method. + + The proxy should print a single "CMETHODS DONE" line after it is + finished telling Tor about the client methods it provides. If it + tries to supply a client method but can't for some reason, it + should say: + CMETHOD-ERROR <methodname> <errormessage> + + A proxy should also tell Tor about the server methods it is providing + by printing zero or more SMETHOD lines. These lines look like: + + SMETHOD <methodname> <address:port> [options] + + If there's an error setting up a configured server method, the + proxy should say: + SMETHOD-ERROR <methodname> <errormessage> + as in + SMETHOD-ERROR trebuchet could not setup 'trebuchet' method + + The 'address:port' part of an SMETHOD line is the address to put + in the bridge line. The Options part is a list of space-separated + K:V flags that Tor should know about. Recognized options are: + + SMETHOD and CMETHOD lines may be interspersed, to allow the proxies to + report methods as they become available, even when some methods may + require probing your network, connecting to some kind of peers, etc + before they are set up. After the final SMETHOD line, the proxy says + "SMETHODS DONE". + + The proxy SHOULD NOT tell Tor about a server or client method + unless it is actually open and ready to use. + + Tor clients SHOULD NOT use any method from a client proxy or + advertise any method from a server proxy UNLESS it is listed as a + possible method for that proxy in torrc, and it is listed by the + proxy as a method it supports. + + Proxies should respond to a single INT signal by closing their + listener ports and not accepting any new connections, but keeping + all connections open, then terminating when connections are all + closed. Proxies should respond to a second INT signal by shutting + down cleanly. + + The managed proxy configuration protocol version defined in this + section is "1". + So, for example, if tor supports this configuration protocol it + should set the environment variable: + TOR_PT_MANAGED_TRANSPORT_VER=1

1 0

[torspec/master] Add a pointer to prop180
by nickm＠torproject.org 14 Jun '12

14 Jun '12

commit 4cd522906f9136dce623a1c2db7487b2f723f0d3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jun 14 12:56:21 2012 -0400 Add a pointer to prop180 --- pt-spec.txt | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/pt-spec.txt b/pt-spec.txt index c8f107f..ffabb4d 100644 --- a/pt-spec.txt +++ b/pt-spec.txt @@ -14,6 +14,10 @@ Overview transport implementations to Tor clients and bridges so that they can negotiate a superencipherment for the Tor protocol. + It is based on Proposal 180: see that document and its discussion for + more background and motivation issue, and a discussion of issues in + writing pluggable transpots. + Specifications: Client behavior We extend the bridge line format to allow you to say which method

1 0

[onionoo/master] Add country parameter to search by country code.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit cedd372a13a49c11da9b4327e8b7e373c750eab6 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 17:36:33 2012 +0200 Add country parameter to search by country code. Implements #5960. --- src/org/torproject/onionoo/CurrentNodes.java | 22 ++++++--- src/org/torproject/onionoo/Node.java | 4 +- src/org/torproject/onionoo/ResourceServlet.java | 56 ++++++++++++++++++++++- web/index.html | 5 ++ 4 files changed, 78 insertions(+), 9 deletions(-) diff --git a/src/org/torproject/onionoo/CurrentNodes.java b/src/org/torproject/onionoo/CurrentNodes.java index 74879af..63fb8c6 100644 --- a/src/org/torproject/onionoo/CurrentNodes.java +++ b/src/org/torproject/onionoo/CurrentNodes.java @@ -71,9 +71,13 @@ public class CurrentNodes { if (parts.length > 9) { consensusWeight = Long.parseLong(parts[9]); } + String countryCode = "??"; + if (parts.length > 10) { + countryCode = parts[10]; + } this.addRelay(nickname, fingerprint, address, validAfterMillis, orPort, dirPort, relayFlags, - consensusWeight); + consensusWeight, countryCode); } else if (line.startsWith("b ")) { String[] parts = line.split(" "); if (parts.length < 9) { @@ -136,9 +140,11 @@ public class CurrentNodes { String relayFlags = sb.toString().substring(1); String consensusWeight = String.valueOf( entry.getConsensusWeight()); + String countryCode = entry.getCountryCode() != null + ? entry.getCountryCode() : "??"; bw.write("r " + nickname + " " + fingerprint + " " + address + " " + validAfter + " " + orPort + " " + dirPort + " " + relayFlags - + " " + consensusWeight + "\n"); + + " " + consensusWeight + " " + countryCode + "\n"); } for (Node entry : this.currentBridges.values()) { String nickname = entry.getNickname(); @@ -155,7 +161,7 @@ public class CurrentNodes { String relayFlags = sb.toString().substring(1); bw.write("b " + nickname + " " + fingerprint + " " + address + " " + published + " " + orPort + " " + dirPort + " " + relayFlags - + " -1\n"); + + " -1 ??\n"); } bw.close(); } catch (IOException e) { @@ -214,19 +220,21 @@ public class CurrentNodes { SortedSet<String> relayFlags = entry.getFlags(); long consensusWeight = entry.getBandwidth(); this.addRelay(nickname, fingerprint, address, validAfterMillis, - orPort, dirPort, relayFlags, consensusWeight); + orPort, dirPort, relayFlags, consensusWeight, null); } } public void addRelay(String nickname, String fingerprint, String address, long validAfterMillis, int orPort, int dirPort, - SortedSet<String> relayFlags, long consensusWeight) { + SortedSet<String> relayFlags, long consensusWeight, + String countryCode) { if (validAfterMillis >= cutoff && (!this.currentRelays.containsKey(fingerprint) || this.currentRelays.get(fingerprint).getLastSeenMillis() < validAfterMillis)) { Node entry = new Node(nickname, fingerprint, address, - validAfterMillis, orPort, dirPort, relayFlags, consensusWeight); + validAfterMillis, orPort, dirPort, relayFlags, consensusWeight, + countryCode); this.currentRelays.put(fingerprint, entry); if (validAfterMillis > this.lastValidAfterMillis) { this.lastValidAfterMillis = validAfterMillis; @@ -333,7 +341,7 @@ public class CurrentNodes { this.currentBridges.get(fingerprint).getLastSeenMillis() < publishedMillis)) { Node entry = new Node(nickname, fingerprint, address, - publishedMillis, orPort, dirPort, relayFlags, -1L); + publishedMillis, orPort, dirPort, relayFlags, -1L, null); this.currentBridges.put(fingerprint, entry); if (publishedMillis > this.lastPublishedMillis) { this.lastPublishedMillis = publishedMillis; diff --git a/src/org/torproject/onionoo/Node.java b/src/org/torproject/onionoo/Node.java index b7bc126..887a600 100644 --- a/src/org/torproject/onionoo/Node.java +++ b/src/org/torproject/onionoo/Node.java @@ -31,7 +31,8 @@ public class Node { private boolean running; public Node(String nickname, String fingerprint, String address, long lastSeenMillis, int orPort, int dirPort, - SortedSet<String> relayFlags, long consensusWeight) { + SortedSet<String> relayFlags, long consensusWeight, + String countryCode) { this.nickname = nickname; this.fingerprint = fingerprint; try { @@ -47,6 +48,7 @@ public class Node { this.dirPort = dirPort; this.relayFlags = relayFlags; this.consensusWeight = consensusWeight; + this.countryCode = countryCode; } public String getFingerprint() { return this.fingerprint; diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index e821500..8744150 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -47,6 +47,8 @@ public class ResourceServlet extends HttpServlet { private Map<String, String> relayFingerprintSummaryLines = new HashMap<String, String>(), bridgeFingerprintSummaryLines = new HashMap<String, String>(); + private Map<String, Set<String>> relaysByCountryCode = + new HashMap<String, Set<String>>(); private void readSummaryFile() { if (!summaryFile.exists()) { readSummaryFile = false; @@ -75,6 +77,16 @@ public class ResourceServlet extends HttpServlet { consensusWeight, fingerprint)); orderRelaysByConsensusWeight.add(String.format("%020d %s", consensusWeight, hashedFingerprint)); + String countryCode = entry.getCountryCode(); + if (countryCode == null) { + countryCode = "??"; + } + if (!this.relaysByCountryCode.containsKey(countryCode)) { + this.relaysByCountryCode.put(countryCode, + new HashSet<String>()); + } + this.relaysByCountryCode.get(countryCode).add(fingerprint); + this.relaysByCountryCode.get(countryCode).add(hashedFingerprint); } Collections.sort(orderRelaysByConsensusWeight); this.relaysByConsensusWeight = new ArrayList<String>(); @@ -178,7 +190,8 @@ public class ResourceServlet extends HttpServlet { /* Make sure that the request doesn't contain any unknown * parameters. */ Set<String> knownParameters = new HashSet<String>(Arrays.asList( - "type,running,search,lookup,order,limit,offset".split(","))); + "type,running,search,lookup,country,order,limit,offset". + split(","))); for (String parameterKey : parameterMap.keySet()) { if (!knownParameters.contains(parameterKey)) { response.sendError(HttpServletResponse.SC_BAD_REQUEST); @@ -235,6 +248,16 @@ public class ResourceServlet extends HttpServlet { this.filterByFingerprint(filteredRelays, filteredBridges, fingerprint); } + if (parameterMap.containsKey("country")) { + String countryCodeParameter = this.parseCountryCodeParameter( + parameterMap.get("country")); + if (countryCodeParameter == null) { + response.sendError(HttpServletResponse.SC_BAD_REQUEST); + return; + } + this.filterByCountryCode(filteredRelays, filteredBridges, + countryCodeParameter); + } /* Re-order and limit results. */ List<String> orderedRelays = new ArrayList<String>(); @@ -389,6 +412,15 @@ public class ResourceServlet extends HttpServlet { return parameter; } + private static Pattern countryCodeParameterPattern = + Pattern.compile("^[0-9a-zA-Z]{2}$"); + private String parseCountryCodeParameter(String parameter) { + if (!countryCodeParameterPattern.matcher(parameter).matches()) { + return null; + } + return parameter; + } + private void filterByType(Map<String, String> filteredRelays, Map<String, String> filteredBridges, boolean relaysRequested) { if (relaysRequested) { @@ -502,6 +534,28 @@ public class ResourceServlet extends HttpServlet { } } + private void filterByCountryCode(Map<String, String> filteredRelays, + Map<String, String> filteredBridges, String countryCodeParameter) { + String countryCode = countryCodeParameter.toLowerCase(); + if (!this.relaysByCountryCode.containsKey(countryCode)) { + filteredRelays.clear(); + } else { + Set<String> relaysWithCountryCode = + this.relaysByCountryCode.get(countryCode); + Set<String> removeRelays = new HashSet<String>(); + for (Map.Entry<String, String> e : filteredRelays.entrySet()) { + String fingerprint = e.getKey(); + if (!relaysWithCountryCode.contains(fingerprint)) { + removeRelays.add(fingerprint); + } + } + for (String fingerprint : removeRelays) { + filteredRelays.remove(fingerprint); + } + } + filteredBridges.clear(); + } + private void writeRelays(List<String> relays, PrintWriter pw, String resourceType) { pw.write("{\"relays_published\":\"" + this.relaysPublishedString diff --git a/web/index.html b/web/index.html index 5ea7743..589d347 100755 --- a/web/index.html +++ b/web/index.html @@ -623,6 +623,11 @@ Fingerprints should always be hashed using SHA-1, regardless of looking up a relay or a bridge, in order to not accidentally leak non-hashed bridge fingerprints in the URL. </td></tr> +<tr><td>country</td><td>Return only +relays which are located in the given country as identified by a +two-letter lower-case country code. +Added parameter on June 14, 2012. +</td></tr> </table> Relay and/or bridge documents in the response can be ordered and limited by providing further parameters.

1 0

[onionoo/master] Clean up previous commit a bit.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit 9aede3cccb380052bee988e19d435fa71ce4567c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 16:55:38 2012 +0200 Clean up previous commit a bit. --- src/org/torproject/onionoo/CurrentNodes.java | 6 +++--- src/org/torproject/onionoo/Main.java | 4 +++- src/org/torproject/onionoo/ResourceServlet.java | 9 +++++---- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/src/org/torproject/onionoo/CurrentNodes.java b/src/org/torproject/onionoo/CurrentNodes.java index 7ed16a8..74879af 100644 --- a/src/org/torproject/onionoo/CurrentNodes.java +++ b/src/org/torproject/onionoo/CurrentNodes.java @@ -35,11 +35,11 @@ import com.maxmind.geoip.regionName; * days. */ public class CurrentNodes { - private File internalRelaySearchDataFile = - new File("status/summary.csv"); + private File internalRelaySearchDataFile; /* Read the internal relay search data file from disk. */ - public void readRelaySearchDataFile() { + public void readRelaySearchDataFile(File internalRelaySearchDataFile) { + this.internalRelaySearchDataFile = internalRelaySearchDataFile; if (this.internalRelaySearchDataFile.exists() && !this.internalRelaySearchDataFile.isDirectory()) { try { diff --git a/src/org/torproject/onionoo/Main.java b/src/org/torproject/onionoo/Main.java index edac926..2c906db 100644 --- a/src/org/torproject/onionoo/Main.java +++ b/src/org/torproject/onionoo/Main.java @@ -2,13 +2,15 @@ * See LICENSE for licensing information */ package org.torproject.onionoo; +import java.io.File; + /* Update search data and status data files. */ public class Main { public static void main(String[] args) { printStatus("Updating internal node list."); CurrentNodes cn = new CurrentNodes(); - cn.readRelaySearchDataFile(); + cn.readRelaySearchDataFile(new File("out/summary")); cn.readRelayNetworkConsensuses(); cn.setRelayRunningBits(); cn.lookUpCountries(); diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index 38ae248..e821500 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -29,13 +29,14 @@ public class ResourceServlet extends HttpServlet { private static final long serialVersionUID = 7236658979947465319L; - private File summaryFile = new File("status/summary.csv"); + private File summaryFile; private String outDirString; public void init(ServletConfig config) throws ServletException { super.init(config); this.outDirString = config.getInitParameter("outDir"); + this.summaryFile = new File(outDirString, "summary"); this.readSummaryFile(); } @@ -53,7 +54,7 @@ public class ResourceServlet extends HttpServlet { } if (summaryFile.lastModified() > this.summaryFileLastModified) { CurrentNodes cn = new CurrentNodes(); - cn.readRelaySearchDataFile(); + cn.readRelaySearchDataFile(this.summaryFile); SimpleDateFormat dateTimeFormat = new SimpleDateFormat( "yyyy-MM-dd HH:mm:ss"); dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); @@ -100,7 +101,7 @@ public class ResourceServlet extends HttpServlet { String fingerprint = entry.getFingerprint(); String running = entry.getRunning() ? "true" : "false"; String address = entry.getAddress(); - return String.format("\n{%s\"f\":\"%s\",\"a\":[\"%s\"],\"r\":%s}", + return String.format("{%s\"f\":\"%s\",\"a\":[\"%s\"],\"r\":%s}", (nickname == null ? "" : "\"n\":\"" + nickname + "\","), fingerprint, address, running); } @@ -110,7 +111,7 @@ public class ResourceServlet extends HttpServlet { entry.getNickname() : null; String hashedFingerprint = entry.getFingerprint(); String running = entry.getRunning() ? "true" : "false"; - return String.format("\n{%s\"h\":\"%s\",\"r\":%s}", + return String.format("{%s\"h\":\"%s\",\"r\":%s}", (nickname == null ? "" : "\"n\":\"" + nickname + "\","), hashedFingerprint, running); }

1 0

[onionoo/master] Use more appropriate node summary file in servlet.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit 26da7450d5aa3bc0d9926d69686565e8fb95e75d Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 16:02:17 2012 +0200 Use more appropriate node summary file in servlet. We used a JSON-formatted summary document and a .csv file to decide which relays and bridges to return in the servlet. This is overly complex and unnecessary. We have a fine summary file that contains all information we need and that can be easily extended. --- src/org/torproject/onionoo/DetailDataWriter.java | 29 ---- src/org/torproject/onionoo/Main.java | 8 +- src/org/torproject/onionoo/Node.java | 15 ++ src/org/torproject/onionoo/ResourceServlet.java | 170 ++++++++------------- src/org/torproject/onionoo/SummaryDataWriter.java | 93 ----------- 5 files changed, 82 insertions(+), 233 deletions(-) diff --git a/src/org/torproject/onionoo/DetailDataWriter.java b/src/org/torproject/onionoo/DetailDataWriter.java index 1e3be72..bb4dbfa 100644 --- a/src/org/torproject/onionoo/DetailDataWriter.java +++ b/src/org/torproject/onionoo/DetailDataWriter.java @@ -211,8 +211,6 @@ public class DetailDataWriter { SimpleDateFormat dateTimeFormat = new SimpleDateFormat( "yyyy-MM-dd HH:mm:ss"); dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - Map<String, Long> relaysByConsensusWeight = - new HashMap<String, Long>(); for (Map.Entry<String, Node> relay : this.relays.entrySet()) { String fingerprint = relay.getKey(); @@ -416,33 +414,6 @@ public class DetailDataWriter { + "broken. Ignoring."); e.printStackTrace(); } - - /* Remember consensus weight to facilitate ordering of results in - * the servlet. */ - relaysByConsensusWeight.put(fingerprint, consensusWeight); - } - - /* Write consensus weights to disk to facilitate ordering of results - * in the servlet. */ - File relaysByConsensusWeightFile = - new File("out/relays-by-consensus-weight.csv"); - try { - relaysByConsensusWeightFile.getParentFile().mkdirs(); - BufferedWriter bw = new BufferedWriter(new FileWriter( - relaysByConsensusWeightFile)); - for (Map.Entry<String, Long> e : - relaysByConsensusWeight.entrySet()) { - String fingerprint = e.getKey(); - long consensusWeight = e.getValue(); - bw.write(fingerprint + "," + String.valueOf(consensusWeight) - + "\n"); - } - bw.close(); - } catch (IOException e) { - System.err.println("Could not write file '" - + relaysByConsensusWeightFile.getAbsolutePath() + "'. " - + "Ordering by consensus_weight may now be broken. Ignoring."); - e.printStackTrace(); } /* Return the files that we didn't update. */ diff --git a/src/org/torproject/onionoo/Main.java b/src/org/torproject/onionoo/Main.java index 20367d2..edac926 100644 --- a/src/org/torproject/onionoo/Main.java +++ b/src/org/torproject/onionoo/Main.java @@ -15,7 +15,6 @@ public class Main { cn.lookUpASes(); cn.readBridgeNetworkStatuses(); cn.setBridgeRunningBits(); - cn.writeRelaySearchDataFile(); printStatus("Updating detail data."); DetailDataWriter ddw = new DetailDataWriter(); @@ -35,12 +34,7 @@ public class Main { bdw.deleteObsoleteBandwidthFiles(); printStatus("Updating summary data."); - SummaryDataWriter sdw = new SummaryDataWriter(); - sdw.setLastValidAfterMillis(cn.getLastValidAfterMillis()); - sdw.setLastPublishedMillis(cn.getLastPublishedMillis()); - sdw.setCurrentRelays(cn.getCurrentRelays()); - sdw.setCurrentBridges(cn.getCurrentBridges()); - sdw.writeSummaryDataFile(); + cn.writeRelaySearchDataFile(); printStatus("Terminating."); } diff --git a/src/org/torproject/onionoo/Node.java b/src/org/torproject/onionoo/Node.java index 3daa4be..b7bc126 100644 --- a/src/org/torproject/onionoo/Node.java +++ b/src/org/torproject/onionoo/Node.java @@ -4,10 +4,15 @@ package org.torproject.onionoo; import java.util.SortedSet; +import org.apache.commons.codec.DecoderException; +import org.apache.commons.codec.binary.Hex; +import org.apache.commons.codec.digest.DigestUtils; + /* Store search data of a single relay that was running in the past seven * days. */ public class Node { private String fingerprint; + private String hashedFingerprint; private String nickname; private String address; private String latitude; @@ -29,6 +34,13 @@ public class Node { SortedSet<String> relayFlags, long consensusWeight) { this.nickname = nickname; this.fingerprint = fingerprint; + try { + this.hashedFingerprint = DigestUtils.shaHex(Hex.decodeHex( + fingerprint.toCharArray())).toUpperCase(); + } catch (DecoderException e) { + throw new IllegalArgumentException("Fingerprint '" + fingerprint + + "' is not a valid fingerprint."); + } this.address = address; this.lastSeenMillis = lastSeenMillis; this.orPort = orPort; @@ -39,6 +51,9 @@ public class Node { public String getFingerprint() { return this.fingerprint; } + public String getHashedFingerprint() { + return this.hashedFingerprint; + } public String getNickname() { return this.nickname; } diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index 4f1749d..38ae248 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -7,6 +7,7 @@ import java.io.File; import java.io.FileReader; import java.io.IOException; import java.io.PrintWriter; +import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -15,6 +16,7 @@ import java.util.HashSet; import java.util.List; import java.util.Map; import java.util.Set; +import java.util.TimeZone; import java.util.regex.Pattern; import javax.servlet.ServletConfig; @@ -23,14 +25,12 @@ import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; -import org.apache.commons.codec.DecoderException; -import org.apache.commons.codec.binary.Hex; -import org.apache.commons.codec.digest.DigestUtils; - public class ResourceServlet extends HttpServlet { private static final long serialVersionUID = 7236658979947465319L; + private File summaryFile = new File("status/summary.csv"); + private String outDirString; public void init(ServletConfig config) throws ServletException { @@ -41,118 +41,80 @@ public class ResourceServlet extends HttpServlet { long summaryFileLastModified = -1L; boolean readSummaryFile = false; - private String relaysPublishedLine = null, bridgesPublishedLine = null; - private List<String> relayLines = new ArrayList<String>(), - bridgeLines = new ArrayList<String>(), - relaysByConsensusWeight = new ArrayList<String>(); + private String relaysPublishedString, bridgesPublishedString; + private List<String> relaysByConsensusWeight = new ArrayList<String>(); private Map<String, String> relayFingerprintSummaryLines = new HashMap<String, String>(), bridgeFingerprintSummaryLines = new HashMap<String, String>(); private void readSummaryFile() { - File summaryFile = new File(this.outDirString + "summary.json"); if (!summaryFile.exists()) { readSummaryFile = false; return; } if (summaryFile.lastModified() > this.summaryFileLastModified) { - this.relayLines.clear(); - this.bridgeLines.clear(); - this.relayFingerprintSummaryLines.clear(); - this.bridgeFingerprintSummaryLines.clear(); - try { - BufferedReader br = new BufferedReader(new FileReader( - summaryFile)); - String line; - while ((line = br.readLine()) != null) { - if (line.contains("\"relays_published\":")) { - this.relaysPublishedLine = line.startsWith("{") ? line : - "{" + line; - } else if (line.startsWith("\"bridges_published\":")) { - this.bridgesPublishedLine = line; - } else if (line.startsWith("\"relays\":")) { - while ((line = br.readLine()) != null && !line.equals("],")) { - this.relayLines.add(line); - int fingerprintStart = line.indexOf("\"f\":\""); - if (fingerprintStart > 0) { - fingerprintStart += "\"f\":\"".length(); - String fingerprint = line.substring(fingerprintStart, - fingerprintStart + 40); - String hashedFingerprint = DigestUtils.shaHex( - Hex.decodeHex(fingerprint.toCharArray())). - toUpperCase(); - this.relayFingerprintSummaryLines.put(fingerprint, line); - this.relayFingerprintSummaryLines.put(hashedFingerprint, - line); - } - } - } else if (line.startsWith("\"bridges\":")) { - while ((line = br.readLine()) != null && !line.equals("]}")) { - this.bridgeLines.add(line); - int hashedFingerprintStart = line.indexOf("\"h\":\""); - if (hashedFingerprintStart > 0) { - hashedFingerprintStart += "\"h\":\"".length(); - String hashedFingerprint = line.substring( - hashedFingerprintStart, hashedFingerprintStart + 40); - String hashedHashedFingerprint = DigestUtils.shaHex( - Hex.decodeHex(hashedFingerprint.toCharArray())). - toUpperCase(); - this.bridgeFingerprintSummaryLines.put(hashedFingerprint, - line); - this.bridgeFingerprintSummaryLines.put( - hashedHashedFingerprint, line); - } - } - } - } - br.close(); - } catch (IOException e) { - return; - } catch (DecoderException e) { - return; - } + CurrentNodes cn = new CurrentNodes(); + cn.readRelaySearchDataFile(); + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + this.relaysPublishedString = dateTimeFormat.format( + cn.getLastValidAfterMillis()); + this.bridgesPublishedString = dateTimeFormat.format( + cn.getLastPublishedMillis()); List<String> orderRelaysByConsensusWeight = new ArrayList<String>(); - File relaysByConsensusWeightFile = - new File(this.outDirString + "/relays-by-consensus-weight.csv"); - if (relaysByConsensusWeightFile.exists()) { - try { - BufferedReader br = new BufferedReader(new FileReader( - relaysByConsensusWeightFile)); - String line; - while ((line = br.readLine()) != null) { - String[] parts = line.split(","); - if (parts.length != 2) { - return; - } - long consensusWeight = Long.parseLong(parts[1]); - String fingerprint = parts[0]; - orderRelaysByConsensusWeight.add( - String.format("%020d %s", consensusWeight, fingerprint)); - String hashedFingerprint = DigestUtils.shaHex( - Hex.decodeHex(fingerprint.toCharArray())). - toUpperCase(); - orderRelaysByConsensusWeight.add( - String.format("%020d %s", consensusWeight, - hashedFingerprint)); - } - br.close(); - Collections.sort(orderRelaysByConsensusWeight); - this.relaysByConsensusWeight = new ArrayList<String>(); - for (String relay : orderRelaysByConsensusWeight) { - this.relaysByConsensusWeight.add(relay.split(" ")[1]); - } - } catch (IOException e) { - return; - } catch (NumberFormatException e) { - return; - } catch (DecoderException e) { - return; - } + for (Node entry : cn.getCurrentRelays().values()) { + String fingerprint = entry.getFingerprint().toUpperCase(); + String hashedFingerprint = entry.getHashedFingerprint(). + toUpperCase(); + String line = this.formatRelaySummaryLine(entry); + this.relayFingerprintSummaryLines.put(fingerprint, line); + this.relayFingerprintSummaryLines.put(hashedFingerprint, line); + long consensusWeight = entry.getConsensusWeight(); + orderRelaysByConsensusWeight.add(String.format("%020d %s", + consensusWeight, fingerprint)); + orderRelaysByConsensusWeight.add(String.format("%020d %s", + consensusWeight, hashedFingerprint)); + } + Collections.sort(orderRelaysByConsensusWeight); + this.relaysByConsensusWeight = new ArrayList<String>(); + for (String relay : orderRelaysByConsensusWeight) { + this.relaysByConsensusWeight.add(relay.split(" ")[1]); + } + for (Node entry : cn.getCurrentBridges().values()) { + String hashedFingerprint = entry.getFingerprint().toUpperCase(); + String hashedHashedFingerprint = entry.getHashedFingerprint(). + toUpperCase(); + String line = this.formatBridgeSummaryLine(entry); + this.bridgeFingerprintSummaryLines.put(hashedFingerprint, line); + this.bridgeFingerprintSummaryLines.put(hashedHashedFingerprint, + line); } } this.summaryFileLastModified = summaryFile.lastModified(); this.readSummaryFile = true; } + private String formatRelaySummaryLine(Node entry) { + String nickname = !entry.getNickname().equals("Unnamed") ? + entry.getNickname() : null; + String fingerprint = entry.getFingerprint(); + String running = entry.getRunning() ? "true" : "false"; + String address = entry.getAddress(); + return String.format("\n{%s\"f\":\"%s\",\"a\":[\"%s\"],\"r\":%s}", + (nickname == null ? "" : "\"n\":\"" + nickname + "\","), + fingerprint, address, running); + } + + private String formatBridgeSummaryLine(Node entry) { + String nickname = !entry.getNickname().equals("Unnamed") ? + entry.getNickname() : null; + String hashedFingerprint = entry.getFingerprint(); + String running = entry.getRunning() ? "true" : "false"; + return String.format("\n{%s\"h\":\"%s\",\"r\":%s}", + (nickname == null ? "" : "\"n\":\"" + nickname + "\","), + hashedFingerprint, running); + } + public long getLastModified(HttpServletRequest request) { this.readSummaryFile(); return this.summaryFileLastModified; @@ -541,8 +503,8 @@ public class ResourceServlet extends HttpServlet { private void writeRelays(List<String> relays, PrintWriter pw, String resourceType) { - pw.print(this.relaysPublishedLine + "\n"); - pw.print("\"relays\":["); + pw.write("{\"relays_published\":\"" + this.relaysPublishedString + + "\",\n\"relays\":["); int written = 0; for (String line : relays) { if (line == null) { @@ -559,8 +521,8 @@ public class ResourceServlet extends HttpServlet { private void writeBridges(List<String> bridges, PrintWriter pw, String resourceType) { - pw.print(this.bridgesPublishedLine + "\n"); - pw.print("\"bridges\":["); + pw.write("\"bridges_published\":\"" + this.bridgesPublishedString + + "\",\n\"bridges\":["); int written = 0; for (String line : bridges) { if (line == null) { diff --git a/src/org/torproject/onionoo/SummaryDataWriter.java b/src/org/torproject/onionoo/SummaryDataWriter.java deleted file mode 100644 index 1391c02..0000000 --- a/src/org/torproject/onionoo/SummaryDataWriter.java +++ /dev/null @@ -1,93 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.onionoo; - -import java.io.BufferedWriter; -import java.io.File; -import java.io.FileWriter; -import java.io.IOException; -import java.text.SimpleDateFormat; -import java.util.SortedMap; -import java.util.TimeZone; - -/* Write relay and bridge summary data to disk. */ -public class SummaryDataWriter { - - private long lastValidAfterMillis; - public void setLastValidAfterMillis(long lastValidAfterMillis) { - this.lastValidAfterMillis = lastValidAfterMillis; - } - - private long lastPublishedMillis; - public void setLastPublishedMillis(long lastPublishedMillis) { - this.lastPublishedMillis = lastPublishedMillis; - } - - private SortedMap<String, Node> currentRelays; - public void setCurrentRelays(SortedMap<String, Node> currentRelays) { - this.currentRelays = currentRelays; - } - - private SortedMap<String, Node> currentBridges; - public void setCurrentBridges(SortedMap<String, Node> currentBridges) { - this.currentBridges = currentBridges; - } - - private File relaySearchDataFile = new File("out/summary.json"); - public void writeSummaryDataFile() { - try { - SimpleDateFormat dateTimeFormat = new SimpleDateFormat( - "yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - String relaysPublishedString = dateTimeFormat.format( - this.lastValidAfterMillis); - String bridgesPublishedString = dateTimeFormat.format( - this.lastPublishedMillis); - this.relaySearchDataFile.getParentFile().mkdirs(); - BufferedWriter bw = new BufferedWriter(new FileWriter( - this.relaySearchDataFile)); - bw.write("{\"relays_published\":\"" + relaysPublishedString - + "\",\n\"relays\":["); - int written = 0; - for (Node entry : this.currentRelays.values()) { - String nickname = !entry.getNickname().equals("Unnamed") ? - entry.getNickname() : null; - String fingerprint = entry.getFingerprint(); - String running = entry.getRunning() ? "true" : "false"; - String address = entry.getAddress(); - if (written++ > 0) { - bw.write(","); - } - bw.write("\n{" - + (nickname == null ? "" : "\"n\":\"" + nickname + "\",") - + "\"f\":\"" + fingerprint + "\"," - + "\"a\":[\"" + address + "\"]," - + "\"r\":" + running + "}"); - } - bw.write("\n],\n\"bridges_published\":\"" + bridgesPublishedString - + "\",\n\"bridges\":["); - written = 0; - for (Node entry : this.currentBridges.values()) { - String nickname = !entry.getNickname().equals("Unnamed") ? - entry.getNickname() : null; - String hashedFingerprint = entry.getFingerprint(); - String running = entry.getRunning() ? "true" : "false"; - if (written++ > 0) { - bw.write(","); - } - bw.write("\n{" - + (nickname == null ? "" : "\"n\":\"" + nickname + "\",") - + "\"h\":\"" + hashedFingerprint + "\"," - + "\"r\":" + running + "}"); - } - bw.write("\n]}\n"); - bw.close(); - } catch (IOException e) { - System.err.println("Could not write " - + this.relaySearchDataFile.getAbsolutePath() + ". Exiting."); - e.printStackTrace(); - System.exit(1); - } - } -} -

1 0

[onionoo/master] Fix nickname search for bridges.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit 04a1db13349344b3d6acd069d4b7f11024adade9 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 14:18:26 2012 +0200 Fix nickname search for bridges. --- src/org/torproject/onionoo/ResourceServlet.java | 27 ++++++++++++++++++---- 1 files changed, 22 insertions(+), 5 deletions(-) diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index e4f5b9f..4f1749d 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -475,7 +475,7 @@ public class ResourceServlet extends HttpServlet { /* Nickname matches. */ lineMatches = true; } else if ("unnamed".startsWith(searchTerm.toLowerCase()) && - (line.startsWith("{\"f\":") || line.startsWith("{\"h\":"))) { + line.startsWith("{\"f\":")) { /* Nickname "Unnamed" matches. */ lineMatches = true; } else if (line.contains("\"f\":\"" + searchTerm.toUpperCase())) { @@ -494,12 +494,29 @@ public class ResourceServlet extends HttpServlet { filteredRelays.remove(fingerprint); } Set<String> removeBridges = new HashSet<String>(); - if (searchTerm.startsWith("$")) { - searchTerm = searchTerm.substring(1); - } for (Map.Entry<String, String> e : filteredBridges.entrySet()) { String line = e.getValue(); - if (!line.contains("\"h\":\"" + searchTerm.toUpperCase())) { + boolean lineMatches = false; + if (searchTerm.startsWith("$")) { + /* Search is for $-prefixed hashed fingerprint. */ + if (line.contains("\"h\":\"" + + searchTerm.substring(1).toUpperCase())) { + /* $-prefixed hashed fingerprint matches. */ + lineMatches = true; + } + } else if (line.toLowerCase().contains("\"n\":\"" + + searchTerm.toLowerCase())) { + /* Nickname matches. */ + lineMatches = true; + } else if ("unnamed".startsWith(searchTerm.toLowerCase()) && + line.startsWith("{\"h\":")) { + /* Nickname "Unnamed" matches. */ + lineMatches = true; + } else if (line.contains("\"h\":\"" + searchTerm.toUpperCase())) { + /* Non-$-prefixed hashed fingerprint matches. */ + lineMatches = true; + } + if (!lineMatches) { removeBridges.add(e.getKey()); } }

1 0

[onionoo/master] Improve documentation of bridge nicknames a bit.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit 173c19261fb0c03fdc06cbbb38d4aa11a68f94c1 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 14:21:25 2012 +0200 Improve documentation of bridge nicknames a bit. --- web/index.html | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a/web/index.html b/web/index.html index 1539705..1cbe975 100755 --- a/web/index.html +++ b/web/index.html @@ -73,6 +73,11 @@ Required field.</li> Required field. Each array object contains the following key-value pairs: <ul> +<li>"n": Bridge nickname consisting of +1–19 alphanumerical characters. +Optional field. +Omitted if the bridge nickname is "Unnamed". +Added field on June 14, 2012.</li> <li>"h": SHA-1 hash of the bridge fingerprint consisting of 40 upper-case hexadecimal characters. Required field.</li> @@ -276,8 +281,8 @@ Required field.</li> Required field. Each array object contains the following key-value pairs: <ul> -<li>"nickname": Bridge nickname consisting of 1–19 -alphanumerical characters. +<li>"nickname": Bridge nickname +consisting of 1–19 alphanumerical characters. Optional field. Omitted if the bridge nickname is "Unnamed". Added field on June 14, 2012.</li>

1 0

[onionoo/master] Stop highlighting news from 1 month ago.
by karsten＠torproject.org 14 Jun '12

14 Jun '12

commit ac4867d4bbd37fe526f1e00d8e7e21022cc56621 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jun 14 14:23:44 2012 +0200 Stop highlighting news from 1 month ago. --- web/index.html | 17 +++-------------- 1 files changed, 3 insertions(+), 14 deletions(-) diff --git a/web/index.html b/web/index.html index 1cbe975..5ea7743 100755 --- a/web/index.html +++ b/web/index.html @@ -240,12 +240,11 @@ relay's first onion-routing IP address. Optional field. Omitted if the relay IP address could not be found in the AS database.</li> -<li>"consensus_weight": Bandwidth weight -assigned to this relay by the directory authorities. +<li>"consensus_weight": Bandwidth weight assigned to this relay by +the directory authorities. The unit is arbitrary; currently it's kilobytes per second, but that might change in the future. -Required field. -Added on May 14, 2012.</li> +Required field.</li> <li>"last_restarted": UTC timestamp (YYYY-MM-DD hh:mm:ss) when the relay was last (re-)started. Optional field.</li> @@ -576,21 +575,18 @@ more relay and/or bridge documents. <td>GET summary</td> <td>Return summaries of all relays and bridges that are currently running or that have been running in the past week. -Method added on May 14, 2012. </td> </tr> <tr> <td>GET details</td> <td>Return details of all relays and bridges that are currently running or that have been running in the past week. -Method added on May 14, 2012. </td> </tr> <tr> <td>GET bandwidth</td> <td>Return bandwidth documents of all relays and bridges that are currently running or that have been running in the past week. -Method added on May 14, 2012. </td> </tr> </table> @@ -604,12 +600,10 @@ of relay and/or bridge documents to be included in the response. <tr><td>type</td><td>Return only relay (parameter value relay) or only bridge documents (parameter value bridge). -Parameter added on May 14, 2012. </td></tr> <tr><td>running</td><td>Return only running (parameter value true) or only non-running relays and/or bridges (paramter value false). -Parameter added on May 14, 2012. </td></tr> <tr><td>search</td><td>Return only relays with the parameter value matching the beginning of a nickname, (possibly $-prefixed) fingerprint, @@ -619,7 +613,6 @@ Searches are case-insensitive. Full fingerprints should always be hashed using SHA-1, regardless of searching for a relay or a bridge, in order to not accidentally leak non-hashed bridge fingerprints in the URL. -Parameter added on May 14, 2012. Extended search to bridge nicknames on June 14, 2012. </td></tr> @@ -629,7 +622,6 @@ matching the hashed fingerprint. Fingerprints should always be hashed using SHA-1, regardless of looking up a relay or a bridge, in order to not accidentally leak non-hashed bridge fingerprints in the URL. -Parameter added on May 14, 2012. </td></tr> </table> Relay and/or bridge documents in the response can be ordered and @@ -650,17 +642,14 @@ Relays which don't have any value for a field to be ordered by are always appended to the end, regardless or sorting order. If no order parameter is given, ordering of results is undefined. -Parameter added on May 14, 2012. </td></tr> <tr><td>offset</td><td>Skip the given number of relays and/or bridges. -Parameter added on May 14, 2012. </td></tr> <tr><td>limit</td><td>Limit result to the given number of relays and/or bridges. When used together with offset, the offsetting step precedes the limiting step. -Parameter added on May 14, 2012. </td></tr> </table> </body>

1 0