March 2012 - tor-commits - lists.torproject.org

[torspec/master] Add "new create and extend cells" proposal by rransom.
by nickm＠torproject.org 22 Mar '12

22 Mar '12

commit a5c38ead59e0c9aac8e209c8a7e15f0fc6d321de Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Mar 22 10:51:56 2012 -0400 Add "new create and extend cells" proposal by rransom. --- proposals/xxx-new-create-and-extend-cells.txt | 136 +++++++++++++++++++++++++ 1 files changed, 136 insertions(+), 0 deletions(-) diff --git a/proposals/xxx-new-create-and-extend-cells.txt b/proposals/xxx-new-create-and-extend-cells.txt new file mode 100644 index 0000000..512eb45 --- /dev/null +++ b/proposals/xxx-new-create-and-extend-cells.txt @@ -0,0 +1,136 @@ +Filename: xxx-new-create-and-extend-cells.txt +Title: Adding new, extensible CREATE, EXTEND, and related cells +Author: Robert Ransom +Created: 2010-12-27 +Status: Open + +Overview and Motivation: + + In Tor's current circuit protocol, every field, including the 'onion + skin', in the EXTEND relay cell has a fixed meaning and length. + This prevents us from extending the current EXTEND cell to support + IPv6 relays, efficient UDP-based link protocols, larger 'onion + keys', new circuit-extension handshake protocols, or larger + identity-key fingerprints. We will need to support all of these + extensions in the near future. This proposal specifies a + replacement EXTEND2 cell and related cells that provide more room + for future extension. + +Design: + + FIXME - allocate command ID numbers (non-RELAY commands for CREATE2 and CREATED2; RELAY commands for EXTEND2 and EXTENDED2) + + The CREATE2 cell contains the following payload: + + Handshake type length [1 byte] + Handshake type [variable] + Handshake data length [2 bytes] + Handshake data [variable] + + The relay payload for an EXTEND2 relay cell contains the following + payload: + + Link target specifier type length [1 byte] + Link target specifier type [variable] + Link target specifier length [2 bytes] + Link target specifier [variable] + Handshake type length [1 byte] + Handshake type [variable] + Handshake data length [2 bytes] + Handshake data [variable] + + The CREATED2 cell and EXTENDED2 relay cell contain the following + payload: + + Handshake data length [2 bytes] + Handshake data [variable] + + All four cell types are padded to 512-byte cells. + + When a relay X receives an EXTEND2 relay cell: + + * X finds or opens a link to the relay Y using the link target + specifier in the EXTEND2 relay cell; if X fails to open a link, it + replies with a TRUNCATED relay cell. (FIXME: what do we do now?) + + * X copies the handshake type and data into a CREATE2 cell and sends + it along the link to Y. + + * If the handshake data is valid, Y replies by sending a CREATED2 + cell along the link to X; otherwise, Y replies with a TRUNCATED + relay cell. (XXX: we currently use a DESTROY cell?) + + * X copies the contents of the CREATED2 cell into an EXTENDED2 relay + cell and sends it along the circuit to the OP. + + + A link target specifier of type “legacy” contains the following + data: + + Relay IP address (FIXME: byte order?) [4 bytes] + Relay OR port (FIXME: byte order?) [2 bytes] + Relay identity key SHA-1 digest [20 bytes] + + These values are processed as section 5.1 of tor-spec.txt specifies + for the current EXTEND relay cell. + + + The first (client->relay) message in a handshake of type “legacy” + contains the following data: + + ‘Onion skin’ (as in CREATE cell) [DH_LEN+KEY_LEN+PK_PAD_LEN bytes] + + This value is generated and processed as sections 5.1 and 5.2 of + tor-spec.txt specify for the current CREATE cell. + + The second (relay->client) message in a handshake of type “legacy” + contains the following data: + + Relay DH public key [DH_LEN bytes] + KH (see section 5.2 of tor-spec.txt) [HASH_LEN bytes] + + These values are generated and processed as sections 5.1 and 5.2 of + tor-spec.txt specify for the current CREATED cell. + + After successfully completing a handshake of type “legacy”, the + client and relay use the current relay cryptography protocol. + +Bugs: + + This specification does not accommodate: + + * circuit-extension handshakes requiring more than one round + + No circuit-extension handshake should ever require more than one + round (i.e. more than one message from the client and one reply + from the relay). We can easily extend the protocol to handle + this, but we will never need to. + + * circuit-extension handshakes in which either message cannot fit in + a single 512-byte cell along with the other required fields + + This can be handled by specifying a dummy handshake type whose + data (sent from the client) consists of another handshake type and + the beginning of the data required by that handshake type, and + then using several (newly defined) HANDSHAKE_COMPLETION relay + cells sent in each direction to transport the remaining handshake + data. + + The specification of a HANDSHAKE_COMPLETION relay cell and its + associated dummy handshake type can safely be postponed until we + develop a circuit-extension handshake protocol that would require + it. + + * link target specifiers that cause EXTEND2 cells to exceed 512 + bytes + + This can be handled by specifying a LONG_COMMAND relay cell type + that can be used to transport a large ‘virtual cell’ in multiple + 512-byte cells. + + The specification of a LONG_COMMAND relay cell can safely be + postponed until we develop a link target specifier, a RELAY_BEGIN2 + relay cell and stream target specifier, or some other relay cell + type that would require it. + +

1 0

[torspec/master] rransom gets proposal 200: "Adding new, extensible CREATE, EXTEND, and related cells"
by nickm＠torproject.org 22 Mar '12

22 Mar '12

commit 2367819cef4faefe0cea1942086a43df30b971a3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Mar 22 11:32:00 2012 -0400 rransom gets proposal 200: "Adding new, extensible CREATE, EXTEND, and related cells" --- proposals/000-index.txt | 2 + proposals/200-new-create-and-extend-cells.txt | 154 +++++++++++++++++++++++++ proposals/xxx-new-create-and-extend-cells.txt | 149 ------------------------ 3 files changed, 156 insertions(+), 149 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 86a6d5b..a15b02f 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -120,6 +120,7 @@ Proposals by number: 197 Message-based Inter-Controller IPC Channel [OPEN] 198 Restore semantics of TLS ClientHello [OPEN] 199 Integration of BridgeFinder and BridgeFinderHelper [OPEN] +200 Adding new, extensible CREATE, EXTEND, and related cells [OPEN] Proposals by status: @@ -160,6 +161,7 @@ Proposals by status: 197 Message-based Inter-Controller IPC Channel [for 0.2.4.x] 198 Restore semantics of TLS ClientHello 199 Integration of BridgeFinder and BridgeFinderHelper [for 0.2.3.x+] + 200 Adding new, extensible CREATE, EXTEND, and related cells ACCEPTED: 117 IPv6 exits [for 0.2.3.x] 140 Provide diffs between consensuses diff --git a/proposals/200-new-create-and-extend-cells.txt b/proposals/200-new-create-and-extend-cells.txt new file mode 100644 index 0000000..05f615e --- /dev/null +++ b/proposals/200-new-create-and-extend-cells.txt @@ -0,0 +1,154 @@ +Filename: 200-new-create-and-extend-cells.txt +Title: Adding new, extensible CREATE, EXTEND, and related cells +Author: Robert Ransom +Created: 2012-03-22 +Status: Open + +History + + The original draft of this proposal was from 2010-12-27; nickm revised + it slightly on 2012-03-22 and added it as proposal 200. + +Overview and Motivation: + + In Tor's current circuit protocol, every field, including the 'onion + skin', in the EXTEND relay cell has a fixed meaning and length. + This prevents us from extending the current EXTEND cell to support + IPv6 relays, efficient UDP-based link protocols, larger 'onion + keys', new circuit-extension handshake protocols, or larger + identity-key fingerprints. We will need to support all of these + extensions in the near future. This proposal specifies a + replacement EXTEND2 cell and related cells that provide more room + for future extension. + +Design: + + FIXME - allocate command ID numbers (non-RELAY commands for CREATE2 and + CREATED2; RELAY commands for EXTEND2 and EXTENDED2) + + The CREATE2 cell contains the following payload: + + Handshake type [2 bytes] + Handshake data length [2 bytes] + Handshake data [variable] + + The relay payload for an EXTEND2 relay cell contains the following + payload: + + Number of link specifiers [1 byte] + N times: + Link specifier type [1 byte] + Link specifier length [1 byte] + Link specifier [variable] + Handshake type [2 bytes] + Handshake data length [2 bytes] + Handshake data [variable] + + The CREATED2 cell and EXTENDED2 relay cell both contain the following + payload: + + Handshake data length [2 bytes] + Handshake data [variable] + + All four cell types are padded to 512-byte cells. + + When a relay X receives an EXTEND2 relay cell: + + * X finds or opens a link to the relay Y using the link target + specifiers in the EXTEND2 relay cell; if X fails to open a link, it + replies with a TRUNCATED relay cell. (FIXME: what do we do now?) + + * X copies the handshake type and data into a CREATE2 cell and sends + it along the link to Y. + + * If the handshake data is valid, Y replies by sending a CREATED2 + cell along the link to X; otherwise, Y replies with a TRUNCATED + relay cell. (XXX: we currently use a DESTROY cell?) + + * X copies the contents of the CREATED2 cell into an EXTENDED2 relay + cell and sends it along the circuit to the OP. + + +Link target specifiers: + + The list of link target specifiers must include at least one address and + at least one identity fingerprint, in a format that the extending node is + known to recognize. + + The extending node MUST NOT accept the connection unless at least one + identity matches, and should follow the current rules for making sure that + addresses match. + + [00] TLS-over-TCP, IPv4 address + A four-byte IPv4 address plus two-byte ORPort + [01] TLS-over-TCP, IPv6 address + A sixteen-byte IPv6 address plus two-byte ORPort + [02] Legacy identity + A 20-byte SHA1 identity fingerprint. At most one may be listed. + + As always, values are sent in network (big-endian) order. + +Legacy handshake type: + + The current "onionskin" handshake type is defined to be handshake type + [00 00], or "legacy". + + The first (client->relay) message in a handshake of type “legacy” + contains the following data: + + ‘Onion skin’ (as in CREATE cell) [DH_LEN+KEY_LEN+PK_PAD_LEN bytes] + + This value is generated and processed as sections 5.1 and 5.2 of + tor-spec.txt specify for the current CREATE cell. + + The second (relay->client) message in a handshake of type “legacy” + contains the following data: + + Relay DH public key [DH_LEN bytes] + KH (see section 5.2 of tor-spec.txt) [HASH_LEN bytes] + + These values are generated and processed as sections 5.1 and 5.2 of + tor-spec.txt specify for the current CREATED cell. + + After successfully completing a handshake of type “legacy”, the + client and relay use the current relay cryptography protocol. + +Bugs: + + This specification does not accommodate: + + * circuit-extension handshakes requiring more than one round + + No circuit-extension handshake should ever require more than one + round (i.e. more than one message from the client and one reply + from the relay). We can easily extend the protocol to handle + this, but we will never need to. + + * circuit-extension handshakes in which either message cannot fit in + a single 512-byte cell along with the other required fields + + This can be handled by specifying a dummy handshake type whose + data (sent from the client) consists of another handshake type and + the beginning of the data required by that handshake type, and + then using several (newly defined) HANDSHAKE_COMPLETION relay + cells sent in each direction to transport the remaining handshake + data. + + The specification of a HANDSHAKE_COMPLETION relay cell and its + associated dummy handshake type can safely be postponed until we + develop a circuit-extension handshake protocol that would require + it. + + * link target specifiers that cause EXTEND2 cells to exceed 512 + bytes + + This can be handled by specifying a LONG_COMMAND relay cell type + that can be used to transport a large ‘virtual cell’ in multiple + 512-byte cells. + + The specification of a LONG_COMMAND relay cell can safely be + postponed until we develop a link target specifier, a RELAY_BEGIN2 + relay cell and stream target specifier, or some other relay cell + type that would require it. + + diff --git a/proposals/xxx-new-create-and-extend-cells.txt b/proposals/xxx-new-create-and-extend-cells.txt deleted file mode 100644 index 538620e..0000000 --- a/proposals/xxx-new-create-and-extend-cells.txt +++ /dev/null @@ -1,149 +0,0 @@ -Filename: xxx-new-create-and-extend-cells.txt -Title: Adding new, extensible CREATE, EXTEND, and related cells -Author: Robert Ransom -Created: 2010-12-27 -Status: Open - -Overview and Motivation: - - In Tor's current circuit protocol, every field, including the 'onion - skin', in the EXTEND relay cell has a fixed meaning and length. - This prevents us from extending the current EXTEND cell to support - IPv6 relays, efficient UDP-based link protocols, larger 'onion - keys', new circuit-extension handshake protocols, or larger - identity-key fingerprints. We will need to support all of these - extensions in the near future. This proposal specifies a - replacement EXTEND2 cell and related cells that provide more room - for future extension. - -Design: - - FIXME - allocate command ID numbers (non-RELAY commands for CREATE2 and - CREATED2; RELAY commands for EXTEND2 and EXTENDED2) - - The CREATE2 cell contains the following payload: - - Handshake type [2 bytes] - Handshake data length [2 bytes] - Handshake data [variable] - - The relay payload for an EXTEND2 relay cell contains the following - payload: - - Number of link specifiers [1 byte] - N times: - Link specifier type [1 byte] - Link specifier length [1 byte] - Link specifier [variable] - Handshake type [2 bytes] - Handshake data length [2 bytes] - Handshake data [variable] - - The CREATED2 cell and EXTENDED2 relay cell both contain the following - payload: - - Handshake data length [2 bytes] - Handshake data [variable] - - All four cell types are padded to 512-byte cells. - - When a relay X receives an EXTEND2 relay cell: - - * X finds or opens a link to the relay Y using the link target - specifiers in the EXTEND2 relay cell; if X fails to open a link, it - replies with a TRUNCATED relay cell. (FIXME: what do we do now?) - - * X copies the handshake type and data into a CREATE2 cell and sends - it along the link to Y. - - * If the handshake data is valid, Y replies by sending a CREATED2 - cell along the link to X; otherwise, Y replies with a TRUNCATED - relay cell. (XXX: we currently use a DESTROY cell?) - - * X copies the contents of the CREATED2 cell into an EXTENDED2 relay - cell and sends it along the circuit to the OP. - - -Link target specifiers: - - The list of link target specifiers must include at least one address and - at least one identity fingerprint, in a format that the extending node is - known to recognize. - - The extending node MUST NOT accept the connection unless at least one - identity matches, and should follow the current rules for making sure that - addresses match. - - [00] TLS-over-TCP, IPv4 address - A four-byte IPv4 address plus two-byte ORPort - [01] TLS-over-TCP, IPv6 address - A sixteen-byte IPv6 address plus two-byte ORPort - [02] Legacy identity - A 20-byte SHA1 identity fingerprint. At most one may be listed. - - As always, values are sent in network (big-endian) order. - -Legacy handshake type: - - The current "onionskin" handshake type is defined to be handshake type - [00 00], or "legacy". - - The first (client->relay) message in a handshake of type “legacy” - contains the following data: - - ‘Onion skin’ (as in CREATE cell) [DH_LEN+KEY_LEN+PK_PAD_LEN bytes] - - This value is generated and processed as sections 5.1 and 5.2 of - tor-spec.txt specify for the current CREATE cell. - - The second (relay->client) message in a handshake of type “legacy” - contains the following data: - - Relay DH public key [DH_LEN bytes] - KH (see section 5.2 of tor-spec.txt) [HASH_LEN bytes] - - These values are generated and processed as sections 5.1 and 5.2 of - tor-spec.txt specify for the current CREATED cell. - - After successfully completing a handshake of type “legacy”, the - client and relay use the current relay cryptography protocol. - -Bugs: - - This specification does not accommodate: - - * circuit-extension handshakes requiring more than one round - - No circuit-extension handshake should ever require more than one - round (i.e. more than one message from the client and one reply - from the relay). We can easily extend the protocol to handle - this, but we will never need to. - - * circuit-extension handshakes in which either message cannot fit in - a single 512-byte cell along with the other required fields - - This can be handled by specifying a dummy handshake type whose - data (sent from the client) consists of another handshake type and - the beginning of the data required by that handshake type, and - then using several (newly defined) HANDSHAKE_COMPLETION relay - cells sent in each direction to transport the remaining handshake - data. - - The specification of a HANDSHAKE_COMPLETION relay cell and its - associated dummy handshake type can safely be postponed until we - develop a circuit-extension handshake protocol that would require - it. - - * link target specifiers that cause EXTEND2 cells to exceed 512 - bytes - - This can be handled by specifying a LONG_COMMAND relay cell type - that can be used to transport a large ‘virtual cell’ in multiple - 512-byte cells. - - The specification of a LONG_COMMAND relay cell can safely be - postponed until we develop a link target specifier, a RELAY_BEGIN2 - relay cell and stream target specifier, or some other relay cell - type that would require it. - -

1 0

[metrics-web/master] Use digests provided by metrics-lib.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit dc925fd19d8ad6344a8eb0560a080a75c8c59dbd Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 15:45:51 2012 +0100 Use digests provided by metrics-lib. --- src/org/torproject/ernie/cron/ArchiveReader.java | 61 +++++----------------- 1 files changed, 13 insertions(+), 48 deletions(-) diff --git a/src/org/torproject/ernie/cron/ArchiveReader.java b/src/org/torproject/ernie/cron/ArchiveReader.java index e32072d..acdbdb5 100644 --- a/src/org/torproject/ernie/cron/ArchiveReader.java +++ b/src/org/torproject/ernie/cron/ArchiveReader.java @@ -117,32 +117,14 @@ public class ArchiveReader { } private void addServerDescriptor(ServerDescriptor descriptor) { - String digest = null; - try { - String ascii = new String(descriptor.getRawDescriptorBytes(), - "US-ASCII"); - String startToken = "router "; - String sigToken = "\nrouter-signature\n"; - int start = ascii.indexOf(startToken); - int sig = ascii.indexOf(sigToken) + sigToken.length(); - if (start >= 0 || sig >= 0 || sig > start) { - byte[] forDigest = new byte[sig - start]; - System.arraycopy(descriptor.getRawDescriptorBytes(), start, - forDigest, 0, sig - start); - digest = DigestUtils.shaHex(forDigest); - } - } catch (UnsupportedEncodingException e) { - } - if (digest != null) { - this.rddi.addServerDescriptor(digest, descriptor.getNickname(), - descriptor.getAddress(), descriptor.getOrPort(), - descriptor.getDirPort(), descriptor.getFingerprint(), - descriptor.getBandwidthRate(), descriptor.getBandwidthBurst(), - descriptor.getBandwidthObserved(), descriptor.getPlatform(), - descriptor.getPublishedMillis(), descriptor.getUptime(), - descriptor.getExtraInfoDigest(), - descriptor.getRawDescriptorBytes()); - } + this.rddi.addServerDescriptor(descriptor.getServerDescriptorDigest(), + descriptor.getNickname(), descriptor.getAddress(), + descriptor.getOrPort(), descriptor.getDirPort(), + descriptor.getFingerprint(), descriptor.getBandwidthRate(), + descriptor.getBandwidthBurst(), descriptor.getBandwidthObserved(), + descriptor.getPlatform(), descriptor.getPublishedMillis(), + descriptor.getUptime(), descriptor.getExtraInfoDigest(), + descriptor.getRawDescriptorBytes()); } private void addExtraInfoDescriptor(ExtraInfoDescriptor descriptor) { @@ -185,28 +167,11 @@ public class ArchiveReader { bandwidthHistoryLines.add( descriptor.getDirreqReadHistory().getLine()); } - String digest = null; - try { - String ascii = new String(descriptor.getRawDescriptorBytes(), - "US-ASCII"); - String startToken = "extra-info "; - String sigToken = "\nrouter-signature\n"; - int start = ascii.indexOf(startToken); - int sig = ascii.indexOf(sigToken) + sigToken.length(); - if (start >= 0 || sig >= 0 || sig > start) { - byte[] forDigest = new byte[sig - start]; - System.arraycopy(descriptor.getRawDescriptorBytes(), start, - forDigest, 0, sig - start); - digest = DigestUtils.shaHex(forDigest); - } - } catch (UnsupportedEncodingException e) { - } - if (digest != null) { - this.rddi.addExtraInfoDescriptor(digest, descriptor.getNickname(), - descriptor.getFingerprint().toLowerCase(), - descriptor.getPublishedMillis(), - descriptor.getRawDescriptorBytes(), bandwidthHistoryLines); - } + this.rddi.addExtraInfoDescriptor(descriptor.getExtraInfoDigest(), + descriptor.getNickname(), + descriptor.getFingerprint().toLowerCase(), + descriptor.getPublishedMillis(), + descriptor.getRawDescriptorBytes(), bandwidthHistoryLines); } }

1 0

[metrics-lib/master] Calculate descriptor digests.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit 8a1dce943a3b5ae791cba66dc3081fa633cee20f Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 15:43:12 2012 +0100 Calculate descriptor digests. --- .../torproject/descriptor/ExtraInfoDescriptor.java | 4 ++ .../torproject/descriptor/ServerDescriptor.java | 4 ++ .../descriptor/impl/ExtraInfoDescriptorImpl.java | 30 ++++++++++++++++++++ .../descriptor/impl/ServerDescriptorImpl.java | 30 ++++++++++++++++++++ 4 files changed, 68 insertions(+), 0 deletions(-) diff --git a/src/org/torproject/descriptor/ExtraInfoDescriptor.java b/src/org/torproject/descriptor/ExtraInfoDescriptor.java index 18a5b97..229b075 100644 --- a/src/org/torproject/descriptor/ExtraInfoDescriptor.java +++ b/src/org/torproject/descriptor/ExtraInfoDescriptor.java @@ -7,6 +7,10 @@ import java.util.SortedMap; public interface ExtraInfoDescriptor extends Descriptor { + /* Return the descriptor digest that is used to reference this + * extra-info descriptor in a server descriptor. */ + public String getExtraInfoDigest(); + /* Return the relay's nickname. */ public String getNickname(); diff --git a/src/org/torproject/descriptor/ServerDescriptor.java b/src/org/torproject/descriptor/ServerDescriptor.java index 1fe1f21..d641d49 100644 --- a/src/org/torproject/descriptor/ServerDescriptor.java +++ b/src/org/torproject/descriptor/ServerDescriptor.java @@ -7,6 +7,10 @@ import java.util.List; /* Contains a relay server descriptor. */ public interface ServerDescriptor extends Descriptor { + /* Return the descriptor digest that is used to reference this server + * descriptor in a network status. */ + public String getServerDescriptorDigest(); + /* Return the relay's nickname. */ public String getNickname(); diff --git a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java index 7f2194e..a198d70 100644 --- a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java +++ b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java @@ -5,6 +5,7 @@ package org.torproject.descriptor.impl; import java.io.BufferedReader; import java.io.IOException; import java.io.StringReader; +import java.io.UnsupportedEncodingException; import java.util.ArrayList; import java.util.Arrays; import java.util.HashSet; @@ -13,6 +14,7 @@ import java.util.Set; import java.util.SortedMap; import java.util.TreeMap; +import org.apache.commons.codec.digest.DigestUtils; import org.torproject.descriptor.BandwidthHistory; import org.torproject.descriptor.ExtraInfoDescriptor; @@ -41,6 +43,7 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl throws DescriptorParseException { super(descriptorBytes, failUnrecognizedDescriptorLines); this.parseDescriptorBytes(); + this.calculateDigest(); Set<String> exactlyOnceKeywords = new HashSet<String>(Arrays.asList(( "extra-info,published").split(","))); this.checkExactlyOnceKeywords(exactlyOnceKeywords); @@ -492,6 +495,33 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl /* Not parsing crypto parts (yet). */ } + private void calculateDigest() throws DescriptorParseException { + try { + String ascii = new String(this.getRawDescriptorBytes(), "US-ASCII"); + String startToken = "router "; + String sigToken = "\nrouter-signature\n"; + int start = ascii.indexOf(startToken); + int sig = ascii.indexOf(sigToken) + sigToken.length(); + if (start >= 0 || sig >= 0 || sig > start) { + byte[] forDigest = new byte[sig - start]; + System.arraycopy(this.getRawDescriptorBytes(), start, + forDigest, 0, sig - start); + this.extraInfoDigest = DigestUtils.shaHex(forDigest); + } + } catch (UnsupportedEncodingException e) { + /* Handle below. */ + } + if (this.extraInfoDigest == null) { + throw new DescriptorParseException("Could not calculate extra-info " + + "descriptor digest."); + } + } + + private String extraInfoDigest; + public String getExtraInfoDigest() { + return this.extraInfoDigest; + } + private String nickname; public String getNickname() { return this.nickname; diff --git a/src/org/torproject/descriptor/impl/ServerDescriptorImpl.java b/src/org/torproject/descriptor/impl/ServerDescriptorImpl.java index 67b4d09..200b8b5 100644 --- a/src/org/torproject/descriptor/impl/ServerDescriptorImpl.java +++ b/src/org/torproject/descriptor/impl/ServerDescriptorImpl.java @@ -5,12 +5,14 @@ package org.torproject.descriptor.impl; import java.io.BufferedReader; import java.io.IOException; import java.io.StringReader; +import java.io.UnsupportedEncodingException; import java.util.ArrayList; import java.util.Arrays; import java.util.HashSet; import java.util.List; import java.util.Set; +import org.apache.commons.codec.digest.DigestUtils; import org.torproject.descriptor.BandwidthHistory; import org.torproject.descriptor.ServerDescriptor; @@ -45,6 +47,7 @@ public class ServerDescriptorImpl extends DescriptorImpl throws DescriptorParseException { super(descriptorBytes, failUnrecognizedDescriptorLines); this.parseDescriptorBytes(); + this.calculateDigest(); Set<String> exactlyOnceKeywords = new HashSet<String>(Arrays.asList( "router,bandwidth,published".split(","))); this.checkExactlyOnceKeywords(exactlyOnceKeywords); @@ -424,6 +427,33 @@ public class ServerDescriptorImpl extends DescriptorImpl this.allowSingleHopExits = true; } + private void calculateDigest() throws DescriptorParseException { + try { + String ascii = new String(this.getRawDescriptorBytes(), "US-ASCII"); + String startToken = "router "; + String sigToken = "\nrouter-signature\n"; + int start = ascii.indexOf(startToken); + int sig = ascii.indexOf(sigToken) + sigToken.length(); + if (start >= 0 || sig >= 0 || sig > start) { + byte[] forDigest = new byte[sig - start]; + System.arraycopy(this.getRawDescriptorBytes(), start, + forDigest, 0, sig - start); + this.serverDescriptorDigest = DigestUtils.shaHex(forDigest); + } + } catch (UnsupportedEncodingException e) { + /* Handle below. */ + } + if (this.serverDescriptorDigest == null) { + throw new DescriptorParseException("Could not calculate server " + + "descriptor digest."); + } + } + + private String serverDescriptorDigest; + public String getServerDescriptorDigest() { + return this.serverDescriptorDigest; + } + private String nickname; public String getNickname() { return this.nickname;

1 0

[tor/master] Never disable debugger attachment for the unit tests
by nickm＠torproject.org 22 Mar '12

22 Mar '12

commit fe2b177cfbcdbc2ad851402a9804f9b5f58d93b0 Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Thu Mar 22 12:50:44 2012 +0100 Never disable debugger attachment for the unit tests --- changes/bug5448 | 5 +++++ src/or/config.c | 4 +++- 2 files changed, 8 insertions(+), 1 deletions(-) diff --git a/changes/bug5448 b/changes/bug5448 new file mode 100644 index 0000000..aaf7dc7 --- /dev/null +++ b/changes/bug5448 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Allow running the unit tests in gdb again. This was accidentally made + impossible when the DisableDebuggerAttachment option was introduced. + Fixes bug 5448; bugfix on 0.2.3.9-alpha. + diff --git a/src/or/config.c b/src/or/config.c index 0c699b0..3de8426 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -1338,7 +1338,9 @@ options_act(const or_options_t *old_options) /* Remember if we already warned about being configured not to disable * debugger attachment */ static int warned_debugger_attach = 0; - if (options->DisableDebuggerAttachment && !disabled_debugger_attach) { + /* Don't disable debugger attachment when we're running the unit tests. */ + if (options->DisableDebuggerAttachment && !disabled_debugger_attach && + running_tor) { int ok = tor_disable_debugger_attach(); if (warned_debugger_attach && ok == 1) { log_notice(LD_CONFIG, "Disabled attaching debuggers for unprivileged "

1 0

[metrics-web/master] Remove now unused database schema parts.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit 3ca51c4a7fd8de5ce8f806a66df43bbde9f48b1d Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 15:23:08 2012 +0100 Remove now unused database schema parts. --- db/tordir.sql | 21 --------------------- 1 files changed, 0 insertions(+), 21 deletions(-) diff --git a/db/tordir.sql b/db/tordir.sql index e1ca324..e7c33f9 100644 --- a/db/tordir.sql +++ b/db/tordir.sql @@ -18,22 +18,10 @@ CREATE TABLE descriptor ( platform CHARACTER VARYING(256), published TIMESTAMP WITHOUT TIME ZONE NOT NULL, uptime BIGINT, - extrainfo CHARACTER(40), rawdesc BYTEA NOT NULL, CONSTRAINT descriptor_pkey PRIMARY KEY (descriptor) ); --- TABLE extrainfo --- Contains all of the extra-info descriptors published by the routers. -CREATE TABLE extrainfo ( - extrainfo CHARACTER(40) NOT NULL, - nickname CHARACTER VARYING(19) NOT NULL, - fingerprint CHARACTER(40) NOT NULL, - published TIMESTAMP WITHOUT TIME ZONE NOT NULL, - rawdesc BYTEA NOT NULL, - CONSTRAINT extrainfo_pkey PRIMARY KEY (extrainfo) -); - -- Contains bandwidth histories reported by relays in extra-info -- descriptors. Each row contains the reported bandwidth in 15-minute -- intervals for each relay and date. @@ -146,15 +134,6 @@ CREATE TABLE consensus ( CONSTRAINT consensus_pkey PRIMARY KEY (validafter) ); --- TABLE vote --- Contains all of the votes published by the directories -CREATE TABLE vote ( - validafter TIMESTAMP WITHOUT TIME ZONE NOT NULL, - dirsource CHARACTER(40) NOT NULL, - rawdesc BYTEA NOT NULL, - CONSTRAINT vote_pkey PRIMARY KEY (validafter, dirsource) -); - -- TABLE connbidirect -- Contain conn-bi-direct stats strings CREATE TABLE connbidirect (

1 0

[metrics-web/master] Rename servlet providing the consensus-health.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit 4a643a4ed21163dccec4226cb7a1164d9d591118 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 14:59:28 2012 +0100 Rename servlet providing the consensus-health. --- etc/web.xml | 6 +- .../ernie/web/ConsensusHealthServlet.java | 57 ++++++++++++++++++ .../ernie/web/ErnieGeneratedFileServlet.java | 63 -------------------- 3 files changed, 60 insertions(+), 66 deletions(-) diff --git a/etc/web.xml b/etc/web.xml index c831f3f..a5bd7be 100644 --- a/etc/web.xml +++ b/etc/web.xml @@ -263,13 +263,13 @@ </servlet-mapping> <servlet> - <servlet-name>ErnieGeneratedFile</servlet-name> + <servlet-name>ConsensusHealthServlet</servlet-name> <servlet-class> - org.torproject.ernie.web.ErnieGeneratedFileServlet + org.torproject.ernie.web.ConsensusHealthServlet </servlet-class> </servlet> <servlet-mapping> - <servlet-name>ErnieGeneratedFile</servlet-name> + <servlet-name>ConsensusHealthServlet</servlet-name> <url-pattern>/consensus-health.html</url-pattern> </servlet-mapping> diff --git a/src/org/torproject/ernie/web/ConsensusHealthServlet.java b/src/org/torproject/ernie/web/ConsensusHealthServlet.java new file mode 100644 index 0000000..2ca26d5 --- /dev/null +++ b/src/org/torproject/ernie/web/ConsensusHealthServlet.java @@ -0,0 +1,57 @@ +/* Copyright 2011, 2012 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.ernie.web; + +import java.io.BufferedInputStream; +import java.io.BufferedOutputStream; +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; + +import javax.servlet.ServletException; +import javax.servlet.http.HttpServlet; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; + +public class ConsensusHealthServlet extends HttpServlet { + + private static final long serialVersionUID = -5230032733057814869L; + + public void doGet(HttpServletRequest request, + HttpServletResponse response) throws IOException, + ServletException { + + /* Read file from disk and write it to response. */ + BufferedInputStream input = null; + BufferedOutputStream output = null; + try { + File f = new File("/srv/metrics.torproject.org/ernie/website/" + + "consensus-health.html"); + if (!f.exists()) { + response.sendError(HttpServletResponse.SC_NOT_FOUND); + return; + } + response.setContentType(this.getServletContext().getMimeType(f.getName())); + response.setHeader("Content-Length", String.valueOf( + f.length())); + response.setHeader("Content-Disposition", + "inline; filename=\"" + f.getName() + "\""); + input = new BufferedInputStream(new FileInputStream(f), + 1024); + output = new BufferedOutputStream(response.getOutputStream(), 1024); + byte[] buffer = new byte[1024]; + int length; + while ((length = input.read(buffer)) > 0) { + output.write(buffer, 0, length); + } + } finally { + if (output != null) { + output.close(); + } + if (input != null) { + input.close(); + } + } + } +} + diff --git a/src/org/torproject/ernie/web/ErnieGeneratedFileServlet.java b/src/org/torproject/ernie/web/ErnieGeneratedFileServlet.java deleted file mode 100644 index a303225..0000000 --- a/src/org/torproject/ernie/web/ErnieGeneratedFileServlet.java +++ /dev/null @@ -1,63 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.BufferedInputStream; -import java.io.BufferedOutputStream; -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; - -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; - -public class ErnieGeneratedFileServlet extends HttpServlet { - - private static final long serialVersionUID = -5230032733057814869L; - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, - ServletException { - - /* Read file from disk and write it to response. */ - String requestedURL = request.getRequestURI(); - if (requestedURL.contains("/")) { - requestedURL = requestedURL.substring(requestedURL. - lastIndexOf("/")); - } - String fn = "/srv/metrics.torproject.org/ernie/website" - + requestedURL; - BufferedInputStream input = null; - BufferedOutputStream output = null; - try { - File f = new File(fn); - if (!f.exists()) { - response.sendError(HttpServletResponse.SC_NOT_FOUND); - return; - } - response.setContentType(this.getServletContext().getMimeType(f.getName())); - response.setHeader("Content-Length", String.valueOf( - f.length())); - response.setHeader("Content-Disposition", - "inline; filename=\"" + f.getName() + "\""); - input = new BufferedInputStream(new FileInputStream(f), - 1024); - output = new BufferedOutputStream(response.getOutputStream(), 1024); - byte[] buffer = new byte[1024]; - int length; - while ((length = input.read(buffer)) > 0) { - output.write(buffer, 0, length); - } - } finally { - if (output != null) { - output.close(); - } - if (input != null) { - input.close(); - } - } - } -} -

1 0

[metrics-web/master] Remove previously disabled or deprecated website parts.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit 82a41fbb14a2e88d8a9494888d2eb2003e76531c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 14:28:10 2012 +0100 Remove previously disabled or deprecated website parts. --- etc/web.xml | 64 ---- src/org/torproject/ernie/cron/ArchiveReader.java | 8 - .../cron/RelayDescriptorDatabaseImporter.java | 165 --------- .../torproject/ernie/web/DescriptorServlet.java | 353 -------------------- .../ernie/web/ExtraInfoDescriptorServlet.java | 113 ------- .../torproject/ernie/web/NetworkStatusServlet.java | 24 -- .../torproject/ernie/web/RouterDetailServlet.java | 24 -- src/org/torproject/ernie/web/VotesServlet.java | 132 -------- web/WEB-INF/banner.jsp | 2 - web/WEB-INF/networkstatus.jsp | 23 -- web/WEB-INF/routerdetail.jsp | 23 -- web/robots.txt | 2 - 12 files changed, 0 insertions(+), 933 deletions(-) diff --git a/etc/web.xml b/etc/web.xml index 12e35fa..c831f3f 100644 --- a/etc/web.xml +++ b/etc/web.xml @@ -130,27 +130,6 @@ </servlet-mapping> <servlet> - <servlet-name>NetworkStatus</servlet-name> - <servlet-class> - org.torproject.ernie.web.NetworkStatusServlet - </servlet-class> - </servlet> - <servlet-mapping> - <servlet-name>NetworkStatus</servlet-name> - <url-pattern>/networkstatus.html</url-pattern> - </servlet-mapping> - <servlet> - <servlet-name>RouterDetail</servlet-name> - <servlet-class> - org.torproject.ernie.web.RouterDetailServlet - </servlet-class> - </servlet> - <servlet-mapping> - <servlet-name>RouterDetail</servlet-name> - <url-pattern>/routerdetail.html</url-pattern> - </servlet-mapping> - - <servlet> <servlet-name>Relay</servlet-name> <servlet-class> org.torproject.ernie.web.RelayServlet @@ -238,10 +217,6 @@ <servlet-name>GraphImage</servlet-name> <url-pattern>/connbidirect.png</url-pattern> </servlet-mapping> - <servlet-mapping> - <servlet-name>GraphImage</servlet-name> - <url-pattern>/routerdetail.png</url-pattern> - </servlet-mapping> <servlet> <servlet-name>Csv</servlet-name> @@ -276,32 +251,6 @@ <url-pattern>/serverdesc</url-pattern> </servlet-mapping> - - - - <servlet> <servlet-name>Consensus</servlet-name> <servlet-class> @@ -313,19 +262,6 @@ <url-pattern>/consensus</url-pattern> </servlet-mapping> - - <servlet> <servlet-name>ErnieGeneratedFile</servlet-name> <servlet-class> diff --git a/src/org/torproject/ernie/cron/ArchiveReader.java b/src/org/torproject/ernie/cron/ArchiveReader.java index e4f6b2a..e32072d 100644 --- a/src/org/torproject/ernie/cron/ArchiveReader.java +++ b/src/org/torproject/ernie/cron/ArchiveReader.java @@ -79,9 +79,6 @@ public class ArchiveReader { if (descriptor instanceof RelayNetworkStatusConsensus) { this.addRelayNetworkStatusConsensus( (RelayNetworkStatusConsensus) descriptor); - } else if (descriptor instanceof RelayNetworkStatusVote) { - this.addRelayNetworkStatusVote( - (RelayNetworkStatusVote) descriptor); } else if (descriptor instanceof ServerDescriptor) { this.addServerDescriptor((ServerDescriptor) descriptor); } else if (descriptor instanceof ExtraInfoDescriptor) { @@ -119,11 +116,6 @@ public class ArchiveReader { consensus.getRawDescriptorBytes()); } - private void addRelayNetworkStatusVote(RelayNetworkStatusVote vote) { - this.rddi.addVote(vote.getValidAfterMillis(), vote.getIdentity(), - vote.getRawDescriptorBytes()); - } - private void addServerDescriptor(ServerDescriptor descriptor) { String digest = null; try { diff --git a/src/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java b/src/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java index 3ee7b6b..3e7e694 100644 --- a/src/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java +++ b/src/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java @@ -71,12 +71,6 @@ public final class RelayDescriptorDatabaseImporter { private PreparedStatement psRs; /** - * Prepared statement to check whether a given extra-info descriptor has - * been imported into the database before. - */ - private PreparedStatement psEs; - - /** * Prepared statement to check whether a given server descriptor has * been imported into the database before. */ @@ -89,12 +83,6 @@ public final class RelayDescriptorDatabaseImporter { private PreparedStatement psCs; /** - * Prepared statement to check whether a given network status vote has - * been imported into the database before. - */ - private PreparedStatement psVs; - - /** * Prepared statement to check whether a given conn-bi-direct stats * string has been imported into the database before. */ @@ -130,12 +118,6 @@ public final class RelayDescriptorDatabaseImporter { private PreparedStatement psD; /** - * Prepared statement to insert an extra-info descriptor into the - * database. - */ - private PreparedStatement psE; - - /** * Callable statement to insert the bandwidth history of an extra-info * descriptor into the database. */ @@ -148,12 +130,6 @@ public final class RelayDescriptorDatabaseImporter { private PreparedStatement psC; /** - * Prepared statement to insert a network status vote into the - * database. - */ - private PreparedStatement psV; - - /** * Prepared statement to insert a conn-bi-direct stats string into the * database. */ @@ -186,11 +162,6 @@ public final class RelayDescriptorDatabaseImporter { private BufferedWriter descriptorOut; /** - * Raw import file containing extra-info descriptors. - */ - private BufferedWriter extrainfoOut; - - /** * Raw import file containing bandwidth histories. */ private BufferedWriter bwhistOut; @@ -201,11 +172,6 @@ public final class RelayDescriptorDatabaseImporter { private BufferedWriter consensusOut; /** - * Raw import file containing votes. - */ - private BufferedWriter voteOut; - - /** * Raw import file containing conn-bi-direct stats strings. */ private BufferedWriter connBiDirectOut; @@ -268,12 +234,8 @@ public final class RelayDescriptorDatabaseImporter { + "fingerprint = ?"); this.psDs = conn.prepareStatement("SELECT COUNT(*) " + "FROM descriptor WHERE descriptor = ?"); - this.psEs = conn.prepareStatement("SELECT COUNT(*) " - + "FROM extrainfo WHERE extrainfo = ?"); this.psCs = conn.prepareStatement("SELECT COUNT(*) " + "FROM consensus WHERE validafter = ?"); - this.psVs = conn.prepareStatement("SELECT COUNT(*) " - + "FROM vote WHERE validafter = ? AND dirsource = ?"); this.psBs = conn.prepareStatement("SELECT COUNT(*) " + "FROM connbidirect WHERE source = ? AND statsend = ?"); this.psQs = conn.prepareStatement("SELECT COUNT(*) " @@ -292,15 +254,10 @@ public final class RelayDescriptorDatabaseImporter { + "bandwidthobserved, platform, published, uptime, " + "extrainfo, rawdesc) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, " + "?, ?, ?, ?)"); - this.psE = conn.prepareStatement("INSERT INTO extrainfo " - + "(extrainfo, nickname, fingerprint, published, rawdesc) " - + "VALUES (?, ?, ?, ?, ?)"); this.csH = conn.prepareCall("{call insert_bwhist(?, ?, ?, ?, ?, " + "?)}"); this.psC = conn.prepareStatement("INSERT INTO consensus " + "(validafter, rawdesc) VALUES (?, ?)"); - this.psV = conn.prepareStatement("INSERT INTO vote " - + "(validafter, dirsource, rawdesc) VALUES (?, ?, ?)"); this.psB = conn.prepareStatement("INSERT INTO connbidirect " + "(source, statsend, seconds, belownum, readnum, writenum, " + "bothnum) VALUES (?, ?, ?, ?, ?, ?, ?)"); @@ -590,60 +547,6 @@ public final class RelayDescriptorDatabaseImporter { public void addExtraInfoDescriptor(String extraInfoDigest, String nickname, String fingerprint, long published, byte[] rawDescriptor, List<String> bandwidthHistoryLines) { - if (this.importIntoDatabase) { - try { - Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("UTC")); - this.psEs.setString(1, extraInfoDigest); - ResultSet rs = psEs.executeQuery(); - rs.next(); - if (rs.getInt(1) == 0) { - this.psE.clearParameters(); - this.psE.setString(1, extraInfoDigest); - this.psE.setString(2, nickname); - this.psE.setString(3, fingerprint); - this.psE.setTimestamp(4, new Timestamp(published), cal); - this.psE.setBytes(5, rawDescriptor); - this.psE.executeUpdate(); - resCount++; - if (resCount % autoCommitCount == 0) { - this.conn.commit(); - } - } - } catch (SQLException e) { - this.logger.log(Level.WARNING, "Could not add extra-info " - + "descriptor. We won't make any further SQL requests in " - + "this execution.", e); - this.importIntoDatabase = false; - } - } - if (this.writeRawImportFiles) { - try { - if (this.extrainfoOut == null) { - new File(rawFilesDirectory).mkdirs(); - this.extrainfoOut = new BufferedWriter(new FileWriter( - rawFilesDirectory + "/extrainfo.sql")); - this.extrainfoOut.write(" COPY extrainfo (extrainfo, nickname, " - + "fingerprint, published, rawdesc) FROM stdin;\n"); - } - this.extrainfoOut.write(extraInfoDigest.toLowerCase() + "\t" - + nickname + "\t" + fingerprint.toLowerCase() + "\t" - + this.dateTimeFormat.format(published) + "\t"); - this.extrainfoOut.write(PGbytea.toPGString(rawDescriptor). - replaceAll("\\\\", "\\\\\\\\") + "\n"); - } catch (IOException e) { - this.logger.log(Level.WARNING, "Could not write extra-info " - + "descriptor to raw database import file. We won't make " - + "any further attempts to write raw import files in this " - + "execution.", e); - this.writeRawImportFiles = false; - } catch (SQLException e) { - this.logger.log(Level.WARNING, "Could not write extra-info " - + "descriptor to raw database import file. We won't make " - + "any further attempts to write raw import files in this " - + "execution.", e); - this.writeRawImportFiles = false; - } - } if (!bandwidthHistoryLines.isEmpty()) { this.addBandwidthHistory(fingerprint.toLowerCase(), published, bandwidthHistoryLines); @@ -955,66 +858,6 @@ public final class RelayDescriptorDatabaseImporter { } /** - * Insert network status vote into database. - */ - public void addVote(long validAfter, String dirSource, - byte[] rawDescriptor) { - if (this.importIntoDatabase) { - try { - Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("UTC")); - Timestamp validAfterTimestamp = new Timestamp(validAfter); - this.psVs.setTimestamp(1, validAfterTimestamp, cal); - this.psVs.setString(2, dirSource); - ResultSet rs = psVs.executeQuery(); - rs.next(); - if (rs.getInt(1) == 0) { - this.psV.clearParameters(); - this.psV.setTimestamp(1, validAfterTimestamp, cal); - this.psV.setString(2, dirSource); - this.psV.setBytes(3, rawDescriptor); - this.psV.executeUpdate(); - rvsCount++; - if (rvsCount % autoCommitCount == 0) { - this.conn.commit(); - } - } - } catch (SQLException e) { - this.logger.log(Level.WARNING, "Could not add network status " - + "vote. We won't make any further SQL requests in this " - + "execution.", e); - this.importIntoDatabase = false; - } - } - if (this.writeRawImportFiles) { - try { - if (this.voteOut == null) { - new File(rawFilesDirectory).mkdirs(); - this.voteOut = new BufferedWriter(new FileWriter( - rawFilesDirectory + "/vote.sql")); - this.voteOut.write(" COPY vote (validafter, dirsource, " - + "rawdesc) FROM stdin;\n"); - } - String validAfterString = this.dateTimeFormat.format(validAfter); - this.voteOut.write(validAfterString + "\t" + dirSource + "\t"); - this.voteOut.write(PGbytea.toPGString(rawDescriptor). - replaceAll("\\\\", "\\\\\\\\") + "\n"); - } catch (SQLException e) { - this.logger.log(Level.WARNING, "Could not write network status " - + "vote to raw database import file. We won't make any " - + "further attempts to write raw import files in this " - + "execution.", e); - this.writeRawImportFiles = false; - } catch (IOException e) { - this.logger.log(Level.WARNING, "Could not write network status " - + "vote to raw database import file. We won't make any " - + "further attempts to write raw import files in this " - + "execution.", e); - this.writeRawImportFiles = false; - } - } - } - - /** * Insert a conn-bi-direct stats string into the database. */ public void addConnBiDirect(String source, long statsEndMillis, @@ -1195,10 +1038,6 @@ public final class RelayDescriptorDatabaseImporter { this.descriptorOut.write("\\.\n"); this.descriptorOut.close(); } - if (this.extrainfoOut != null) { - this.extrainfoOut.write("\\.\n"); - this.extrainfoOut.close(); - } if (this.bwhistOut != null) { this.bwhistOut.write("\\.\n"); this.bwhistOut.close(); @@ -1207,10 +1046,6 @@ public final class RelayDescriptorDatabaseImporter { this.consensusOut.write("\\.\n"); this.consensusOut.close(); } - if (this.voteOut != null) { - this.voteOut.write("\\.\n"); - this.voteOut.close(); - } if (this.connBiDirectOut != null) { this.connBiDirectOut.write("\\.\n"); this.connBiDirectOut.close(); diff --git a/src/org/torproject/ernie/web/DescriptorServlet.java b/src/org/torproject/ernie/web/DescriptorServlet.java deleted file mode 100644 index 7b3a1b7..0000000 --- a/src/org/torproject/ernie/web/DescriptorServlet.java +++ /dev/null @@ -1,353 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.BufferedReader; -import java.io.IOException; -import java.io.PrintWriter; -import java.io.StringReader; -import java.sql.Connection; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Statement; -import java.util.SortedSet; -import java.util.TreeSet; -import java.util.logging.Level; -import java.util.logging.Logger; -import java.util.regex.Pattern; - -import javax.naming.Context; -import javax.naming.InitialContext; -import javax.naming.NamingException; -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import javax.sql.DataSource; - -import org.apache.commons.lang.StringEscapeUtils; - -public class DescriptorServlet extends HttpServlet { - - private static final long serialVersionUID = 7933295702958621380L; - - private DataSource ds; - - private Logger logger; - - public void init() { - - /* Initialize logger. */ - this.logger = Logger.getLogger(DescriptorServlet.class.toString()); - - /* Look up data source. */ - try { - Context cxt = new InitialContext(); - this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/tordir"); - this.logger.info("Successfully looked up data source."); - } catch (NamingException e) { - this.logger.log(Level.WARNING, "Could not look up data source", e); - } - } - - private void writeHeader(PrintWriter out) throws IOException { - out.println("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 " - + "Transitional//EN\"\n" - + "\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n" - + "<html xmlns=\"http://www.w3.org/1999/xhtml\">\n" - + " <head>\n" - + " <meta content=\"text/html; charset=ISO-8859-1\"\n" - + " http-equiv=\"content-type\" />\n" - + " <title>Relay Descriptor</title>\n" - + " <meta http-equiv=Content-Type content=\"text/html; " - + "charset=iso-8859-1\">\n" - + " <link href=\"/css/stylesheet-ltr.css\" type=text/css " - + "rel=stylesheet>\n" - + " <link href=\"/images/favicon.ico\" " - + "type=image/x-icon rel=\"shortcut icon\">\n" - + " </head>\n" - + " <body>\n" - + " <div class=\"center\">\n" - + " <table class=\"banner\" border=\"0\" cellpadding=\"0\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <tr>\n" - + " <td class=\"banner-left\"><a " - + "href=\"/index.html\"><img src=\"/images/top-left.png\" " - + "alt=\"Click to go to home page\" width=\"193\" " - + "height=\"79\"></a></td>\n" - + " <td class=\"banner-middle\">\n" - + " <a href=\"/\">Home</a>\n" - + " <a href=\"graphs.html\">Graphs</a>\n" - + " <a href=\"research.html\">Research</a>\n" - + " <a href=\"status.html\">Status</a>\n" - + " <br/>\n" - + " <font size=\"2\">\n" - + " <a href=\"exonerator.html\">ExoneraTor</a>\n" - + " <a class=\"current\">Relay Search</a>\n" - + " <a href=\"consensus-health.html\">Consensus " - + "Health</a>\n" - + " </font>\n" - + " </td>\n" - + " <td class=\"banner-right\"></td>\n" - + " </tr>\n" - + " </table>\n" - + " <div class=\"main-column\" style=\"margin:5; " - + "Padding:0;\">\n" - + " <h2>Relay Descriptor</h2>\n"); - } - - private void writeFooter(PrintWriter out) throws IOException { - out.println(" <br/>\n" - + " </div>\n" - + " </div>\n" - + " <div class=\"bottom\" id=\"bottom\">\n" - + " <p>This material is supported in part by the National " - + "Science Foundation under Grant No. CNS-0959138. Any " - + "opinions, finding, and conclusions or recommendations " - + "expressed in this material are those of the author(s) and " - + "do not necessarily reflect the views of the National " - + "Science Foundation.</p>\n" - + " <p>\"Tor\" and the \"Onion Logo\" are <a " - + "href=\"https://www.torproject.org/docs/trademark-faq.html.en\">" - + "registered trademarks</a> of The Tor Project, Inc.</p>\n" - + " <p>Data on this site is freely available under a <a " - + "href=\"http://creativecommons.org/publicdomain/zero/1.0/\">" - + "CC0 no copyright declaration</a>: To the extent possible " - + "under law, the Tor Project has waived all copyright and " - + "related or neighboring rights in the data. Graphs are " - + "licensed under a <a " - + "href=\"http://creativecommons.org/licenses/by/3.0/us/\">" - + "Creative Commons Attribution 3.0 United States " - + "License</a>.</p>\n" - + " </div>\n" - + " </body>\n" - + "</html>"); - out.close(); - } - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, - ServletException { - - /* Measure how long it takes to process this request. */ - long started = System.currentTimeMillis(); - - /* Get print writer and start writing response. */ - PrintWriter out = response.getWriter(); - writeHeader(out); - - /* Check desc-id parameter. */ - String descIdParameter = request.getParameter("desc-id"); - String descId = null; - if (descIdParameter != null && descIdParameter.length() >= 8 && - descIdParameter.length() <= 40) { - Pattern descIdPattern = Pattern.compile("^[0-9a-f]{8,40}$"); - if (descIdPattern.matcher(descIdParameter.toLowerCase()). - matches()) { - descId = descIdParameter.toLowerCase(); - } - } - if (descId == null) { - out.write(" <br/><p>Sorry, \"" - + StringEscapeUtils.escapeHtml(descIdParameter) + "\" is not a " - + "valid descriptor identifier. Please provide at least the " - + "first 8 hex characters of a descriptor identifier.</p>\n"); - writeFooter(out); - return; - } - - /* If we were only given a partial descriptor identifier, look up all - * descriptor identifiers starting with that part to see if it's - * unique. */ - if (descId.length() < 40) { - SortedSet<String> allDescIds = new TreeSet<String>(); - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT DISTINCT descriptor FROM statusentry " - + "WHERE descriptor LIKE '" + descId + "%'"; - ResultSet rs = statement.executeQuery(query); - while (rs.next()) { - allDescIds.add(rs.getString(1)); - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool " - + "after " + (System.currentTimeMillis() - - requestedConnection) + " millis."); - } catch (SQLException e) { - out.println("<p><font color=\"red\"><b>Warning: </b></font>We " - + "experienced an unknown database problem while looking up " - + "descriptors with identifier starting with " + descId - + ". If this problem persists, please " - + "<a href=\"mailto:tor-assistants@torproject.org\">let us " - + "know</a>!</p>\n"); - writeFooter(out); - return; - } - if (allDescIds.size() == 0) { - out.write("<p>No descriptor found " + (descId.length() < 40 - ? "starting " : "") + "with identifier " + descId + ".</p>"); - writeFooter(out); - return; - } else if (allDescIds.size() > 1) { - out.println("<p>The descriptor identifier part " + descIdParameter - + " is not unique. Please choose one of the following " - + "descriptors:</p><ul>"); - for (String f : allDescIds) { - out.println("<li><a href=\"descriptor.html?desc-id=" + f + "\">" - + f + "</a></li>"); - } - out.write("</ul><br/>"); - writeFooter(out); - return; - } else { - descId = allDescIds.first(); - } - } - - /* Look up descriptor in the database. */ - String descriptor = null, nickname = null, published = null, - extrainfo = null; - byte[] rawDescriptor = null, rawExtrainfo = null; - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT descriptor, nickname, published, extrainfo, " - + "rawdesc FROM descriptor WHERE descriptor = '" + descId - + "'"; - ResultSet rs = statement.executeQuery(query); - if (rs.next()) { - descriptor = rs.getString(1); - nickname = rs.getString(2); - published = rs.getTimestamp(3).toString().substring(0, 19); - extrainfo = rs.getString(4); - rawDescriptor = rs.getBytes(5); - } - query = "SELECT rawdesc FROM extrainfo WHERE extrainfo = '" - + extrainfo + "'"; - rs = statement.executeQuery(query); - if (rs.next()) { - rawExtrainfo = rs.getBytes(1); - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool after " - + (System.currentTimeMillis() - requestedConnection) - + " millis."); - } catch (SQLException e) { - out.write("<br/><p><font color=\"red\"><b>Warning: </b></font>" - + "Internal server error when looking up descriptor. If this " - + "problem persists, please " - + "<a href=\"mailto:tor-assistants@torproject.org\">let us " - + "know</a>!</p>\n"); - writeFooter(out); - return; - } - - /* If no descriptor was found, stop here. */ - if (descriptor == null) { - out.write("<p>No descriptor found " + (descIdParameter.length() < 40 - ? "starting " : "") + "with identifier " + descIdParameter - + ".</p>"); - writeFooter(out); - return; - } - - /* Print out both server and extra-info descriptor. */ - out.write("<br/><p>The following server descriptor was published by " - + "relay " + nickname + " at " + published + " UTC:</p>"); - String descriptorString = new String(rawDescriptor, "US-ASCII"); - String escapedDescriptorString = StringEscapeUtils.escapeHtml( - descriptorString); - BufferedReader br = new BufferedReader(new StringReader( - escapedDescriptorString)); - String line = null; - while ((line = br.readLine()) != null) { - out.println(" <tt>" + line + "</tt><br/>"); - } - br.close(); - if (rawExtrainfo != null) { - out.println("<br/><p>Together with this server descriptor, the " - + "relay published the following extra-info descriptor:</p>"); - String extraInfoString = new String(rawExtrainfo, "US-ASCII"); - String escapedExtraInfoString = StringEscapeUtils.escapeHtml( - extraInfoString); - br = new BufferedReader(new StringReader(escapedExtraInfoString)); - line = null; - while ((line = br.readLine()) != null) { - out.println(" <tt>" + line + "</tt><br/>"); - } - } - - /* Print out in which consensuses this descriptor is referenced. */ - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT validafter, rawdesc FROM statusentry " - + "WHERE descriptor = '" + descriptor + "' ORDER BY validafter " - + "DESC"; - ResultSet rs = statement.executeQuery(query); - boolean printedDescription = false; - while (rs.next()) { - if (!printedDescription) { - out.println("<br/><p>This server descriptor is referenced from " - + "the following network status consensuses:</p>"); - printedDescription = true; - } - String validAfter = rs.getTimestamp(1).toString(). - substring(0, 19); - out.println(" <br/><tt>valid-after " - + "<a href=\"consensus?valid-after=" - + validAfter.replaceAll(":", "-").replaceAll(" ", "-") - + "\" target=\"_blank\">" + validAfter + "</a></tt><br/>"); - byte[] rawStatusEntry = rs.getBytes(2); - br = new BufferedReader(new StringReader(new String( - rawStatusEntry, "US-ASCII"))); - line = null; - while ((line = br.readLine()) != null) { - out.println(" <tt>" + line + "</tt><br/>"); - } - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool after " - + (System.currentTimeMillis() - requestedConnection) - + " millis."); - } catch (SQLException e) { - out.println("<p><font color=\"red\"><b>Warning: </b></font>We " - + "experienced an unknown database problem while looking up " - + "the network status consensuses referencing descriptor " - + descId + ". If this problem persists, please " - + "<a href=\"mailto:tor-assistants@torproject.org\">let us " - + "know</a>!</p>\n"); - } - - /* Provide links to raw descriptors, too. */ - out.println("<br/><p>Note that the descriptor" + (rawExtrainfo != null - ? "s have" : " has") + " been converted to ASCII and reformatted " - + "for display purposes. You may also download the raw " - + "<a href=\"serverdesc?desc-id=" + descriptor - + "\" target=\"_blank\">server " + "descriptor</a>" - + (extrainfo != null ? " and <a href=\"extrainfodesc?desc-id=" - + extrainfo + "\" target=\"_blank\">extra-info descriptor</a>" - : "") + " as " + (extrainfo != null ? "they were" : "it was") - + " published to the directory authorities.</p>"); - - /* Display total lookup time on the results page. */ - long searchTime = System.currentTimeMillis() - started; - out.write(" <br/><p>Looking up this descriptor took us " - + String.format("%d.%03d", searchTime / 1000, searchTime % 1000) - + " seconds.</p>\n"); - - /* Finish writing response. */ - writeFooter(out); - } -} - diff --git a/src/org/torproject/ernie/web/ExtraInfoDescriptorServlet.java b/src/org/torproject/ernie/web/ExtraInfoDescriptorServlet.java deleted file mode 100644 index 8b532ce..0000000 --- a/src/org/torproject/ernie/web/ExtraInfoDescriptorServlet.java +++ /dev/null @@ -1,113 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.BufferedOutputStream; -import java.io.IOException; -import java.sql.Connection; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Statement; -import java.util.logging.Level; -import java.util.logging.Logger; -import java.util.regex.Matcher; -import java.util.regex.Pattern; - -import javax.naming.Context; -import javax.naming.InitialContext; -import javax.naming.NamingException; -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import javax.sql.DataSource; - -public class ExtraInfoDescriptorServlet extends HttpServlet { - - private static final long serialVersionUID = -177072581949479416L; - - private DataSource ds; - - private Logger logger; - - public void init() { - - /* Initialize logger. */ - this.logger = Logger.getLogger( - ExtraInfoDescriptorServlet.class.toString()); - - /* Look up data source. */ - try { - Context cxt = new InitialContext(); - this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/tordir"); - this.logger.info("Successfully looked up data source."); - } catch (NamingException e) { - this.logger.log(Level.WARNING, "Could not look up data source", e); - } - } - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, - ServletException { - - /* Check desc-id parameter. */ - String descIdParameter = request.getParameter("desc-id"); - if (descIdParameter == null || descIdParameter.length() < 8) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - String descId = descIdParameter.toLowerCase(); - Pattern descIdPattern = Pattern.compile("^[0-9a-f]+$"); - Matcher descIdMatcher = descIdPattern.matcher(descId); - if (!descIdMatcher.matches()) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - - /* Look up descriptor in the database. */ - String extrainfo = null; - byte[] rawDescriptor = null; - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT extrainfo, rawdesc FROM extrainfo " - + "WHERE extrainfo LIKE '" + descId + "%'"; - ResultSet rs = statement.executeQuery(query); - if (rs.next()) { - extrainfo = rs.getString(1); - rawDescriptor = rs.getBytes(2); - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool after " - + (System.currentTimeMillis() - requestedConnection) - + " millis."); - } catch (SQLException e) { - response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR); - return; - } - - /* Write response. */ - if (rawDescriptor == null) { - response.sendError(HttpServletResponse.SC_NOT_FOUND); - return; - } - try { - response.setContentType("text/plain"); - response.setHeader("Content-Length", String.valueOf( - rawDescriptor.length)); - response.setHeader("Content-Disposition", "inline; filename=\"" - + extrainfo + "\""); - BufferedOutputStream output = new BufferedOutputStream( - response.getOutputStream()); - output.write(rawDescriptor); - output.flush(); - output.close(); - } finally { - /* Nothing to do here. */ - } - } -} - diff --git a/src/org/torproject/ernie/web/NetworkStatusServlet.java b/src/org/torproject/ernie/web/NetworkStatusServlet.java deleted file mode 100644 index 844fc4f..0000000 --- a/src/org/torproject/ernie/web/NetworkStatusServlet.java +++ /dev/null @@ -1,24 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.IOException; - -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; - -public class NetworkStatusServlet extends HttpServlet { - - private static final long serialVersionUID = 1762552983543719111L; - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, ServletException { - - /* Forward the request to the JSP. */ - request.getRequestDispatcher("WEB-INF/networkstatus.jsp").forward( - request, response); - } -} - diff --git a/src/org/torproject/ernie/web/RouterDetailServlet.java b/src/org/torproject/ernie/web/RouterDetailServlet.java deleted file mode 100644 index fcbbe74..0000000 --- a/src/org/torproject/ernie/web/RouterDetailServlet.java +++ /dev/null @@ -1,24 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.IOException; - -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; - -public class RouterDetailServlet extends HttpServlet { - - private static final long serialVersionUID = -5740769933146059947L; - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, ServletException { - - /* Forward the request to the JSP. */ - request.getRequestDispatcher("WEB-INF/routerdetail.jsp").forward( - request, response); - } -} - diff --git a/src/org/torproject/ernie/web/VotesServlet.java b/src/org/torproject/ernie/web/VotesServlet.java deleted file mode 100644 index 0821412..0000000 --- a/src/org/torproject/ernie/web/VotesServlet.java +++ /dev/null @@ -1,132 +0,0 @@ -/* Copyright 2011, 2012 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.ernie.web; - -import java.io.BufferedOutputStream; -import java.io.IOException; -import java.sql.Connection; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Statement; -import java.text.ParseException; -import java.text.SimpleDateFormat; -import java.util.ArrayList; -import java.util.List; -import java.util.TimeZone; -import java.util.logging.Level; -import java.util.logging.Logger; - -import javax.naming.Context; -import javax.naming.InitialContext; -import javax.naming.NamingException; -import javax.servlet.ServletException; -import javax.servlet.http.HttpServlet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import javax.sql.DataSource; - -public class VotesServlet extends HttpServlet { - - private static final long serialVersionUID = 246279501605993024L; - - private DataSource ds; - - private Logger logger; - - public void init() { - - /* Initialize logger. */ - this.logger = Logger.getLogger(VotesServlet.class.toString()); - - /* Look up data source. */ - try { - Context cxt = new InitialContext(); - this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/tordir"); - this.logger.info("Successfully looked up data source."); - } catch (NamingException e) { - this.logger.log(Level.WARNING, "Could not look up data source", e); - } - } - - public void doGet(HttpServletRequest request, - HttpServletResponse response) throws IOException, - ServletException { - - /* Check valid-after parameter. */ - String validAfterParameter = request.getParameter("valid-after"); - if (validAfterParameter == null || - validAfterParameter.length() != "yyyy-MM-dd-HH-mm-ss".length()) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - SimpleDateFormat parameterFormat = new SimpleDateFormat( - "yyyy-MM-dd-HH-mm-ss"); - parameterFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - long parsedTimestamp = -1L; - try { - parsedTimestamp = parameterFormat.parse(validAfterParameter). - getTime(); - } catch (ParseException e) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - if (parsedTimestamp < 0L) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - - /* Look up consensus in the database. */ - SimpleDateFormat databaseFormat = new SimpleDateFormat( - "yyyy-MM-dd HH:mm:ss"); - databaseFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - String databaseParameter = databaseFormat.format(parsedTimestamp); - List<byte[]> rawDescriptors = new ArrayList<byte[]>(); - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT rawdesc FROM vote " - + "WHERE validafter = '" + databaseParameter + "'"; - ResultSet rs = statement.executeQuery(query); - while (rs.next()) { - rawDescriptors.add(rs.getBytes(1)); - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool after " - + (System.currentTimeMillis() - requestedConnection) - + " millis."); - } catch (SQLException e) { - response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR); - return; - } - - /* Write response. */ - if (rawDescriptors.size() == 0) { - response.sendError(HttpServletResponse.SC_NOT_FOUND); - return; - } - try { - response.setContentType("text/plain"); - int responseLength = 0; - for (byte[] rawDescriptor : rawDescriptors) { - responseLength += rawDescriptor.length; - } - response.setHeader("Content-Length", String.valueOf( - responseLength)); - response.setHeader("Content-Disposition", "inline; filename=\"" - + validAfterParameter + "-votes\""); - BufferedOutputStream output = new BufferedOutputStream( - response.getOutputStream()); - for (byte[] rawDescriptor : rawDescriptors) { - output.write(rawDescriptor); - } - output.flush(); - output.close(); - } finally { - /* Nothing to do here. */ - } - } -} - diff --git a/web/WEB-INF/banner.jsp b/web/WEB-INF/banner.jsp index a081423..c31875d 100644 --- a/web/WEB-INF/banner.jsp +++ b/web/WEB-INF/banner.jsp @@ -35,8 +35,6 @@ %>>Performance</a> </font> <%} else if (currentPage.endsWith("status.jsp") || - currentPage.endsWith("networkstatus.jsp") || - currentPage.endsWith("routerdetail.jsp") || currentPage.endsWith("exonerator.jsp") || currentPage.endsWith("relay-search.jsp") || currentPage.endsWith("consensus-health.jsp")) { diff --git a/web/WEB-INF/networkstatus.jsp b/web/WEB-INF/networkstatus.jsp deleted file mode 100644 index 8f7b379..0000000 --- a/web/WEB-INF/networkstatus.jsp +++ /dev/null @@ -1,23 +0,0 @@ -<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> -<html> -<head> - <title>Tor Metrics Portal: Network Status</title> - <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> - <link href="/css/stylesheet-ltr.css" type="text/css" rel="stylesheet"> - <link href="/images/favicon.ico" type="image/x-icon" rel="shortcut icon"> -</head> -<body> - <div class="center"> - <%@ include file="banner.jsp"%> - <div class="main-column"> - <h2>Tor Metrics Portal: Network Status</h2> - <br> - <p>Sorry, this page has been discontinued.</p> - </div> - </div> - <div class="bottom" id="bottom"> - <%@ include file="footer.jsp"%> - </div> -</body> -</html> diff --git a/web/WEB-INF/routerdetail.jsp b/web/WEB-INF/routerdetail.jsp deleted file mode 100644 index 416285d..0000000 --- a/web/WEB-INF/routerdetail.jsp +++ /dev/null @@ -1,23 +0,0 @@ -<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> -<html> -<head> - <title>Tor Metrics Portal: Router Detail</title> - <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> - <link href="/css/stylesheet-ltr.css" type="text/css" rel="stylesheet"> - <link href="/images/favicon.ico" type="image/x-icon" rel="shortcut icon"> -</head> -<body> - <div class="center"> - <%@ include file="banner.jsp"%> - <div class="main-column"> - <h2>Tor Metrics Portal: Router Detail</h2> - <br> - <p>Sorry, this page has been discontinued.</p> - </div> - </div> - <div class="bottom" id="bottom"> - <%@ include file="footer.jsp"%> - </div> -</body> -</html> diff --git a/web/robots.txt b/web/robots.txt index 7d138ee..f3ffac3 100644 --- a/web/robots.txt +++ b/web/robots.txt @@ -1,6 +1,4 @@ User-agent: * -Disallow: /networkstatus.html -Disallow: /routerdetail.html Disallow: /relay.html Disallow: /csv/ Disallow: /serverdesc

1 0

[metrics-web/master] Remove functionality to look up all descriptors of a consensus.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit b906c881998a1682d3c4e13c61fbb23e78754018 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 14:53:00 2012 +0100 Remove functionality to look up all descriptors of a consensus. Due to Apache's logs, this functionality was last used in November 2011. Could be a useful feature, but nothing that should prevent us from removing details from the metrics database. --- .../ernie/web/ServerDescriptorServlet.java | 65 +------------------- 1 files changed, 3 insertions(+), 62 deletions(-) diff --git a/src/org/torproject/ernie/web/ServerDescriptorServlet.java b/src/org/torproject/ernie/web/ServerDescriptorServlet.java index c8f3d9c..d0d1f1f 100644 --- a/src/org/torproject/ernie/web/ServerDescriptorServlet.java +++ b/src/org/torproject/ernie/web/ServerDescriptorServlet.java @@ -8,11 +8,8 @@ import java.sql.Connection; import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Statement; -import java.text.ParseException; -import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.List; -import java.util.TimeZone; import java.util.logging.Level; import java.util.logging.Logger; import java.util.regex.Matcher; @@ -55,15 +52,14 @@ public class ServerDescriptorServlet extends HttpServlet { HttpServletResponse response) throws IOException, ServletException { - /* Read desc-id and/or valid-after parameters. */ - String validAfterParameter = request.getParameter("valid-after"); + /* Read desc-id parameter. */ String descIdParameter = request.getParameter("desc-id"); /* See if we were given a desc-id parameter. If so, look up this * descriptor and return it. */ List<byte[]> rawDescriptors = new ArrayList<byte[]>(); String filename = null; - if (descIdParameter != null && validAfterParameter == null) { + if (descIdParameter != null) { if (descIdParameter.length() < 8 || descIdParameter.length() > 40) { response.sendError(HttpServletResponse.SC_BAD_REQUEST); @@ -100,62 +96,7 @@ public class ServerDescriptorServlet extends HttpServlet { return; } - /* See if we were given a valid-after parameter. If so, return all - * descriptors referenced from the consensus published at that - * time. */ - } else if (descIdParameter == null && validAfterParameter != null) { - if (validAfterParameter.length() != - "yyyy-MM-dd-HH-mm-ss".length()) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - SimpleDateFormat parameterFormat = new SimpleDateFormat( - "yyyy-MM-dd-HH-mm-ss"); - parameterFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - long parsedTimestamp = -1L; - try { - parsedTimestamp = parameterFormat.parse(validAfterParameter). - getTime(); - } catch (ParseException e) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - if (parsedTimestamp < 0L) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - filename = validAfterParameter + "-descriptors"; - - /* Look up descriptors in the database. */ - SimpleDateFormat databaseFormat = new SimpleDateFormat( - "yyyy-MM-dd HH:mm:ss"); - databaseFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - String databaseParameter = databaseFormat.format(parsedTimestamp); - try { - long requestedConnection = System.currentTimeMillis(); - Connection conn = this.ds.getConnection(); - Statement statement = conn.createStatement(); - String query = "SELECT descriptor.rawdesc FROM statusentry " - + "JOIN descriptor ON statusentry.descriptor = " - + "descriptor.descriptor WHERE validafter = '" - + databaseParameter + "'"; - ResultSet rs = statement.executeQuery(query); - while (rs.next()) { - rawDescriptors.add(rs.getBytes(1)); - } - rs.close(); - statement.close(); - conn.close(); - this.logger.info("Returned a database connection to the pool " - + "after " + (System.currentTimeMillis() - - requestedConnection) + " millis."); - } catch (SQLException e) { - response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR); - return; - } - - /* Return an error if neither desc-id nor valid-after parameter was - * given (or both of them). */ + /* Return an error if no desc-id parameter was given. */ } else { response.sendError(HttpServletResponse.SC_BAD_REQUEST); return;

1 0

[metrics-web/master] Use the ExoneraTor database for descriptor lookups.
by karsten＠torproject.org 22 Mar '12

22 Mar '12

commit fbeaeb8484cc63a78a3c77a908c143741433bbfc Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 22 14:55:54 2012 +0100 Use the ExoneraTor database for descriptor lookups. --- src/org/torproject/ernie/web/ConsensusServlet.java | 4 ++-- .../ernie/web/ServerDescriptorServlet.java | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/org/torproject/ernie/web/ConsensusServlet.java b/src/org/torproject/ernie/web/ConsensusServlet.java index 20cff07..5f642fe 100644 --- a/src/org/torproject/ernie/web/ConsensusServlet.java +++ b/src/org/torproject/ernie/web/ConsensusServlet.java @@ -39,7 +39,7 @@ public class ConsensusServlet extends HttpServlet { /* Look up data source. */ try { Context cxt = new InitialContext(); - this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/tordir"); + this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/exonerator"); this.logger.info("Successfully looked up data source."); } catch (NamingException e) { this.logger.log(Level.WARNING, "Could not look up data source", e); @@ -83,7 +83,7 @@ public class ConsensusServlet extends HttpServlet { long requestedConnection = System.currentTimeMillis(); Connection conn = this.ds.getConnection(); Statement statement = conn.createStatement(); - String query = "SELECT rawdesc FROM consensus " + String query = "SELECT rawconsensus FROM consensus " + "WHERE validafter = '" + databaseParameter + "'"; ResultSet rs = statement.executeQuery(query); if (rs.next()) { diff --git a/src/org/torproject/ernie/web/ServerDescriptorServlet.java b/src/org/torproject/ernie/web/ServerDescriptorServlet.java index d0d1f1f..5f8ba5b 100644 --- a/src/org/torproject/ernie/web/ServerDescriptorServlet.java +++ b/src/org/torproject/ernie/web/ServerDescriptorServlet.java @@ -41,7 +41,7 @@ public class ServerDescriptorServlet extends HttpServlet { /* Look up data source. */ try { Context cxt = new InitialContext(); - this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/tordir"); + this.ds = (DataSource) cxt.lookup("java:comp/env/jdbc/exonerator"); this.logger.info("Successfully looked up data source."); } catch (NamingException e) { this.logger.log(Level.WARNING, "Could not look up data source", e); @@ -78,7 +78,7 @@ public class ServerDescriptorServlet extends HttpServlet { long requestedConnection = System.currentTimeMillis(); Connection conn = ds.getConnection(); Statement statement = conn.createStatement(); - String query = "SELECT descriptor, rawdesc FROM descriptor " + String query = "SELECT descriptor, rawdescriptor FROM descriptor " + "WHERE descriptor LIKE '" + descId + "%'"; ResultSet rs = statement.executeQuery(query); if (rs.next()) {

1 0