April 2012 - tor-commits - lists.torproject.org

r25627: {website} Adding Ravi's stem proposal as an example Got permission to (in website/trunk/about: en gsocProposal)
by Damian Johnson 26 Apr '12

26 Apr '12

Author: atagar Date: 2012-04-26 16:18:31 +0000 (Thu, 26 Apr 2012) New Revision: 25627 Added: website/trunk/about/gsocProposal/gsoc12-proposal-stemImprovements.html Modified: website/trunk/about/en/gsoc.wml Log: Adding Ravi's stem proposal as an example Got permission to use his proposal as an example on the GSoC page. Modified: website/trunk/about/en/gsoc.wml =================================================================== --- website/trunk/about/en/gsoc.wml 2012-04-26 00:36:52 UTC (rev 25626) +++ website/trunk/about/en/gsoc.wml 2012-04-26 16:18:31 UTC (rev 25627) @@ -226,6 +226,7 @@ <ul> + <li><h4><a href="../about/gsocProposal/gsoc12-proposal-stemImprovements.html">Stem Improvements and Arm port</a> by Ravi Padmala</h4></li> <li><h4><a href="../about/gsocProposal/gsoc10-proposal-soat.txt">SOAT Expansion</a> by John Schanck</h4></li> <li><h4><a href="http://inspirated.com/uploads/tor-gsoc-11.pdf">GTK+ Frontend and Client Mode Improvements for arm</a> by Kamran Khan</h4></li> <li><h4><a href="http://www.gsathya.in/gsoc11.html">Orbot + ORLib</a> by Sathya Gunasekaran</h4></li> Added: website/trunk/about/gsocProposal/gsoc12-proposal-stemImprovements.html =================================================================== --- website/trunk/about/gsocProposal/gsoc12-proposal-stemImprovements.html (rev 0) +++ website/trunk/about/gsocProposal/gsoc12-proposal-stemImprovements.html 2012-04-26 16:18:31 UTC (rev 25627) @@ -0,0 +1,100 @@ +<head> + <title>Stem Improvements and Arm port</title> + <style> +h1 {font-size: 20pt;} + </style> +</head> + +<body> +<h1>What project would you like to work on?</h1> + I would like to work on adding multiple features to Stem and on porting Arm to use Stem. I originally intended to work on implementing a PathSupport equivalent module for Stem. But, I was convinced that it wouldn't be the best idea for a gsoc project. The improvements I am going to make to stem in particular are, addition of a general controller class,  integrating the safe cookie authentication client into Stem and extending stem.descriptor by implementing network status descriptor and microdescriptor parsing functionality. I will also begin porting Arm to use Stem. + +<h2> </h2> +<h2>General Controller Class</h2> +The general controller class will provide basic functionality for accessing control port commands at a higher level (using methods instead of sending the commands to the socket). This will also (ideally) be used a base class for implementing any advanced controllers, such as a consensus tracking controller that would override the newconsensus/ns methods to keep track of the current consensus. * This class will also implement other necessary helper classes/functions, such     as the Router class. * This class will implements methods that can be called to send every know n     control command to the control port. This includes the following commands:     SETCONF, RESETCONF, GETCONF, SETEVENTS, SAVECONF, SIGNAL, AUTHENTICATE, MAPADDRESS, GETINFO, EXTENDCIRCUIT, SETCIRCUITPURPOSE, SETROUTERPURPOSE, ATTACHSTREAM, AUTHCHALLENGE, POSTDESCRIPTOR, REDIRECTSTREAM, CLOSESTREAM, CLOSECIRCUIT, QUIT, USEFEATURE, RESOLVE, LOADCONF, PROTOCOLINFO, TAKEOWNERSHIP. + +* This package implements classes which can parse the output of the above control commands and store the parsed data. These classes will be subclasses of ControlMessage, and the corresponding class for a control command FOOBAR will be called FooBarResponse, usually. For example, the GetInfoResponse class would look something like this (minus the error checking, documentation etc.): +<code><samp>class GetInfoResponse(stem.socket.ControlMessage):   @staticmethod   def convert(control_message):     control_message.__class__ = GetInfoResponse     control_message._parse_message()     return control_message   def _parse_message(self):     self.responses = {}     for reply in self:       if reply == "OK": break       elif reply.is_empty(): continue       k, v = reply.split("=", 1)       if v[0] == "\n": # multiline reply         self.responses[k] = v[1:-2] #strip \n after the = and the \n. at the end       else:         self.responses[k] = v </samp></code> + +* Implement methods which are called when an asynchronous message/command is     received. The events that are received can be modified using the wrapper     around SETEVENTS. The BaseController takes care of queuing the events for     us. * This package will contain exceptions for the various error codes that the control     spec defines. The general controller itself will try to handle all cases in     which an exception is raised. These exceptions will also be used by     controllers which will subclass the general controller class. * This will also involve writing a lot of unit and integration tests. I plan to     write the integration tests by creating a test controller that is a subclass     of the general controller class and running the tests against it. The unit     tests will be written against an object of the general controller class and against objects of the individual *Response classes. * I will also be writing comprehensive documentation that will help people write     their own controller classes which subclasses this without having to go     through the sources. This mostly involves writing concise (and somet imes elaborate) docstrings. I will also be writing a few (2-4) self contained examples of code that subclasses the general controller class. These will be placed in the examples directory. + +<h2> Safe Cookie Authentication</h2> +Currently, Stem does not understand the new safe cookie authentication method. There exists a python script that does the authentication. This task entails distilling the authentication client script to extract the authentication specific parts, and then integrating it into the connection module while writing new code as necessary. I will also be writing integration tests. These will be inspired by the current tests for cookie based authentication, if not completely mimicking them This task will end with closing <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a> +<h2> </h2> +<h2>Descriptor</h2> +stem.descriptor is a python counterpart of metrics-lib. Currently, it is missing a lot of functionality provided by metrics-lib. At this moment, it can parse V3 server descriptors using parse_file_v3 in stem.descriptor.server_descriptor. I will be extending stem.descriptor by implementing methods and classes to handle the following descriptor formats: + +<ol> +<li>V3 network status documents</li> +<li>Microdescriptor documents</li> +<li>Extra-info documents </li> +</ol> + I will be implementing the parsers and classes that contain the parsed data. I will also be implementing methods that will get this data from common data sources. I will also be writing unit and integration tests to ensure the parsers are working as expected. While this task will require writing a large number of tests, documentation will be a relatively simple subtask and won't require as much time as the documentation of the general controller class. +<h2> Arm Port</h2> + +Porting an application to use Stem is an exercise that will test its design. Arm is the Anonymous Relay Monitor. I will port Arm to use Stem instead of TorCtl. This requires the general controller class to be implemented. I assume that I will make atleast some minor modifications to the Stem API during this process. Though at first, this appears like it is a very large task, the time to implement is reduced greatly because many things that Arm does on its own are now done by Stem. Ex: version parsing, connection handling, get_pid methods etc. The newly implemented general controller class will also help greatly in reducing the amount of time taken to port this application, since the Controller class forms a significant chunk of the code that needs to be ported. +Almost all of the work that needs to be here is porting a single file - src/util/torTools.py which also contains the controller class. This file currently consist of a few functions that are already implemented in Stem (the version parsing, get_pid mentioned above). I will begin with replacing the connection handling code and the miscellaneous functions defined in this file. The code that requires TorCtl within the controller will be substituted with the appropriate Stem code. (At a later date, 'outside' of gsoc, this will be refactored to use Stem better). + +There is almost negligible code outside of this single file that imports TorCtl and uses it directly. I will refactor these files such that they make these calls via torTools.py so that the controller library is fully abstracted in that single file. +<h2> Deliverables</h2> +<h3> Mid term evaluation</h3> +* Implementation of a fully documented General controller class with sufficient     test coverage. * Implementation of parsers for parsing network status documents and     microdescriptor documents with significant test coverage. * Integrate the Safe Cookie authentication client into Stem. (Close <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a>). +<h3> </h3> + +<h3>Final Evaluation</h3> +* Implementation of extra-info document parser. +* A port of Arm which uses Stem. Ideally I will merge my stem port with the     master branch to completely remove the TorCtl dependency and make the next     Arm release Stem-based +<h2> Timeline</h2> +<h3> April 23rd - May 20th Community bonding period</h3> +I'll spend this time working on minor stem bugs. I will also be familiarizing myself with the Arm codebase. + +  +<h3>May 20th - July 9th Coding Period (Pre-mid term evaluation)</h3> +Week 1 +Integrate safe cookie authentication into Stem. Write integration tests. Close <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a> +Implement the classes required to for the general controller class and other helper functions (Router, Exceptions, etc.). Week 2 Implement wrapper classes for AUTHENTICATE and PROTOCOLINFO. Implement the following control command parser/container classes for the following commands: * GETINFO * GETCONF * SETCONF * LOADCONF * RESETCONF * SAVECONF * SIGNAL * TAKEOWNERSHIP * USEFEATURE * QUIT Week 3 Implement control command parser/container for the following commands: * EXTENDCIRCUIT * SETCIRCUITPURPOSE * ATTACHSTREAM * REDIRECTSTREAM * CLOSESTREAM * CLOSECIRCUIT * MAPADDRESS * RESOLVE * POSTDESCRIPTOR Week 4 Implement control command parser/container for the SETEVENTS command. Implement the event handling methods for the following asynchronous events: * Log messages (DEBUG/INFO/NOTICE/WARN/ERR) * CIRC * STREAM * OR * BW * SIGNAL * STREAM_BW Week 5 Implement the event handling methods for the following asynchronous events: * NEWDESC * DESCCHANGED * ADDRMAP * Status reports * GUARD <spa n style="font-size: small;"> * BUILDTIMEOUT_SET * CIRC_MINOR * CLIENTS_SEEN Week 6 Implement the event handling methods for the following asynchronous events: * NS * NEWCONSENSUS Writing additional documentation for the controller class. Writing any additional tests. Buffer time. Week 7 Implement a parser for parsing network status descriptors and the microdescriptors. Write integration tests for the parser. Implement classes for storing this parsed descriptor data. Write functions to get them from common data sources. + +Week 8 +Additional buffer time to complete any pending controller class work. (Otherwise, begin work on the extra-info document parser). +<h3>July 9th - August 13th (Post-mid term evaluation)</h3> +Week 9 +Implement a parser for parsing extra-info documents. +  +Week 10 Begin porting Arm to use Stem. Start with replacing the functions in torTools.py with their stem equivalents. Begin replacing TorCtl code in the controller class with the equivalent Stem code. + Week 11-12 More controller class work. Replace the little TorCtl elsewhere in the codebase with equivalent Stem code. I will be left with some buffer time here. + +<h3> August 13th - August 20th (Post-soft pencils down deadline)</h3> +Week 13 Post soft-pencils down week. Buffer time to finish any unfinished work. Any extra time will be spent on documenting Stem. +<h1> </h1> +<h1>Point us to a code sample: something good and clean to demonstrate that you know what you're doing, ideally from an existing project.</h1> +I have written a few patches for some Tor Project projects, <a title="#1667" href="https://trac.torproject.org/projects/tor/ticket/1667">#1667</a> (Tor), <a title="#5032" href="https://trac.torproject.org/projects/tor/ticket/5032">#5032</a> (Thandy). Two to Stem, which have been committed to the repository <a title="#5199" href="https://trac.torproject.org/projects/tor/ticket/5199">#5199</a> and <a title="#5472" href="https://trac.torproject.org/projects/tor/ticket/5472">#5472</a>. + +<h1> </h1> +<h1>Why do you want to work with The Tor Project / EFF in particular?</h1> + I began reading stuff about The Tor Project about 2 months ago after Sathyanarayanan suggested that I contribute to it. Now, I love the internet, and it is responsible for a large part who I am. The Tor Project and the EFF work to defend the things that make the internet what it is, i.e. (among other things) free speech. I can relate with this goal, and this is why I want to work with The Tor Project/EFF. +<h1> Tell us about your experiences in free software development environments. We especially want to hear examples of how you have collaborated with others rather than just working on a project by yourself.</h1> + Though I have been using Free software for a long time (I switched to Linux about 7 years ago), I haven't made any significant contributions to free software, apart from a few bugs reports and minor patches. However, I am familiar with version control software, bug trackers etc. I have used them while submitting the patches mentioned earlier. + +<h1> </h1> +<h1> Will you be working full-time on the  project for the summer, or will you have other commitments too (a second job,  classes, etc)? If you won't be available full-time, please explain, and list timing  if you know them for other major deadlines (e.g. exams). Having other  activities isn't a deal-breaker, but we don't want to be surprised.</h1> +  +I have exams until the 29th of April, so I will be missing a few days of the community bonding period, though, I hope to show up on the IRC channels even then, albeit sporadically. I also might have to write an exam either in july or august. Though, that depends on me flunking. It won't cost me more than 2 days, and I will work extra during the weekends to make up for it. + +<h1> Will your project need more work and/or maintenance after the summer ends?  What are the chances you will stick  around and help out with that and other  related projects?</h1> + Stem, like all libraries implementing an API for a moving target requires maintenance. I will co-maintain Stem in the future. By the time I'm done with the SoC program. I would've also gained familiarity with Arm. I'll be able to contribute to it, and I will. Personally, I am also interested in getting involved in Tor development, and the re-implementation of Thandy (if/when it happens). +<h1> What is your ideal approach to keeping  everybody informed of your progress, problems, and questions over the  course of the project? Said another way, how much of a "manager" will you need your mentor to be?</h1> + + IRC is my preferred mode of communication, and I will be using it to ask questions and for help with my problems. If I'm unable to get the answer I want on the IRC, I will ask them on the mailing list. I will keep people informed about my progress by sending (probably monthly, or as often as required) reports the mailing list. +<h1> </h1> +<h1>What school are you attending? What  year are you, and what's your major/degree/focus? If you're part of a  research group, which one?</h1> + I'm an undergraduate student majoring in computer science studying at GITAM University  I'm currently working on my final year project which involves computer network modelling. + +<h1> </h1> +<h1> How can we contact you to ask you further questions? Google doesn't share your contact details with us automatically, so you should include that in your application. In addition, what's your IRC nickname? Interacting with us on IRC will help us get to know you, and help you get to know our  community.</h1> + I'm available via email at <removed>. I'm also subscribed to many of the tor-* mailing lists, including tor-dev, tor-talk, tor-bugs and tor-commits. My nickname on OFTC is 'neena'. My email account also doubles up as my Jabber account, though, I prefer IRC.   +<h1> Are you applying to other projects for  GSoC and, if so, what would be your preference if you're accepted to both?  Having a stated preference helps with  the deduplication process and will not  impact if we accept your application or  not.</h1> + + I am not applying to any other projects for GSoC. +  +</body> +

1 0

[obfsproxy/master] Produce an error if pkg-config isn't found.
by asn＠torproject.org 26 Apr '12

26 Apr '12

commit 5830a400052cfbb397d67aa2149a564af9f2df26 Author: Rob Savoye <rob(a)welcomehome.org> Date: Thu Apr 26 08:40:57 2012 -0600 Produce an error if pkg-config isn't found. --- m4/pkg.m4 | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/m4/pkg.m4 b/m4/pkg.m4 index 31169b4..c9d0dd0 100644 --- a/m4/pkg.m4 +++ b/m4/pkg.m4 @@ -43,6 +43,8 @@ if test -n "$PKG_CONFIG"; then AC_MSG_RESULT([no]) PKG_CONFIG="" fi +else + AC_MSG_ERROR([pkg-config not found!]) fi[]dnl ])# PKG_PROG_PKG_CONFIG

1 0

[obfsproxy/master] Produce an error if pkg-config's version is old.
by asn＠torproject.org 26 Apr '12

26 Apr '12

commit e0e53de4f5ea91762fd810f2c5755f0c22e578ef Author: George Kadianakis <desnacked(a)riseup.net> Date: Thu Apr 26 18:05:19 2012 +0300 Produce an error if pkg-config's version is old. --- m4/pkg.m4 | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/m4/pkg.m4 b/m4/pkg.m4 index c9d0dd0..cb5292d 100644 --- a/m4/pkg.m4 +++ b/m4/pkg.m4 @@ -41,10 +41,10 @@ if test -n "$PKG_CONFIG"; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) - PKG_CONFIG="" + AC_MSG_ERROR([pkg-config must be at least version $_pkg_min_version.]) fi else - AC_MSG_ERROR([pkg-config not found!]) + AC_MSG_ERROR([pkg-config not found!]) fi[]dnl ])# PKG_PROG_PKG_CONFIG

1 0

[metrics-tasks/master] Add code to verify consensuses (#2768).
by karsten＠torproject.org 26 Apr '12

26 Apr '12

commit 90ff9b555c9d043d8f4392e08ee6a5e5dcdc9f68 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Apr 26 16:14:58 2012 +0200 Add code to verify consensuses (#2768). --- task-2768/VerifyDescriptors.java | 306 ++++++++++++++++++++++++++++++++ task-2768/VerifyServerDescriptors.java | 145 --------------- task-2768/run.sh | 2 +- 3 files changed, 307 insertions(+), 146 deletions(-) diff --git a/task-2768/VerifyDescriptors.java b/task-2768/VerifyDescriptors.java new file mode 100644 index 0000000..b784cdd --- /dev/null +++ b/task-2768/VerifyDescriptors.java @@ -0,0 +1,306 @@ +/* Copyright 2012 The Tor Project + * See LICENSE for licensing information */ +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.StringReader; +import java.security.Security; +import java.security.interfaces.RSAPublicKey; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; + +import org.apache.commons.codec.binary.Base64; +import org.apache.commons.codec.binary.Hex; +import org.bouncycastle.asn1.ASN1OutputStream; +import org.bouncycastle.crypto.digests.SHA1Digest; +import org.bouncycastle.crypto.encodings.PKCS1Encoding; +import org.bouncycastle.crypto.engines.RSAEngine; +import org.bouncycastle.crypto.params.RSAKeyParameters; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.bouncycastle.openssl.PEMReader; +import org.torproject.descriptor.Descriptor; +import org.torproject.descriptor.DescriptorFile; +import org.torproject.descriptor.DescriptorReader; +import org.torproject.descriptor.DescriptorSourceFactory; +import org.torproject.descriptor.DirectoryKeyCertificate; +import org.torproject.descriptor.DirectorySignature; +import org.torproject.descriptor.RelayNetworkStatusConsensus; +import org.torproject.descriptor.ServerDescriptor; + +/* + * Verify server descriptors using the contained signing key. Verify that + * 1) a contained fingerprint is actually a hash of the signing key and + * 2) a router signature was created using the signing key. + * + * Verify consensuses using the separate certs. Verify that + * 1) the fingerprint in a cert is actually a hash of the identity key, + * 2) a cert was signed using the identity key, + * 3) a consensus was signed using the signing key from the cert. + * + * Usage: + * - Put certs in in/certs/, consensuses in in/consensuses/, and server + * descriptors in in/server-descriptors/. + * - Clone metrics-lib, run `ant jar`, and copy descriptor.jar to this + * directory. + * - Download Apache Commons Codec and Compress jar files + * commons-codec-1.4.jar and commons-compress-1.3.jar and put them in + * this directory. + * - Download BouncyCastle 1.47 jar files bcprov-jdk15on-147.jar and + * bcpkix-jdk15on-147.jar and put them in this directory. + * - Compile and run this class: ./run.sh. + */ +public class VerifyDescriptors { + public static void main(String[] args) throws Exception { + System.out.println("Verifying consensuses..."); + if (Security.getProvider("BC") == null) { + Security.addProvider(new BouncyCastleProvider()); + } + verifyServerDescriptors(); + verifyConsensuses(); + } + + private static void verifyServerDescriptors() throws Exception { + File serverDescriptorDirectory = new File("in/server-descriptors"); + if (!serverDescriptorDirectory.exists()) { + return; + } + DescriptorReader descriptorReader = DescriptorSourceFactory + .createDescriptorReader(); + descriptorReader.addDirectory(serverDescriptorDirectory); + Iterator<DescriptorFile> descriptorFiles = + descriptorReader.readDescriptors(); + int processedDescriptors = 0, verifiedDescriptors = 0; + while (descriptorFiles.hasNext()) { + DescriptorFile descriptorFile = descriptorFiles.next(); + if (descriptorFile.getException() != null) { + System.err.println("Could not read/parse descriptor file " + + descriptorFile.getFileName() + ": " + + descriptorFile.getException().getMessage()); + continue; + } + if (descriptorFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : descriptorFile.getDescriptors()) { + if (!(descriptor instanceof ServerDescriptor)) { + continue; + } + ServerDescriptor serverDescriptor = (ServerDescriptor) descriptor; + boolean isVerified = true; + + /* Verify that the contained fingerprint is a hash of the signing + * key. */ + String signingKeyHashString = determineKeyHash( + serverDescriptor.getSigningKey()); + String fingerprintString = + serverDescriptor.getFingerprint().toLowerCase(); + if (!signingKeyHashString.equals(fingerprintString)) { + System.out.println("In " + descriptorFile.getFile() + + ", server descriptor, the calculated signing key hash " + + " does not match the contained fingerprint!"); + isVerified = false; + } + + /* Verify that the router signature was created using the signing + * key. */ + if (!verifySignature(serverDescriptor.getServerDescriptorDigest(), + serverDescriptor.getRouterSignature(), + serverDescriptor.getSigningKey())) { + System.out.println("In " + descriptorFile.getFile() + + ", the decrypted signature does not match the descriptor " + + "digest!"); + isVerified = false; + } + + processedDescriptors++; + if (isVerified) { + verifiedDescriptors++; + } + } + } + System.out.println("Verified " + verifiedDescriptors + "/" + + processedDescriptors + " server descriptors."); + } + + private static void verifyConsensuses() throws Exception { + File certsDirectory = new File("in/certs"); + File consensusDirectory = new File("in/consensuses"); + if (!certsDirectory.exists() || !consensusDirectory.exists()) { + return; + } + Map<String, String> signingKeys = new HashMap<String, String>(); + + DescriptorReader certsReader = DescriptorSourceFactory + .createDescriptorReader(); + certsReader.addDirectory(certsDirectory); + Iterator<DescriptorFile> descriptorFiles = + certsReader.readDescriptors(); + int processedCerts = 0, verifiedCerts = 0; + while (descriptorFiles.hasNext()) { + DescriptorFile descriptorFile = descriptorFiles.next(); + if (descriptorFile.getException() != null) { + System.err.println("Could not read/parse descriptor file " + + descriptorFile.getFileName() + ": " + + descriptorFile.getException().getMessage()); + continue; + } + if (descriptorFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : descriptorFile.getDescriptors()) { + if (!(descriptor instanceof DirectoryKeyCertificate)) { + continue; + } + DirectoryKeyCertificate cert = + (DirectoryKeyCertificate) descriptor; + boolean isVerified = true; + + /* Verify that the contained fingerprint is a hash of the signing + * key. */ + String dirIdentityKeyHashString = determineKeyHash( + cert.getDirIdentityKey()); + String fingerprintString = cert.getFingerprint().toLowerCase(); + if (!dirIdentityKeyHashString.equals(fingerprintString)) { + System.out.println("In " + descriptorFile.getFile() + + ", the calculated directory identity key hash " + + dirIdentityKeyHashString + + " does not match the contained fingerprint " + + fingerprintString + "!"); + isVerified = false; + } + + /* Verify that the router signature was created using the signing + * key. */ + if (!verifySignature(cert.getCertificateDigest(), + cert.getDirKeyCertification(), cert.getDirIdentityKey())) { + System.out.println("In " + descriptorFile.getFile() + + ", the decrypted directory key certification does not " + + "match the certificate digest!"); + isVerified = false; + } + + /* Determine the signing key digest and remember the signing key + * to verify consensus signatures. */ + String dirSigningKeyString = cert.getDirSigningKey(); + PEMReader pemReader2 = new PEMReader(new StringReader( + dirSigningKeyString)); + RSAPublicKey dirSigningKey = + (RSAPublicKey) pemReader2.readObject(); + ByteArrayOutputStream baos2 = new ByteArrayOutputStream(); + new ASN1OutputStream(baos2).writeObject( + new org.bouncycastle.asn1.pkcs.RSAPublicKey( + dirSigningKey.getModulus(), + dirSigningKey.getPublicExponent()).toASN1Primitive()); + byte[] pkcs2 = baos2.toByteArray(); + byte[] dirSigningKeyHashBytes = new byte[20]; + SHA1Digest sha1_2 = new SHA1Digest(); + sha1_2.update(pkcs2, 0, pkcs2.length); + sha1_2.doFinal(dirSigningKeyHashBytes, 0); + String dirSigningKeyHashString = Hex.encodeHexString( + dirSigningKeyHashBytes).toUpperCase(); + signingKeys.put(dirSigningKeyHashString, cert.getDirSigningKey()); + + processedCerts++; + if (isVerified) { + verifiedCerts++; + } + } + } + System.out.println("Verified " + verifiedCerts + "/" + + processedCerts + " certs."); + + DescriptorReader consensusReader = DescriptorSourceFactory + .createDescriptorReader(); + consensusReader.addDirectory(consensusDirectory); + Iterator<DescriptorFile> consensusFiles = + consensusReader.readDescriptors(); + int processedConsensuses = 0, verifiedConsensuses = 0; + while (consensusFiles.hasNext()) { + DescriptorFile consensusFile = consensusFiles.next(); + if (consensusFile.getException() != null) { + System.err.println("Could not read/parse descriptor file " + + consensusFile.getFileName() + ": " + + consensusFile.getException().getMessage()); + continue; + } + if (consensusFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : consensusFile.getDescriptors()) { + if (!(descriptor instanceof RelayNetworkStatusConsensus)) { + continue; + } + RelayNetworkStatusConsensus consensus = + (RelayNetworkStatusConsensus) descriptor; + boolean isVerified = true; + + /* Verify all signatures using the corresponding certificates. */ + if (consensus.getDirectorySignatures().isEmpty()) { + System.out.println(consensusFile.getFile() + + " does not contain any signatures."); + continue; + } + for (DirectorySignature signature : + consensus.getDirectorySignatures().values()) { + String signingKeyDigest = signature.getSigningKeyDigest(); + if (!signingKeys.containsKey(signingKeyDigest)) { + System.out.println("Cannot find signing key with digest " + + signingKeyDigest + "!"); + } + if (!verifySignature(consensus.getConsensusDigest(), + signature.getSignature(), + signingKeys.get(signingKeyDigest))) { + System.out.println("In " + consensusFile.getFile() + + ", the decrypted signature digest does not match the " + + "consensus digest!"); + isVerified = false; + } + } + processedConsensuses++; + if (isVerified) { + verifiedConsensuses++; + } + } + } + System.out.println("Verified " + verifiedConsensuses + "/" + + processedConsensuses + " consensuses."); + } + + private static String determineKeyHash(String key) throws Exception { + PEMReader pemReader = new PEMReader(new StringReader(key)); + RSAPublicKey dirIdentityKey = (RSAPublicKey) pemReader.readObject(); + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + new ASN1OutputStream(baos).writeObject( + new org.bouncycastle.asn1.pkcs.RSAPublicKey( + dirIdentityKey.getModulus(), + dirIdentityKey.getPublicExponent()).toASN1Primitive()); + byte[] pkcs = baos.toByteArray(); + byte[] dirIdentityKeyHashBytes = new byte[20]; + SHA1Digest sha1 = new SHA1Digest(); + sha1.update(pkcs, 0, pkcs.length); + sha1.doFinal(dirIdentityKeyHashBytes, 0); + String keyHash = Hex.encodeHexString(dirIdentityKeyHashBytes); + return keyHash; + } + + private static boolean verifySignature(String digest, String signature, + String signingKey) throws Exception { + byte[] signatureBytes = Base64.decodeBase64(signature.substring( + 0 + "-----BEGIN SIGNATURE-----\n".length(), + signature.length() - "-----END SIGNATURE-----\n".length()). + replaceAll("\n", "")); + RSAPublicKey rsaSigningKey = (RSAPublicKey) new PEMReader( + new StringReader(signingKey)).readObject(); + RSAKeyParameters rsakp = new RSAKeyParameters(false, + rsaSigningKey.getModulus(), + rsaSigningKey.getPublicExponent()); + PKCS1Encoding pe = new PKCS1Encoding(new RSAEngine()); + pe.init(false, rsakp); + byte[] decryptedSignatureDigest = pe.processBlock( + signatureBytes, 0, signatureBytes.length); + String decryptedSignatureDigestString = + Hex.encodeHexString(decryptedSignatureDigest); + return decryptedSignatureDigestString.equalsIgnoreCase(digest); + } +} + diff --git a/task-2768/VerifyServerDescriptors.java b/task-2768/VerifyServerDescriptors.java deleted file mode 100644 index e648d92..0000000 --- a/task-2768/VerifyServerDescriptors.java +++ /dev/null @@ -1,145 +0,0 @@ -/* Copyright 2012 The Tor Project - * See LICENSE for licensing information */ -import java.io.ByteArrayOutputStream; -import java.io.File; -import java.io.StringReader; -import java.security.Security; -import java.security.interfaces.RSAPublicKey; -import java.util.Iterator; - -import org.apache.commons.codec.binary.Base64; -import org.apache.commons.codec.binary.Hex; -import org.bouncycastle.asn1.ASN1OutputStream; -import org.bouncycastle.crypto.digests.SHA1Digest; -import org.bouncycastle.crypto.encodings.PKCS1Encoding; -import org.bouncycastle.crypto.engines.RSAEngine; -import org.bouncycastle.crypto.params.RSAKeyParameters; -import org.bouncycastle.jce.provider.BouncyCastleProvider; -import org.bouncycastle.openssl.PEMReader; -import org.torproject.descriptor.Descriptor; -import org.torproject.descriptor.DescriptorFile; -import org.torproject.descriptor.DescriptorReader; -import org.torproject.descriptor.DescriptorSourceFactory; -import org.torproject.descriptor.ServerDescriptor; - -/* - * Verify server descriptors using the contained signing key. Verify that - * 1) the contained fingerprint is actually a hash of the signing key and - * 2) the router signature was created using the signing key. - * - * Usage: - * - Extract server descriptors to in/. - * - Clone metrics-lib, run `ant jar`, and copy descriptor.jar to this - * directory. - * - Download Apache Commons Codec and Compress jar files - * commons-codec-1.4.jar and commons-compress-1.3.jar and put them in - * this directory. - * - Download BouncyCastle 1.47 jar files bcprov-jdk15on-147.jar and - * bcpkix-jdk15on-147.jar and put them in this directory. - * - Compile and run this class: ./run.sh. - */ -public class VerifyServerDescriptors { - public static void main(String[] args) throws Exception { - System.out.println("Verifying descriptors..."); - if (Security.getProvider("BC") == null) { - Security.addProvider(new BouncyCastleProvider()); - } - File inputDirectory = new File("in/"); - DescriptorReader reader = DescriptorSourceFactory - .createDescriptorReader(); - reader.addDirectory(inputDirectory); - Iterator<DescriptorFile> descriptorFiles = reader.readDescriptors(); - int processedDescriptors = 0, verifiedDescriptors = 0; - while (descriptorFiles.hasNext()) { - DescriptorFile descriptorFile = descriptorFiles.next(); - if (descriptorFile.getException() != null) { - System.err.println("Could not read/parse descriptor file " - + descriptorFile.getFileName() + ": " - + descriptorFile.getException().getMessage()); - continue; - } - if (descriptorFile.getDescriptors() == null) { - continue; - } - for (Descriptor descriptor : descriptorFile.getDescriptors()) { - if (!(descriptor instanceof ServerDescriptor)) { - continue; - } - ServerDescriptor serverDescriptor = (ServerDescriptor) descriptor; - boolean isVerified = true; - - /* Verify that the contained fingerprint is a hash of the signing - * key. */ - String signingKeyString = serverDescriptor.getSigningKey(); - String fingerprintString = - serverDescriptor.getFingerprint().toLowerCase(); - PEMReader pemReader = new PEMReader(new StringReader( - signingKeyString)); - RSAPublicKey signingKey = (RSAPublicKey) pemReader.readObject(); - - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - new ASN1OutputStream(baos).writeObject( - new org.bouncycastle.asn1.pkcs.RSAPublicKey( - signingKey.getModulus(), - signingKey.getPublicExponent()).toASN1Primitive()); - byte[] pkcs = baos.toByteArray(); - byte[] signingKeyHashBytes = new byte[20]; - SHA1Digest sha1 = new SHA1Digest(); - sha1.update(pkcs, 0, pkcs.length); - sha1.doFinal(signingKeyHashBytes, 0); - String signingKeyHashString = Hex.encodeHexString( - signingKeyHashBytes); - if (!signingKeyHashString.equals(fingerprintString)) { - System.out.println("In " + descriptorFile.getFile() - + ", server descriptor " - + serverDescriptor.getServerDescriptorDigest() - + ", the calculated signing key hash " - + signingKeyHashString - + " does not match the contained fingerprint " - + fingerprintString + "!"); - isVerified = false; - } - - /* Verify that the router signature was created using the signing - * key. */ - String serverDescriptorDigestString = serverDescriptor - .getServerDescriptorDigest().toLowerCase(); - String routerSignatureString = serverDescriptor - .getRouterSignature(); - byte[] routerSignature = Base64 - .decodeBase64(routerSignatureString.substring( - 0 + "-----BEGIN SIGNATURE-----\n".length(), - routerSignatureString.length() - - "-----END SIGNATURE-----\n".length()).replaceAll( - "\n", "")); - RSAKeyParameters rsakp = new RSAKeyParameters(false, - signingKey.getModulus(), signingKey.getPublicExponent()); - PKCS1Encoding pe = new PKCS1Encoding(new RSAEngine()); - pe.init(false, rsakp); - byte[] decryptedSignature = pe.processBlock(routerSignature, 0, - routerSignature.length); - String decryptedSignatureString = - Hex.encodeHexString(decryptedSignature); - if (!decryptedSignatureString.equals( - serverDescriptorDigestString)) { - System.out.println("In " + descriptorFile.getFile() - + ", server descriptor " - + serverDescriptor.getServerDescriptorDigest() - + ", the decrypted signature " - + decryptedSignatureString - + " does not match the descriptor digest " - + serverDescriptorDigestString + "!"); - isVerified = false; - } - - processedDescriptors++; - if (isVerified) { - verifiedDescriptors++; - } - } - } - System.out.println("Verified " + verifiedDescriptors + "/" - + processedDescriptors + " server descriptors."); - } -} - diff --git a/task-2768/run.sh b/task-2768/run.sh index ed5fcc4..658ed6a 100755 --- a/task-2768/run.sh +++ b/task-2768/run.sh @@ -1,3 +1,3 @@ #!/bin/bash -javac -cp descriptor.jar:commons-codec-1.4.jar:commons-compress-1.3.jar:bcprov-jdk15on-147.jar:bcpkix-jdk15on-147.jar VerifyServerDescriptors.java && java -cp descriptor.jar:commons-codec-1.4.jar:commons-compress-1.3.jar:bcprov-jdk15on-147.jar:bcpkix-jdk15on-147.jar:. VerifyServerDescriptors +javac -cp descriptor.jar:commons-codec-1.4.jar:commons-compress-1.3.jar:bcprov-jdk15on-147.jar:bcpkix-jdk15on-147.jar VerifyDescriptors.java && java -cp descriptor.jar:commons-codec-1.4.jar:commons-compress-1.3.jar:bcprov-jdk15on-147.jar:bcpkix-jdk15on-147.jar:. VerifyDescriptors

1 0

[metrics-lib/master] Add unit tests for all stats in extra-info descriptors.
by karsten＠torproject.org 26 Apr '12

26 Apr '12

commit 18ad99171f4fc51112b7e64aab8c503f6393ca94 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Apr 26 11:28:31 2012 +0200 Add unit tests for all stats in extra-info descriptors. --- .../impl/ExtraInfoDescriptorImplTest.java | 460 +++++++++++++++++++- 1 files changed, 438 insertions(+), 22 deletions(-) diff --git a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java index ae1b87a..c21a4ec 100644 --- a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java +++ b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java @@ -3,11 +3,14 @@ package org.torproject.descriptor.impl; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; import java.util.ArrayList; import java.util.List; import java.util.Map; +import java.util.SortedMap; import org.junit.Test; import org.torproject.descriptor.ExtraInfoDescriptor; @@ -743,6 +746,66 @@ public class ExtraInfoDescriptorImplTest { assertEquals(1328951316000L, descriptor.getPublishedMillis()); } + /* TODO This test should fail, even though dir-spec.txt doesn't + * explicitly say that reported bytes must be non-negative. */ + @Test() + public void testWriteHistoryNegativeBytes() + throws DescriptorParseException { + DescriptorBuilder.createWithWriteHistoryLine("write-history " + + "2012-02-11 09:03:39 (900 s) " + + "-4713350144,-4723824640,-4710717440,-4572675072"); + } + + @Test(expected = DescriptorParseException.class) + public void testReadHistoryTabInterval() + throws DescriptorParseException { + DescriptorBuilder.createWithReadHistoryLine("read-history " + + "2012-02-11 09:03:39 (900\ts) " + + "4707695616,4699666432,4650004480,4489718784"); + } + + @Test(expected = DescriptorParseException.class) + public void testReadHistoryTabIntervalBytes() + throws DescriptorParseException { + DescriptorBuilder.createWithReadHistoryLine("read-history " + + "2012-02-11 09:03:39 (900 s)\t" + + "4707695616,4699666432,4650004480,4489718784"); + } + + /* TODO This test should fail, even though dir-spec.txt doesn't + * explicitly say that the interval must be positive. */ + @Test() + public void testReadHistoryNegativeInterval() + throws DescriptorParseException { + DescriptorBuilder.createWithReadHistoryLine("read-history " + + "2012-02-11 09:03:39 (-900 s) " + + "4707695616,4699666432,4650004480,4489718784"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqWriteHistoryMissingBytesBegin() + throws DescriptorParseException { + DescriptorBuilder.createWithDirreqWriteHistoryLine( + "dirreq-write-history 2012-02-11 09:03:39 (900 s) " + + ",64996352,60625920,67922944"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqWriteHistoryMissingBytesMiddle() + throws DescriptorParseException { + DescriptorBuilder.createWithDirreqWriteHistoryLine( + "dirreq-write-history 2012-02-11 09:03:39 (900 s) " + + "81281024,,60625920,67922944"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqReadHistoryMissingBytesEnd() + throws DescriptorParseException { + DescriptorBuilder.createWithDirreqReadHistoryLine( + "dirreq-read-history 2012-02-11 09:03:39 (900 s) " + + "17074176,16235520,16005120,"); + } + @Test() public void testGeoipDbDigestValid() throws DescriptorParseException { ExtraInfoDescriptor descriptor = DescriptorBuilder. @@ -752,49 +815,356 @@ public class ExtraInfoDescriptorImplTest { descriptor.getGeoipDbDigest()); } - /* TODO Add tests for invalid geoip-db-digest lines. */ + @Test(expected = DescriptorParseException.class) + public void testGeoipDbDigestTooShort() + throws DescriptorParseException { + DescriptorBuilder.createWithGeoipDbDigestLine("geoip-db-digest " + + "916A3CA8B7DF61473D5AE5B21711F35F301C"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipDbDigestIllegalChars() + throws DescriptorParseException { + DescriptorBuilder.createWithGeoipDbDigestLine("geoip-db-digest " + + "&%6A3CA8B7DF61473D5AE5B21711F35F301CE9E8"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipDbDigestMissing() + throws DescriptorParseException { + DescriptorBuilder.createWithGeoipDbDigestLine("geoip-db-digest"); + } @Test() public void testGeoipStatsValid() throws DescriptorParseException { - GeoipStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = GeoipStatsBuilder. + createWithDefaultLines(); + assertEquals(1328898771000L, descriptor.getGeoipStartTimeMillis()); + SortedMap<String, Integer> ips = descriptor.getGeoipClientOrigins(); + assertNotNull(ips); + assertEquals(1152, ips.get("de").intValue()); + assertEquals(896, ips.get("cn").intValue()); + assertFalse(ips.containsKey("pl")); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipStartTimeDateOnly() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipStartTimeLine("geoip-start-time " + + "2012-02-10"); } - /* TODO Add tests for invalid geoip stats. */ + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsDash() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de-1152,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,ir=200"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsZero() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=zero,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,ir=200"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsNone() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=none,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,ir=200"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsOther() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=1152,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,other=200"); + } + + @Test() + public void testGeoipClientOriginsQuestionMarks() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=1152,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,??=200"); + } + + @Test() + public void testGeoipClientOriginsCapital() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins DE=1152,CN=896,US=712,IT=504,RU=352,FR=208," + + "GB=208,IR=200"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsMissingBegin() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins ,cn=896,us=712,it=504,ru=352,fr=208,gb=208," + + "ir=200"); + } + + @Test(expected = DescriptorParseException.class) + public void testGeoipClientOriginsMissingMiddle() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=1152,,us=712,it=504,ru=352,fr=208," + + "gb=208,ir=200"); + } + + /* TODO Lines shouldn't be allowed to end with a comma. This also + * applies to all other key-value pair lines. */ + @Test() + public void testGeoipClientOriginsMissingEnd() + throws DescriptorParseException { + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=1152,cn=896,us=712,it=504,ru=352,fr=208," + + "gb=208,"); + } + + @Test() + public void testGeoipClientOriginsDuplicate() + throws DescriptorParseException { + /* dir-spec.txt doesn't say anything about duplicate country codes, so + * this line is valid, even though it leads to a somewhat undefined + * parse result. */ + GeoipStatsBuilder.createWithGeoipClientOriginsLine( + "geoip-client-origins de=1152,de=952,cn=896,us=712,it=504," + + "ru=352,fr=208,gb=208,ir=200"); + } @Test() public void testDirreqStatsValid() throws DescriptorParseException { - DirreqStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = DirreqStatsBuilder. + createWithDefaultLines(); + assertEquals(1328921993000L, descriptor.getDirreqStatsEndMillis()); + assertEquals(86400L, descriptor.getDirreqStatsIntervalLength()); + SortedMap<String, Integer> ips = descriptor.getDirreqV3Ips(); + assertNotNull(ips); + assertEquals(1544, ips.get("us").intValue()); + assertFalse(ips.containsKey("no")); + assertTrue(descriptor.getDirreqV2Ips().isEmpty()); + SortedMap<String, Integer> reqs = descriptor.getDirreqV3Reqs(); + assertEquals(832, reqs.get("fr").intValue()); + assertTrue(descriptor.getDirreqV2Reqs().isEmpty()); + SortedMap<String, Integer> resp = descriptor.getDirreqV3Resp(); + assertEquals(10848, resp.get("ok").intValue()); + assertEquals(8, resp.get("not-enough-sigs").intValue()); + resp = descriptor.getDirreqV2Resp(); + assertEquals(1576, resp.get("not-found").intValue()); + assertEquals(0.37, descriptor.getDirreqV2Share(), 0.0001); + assertEquals(0.37, descriptor.getDirreqV3Share(), 0.0001); + SortedMap<String, Integer> dl = descriptor.getDirreqV3DirectDl(); + assertEquals(36, dl.get("complete").intValue()); + dl = descriptor.getDirreqV2DirectDl(); + assertEquals(0, dl.get("timeout").intValue()); + dl = descriptor.getDirreqV3TunneledDl(); + assertEquals(10608, dl.get("complete").intValue()); + dl = descriptor.getDirreqV2TunneledDl(); + assertEquals(0, dl.get("complete").intValue()); + } + + @Test() + public void testDirreqStatsIntervalTwoDays() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqStatsEndLine("dirreq-stats-end " + + "2012-02-11 00:59:53 (172800 s)"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3IpsThreeLetterCountry() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3IpsLine("dirreq-v3-ips " + + "usa=1544"); + } + + @Test() + public void testDirreqV2IpsDigitCountry() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2IpsLine("dirreq-v2-ips 00=8"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3ReqsOneLetterCountry() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3ReqsLine("dirreq-v3-reqs " + + "u=1744"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV2ReqsNoNumber() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2ReqsLine("dirreq-v2-reqs us="); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3RespTwoEqualSigns() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3RespLine("dirreq-v3-resp " + + "ok==10848"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV2RespNull() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2RespLine("dirreq-v2-resp " + + "ok=null"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV2ShareComma() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2ShareLine("dirreq-v2-share " + + "0,37%"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3ShareNoPercent() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3ShareLine("dirreq-v3-share " + + "0.37"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3DirectDlSpace() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3DirectDlLine( + "dirreq-v3-direct-dl complete 36"); + } + + @Test() + public void testDirreqV2DirectDlNegative() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2DirectDlLine( + "dirreq-v2-direct-dl complete=-8"); + } + + @Test(expected = DescriptorParseException.class) + public void testDirreqV3TunneledDlTooLarge() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV3TunneledDlLine( + "dirreq-v3-tunneled-dl complete=2147483648"); } - /* TODO Add tests for invalid dirreq stats. */ + @Test(expected = DescriptorParseException.class) + public void testDirreqV3TunneledDlDouble() + throws DescriptorParseException { + DirreqStatsBuilder.createWithDirreqV2TunneledDlLine( + "dirreq-v2-tunneled-dl complete=0.001"); + } @Test() public void testEntryStatsValid() throws DescriptorParseException { - EntryStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = EntryStatsBuilder. + createWithDefaultLines(); + assertEquals(1328925579000L, descriptor.getEntryStatsEndMillis()); + assertEquals(86400L, descriptor.getEntryStatsIntervalLength()); + SortedMap<String, Integer> ips = descriptor.getEntryIps(); + assertNotNull(ips); + assertEquals(25368, ips.get("ir").intValue()); + assertFalse(ips.containsKey("no")); + } + + @Test(expected = DescriptorParseException.class) + public void testEntryStatsEndNoDate() throws DescriptorParseException { + EntryStatsBuilder.createWithEntryStatsEndLine("entry-stats-end " + + "01:59:39 (86400 s)"); } - /* TODO Add tests for invalid entry stats. */ + @Test(expected = DescriptorParseException.class) + public void testEntryStatsIpsSemicolon() + throws DescriptorParseException { + EntryStatsBuilder.createWithEntryIpsLine("entry-ips " + + "ir=25368;us=15744"); + } @Test() public void testCellStatsValid() throws DescriptorParseException { - CellStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = CellStatsBuilder. + createWithDefaultLines(); + assertEquals(1328925579000L, descriptor.getCellStatsEndMillis()); + assertEquals(86400L, descriptor.getCellStatsIntervalLength()); + List<Integer> processedCells = descriptor.getCellProcessedCells(); + assertEquals(10, processedCells.size()); + assertEquals(1441, processedCells.get(0).intValue()); + assertEquals(11, processedCells.get(1).intValue()); + List<Double> queuedCells = descriptor.getCellQueuedCells(); + assertEquals(10, queuedCells.size()); + assertEquals(3.29, queuedCells.get(0), 0.001); + assertEquals(0.00, queuedCells.get(1), 0.001); + List<Integer> timeInQueue = descriptor.getCellTimeInQueue(); + assertEquals(10, timeInQueue.size()); + assertEquals(524, timeInQueue.get(0).intValue()); + assertEquals(1, timeInQueue.get(1).intValue()); + assertEquals(866, descriptor.getCellCircuitsPerDecile()); } - /* TODO Add tests for invalid cell stats. */ + @Test(expected = DescriptorParseException.class) + public void testCellStatsEndNoSeconds() + throws DescriptorParseException { + CellStatsBuilder.createWithCellStatsEndLine("cell-stats-end " + + "2012-02-11 01:59:39 (86400)"); + } + + /* TODO Lines shouldn't be allowed to end with a comma. This also + * applies to all other comma-separated value lines. */ + @Test() + public void testCellProcessedCellsNineComma() + throws DescriptorParseException { + CellStatsBuilder.createWithCellProcessedCellsLine( + "cell-processed-cells 1441,11,6,4,2,1,1,1,1,"); + } + + /* TODO We should check that there are really ten values, not more or + * less. Applies to most cell-stats lines. */ + @Test() + public void testCellProcessedCellsEleven() + throws DescriptorParseException { + CellStatsBuilder.createWithCellQueuedCellsLine("cell-queued-cells " + + "3.29,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00"); + } + + @Test(expected = DescriptorParseException.class) + public void testCellTimeInQueueDouble() + throws DescriptorParseException { + CellStatsBuilder.createWithCellTimeInQueueLine("cell-time-in-queue " + + "524.0,1.0,1.0,0.0,0.0,25.0,0.0,0.0,0.0,0.0"); + } + + @Test(expected = DescriptorParseException.class) + public void testCellCircuitsPerDecileNegative() + throws DescriptorParseException { + CellStatsBuilder.createWithCellCircuitsPerDecileLine( + "cell-circuits-per-decile -866"); + } @Test() public void testConnBiDirectValid() throws DescriptorParseException { - DescriptorBuilder.createWithConnBiDirectLine("conn-bi-direct " - + "2012-02-11 01:59:39 (86400 s) 42173,1591,1310,1744"); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = DescriptorBuilder. + createWithConnBiDirectLine("conn-bi-direct 2012-02-11 01:59:39 " + + "(86400 s) 42173,1591,1310,1744"); + assertEquals(1328925579000L, + descriptor.getConnBiDirectStatsEndMillis()); + assertEquals(86400L, descriptor.getConnBiDirectStatsIntervalLength()); + assertEquals(42173, descriptor.getConnBiDirectBelow()); + assertEquals(1591, descriptor.getConnBiDirectRead()); + assertEquals(1310, descriptor.getConnBiDirectWrite()); + assertEquals(1744, descriptor.getConnBiDirectBoth()); } - /* TODO Add tests for invalid conn-bi-direct stats. */ + @Test(expected = DescriptorParseException.class) + public void testConnBiDirectStatsFive() + throws DescriptorParseException { + DescriptorBuilder.createWithConnBiDirectLine("conn-bi-direct " + + "2012-02-11 01:59:39 (86400 s) 42173,1591,1310,1744,42"); + } @Test() public void testExitStatsValid() throws DescriptorParseException { @@ -805,7 +1175,7 @@ public class ExtraInfoDescriptorImplTest { String[] ports = new String[] { "25", "80", "443", "49755", "52563", "52596", "57528", "60912", "61351", "64811", "other" }; int[] writtenValues = new int[] { 74647, 31370, 20577, 23, 12, 1111, - 4, 11, 6, 3365, 2592}; + 4, 11, 6, 3365, 2592 }; int i = 0; for (Map.Entry<String, Integer> e : descriptor.getExitKibibytesWritten().entrySet()) { @@ -830,16 +1200,62 @@ public class ExtraInfoDescriptorImplTest { } } - /* TODO Add tests for invalid exit stats. */ + @Test(expected = DescriptorParseException.class) + public void testExitStatsEndNoSeconds() + throws DescriptorParseException { + ExitStatsBuilder.createWithExitStatsEndLine("exit-stats-end " + + "2012-02-11 01:59 (86400 s)"); + } + + /* TODO Negative ports should not be allowed. */ + @Test() + public void testExitStatsWrittenNegativePort() + throws DescriptorParseException { + ExitStatsBuilder.createWithExitKibibytesWrittenLine( + "exit-kibibytes-written -25=74647"); + } + + /* TODO Negative bytes should not be allowed. */ + @Test() + public void testExitStatsReadNegativeBytes() + throws DescriptorParseException { + ExitStatsBuilder.createWithExitKibibytesReadLine( + "exit-kibibytes-read 25=-35562"); + } + + @Test(expected = DescriptorParseException.class) + public void testExitStatsStreamsTooLarge() + throws DescriptorParseException { + ExitStatsBuilder.createWithExitStreamsOpenedLine( + "exit-streams-opened 25=2147483648"); + } @Test() public void testBridgeStatsValid() throws DescriptorParseException { - BridgeStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = BridgeStatsBuilder. + createWithDefaultLines(); + assertEquals(1328925579000L, descriptor.getBridgeStatsEndMillis()); + assertEquals(86400L, descriptor.getBridgeStatsIntervalLength()); + SortedMap<String, Integer> ips = descriptor.getBridgeIps(); + assertNotNull(ips); + assertEquals(24, ips.get("ir").intValue()); + assertEquals(16, ips.get("sy").intValue()); + assertFalse(ips.containsKey("no")); } - /* TODO Add tests for invalid bridge stats. */ + /* TODO Only positive intervals should be allowed, for all stats. */ + @Test() + public void testBridgeStatsEndIntervalZero() + throws DescriptorParseException { + BridgeStatsBuilder.createWithBridgeStatsEndLine("bridge-stats-end " + + "2012-02-11 01:59:39 (0 s)"); + } + @Test(expected = DescriptorParseException.class) + public void testBridgeIpsDouble() + throws DescriptorParseException { + BridgeStatsBuilder.createWithBridgeIpsLine("bridge-ips ir=24.5"); + } @Test() public void testRouterSignatureOpt() throws DescriptorParseException {

1 0

[metrics-lib/master] Fix the problems found while writing unit tests.
by karsten＠torproject.org 26 Apr '12

26 Apr '12

commit a287fa0c290fbe70de4f52c3c6b54df6b85ae8d1 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Apr 26 11:54:17 2012 +0200 Fix the problems found while writing unit tests. --- .../descriptor/impl/BandwidthHistoryImpl.java | 8 +++ .../descriptor/impl/ExtraInfoDescriptorImpl.java | 58 ++++++++++++++++++++ .../torproject/descriptor/impl/ParseHelper.java | 6 +- .../impl/ExtraInfoDescriptorImplTest.java | 36 +++++------- 4 files changed, 84 insertions(+), 24 deletions(-) diff --git a/src/org/torproject/descriptor/impl/BandwidthHistoryImpl.java b/src/org/torproject/descriptor/impl/BandwidthHistoryImpl.java index c696624..56ae512 100644 --- a/src/org/torproject/descriptor/impl/BandwidthHistoryImpl.java +++ b/src/org/torproject/descriptor/impl/BandwidthHistoryImpl.java @@ -26,6 +26,10 @@ public class BandwidthHistoryImpl implements BandwidthHistory { partsNoOpt[4].equals("s)")) { this.intervalLength = Long.parseLong(partsNoOpt[3]. substring(1)); + if (this.intervalLength <= 0L) { + throw new DescriptorParseException("Only positive interval " + + "lengths are allowed in line '" + line + "'."); + } if (partsNoOpt.length > 6) { /* Invalid line, handle below. */ } else if (partsNoOpt.length == 5) { @@ -36,6 +40,10 @@ public class BandwidthHistoryImpl implements BandwidthHistory { String[] values = partsNoOpt[5].split(",", -1); for (int i = values.length - 1; i >= 0; i--) { long bandwidthValue = Long.parseLong(values[i]); + if (bandwidthValue < 0L) { + throw new DescriptorParseException("Negative bandwidth " + + "values are not allowed in line '" + line + "'."); + } this.bandwidthValues.put(endMillis, bandwidthValue); endMillis -= this.intervalLength * 1000L; } diff --git a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java index 3520934..184598a 100644 --- a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java +++ b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java @@ -5,6 +5,7 @@ package org.torproject.descriptor.impl; import java.io.UnsupportedEncodingException; import java.util.ArrayList; import java.util.Arrays; +import java.util.Collection; import java.util.Comparator; import java.util.HashSet; import java.util.List; @@ -259,6 +260,10 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl result[0] = ParseHelper.parseTimestampAtIndex(line, partsNoOpt, 1, 2); result[1] = ParseHelper.parseSeconds(line, partsNoOpt[3].substring(1)); + if (result[1] <= 0) { + throw new DescriptorParseException("Interval length must be " + + "positive in line '" + line + "'."); + } return result; } @@ -389,18 +394,30 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl String[] partsNoOpt) throws DescriptorParseException { this.cellProcessedCells = ParseHelper. parseCommaSeparatedIntegerValueList(line, partsNoOpt, 1); + if (this.cellProcessedCells.size() != 10) { + throw new DescriptorParseException("There must be exact ten values " + + "in line '" + line + "'."); + } } private void parseCellQueuedCellsLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { this.cellQueuedCells = ParseHelper.parseCommaSeparatedDoubleValueList( line, partsNoOpt, 1); + if (this.cellQueuedCells.size() != 10) { + throw new DescriptorParseException("There must be exact ten values " + + "in line '" + line + "'."); + } } private void parseCellTimeInQueueLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { this.cellTimeInQueue = ParseHelper. parseCommaSeparatedIntegerValueList(line, partsNoOpt, 1); + if (this.cellTimeInQueue.size() != 10) { + throw new DescriptorParseException("There must be exact ten values " + + "in line '" + line + "'."); + } } private void parseCellCircuitsPerDecileLine(String line, @@ -451,18 +468,24 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl throws DescriptorParseException { this.exitKibibytesWritten = this.sortByPorts(ParseHelper. parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); + this.verifyPorts(line, this.exitKibibytesWritten.keySet()); + this.verifyBytesOrStreams(line, this.exitKibibytesWritten.values()); } private void parseExitKibibytesReadLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { this.exitKibibytesRead = this.sortByPorts(ParseHelper. parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); + this.verifyPorts(line, this.exitKibibytesRead.keySet()); + this.verifyBytesOrStreams(line, this.exitKibibytesRead.values()); } private void parseExitStreamsOpenedLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { this.exitStreamsOpened = this.sortByPorts(ParseHelper. parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); + this.verifyPorts(line, this.exitStreamsOpened.keySet()); + this.verifyBytesOrStreams(line, this.exitStreamsOpened.values()); } private SortedMap<String, Integer> sortByPorts( @@ -493,6 +516,41 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl return byPortNumber; } + private void verifyPorts(String line, Set<String> ports) + throws DescriptorParseException { + boolean valid = true; + try { + for (String port : ports) { + if (!port.equals("other") && Integer.parseInt(port) <= 0) { + valid = false; + break; + } + } + } catch (NumberFormatException e) { + valid = false; + } + if (!valid) { + throw new DescriptorParseException("Invalid port in line '" + line + + "'."); + } + } + + private void verifyBytesOrStreams(String line, + Collection<Integer> bytesOrStreams) + throws DescriptorParseException { + boolean valid = true; + for (int s : bytesOrStreams) { + if (s < 0) { + valid = false; + break; + } + } + if (!valid) { + throw new DescriptorParseException("Invalid value in line '" + line + + "'."); + } + } + private void parseBridgeStatsEndLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { long[] parsedStatsEndData = this.parseStatsEndLine(line, partsNoOpt, diff --git a/src/org/torproject/descriptor/impl/ParseHelper.java b/src/org/torproject/descriptor/impl/ParseHelper.java index daea018..a3f907d 100644 --- a/src/org/torproject/descriptor/impl/ParseHelper.java +++ b/src/org/torproject/descriptor/impl/ParseHelper.java @@ -218,7 +218,7 @@ public class ParseHelper { + "unrecognized values beyond the expected key-value list at " + "index " + index + "."); } else if (partsNoOpt.length > index) { - String[] listElements = partsNoOpt[index].split(","); + String[] listElements = partsNoOpt[index].split(",", -1); for (String listElement : listElements) { String[] keyAndValue = listElement.split("="); String key = null; @@ -256,7 +256,7 @@ public class ParseHelper { + "unrecognized values beyond the expected comma-separated " + "value list at index " + index + "."); } else if (partsNoOpt.length > index) { - String[] listElements = partsNoOpt[index].split(","); + String[] listElements = partsNoOpt[index].split(",", -1); for (String listElement : listElements) { try { result.add(Integer.parseInt(listElement)); @@ -283,7 +283,7 @@ public class ParseHelper { + "unrecognized values beyond the expected comma-separated " + "value list at index " + index + "."); } else if (partsNoOpt.length > index) { - String[] listElements = partsNoOpt[index].split(","); + String[] listElements = partsNoOpt[index].split(",", -1); for (String listElement : listElements) { try { result.add(Double.parseDouble(listElement)); diff --git a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java index c21a4ec..c8df3ce 100644 --- a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java +++ b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java @@ -746,9 +746,7 @@ public class ExtraInfoDescriptorImplTest { assertEquals(1328951316000L, descriptor.getPublishedMillis()); } - /* TODO This test should fail, even though dir-spec.txt doesn't - * explicitly say that reported bytes must be non-negative. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testWriteHistoryNegativeBytes() throws DescriptorParseException { DescriptorBuilder.createWithWriteHistoryLine("write-history " @@ -772,9 +770,7 @@ public class ExtraInfoDescriptorImplTest { + "4707695616,4699666432,4650004480,4489718784"); } - /* TODO This test should fail, even though dir-spec.txt doesn't - * explicitly say that the interval must be positive. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testReadHistoryNegativeInterval() throws DescriptorParseException { DescriptorBuilder.createWithReadHistoryLine("read-history " @@ -918,9 +914,7 @@ public class ExtraInfoDescriptorImplTest { + "gb=208,ir=200"); } - /* TODO Lines shouldn't be allowed to end with a comma. This also - * applies to all other key-value pair lines. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testGeoipClientOriginsMissingEnd() throws DescriptorParseException { GeoipStatsBuilder.createWithGeoipClientOriginsLine( @@ -1112,18 +1106,14 @@ public class ExtraInfoDescriptorImplTest { + "2012-02-11 01:59:39 (86400)"); } - /* TODO Lines shouldn't be allowed to end with a comma. This also - * applies to all other comma-separated value lines. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testCellProcessedCellsNineComma() throws DescriptorParseException { CellStatsBuilder.createWithCellProcessedCellsLine( "cell-processed-cells 1441,11,6,4,2,1,1,1,1,"); } - /* TODO We should check that there are really ten values, not more or - * less. Applies to most cell-stats lines. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testCellProcessedCellsEleven() throws DescriptorParseException { CellStatsBuilder.createWithCellQueuedCellsLine("cell-queued-cells " @@ -1207,16 +1197,21 @@ public class ExtraInfoDescriptorImplTest { + "2012-02-11 01:59 (86400 s)"); } - /* TODO Negative ports should not be allowed. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testExitStatsWrittenNegativePort() throws DescriptorParseException { ExitStatsBuilder.createWithExitKibibytesWrittenLine( "exit-kibibytes-written -25=74647"); } - /* TODO Negative bytes should not be allowed. */ - @Test() + @Test(expected = DescriptorParseException.class) + public void testExitStatsWrittenUnknown() + throws DescriptorParseException { + ExitStatsBuilder.createWithExitKibibytesWrittenLine( + "exit-kibibytes-written unknown=74647"); + } + + @Test(expected = DescriptorParseException.class) public void testExitStatsReadNegativeBytes() throws DescriptorParseException { ExitStatsBuilder.createWithExitKibibytesReadLine( @@ -1243,8 +1238,7 @@ public class ExtraInfoDescriptorImplTest { assertFalse(ips.containsKey("no")); } - /* TODO Only positive intervals should be allowed, for all stats. */ - @Test() + @Test(expected = DescriptorParseException.class) public void testBridgeStatsEndIntervalZero() throws DescriptorParseException { BridgeStatsBuilder.createWithBridgeStatsEndLine("bridge-stats-end "

1 0

[metrics-lib/master] Order exit-stats numerically, not alphanumerically.
by karsten＠torproject.org 26 Apr '12

26 Apr '12

commit cd2fb43d1511568ee33a3b674bed79f84e397de4 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Apr 26 08:47:41 2012 +0200 Order exit-stats numerically, not alphanumerically. --- .../descriptor/impl/ExtraInfoDescriptorImpl.java | 45 +++++++++++++++---- .../impl/ExtraInfoDescriptorImplTest.java | 37 +++++++++++++--- 2 files changed, 66 insertions(+), 16 deletions(-) diff --git a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java index 763c9bf..3520934 100644 --- a/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java +++ b/src/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java @@ -5,6 +5,7 @@ package org.torproject.descriptor.impl; import java.io.UnsupportedEncodingException; import java.util.ArrayList; import java.util.Arrays; +import java.util.Comparator; import java.util.HashSet; import java.util.List; import java.util.Scanner; @@ -448,20 +449,48 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl private void parseExitKibibytesWrittenLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { - this.exitKibibytesWritten = ParseHelper. - parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0); + this.exitKibibytesWritten = this.sortByPorts(ParseHelper. + parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); } private void parseExitKibibytesReadLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { - this.exitKibibytesRead = ParseHelper.parseCommaSeparatedKeyValueList( - line, partsNoOpt, 1, 0); + this.exitKibibytesRead = this.sortByPorts(ParseHelper. + parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); } private void parseExitStreamsOpenedLine(String line, String lineNoOpt, String[] partsNoOpt) throws DescriptorParseException { - this.exitStreamsOpened = ParseHelper.parseCommaSeparatedKeyValueList( - line, partsNoOpt, 1, 0); + this.exitStreamsOpened = this.sortByPorts(ParseHelper. + parseCommaSeparatedKeyValueList(line, partsNoOpt, 1, 0)); + } + + private SortedMap<String, Integer> sortByPorts( + SortedMap<String, Integer> naturalOrder) { + SortedMap<String, Integer> byPortNumber = + new TreeMap<String, Integer>(new Comparator<String>() { + public int compare(String arg0, String arg1) { + int port0 = 0, port1 = 0; + try { + port1 = Integer.parseInt(arg1); + } catch (NumberFormatException e) { + return -1; + } + try { + port0 = Integer.parseInt(arg0); + } catch (NumberFormatException e) { + return 1; + } + if (port0 < port1) { + return -1; + } else if (port0 > port1) { + return 1; + } else { + return 0; + } + }}); + byPortNumber.putAll(naturalOrder); + return byPortNumber; } private void parseBridgeStatsEndLine(String line, String lineNoOpt, @@ -722,10 +751,6 @@ public class ExtraInfoDescriptorImpl extends DescriptorImpl return this.exitStatsIntervalLength; } - /* TODO Add custom comparators to the maps returned by all three - * exit-stats methods to sort keys alphanumerically, not - * alphabetically. */ - private SortedMap<String, Integer> exitKibibytesWritten; public SortedMap<String, Integer> getExitKibibytesWritten() { return this.exitKibibytesWritten == null ? null : diff --git a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java index 1eab3b9..ae1b87a 100644 --- a/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java +++ b/test/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java @@ -3,16 +3,13 @@ package org.torproject.descriptor.impl; import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; import java.util.ArrayList; import java.util.List; +import java.util.Map; import org.junit.Test; -import org.torproject.descriptor.BandwidthHistory; import org.torproject.descriptor.ExtraInfoDescriptor; /* Test parsing of extra-info descriptors. */ @@ -801,8 +798,36 @@ public class ExtraInfoDescriptorImplTest { @Test() public void testExitStatsValid() throws DescriptorParseException { - ExitStatsBuilder.createWithDefaultLines(); - /* TODO Check stats parts. */ + ExtraInfoDescriptor descriptor = ExitStatsBuilder. + createWithDefaultLines(); + assertEquals(1328925579000L, descriptor.getExitStatsEndMillis()); + assertEquals(86400L, descriptor.getExitStatsIntervalLength()); + String[] ports = new String[] { "25", "80", "443", "49755", + "52563", "52596", "57528", "60912", "61351", "64811", "other" }; + int[] writtenValues = new int[] { 74647, 31370, 20577, 23, 12, 1111, + 4, 11, 6, 3365, 2592}; + int i = 0; + for (Map.Entry<String, Integer> e : + descriptor.getExitKibibytesWritten().entrySet()) { + assertEquals(ports[i], e.getKey()); + assertEquals(writtenValues[i++], e.getValue().intValue()); + } + int[] readValues = new int[] { 35562, 1254256, 110279, 9396, 1911, + 648, 1188, 1427, 1824, 14, 3054 }; + i = 0; + for (Map.Entry<String, Integer> e : + descriptor.getExitKibibytesRead().entrySet()) { + assertEquals(ports[i], e.getKey()); + assertEquals(readValues[i++], e.getValue().intValue()); + } + int[] streamsValues = new int[] { 369748, 64212, 151660, 4, 4, 4, 4, + 4, 4, 4, 1212 }; + i = 0; + for (Map.Entry<String, Integer> e : + descriptor.getExitStreamsOpened().entrySet()) { + assertEquals(ports[i], e.getKey()); + assertEquals(streamsValues[i++], e.getValue().intValue()); + } } /* TODO Add tests for invalid exit stats. */

1 0

[metrics-lib/master] Parse certs and everything we need to verify consensuses.
by karsten＠torproject.org 26 Apr '12

26 Apr '12

commit 47e771f2b772f21563d38a98bb7c7fe284b474d8 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Apr 26 16:07:55 2012 +0200 Parse certs and everything we need to verify consensuses. --- .../descriptor/DirectoryKeyCertificate.java | 45 ++++ .../torproject/descriptor/DirectorySignature.java | 16 ++ .../descriptor/RelayNetworkStatusConsensus.java | 6 +- .../descriptor/RelayNetworkStatusVote.java | 2 +- .../torproject/descriptor/impl/DescriptorImpl.java | 4 +- .../impl/DirectoryKeyCertificateImpl.java | 270 ++++++++++++++++++++ .../descriptor/impl/DirectorySignatureImpl.java | 91 +++++++ .../descriptor/impl/NetworkStatusImpl.java | 50 +--- .../impl/RelayNetworkStatusConsensusImpl.java | 33 +++ 9 files changed, 479 insertions(+), 38 deletions(-) diff --git a/src/org/torproject/descriptor/DirectoryKeyCertificate.java b/src/org/torproject/descriptor/DirectoryKeyCertificate.java new file mode 100644 index 0000000..9a4aeae --- /dev/null +++ b/src/org/torproject/descriptor/DirectoryKeyCertificate.java @@ -0,0 +1,45 @@ +/* Copyright 2012 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.descriptor; + +public interface DirectoryKeyCertificate extends Descriptor { + + /* Return the directory key certificate version. */ + public int getDirKeyCertificateVersion(); + + /* Return the IP address, or null if the certificate does not contain an + * address. */ + public String getAddress(); + + /* Return the directory port, or -1 if the certificate does not contain + * one. */ + public int getPort(); + + /* Return the directory identity fingerprint. */ + public String getFingerprint(); + + /* Return the directory identity key. */ + public String getDirIdentityKey(); + + /* Return the directory key certificate publication timestamp. */ + public long getDirKeyPublishedMillis(); + + /* Return the directory key certificate expiration timestamp. */ + public long getDirKeyExpiresMillis(); + + /* Return the directory signing key digest. */ + public String getDirSigningKey(); + + /* Return the signature of the directory identity key made using the + * directory signing key, or null if the certificate does not contain + * this signature. */ + public String getDirKeyCrosscert(); + + /* Return the certificate signature made using the directory identity + * key. */ + public String getDirKeyCertification(); + + /* Return the calculated certificate digest. */ + public String getCertificateDigest(); +} + diff --git a/src/org/torproject/descriptor/DirectorySignature.java b/src/org/torproject/descriptor/DirectorySignature.java new file mode 100644 index 0000000..29eb055 --- /dev/null +++ b/src/org/torproject/descriptor/DirectorySignature.java @@ -0,0 +1,16 @@ +/* Copyright 2012 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.descriptor; + +public interface DirectorySignature { + + /* Return the directory identity fingerprint. */ + public String getIdentity(); + + /* Return the directory signing key digest. */ + public String getSigningKeyDigest(); + + /* Return the directory signature made using the signing key. */ + public String getSignature(); +} + diff --git a/src/org/torproject/descriptor/RelayNetworkStatusConsensus.java b/src/org/torproject/descriptor/RelayNetworkStatusConsensus.java index 2c0ff4f..4a1634f 100644 --- a/src/org/torproject/descriptor/RelayNetworkStatusConsensus.java +++ b/src/org/torproject/descriptor/RelayNetworkStatusConsensus.java @@ -60,10 +60,14 @@ public interface RelayNetworkStatusConsensus extends Descriptor { public NetworkStatusEntry getStatusEntry(String fingerprint); /* Return directory signatures. */ - public SortedMap<String, String> getDirectorySignatures(); + public SortedMap<String, DirectorySignature> getDirectorySignatures(); /* Return bandwidth weights or null if the consensus doesn't contain * bandwidth weights. */ public SortedMap<String, Integer> getBandwidthWeights(); + + /* Return the consensus digest that directory authorities use to sign + * the consensus. */ + public String getConsensusDigest(); } diff --git a/src/org/torproject/descriptor/RelayNetworkStatusVote.java b/src/org/torproject/descriptor/RelayNetworkStatusVote.java index f9635e2..eda0cef 100644 --- a/src/org/torproject/descriptor/RelayNetworkStatusVote.java +++ b/src/org/torproject/descriptor/RelayNetworkStatusVote.java @@ -92,6 +92,6 @@ public interface RelayNetworkStatusVote extends Descriptor { public NetworkStatusEntry getStatusEntry(String fingerprint); /* Return directory signatures. */ - public SortedMap<String, String> getDirectorySignatures(); + public SortedMap<String, DirectorySignature> getDirectorySignatures(); } diff --git a/src/org/torproject/descriptor/impl/DescriptorImpl.java b/src/org/torproject/descriptor/impl/DescriptorImpl.java index 33b94e9..6b1b167 100644 --- a/src/org/torproject/descriptor/impl/DescriptorImpl.java +++ b/src/org/torproject/descriptor/impl/DescriptorImpl.java @@ -60,7 +60,9 @@ public abstract class DescriptorImpl implements Descriptor { parseDescriptors(rawDescriptorBytes, failUnrecognizedDescriptorLines)); } else if (firstLines.startsWith("dir-key-certificate-version ")) { - /* TODO Implement parsing of directory certificates. */ + parsedDescriptors.addAll(DirectoryKeyCertificateImpl. + parseDescriptors(rawDescriptorBytes, + failUnrecognizedDescriptorLines)); } else if (firstLines.startsWith("ExitNode ")) { parsedDescriptors.add(new ExitListImpl(rawDescriptorBytes, fileName, failUnrecognizedDescriptorLines)); diff --git a/src/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java b/src/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java new file mode 100644 index 0000000..2483aa1 --- /dev/null +++ b/src/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java @@ -0,0 +1,270 @@ +/* Copyright 2012 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.descriptor.impl; + +import java.io.UnsupportedEncodingException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashSet; +import java.util.List; +import java.util.Scanner; +import java.util.Set; + +import org.apache.commons.codec.digest.DigestUtils; +import org.torproject.descriptor.DirectoryKeyCertificate; + +/* TODO Add test class. */ + +public class DirectoryKeyCertificateImpl extends DescriptorImpl + implements DirectoryKeyCertificate { + + protected static List<DirectoryKeyCertificate> parseDescriptors( + byte[] descriptorsBytes, boolean failUnrecognizedDescriptorLines) + throws DescriptorParseException { + List<DirectoryKeyCertificate> parsedDescriptors = + new ArrayList<DirectoryKeyCertificate>(); + List<byte[]> splitDescriptorsBytes = + DirectoryKeyCertificateImpl.splitRawDescriptorBytes( + descriptorsBytes, "dir-key-certificate-version "); + for (byte[] descriptorBytes : splitDescriptorsBytes) { + DirectoryKeyCertificate parsedDescriptor = + new DirectoryKeyCertificateImpl(descriptorBytes, + failUnrecognizedDescriptorLines); + parsedDescriptors.add(parsedDescriptor); + } + return parsedDescriptors; + } + + protected DirectoryKeyCertificateImpl(byte[] rawDescriptorBytes, + boolean failUnrecognizedDescriptorLines) + throws DescriptorParseException { + super(rawDescriptorBytes, failUnrecognizedDescriptorLines); + this.parseDescriptorBytes(); + this.calculateDigest(); + Set<String> exactlyOnceKeywords = new HashSet<String>(Arrays.asList(( + "dir-key-certificate-version,fingerprint,dir-identity-key," + + "dir-key-published,dir-key-expires,dir-signing-key," + + "dir-key-certification").split(","))); + this.checkExactlyOnceKeywords(exactlyOnceKeywords); + Set<String> atMostOnceKeywords = new HashSet<String>(Arrays.asList(( + "dir-address,dir-key-crosscert").split(","))); + this.checkAtMostOnceKeywords(atMostOnceKeywords); + this.checkFirstKeyword("dir-key-certificate-version"); + this.checkLastKeyword("dir-key-certification"); + } + + private void parseDescriptorBytes() throws DescriptorParseException { + Scanner s = new Scanner(new String(this.rawDescriptorBytes)). + useDelimiter("\n"); + String nextCrypto = null; + StringBuilder crypto = null; + while (s.hasNext()) { + String line = s.next(); + String[] parts = line.split(" "); + String keyword = parts[0]; + if (keyword.equals("dir-key-certificate-version")) { + this.parseDirKeyCertificateVersionLine(line, parts); + } else if (keyword.equals("dir-address")) { + this.parseDirAddressLine(line, parts); + } else if (keyword.equals("fingerprint")) { + this.parseFingerprintLine(line, parts); + } else if (keyword.equals("dir-identity-key")) { + this.parseDirIdentityKeyLine(line, parts); + nextCrypto = "dir-identity-key"; + } else if (keyword.equals("dir-key-published")) { + this.parseDirKeyPublishedLine(line, parts); + } else if (keyword.equals("dir-key-expires")) { + this.parseDirKeyExpiresLine(line, parts); + } else if (keyword.equals("dir-signing-key")) { + this.parseDirSigningKeyLine(line, parts); + nextCrypto = "dir-signing-key"; + } else if (keyword.equals("dir-key-crosscert")) { + this.parseDirKeyCrosscertLine(line, parts); + nextCrypto = "dir-key-crosscert"; + } else if (keyword.equals("dir-key-certification")) { + this.parseDirKeyCertificationLine(line, parts); + nextCrypto = "dir-key-certification"; + } else if (line.startsWith("-----BEGIN")) { + crypto = new StringBuilder(); + crypto.append(line + "\n"); + } else if (line.startsWith("-----END")) { + crypto.append(line + "\n"); + String cryptoString = crypto.toString(); + crypto = null; + if (nextCrypto.equals("dir-identity-key")) { + this.dirIdentityKey = cryptoString; + } else if (nextCrypto.equals("dir-signing-key")) { + this.dirSigningKey = cryptoString; + } else if (nextCrypto.equals("dir-key-crosscert")) { + this.dirKeyCrosscert = cryptoString; + } else if (nextCrypto.equals("dir-key-certification")) { + this.dirKeyCertification = cryptoString; + } else { + throw new DescriptorParseException("Unrecognized crypto " + + "block in directory key certificate."); + } + nextCrypto = null; + } else if (crypto != null) { + crypto.append(line + "\n"); + } else { + if (this.failUnrecognizedDescriptorLines) { + throw new DescriptorParseException("Unrecognized line '" + + line + "' in directory key certificate."); + } else { + if (this.unrecognizedLines == null) { + this.unrecognizedLines = new ArrayList<String>(); + } + this.unrecognizedLines.add(line); + } + } + } + } + + private void parseDirKeyCertificateVersionLine(String line, + String[] parts) throws DescriptorParseException { + if (!line.equals("dir-key-certificate-version 3")) { + throw new DescriptorParseException("Illegal directory key " + + "certificate version number in line '" + line + "'."); + } + this.dirKeyCertificateVersion = 3; + } + + private void parseDirAddressLine(String line, String[] parts) + throws DescriptorParseException { + if (parts.length != 2 || parts[1].split(":").length != 2) { + throw new DescriptorParseException("Illegal line '" + line + + "' in directory key certificate."); + } + this.address = ParseHelper.parseIpv4Address(line, + parts[1].split(":")[0]); + this.port = ParseHelper.parsePort(line, parts[1].split(":")[1]); + } + + private void parseFingerprintLine(String line, String[] parts) + throws DescriptorParseException { + if (parts.length != 2) { + throw new DescriptorParseException("Illegal line '" + line + + "' in directory key certificate."); + } + this.fingerprint = ParseHelper.parseTwentyByteHexString(line, + parts[1]); + } + + private void parseDirIdentityKeyLine(String line, String[] parts) + throws DescriptorParseException { + if (!line.equals("dir-identity-key")) { + throw new DescriptorParseException("Illegal line '" + line + "'."); + } + } + + private void parseDirKeyPublishedLine(String line, String[] parts) + throws DescriptorParseException { + this.dirKeyPublishedMillis = ParseHelper.parseTimestampAtIndex(line, + parts, 1, 2); + } + + private void parseDirKeyExpiresLine(String line, String[] parts) + throws DescriptorParseException { + this.dirKeyExpiresMillis = ParseHelper.parseTimestampAtIndex(line, + parts, 1, 2); + } + + private void parseDirSigningKeyLine(String line, String[] parts) + throws DescriptorParseException { + if (!line.equals("dir-signing-key")) { + throw new DescriptorParseException("Illegal line '" + line + "'."); + } + } + + private void parseDirKeyCrosscertLine(String line, String[] parts) + throws DescriptorParseException { + if (!line.equals("dir-key-crosscert")) { + throw new DescriptorParseException("Illegal line '" + line + "'."); + } + } + + private void parseDirKeyCertificationLine(String line, String[] parts) + throws DescriptorParseException { + if (!line.equals("dir-key-certification")) { + throw new DescriptorParseException("Illegal line '" + line + "'."); + } + } + + private void calculateDigest() throws DescriptorParseException { + try { + String ascii = new String(this.getRawDescriptorBytes(), "US-ASCII"); + String startToken = "dir-key-certificate-version "; + String sigToken = "\ndir-key-certification\n"; + int start = ascii.indexOf(startToken); + int sig = ascii.indexOf(sigToken) + sigToken.length(); + if (start >= 0 && sig >= 0 && sig > start) { + byte[] forDigest = new byte[sig - start]; + System.arraycopy(this.getRawDescriptorBytes(), start, + forDigest, 0, sig - start); + this.certificateDigest = DigestUtils.shaHex(forDigest); + } + } catch (UnsupportedEncodingException e) { + /* Handle below. */ + } + if (this.certificateDigest == null) { + throw new DescriptorParseException("Could not calculate " + + "certificate digest."); + } + } + + private int dirKeyCertificateVersion; + public int getDirKeyCertificateVersion() { + return this.dirKeyCertificateVersion; + } + + private String address; + public String getAddress() { + return this.address; + } + + private int port = -1; + public int getPort() { + return this.port; + } + + private String fingerprint; + public String getFingerprint() { + return this.fingerprint; + } + + private String dirIdentityKey; + public String getDirIdentityKey() { + return this.dirIdentityKey; + } + + private long dirKeyPublishedMillis; + public long getDirKeyPublishedMillis() { + return this.dirKeyPublishedMillis; + } + + private long dirKeyExpiresMillis; + public long getDirKeyExpiresMillis() { + return this.dirKeyExpiresMillis; + } + + private String dirSigningKey; + public String getDirSigningKey() { + return this.dirSigningKey; + } + + private String dirKeyCrosscert; + public String getDirKeyCrosscert() { + return this.dirKeyCrosscert; + } + + private String dirKeyCertification; + public String getDirKeyCertification() { + return this.dirKeyCertification; + } + + private String certificateDigest; + public String getCertificateDigest() { + return this.certificateDigest; + } +} + diff --git a/src/org/torproject/descriptor/impl/DirectorySignatureImpl.java b/src/org/torproject/descriptor/impl/DirectorySignatureImpl.java new file mode 100644 index 0000000..d205345 --- /dev/null +++ b/src/org/torproject/descriptor/impl/DirectorySignatureImpl.java @@ -0,0 +1,91 @@ +/* Copyright 2012 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.descriptor.impl; + +import java.util.ArrayList; +import java.util.List; +import java.util.Scanner; + +import org.torproject.descriptor.DirectorySignature; + +public class DirectorySignatureImpl implements DirectorySignature { + + private byte[] directorySignatureBytes; + public byte[] getDirectorySignatureBytes() { + return this.directorySignatureBytes; + } + + private boolean failUnrecognizedDescriptorLines; + private List<String> unrecognizedLines; + protected List<String> getAndClearUnrecognizedLines() { + List<String> lines = this.unrecognizedLines; + this.unrecognizedLines = null; + return lines; + } + + protected DirectorySignatureImpl(byte[] directorySignatureBytes, + boolean failUnrecognizedDescriptorLines) + throws DescriptorParseException { + this.directorySignatureBytes = directorySignatureBytes; + this.failUnrecognizedDescriptorLines = + failUnrecognizedDescriptorLines; + this.parseDirectorySignatureBytes(); + } + + private void parseDirectorySignatureBytes() + throws DescriptorParseException { + Scanner s = new Scanner(new String(this.directorySignatureBytes)). + useDelimiter("\n"); + StringBuilder crypto = null; + while (s.hasNext()) { + String line = s.next(); + if (line.startsWith("directory-signature ")) { + String[] parts = line.split(" ", -1); + if (parts.length != 3) { + throw new DescriptorParseException("Illegal line '" + line + + "'."); + } + this.identity = ParseHelper.parseTwentyByteHexString(line, + parts[1]); + this.signingKeyDigest = ParseHelper.parseTwentyByteHexString( + line, parts[2]); + } else if (line.startsWith("-----BEGIN")) { + crypto = new StringBuilder(); + crypto.append(line + "\n"); + } else if (line.startsWith("-----END")) { + crypto.append(line + "\n"); + String cryptoString = crypto.toString(); + crypto = null; + this.signature = cryptoString; + } else if (crypto != null) { + crypto.append(line + "\n"); + } else { + if (this.failUnrecognizedDescriptorLines) { + throw new DescriptorParseException("Unrecognized line '" + + line + "' in dir-source entry."); + } else { + if (this.unrecognizedLines == null) { + this.unrecognizedLines = new ArrayList<String>(); + } + this.unrecognizedLines.add(line); + } + } + } + } + + private String identity; + public String getIdentity() { + return this.identity; + } + + private String signingKeyDigest; + public String getSigningKeyDigest() { + return this.signingKeyDigest; + } + + private String signature; + public String getSignature() { + return this.signature; + } +} + diff --git a/src/org/torproject/descriptor/impl/NetworkStatusImpl.java b/src/org/torproject/descriptor/impl/NetworkStatusImpl.java index f080171..d27e651 100644 --- a/src/org/torproject/descriptor/impl/NetworkStatusImpl.java +++ b/src/org/torproject/descriptor/impl/NetworkStatusImpl.java @@ -4,11 +4,11 @@ package org.torproject.descriptor.impl; import java.util.ArrayList; import java.util.List; -import java.util.Scanner; import java.util.SortedMap; import java.util.TreeMap; import org.torproject.descriptor.DirSourceEntry; +import org.torproject.descriptor.DirectorySignature; import org.torproject.descriptor.NetworkStatusEntry; /* Parse the common parts of v3 consensuses, v3 votes, v3 microdesc @@ -195,39 +195,19 @@ public abstract class NetworkStatusImpl extends DescriptorImpl { protected void parseDirectorySignature(byte[] directorySignatureBytes) throws DescriptorParseException { if (this.directorySignatures == null) { - this.directorySignatures = new TreeMap<String, String>(); + this.directorySignatures = new TreeMap<String, + DirectorySignature>(); } - Scanner s = new Scanner(new String(directorySignatureBytes)). - useDelimiter("\n"); - boolean skipCrypto = false; - while (s.hasNext()) { - String line = s.next(); - if (line.startsWith("directory-signature ")) { - String[] parts = line.split(" ", -1); - if (parts.length != 3) { - throw new DescriptorParseException("Illegal line '" + line - + "'."); - } - String identity = ParseHelper.parseTwentyByteHexString(line, - parts[1]); - String signingKeyDigest = ParseHelper.parseTwentyByteHexString( - line, parts[2]); - this.directorySignatures.put(identity, signingKeyDigest); - } else if (line.startsWith("-----BEGIN")) { - skipCrypto = true; - } else if (line.startsWith("-----END")) { - skipCrypto = false; - } else if (!skipCrypto) { - if (this.failUnrecognizedDescriptorLines) { - throw new DescriptorParseException("Unrecognized line '" - + line + "' in dir-source entry."); - } else { - if (this.unrecognizedLines == null) { - this.unrecognizedLines = new ArrayList<String>(); - } - this.unrecognizedLines.add(line); - } + DirectorySignatureImpl signature = new DirectorySignatureImpl( + directorySignatureBytes, failUnrecognizedDescriptorLines); + this.directorySignatures.put(signature.getIdentity(), signature); + List<String> unrecognizedStatusEntryLines = signature. + getAndClearUnrecognizedLines(); + if (unrecognizedStatusEntryLines != null) { + if (this.unrecognizedLines == null) { + this.unrecognizedLines = new ArrayList<String>(); } + this.unrecognizedLines.addAll(unrecognizedStatusEntryLines); } } @@ -249,10 +229,10 @@ public abstract class NetworkStatusImpl extends DescriptorImpl { return this.statusEntries.get(fingerprint); } - private SortedMap<String, String> directorySignatures; - public SortedMap<String, String> getDirectorySignatures() { + private SortedMap<String, DirectorySignature> directorySignatures; + public SortedMap<String, DirectorySignature> getDirectorySignatures() { return this.directorySignatures == null ? null : - new TreeMap<String, String>(this.directorySignatures); + new TreeMap<String, DirectorySignature>(this.directorySignatures); } } diff --git a/src/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java b/src/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java index 564beb2..3d2af37 100644 --- a/src/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java +++ b/src/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java @@ -2,6 +2,7 @@ * See LICENSE for licensing information */ package org.torproject.descriptor.impl; +import java.io.UnsupportedEncodingException; import java.util.ArrayList; import java.util.Arrays; import java.util.HashSet; @@ -13,6 +14,7 @@ import java.util.SortedSet; import java.util.TreeMap; import java.util.TreeSet; +import org.apache.commons.codec.digest.DigestUtils; import org.torproject.descriptor.RelayNetworkStatusConsensus; /* Contains a network status consensus. */ @@ -49,6 +51,32 @@ public class RelayNetworkStatusConsensusImpl extends NetworkStatusImpl + "bandwidth-weights").split(","))); this.checkAtMostOnceKeywords(atMostOnceKeywords); this.checkFirstKeyword("network-status-version"); + this.calculateDigest(); + } + + private void calculateDigest() throws DescriptorParseException { + try { + String ascii = new String(this.getRawDescriptorBytes(), "US-ASCII"); + String startToken = "network-status-version "; + String sigToken = "\ndirectory-signature "; + if (!ascii.contains(sigToken)) { + return; + } + int start = ascii.indexOf(startToken); + int sig = ascii.indexOf(sigToken) + sigToken.length(); + if (start >= 0 && sig >= 0 && sig > start) { + byte[] forDigest = new byte[sig - start]; + System.arraycopy(this.getRawDescriptorBytes(), start, + forDigest, 0, sig - start); + this.consensusDigest = DigestUtils.shaHex(forDigest); + } + } catch (UnsupportedEncodingException e) { + /* Handle below. */ + } + if (this.consensusDigest == null) { + throw new DescriptorParseException("Could not calculate consensus " + + "digest."); + } } protected void parseHeader(byte[] headerBytes) @@ -238,6 +266,11 @@ public class RelayNetworkStatusConsensusImpl extends NetworkStatusImpl 1); } + private String consensusDigest; + public String getConsensusDigest() { + return this.consensusDigest; + } + private int networkStatusVersion; public int getNetworkStatusVersion() { return this.networkStatusVersion;

1 0

[torbrowser/maint-2.3] Merge branch 'maint-2.2' into maint-2.3
by erinn＠torproject.org 26 Apr '12

26 Apr '12

commit 92f635537c9c95174346d2552b5f81e6a2151199 Merge: f81c8eb 77ba44e Author: Erinn Clark <erinn(a)torproject.org> Date: Thu Apr 26 10:04:55 2012 -0300 Merge branch 'maint-2.2' into maint-2.3 build-scripts/linux.mk | 5 ---- build-scripts/osx.mk | 9 +----- build-scripts/versions.mk | 9 ++++++- build-scripts/windows.mk | 7 +---- .../openssl/0002-Fix-openssl-1.0.1a-compile.patch | 26 -------------------- 5 files changed, 11 insertions(+), 45 deletions(-)

1 0

[torbrowser/maint-2.3] Openssl 1.0.1b released, fixes 1.0.1a compile on osx
by erinn＠torproject.org 26 Apr '12

26 Apr '12

commit 77ba44ef96d1005810c38a80f89aae78b419925a Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Thu Apr 26 14:59:20 2012 +0200 Openssl 1.0.1b released, fixes 1.0.1a compile on osx That means we can get rid of the patch again... --- build-scripts/versions.mk | 2 +- .../openssl/0002-Fix-openssl-1.0.1a-compile.patch | 26 -------------------- 2 files changed, 1 insertions(+), 27 deletions(-) diff --git a/build-scripts/versions.mk b/build-scripts/versions.mk index fdc67c0..9788cfb 100644 --- a/build-scripts/versions.mk +++ b/build-scripts/versions.mk @@ -3,7 +3,7 @@ RELEASE_VER=2.2.35 ZLIB_VER=1.2.6 -OPENSSL_VER=1.0.1a +OPENSSL_VER=1.0.1b LIBPNG_VER=1.5.10 QT_VER=4.8.1 VIDALIA_VER=0.2.17 diff --git a/src/current-patches/openssl/0002-Fix-openssl-1.0.1a-compile.patch b/src/current-patches/openssl/0002-Fix-openssl-1.0.1a-compile.patch deleted file mode 100644 index 3fbd39c..0000000 --- a/src/current-patches/openssl/0002-Fix-openssl-1.0.1a-compile.patch +++ /dev/null @@ -1,26 +0,0 @@ -From 24d73c226f53c01d1d8400cc4fdad23f5a45412b Mon Sep 17 00:00:00 2001 -From: Sebastian Hahn <sebastian(a)torproject.org> -Date: Wed, 25 Apr 2012 23:58:47 +0200 -Subject: [PATCH] Fix openssl 1.0.1a compile - ---- - crypto/evp/e_rc4_hmac_md5.c | 3 ++- - 1 file changed, 2 insertions(+), 1 deletion(-) - -diff --git a/crypto/evp/e_rc4_hmac_md5.c b/crypto/evp/e_rc4_hmac_md5.c -index 3f32b25..e65380d 100644 ---- a/crypto/evp/e_rc4_hmac_md5.c -+++ b/crypto/evp/e_rc4_hmac_md5.c -@@ -103,7 +103,8 @@ static int rc4_hmac_md5_init_key(EVP_CIPHER_CTX *ctx, - #if !defined(OPENSSL_NO_ASM) && ( \ - defined(__x86_64) || defined(__x86_64__) || \ - defined(_M_AMD64) || defined(_M_X64) || \ -- defined(__INTEL__) ) -+ defined(__INTEL__) ) && \ -+ !(defined(__APPLE__) && defined(__MACH__)) - #define STITCHED_CALL - #endif - --- -1.7.10 -

1 0