April 2018 - tor-commits - lists.torproject.org

[translation/bridgedb] Update translations for bridgedb
by translation＠torproject.org 18 Apr '18

18 Apr '18

commit a4f1ffe284b3a2dc729b7b8e88a9ea92e8c21773 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 18 16:15:04 2018 +0000 Update translations for bridgedb --- fr/LC_MESSAGES/bridgedb.po | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fr/LC_MESSAGES/bridgedb.po b/fr/LC_MESSAGES/bridgedb.po index de85853fd..fe7dd4f89 100644 --- a/fr/LC_MESSAGES/bridgedb.po +++ b/fr/LC_MESSAGES/bridgedb.po @@ -29,7 +29,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: 'https://trac.torproject.org/projects/tor/newticket?component=BridgeDB&keywo…'\n" "POT-Creation-Date: 2015-07-25 03:40+0000\n" -"PO-Revision-Date: 2018-04-12 20:34+0000\n" +"PO-Revision-Date: 2018-04-18 15:58+0000\n" "Last-Translator: French language coordinator <french.coordinator(a)rbox.me>\n" "Language-Team: French (http://www.transifex.com/otf/torproject/language/fr/)\n" "MIME-Version: 1.0\n" @@ -120,7 +120,7 @@ msgstr "Étape %s1%s" #: bridgedb/https/templates/index.html:13 #, python-format msgid "Download %s Tor Browser %s" -msgstr "Télécharger %s le navigateur Tor %s" +msgstr "Télécharger %s le Navigateur Tor %s" #: bridgedb/https/templates/index.html:25 #, python-format @@ -140,7 +140,7 @@ msgstr "Étape %s3%s" #: bridgedb/https/templates/index.html:38 #, python-format msgid "Now %s add the bridges to Tor Browser %s" -msgstr "Maintenant, %s ajouter les ponts au navigateur Tor %s" +msgstr "Maintenant, %s ajouter les ponts au Navigateur Tor %s" #. TRANSLATORS: Please make sure the '%s' surrounding single letters at the #. beginning of words are present in your final translation. Thanks! @@ -307,7 +307,7 @@ msgid "" "Try including as much info about your case as you can, including the list of\n" "bridges and Pluggable Transports you tried to use, your Tor Browser version,\n" "and any messages which Tor gave out, etc." -msgstr "Essayez d’inclure autant de renseignements sur votre cas que possible, incluant la liste de ponts et de transports enfichables que vous avez essayé d’utiliser, votre version du navigateur Tor et tout message donné par Tor, etc." +msgstr "Essayez d’inclure autant de renseignements sur votre cas que possible, incluant la liste de ponts et de transports enfichables que vous avez essayé d’utiliser, votre version du Navigateur Tor et tout message donné par Tor, etc." #: bridgedb/strings.py:128 msgid "Here are your bridge lines:" @@ -349,7 +349,7 @@ msgid "" "To enter bridges into Tor Browser, first go to the %s Tor Browser download\n" "page %s and then follow the instructions there for downloading and starting\n" "Tor Browser." -msgstr "Pour saisir des ponts dans le navigateur Tor, rendez-vous d’abord à la %s page de téléchargement du navigateur Tor %s et suivez les instructions que vous y trouverez pour télécharger et démarrer le navigateur Tor." +msgstr "Pour saisir des ponts dans le Navigateur Tor, rendez-vous d’abord à la %s page de téléchargement du Navigateur Tor %s et suivez les instructions que vous y trouverez pour télécharger et démarrer le Navigateur Tor." #. TRANSLATORS: Please DO NOT translate "Tor". #: bridgedb/strings.py:151

1 0

[translation/support-miscellaneous] Update translations for support-miscellaneous
by translation＠torproject.org 18 Apr '18

18 Apr '18

commit 66290e81514546aba1e92874a8c28dd8340f54ff Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 18 15:50:18 2018 +0000 Update translations for support-miscellaneous --- nl.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/nl.json b/nl.json index 7400fcbe9..948d12c3c 100644 --- a/nl.json +++ b/nl.json @@ -9,7 +9,7 @@ "id": "#misc-2", "control": "misc-2", "title": "Waarom voorkom je niet dat slechte mensen slechte dingen doen bij het gebruik van Tor?", - "description": "Tor is designed to defend human rights and privacy by preventing anyone from censoring things, even us. We hate that there are some people who use Tor to do terrible things, but we can't do anything to get rid of them without also undermining the human rights activists, journalists, abuse survivors, and other people who use Tor for good things. If we wanted to block certain people from using Tor, we'd basically be adding a backdoor to the software, which would open up our vulnerable users to attacks from bad regimes and other adversaries." + "description": "Tor is ontworpen om mensenrechten en privacy te beschermen door te voorkomen dat iemand dingen censureert, zelfs wij. We haten het dat sommige mensen Tor gebruiken om vreselijke dingen te doen, maar we kunnen niets doen om ervan af te komen zonder de mensenrechtenactivisten, journalisten, misbruikende overlevenden en andere mensen die Tor gebruiken voor goede dingen te ondermijnen. Als we bepaalde mensen wilden blokkeren voor het gebruik van Tor, zouden we in feite een achterdeur toevoegen aan de software, waardoor onze kwetsbare gebruikers zouden worden blootgesteld aan aanvallen van slechte regimes en andere tegenstanders." }, "misc-3": { "id": "#misc-3", @@ -62,7 +62,7 @@ "misc-11": { "id": "#misc-11", "control": "misc-11", - "title": "Can I change the number of hops Tor uses?", + "title": "Kan ik het aantal hops dat Tor gebruikt wijzigen?", "description": "Right now the path length is hard-coded at 3 plus the number of nodes in your path that are sensitive. That is, in normal cases it's 3, but for example if you're accessing an onion service or a \".exit\" address it could be more.<p class=\"mb-3\"We don't want to encourage people to use paths longer than this as it increases load on the network without (as far as we can tell) providing any more security. Also, using paths longer than 3 could harm anonymity, first because it makes<a href=\"https://www.freehaven.net/anonbib/#ccs07-doa\">denial of security</a> ttacks easier, and second because it could act as an identifier if only a small number of users have the same path length as you." }, "misc-12": {

1 0

[translation/support-miscellaneous] Update translations for support-miscellaneous
by translation＠torproject.org 18 Apr '18

18 Apr '18

commit fbb72162432e17930c0de2dceb7864461108d13d Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 18 15:20:29 2018 +0000 Update translations for support-miscellaneous --- nl.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nl.json b/nl.json index df240a0cf..7400fcbe9 100644 --- a/nl.json +++ b/nl.json @@ -3,7 +3,7 @@ "id": "#misc-1", "control": "misc-1", "title": "Ik heb een dringende reden om een Tor gebruiker te traceren. Kun je helpen?", - "description": "There is nothing the Tor developers can do to trace Tor users. The same protections that keep bad people from breaking Tor's anonymity also prevent us from tracking users." + "description": "De Tor-ontwikkelaars kunnen niets doen om Tor-gebruikers op te sporen. Dezelfde bescherming die slechte mensen ervan weerhoudt de anonimiteit van Tor te schenden, verhindert ons ook gebruikers te volgen." }, "misc-2": { "id": "#misc-2",

1 0

[metrics-lib/master] Bump version to 2.3.0-dev.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit 77bbd742b63fd9708694d7d8623d82a9654b770e Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Apr 18 16:43:13 2018 +0200 Bump version to 2.3.0-dev. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 9bff4b2..61cd45d 100644 --- a/build.xml +++ b/build.xml @@ -6,7 +6,7 @@ <project default="usage" name="metrics-lib" basedir="."> - <property name="release.version" value="2.3.0" /> + <property name="release.version" value="2.3.0-dev" /> <property name="javadoc-title" value="Tor Metrics Library API Documentation"/> <property name="javadoc-excludes" value="**/impl/** **/index/** **/internal/** **/log/**" /> <property name="implementation-title" value="Tor Metrics Library" />

1 0

[onionoo/master] Bump version to 6.0-1.13.0-dev.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit 960ded0162b99d3ed8b7af976d92eb422c5969d0 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Apr 18 16:42:38 2018 +0200 Bump version to 6.0-1.13.0-dev. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 315989b..557297d 100644 --- a/build.xml +++ b/build.xml @@ -10,7 +10,7 @@ <property name="implementation-title" value="Onionoo" /> <property name="onionoo.protocol.version" value="6.0"/> <property name="release.version" - value="${onionoo.protocol.version}-1.13.0"/> + value="${onionoo.protocol.version}-1.13.0-dev"/> <property name="metricslibversion" value="2.2.0"/> <property name="jetty.version" value="-9.2.21.v20170120" /> <property name="warfile"

1 0

[metrics-lib/release] Prepare for 2.3.0 release.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit bb087e88d211f6b75a168158a5a82ee477c9fc22 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Apr 18 11:42:54 2018 +0200 Prepare for 2.3.0 release. --- CHANGELOG.md | 2 +- build.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5edc519..1e9069e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -# Changes in version 2.3.0 - 2018-0?-?? +# Changes in version 2.3.0 - 2018-04-18 * Medium changes - Replace ServerDescriptor#getHiddenServiceDirVersions with diff --git a/build.xml b/build.xml index 199ba7a..9bff4b2 100644 --- a/build.xml +++ b/build.xml @@ -6,7 +6,7 @@ <project default="usage" name="metrics-lib" basedir="."> - <property name="release.version" value="2.2.0-dev" /> + <property name="release.version" value="2.3.0" /> <property name="javadoc-title" value="Tor Metrics Library API Documentation"/> <property name="javadoc-excludes" value="**/impl/** **/index/** **/internal/** **/log/**" /> <property name="implementation-title" value="Tor Metrics Library" />

1 0

[metrics-lib/release] Use the descriptor file name for determining descriptor type.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit 7b859f8dc21c801f0bf707ceac6703f84c056497 Author: iwakeh <iwakeh(a)torproject.org> Date: Tue Mar 27 07:25:37 2018 +0000 Use the descriptor file name for determining descriptor type. Initial solution suggested by Karsten. In addition, add a test that would fail without this patch and adapt old test to expect the correct behavior. Also provide the origin of the descriptor with getDescriptorFile. Include a test for this functionality in log descriptors. Provide proper file name to log descriptor classes, because this filename contains log metadata for storing log files properly. Solves task-25332. --- CHANGELOG.md | 3 +++ .../descriptor/impl/DescriptorParserImpl.java | 4 ++-- .../descriptor/log/LogDescriptorImpl.java | 19 ++++++++-------- .../descriptor/log/WebServerAccessLogImpl.java | 25 +++++++++++++-------- .../descriptor/impl/DescriptorParserImplTest.java | 18 +++++++++++++++ .../descriptor/log/LogDescriptorTest.java | 4 ++-- .../descriptor/log/WebServerAccessLogTest.java | 2 +- .../descriptor/log/WebServerModuleTest.java | 18 +++++---------- src/test/resources/webstats-2015-02.tar | Bin 0 -> 20480 bytes 9 files changed, 58 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d64d58d..5edc519 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,9 @@ - Replace ServerDescriptor#getHiddenServiceDirVersions with ServerDescriptor#isHiddenServiceDir, because Tor has never supported versions in the hidden-service-dir descriptor line. + - Add support for reading web server logs contained in tarballs by + providing file names of both the tarball and of contained files + to WebServerAccessLog. * Minor changes - Override logLines() method from LogDescriptor in diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java index be9041f..713170e 100644 --- a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java +++ b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java @@ -130,8 +130,8 @@ public class DescriptorParserImpl implements DescriptorParser { } else if (firstLines.startsWith("@type torperf 1.")) { return TorperfResultImpl.parseTorperfResults(rawDescriptorBytes, sourceFile); - } else if (sourceFile.getName().contains(LogDescriptorImpl.MARKER)) { - return LogDescriptorImpl.parse(rawDescriptorBytes, sourceFile); + } else if (fileName.contains(LogDescriptorImpl.MARKER)) { + return LogDescriptorImpl.parse(rawDescriptorBytes, sourceFile, fileName); } else { throw new DescriptorParseException("Could not detect descriptor " + "type in descriptor starting with '" + firstLines + "'."); diff --git a/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java b/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java index 3583d26..b730465 100644 --- a/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java +++ b/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java @@ -64,14 +64,15 @@ public abstract class LogDescriptorImpl * @since 2.2.0 */ protected LogDescriptorImpl(byte[] logBytes, File descriptorFile, - FileType defaultCompression) throws DescriptorParseException { + String logName, FileType defaultCompression) + throws DescriptorParseException { this.logBytes = logBytes; this.descriptorFile = descriptorFile; try { - Matcher mat = filenamePattern.matcher(descriptorFile.getName()); + Matcher mat = filenamePattern.matcher(logName); if (!mat.find()) { throw new DescriptorParseException( - "Log file name doesn't comply to standard: " + descriptorFile); + "Log file name doesn't comply to standard: " + logName); } this.fileType = FileType.findType(mat.group(1).toUpperCase()); if (FileType.PLAIN == this.fileType) { @@ -80,7 +81,7 @@ public abstract class LogDescriptorImpl } } catch (Exception ex) { throw new DescriptorParseException("Cannot parse file " - + descriptorFile.getName(), ex); + + logName + " from file " + descriptorFile.getName(), ex); } } @@ -112,13 +113,13 @@ public abstract class LogDescriptorImpl * @since 2.2.0 */ public static List<Descriptor> parse(byte[] logBytes, - File descriptorFile) throws DescriptorParseException { - if (descriptorFile.getName().contains(InternalWebServerAccessLog.MARKER)) { + File descriptorFile, String logName) throws DescriptorParseException { + if (logName.contains(InternalWebServerAccessLog.MARKER)) { return Arrays.asList(new Descriptor[]{ - new WebServerAccessLogImpl(logBytes, descriptorFile)}); + new WebServerAccessLogImpl(logBytes, descriptorFile, logName)}); } else { - throw new DescriptorParseException("Cannot parse file " - + descriptorFile.getName()); + throw new DescriptorParseException("Cannot parse file " + logName + + " from file " + descriptorFile.getName()); } } diff --git a/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java index 3666d5d..ffaef7b 100644 --- a/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java +++ b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java @@ -13,6 +13,7 @@ import org.slf4j.LoggerFactory; import java.io.BufferedReader; import java.io.File; import java.io.InputStreamReader; +import java.nio.file.Paths; import java.time.LocalDate; import java.time.format.DateTimeFormatter; import java.util.ArrayList; @@ -65,28 +66,34 @@ public class WebServerAccessLogImpl extends LogDescriptorImpl * The immediate parent name is taken to be the physical host collecting the * logs. */ - protected WebServerAccessLogImpl(byte[] logBytes, File file) + protected WebServerAccessLogImpl(byte[] logBytes, File file, String logName) throws DescriptorParseException { - this(logBytes, file, FileType.XZ); + this(logBytes, file, logName, FileType.XZ); } /** For internal use only. */ public WebServerAccessLogImpl(byte[] bytes, String filename, boolean validate) throws DescriptorParseException { - this(bytes, new File(filename), FileType.XZ, validate); + this(bytes, null, filename, FileType.XZ, validate); } - private WebServerAccessLogImpl(byte[] logBytes, File file, + /** For internal use only. */ + public WebServerAccessLogImpl(byte[] bytes, File sourceFile, String filename, + boolean validate) throws DescriptorParseException { + this(bytes, sourceFile, filename, FileType.XZ, validate); + } + + private WebServerAccessLogImpl(byte[] logBytes, File file, String logName, FileType defaultCompression) throws DescriptorParseException { - this(logBytes, file, defaultCompression, true); + this(logBytes, file, logName, defaultCompression, true); } - private WebServerAccessLogImpl(byte[] logBytes, File file, + private WebServerAccessLogImpl(byte[] logBytes, File file, String logName, FileType defaultCompression, boolean validate) throws DescriptorParseException { - super(logBytes, file, defaultCompression); + super(logBytes, file, logName, defaultCompression); try { - String fn = file.toPath().getFileName().toString(); + String fn = Paths.get(logName).getFileName().toString(); Matcher mat = filenamePattern.matcher(fn); if (!mat.find()) { throw new DescriptorParseException( @@ -110,7 +117,7 @@ public class WebServerAccessLogImpl extends LogDescriptorImpl throw dpe; // escalate } catch (Exception pe) { throw new DescriptorParseException( - "Cannot parse WebServerAccessLog file: " + file, pe); + "Cannot parse WebServerAccessLog file: " + logName, pe); } } diff --git a/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java b/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java index d9dd178..70594d8 100644 --- a/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java +++ b/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java @@ -4,9 +4,13 @@ package org.torproject.descriptor.impl; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import org.torproject.descriptor.Descriptor; import org.torproject.descriptor.DescriptorParseException; +import org.torproject.descriptor.DescriptorReader; +import org.torproject.descriptor.DescriptorSourceFactory; +import org.torproject.descriptor.WebServerAccessLog; import org.junit.Rule; import org.junit.Test; @@ -48,6 +52,20 @@ public class DescriptorParserImplTest { } @Test + public void testParseDescriptorTar() throws Exception { + DescriptorReader dr = DescriptorSourceFactory.createDescriptorReader(); + int count = 0; + for (Descriptor desc : dr.readDescriptors(new File(getClass() + .getClassLoader().getResource("webstats-2015-02.tar").toURI()))) { + assertTrue(desc instanceof WebServerAccessLog); + assertTrue(desc.getDescriptorFile().toString() + .endsWith("webstats-2015-02.tar")); + count++; + } + assertEquals(7, count); + } + + @Test public void testParseDescriptor() throws DescriptorParseException { Constructor<? extends DescriptorImpl> constructor; try { diff --git a/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java b/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java index 0ff3e62..e76566a 100644 --- a/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java +++ b/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java @@ -149,8 +149,8 @@ public class LogDescriptorTest { File invalidFile = new File(this.reader.getParsedFiles().firstKey() .replace("access", "-")); List<Descriptor> descs = new ArrayList<>(); - for (Descriptor desc // note: only 'invalidFile' is used by LogDescriptor - : dp.parseDescriptors(new byte[]{}, invalidFile, logFile.getName())) { + for (Descriptor desc // note: only 'invalidFile' is used by DescriptorReader + : dp.parseDescriptors(new byte[]{}, logFile, invalidFile.getName())) { descs.add(desc); } assertTrue(dataUsed() + "\nWrong type: " diff --git a/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java index 3e98f13..b64c4df 100644 --- a/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java +++ b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java @@ -79,7 +79,7 @@ public class WebServerAccessLogTest { @Test public void testValidation() throws Exception { WebServerAccessLogImpl wsal - = new WebServerAccessLogImpl(real.getBytes(), file); + = new WebServerAccessLogImpl(real.getBytes(), file, fn); wsal.validate(); if (valid) { assertEquals(0, wsal.getUnrecognizedLines().size()); diff --git a/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java b/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java index 66c8f54..c5cf6e5 100644 --- a/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java +++ b/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java @@ -14,7 +14,6 @@ import org.junit.Rule; import org.junit.Test; import org.junit.rules.ExpectedException; -import java.io.File; import java.util.Arrays; import java.util.stream.Collectors; @@ -31,8 +30,7 @@ public class WebServerModuleTest { thrown.expectMessage(Matchers .containsString("Cannot parse WebServerAccessLog file: " + filename)); - new WebServerAccessLogImpl(new byte[0], - new File(filename)); + new WebServerAccessLogImpl(new byte[0], null, filename); } @Test @@ -42,8 +40,7 @@ public class WebServerModuleTest { thrown.expectMessage(Matchers .containsString("Cannot parse WebServerAccessLog file: " + filename)); - new WebServerAccessLogImpl(new byte[0], - new File(filename)); + new WebServerAccessLogImpl(new byte[0], null, filename); } @Test @@ -53,8 +50,7 @@ public class WebServerModuleTest { thrown.expectMessage(Matchers .containsString("WebServerAccessLog " + "file name doesn't comply to standard: " + filename)); - new WebServerAccessLogImpl(new byte[0], - new File(filename)); + new WebServerAccessLogImpl(new byte[0], null, filename); } @Test @@ -64,8 +60,7 @@ public class WebServerModuleTest { thrown.expectMessage(Matchers .containsString("WebServerAccessLog " + "file name doesn't comply to standard: " + filename)); - new WebServerAccessLogImpl(new byte[0], - new File(filename)); + new WebServerAccessLogImpl(new byte[0], null, filename); } @Test @@ -75,8 +70,7 @@ public class WebServerModuleTest { thrown.expectMessage(Matchers .containsString("WebServerAccessLog " + "file name doesn't comply to standard: " + filename)); - new WebServerAccessLogImpl(new byte[0], - new File(filename)); + new WebServerAccessLogImpl(new byte[0], null, filename); } private static String[] logLines = { @@ -103,7 +97,7 @@ public class WebServerModuleTest { @Test public void testBasics() throws Exception { WebServerAccessLogImpl wsal = new WebServerAccessLogImpl(logText.getBytes(), - new File("vhost_host7_access.log_20170530")); + null, "vhost_host7_access.log_20170530"); assertEquals(wsal.getAnnotations().size(), 0); assertEquals(logText, new String(FileType.XZ.decompress(wsal.getRawDescriptorBytes()))); diff --git a/src/test/resources/webstats-2015-02.tar b/src/test/resources/webstats-2015-02.tar new file mode 100644 index 0000000..ffb3fc7 Binary files /dev/null and b/src/test/resources/webstats-2015-02.tar differ

1 0

[metrics-lib/release] Clarify parameter description in DescriptorParser's javadoc.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit 42daf8e5e12f99ab78c2dd53b4cfe729e8171ded Author: iwakeh <iwakeh(a)torproject.org> Date: Tue Mar 27 07:25:36 2018 +0000 Clarify parameter description in DescriptorParser's javadoc. Also rename parameter for clarity and apply the name change to the implementation. --- .../torproject/descriptor/DescriptorParser.java | 9 ++-- .../descriptor/impl/DescriptorParserImpl.java | 52 +++++++++++----------- 2 files changed, 31 insertions(+), 30 deletions(-) diff --git a/src/main/java/org/torproject/descriptor/DescriptorParser.java b/src/main/java/org/torproject/descriptor/DescriptorParser.java index d530a2b..064e3ab 100644 --- a/src/main/java/org/torproject/descriptor/DescriptorParser.java +++ b/src/main/java/org/torproject/descriptor/DescriptorParser.java @@ -28,9 +28,10 @@ public interface DescriptorParser { * * @param rawDescriptorBytes Raw descriptor bytes containing one or more * descriptors - * @param descriptorFile Optional descriptor file reference included in - * parsed/unparseable descriptors - * @param fileName Descriptor file name used for parsing the descriptor + * @param sourceFile Optional descriptor source file reference, e.g., the name + * of a tar file that contains descriptors, included in parsed/unparseable + * descriptors. + * @param fileName Proper file name for the descriptor used as a parser hint, * publication time of some descriptor types * * @return Parsed/unparseable descriptors @@ -38,5 +39,5 @@ public interface DescriptorParser { * @since 1.9.0 */ public Iterable<Descriptor> parseDescriptors(byte[] rawDescriptorBytes, - File descriptorFile, String fileName); + File sourceFile, String fileName); } diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java index f244abb..be9041f 100644 --- a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java +++ b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java @@ -28,21 +28,21 @@ public class DescriptorParserImpl implements DescriptorParser { @Override public Iterable<Descriptor> parseDescriptors(byte[] rawDescriptorBytes, - File descriptorFile, String fileName) { + File sourceFile, String fileName) { try { return this.detectTypeAndParseDescriptors(rawDescriptorBytes, - descriptorFile, fileName); + sourceFile, fileName); } catch (DescriptorParseException e) { - log.debug("Cannot parse descriptor file ’{}’.", descriptorFile, e); + log.debug("Cannot parse descriptor file ’{}’.", sourceFile, e); List<Descriptor> parsedDescriptors = new ArrayList<>(); parsedDescriptors.add(new UnparseableDescriptorImpl(rawDescriptorBytes, - new int[] { 0, rawDescriptorBytes.length }, descriptorFile, e)); + new int[] { 0, rawDescriptorBytes.length }, sourceFile, e)); return parsedDescriptors; } } private List<Descriptor> detectTypeAndParseDescriptors( - byte[] rawDescriptorBytes, File descriptorFile, String fileName) + byte[] rawDescriptorBytes, File sourceFile, String fileName) throws DescriptorParseException { byte[] first100Chars = new byte[Math.min(100, rawDescriptorBytes.length)]; @@ -58,7 +58,7 @@ public class DescriptorParserImpl implements DescriptorParser { NL + Key.NETWORK_STATUS_VERSION.keyword + SP + "3")) && firstLines.contains( NL + Key.VOTE_STATUS.keyword + SP + "consensus" + NL))) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.NETWORK_STATUS_VERSION, RelayNetworkStatusConsensusImpl.class); } else if (firstLines.startsWith("@type network-status-vote-3 1.") || ((firstLines.startsWith( @@ -67,52 +67,52 @@ public class DescriptorParserImpl implements DescriptorParser { NL + Key.NETWORK_STATUS_VERSION.keyword + SP + "3" + NL)) && firstLines.contains( NL + Key.VOTE_STATUS.keyword + SP + "vote" + NL))) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.NETWORK_STATUS_VERSION, RelayNetworkStatusVoteImpl.class); } else if (firstLines.startsWith("@type bridge-network-status 1.") || firstLines.startsWith(Key.R.keyword + SP)) { List<Descriptor> parsedDescriptors = new ArrayList<>(); parsedDescriptors.add(new BridgeNetworkStatusImpl( rawDescriptorBytes, new int[] { 0, rawDescriptorBytes.length }, - descriptorFile, fileName)); + sourceFile, fileName)); return parsedDescriptors; } else if (firstLines.startsWith("@type bridge-server-descriptor 1.")) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.ROUTER, BridgeServerDescriptorImpl.class); } else if (firstLines.startsWith("@type server-descriptor 1.") || firstLines.startsWith(Key.ROUTER.keyword + SP) || firstLines.contains(NL + Key.ROUTER.keyword + SP)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.ROUTER, RelayServerDescriptorImpl.class); } else if (firstLines.startsWith("@type bridge-extra-info 1.")) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.EXTRA_INFO, BridgeExtraInfoDescriptorImpl.class); } else if (firstLines.startsWith("@type extra-info 1.") || firstLines.startsWith(Key.EXTRA_INFO.keyword + SP) || firstLines.contains(NL + Key.EXTRA_INFO.keyword + SP)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.EXTRA_INFO, RelayExtraInfoDescriptorImpl.class); } else if (firstLines.startsWith("@type microdescriptor 1.") || firstLines.startsWith(Key.ONION_KEY.keyword + NL) || firstLines.contains(NL + Key.ONION_KEY.keyword + NL)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.ONION_KEY, MicrodescriptorImpl.class); } else if (firstLines.startsWith("@type bridge-pool-assignment 1.") || firstLines.startsWith(Key.BRIDGE_POOL_ASSIGNMENT.keyword + SP) || firstLines.contains(NL + Key.BRIDGE_POOL_ASSIGNMENT.keyword + SP)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.BRIDGE_POOL_ASSIGNMENT, BridgePoolAssignmentImpl.class); } else if (firstLines.startsWith("@type dir-key-certificate-3 1.") || firstLines.startsWith(Key.DIR_KEY_CERTIFICATE_VERSION.keyword + SP) || firstLines.contains( NL + Key.DIR_KEY_CERTIFICATE_VERSION.keyword + SP)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.DIR_KEY_CERTIFICATE_VERSION, DirectoryKeyCertificateImpl.class); } else if (firstLines.startsWith("@type tordnsel 1.") || firstLines.startsWith("ExitNode" + SP) || firstLines.contains(NL + "ExitNode" + SP)) { List<Descriptor> parsedDescriptors = new ArrayList<>(); - parsedDescriptors.add(new ExitListImpl(rawDescriptorBytes, descriptorFile, + parsedDescriptors.add(new ExitListImpl(rawDescriptorBytes, sourceFile, fileName)); return parsedDescriptors; } else if (firstLines.startsWith("@type network-status-2 1.") @@ -120,18 +120,18 @@ public class DescriptorParserImpl implements DescriptorParser { Key.NETWORK_STATUS_VERSION.keyword + SP + "2" + NL) || firstLines.contains( NL + Key.NETWORK_STATUS_VERSION.keyword + SP + "2" + NL)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.NETWORK_STATUS_VERSION, RelayNetworkStatusImpl.class); } else if (firstLines.startsWith("@type directory 1.") || firstLines.startsWith(Key.SIGNED_DIRECTORY.keyword + NL) || firstLines.contains(NL + Key.SIGNED_DIRECTORY.keyword + NL)) { - return this.parseOneOrMoreDescriptors(rawDescriptorBytes, descriptorFile, + return this.parseOneOrMoreDescriptors(rawDescriptorBytes, sourceFile, Key.SIGNED_DIRECTORY, RelayDirectoryImpl.class); } else if (firstLines.startsWith("@type torperf 1.")) { return TorperfResultImpl.parseTorperfResults(rawDescriptorBytes, - descriptorFile); - } else if (descriptorFile.getName().contains(LogDescriptorImpl.MARKER)) { - return LogDescriptorImpl.parse(rawDescriptorBytes, descriptorFile); + sourceFile); + } else if (sourceFile.getName().contains(LogDescriptorImpl.MARKER)) { + return LogDescriptorImpl.parse(rawDescriptorBytes, sourceFile); } else { throw new DescriptorParseException("Could not detect descriptor " + "type in descriptor starting with '" + firstLines + "'."); @@ -139,7 +139,7 @@ public class DescriptorParserImpl implements DescriptorParser { } private List<Descriptor> parseOneOrMoreDescriptors(byte[] rawDescriptorBytes, - File descriptorFile, Key key, + File sourceFile, Key key, Class<? extends DescriptorImpl> descriptorClass) throws DescriptorParseException { List<Descriptor> parsedDescriptors = new ArrayList<>(); @@ -192,10 +192,10 @@ public class DescriptorParserImpl implements DescriptorParser { endDescriptor - startAnnotations }; try { parsedDescriptors.add(this.parseOneDescriptor(rawDescriptorBytes, - offsetAndLength, descriptorFile, constructor)); + offsetAndLength, sourceFile, constructor)); } catch (DescriptorParseException e) { parsedDescriptors.add(new UnparseableDescriptorImpl( - rawDescriptorBytes, offsetAndLength, descriptorFile, e)); + rawDescriptorBytes, offsetAndLength, sourceFile, e)); } startAnnotations = endDescriptor; } @@ -203,12 +203,12 @@ public class DescriptorParserImpl implements DescriptorParser { } Descriptor parseOneDescriptor(byte[] rawDescriptorBytes, - int[] offsetAndLength, File descriptorFile, + int[] offsetAndLength, File sourceFile, Constructor<? extends DescriptorImpl> constructor) throws DescriptorParseException { try { return constructor.newInstance(rawDescriptorBytes, offsetAndLength, - descriptorFile); + sourceFile); } catch (InvocationTargetException e) { if (null != e.getCause() && e.getCause() instanceof DescriptorParseException) {

1 0

[metrics-lib/release] Bump version to 2.2.0-dev.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit 7648780844f809f3068d3e625973fe43c78ccbb3 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Feb 26 16:24:25 2018 +0100 Bump version to 2.2.0-dev. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 3f081f7..199ba7a 100644 --- a/build.xml +++ b/build.xml @@ -6,7 +6,7 @@ <project default="usage" name="metrics-lib" basedir="."> - <property name="release.version" value="2.2.0" /> + <property name="release.version" value="2.2.0-dev" /> <property name="javadoc-title" value="Tor Metrics Library API Documentation"/> <property name="javadoc-excludes" value="**/impl/** **/index/** **/internal/** **/log/**" /> <property name="implementation-title" value="Tor Metrics Library" />

1 0

[metrics-lib/release] Override logLines method from LogDescriptor.
by karsten＠torproject.org 18 Apr '18

18 Apr '18

commit c54b264ad865d522bceb135d0fd0ab6b116e5f81 Author: iwakeh <iwakeh(a)torproject.org> Date: Mon Feb 26 15:24:26 2018 +0000 Override logLines method from LogDescriptor. --- .../java/org/torproject/descriptor/WebServerAccessLog.java | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/main/java/org/torproject/descriptor/WebServerAccessLog.java b/src/main/java/org/torproject/descriptor/WebServerAccessLog.java index 5f3ad73..a24b9cd 100644 --- a/src/main/java/org/torproject/descriptor/WebServerAccessLog.java +++ b/src/main/java/org/torproject/descriptor/WebServerAccessLog.java @@ -6,6 +6,7 @@ package org.torproject.descriptor; import java.time.LocalDate; import java.util.List; import java.util.Optional; +import java.util.stream.Stream; /** * Contains a sanitized web server access log file from a {@code torproject.org} @@ -63,6 +64,15 @@ public interface WebServerAccessLog extends LogDescriptor { public List<String> getUnrecognizedLines(); /** + * Returns a stream of all valid log lines. + * + * @since 2.3.0 + */ + @Override + public Stream<WebServerAccessLog.Line> logLines() + throws DescriptorParseException; + + /** * Facilitates access to all log line fields that don't only contain * default values post sanitization. *

1 0