tor-commits
Threads by month
- ----- 2025 -----
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
January 2018
- 18 participants
- 1738 discussions
31 Jan '18
commit 7dce6a5f6bd36927c0875d2b5f5d02e67698b64a
Author: Karsten Loesing <karsten.loesing(a)gmx.net>
Date: Wed Jan 24 17:45:56 2018 +0100
Fix "requests" column in torperf-1.1.csv.
It looks like we implemented the database query for filling the
"requests" column as completed requests. What we really should have
implemented is total requests made, which is what we're doing here.
Fixes #24996.
---
src/main/sql/onionperf/init-onionperf.sql | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/src/main/sql/onionperf/init-onionperf.sql b/src/main/sql/onionperf/init-onionperf.sql
index eddfe0f..9207bcf 100644
--- a/src/main/sql/onionperf/init-onionperf.sql
+++ b/src/main/sql/onionperf/init-onionperf.sql
@@ -67,8 +67,7 @@ SELECT DATE(start) AS date,
AS timeouts,
COUNT(CASE WHEN NOT didtimeout AND datacomplete >= 1
AND readbytes < filesize THEN 1 ELSE NULL END) AS failures,
- COUNT(CASE WHEN NOT didtimeout AND datacomplete >= 1
- AND readbytes >= filesize then 1 else null end) AS requests
+ COUNT(*) AS requests
FROM measurements
GROUP BY date, filesize, source, server
UNION
@@ -82,8 +81,7 @@ SELECT DATE(start) AS date,
AS timeouts,
COUNT(CASE WHEN NOT didtimeout AND datacomplete >= 1
AND readbytes < filesize THEN 1 ELSE NULL END) AS failures,
- COUNT(CASE WHEN NOT didtimeout AND datacomplete >= 1
- AND readbytes >= filesize then 1 else null end) AS requests
+ COUNT(*) AS requests
FROM measurements
GROUP BY date, filesize, 3, server) sub
ORDER BY date, filesize, source, server;
1
0
commit 2152ece320cf2ed797fe438ff61882ccc7a30b2f
Author: Karsten Loesing <karsten.loesing(a)gmx.net>
Date: Tue Jan 9 10:24:25 2018 +0100
Update copyright to 2018.
---
src/main/java/org/torproject/descriptor/BandwidthHistory.java | 2 +-
src/main/java/org/torproject/descriptor/BridgeExtraInfoDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/BridgeNetworkStatus.java | 2 +-
src/main/java/org/torproject/descriptor/BridgePoolAssignment.java | 2 +-
src/main/java/org/torproject/descriptor/BridgeServerDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/Descriptor.java | 2 +-
src/main/java/org/torproject/descriptor/DescriptorCollector.java | 2 +-
src/main/java/org/torproject/descriptor/DescriptorParseException.java | 2 +-
src/main/java/org/torproject/descriptor/DescriptorParser.java | 2 +-
src/main/java/org/torproject/descriptor/DescriptorReader.java | 2 +-
src/main/java/org/torproject/descriptor/DescriptorSourceFactory.java | 2 +-
src/main/java/org/torproject/descriptor/DirSourceEntry.java | 2 +-
src/main/java/org/torproject/descriptor/DirectoryKeyCertificate.java | 2 +-
src/main/java/org/torproject/descriptor/DirectorySignature.java | 2 +-
src/main/java/org/torproject/descriptor/ExitList.java | 2 +-
src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/Microdescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/NetworkStatusEntry.java | 2 +-
src/main/java/org/torproject/descriptor/RelayDirectory.java | 2 +-
src/main/java/org/torproject/descriptor/RelayExtraInfoDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/RelayNetworkStatus.java | 2 +-
.../java/org/torproject/descriptor/RelayNetworkStatusConsensus.java | 2 +-
src/main/java/org/torproject/descriptor/RelayNetworkStatusVote.java | 2 +-
src/main/java/org/torproject/descriptor/RelayServerDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/RouterStatusEntry.java | 2 +-
src/main/java/org/torproject/descriptor/ServerDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/TorperfResult.java | 2 +-
src/main/java/org/torproject/descriptor/UnparseableDescriptor.java | 2 +-
src/main/java/org/torproject/descriptor/impl/BandwidthHistoryImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/BlockingIteratorImpl.java | 2 +-
.../org/torproject/descriptor/impl/BridgeExtraInfoDescriptorImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/BridgeNetworkStatusImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/BridgePoolAssignmentImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/BridgeServerDescriptorImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/DescriptorImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/DescriptorReaderImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/DirSourceEntryImpl.java | 2 +-
.../org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/DirectorySignatureImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/ExitListEntryImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/ExitListImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/KeyValueMap.java | 2 +-
src/main/java/org/torproject/descriptor/impl/MicrodescriptorImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/NetworkStatusEntryImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/NetworkStatusImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/ParseHelper.java | 2 +-
src/main/java/org/torproject/descriptor/impl/RelayDirectoryImpl.java | 2 +-
.../org/torproject/descriptor/impl/RelayExtraInfoDescriptorImpl.java | 2 +-
.../org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/RelayNetworkStatusImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/RelayServerDescriptorImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/RouterStatusEntryImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java | 2 +-
src/main/java/org/torproject/descriptor/impl/TorperfResultImpl.java | 2 +-
.../java/org/torproject/descriptor/impl/UnparseableDescriptorImpl.java | 2 +-
.../java/org/torproject/descriptor/index/DescriptorIndexCollector.java | 2 +-
src/main/java/org/torproject/descriptor/index/DirectoryNode.java | 2 +-
src/main/java/org/torproject/descriptor/index/FileNode.java | 2 +-
src/main/java/org/torproject/descriptor/index/IndexNode.java | 2 +-
src/main/java/org/torproject/descriptor/index/package-info.java | 2 +-
src/main/java/org/torproject/descriptor/internal/FileType.java | 2 +-
src/main/java/org/torproject/descriptor/package-info.java | 2 +-
src/main/resources/bootstrap-development.sh | 2 +-
src/main/resources/examples/ConsensusWeightByVersion.java | 2 +-
src/main/resources/examples/DownloadConsensuses.java | 2 +-
src/main/resources/examples/PluggableTransports.java | 2 +-
.../java/org/torproject/descriptor/DescriptorSourceFactoryTest.java | 2 +-
src/test/java/org/torproject/descriptor/DescriptorTest.java | 2 +-
.../java/org/torproject/descriptor/DummyCollectorImplementation.java | 2 +-
src/test/java/org/torproject/descriptor/DummyParserImplementation.java | 2 +-
src/test/java/org/torproject/descriptor/DummyReaderImplementation.java | 2 +-
.../java/org/torproject/descriptor/benchmark/MeasurePerformance.java | 2 +-
.../java/org/torproject/descriptor/impl/BridgeNetworkStatusTest.java | 2 +-
src/test/java/org/torproject/descriptor/impl/ConsensusBuilder.java | 2 +-
src/test/java/org/torproject/descriptor/impl/DescriptorImplTest.java | 2 +-
.../java/org/torproject/descriptor/impl/DescriptorParserImplTest.java | 2 +-
.../java/org/torproject/descriptor/impl/DescriptorReaderImplTest.java | 2 +-
src/test/java/org/torproject/descriptor/impl/ExitListImplTest.java | 2 +-
.../org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java | 2 +-
.../java/org/torproject/descriptor/impl/MicrodescriptorImplTest.java | 2 +-
.../torproject/descriptor/impl/RelayNetworkStatusConsensusImplTest.java | 2 +-
.../java/org/torproject/descriptor/impl/RelayNetworkStatusImplTest.java | 2 +-
.../org/torproject/descriptor/impl/RelayNetworkStatusVoteImplTest.java | 2 +-
.../java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java | 2 +-
src/test/java/org/torproject/descriptor/impl/TestDescriptor.java | 2 +-
src/test/java/org/torproject/descriptor/impl/TestServerDescriptor.java | 2 +-
src/test/java/org/torproject/descriptor/impl/TorperfResultImplTest.java | 2 +-
.../org/torproject/descriptor/index/DescriptorIndexCollectorTest.java | 2 +-
src/test/java/org/torproject/descriptor/index/DirectoryNodeTest.java | 2 +-
src/test/java/org/torproject/descriptor/index/FileNodeTest.java | 2 +-
src/test/java/org/torproject/descriptor/index/IndexNodeTest.java | 2 +-
94 files changed, 94 insertions(+), 94 deletions(-)
diff --git a/src/main/java/org/torproject/descriptor/BandwidthHistory.java b/src/main/java/org/torproject/descriptor/BandwidthHistory.java
index 344d0df..cc1e58f 100644
--- a/src/main/java/org/torproject/descriptor/BandwidthHistory.java
+++ b/src/main/java/org/torproject/descriptor/BandwidthHistory.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/BridgeExtraInfoDescriptor.java b/src/main/java/org/torproject/descriptor/BridgeExtraInfoDescriptor.java
index e3cd5d1..d2fa37d 100644
--- a/src/main/java/org/torproject/descriptor/BridgeExtraInfoDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/BridgeExtraInfoDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/BridgeNetworkStatus.java b/src/main/java/org/torproject/descriptor/BridgeNetworkStatus.java
index ca2979c..62f9edf 100644
--- a/src/main/java/org/torproject/descriptor/BridgeNetworkStatus.java
+++ b/src/main/java/org/torproject/descriptor/BridgeNetworkStatus.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/BridgePoolAssignment.java b/src/main/java/org/torproject/descriptor/BridgePoolAssignment.java
index 8f40a74..4d15b81 100644
--- a/src/main/java/org/torproject/descriptor/BridgePoolAssignment.java
+++ b/src/main/java/org/torproject/descriptor/BridgePoolAssignment.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/BridgeServerDescriptor.java b/src/main/java/org/torproject/descriptor/BridgeServerDescriptor.java
index 2a2fdc4..47ee9f4 100644
--- a/src/main/java/org/torproject/descriptor/BridgeServerDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/BridgeServerDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/Descriptor.java b/src/main/java/org/torproject/descriptor/Descriptor.java
index 35ef38b..da51ce2 100644
--- a/src/main/java/org/torproject/descriptor/Descriptor.java
+++ b/src/main/java/org/torproject/descriptor/Descriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DescriptorCollector.java b/src/main/java/org/torproject/descriptor/DescriptorCollector.java
index fb0668c..f27d8fc 100644
--- a/src/main/java/org/torproject/descriptor/DescriptorCollector.java
+++ b/src/main/java/org/torproject/descriptor/DescriptorCollector.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DescriptorParseException.java b/src/main/java/org/torproject/descriptor/DescriptorParseException.java
index c9e7d93..7594b45 100644
--- a/src/main/java/org/torproject/descriptor/DescriptorParseException.java
+++ b/src/main/java/org/torproject/descriptor/DescriptorParseException.java
@@ -1,4 +1,4 @@
-/* Copyright 2014--2017 The Tor Project
+/* Copyright 2014--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DescriptorParser.java b/src/main/java/org/torproject/descriptor/DescriptorParser.java
index 259510f..d530a2b 100644
--- a/src/main/java/org/torproject/descriptor/DescriptorParser.java
+++ b/src/main/java/org/torproject/descriptor/DescriptorParser.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DescriptorReader.java b/src/main/java/org/torproject/descriptor/DescriptorReader.java
index 4adff39..d41e6b4 100644
--- a/src/main/java/org/torproject/descriptor/DescriptorReader.java
+++ b/src/main/java/org/torproject/descriptor/DescriptorReader.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DescriptorSourceFactory.java b/src/main/java/org/torproject/descriptor/DescriptorSourceFactory.java
index f2c1a43..495968c 100644
--- a/src/main/java/org/torproject/descriptor/DescriptorSourceFactory.java
+++ b/src/main/java/org/torproject/descriptor/DescriptorSourceFactory.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DirSourceEntry.java b/src/main/java/org/torproject/descriptor/DirSourceEntry.java
index 69e8525..7540c2c 100644
--- a/src/main/java/org/torproject/descriptor/DirSourceEntry.java
+++ b/src/main/java/org/torproject/descriptor/DirSourceEntry.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DirectoryKeyCertificate.java b/src/main/java/org/torproject/descriptor/DirectoryKeyCertificate.java
index 2819fc3..6a02ac3 100644
--- a/src/main/java/org/torproject/descriptor/DirectoryKeyCertificate.java
+++ b/src/main/java/org/torproject/descriptor/DirectoryKeyCertificate.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/DirectorySignature.java b/src/main/java/org/torproject/descriptor/DirectorySignature.java
index 9ce44df..fb66faa 100644
--- a/src/main/java/org/torproject/descriptor/DirectorySignature.java
+++ b/src/main/java/org/torproject/descriptor/DirectorySignature.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/ExitList.java b/src/main/java/org/torproject/descriptor/ExitList.java
index 2623ee5..e549da8 100644
--- a/src/main/java/org/torproject/descriptor/ExitList.java
+++ b/src/main/java/org/torproject/descriptor/ExitList.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java b/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java
index 333c853..b668eff 100644
--- a/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/Microdescriptor.java b/src/main/java/org/torproject/descriptor/Microdescriptor.java
index ef69417..5490eba 100644
--- a/src/main/java/org/torproject/descriptor/Microdescriptor.java
+++ b/src/main/java/org/torproject/descriptor/Microdescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2014--2017 The Tor Project
+/* Copyright 2014--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/NetworkStatusEntry.java b/src/main/java/org/torproject/descriptor/NetworkStatusEntry.java
index bbc0173..81efde0 100644
--- a/src/main/java/org/torproject/descriptor/NetworkStatusEntry.java
+++ b/src/main/java/org/torproject/descriptor/NetworkStatusEntry.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayDirectory.java b/src/main/java/org/torproject/descriptor/RelayDirectory.java
index f05f65d..3e00ebb 100644
--- a/src/main/java/org/torproject/descriptor/RelayDirectory.java
+++ b/src/main/java/org/torproject/descriptor/RelayDirectory.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayExtraInfoDescriptor.java b/src/main/java/org/torproject/descriptor/RelayExtraInfoDescriptor.java
index 8c7c06a..133d9e3 100644
--- a/src/main/java/org/torproject/descriptor/RelayExtraInfoDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/RelayExtraInfoDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayNetworkStatus.java b/src/main/java/org/torproject/descriptor/RelayNetworkStatus.java
index 1197c2e..5080282 100644
--- a/src/main/java/org/torproject/descriptor/RelayNetworkStatus.java
+++ b/src/main/java/org/torproject/descriptor/RelayNetworkStatus.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayNetworkStatusConsensus.java b/src/main/java/org/torproject/descriptor/RelayNetworkStatusConsensus.java
index 2b433dc..9885172 100644
--- a/src/main/java/org/torproject/descriptor/RelayNetworkStatusConsensus.java
+++ b/src/main/java/org/torproject/descriptor/RelayNetworkStatusConsensus.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayNetworkStatusVote.java b/src/main/java/org/torproject/descriptor/RelayNetworkStatusVote.java
index 1e4b0a5..d14b39e 100644
--- a/src/main/java/org/torproject/descriptor/RelayNetworkStatusVote.java
+++ b/src/main/java/org/torproject/descriptor/RelayNetworkStatusVote.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RelayServerDescriptor.java b/src/main/java/org/torproject/descriptor/RelayServerDescriptor.java
index eeb1c01..2e8118a 100644
--- a/src/main/java/org/torproject/descriptor/RelayServerDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/RelayServerDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/RouterStatusEntry.java b/src/main/java/org/torproject/descriptor/RouterStatusEntry.java
index c2ac5ec..a41b4bd 100644
--- a/src/main/java/org/torproject/descriptor/RouterStatusEntry.java
+++ b/src/main/java/org/torproject/descriptor/RouterStatusEntry.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/ServerDescriptor.java b/src/main/java/org/torproject/descriptor/ServerDescriptor.java
index 0aeb543..ac9c645 100644
--- a/src/main/java/org/torproject/descriptor/ServerDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/ServerDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/TorperfResult.java b/src/main/java/org/torproject/descriptor/TorperfResult.java
index 1958ed2..f66dbde 100644
--- a/src/main/java/org/torproject/descriptor/TorperfResult.java
+++ b/src/main/java/org/torproject/descriptor/TorperfResult.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/UnparseableDescriptor.java b/src/main/java/org/torproject/descriptor/UnparseableDescriptor.java
index 6282222..b4dc103 100644
--- a/src/main/java/org/torproject/descriptor/UnparseableDescriptor.java
+++ b/src/main/java/org/torproject/descriptor/UnparseableDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/main/java/org/torproject/descriptor/impl/BandwidthHistoryImpl.java b/src/main/java/org/torproject/descriptor/impl/BandwidthHistoryImpl.java
index ef29a3b..79c185d 100644
--- a/src/main/java/org/torproject/descriptor/impl/BandwidthHistoryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BandwidthHistoryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/BlockingIteratorImpl.java b/src/main/java/org/torproject/descriptor/impl/BlockingIteratorImpl.java
index 4921ac4..0c171ed 100644
--- a/src/main/java/org/torproject/descriptor/impl/BlockingIteratorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BlockingIteratorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/BridgeExtraInfoDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/BridgeExtraInfoDescriptorImpl.java
index d8941a6..0ca4cfd 100644
--- a/src/main/java/org/torproject/descriptor/impl/BridgeExtraInfoDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BridgeExtraInfoDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/BridgeNetworkStatusImpl.java b/src/main/java/org/torproject/descriptor/impl/BridgeNetworkStatusImpl.java
index 66a238e..c3e7cb4 100644
--- a/src/main/java/org/torproject/descriptor/impl/BridgeNetworkStatusImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BridgeNetworkStatusImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/BridgePoolAssignmentImpl.java b/src/main/java/org/torproject/descriptor/impl/BridgePoolAssignmentImpl.java
index e0e5cd8..58fc872 100644
--- a/src/main/java/org/torproject/descriptor/impl/BridgePoolAssignmentImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BridgePoolAssignmentImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/BridgeServerDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/BridgeServerDescriptorImpl.java
index 8125102..f2de3f2 100644
--- a/src/main/java/org/torproject/descriptor/impl/BridgeServerDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/BridgeServerDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorImpl.java
index eec89a4..918fef8 100644
--- a/src/main/java/org/torproject/descriptor/impl/DescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
index ac7ff9c..d32c031 100644
--- a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorReaderImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorReaderImpl.java
index 2d58573..4da474e 100644
--- a/src/main/java/org/torproject/descriptor/impl/DescriptorReaderImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DescriptorReaderImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DirSourceEntryImpl.java b/src/main/java/org/torproject/descriptor/impl/DirSourceEntryImpl.java
index 630b886..a00ffa1 100644
--- a/src/main/java/org/torproject/descriptor/impl/DirSourceEntryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DirSourceEntryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java b/src/main/java/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java
index 81ece3c..cdc9d44 100644
--- a/src/main/java/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DirectoryKeyCertificateImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/DirectorySignatureImpl.java b/src/main/java/org/torproject/descriptor/impl/DirectorySignatureImpl.java
index b638002..6430e7f 100644
--- a/src/main/java/org/torproject/descriptor/impl/DirectorySignatureImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DirectorySignatureImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/ExitListEntryImpl.java b/src/main/java/org/torproject/descriptor/impl/ExitListEntryImpl.java
index 06b6a96..5389ce3 100644
--- a/src/main/java/org/torproject/descriptor/impl/ExitListEntryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/ExitListEntryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/ExitListImpl.java b/src/main/java/org/torproject/descriptor/impl/ExitListImpl.java
index 454efdc..bd5b2b6 100644
--- a/src/main/java/org/torproject/descriptor/impl/ExitListImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/ExitListImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java
index 3f84902..99ea13e 100644
--- a/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java b/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
index a1eab05..c048b94 100644
--- a/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
+++ b/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/MicrodescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/MicrodescriptorImpl.java
index 6acb845..f91362e 100644
--- a/src/main/java/org/torproject/descriptor/impl/MicrodescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/MicrodescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2014--2017 The Tor Project
+/* Copyright 2014--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/NetworkStatusEntryImpl.java b/src/main/java/org/torproject/descriptor/impl/NetworkStatusEntryImpl.java
index e6f7bad..f869918 100644
--- a/src/main/java/org/torproject/descriptor/impl/NetworkStatusEntryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/NetworkStatusEntryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/NetworkStatusImpl.java b/src/main/java/org/torproject/descriptor/impl/NetworkStatusImpl.java
index b025c1b..60a9ff3 100644
--- a/src/main/java/org/torproject/descriptor/impl/NetworkStatusImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/NetworkStatusImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/ParseHelper.java b/src/main/java/org/torproject/descriptor/impl/ParseHelper.java
index 1c7aa93..d0c83f0 100644
--- a/src/main/java/org/torproject/descriptor/impl/ParseHelper.java
+++ b/src/main/java/org/torproject/descriptor/impl/ParseHelper.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayDirectoryImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayDirectoryImpl.java
index a312f24..4c457f8 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayDirectoryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayDirectoryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayExtraInfoDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayExtraInfoDescriptorImpl.java
index 7ed696e..e24645f 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayExtraInfoDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayExtraInfoDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java
index 87ce59a..bb14c28 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusImpl.java
index d2be570..6158a54 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImpl.java
index df36b26..bc87c66 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RelayServerDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/RelayServerDescriptorImpl.java
index fd2464b..0d1b7a0 100644
--- a/src/main/java/org/torproject/descriptor/impl/RelayServerDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RelayServerDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/RouterStatusEntryImpl.java b/src/main/java/org/torproject/descriptor/impl/RouterStatusEntryImpl.java
index 6f07b81..6678ce7 100644
--- a/src/main/java/org/torproject/descriptor/impl/RouterStatusEntryImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/RouterStatusEntryImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
index 80ed473..ca79cd2 100644
--- a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/TorperfResultImpl.java b/src/main/java/org/torproject/descriptor/impl/TorperfResultImpl.java
index 65b9117..d61afec 100644
--- a/src/main/java/org/torproject/descriptor/impl/TorperfResultImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/TorperfResultImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/impl/UnparseableDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/UnparseableDescriptorImpl.java
index 7174621..eca29c9 100644
--- a/src/main/java/org/torproject/descriptor/impl/UnparseableDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/UnparseableDescriptorImpl.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/main/java/org/torproject/descriptor/index/DescriptorIndexCollector.java b/src/main/java/org/torproject/descriptor/index/DescriptorIndexCollector.java
index 4796263..be79386 100644
--- a/src/main/java/org/torproject/descriptor/index/DescriptorIndexCollector.java
+++ b/src/main/java/org/torproject/descriptor/index/DescriptorIndexCollector.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/main/java/org/torproject/descriptor/index/DirectoryNode.java b/src/main/java/org/torproject/descriptor/index/DirectoryNode.java
index ad9ba74..859493b 100644
--- a/src/main/java/org/torproject/descriptor/index/DirectoryNode.java
+++ b/src/main/java/org/torproject/descriptor/index/DirectoryNode.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/main/java/org/torproject/descriptor/index/FileNode.java b/src/main/java/org/torproject/descriptor/index/FileNode.java
index aa7c1fb..eb34131 100644
--- a/src/main/java/org/torproject/descriptor/index/FileNode.java
+++ b/src/main/java/org/torproject/descriptor/index/FileNode.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/main/java/org/torproject/descriptor/index/IndexNode.java b/src/main/java/org/torproject/descriptor/index/IndexNode.java
index 8b61b75..4c4c884 100644
--- a/src/main/java/org/torproject/descriptor/index/IndexNode.java
+++ b/src/main/java/org/torproject/descriptor/index/IndexNode.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/main/java/org/torproject/descriptor/index/package-info.java b/src/main/java/org/torproject/descriptor/index/package-info.java
index cfb77ca..c685c63 100644
--- a/src/main/java/org/torproject/descriptor/index/package-info.java
+++ b/src/main/java/org/torproject/descriptor/index/package-info.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
/**
diff --git a/src/main/java/org/torproject/descriptor/internal/FileType.java b/src/main/java/org/torproject/descriptor/internal/FileType.java
index 47bbd28..36b5df8 100644
--- a/src/main/java/org/torproject/descriptor/internal/FileType.java
+++ b/src/main/java/org/torproject/descriptor/internal/FileType.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.internal;
diff --git a/src/main/java/org/torproject/descriptor/package-info.java b/src/main/java/org/torproject/descriptor/package-info.java
index 5827115..0410bac 100644
--- a/src/main/java/org/torproject/descriptor/package-info.java
+++ b/src/main/java/org/torproject/descriptor/package-info.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
/**
diff --git a/src/main/resources/bootstrap-development.sh b/src/main/resources/bootstrap-development.sh
index 28fb975..d301e25 100755
--- a/src/main/resources/bootstrap-development.sh
+++ b/src/main/resources/bootstrap-development.sh
@@ -1,6 +1,6 @@
#!/bin/sh
#
-# Copyright 2016--2017 The Tor Project
+# Copyright 2016--2018 The Tor Project
# See LICENSE for licensing information
#
# Use for submodule initialization and checkout.
diff --git a/src/main/resources/examples/ConsensusWeightByVersion.java b/src/main/resources/examples/ConsensusWeightByVersion.java
index 5fc7c1c..5026999 100644
--- a/src/main/resources/examples/ConsensusWeightByVersion.java
+++ b/src/main/resources/examples/ConsensusWeightByVersion.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
import org.torproject.descriptor.*;
diff --git a/src/main/resources/examples/DownloadConsensuses.java b/src/main/resources/examples/DownloadConsensuses.java
index a592928..359c76c 100644
--- a/src/main/resources/examples/DownloadConsensuses.java
+++ b/src/main/resources/examples/DownloadConsensuses.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
import org.torproject.descriptor.*;
diff --git a/src/main/resources/examples/PluggableTransports.java b/src/main/resources/examples/PluggableTransports.java
index 65ce732..102adf0 100644
--- a/src/main/resources/examples/PluggableTransports.java
+++ b/src/main/resources/examples/PluggableTransports.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
import org.torproject.descriptor.*;
diff --git a/src/test/java/org/torproject/descriptor/DescriptorSourceFactoryTest.java b/src/test/java/org/torproject/descriptor/DescriptorSourceFactoryTest.java
index 3a9d502..d9549e5 100644
--- a/src/test/java/org/torproject/descriptor/DescriptorSourceFactoryTest.java
+++ b/src/test/java/org/torproject/descriptor/DescriptorSourceFactoryTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/test/java/org/torproject/descriptor/DescriptorTest.java b/src/test/java/org/torproject/descriptor/DescriptorTest.java
index 9a07cdf..2dd72e7 100644
--- a/src/test/java/org/torproject/descriptor/DescriptorTest.java
+++ b/src/test/java/org/torproject/descriptor/DescriptorTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/test/java/org/torproject/descriptor/DummyCollectorImplementation.java b/src/test/java/org/torproject/descriptor/DummyCollectorImplementation.java
index 130a718..219bd2a 100644
--- a/src/test/java/org/torproject/descriptor/DummyCollectorImplementation.java
+++ b/src/test/java/org/torproject/descriptor/DummyCollectorImplementation.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/test/java/org/torproject/descriptor/DummyParserImplementation.java b/src/test/java/org/torproject/descriptor/DummyParserImplementation.java
index 42e67c9..54fc254 100644
--- a/src/test/java/org/torproject/descriptor/DummyParserImplementation.java
+++ b/src/test/java/org/torproject/descriptor/DummyParserImplementation.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/test/java/org/torproject/descriptor/DummyReaderImplementation.java b/src/test/java/org/torproject/descriptor/DummyReaderImplementation.java
index 8f74337..f9d9f3d 100644
--- a/src/test/java/org/torproject/descriptor/DummyReaderImplementation.java
+++ b/src/test/java/org/torproject/descriptor/DummyReaderImplementation.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
diff --git a/src/test/java/org/torproject/descriptor/benchmark/MeasurePerformance.java b/src/test/java/org/torproject/descriptor/benchmark/MeasurePerformance.java
index 09c581d..20df999 100644
--- a/src/test/java/org/torproject/descriptor/benchmark/MeasurePerformance.java
+++ b/src/test/java/org/torproject/descriptor/benchmark/MeasurePerformance.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.benchmark;
diff --git a/src/test/java/org/torproject/descriptor/impl/BridgeNetworkStatusTest.java b/src/test/java/org/torproject/descriptor/impl/BridgeNetworkStatusTest.java
index 22d24fd..fd26eec 100644
--- a/src/test/java/org/torproject/descriptor/impl/BridgeNetworkStatusTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/BridgeNetworkStatusTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/ConsensusBuilder.java b/src/test/java/org/torproject/descriptor/impl/ConsensusBuilder.java
index cb8d0aa..1c0fa81 100644
--- a/src/test/java/org/torproject/descriptor/impl/ConsensusBuilder.java
+++ b/src/test/java/org/torproject/descriptor/impl/ConsensusBuilder.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/DescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/DescriptorImplTest.java
index 09bac87..a0a32cf 100644
--- a/src/test/java/org/torproject/descriptor/impl/DescriptorImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/DescriptorImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java b/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java
index 42d0cde..d9dd178 100644
--- a/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/DescriptorParserImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/DescriptorReaderImplTest.java b/src/test/java/org/torproject/descriptor/impl/DescriptorReaderImplTest.java
index 0a3791e..01d190b 100644
--- a/src/test/java/org/torproject/descriptor/impl/DescriptorReaderImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/DescriptorReaderImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/ExitListImplTest.java b/src/test/java/org/torproject/descriptor/impl/ExitListImplTest.java
index 8365d4c..0b53250 100644
--- a/src/test/java/org/torproject/descriptor/impl/ExitListImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/ExitListImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java
index 5a3f0f2..c90207a 100644
--- a/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/MicrodescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/MicrodescriptorImplTest.java
index a99895f..890071a 100644
--- a/src/test/java/org/torproject/descriptor/impl/MicrodescriptorImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/MicrodescriptorImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImplTest.java b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImplTest.java
index cafbf31..ca8c8e3 100644
--- a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusConsensusImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusImplTest.java b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusImplTest.java
index a2328d3..9ecf391 100644
--- a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImplTest.java b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImplTest.java
index aba55f4..be6567c 100644
--- a/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/RelayNetworkStatusVoteImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2011--2017 The Tor Project
+/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java
index e814b4c..f121cb3 100644
--- a/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2012--2017 The Tor Project
+/* Copyright 2012--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/TestDescriptor.java b/src/test/java/org/torproject/descriptor/impl/TestDescriptor.java
index 48dcb98..1309070 100644
--- a/src/test/java/org/torproject/descriptor/impl/TestDescriptor.java
+++ b/src/test/java/org/torproject/descriptor/impl/TestDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/TestServerDescriptor.java b/src/test/java/org/torproject/descriptor/impl/TestServerDescriptor.java
index 406e9c0..9649bb0 100644
--- a/src/test/java/org/torproject/descriptor/impl/TestServerDescriptor.java
+++ b/src/test/java/org/torproject/descriptor/impl/TestServerDescriptor.java
@@ -1,4 +1,4 @@
-/* Copyright 2017 The Tor Project
+/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/impl/TorperfResultImplTest.java b/src/test/java/org/torproject/descriptor/impl/TorperfResultImplTest.java
index d9e134d..7bc689e 100644
--- a/src/test/java/org/torproject/descriptor/impl/TorperfResultImplTest.java
+++ b/src/test/java/org/torproject/descriptor/impl/TorperfResultImplTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2015--2017 The Tor Project
+/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.impl;
diff --git a/src/test/java/org/torproject/descriptor/index/DescriptorIndexCollectorTest.java b/src/test/java/org/torproject/descriptor/index/DescriptorIndexCollectorTest.java
index 15ca5b5..1180301 100644
--- a/src/test/java/org/torproject/descriptor/index/DescriptorIndexCollectorTest.java
+++ b/src/test/java/org/torproject/descriptor/index/DescriptorIndexCollectorTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/test/java/org/torproject/descriptor/index/DirectoryNodeTest.java b/src/test/java/org/torproject/descriptor/index/DirectoryNodeTest.java
index c98abb8..fc545f6 100644
--- a/src/test/java/org/torproject/descriptor/index/DirectoryNodeTest.java
+++ b/src/test/java/org/torproject/descriptor/index/DirectoryNodeTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/test/java/org/torproject/descriptor/index/FileNodeTest.java b/src/test/java/org/torproject/descriptor/index/FileNodeTest.java
index 5af89ff..0fed293 100644
--- a/src/test/java/org/torproject/descriptor/index/FileNodeTest.java
+++ b/src/test/java/org/torproject/descriptor/index/FileNodeTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
diff --git a/src/test/java/org/torproject/descriptor/index/IndexNodeTest.java b/src/test/java/org/torproject/descriptor/index/IndexNodeTest.java
index f1e044c..c733e47 100644
--- a/src/test/java/org/torproject/descriptor/index/IndexNodeTest.java
+++ b/src/test/java/org/torproject/descriptor/index/IndexNodeTest.java
@@ -1,4 +1,4 @@
-/* Copyright 2016--2017 The Tor Project
+/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor.index;
1
0
commit c29327ddc90e488c1dc31dfa46776345ef022a84
Author: iwakeh <iwakeh(a)torproject.org>
Date: Fri Sep 15 14:07:06 2017 +0000
Added a space.
---
src/main/java/org/torproject/descriptor/impl/KeyValueMap.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java b/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
index c048b94..08ad84e 100644
--- a/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
+++ b/src/main/java/org/torproject/descriptor/impl/KeyValueMap.java
@@ -60,7 +60,7 @@ public class KeyValueMap<T> extends TreeMap<String, T> {
} catch (IllegalArgumentException | InvocationTargetException e) {
value = null;
} catch (NoSuchMethodException e) { // use the String value
- value = (T)keyAndValue[1];
+ value = (T) keyAndValue[1];
}
}
this.putPair(key, value, line, listElement, keyLength);
1
0
commit 5903c105652f3aa11da0e3102e170a88a2374e65
Author: iwakeh <iwakeh(a)torproject.org>
Date: Fri Sep 15 14:07:07 2017 +0000
Added package-info.
---
.../org/torproject/descriptor/internal/package-info.java | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/src/main/java/org/torproject/descriptor/internal/package-info.java b/src/main/java/org/torproject/descriptor/internal/package-info.java
new file mode 100644
index 0000000..5bc7bcb
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/internal/package-info.java
@@ -0,0 +1,14 @@
+/* Copyright 2017 The Tor Project
+ * See LICENSE for licensing information */
+
+/**
+ * <h1>This package is part of the implementation not the public API.</h1>
+ * <p>The public interface might change in unexpected ways.</p>
+ *
+ * <p>Interfaces and essential classes for obtaining and processing
+ * descriptors.</p>
+ *
+ * @since 2.1.0
+ */
+package org.torproject.descriptor.internal;
+
1
0
[metrics-lib/master] Add new descriptor type for web server access logs.
by karsten＠torproject.org 31 Jan '18
by karsten＠torproject.org 31 Jan '18
31 Jan '18
commit 3cd814d8481c87ee3609783d66ae4e2eec81d290
Author: iwakeh <iwakeh(a)torproject.org>
Date: Fri Sep 15 14:07:08 2017 +0000
Add new descriptor type for web server access logs.
Implements task-22983 and is based on the log-descriptor
specification.
---
CHANGELOG.md | 7 +
.../org/torproject/descriptor/LogDescriptor.java | 47 ++++++
.../torproject/descriptor/WebServerAccessLog.java | 65 ++++++++
.../descriptor/impl/DescriptorParserImpl.java | 12 +-
.../torproject/descriptor/index/package-info.java | 6 +-
.../torproject/descriptor/internal/FileType.java | 53 +++++-
.../descriptor/internal/package-info.java | 10 +-
.../descriptor/log/InternalLogDescriptor.java | 63 ++++++++
.../descriptor/log/InternalWebServerAccessLog.java | 17 ++
.../descriptor/log/LogDescriptorImpl.java | 163 +++++++++++++++++++
.../descriptor/log/WebServerAccessLogImpl.java | 119 ++++++++++++++
.../descriptor/log/WebServerAccessLogLine.java | 135 ++++++++++++++++
.../torproject/descriptor/log/package-info.java | 14 ++
.../org/torproject/descriptor/package-info.java | 6 +-
.../descriptor/log/LogDescriptorTest.java | 178 +++++++++++++++++++++
.../descriptor/log/WebServerAccessLogLineTest.java | 140 ++++++++++++++++
.../descriptor/log/WebServerAccessLogTest.java | 94 +++++++++++
.../descriptor/log/WebServerModuleTest.java | 113 +++++++++++++
...eotrichon.torproject.org_access.log_20151007.xz | Bin 0 -> 4056 bytes
...rver.org_dummy.host.net_access.log_20111111.bz2 | Bin 0 -> 76 bytes
...meronense.torproject.org_access.log_20170530.gz | Bin 0 -> 388 bytes
...meronense.torproject.org_access.log_20170531.gz | Bin 0 -> 388 bytes
...eronense.torproject.org_access.log_20170530.log | 26 +++
...eronense.torproject.org_access.log_20170607.log | 26 +++
24 files changed, 1280 insertions(+), 14 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cd0dc6a..42e0e09 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,10 @@
+# Changes in version 2.2.0 - 2018-01-??
+
+ * Major changes
+ - Add new descriptor type WebServerAccessLog to parse web server
+ access logs.
+
+
# Changes in version 2.1.1 - 2017-10-09
* Minor changes
diff --git a/src/main/java/org/torproject/descriptor/LogDescriptor.java b/src/main/java/org/torproject/descriptor/LogDescriptor.java
new file mode 100644
index 0000000..ff02cae
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/LogDescriptor.java
@@ -0,0 +1,47 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor;
+
+import java.util.List;
+
+/**
+ * Contains a log file.
+ *
+ * <p>Unlike other descriptors, logs can get very large and are typically stored
+ * on disk in compressed form. However, all access to log contents through this
+ * interface and its subinterfaces is made available in uncompressed form.</p>
+ *
+ * @since 2.2.0
+ */
+public interface LogDescriptor extends Descriptor {
+
+ /**
+ * Returns the decompressed raw descriptor bytes of the log.
+ *
+ * @since 2.2.0
+ */
+ @Override
+ public byte[] getRawDescriptorBytes();
+
+ /**
+ * Returns annotations found in the log file, which may be an empty List if a
+ * log format does not support adding annotations.
+ *
+ * @since 2.2.0
+ */
+ @Override
+ public List<String> getAnnotations();
+
+ /**
+ * Returns unrecognized lines encountered while parsing the log, which may be
+ * an empty list or a fixed-size list with only a few entries, depending on
+ * the log type.
+ *
+ * @since 2.2.0
+ */
+ @Override
+ public List<String> getUnrecognizedLines();
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/WebServerAccessLog.java b/src/main/java/org/torproject/descriptor/WebServerAccessLog.java
new file mode 100644
index 0000000..b94bc30
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/WebServerAccessLog.java
@@ -0,0 +1,65 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor;
+
+import java.time.LocalDate;
+import java.util.List;
+
+/**
+ * Contains a sanitized web server access log file from a {@code torproject.org}
+ * web server.
+ *
+ * <p>Parsing non-sanitized web server access logs from {@code torproject.org}
+ * web servers or other web servers is not explicitly supported, but may work
+ * anyway.</p>
+ *
+ * @since 2.2.0
+ */
+public interface WebServerAccessLog extends LogDescriptor {
+
+ /**
+ * Returns the date when requests contained in the log have been started,
+ * which is parsed from the log file path.
+ *
+ * <p>Typical web server access logs may contain date information in their
+ * file path, too, but that would be the date when the log file was rotated,
+ * which is not necessary the same date as the date in contained request
+ * lines.</p>
+ *
+ * @since 2.2.0
+ */
+ public LocalDate getLogDate();
+
+ /**
+ * Returns the hostname of the physical host writing this log file, which is
+ * parsed from the log file path.
+ *
+ * <p>A physical host can serve multiple virtual hosts, and a virtual host can
+ * be served by multiple physical hosts.</p>
+ *
+ * @since 2.2.0
+ */
+ public String getPhysicalHost();
+
+ /**
+ * Returns the hostname of the virtual host that this log file was written
+ * for, which is parsed from the log file path.
+ *
+ * <p>A physical host can serve multiple virtual hosts, and a virtual host can
+ * be served by multiple physical hosts.</p>
+ *
+ * @since 2.2.0
+ */
+ public String getVirtualHost();
+
+ /**
+ * Returns at most three unrecognized lines encountered while parsing the log.
+ *
+ * @since 2.2.0
+ */
+ @Override
+ public List<String> getUnrecognizedLines();
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
index d32c031..f244abb 100644
--- a/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/DescriptorParserImpl.java
@@ -9,6 +9,10 @@ import static org.torproject.descriptor.impl.DescriptorImpl.SP;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorParseException;
import org.torproject.descriptor.DescriptorParser;
+import org.torproject.descriptor.log.LogDescriptorImpl;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import java.io.File;
import java.lang.reflect.Constructor;
@@ -19,6 +23,9 @@ import java.util.List;
public class DescriptorParserImpl implements DescriptorParser {
+ private static final Logger log
+ = LoggerFactory.getLogger(DescriptorParserImpl.class);
+
@Override
public Iterable<Descriptor> parseDescriptors(byte[] rawDescriptorBytes,
File descriptorFile, String fileName) {
@@ -26,8 +33,7 @@ public class DescriptorParserImpl implements DescriptorParser {
return this.detectTypeAndParseDescriptors(rawDescriptorBytes,
descriptorFile, fileName);
} catch (DescriptorParseException e) {
- /* Looks like we attempted to parse the whole raw descriptor bytes at once
- * below and ran into a parse issue. */
+ log.debug("Cannot parse descriptor file ’{}’.", descriptorFile, e);
List<Descriptor> parsedDescriptors = new ArrayList<>();
parsedDescriptors.add(new UnparseableDescriptorImpl(rawDescriptorBytes,
new int[] { 0, rawDescriptorBytes.length }, descriptorFile, e));
@@ -124,6 +130,8 @@ public class DescriptorParserImpl implements DescriptorParser {
} else if (firstLines.startsWith("@type torperf 1.")) {
return TorperfResultImpl.parseTorperfResults(rawDescriptorBytes,
descriptorFile);
+ } else if (descriptorFile.getName().contains(LogDescriptorImpl.MARKER)) {
+ return LogDescriptorImpl.parse(rawDescriptorBytes, descriptorFile);
} else {
throw new DescriptorParseException("Could not detect descriptor "
+ "type in descriptor starting with '" + firstLines + "'.");
diff --git a/src/main/java/org/torproject/descriptor/index/package-info.java b/src/main/java/org/torproject/descriptor/index/package-info.java
index c685c63..021cbea 100644
--- a/src/main/java/org/torproject/descriptor/index/package-info.java
+++ b/src/main/java/org/torproject/descriptor/index/package-info.java
@@ -2,14 +2,12 @@
* See LICENSE for licensing information */
/**
- * <h1>This package is still in alpha stage.</h1>
- * <p>The public interface might still change in unexpected ways.</p>
+ * <h1>This package is part of the implementation not the public API.</h1>
+ * <p>The public interface might change in unexpected ways.</p>
*
* <p>Interfaces and essential classes for obtaining and processing
* CollecTor's index.json file.</p>
*
- * <p>Interfaces and classes make the content of index.json available.</p>
- *
*
* @since 1.4.0
*/
diff --git a/src/main/java/org/torproject/descriptor/internal/FileType.java b/src/main/java/org/torproject/descriptor/internal/FileType.java
index 36b5df8..353f0bb 100644
--- a/src/main/java/org/torproject/descriptor/internal/FileType.java
+++ b/src/main/java/org/torproject/descriptor/internal/FileType.java
@@ -12,6 +12,8 @@ import org.apache.commons.compress.compressors.xz.XZCompressorOutputStream;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
@@ -43,6 +45,8 @@ public enum FileType {
/**
* Returns <code>valueOf</code> or the default enum {@link #PLAIN}, i.e.,
* this method doesn't throw any exceptions and allways returns a valid enum.
+ *
+ * @since 2.1.0
*/
public static FileType findType(String ext) {
FileType res = null;
@@ -54,16 +58,61 @@ public enum FileType {
}
}
- /** Return the appropriate input stream. */
+ /**
+ * Return the appropriate input stream.
+ *
+ * @since 1.4.0
+ */
public InputStream inputStream(InputStream is) throws Exception {
return this.inClass.getConstructor(new Class[]{InputStream.class})
.newInstance(is);
}
- /** Return the appropriate output stream. */
+ /**
+ * Return the appropriate output stream.
+ *
+ * @since 1.4.0
+ */
public OutputStream outputStream(OutputStream os) throws Exception {
return this.outClass.getConstructor(new Class[]{OutputStream.class})
.newInstance(os);
}
+
+ /**
+ * Compresses the given bytes in memory and returns the compressed bytes.
+ *
+ * @since 2.2.0
+ */
+ public byte[] compress(byte[] bytes) throws Exception {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ try (OutputStream os = this.outputStream(baos)) {
+ os.write(bytes);
+ os.flush();
+ }
+ return baos.toByteArray();
+ }
+
+ /**
+ * Decompresses the given bytes in memory and returns the decompressed bytes.
+ *
+ * @since 2.2.0
+ */
+ public byte[] decompress(byte[] bytes) throws Exception {
+ if (0 == bytes.length) {
+ return bytes;
+ }
+ try (InputStream is
+ = this.inputStream(new ByteArrayInputStream(bytes));
+ ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
+ int readByte = is.read();
+ while (readByte > 0) {
+ baos.write(readByte);
+ readByte = is.read();
+ }
+ baos.flush();
+ return baos.toByteArray();
+ }
+ }
+
}
diff --git a/src/main/java/org/torproject/descriptor/internal/package-info.java b/src/main/java/org/torproject/descriptor/internal/package-info.java
index 5bc7bcb..b845921 100644
--- a/src/main/java/org/torproject/descriptor/internal/package-info.java
+++ b/src/main/java/org/torproject/descriptor/internal/package-info.java
@@ -2,11 +2,13 @@
* See LICENSE for licensing information */
/**
- * <h1>This package is part of the implementation not the public API.</h1>
- * <p>The public interface might change in unexpected ways.</p>
+ * Interfaces and essential classes for obtaining and processing
+ * descriptors.
+ *
+ * <p><strong>This package is part of the implementation not the
+ * public API.</strong></p>
*
- * <p>Interfaces and essential classes for obtaining and processing
- * descriptors.</p>
+ * <p>The public interface might change in unexpected ways.</p>
*
* @since 2.1.0
*/
diff --git a/src/main/java/org/torproject/descriptor/log/InternalLogDescriptor.java b/src/main/java/org/torproject/descriptor/log/InternalLogDescriptor.java
new file mode 100644
index 0000000..3c0039b
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/InternalLogDescriptor.java
@@ -0,0 +1,63 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import org.torproject.descriptor.DescriptorParseException;
+import org.torproject.descriptor.LogDescriptor;
+
+/**
+ * This interface provides methods for internal use only.
+ *
+ * @since 2.2.0
+ */
+public interface InternalLogDescriptor extends LogDescriptor {
+
+ /** Logfile name parts separator. */
+ public static final String SEP = "_";
+
+ /**
+ * Validate log lines.
+ *
+ * @since 2.2.0
+ */
+ public void validate() throws DescriptorParseException;
+
+ /**
+ * Set the <code>Validator</code> that will perform the validation on log
+ * lines.
+ *
+ * <p>Usually set by the implementing class.</p>
+ *
+ * @since 2.2.0
+ */
+ public void setValidator(Validator validator);
+
+ /**
+ * Set the descriptor's bytes.
+ *
+ * @since 2.2.0
+ */
+ public void setRawDescriptorBytes(byte[] bytes);
+
+ /** Return the descriptor's preferred compression. */
+ public String getCompressionType();
+
+ /**
+ * Provides a single function for validating a single log line.
+ *
+ * @since 2.2.0
+ */
+ public interface Validator {
+
+ /**
+ * Verifies a log line.
+ *
+ * @since 2.2.0
+ */
+ public boolean validate(String line);
+
+ }
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/log/InternalWebServerAccessLog.java b/src/main/java/org/torproject/descriptor/log/InternalWebServerAccessLog.java
new file mode 100644
index 0000000..540f25d
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/InternalWebServerAccessLog.java
@@ -0,0 +1,17 @@
+/* Copyright 2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+/**
+ * This interface provides methods for internal use only.
+ *
+ * @since 2.2.0
+ */
+public interface InternalWebServerAccessLog extends InternalLogDescriptor {
+
+ /** The log's name should include this string. */
+ public static final String MARKER = "access.log";
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java b/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java
new file mode 100644
index 0000000..97854e4
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/LogDescriptorImpl.java
@@ -0,0 +1,163 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import org.torproject.descriptor.Descriptor;
+import org.torproject.descriptor.DescriptorParseException;
+import org.torproject.descriptor.LogDescriptor;
+import org.torproject.descriptor.internal.FileType;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedReader;
+import java.io.ByteArrayInputStream;
+import java.io.File;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * Base class for log descriptors.
+ *
+ * @since 2.2.0
+ */
+public abstract class LogDescriptorImpl
+ implements LogDescriptor, InternalLogDescriptor {
+
+ /** The log's file name should contain this string. */
+ public static final String MARKER = ".log";
+
+ private static final int unrecognizedLinesLimit = 3;
+
+ private static final Logger log
+ = LoggerFactory.getLogger(LogDescriptorImpl.class);
+
+ private static Pattern filenamePattern = Pattern.compile(
+ "(?:\\S*)" + MARKER + SEP + "(?:[0-9a-zA-Z]*)(?:\\.?)([a-zA-Z2]*)");
+
+ private final File descriptorFile;
+
+ /** Byte array for plain, i.e. uncompressed, log data. */
+ private byte[] logBytes;
+
+ private FileType fileType;
+
+ private List<String> unrecognizedLines = new ArrayList<>();
+
+ private Validator validator = (String line) -> true;
+
+ /**
+ * This constructor performs basic operations on the given bytes.
+ *
+ * <p>An unknown compression type (see {@link #getCompressionType})
+ * is interpreted as missing compression. In this case the bytes
+ * will be compressed to the given compression type.</p>
+ *
+ * @since 2.2.0
+ */
+ protected LogDescriptorImpl(byte[] logBytes, File descriptorFile,
+ FileType defaultCompression) throws DescriptorParseException {
+ this.logBytes = logBytes;
+ this.descriptorFile = descriptorFile;
+ try {
+ Matcher mat = filenamePattern.matcher(descriptorFile.getName());
+ if (!mat.find()) {
+ throw new DescriptorParseException(
+ "Log file name doesn't comply to standard: " + descriptorFile);
+ }
+ this.fileType = FileType.findType(mat.group(1).toUpperCase());
+ if (FileType.PLAIN == this.fileType) {
+ this.fileType = defaultCompression;
+ } else {
+ this.logBytes = this.fileType.decompress(this.logBytes);
+ }
+ } catch (Exception ex) {
+ throw new DescriptorParseException("Cannot parse file "
+ + descriptorFile.getName(), ex);
+ }
+ }
+
+ @Override
+ public void validate() throws DescriptorParseException {
+ try (BufferedReader br
+ = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(
+ this.logBytes)))) {
+ this.unrecognizedLines.addAll(br.lines().parallel().filter((line)
+ -> null != line && !line.isEmpty() && !validator.validate(line))
+ .limit(unrecognizedLinesLimit).collect(Collectors.toList()));
+ } catch (Exception ex) {
+ throw new DescriptorParseException("Cannot validate log lines.", ex);
+ }
+ }
+
+ /**
+ * Assemble a LogDescriptor.
+ *
+ * @since 2.2.0
+ */
+ public static List<Descriptor> parse(byte[] logBytes,
+ File descriptorFile) throws DescriptorParseException {
+ if (descriptorFile.getName().contains(InternalWebServerAccessLog.MARKER)) {
+ return Arrays.asList(new Descriptor[]{
+ new WebServerAccessLogImpl(logBytes, descriptorFile)});
+ } else {
+ throw new DescriptorParseException("Cannot parse file "
+ + descriptorFile.getName());
+ }
+ }
+
+ public static byte[] collectionToBytes(Collection<String> lines) {
+ return lines.stream().collect(Collectors.joining("\n", "", "\n"))
+ .getBytes();
+ }
+
+ @Override
+ public void setValidator(Validator validator) {
+ this.validator = validator;
+ }
+
+ @Override
+ public String getCompressionType() {
+ return this.fileType.name().toLowerCase();
+ }
+
+ @Override
+ public byte[] getRawDescriptorBytes() {
+ return this.logBytes;
+ }
+
+ @Override
+ public void setRawDescriptorBytes(byte[] bytes) {
+ this.logBytes = bytes;
+ }
+
+ @Override
+ public int getRawDescriptorLength() {
+ return this.logBytes.length;
+ }
+
+ @Override
+ public List<String> getAnnotations() {
+ return Collections.emptyList();
+ }
+
+ @Override
+ public List<String> getUnrecognizedLines() {
+ return this.unrecognizedLines;
+ }
+
+ @Override
+ public File getDescriptorFile() {
+ return descriptorFile;
+ }
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java
new file mode 100644
index 0000000..6708c3a
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogImpl.java
@@ -0,0 +1,119 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import org.torproject.descriptor.DescriptorParseException;
+import org.torproject.descriptor.WebServerAccessLog;
+import org.torproject.descriptor.internal.FileType;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.time.LocalDate;
+import java.time.format.DateTimeFormatter;
+import java.util.Collection;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * Implementation of web server access log descriptors.
+ *
+ * <p>Defines sanitization and validation for web server access logs.</p>
+ *
+ * @since 2.2.0
+ */
+public class WebServerAccessLogImpl extends LogDescriptorImpl
+ implements InternalWebServerAccessLog, WebServerAccessLog {
+
+ private static final Logger log
+ = LoggerFactory.getLogger(WebServerAccessLogImpl.class);
+
+ /** The log's name should include this string. */
+ public static final String MARKER = InternalWebServerAccessLog.MARKER;
+
+ /** The mandatory web server log descriptor file name pattern. */
+ public static final Pattern filenamePattern
+ = Pattern.compile("(\\S*)" + SEP + "(\\S*)" + SEP + "" + MARKER
+ + SEP + "(\\d*)(?:\\.?)([a-zA-Z]*)");
+
+ private final String physicalHost;
+
+ private final String virtualHost;
+
+ private final LocalDate logDate;
+
+ /**
+ * Creates a WebServerAccessLog from the given bytes and filename.
+ *
+ * <p>The given bytes are read, whereas the file is not read.</p>
+ *
+ * <p>The path of the given file has to be compliant to the following
+ * naming pattern
+ * {@code
+ * <virtualHost>-<physicalHost>-access.log-<yyyymmdd>.<compression>},
+ * where an unknown compression type (see {@link #getCompressionType})
+ * is interpreted as missing compression. In this case the bytes
+ * will be compressed to the default compression type.
+ * The immediate parent name is taken to be the physical host collecting the
+ * logs.</p>
+ */
+ protected WebServerAccessLogImpl(byte[] logBytes, File file)
+ throws DescriptorParseException {
+ this(logBytes, file, FileType.XZ);
+ }
+
+ /** For internal use only. */
+ public WebServerAccessLogImpl(Collection<String> lines, String filename)
+ throws DescriptorParseException {
+ this(LogDescriptorImpl.collectionToBytes(lines), new File(filename));
+ }
+
+ private WebServerAccessLogImpl(byte[] logBytes, File file,
+ FileType defaultCompression) throws DescriptorParseException {
+ super(logBytes, file, defaultCompression);
+ try {
+ String fn = file.toPath().getFileName().toString();
+ Matcher mat = filenamePattern.matcher(fn);
+ if (!mat.find()) {
+ throw new DescriptorParseException(
+ "WebServerAccessLog file name doesn't comply to standard: " + fn);
+ }
+ this.virtualHost = mat.group(1);
+ this.physicalHost = mat.group(2);
+ if (null == this.virtualHost || null == this.physicalHost
+ || this.virtualHost.isEmpty() || this.physicalHost.isEmpty()) {
+ throw new DescriptorParseException(
+ "WebServerAccessLog file name doesn't comply to standard: " + fn);
+ }
+ String ymd = mat.group(3);
+ this.logDate = LocalDate.parse(ymd, DateTimeFormatter.BASIC_ISO_DATE);
+ this.setValidator((line)
+ -> WebServerAccessLogLine.makeLine(line).isValid());
+ this.validate();
+ } catch (DescriptorParseException dpe) {
+ throw dpe; // escalate
+ } catch (Exception pe) {
+ throw new DescriptorParseException(
+ "Cannot parse WebServerAccessLog file: " + file, pe);
+ }
+ }
+
+ @Override
+ public String getPhysicalHost() {
+ return this.physicalHost;
+ }
+
+ @Override
+ public String getVirtualHost() {
+ return this.virtualHost;
+ }
+
+ @Override
+ public LocalDate getLogDate() {
+ return this.logDate;
+ }
+
+}
+
diff --git a/src/main/java/org/torproject/descriptor/log/WebServerAccessLogLine.java b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogLine.java
new file mode 100644
index 0000000..ab20dd2
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/WebServerAccessLogLine.java
@@ -0,0 +1,135 @@
+/* Copyright 2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDate;
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class WebServerAccessLogLine {
+
+ private static final Logger log = LoggerFactory
+ .getLogger(WebServerAccessLogLine.class);
+
+ private static final String DATE_PATTERN = "dd/MMM/yyyy";
+
+ private static final DateTimeFormatter dateTimeFormatter
+ = DateTimeFormatter.ofPattern(DATE_PATTERN + ":HH:mm:ss xxxx");
+
+ private static Pattern logLinePattern = Pattern.compile(
+ "^((?:\\d{1,3}\\.){3}\\d{1,3}) (\\S+) (\\S+) "
+ + "\\[([\\w/]+)([\\w:]+)(\\s[+\\-]\\d{4})\\] "
+ + "\"([A-Z]+) ([^\"]+) ([A-Z]+/\\d\\.\\d)\" "
+ + "(\\d{3}) (\\d+|-)(.*)");
+
+ private String ip;
+ private int response;
+ private String request;
+ private String method;
+ private String dateString;
+ private LocalDate date;
+ private String protocol;
+ private Optional<Integer> size;
+ private boolean valid = false;
+ private String type;
+
+ /** Returns a log line string. Possibly empty. */
+ public String toLogString() {
+ if (!this.valid) {
+ return "";
+ }
+ return toString();
+ }
+
+ @Override
+ public String toString() {
+ return String.format("%s - - [%s:00:00:00 +0000] \"%s %s %s\" %d %s",
+ this.ip, this.dateString, this.method, this.request, this.type,
+ this.response, this.size.isPresent() ? this.size.get() : "-");
+ }
+
+ /** Returns the string of the date using 'yyyymmdd' format. */
+ public String getDateString() {
+ return dateString;
+ }
+
+ /** Returns a string containing the ip. */
+ public String getIp() {
+ return this.ip;
+ }
+
+ /** Only used internally during sanitization. */
+ public void setIp(String ip) {
+ this.ip = ip;
+ }
+
+ public String getMethod() {
+ return this.method;
+ }
+
+ public String getProtocol() {
+ return this.protocol;
+ }
+
+ public String getRequest() {
+ return this.request;
+ }
+
+ public int getResponse() {
+ return this.response;
+ }
+
+ /** Only used internally during sanitization. */
+ public void setRequest(String request) {
+ this.request = request;
+ }
+
+ public LocalDate getDate() {
+ return this.date;
+ }
+
+ public boolean isValid() {
+ return this.valid;
+ }
+
+ /** Creates a Line from a string. */
+ public static WebServerAccessLogLine makeLine(String line) {
+ WebServerAccessLogLine res = new WebServerAccessLogLine();
+ try {
+ Matcher mat = logLinePattern.matcher(line);
+ if (mat.find()) {
+ res.response = Integer.valueOf(mat.group(10));
+ res.method = mat.group(7);
+ res.protocol = mat.group(9);
+ String dateTimeString = mat.group(4) + mat.group(5) + mat.group(6);
+ res.date = ZonedDateTime.parse(dateTimeString,
+ dateTimeFormatter).withZoneSameInstant(ZoneOffset.UTC)
+ .toLocalDate();
+ res.dateString = res.date
+ .format(DateTimeFormatter.ofPattern(DATE_PATTERN));
+ res.ip = mat.group(1);
+ res.request = mat.group(8);
+ res.type = mat.group(9);
+ if ("-".equals(mat.group(11))) {
+ res.size = Optional.empty();
+ } else {
+ res.size = Optional.of(Integer.valueOf(mat.group(11)));
+ }
+ res.valid = true;
+ }
+ } catch (Throwable th) {
+ log.debug("Unmatchable line: '{}'.", line, th);
+ return new WebServerAccessLogLine();
+ }
+ return res;
+ }
+
+}
diff --git a/src/main/java/org/torproject/descriptor/log/package-info.java b/src/main/java/org/torproject/descriptor/log/package-info.java
new file mode 100644
index 0000000..68bcfa1
--- /dev/null
+++ b/src/main/java/org/torproject/descriptor/log/package-info.java
@@ -0,0 +1,14 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+/**
+ * <h1>This package is part of the implementation not the public API.</h1>
+ * <p>The public interface might change in unexpected ways.</p>
+ *
+ * <p>Interfaces and essential classes for obtaining and processing
+ * log descriptors.</p>
+ *
+ * @since 2.2.0
+ */
+package org.torproject.descriptor.log;
+
diff --git a/src/main/java/org/torproject/descriptor/package-info.java b/src/main/java/org/torproject/descriptor/package-info.java
index 0410bac..d844d40 100644
--- a/src/main/java/org/torproject/descriptor/package-info.java
+++ b/src/main/java/org/torproject/descriptor/package-info.java
@@ -65,9 +65,11 @@
* connected to the Tor network rather than by the Tor software. This
* group comprises descriptors by the bridge distribution service BridgeDB
* ({@link org.torproject.descriptor.BridgePoolAssignment}), the exit list
- * service TorDNSEL ({@link org.torproject.descriptor.ExitList}), and the
+ * service TorDNSEL ({@link org.torproject.descriptor.ExitList}), the
* performance measurement service Torperf
- * ({@link org.torproject.descriptor.TorperfResult}).</li>
+ * ({@link org.torproject.descriptor.TorperfResult}), and sanitized access logs
+ * of Tor's web servers
+ * ({@link org.torproject.descriptor.WebServerAccessLog}).</li>
* </ol>
*
* @since 1.0.0
diff --git a/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java b/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java
new file mode 100644
index 0000000..b12cfc0
--- /dev/null
+++ b/src/test/java/org/torproject/descriptor/log/LogDescriptorTest.java
@@ -0,0 +1,178 @@
+
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+import org.torproject.descriptor.Descriptor;
+import org.torproject.descriptor.DescriptorParser;
+import org.torproject.descriptor.DescriptorReader;
+import org.torproject.descriptor.DescriptorSourceFactory;
+import org.torproject.descriptor.LogDescriptor;
+import org.torproject.descriptor.UnparseableDescriptor;
+import org.torproject.descriptor.WebServerAccessLog;
+
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.List;
+
+(a)RunWith(Parameterized.class)
+public class LogDescriptorTest {
+
+ /** Temporary folder containing all files for this test. */
+ @Rule
+ public TemporaryFolder temp = new TemporaryFolder();
+
+ /** Directory containing two input descriptor files. */
+ protected File indir;
+
+ /** Descriptor reader used in this test. */
+ protected DescriptorReader reader
+ = DescriptorSourceFactory.createDescriptorReader();
+
+ protected int size;
+ protected String[] pan;
+ protected Class<LogDescriptor> type;
+ protected boolean isDecompressionTest;
+
+ /** All types of data that can be encountered during sync. */
+ @Parameters
+ public static Collection<Object[]> pathAndName() {
+ return Arrays.asList(new Object[][] {
+ {Boolean.TRUE, 1878, new String[]{"meronense.torproject.org",
+ "metrics.torproject.org_meronense.torproject.org_access.log"
+ + "_20170530.gz",
+ "metrics.torproject.org", "20170530", "gz"},
+ WebServerAccessLog.class},
+ {Boolean.FALSE, 1878, new String[]{"meronense.torproject.org",
+ "xy.host.org_meronense.torproject.org_access.log_20170530.log",
+ "metrics.torproject.org", "20170530", "xz"},
+ WebServerAccessLog.class},
+ {Boolean.TRUE, 70730, new String[]{"archeotrichon.torproject.org",
+ "archive.torproject.org_archeotrichon.torproject.org_access.log_"
+ + "20151007.xz",
+ "archive.torproject.org", "20151007", "xz"},
+ WebServerAccessLog.class},
+ {Boolean.TRUE, 0, new String[]{"dummy.host.net",
+ "nix.server.org_dummy.host.net_access.log_20111111.bz2",
+ "nix.server.org", "20111111", "bz2"},
+ WebServerAccessLog.class}});
+ }
+
+ /** This constructor receives the above defined data for each run. */
+ public LogDescriptorTest(boolean decompression, int size, String[] pan,
+ Class<LogDescriptor> type) {
+ this.pan = pan;
+ this.size = size;
+ this.type = type;
+ this.isDecompressionTest = decompression;
+ }
+
+ /** Prepares the temporary folder and writes files to it for this test. */
+ private void createTemporaryFolderAndContents() throws IOException {
+ this.indir = this.temp.newFolder();
+ String path = this.pan[0];
+ String name = this.pan[1];
+ File logdir = new File(indir, path);
+ logdir.mkdir();
+ File accessLogFile = new File(logdir, name);
+ Files.copy(getClass().getClassLoader().getResource(path + "/" + name)
+ .openStream(), accessLogFile.toPath());
+ }
+
+ /** Read the test files. */
+ @Before
+ public void readAll() throws IOException {
+ createTemporaryFolderAndContents();
+ Iterator<Descriptor> descs = this.reader
+ .readDescriptors(this.indir).iterator();
+ while (descs.hasNext()) {
+ descs.next();
+ }
+ }
+
+ protected List<Descriptor> retrieve() throws Exception {
+ assertEquals(1, this.reader.getParsedFiles().size());
+ File logFile = new File(this.reader.getParsedFiles().firstKey());
+ byte[] raw = Files.readAllBytes(logFile.toPath());
+ DescriptorParser dp = DescriptorSourceFactory.createDescriptorParser();
+ List<Descriptor> descs = new ArrayList<>();
+ for (Descriptor desc
+ : dp.parseDescriptors(raw, logFile, logFile.getName())) {
+ descs.add(desc);
+ }
+ return descs;
+ }
+
+ @Test
+ public void testParsing() throws Exception {
+ List<Descriptor> descs = retrieve();
+ assertTrue("Wrong type. " + dataUsed(),
+ (descs.get(0) instanceof LogDescriptor));
+ InternalLogDescriptor ld = (InternalLogDescriptor) descs.get(0);
+ assertEquals("Wrong compression type string. " + dataUsed(),
+ pan[4], ld.getCompressionType());
+ }
+
+ private String dataUsed() {
+ return "Used data: " + Arrays.toString(pan);
+ }
+
+ @Test
+ public void testUnknownLogType() throws Exception {
+ assertEquals(dataUsed(), 1, this.reader.getParsedFiles().size());
+ File logFile = new File(this.reader.getParsedFiles().firstKey());
+ DescriptorParser dp = DescriptorSourceFactory.createDescriptorParser();
+ File invalidFile = new File(this.reader.getParsedFiles().firstKey()
+ .replace("access", "-"));
+ List<Descriptor> descs = new ArrayList<>();
+ for (Descriptor desc // note: only 'invalidFile' is used by LogDescriptor
+ : dp.parseDescriptors(new byte[]{}, invalidFile, logFile.getName())) {
+ descs.add(desc);
+ }
+ assertTrue(dataUsed() + "\nWrong type: "
+ + Arrays.toString(descs.get(0).getClass().getInterfaces()),
+ (descs.get(0) instanceof UnparseableDescriptor));
+ }
+
+ @Test
+ public void testCompressionInvalid() throws Exception {
+ if (!isDecompressionTest) {
+ return;
+ }
+ assertEquals(1, this.reader.getParsedFiles().size());
+ File logFile = new File(this.reader.getParsedFiles().firstKey());
+ byte[] raw = Files.readAllBytes(logFile.toPath());
+ for (int i = 0; i < 3; i++) {
+ raw[0] = (byte) i;
+ }
+ DescriptorParser dp = DescriptorSourceFactory.createDescriptorParser();
+ List<Descriptor> descs = new ArrayList<>();
+ for (Descriptor desc
+ : dp.parseDescriptors(raw, logFile, logFile.getName())) {
+ descs.add(desc);
+ }
+ assertTrue(dataUsed() + "\nWrong type: "
+ + Arrays.toString(descs.get(0).getClass().getInterfaces()),
+ (descs.get(0) instanceof UnparseableDescriptor));
+ assertArrayEquals(dataUsed(), raw, descs.get(0).getRawDescriptorBytes());
+ }
+}
+
diff --git a/src/test/java/org/torproject/descriptor/log/WebServerAccessLogLineTest.java b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogLineTest.java
new file mode 100644
index 0000000..ec23b61
--- /dev/null
+++ b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogLineTest.java
@@ -0,0 +1,140 @@
+/* Copyright 2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import static org.junit.Assert.assertEquals;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import java.util.Arrays;
+import java.util.Collection;
+
+(a)RunWith(Parameterized.class)
+public class WebServerAccessLogLineTest {
+
+ /** Test data structure:
+ * reference date, real log line, cleaned line, is valid.
+ */
+ @Parameters
+ public static Collection<Object[]> logData() {
+ return Arrays.asList(new Object[][] {
+ { "0.0.0.0 - - [22/Jan/2018:00:00:00 +0000] \"GET "
+ + "/collector/archive HTTP/1.1\" 301 -",
+ "0.0.0.0 - - [22/Jan/2018:00:00:00 +0000] \"GET "
+ + "/collector/archive HTTP/1.1\" 301 -", Boolean.TRUE},
+ { "0.0.0.0 - - [22/Jan/2018:00:00:00 +0000] \"GET "
+ + "/collector/archive HTTP/1.1\" 301 X \"ccc\"",
+ "", Boolean.FALSE},
+ { "123.98.100.23 xyz xyz [22/Jan/2018:01:20:03 +0000] \"GET "
+ + "/collector/archive HTTP/1.1\" 301 - xyz abc xxxXXXXXXXX",
+ "123.98.100.23 - - [22/Jan/2018:00:00:00 +0000] \"GET "
+ + "/collector/archive HTTP/1.1\" 301 -", Boolean.TRUE},
+ { "127.0.0.1 abc xyz [03/May/2017:06:07:08 +0000] "
+ + "\"GET /server-status HTTP/1.1\" 303 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "127.0.0.1 - - [03/May/2017:00:00:00 +0000] \"GET /server-status"
+ + " HTTP/1.1\" 303 294", Boolean.TRUE},
+ { "127.0.0.1 abc xyz [03/May/2017:06:07:08 +0000] "
+ + "\"GET /server-status?auto HTTP/1.1\" 303 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "127.0.0.1 - - [03/May/2017:00:00:00 +0000] \"GET /server-status"
+ + "?auto HTTP/1.1\" 303 294", Boolean.TRUE},
+ { "42.41.40.39 - - [04/May/2017:06:07:08 +0000] "
+ + "\"HEAD /server-status?auto HTTP/1.1\" 200 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "42.41.40.39 - - [04/May/2017:00:00:00 +0000] \"HEAD /server-status"
+ + "?auto HTTP/1.1\" 200 294", Boolean.TRUE},
+ { "42.41.39 - - [04/May/2017:06:07:08 +0000] "
+ + "\"HEAD /server-status?auto HTTP/1.1\" 200 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "", Boolean.FALSE},
+ { "42.41.40.1039 - - [04/May/2017:06:07:08 +0000] "
+ + "\"HEAD /server-status?auto HTTP/1.1\" 200 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "", Boolean.FALSE},
+ { "42.41.40_039 - - [04/May/2017:06:07:08 +0000] "
+ + "\"HEAD /server-status?auto HTTP/1.1\" 200 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "", Boolean.FALSE},
+ { "0.0.0.2 - - [05/May/2017:15:16:17 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "404 536 \"-\" \"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;"
+ + " SLCC1; .NET CLR 2.0.50; Media Center PC 5.0; .NET CLR 3.5.2;\"",
+ "0.0.0.2 - - [05/May/2017:00:00:00 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "404 536",
+ Boolean.TRUE},
+ { "0.0.0.99 - - [05/June/2017:15:16:17 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico FTP/1.0\" "
+ + "300 536 \"-\" \"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;"
+ + " SLCC1; .NET CLR 2.0.50; Media Center PC 5.0; .NET CLR 3.5.2;\"",
+ "", Boolean.FALSE},
+ { "0.0.0.99 - - [05/Jun/2017:15:16:17 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico FTP/1.0\" "
+ + "300 536 \"-\" \"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;"
+ + " SLCC1; .NET CLR 2.0.50; Media Center PC 5.0; .NET CLR 3.5.2;\"",
+ "0.0.0.99 - - [05/Jun/2017:00:00:00 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico FTP/1.0\" 300 536",
+ Boolean.TRUE},
+ { "0.0.0.7 - - [06/May/2017:00:16:17 +0100] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536 \"-\" \"Mozilla/4.0 (compatible; Opera 7.0; Windows 6.0;"
+ + " funky values ; \"",
+ "0.0.0.7 - - [05/May/2017:00:00:00 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536", Boolean.TRUE},
+ { "0.0.0.1 - - [07/Dec/2016:20:16:18 -1000] "
+ + "\"GET http://t3.torproject.org/?query=what HTTP/1.1\" "
+ + "200 777 \"-\" \"Mozilla/4.0 (compatible; MSIE 7.0; Windows 10;"
+ + " SLCC1; .NET CLR 2.0; Media Center PC 5.0; .NET CLR 3.5.2)\"",
+ "0.0.0.1 - - [08/Dec/2016:00:00:00 +0000] "
+ + "\"GET http://t3.torproject.org/?query=what HTTP/1.1\" 200 777",
+ Boolean.TRUE},
+ { "abcdefghijklmnop1234567890", "", Boolean.FALSE},
+ { "", "", Boolean.FALSE},
+ { "0.0.0.7 - - [06/May/2017:00:16:17 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536 \"-\" \"Mozilla/4.0 (compatible; Opera 7.0; Windows 8.0;",
+ "0.0.0.7 - - [06/May/2017:00:00:00 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536", Boolean.TRUE},
+ { "0.0.0.7 - - [06/May/2017:00:16:17 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536 \"-\" \"Mozilla/4.0 (compatible; Opera 7.0; Windows XT;",
+ "0.0.0.7 - - [06/May/2017:00:00:00 +0000] "
+ + "\"GET http://metrics.torproject.org/favicon.ico HTTP/1.1\" "
+ + "333 536", Boolean.TRUE},
+ { "0.0.0.0 - - [08/May/2017:00:00:00 +0000] "
+ + "\"GET /server-status HTTP/1.1\" 200 1294",
+ "0.0.0.0 - - [08/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1294", Boolean.TRUE}
+ });
+ }
+
+ @Parameter(0)
+ public String real;
+
+ @Parameter(1)
+ public String clean;
+
+ @Parameter(2)
+ public boolean valid;
+
+ @Test
+ public void testValidation() {
+ WebServerAccessLogLine line = WebServerAccessLogLine.makeLine(real);
+ assertEquals("Failed on line: " + real, valid, line.isValid());
+ assertEquals("Failed on line: " + real, clean, line.toLogString());
+ if (valid && !"".equals(clean)) { // A cleaned, accepted line is valid.
+ assertEquals("Failed on line: " + clean, clean,
+ WebServerAccessLogLine.makeLine(clean).toLogString());
+ }
+ }
+
+}
+
diff --git a/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java
new file mode 100644
index 0000000..3e98f13
--- /dev/null
+++ b/src/test/java/org/torproject/descriptor/log/WebServerAccessLogTest.java
@@ -0,0 +1,94 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import static org.junit.Assert.assertEquals;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.Collection;
+
+(a)RunWith(Parameterized.class)
+public class WebServerAccessLogTest {
+
+ /** Test data structure: given line, cleaned line, valid, filename. */
+ @Parameters
+ public static Collection<Object[]> logData() {
+ return Arrays.asList(new Object[][] {
+ { "0.0.0.0 - - [20/Sep/2017:00:00:00 +0000] "
+ + "\"GET /fonts/WOFF/OTF/SourceSansPro-It.otf.woff HTTP/1.1\" "
+ + "200 50556 \"https://metrics.torproject.org/\" \"-\" -",
+ "0.0.0.0 - - [20/Sep/2017:00:00:00 +0000] \"GET "
+ + "/fonts/WOFF/OTF/SourceSansPro-It.otf.woff HTTP/1.1\" 200 50556\n",
+ Boolean.TRUE, "virt.host0_phys.host1a_access.log_20170920"},
+ { "127.0.0.1 qwer 123 [30/May/2017:06:07:08 +0000] "
+ + "\"GET /server-status?auto HTTP/1.1\" 333 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET /server-status"
+ + " HTTP/1.1\" 333 294\n", Boolean.TRUE,
+ "virt.host1_phys.host2a_access.log_20170530"},
+ { "0.0.0.3 abc 567 [30/May/2017:06:07:08 +0000] "
+ + "\"GET /server-status?auto HTTP/1.1\" 333 294 "
+ + "\"-\" \"munin/2.0.25-1+deb8u3 (libwww-perl/6.08)\"",
+ "0.0.0.3 - - [30/May/2017:00:00:00 +0000] \"GET /server-status"
+ + " HTTP/1.1\" 333 294\n", Boolean.TRUE,
+ "virt-host1_phys.host2a_access.log_20170530"},
+ { "11.22.33.44 - - [30/Jul/2017:15:16:17 +0000] "
+ + "\"GET http://www.torproject.org/favicon.ico HTTP/1.1\" "
+ + "100 536 \"-\" \"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;"
+ + " SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.2; "
+ + ".NET CLR 3.5.30729; .NET CLR 3.0.30618)\"",
+ "0.0.0.0 - - [30/Jul/2017:00:00:00 +0000] "
+ + "\"GET http://www.torproject.org/favicon.ico HTTP/1.1\" "
+ + "100 536\n", Boolean.TRUE,
+ "virt.host1_phys.host2b_access.log_20170730"},
+ { "abcdefghijklmnop1234567890", "", Boolean.FALSE,
+ "vhost1_phys.host2c_access.log_20170731.log"},
+ { "", "", Boolean.FALSE, "host2d_host1_access.log_20170731.log"},
+ { "0.0.0.0 - - [30/May/2017:00:00:00 +0000] "
+ + "\"GET /server-status HTTP/1.1\" 200 1294 \"-\" \"-\" -",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1294\n", Boolean.TRUE,
+ "some/other/path/virtual_physical_access.log_20170530.log"}
+ });
+ }
+
+ private String real;
+ private String clean;
+ private int count;
+ private boolean valid;
+ private String fn;
+ private File file;
+
+ /** Set the above test data. */
+ public WebServerAccessLogTest(String in, String out, boolean valid,
+ String filename) {
+ this.real = in;
+ this.clean = out;
+ this.valid = valid;
+ this.fn = filename;
+ this.file = new File(filename);
+ }
+
+ @Test
+ public void testValidation() throws Exception {
+ WebServerAccessLogImpl wsal
+ = new WebServerAccessLogImpl(real.getBytes(), file);
+ wsal.validate();
+ if (valid) {
+ assertEquals(0, wsal.getUnrecognizedLines().size());
+ } else {
+ if (!real.isEmpty()) {
+ assertEquals(real, wsal.getUnrecognizedLines().get(0));
+ }
+ }
+ }
+
+}
+
diff --git a/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java b/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java
new file mode 100644
index 0000000..a11bc30
--- /dev/null
+++ b/src/test/java/org/torproject/descriptor/log/WebServerModuleTest.java
@@ -0,0 +1,113 @@
+/* Copyright 2017--2018 The Tor Project
+ * See LICENSE for licensing information */
+
+package org.torproject.descriptor.log;
+
+import static org.junit.Assert.assertEquals;
+
+import org.torproject.descriptor.DescriptorParseException;
+
+import org.hamcrest.Matchers;
+
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+/** This class contains various tests for the webstats module. */
+public class WebServerModuleTest {
+
+ @Rule
+ public ExpectedException thrown = ExpectedException.none();
+
+ @Test
+ public void testWrongFormat() throws Exception {
+ String filename = "h1_phys1_access.log_no-date.log";
+ thrown.expect(DescriptorParseException.class);
+ thrown.expectMessage(Matchers
+ .containsString("Cannot parse WebServerAccessLog file: "
+ + filename));
+ new WebServerAccessLogImpl(new byte[0],
+ new File(filename));
+ }
+
+ @Test
+ public void testDateFormat() throws Exception {
+ String filename = "h2_phys2_access.log_05001713";
+ thrown.expect(DescriptorParseException.class);
+ thrown.expectMessage(Matchers
+ .containsString("Cannot parse WebServerAccessLog file: "
+ + filename));
+ new WebServerAccessLogImpl(new byte[0],
+ new File(filename));
+ }
+
+ @Test
+ public void testNoParentPathRoot() throws Exception {
+ String filename = "h3_access.log_05001213";
+ thrown.expect(DescriptorParseException.class);
+ thrown.expectMessage(Matchers
+ .containsString("WebServerAccessLog "
+ + "file name doesn't comply to standard: " + filename));
+ new WebServerAccessLogImpl(new byte[0],
+ new File(filename));
+ }
+
+ @Test
+ public void testNoParentPathThis() throws Exception {
+ String filename = "_h3_access.log_05001213";
+ thrown.expect(DescriptorParseException.class);
+ thrown.expectMessage(Matchers
+ .containsString("WebServerAccessLog "
+ + "file name doesn't comply to standard: " + filename));
+ new WebServerAccessLogImpl(new byte[0],
+ new File(filename));
+ }
+
+ @Test
+ public void testNoParentPathParent() throws Exception {
+ String filename = "h3__access.log_05001213";
+ thrown.expect(DescriptorParseException.class);
+ thrown.expectMessage(Matchers
+ .containsString("WebServerAccessLog "
+ + "file name doesn't comply to standard: " + filename));
+ new WebServerAccessLogImpl(new byte[0],
+ new File(filename));
+ }
+
+ private static String[] logLines = {
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1205",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1203",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1207",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1204",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1202",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1206",
+ "0.0.0.0 - - [30/May/2017:00:00:00 +0000] \"GET "
+ + "/server-status HTTP/1.1\" 200 1201"
+ };
+
+ private static String logText = Arrays.asList(logLines).stream()
+ .map((String line) -> line + (" some content"))
+ .collect(Collectors.joining("\n"));
+
+ @Test
+ public void testBasics() throws Exception {
+ WebServerAccessLogImpl wsal = new WebServerAccessLogImpl(logText.getBytes(),
+ new File("vhost_host7_access.log_20170530"));
+ assertEquals(wsal.getAnnotations().size(), 0);
+ assertEquals(logText, new String(wsal.getRawDescriptorBytes()));
+ assertEquals("host7", wsal.getPhysicalHost());
+ assertEquals("vhost", wsal.getVirtualHost());
+ }
+
+}
+
diff --git a/src/test/resources/archeotrichon.torproject.org/archive.torproject.org_ar… b/src/test/resources/archeotrichon.torproject.org/archive.torproject.org_ar…
new file mode 100644
index 0000000..b459742
Binary files /dev/null and b/src/test/resources/archeotrichon.torproject.org/archive.torproject.org_ar… differ
diff --git a/src/test/resources/dummy.host.net/nix.server.org_dummy.host.net_access.lo… b/src/test/resources/dummy.host.net/nix.server.org_dummy.host.net_access.lo…
new file mode 100644
index 0000000..17f335d
Binary files /dev/null and b/src/test/resources/dummy.host.net/nix.server.org_dummy.host.net_access.lo… differ
diff --git a/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone… b/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone…
new file mode 100644
index 0000000..8c2333b
Binary files /dev/null and b/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone… differ
diff --git a/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone… b/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone…
new file mode 100644
index 0000000..8c2333b
Binary files /dev/null and b/src/test/resources/meronense.torproject.org/metrics.torproject.org_merone… differ
diff --git a/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj… b/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj…
new file mode 100644
index 0000000..eee478b
--- /dev/null
+++ b/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj…
@@ -0,0 +1,26 @@
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2873 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2877 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:39:44 +0000] "GET / HTTP/1.1" 200 868 "-" "check_http/v2.1.1 (monitoring-plugins 2.1.1)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:42:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:49:50 +0000] "GET / HTTP/1.1" 200 868 "-" "Wget/1.15 (linux-gnu)"
+123.456.789.0 - - [30/May/2017:06:52:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2872 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:54:44 +0000] "GET / HTTP/1.1" 200 868 "-" "check_http/v2.1.1 (monitoring-plugins 2.1.1)"
+123.456.789.0 - - [30/May/2017:06:56:54 +0000] "-" 408 0 "-" "-"
+123.456.789.0 - - [30/May/2017:06:57:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2876 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+
diff --git a/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj… b/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj…
new file mode 100644
index 0000000..eee478b
--- /dev/null
+++ b/src/test/resources/meronense.torproject.org/xy.host.org_meronense.torproj…
@@ -0,0 +1,26 @@
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2875 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:32:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2873 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2877 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:37:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:39:44 +0000] "GET / HTTP/1.1" 200 868 "-" "check_http/v2.1.1 (monitoring-plugins 2.1.1)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:42:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:42:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2795 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2878 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:47:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2880 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:49:50 +0000] "GET / HTTP/1.1" 200 868 "-" "Wget/1.15 (linux-gnu)"
+123.456.789.0 - - [30/May/2017:06:52:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2872 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "apache-munin/1.0libwww-perl/6.08"
+123.456.789.0 - - [30/May/2017:06:52:40 +0000] "GET /server-status?auto HTTP/1.1" 200 2874 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+123.456.789.0 - - [30/May/2017:06:54:44 +0000] "GET / HTTP/1.1" 200 868 "-" "check_http/v2.1.1 (monitoring-plugins 2.1.1)"
+123.456.789.0 - - [30/May/2017:06:56:54 +0000] "-" 408 0 "-" "-"
+123.456.789.0 - - [30/May/2017:06:57:39 +0000] "GET /server-status?auto HTTP/1.1" 200 2876 "-" "munin/2.0.25-1+deb8u3 (libwww-perl/6.08)"
+
1
0
commit 92567231daea83b07453a77b86abece823b83567
Author: Karsten Loesing <karsten.loesing(a)gmx.net>
Date: Tue Jan 30 11:48:43 2018 +0100
Rename packages.
- Rename root package org.torproject.metrics to
org.torproject.metrics.stats to make it part of the Tor Metrics
name space.
- ernie.cron becomes org.torproject.metrics.stats.servers to get rid
of the legacy name and to reflect that it's all about servers and
bandwidth (provided by servers).
- org.torproject.metrics.web remains unchanged.
- org.torproject.metrics.web.graphs and
org.torproject.metrics.web.research are merged into
org.torproject.metrics.web.
- Remove special treatment for legacy package names, which is not
necessary anymore.
Resolves #24294.
---
build.xml | 9 +++------
.../java/org/torproject/metrics/{ => stats}/advbwdist/Main.java | 2 +-
.../java/org/torproject/metrics/{ => stats}/clients/Main.java | 2 +-
.../org/torproject/metrics/{ => stats}/collectdescs/Main.java | 2 +-
.../org/torproject/metrics/{ => stats}/connbidirect/Main.java | 2 +-
.../org/torproject/metrics/{ => stats}/hidserv/Aggregator.java | 2 +-
.../metrics/{ => stats}/hidserv/ComputedNetworkFractions.java | 2 +-
.../torproject/metrics/{ => stats}/hidserv/DateTimeHelper.java | 2 +-
.../org/torproject/metrics/{ => stats}/hidserv/Document.java | 2 +-
.../torproject/metrics/{ => stats}/hidserv/DocumentStore.java | 2 +-
.../metrics/{ => stats}/hidserv/ExtrapolatedHidServStats.java | 2 +-
.../org/torproject/metrics/{ => stats}/hidserv/Extrapolator.java | 2 +-
.../java/org/torproject/metrics/{ => stats}/hidserv/Main.java | 2 +-
.../java/org/torproject/metrics/{ => stats}/hidserv/Parser.java | 2 +-
.../metrics/{ => stats}/hidserv/ReportedHidServStats.java | 2 +-
.../org/torproject/metrics/{ => stats}/hidserv/Simulate.java | 2 +-
.../java/org/torproject/metrics/{ => stats}/onionperf/Main.java | 2 +-
.../{ernie/cron => metrics/stats/servers}/Configuration.java | 2 +-
.../stats/servers}/ConsensusStatsFileHandler.java | 2 +-
.../{ernie/cron => metrics/stats/servers}/LockFile.java | 2 +-
.../cron => metrics/stats/servers}/LoggingConfiguration.java | 2 +-
.../torproject/{ernie/cron => metrics/stats/servers}/Main.java | 4 +---
.../stats/servers}/RelayDescriptorDatabaseImporter.java | 2 +-
.../java/org/torproject/metrics/{ => stats}/webstats/Main.java | 2 +-
.../org/torproject/metrics/web/{graphs => }/BubblesServlet.java | 5 +----
.../java/org/torproject/metrics/web/{graphs => }/Countries.java | 2 +-
.../torproject/metrics/web/{graphs => }/GraphImageServlet.java | 2 +-
.../metrics/web/{graphs => }/GraphParameterChecker.java | 5 +----
src/main/java/org/torproject/metrics/web/GraphServlet.java | 3 ---
src/main/java/org/torproject/metrics/web/News.java | 2 --
src/main/java/org/torproject/metrics/web/NewsServlet.java | 2 --
.../java/org/torproject/metrics/web/{graphs => }/RObject.java | 2 +-
.../torproject/metrics/web/{graphs => }/RObjectGenerator.java | 5 +----
.../metrics/web/{research => }/ResearchStatsServlet.java | 2 +-
.../metrics/web/{graphs => }/TableParameterChecker.java | 5 +----
src/main/java/org/torproject/metrics/web/TableServlet.java | 3 ---
src/main/resources/web.xml | 8 ++++----
.../torproject/metrics/{ => stats}/connbidirect/MainTest.java | 2 +-
.../org/torproject/metrics/{ => stats}/webstats/MainTest.java | 2 +-
39 files changed, 40 insertions(+), 67 deletions(-)
diff --git a/build.xml b/build.xml
index 996d2a3..c9005d1 100644
--- a/build.xml
+++ b/build.xml
@@ -265,7 +265,7 @@
<!-- Expects legacy.config file in the base directory. -->
<target name="legacy" >
- <property name="module.name" value="legacy" />
+ <property name="module.name" value="servers" />
<property name="localmoddir" value="${modulebase}/${module.name}" />
<property name="statsdir"
value="${localmoddir}/stats" />
@@ -274,10 +274,7 @@
<copy file="${basedir}/legacy.config"
tofile="${localmoddir}/config"/>
- <antcall target="run-java">
- <param name="module.main"
- value="org.torproject.ernie.cron.Main" />
- </antcall>
+ <antcall target="run-java" />
<exec executable="psql"
dir="${localmoddir}"
@@ -443,7 +440,7 @@
<fail unless="have.jar" message="Please run 'ant jar' first."/>
<condition property="mainclass"
value="${module.main}"
- else="org.torproject.metrics.${module.name}.Main" >
+ else="org.torproject.metrics.stats.${module.name}.Main" >
<isset property="module.main"/>
</condition>
<property name="workingdir" value="${modulebase}/${module.name}" />
diff --git a/src/main/java/org/torproject/metrics/advbwdist/Main.java b/src/main/java/org/torproject/metrics/stats/advbwdist/Main.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/advbwdist/Main.java
rename to src/main/java/org/torproject/metrics/stats/advbwdist/Main.java
index 76c701f..6565961 100644
--- a/src/main/java/org/torproject/metrics/advbwdist/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/advbwdist/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.advbwdist;
+package org.torproject.metrics.stats.advbwdist;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorReader;
diff --git a/src/main/java/org/torproject/metrics/clients/Main.java b/src/main/java/org/torproject/metrics/stats/clients/Main.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/clients/Main.java
rename to src/main/java/org/torproject/metrics/stats/clients/Main.java
index 0ad3a49..4f083b9 100644
--- a/src/main/java/org/torproject/metrics/clients/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/clients/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2013--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.clients;
+package org.torproject.metrics.stats.clients;
import org.torproject.descriptor.BandwidthHistory;
import org.torproject.descriptor.BridgeNetworkStatus;
diff --git a/src/main/java/org/torproject/metrics/collectdescs/Main.java b/src/main/java/org/torproject/metrics/stats/collectdescs/Main.java
similarity index 95%
rename from src/main/java/org/torproject/metrics/collectdescs/Main.java
rename to src/main/java/org/torproject/metrics/stats/collectdescs/Main.java
index 3ed0e71..04dc86d 100644
--- a/src/main/java/org/torproject/metrics/collectdescs/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/collectdescs/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.collectdescs;
+package org.torproject.metrics.stats.collectdescs;
import org.torproject.descriptor.DescriptorCollector;
import org.torproject.descriptor.DescriptorSourceFactory;
diff --git a/src/main/java/org/torproject/metrics/connbidirect/Main.java b/src/main/java/org/torproject/metrics/stats/connbidirect/Main.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/connbidirect/Main.java
rename to src/main/java/org/torproject/metrics/stats/connbidirect/Main.java
index fa24e6e..1911472 100644
--- a/src/main/java/org/torproject/metrics/connbidirect/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/connbidirect/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2015--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.connbidirect;
+package org.torproject.metrics.stats.connbidirect;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorReader;
diff --git a/src/main/java/org/torproject/metrics/hidserv/Aggregator.java b/src/main/java/org/torproject/metrics/stats/hidserv/Aggregator.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/Aggregator.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Aggregator.java
index c492f4d..a57fde5 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Aggregator.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Aggregator.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.io.BufferedWriter;
import java.io.File;
diff --git a/src/main/java/org/torproject/metrics/hidserv/ComputedNetworkFractions.java b/src/main/java/org/torproject/metrics/stats/hidserv/ComputedNetworkFractions.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/ComputedNetworkFractions.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/ComputedNetworkFractions.java
index 9f14cf7..48f3679 100644
--- a/src/main/java/org/torproject/metrics/hidserv/ComputedNetworkFractions.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/ComputedNetworkFractions.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.util.Collections;
import java.util.HashMap;
diff --git a/src/main/java/org/torproject/metrics/hidserv/DateTimeHelper.java b/src/main/java/org/torproject/metrics/stats/hidserv/DateTimeHelper.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/hidserv/DateTimeHelper.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/DateTimeHelper.java
index 278ef80..58a785e 100644
--- a/src/main/java/org/torproject/metrics/hidserv/DateTimeHelper.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/DateTimeHelper.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.text.DateFormat;
import java.text.ParseException;
diff --git a/src/main/java/org/torproject/metrics/hidserv/Document.java b/src/main/java/org/torproject/metrics/stats/hidserv/Document.java
similarity index 95%
rename from src/main/java/org/torproject/metrics/hidserv/Document.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Document.java
index af1489a..7c915a1 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Document.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Document.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
/** Common interface of documents that are supposed to be serialized and
* stored in document files and later retrieved and de-serialized. */
diff --git a/src/main/java/org/torproject/metrics/hidserv/DocumentStore.java b/src/main/java/org/torproject/metrics/stats/hidserv/DocumentStore.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/DocumentStore.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/DocumentStore.java
index 488b89e..118071e 100644
--- a/src/main/java/org/torproject/metrics/hidserv/DocumentStore.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/DocumentStore.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.io.BufferedReader;
import java.io.BufferedWriter;
diff --git a/src/main/java/org/torproject/metrics/hidserv/ExtrapolatedHidServStats.java b/src/main/java/org/torproject/metrics/stats/hidserv/ExtrapolatedHidServStats.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/ExtrapolatedHidServStats.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/ExtrapolatedHidServStats.java
index 2d89d15..d4a741d 100644
--- a/src/main/java/org/torproject/metrics/hidserv/ExtrapolatedHidServStats.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/ExtrapolatedHidServStats.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
/** Extrapolated network totals of hidden-service statistics reported by a
* single relay. Extrapolated values are based on reported statistics and
diff --git a/src/main/java/org/torproject/metrics/hidserv/Extrapolator.java b/src/main/java/org/torproject/metrics/stats/hidserv/Extrapolator.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/Extrapolator.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Extrapolator.java
index 2372bed..4bce0e6 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Extrapolator.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Extrapolator.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.io.File;
import java.util.HashSet;
diff --git a/src/main/java/org/torproject/metrics/hidserv/Main.java b/src/main/java/org/torproject/metrics/stats/hidserv/Main.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/hidserv/Main.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Main.java
index f133e7b..a23c17f 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.io.File;
diff --git a/src/main/java/org/torproject/metrics/hidserv/Parser.java b/src/main/java/org/torproject/metrics/stats/hidserv/Parser.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/Parser.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Parser.java
index 9d3f8dd..f2abc78 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Parser.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Parser.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorReader;
diff --git a/src/main/java/org/torproject/metrics/hidserv/ReportedHidServStats.java b/src/main/java/org/torproject/metrics/stats/hidserv/ReportedHidServStats.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/ReportedHidServStats.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/ReportedHidServStats.java
index 1bc2c6b..0ff5fd4 100644
--- a/src/main/java/org/torproject/metrics/hidserv/ReportedHidServStats.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/ReportedHidServStats.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
/* Hidden-service statistics reported by a single relay covering a single
* statistics interval of usually 24 hours. These statistics are reported
diff --git a/src/main/java/org/torproject/metrics/hidserv/Simulate.java b/src/main/java/org/torproject/metrics/stats/hidserv/Simulate.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/hidserv/Simulate.java
rename to src/main/java/org/torproject/metrics/stats/hidserv/Simulate.java
index 1bb1167..a7bc1c3 100644
--- a/src/main/java/org/torproject/metrics/hidserv/Simulate.java
+++ b/src/main/java/org/torproject/metrics/stats/hidserv/Simulate.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.hidserv;
+package org.torproject.metrics.stats.hidserv;
import java.io.BufferedWriter;
import java.io.File;
diff --git a/src/main/java/org/torproject/metrics/onionperf/Main.java b/src/main/java/org/torproject/metrics/stats/onionperf/Main.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/onionperf/Main.java
rename to src/main/java/org/torproject/metrics/stats/onionperf/Main.java
index f77b85d..76c2809 100644
--- a/src/main/java/org/torproject/metrics/onionperf/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/onionperf/Main.java
@@ -1,4 +1,4 @@
-package org.torproject.metrics.onionperf;
+package org.torproject.metrics.stats.onionperf;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorReader;
diff --git a/src/main/java/org/torproject/ernie/cron/Configuration.java b/src/main/java/org/torproject/metrics/stats/servers/Configuration.java
similarity index 99%
rename from src/main/java/org/torproject/ernie/cron/Configuration.java
rename to src/main/java/org/torproject/metrics/stats/servers/Configuration.java
index e223566..5be8d88 100644
--- a/src/main/java/org/torproject/ernie/cron/Configuration.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/Configuration.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron;
+package org.torproject.metrics.stats.servers;
import java.io.BufferedReader;
import java.io.File;
diff --git a/src/main/java/org/torproject/ernie/cron/network/ConsensusStatsFileHandler.java b/src/main/java/org/torproject/metrics/stats/servers/ConsensusStatsFileHandler.java
similarity index 99%
rename from src/main/java/org/torproject/ernie/cron/network/ConsensusStatsFileHandler.java
rename to src/main/java/org/torproject/metrics/stats/servers/ConsensusStatsFileHandler.java
index c47db13..8ccb3b5 100644
--- a/src/main/java/org/torproject/ernie/cron/network/ConsensusStatsFileHandler.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/ConsensusStatsFileHandler.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron.network;
+package org.torproject.metrics.stats.servers;
import org.torproject.descriptor.BridgeNetworkStatus;
import org.torproject.descriptor.Descriptor;
diff --git a/src/main/java/org/torproject/ernie/cron/LockFile.java b/src/main/java/org/torproject/metrics/stats/servers/LockFile.java
similarity index 97%
rename from src/main/java/org/torproject/ernie/cron/LockFile.java
rename to src/main/java/org/torproject/metrics/stats/servers/LockFile.java
index d8ff985..c6c63bb 100644
--- a/src/main/java/org/torproject/ernie/cron/LockFile.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/LockFile.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron;
+package org.torproject.metrics.stats.servers;
import java.io.BufferedReader;
import java.io.BufferedWriter;
diff --git a/src/main/java/org/torproject/ernie/cron/LoggingConfiguration.java b/src/main/java/org/torproject/metrics/stats/servers/LoggingConfiguration.java
similarity index 98%
rename from src/main/java/org/torproject/ernie/cron/LoggingConfiguration.java
rename to src/main/java/org/torproject/metrics/stats/servers/LoggingConfiguration.java
index 9f49dff..f14241b 100644
--- a/src/main/java/org/torproject/ernie/cron/LoggingConfiguration.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/LoggingConfiguration.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron;
+package org.torproject.metrics.stats.servers;
import java.io.IOException;
import java.text.SimpleDateFormat;
diff --git a/src/main/java/org/torproject/ernie/cron/Main.java b/src/main/java/org/torproject/metrics/stats/servers/Main.java
similarity index 95%
rename from src/main/java/org/torproject/ernie/cron/Main.java
rename to src/main/java/org/torproject/metrics/stats/servers/Main.java
index 4e272bc..5db867d 100644
--- a/src/main/java/org/torproject/ernie/cron/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/Main.java
@@ -1,9 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron;
-
-import org.torproject.ernie.cron.network.ConsensusStatsFileHandler;
+package org.torproject.metrics.stats.servers;
import java.io.File;
import java.util.logging.Logger;
diff --git a/src/main/java/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java b/src/main/java/org/torproject/metrics/stats/servers/RelayDescriptorDatabaseImporter.java
similarity index 99%
rename from src/main/java/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java
rename to src/main/java/org/torproject/metrics/stats/servers/RelayDescriptorDatabaseImporter.java
index 99b931c..38afeec 100644
--- a/src/main/java/org/torproject/ernie/cron/RelayDescriptorDatabaseImporter.java
+++ b/src/main/java/org/torproject/metrics/stats/servers/RelayDescriptorDatabaseImporter.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.ernie.cron;
+package org.torproject.metrics.stats.servers;
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorReader;
diff --git a/src/main/java/org/torproject/metrics/webstats/Main.java b/src/main/java/org/torproject/metrics/stats/webstats/Main.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/webstats/Main.java
rename to src/main/java/org/torproject/metrics/stats/webstats/Main.java
index 18433d2..f70963f 100644
--- a/src/main/java/org/torproject/metrics/webstats/Main.java
+++ b/src/main/java/org/torproject/metrics/stats/webstats/Main.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.webstats;
+package org.torproject.metrics.stats.webstats;
import org.apache.commons.compress.compressors.xz.XZCompressorInputStream;
import org.slf4j.Logger;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/BubblesServlet.java b/src/main/java/org/torproject/metrics/web/BubblesServlet.java
similarity index 92%
rename from src/main/java/org/torproject/metrics/web/graphs/BubblesServlet.java
rename to src/main/java/org/torproject/metrics/web/BubblesServlet.java
index 1cea611..8f83184 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/BubblesServlet.java
+++ b/src/main/java/org/torproject/metrics/web/BubblesServlet.java
@@ -1,10 +1,7 @@
/* Copyright 2013--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
-
-import org.torproject.metrics.web.Category;
-import org.torproject.metrics.web.MetricServlet;
+package org.torproject.metrics.web;
import java.io.IOException;
import java.util.ArrayList;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/Countries.java b/src/main/java/org/torproject/metrics/web/Countries.java
similarity index 99%
rename from src/main/java/org/torproject/metrics/web/graphs/Countries.java
rename to src/main/java/org/torproject/metrics/web/Countries.java
index 8472bd7..a2d3cea 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/Countries.java
+++ b/src/main/java/org/torproject/metrics/web/Countries.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
+package org.torproject.metrics.web;
import java.util.ArrayList;
import java.util.List;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/GraphImageServlet.java b/src/main/java/org/torproject/metrics/web/GraphImageServlet.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/web/graphs/GraphImageServlet.java
rename to src/main/java/org/torproject/metrics/web/GraphImageServlet.java
index 473c743..0644fa2 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/GraphImageServlet.java
+++ b/src/main/java/org/torproject/metrics/web/GraphImageServlet.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
+package org.torproject.metrics.web;
import java.io.BufferedOutputStream;
import java.io.IOException;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/GraphParameterChecker.java b/src/main/java/org/torproject/metrics/web/GraphParameterChecker.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/web/graphs/GraphParameterChecker.java
rename to src/main/java/org/torproject/metrics/web/GraphParameterChecker.java
index ed4943b..2168ab5 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/GraphParameterChecker.java
+++ b/src/main/java/org/torproject/metrics/web/GraphParameterChecker.java
@@ -1,10 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
-
-import org.torproject.metrics.web.ContentProvider;
-import org.torproject.metrics.web.Metric;
+package org.torproject.metrics.web;
import java.text.ParseException;
import java.text.SimpleDateFormat;
diff --git a/src/main/java/org/torproject/metrics/web/GraphServlet.java b/src/main/java/org/torproject/metrics/web/GraphServlet.java
index e05e15d..31116b4 100644
--- a/src/main/java/org/torproject/metrics/web/GraphServlet.java
+++ b/src/main/java/org/torproject/metrics/web/GraphServlet.java
@@ -3,9 +3,6 @@
package org.torproject.metrics.web;
-import org.torproject.metrics.web.graphs.Countries;
-import org.torproject.metrics.web.graphs.GraphParameterChecker;
-
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
diff --git a/src/main/java/org/torproject/metrics/web/News.java b/src/main/java/org/torproject/metrics/web/News.java
index 53ca7b4..f21d4e3 100644
--- a/src/main/java/org/torproject/metrics/web/News.java
+++ b/src/main/java/org/torproject/metrics/web/News.java
@@ -3,8 +3,6 @@
package org.torproject.metrics.web;
-import org.torproject.metrics.web.graphs.Countries;
-
import java.util.List;
import java.util.SortedMap;
import java.util.TreeMap;
diff --git a/src/main/java/org/torproject/metrics/web/NewsServlet.java b/src/main/java/org/torproject/metrics/web/NewsServlet.java
index 15717bd..131bd11 100644
--- a/src/main/java/org/torproject/metrics/web/NewsServlet.java
+++ b/src/main/java/org/torproject/metrics/web/NewsServlet.java
@@ -3,8 +3,6 @@
package org.torproject.metrics.web;
-import org.torproject.metrics.web.graphs.Countries;
-
import java.io.IOException;
import java.util.ArrayList;
import java.util.Calendar;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/RObject.java b/src/main/java/org/torproject/metrics/web/RObject.java
similarity index 93%
rename from src/main/java/org/torproject/metrics/web/graphs/RObject.java
rename to src/main/java/org/torproject/metrics/web/RObject.java
index 8f5c231..3e38dd0 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/RObject.java
+++ b/src/main/java/org/torproject/metrics/web/RObject.java
@@ -1,7 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
+package org.torproject.metrics.web;
public class RObject {
diff --git a/src/main/java/org/torproject/metrics/web/graphs/RObjectGenerator.java b/src/main/java/org/torproject/metrics/web/RObjectGenerator.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/web/graphs/RObjectGenerator.java
rename to src/main/java/org/torproject/metrics/web/RObjectGenerator.java
index 4ea8186..06177bf 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/RObjectGenerator.java
+++ b/src/main/java/org/torproject/metrics/web/RObjectGenerator.java
@@ -1,10 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
-
-import org.torproject.metrics.web.ContentProvider;
-import org.torproject.metrics.web.Metric;
+package org.torproject.metrics.web;
import org.rosuda.REngine.Rserve.RConnection;
import org.rosuda.REngine.Rserve.RserveException;
diff --git a/src/main/java/org/torproject/metrics/web/research/ResearchStatsServlet.java b/src/main/java/org/torproject/metrics/web/ResearchStatsServlet.java
similarity index 98%
rename from src/main/java/org/torproject/metrics/web/research/ResearchStatsServlet.java
rename to src/main/java/org/torproject/metrics/web/ResearchStatsServlet.java
index c173000..415e5b5 100644
--- a/src/main/java/org/torproject/metrics/web/research/ResearchStatsServlet.java
+++ b/src/main/java/org/torproject/metrics/web/ResearchStatsServlet.java
@@ -1,7 +1,7 @@
/* Copyright 2013--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.research;
+package org.torproject.metrics.web;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
diff --git a/src/main/java/org/torproject/metrics/web/graphs/TableParameterChecker.java b/src/main/java/org/torproject/metrics/web/TableParameterChecker.java
similarity index 96%
rename from src/main/java/org/torproject/metrics/web/graphs/TableParameterChecker.java
rename to src/main/java/org/torproject/metrics/web/TableParameterChecker.java
index 3a13439..02e9acb 100644
--- a/src/main/java/org/torproject/metrics/web/graphs/TableParameterChecker.java
+++ b/src/main/java/org/torproject/metrics/web/TableParameterChecker.java
@@ -1,10 +1,7 @@
/* Copyright 2011--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.web.graphs;
-
-import org.torproject.metrics.web.ContentProvider;
-import org.torproject.metrics.web.Metric;
+package org.torproject.metrics.web;
import java.text.ParseException;
import java.text.SimpleDateFormat;
diff --git a/src/main/java/org/torproject/metrics/web/TableServlet.java b/src/main/java/org/torproject/metrics/web/TableServlet.java
index e745da0..d37852f 100644
--- a/src/main/java/org/torproject/metrics/web/TableServlet.java
+++ b/src/main/java/org/torproject/metrics/web/TableServlet.java
@@ -3,9 +3,6 @@
package org.torproject.metrics.web;
-import org.torproject.metrics.web.graphs.RObjectGenerator;
-import org.torproject.metrics.web.graphs.TableParameterChecker;
-
import org.apache.commons.lang3.text.StrSubstitutor;
import java.io.IOException;
diff --git a/src/main/resources/web.xml b/src/main/resources/web.xml
index 09ab129..e6e6a69 100644
--- a/src/main/resources/web.xml
+++ b/src/main/resources/web.xml
@@ -86,7 +86,7 @@
<servlet>
<servlet-name>Bubbles</servlet-name>
<servlet-class>
- org.torproject.metrics.web.graphs.BubblesServlet
+ org.torproject.metrics.web.BubblesServlet
</servlet-class>
</servlet>
<servlet-mapping>
@@ -97,7 +97,7 @@
<servlet>
<servlet-name>ResearchStats</servlet-name>
<servlet-class>
- org.torproject.metrics.web.research.ResearchStatsServlet
+ org.torproject.metrics.web.ResearchStatsServlet
</servlet-class>
<init-param>
<param-name>statsDir</param-name>
@@ -114,7 +114,7 @@
<servlet>
<servlet-name>GraphImage</servlet-name>
<servlet-class>
- org.torproject.metrics.web.graphs.GraphImageServlet
+ org.torproject.metrics.web.GraphImageServlet
</servlet-class>
</servlet>
<servlet-mapping>
@@ -408,7 +408,7 @@
<listener>
<listener-class>
- org.torproject.metrics.web.graphs.RObjectGenerator
+ org.torproject.metrics.web.RObjectGenerator
</listener-class>
</listener>
diff --git a/src/test/java/org/torproject/metrics/connbidirect/MainTest.java b/src/test/java/org/torproject/metrics/stats/connbidirect/MainTest.java
similarity index 99%
rename from src/test/java/org/torproject/metrics/connbidirect/MainTest.java
rename to src/test/java/org/torproject/metrics/stats/connbidirect/MainTest.java
index 0f387a6..255cccf 100644
--- a/src/test/java/org/torproject/metrics/connbidirect/MainTest.java
+++ b/src/test/java/org/torproject/metrics/stats/connbidirect/MainTest.java
@@ -1,7 +1,7 @@
/* Copyright 2016--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.connbidirect;
+package org.torproject.metrics.stats.connbidirect;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
diff --git a/src/test/java/org/torproject/metrics/webstats/MainTest.java b/src/test/java/org/torproject/metrics/stats/webstats/MainTest.java
similarity index 98%
rename from src/test/java/org/torproject/metrics/webstats/MainTest.java
rename to src/test/java/org/torproject/metrics/stats/webstats/MainTest.java
index 76268b1..a4e88d1 100644
--- a/src/test/java/org/torproject/metrics/webstats/MainTest.java
+++ b/src/test/java/org/torproject/metrics/stats/webstats/MainTest.java
@@ -1,7 +1,7 @@
/* Copyright 2017--2018 The Tor Project
* See LICENSE for licensing information */
-package org.torproject.metrics.webstats;
+package org.torproject.metrics.stats.webstats;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
1
0
[translation/tor-browser-manual] Update translations for tor-browser-manual
by translation＠torproject.org 31 Jan '18
by translation＠torproject.org 31 Jan '18
31 Jan '18
commit 98c2bf5b85e40e684cf3291fb0296e762e3d6eae
Author: Translation commit bot <translation(a)torproject.org>
Date: Wed Jan 31 08:54:07 2018 +0000
Update translations for tor-browser-manual
---
is/is.po | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/is/is.po b/is/is.po
index c0a97bd12..c7518c42f 100644
--- a/is/is.po
+++ b/is/is.po
@@ -213,7 +213,7 @@ msgstr ""
#: circumvention.page:22
msgid "Using pluggable transports"
-msgstr ""
+msgstr "Nota 'pluggable transport' tengileiðir"
#. This is a reference to an external file such as an image or video. When
#. the file changes, the md5 hash will change to let you know you need to
@@ -577,6 +577,8 @@ msgid ""
"The Tor Browser package is dated January 1, 2000 00:00:00 UTC. This is to "
"ensure that each software build is exactly reproducible."
msgstr ""
+"Tor-vafrapakkinn er dagsettur þann 1. janúar, 2000 kl. 00:00:00 UTC. Þetta "
+"er til að tryggja að hver einasta byggingarútgáfa sé eins."
#: known-issues.page:54
msgid ""
@@ -1152,11 +1154,11 @@ msgstr ""
#: transports.page:6 transports.page:20
msgid "Types of pluggable transport"
-msgstr ""
+msgstr "Gerðir 'pluggable transport' tengileiða"
#: transports.page:10
msgid "Pluggable Transports"
-msgstr ""
+msgstr "Pluggable Transport tengileiðir"
#: transports.page:12
msgid ""
@@ -1171,6 +1173,8 @@ msgid ""
"Currently there are six pluggable transports available, but more are being "
"developed."
msgstr ""
+"Í augnablikinu eru tiltækar sex 'pluggable transport' tengileiðir, en fleiri"
+" eru í undirbúningi."
#: transports.page:28
msgid "obfs3"
1
0
[translation/torbutton-torbuttondtd] Update translations for torbutton-torbuttondtd
by translation＠torproject.org 31 Jan '18
by translation＠torproject.org 31 Jan '18
31 Jan '18
commit 8021515c0e8e3c9dbb124c8eecd241007100dd72
Author: Translation commit bot <translation(a)torproject.org>
Date: Wed Jan 31 08:47:44 2018 +0000
Update translations for torbutton-torbuttondtd
---
is/torbutton.dtd | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/is/torbutton.dtd b/is/torbutton.dtd
index 8a338b596..805968a1f 100644
--- a/is/torbutton.dtd
+++ b/is/torbutton.dtd
@@ -45,6 +45,6 @@
<!ENTITY torbutton.prefs.sec_js_on_https_sites_only "JavaScript er óvirkt á öllum ekki-HTTPS vefjum.">
<!ENTITY torbutton.prefs.sec_js_disabled "JavaScript er sjálfgefið óvirkt á öllum vefsvæðum.">
<!ENTITY torbutton.prefs.sec_limit_typography "Sumt letur og stærðfræðitákn eru óvirk.">
-<!ENTITY torbutton.prefs.sec_limit_graphics_and_typography "Some fonts, icons, math symbols, and images are disabled.">
-<!ENTITY torbutton.prefs.sec_click_to_play_media "Audio and video (HTML5 media) are click-to-play.">
+<!ENTITY torbutton.prefs.sec_limit_graphics_and_typography "Sumt letur, táknmyndir, myndir og stærðfræðitákn eru óvirk.">
+<!ENTITY torbutton.prefs.sec_click_to_play_media "Hljóð og myndskeið (HTML5-gagnamiðlar) þarf að smella á til að spila.">
<!ENTITY torbutton.circuit_display.title "Tor-rás fyrir þetta vefsvæði">
1
0
[translation/tor-launcher-properties] Update translations for tor-launcher-properties
by translation＠torproject.org 31 Jan '18
by translation＠torproject.org 31 Jan '18
31 Jan '18
commit c93dff28b8d012216fcb98e3edd813286f584488
Author: Translation commit bot <translation(a)torproject.org>
Date: Wed Jan 31 08:46:34 2018 +0000
Update translations for tor-launcher-properties
---
is/torlauncher.properties | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/is/torlauncher.properties b/is/torlauncher.properties
index cfa135b8a..ace645ec8 100644
--- a/is/torlauncher.properties
+++ b/is/torlauncher.properties
@@ -6,16 +6,16 @@ torlauncher.error_title=Tor-ræsir
torlauncher.tor_exited_during_startup=Tor exited during startup. This might be due to an error in your torrc file, a bug in Tor or another program on your system, or faulty hardware. Until you fix the underlying problem and restart Tor, Tor Browser will not start.
torlauncher.tor_exited=Tor unexpectedly exited. This might be due to a bug in Tor itself, another program on your system, or faulty hardware. Until you restart Tor, the Tor Browser will not able to reach any websites. If the problem persists, please send a copy of your Tor Log to the support team.
torlauncher.tor_exited2=Endurræsing Tor mun ekki loka flipum vafrans.
-torlauncher.tor_controlconn_failed=Could not connect to Tor control port.
+torlauncher.tor_controlconn_failed=Gat ekki tengst stýrigátt Tor.
torlauncher.tor_failed_to_start=Tor ræstist ekki.
torlauncher.tor_control_failed=Mistókst að ná stjórn yfir Tor.
-torlauncher.tor_bootstrap_failed=Tor failed to establish a Tor network connection.
+torlauncher.tor_bootstrap_failed=Tor mistókst að koma á Tor-nettengingu.
torlauncher.tor_bootstrap_failed_details=%1$S mistókst (%2$S).
torlauncher.unable_to_start_tor=Tókst ekki að ræsa Tor.\n\n%S
torlauncher.tor_missing=Vantar Tor-keyrsluskrána.
-torlauncher.torrc_missing=The torrc file is missing and could not be created.
-torlauncher.datadir_missing=The Tor data directory does not exist and could not be created.
+torlauncher.torrc_missing=Skrána torrc vantar og var ekki hægt að útbúa hana.
+torlauncher.datadir_missing=Tor-gagnamappan er ekki til og var ekki hægt að útbúa hana.
torlauncher.password_hash_missing=Mistókst að fá hakkað lykilorð.
torlauncher.failed_to_get_settings=Tókst ekki að nálgast stillingar fyrir Tor.\n\n%S
@@ -28,8 +28,8 @@ torlauncher.error_bridges_missing=Þú verður að tilgreina eina eða fleiri br
torlauncher.error_default_bridges_type_missing=You must select a transport type for the provided bridges.
torlauncher.error_bridge_bad_default_type=No provided bridges that have the transport type %S are available. Please adjust your settings.
-torlauncher.bridge_suffix.meek-amazon=(works in China)
-torlauncher.bridge_suffix.meek-azure=(works in China)
+torlauncher.bridge_suffix.meek-amazon=(virkar í Kína)
+torlauncher.bridge_suffix.meek-azure=(virkar í Kína)
torlauncher.connect=Tengjast
torlauncher.restart_tor=Endurræsa Tor
@@ -61,4 +61,4 @@ torlauncher.bootstrapWarning.identity=auðkenni samsvara ekki
torlauncher.bootstrapWarning.timeout=tenging féll á tíma
torlauncher.bootstrapWarning.noroute=no route to host
torlauncher.bootstrapWarning.ioerror=les/skrifvilla
-torlauncher.bootstrapWarning.pt_missing=missing pluggable transport
+torlauncher.bootstrapWarning.pt_missing=vantar 'pluggable transport' tengileiðir
1
0
[translation/tor-browser-manual] Update translations for tor-browser-manual
by translation＠torproject.org 31 Jan '18
by translation＠torproject.org 31 Jan '18
31 Jan '18
commit e456e33b286bc77e2e8c9282fd8de1401c8e018d
Author: Translation commit bot <translation(a)torproject.org>
Date: Wed Jan 31 08:20:43 2018 +0000
Update translations for tor-browser-manual
---
is/is.po | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/is/is.po b/is/is.po
index 899b26dd1..c0a97bd12 100644
--- a/is/is.po
+++ b/is/is.po
@@ -570,7 +570,7 @@ msgstr ""
#: known-issues.page:43
msgid "Tor can not use a bridge if a proxy is set."
-msgstr ""
+msgstr "Tor getur ekki notað brú ef milliþjónn er stilltur."
#: known-issues.page:48
msgid ""
1
0