tor-commits August 2017

tor-commits@lists.torproject.org

17 participants
1132 discussions

[metrics-web/master] Update metrics-lib subproject.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 58d1c2bf2873de53d3bc39e8006200dfcb26a338 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Aug 31 21:16:25 2017 +0200 Update metrics-lib subproject. --- submods/metrics-lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submods/metrics-lib b/submods/metrics-lib index 925b4a1..f533c3d 160000 --- a/submods/metrics-lib +++ b/submods/metrics-lib @@ -1 +1 @@ -Subproject commit 925b4a1f80fbe028d6a5bf711d798c0b5ba8e636 +Subproject commit f533c3d91de39857a0fc0fd5b7cae6239dfeb411

1 0

[webwml/master] Fix typo in url in press article
by hiro＠torproject.org 31 Aug '17

31 Aug '17

commit 72eb2dcfa4b5fa5df5e7591159f1e673d192e32a Author: hiromipaw <hiro(a)torproject.org> Date: Thu Aug 31 18:46:56 2017 +0200 Fix typo in url in press article --- press/en/press.wml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/press/en/press.wml b/press/en/press.wml index 6089c1d4..cb642cb1 100644 --- a/press/en/press.wml +++ b/press/en/press.wml @@ -65,7 +65,7 @@ some of the significant Tor-related stories that have popped up.</p> <tr style=""… [View More]

1 0

[webwml/master] Fix typo in press release
by hiro＠torproject.org 31 Aug '17

31 Aug '17

commit 52aaa12090215e5468419fd04c72aa8c52413409 Author: hiromipaw <hiro(a)torproject.org> Date: Thu Aug 31 18:46:01 2017 +0200 Fix typo in press release --- press/en/press.wml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/press/en/press.wml b/press/en/press.wml index 41b57609..6089c1d4 100644 --- a/press/en/press.wml +++ b/press/en/press.wml @@ -63,7 +63,8 @@ some of the significant Tor-related stories that have popped up.</p> </tr> <tr … [View More]

1 0

[webwml/master] Update press release
by hiro＠torproject.org 31 Aug '17

31 Aug '17

commit a74316fe9095a52ae9888324720305cdc9c5ac9d Author: hiromipaw <hiro(a)torproject.org> Date: Thu Aug 31 18:26:45 2017 +0200 Update press release --- press/en/press.wml | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/press/en/press.wml b/press/en/press.wml index 7d0ac691..41b57609 100644 --- a/press/en/press.wml +++ b/press/en/press.wml @@ -51,6 +51,29 @@ some of the significant Tor-related stories that have popped up.</p> </thead> &… [View More]

1 0

[onionoo/master] Bump version to 4.1-1.4.1-dev.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 99723185a504562a55d5f192b8f264147af9ae6c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Aug 31 17:02:54 2017 +0200 Bump version to 4.1-1.4.1-dev. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 921768f..b807e18 100644 --- a/build.xml +++ b/build.xml @@ -10,7 +10,7 @@ <property name="implementation-title" value="Onionoo" /> <property name="onionoo.protocol.version" value="4.1"/> … [View More]

1 0

[onionoo/release] Reset IPv6 exit-policy summary in details status.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 6d23a7ce43e5d88cff132cacd38206ebb5d7e62c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon May 29 09:32:48 2017 +0200 Reset IPv6 exit-policy summary in details status. Fixes #22030. --- CHANGELOG.md | 7 +++++++ .../org/torproject/onionoo/updater/NodeDetailsStatusUpdater.java | 2 ++ 2 files changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b2922d9..8412f87 100644 --- a/… [View More]

1 0

[onionoo/release] Bump version to 4.0-1.3.0-dev.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 9938b622ae833069b513f86a738e4436fc4d8833 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Aug 8 10:50:42 2017 +0200 Bump version to 4.0-1.3.0-dev. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index ef3ae50..a139d4e 100644 --- a/build.xml +++ b/build.xml @@ -10,7 +10,7 @@ <property name="implementation-title" value="Onionoo" /> <property name="onionoo.protocol.version" value="4.0"/> … [View More]

1 0

[onionoo/release] Add change log entry and bump protocol version.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit cdf8fa8daced0ecf364eae4fcd8a600ebe91e475 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Aug 29 11:59:50 2017 +0200 Add change log entry and bump protocol version. --- CHANGELOG.md | 1 + build.xml | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index acfe8df..bb38dd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ server descriptor doesn't contain such a summary anymore. - Remove … [View More]

1 0

[onionoo/release] Add test for exit policy reset.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 6473fdc8dcee245370cc5efa411c177bd1d8cfb2 Author: iwakeh <iwakeh(a)torproject.org> Date: Tue Aug 8 08:50:44 2017 +0000 Add test for exit policy reset. Test part of task-22030. --- .../onionoo/docs/DummyDocumentStore.java | 2 +- .../updater/NodeDetailsStatusUpdaterTest.java | 142 +++++++++++++++++++++ 2 files changed, 143 insertions(+), 1 deletion(-) diff --git a/src/test/java/org/torproject/onionoo/docs/DummyDocumentStore.java b/src/test/java/org/… [View More]torproject/onionoo/docs/DummyDocumentStore.java index 4904044..2f8063e 100644 --- a/src/test/java/org/torproject/onionoo/docs/DummyDocumentStore.java +++ b/src/test/java/org/torproject/onionoo/docs/DummyDocumentStore.java @@ -12,7 +12,7 @@ import java.util.TreeSet; public class DummyDocumentStore extends DocumentStore { - private Map<Class<? extends Document>, SortedMap<String, Document>> + public Map<Class<? extends Document>, SortedMap<String, Document>> storedDocuments = new HashMap<>(); private static final String FINGERPRINT_NULL = ""; diff --git a/src/test/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdaterTest.java b/src/test/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdaterTest.java new file mode 100644 index 0000000..84c1fa6 --- /dev/null +++ b/src/test/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdaterTest.java @@ -0,0 +1,142 @@ +/* Copyright 2014--2017 The Tor Project + * See LICENSE for licensing information */ + +package org.torproject.onionoo.updater; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import org.torproject.descriptor.Descriptor; +import org.torproject.descriptor.DescriptorParser; +import org.torproject.descriptor.DescriptorSourceFactory; +import org.torproject.descriptor.ServerDescriptor; +import org.torproject.onionoo.docs.DetailsStatus; +import org.torproject.onionoo.docs.DocumentStoreFactory; +import org.torproject.onionoo.docs.DummyDocumentStore; + +import org.junit.Before; +import org.junit.Test; + +import java.io.File; +import java.util.List; + +public class NodeDetailsStatusUpdaterTest { + + private DummyDocumentStore docStore; + + @Before + public void createDummyDocumentStore() { + this.docStore = new DummyDocumentStore(); + DocumentStoreFactory.setDocumentStore(this.docStore); + } + + @Test + public void testExitReset() throws Exception { + NodeDetailsStatusUpdater ndsu = new NodeDetailsStatusUpdater(null, null); + DescriptorParser dp = DescriptorSourceFactory.createDescriptorParser(); + String descString = RELAY1 + PUB1 + RELAY2 + POLICY1 + RELAY3; + for (Descriptor desc : dp.parseDescriptors(descString.getBytes(), + new File("dummy"), "dummy")) { + assertTrue(desc.getClass().getName(), desc instanceof ServerDescriptor); + ndsu.processDescriptor(desc, true); + } + assertEquals(1, this.docStore.getPerformedStoreOperations()); + assertEquals(1, this.docStore.storedDocuments.size()); + DetailsStatus dd = this.docStore.getDocument(DetailsStatus.class, FP); + assertNotNull("docs: " + this.docStore.storedDocuments, dd); + assertEquals("Found: " + dd.getExitPolicy(), 1, dd.getExitPolicy().size()); + assertEquals("Found: " + dd.getExitPolicy().get(0), + "reject *:25", dd.getExitPolicy().get(0)); + List<String> acc = dd.getExitPolicyV6Summary().get("accept"); + List<String> rej = dd.getExitPolicyV6Summary().get("reject"); + assertNull(rej); + assertEquals("Found: " + dd.getExitPolicyV6Summary(), "23", acc.get(0)); + assertEquals("Found: " + dd.getExitPolicyV6Summary(), "42", acc.get(1)); + + descString = RELAY1 + PUB2 + RELAY2 + POLICY2 + RELAY3; + for (Descriptor desc : dp.parseDescriptors(descString.getBytes(), + new File("dummy"), "dummy")) { + assertTrue(desc.getClass().getName(), desc instanceof ServerDescriptor); + ndsu.processDescriptor(desc, true); + } + assertEquals(2, this.docStore.getPerformedStoreOperations()); + assertEquals(1, this.docStore.storedDocuments.size()); + dd = this.docStore.getDocument(DetailsStatus.class, FP); + assertNotNull("docs: " + this.docStore.storedDocuments, dd); + assertEquals("Found: " + dd.getExitPolicy(), 1, dd.getExitPolicy().size()); + assertEquals("Found: " + dd.getExitPolicy().get(0), + "reject *:*", dd.getExitPolicy().get(0)); + assertNull(dd.getExitPolicyV6Summary()); + } + + private static final String FP = "42CAF9C0588BBADDD338025E8F2D3CCF35CEEC25"; + + private static final String RELAY1 = "@type server-descriptor 1.0\n" + + "router impedance 84.201.150.89 443 0 0\n" + + "identity-ed25519\n" + + "-----BEGIN ED25519 CERT-----\n" + + "AQQABkIAAS889JQwk/Acb0kWwkX1lolsL72YAGUvNgrLjz8fxczJAQAgBACw4otT\n" + + "Ajr+/0iRTRaCbdHs7BHSZ5qA3c4DG6a71zMzM0ismH/bVLbTuIsRrOuBfpOux+Cs\n" + + "ONO0fGuk5uazrY0iGcKJtZckyW4W7RYLkeGdNnYJ6BpshTEP0Rd61roeBQs=\n" + + "-----END ED25519 CERT-----\n" + + "master-key-ed25519 sOKLUwI6/v9IkU0Wgm3R7OwR0meagN3OAxumu9czMzM\n" + + "platform Tor 0.2.7.6 on Linux\n" + + "protocols Link 1 2 Circuit 1\n"; + + private static final String PUB1 = "published 2016-10-02 15:01:09\n"; + + private static final String PUB2 = "published 2016-10-03 15:01:09\n"; + + private static final String RELAY2 + = "fingerprint 42CA F9C0 588B BADD D338 025E 8F2D 3CCF 35CE EC25\n" + + "uptime 329888\n" + + "bandwidth 10240000 10240000 167976\n" + + "extra-info-digest 43A1C5D7EBA550CEC7465E20E3027772D5C552DD " + + "O3s96zE6cCXRrv3+KKpHm20GATS1s/9jyxbEv91v9mk\n" + + "onion-key\n" + + "-----BEGIN RSA PUBLIC KEY-----\n" + + "MIGJAoGBAMyCYbuR3p6mQDGSjkeO37/BUsGEPiFtKsd7e3m59cXB5oXmEqz0n2dp\n" + + "6qQAfyjPLP7Q1VZvvHxhD9n/qVXOmJRJGdlzhNBx1kIfvFhqKPop68cbwkcOlJ8r\n" + + "c3RjwDiv24q2kVAAq+NsYEhVECeshodf1A0off3J/cNsP6zXHn2HAgMBAAE=\n" + + "-----END RSA PUBLIC KEY-----\n" + + "signing-key\n" + + "-----BEGIN RSA PUBLIC KEY-----\n" + + "MIGJAoGBAMl++uLczhgkVvFLWuPON5Ynk820A/5W4Sm/tAg5klhRj1fvn5SxnSIZ\n" + + "l4VvAHSCZvojqEDCQvWwJUDEfEgT7tSlkq0Xjfsr7IhipnOkqhJFceLVCCdPHkkn\n" + + "L4maXPKcJ+JjhRQABEUM/+HOhBJHWyJ2V1Dc2C/8VISH+bG6BE/1AgMBAAE=\n" + + "-----END RSA PUBLIC KEY-----\n" + + "onion-key-crosscert\n" + + "-----BEGIN CROSSCERT-----\n" + + "AY2c8r2+VIHDo5ga+vSCfOt327496XcMSMJp/9DNQrWiXk+Xwr9z4cTTvcHun7Tc\n" + + "A17/Gjyo2yXDJABOQyZFbbKlCNUaoddOJc/s9hLCnkB//pxkO3sz7BwF+eOxqr9z\n" + + "u4TV8c2yOPqPybsm5ctmNmmP6eecUcNFmBA4uX7QSMs=\n" + + "-----END CROSSCERT-----\n" + + "ntor-onion-key-crosscert 0\n" + + "-----BEGIN ED25519 CERT-----\n" + + "AQoABkGYAbDii1MCOv7/SJFNFoJt0ezsEdJnmoDdzgMbprvXMzMzAAN9vMECBiJ2\n" + + "4Gbu9L3oZ983R5LwaE92TObml1anqe7W5uQDSskkIpEauL8tt029rS+xzQ7pCpdy\n" + + "EgyktjqXGQM=\n" + + "-----END ED25519 CERT-----\n" + + "hidden-service-dir\n" + + "contact dimanne dimanne(a)ya.ru\n" + + "ntor-onion-key DpZv/Br0X2JgxtohD5Kr7A7NgAK4HjBgTbTB+uVSolE=\n"; + + private static final String POLICY1 + = "reject *:25\nipv6-policy accept 23,42\n"; + + private static final String POLICY2 = "reject *:*\n"; + + private static final String RELAY3 + = "router-sig-ed25519 8pfPgYjlpwDoyESOZJHQwMwpmoyWCFg9dcswb8RTra4FT5jgol" + + "HTgkX51h/yUXBx7jUibs2EVaRTOPm9TuiVDA\n" + + "router-signature\n" + + "-----BEGIN SIGNATURE-----\n" + + "el2XtigqzbiSbUW11POx+l4kThP7c12JMyLlMPJXDncEYEa4F0+M0dQQF0BOeEIT\n" + + "D0lj1h4iWT4R+uO5/7umXyAhZidfbhLoQsWa/dGs5BfO4ROgvVLc8o4Za6PYcPE9\n" + + "DXd4yh+SZ86zaAWLUbr1VhRvSLWbFJwNn/aAQdAu70M=\n" + + "-----END SIGNATURE-----\n"; + +} + [View Less]

1 0

[onionoo/release] Add version parameter to filter for Tor version.
by karsten＠torproject.org 31 Aug '17

31 Aug '17

commit 249c8549c0536e7c8e71b5f7f500f81b2f203e6d Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Aug 7 23:06:36 2017 +0200 Add version parameter to filter for Tor version. Implements #21427. Requires minor protocol update. --- .../org/torproject/onionoo/docs/DocumentStore.java | 3 +- .../org/torproject/onionoo/docs/NodeStatus.java | 14 +++++ .../torproject/onionoo/docs/SummaryDocument.java | 13 +++- .../org/torproject/onionoo/server/… [View More]NodeIndex.java | 10 +++ .../org/torproject/onionoo/server/NodeIndexer.java | 8 +++ .../torproject/onionoo/server/RequestHandler.java | 26 ++++++++ .../torproject/onionoo/server/ResourceServlet.java | 24 +++++++- .../onionoo/updater/NodeDetailsStatusUpdater.java | 2 + .../onionoo/writer/SummaryDocumentWriter.java | 3 +- .../onionoo/docs/SummaryDocumentTest.java | 2 +- .../onionoo/server/ResourceServletTest.java | 72 +++++++++++++++++++--- .../server/SummaryDocumentComparatorTest.java | 2 +- 12 files changed, 166 insertions(+), 13 deletions(-) diff --git a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java index 56ca406..7819d0f 100644 --- a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java +++ b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java @@ -435,13 +435,14 @@ public class DocumentStore { } SortedSet<String> relayFlags = new TreeSet<>(); SortedSet<String> family = null; + String version = null; long lastSeenMillis = -1L; long consensusWeight = -1L; long firstSeenMillis = -1L; SummaryDocument summaryDocument = new SummaryDocument(isRelay, nickname, fingerprint, addresses, lastSeenMillis, running, relayFlags, consensusWeight, countryCode, firstSeenMillis, - asNumber, contact, family, family); + asNumber, contact, family, family, version); return summaryDocument; } diff --git a/src/main/java/org/torproject/onionoo/docs/NodeStatus.java b/src/main/java/org/torproject/onionoo/docs/NodeStatus.java index 6a71fb6..759d83e 100644 --- a/src/main/java/org/torproject/onionoo/docs/NodeStatus.java +++ b/src/main/java/org/torproject/onionoo/docs/NodeStatus.java @@ -321,6 +321,16 @@ public class NodeStatus extends Document { return this.recommendedVersion; } + private String version; + + public void setVersion(String version) { + this.version = version.substring(version.lastIndexOf(" ") + 1); + } + + public String getVersion() { + return this.version; + } + /* From exit lists: */ private SortedSet<String> exitAddresses; @@ -547,6 +557,9 @@ public class NodeStatus extends Document { extendedFamily.addAll(indirectFamily); nodeStatus.setExtendedFamily(extendedFamily); } + if (parts.length >= 24 && !parts[23].isEmpty()) { + nodeStatus.setVersion(parts[23]); + } return nodeStatus; } catch (NumberFormatException e) { log.error("Number format exception while parsing node " @@ -610,6 +623,7 @@ public class NodeStatus extends Document { sb.append("\t" + StringUtils.join(this.getAllegedFamily(), ";") + ":" + StringUtils.join(this.getEffectiveFamily(), ";") + ":" + StringUtils.join(this.getIndirectFamily(), ";")); + sb.append("\t" + (this.getVersion() != null ? this.getVersion() : "")); return sb.toString(); } } diff --git a/src/main/java/org/torproject/onionoo/docs/SummaryDocument.java b/src/main/java/org/torproject/onionoo/docs/SummaryDocument.java index 3ca4960..527f91d 100644 --- a/src/main/java/org/torproject/onionoo/docs/SummaryDocument.java +++ b/src/main/java/org/torproject/onionoo/docs/SummaryDocument.java @@ -269,6 +269,16 @@ public class SummaryDocument extends Document { return this.stringArrayToSortedSet(this.ef); } + private String v; + + public void setVersion(String version) { + this.v = version; + } + + public String getVersion() { + return this.v; + } + /* The familyFingerprints parameter can go away after September 8, 2015. * See above. */ /** Instantiates a summary document with all given properties. */ @@ -277,7 +287,7 @@ public class SummaryDocument extends Document { boolean running, SortedSet<String> relayFlags, long consensusWeight, String countryCode, long firstSeenMillis, String asNumber, String contact, SortedSet<String> familyFingerprints, - SortedSet<String> effectiveFamily) { + SortedSet<String> effectiveFamily, String version) { this.setRelay(isRelay); this.setNickname(nickname); this.setFingerprint(fingerprint); @@ -292,6 +302,7 @@ public class SummaryDocument extends Document { this.setContact(contact); this.setFamilyFingerprints(familyFingerprints); this.setEffectiveFamily(effectiveFamily); + this.setVersion(version); } } diff --git a/src/main/java/org/torproject/onionoo/server/NodeIndex.java b/src/main/java/org/torproject/onionoo/server/NodeIndex.java index 439d302..dd4bd4d 100644 --- a/src/main/java/org/torproject/onionoo/server/NodeIndex.java +++ b/src/main/java/org/torproject/onionoo/server/NodeIndex.java @@ -169,5 +169,15 @@ class NodeIndex { public SortedMap<Integer, Set<String>> getBridgesByLastSeenDays() { return bridgesByLastSeenDays; } + + private Map<String, Set<String>> relaysByVersion; + + public void setRelaysByVersion(Map<String, Set<String>> relaysByVersion) { + this.relaysByVersion = relaysByVersion; + } + + public Map<String, Set<String>> getRelaysByVersion() { + return this.relaysByVersion; + } } diff --git a/src/main/java/org/torproject/onionoo/server/NodeIndexer.java b/src/main/java/org/torproject/onionoo/server/NodeIndexer.java index f6b9510..7ed9ea6 100644 --- a/src/main/java/org/torproject/onionoo/server/NodeIndexer.java +++ b/src/main/java/org/torproject/onionoo/server/NodeIndexer.java @@ -150,6 +150,7 @@ public class NodeIndexer implements ServletContextListener, Runnable { Map<String, Set<String>> newBridgesByFlag = new HashMap<>(); Map<String, Set<String>> newRelaysByContact = new HashMap<>(); Map<String, Set<String>> newRelaysByFamily = new HashMap<>(); + Map<String, Set<String>> newRelaysByVersion = new HashMap<>(); SortedMap<Integer, Set<String>> newRelaysByFirstSeenDays = new TreeMap<>(); SortedMap<Integer, Set<String>> newBridgesByFirstSeenDays = new TreeMap<>(); SortedMap<Integer, Set<String>> newRelaysByLastSeenDays = new TreeMap<>(); @@ -246,6 +247,12 @@ public class NodeIndexer implements ServletContextListener, Runnable { } newRelaysByContact.get(contact).add(fingerprint); newRelaysByContact.get(contact).add(hashedFingerprint); + String version = entry.getVersion(); + if (!newRelaysByVersion.containsKey(version)) { + newRelaysByVersion.put(version, new HashSet<String>()); + } + newRelaysByVersion.get(version).add(fingerprint); + newRelaysByVersion.get(version).add(hashedFingerprint); } /* This loop can go away once all Onionoo services had their hourly * updater write effective families to summary documents at least @@ -319,6 +326,7 @@ public class NodeIndexer implements ServletContextListener, Runnable { newNodeIndex.setBridgesByLastSeenDays(newBridgesByLastSeenDays); newNodeIndex.setRelaysPublishedMillis(relaysLastValidAfterMillis); newNodeIndex.setBridgesPublishedMillis(bridgesLastPublishedMillis); + newNodeIndex.setRelaysByVersion(newRelaysByVersion); synchronized (this) { this.lastIndexed = updateStatusMillis; this.latestNodeIndex = newNodeIndex; diff --git a/src/main/java/org/torproject/onionoo/server/RequestHandler.java b/src/main/java/org/torproject/onionoo/server/RequestHandler.java index 98ece58..1e08c24 100644 --- a/src/main/java/org/torproject/onionoo/server/RequestHandler.java +++ b/src/main/java/org/torproject/onionoo/server/RequestHandler.java @@ -7,6 +7,9 @@ import org.torproject.onionoo.docs.DocumentStore; import org.torproject.onionoo.docs.DocumentStoreFactory; import org.torproject.onionoo.docs.SummaryDocument; +import java.io.BufferedWriter; +import java.io.FileWriter; +import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -91,6 +94,12 @@ public class RequestHandler { System.arraycopy(contact, 0, this.contact, 0, contact.length); } + private String version; + + public void setVersion(String version) { + this.version = version; + } + private String[] order; public void setOrder(String[] order) { @@ -158,6 +167,7 @@ public class RequestHandler { this.filterNodesByLastSeenDays(); this.filterByContact(); this.filterByFamily(); + this.filterByVersion(); this.order(); this.offset(); this.limit(); @@ -514,6 +524,22 @@ public class RequestHandler { this.filteredBridges.clear(); } + private void filterByVersion() { + if (null == this.version) { + /* Not filtering by version. */ + return; + } + Set<String> keepRelays = new HashSet<>(); + for (Map.Entry<String, Set<String>> e + : this.nodeIndex.getRelaysByVersion().entrySet()) { + if (e.getKey().startsWith(this.version)) { + keepRelays.addAll(e.getValue()); + } + } + this.filteredRelays.keySet().retainAll(keepRelays); + this.filteredBridges.clear(); + } + private void order() { List<SummaryDocument> uniqueRelays = new ArrayList<>(); List<SummaryDocument> uniqueBridges = new ArrayList<>(); diff --git a/src/main/java/org/torproject/onionoo/server/ResourceServlet.java b/src/main/java/org/torproject/onionoo/server/ResourceServlet.java index 869574c..164b770 100644 --- a/src/main/java/org/torproject/onionoo/server/ResourceServlet.java +++ b/src/main/java/org/torproject/onionoo/server/ResourceServlet.java @@ -3,6 +3,8 @@ package org.torproject.onionoo.server; +import java.io.BufferedWriter; +import java.io.FileWriter; import java.io.IOException; import java.io.PrintWriter; import java.util.ArrayList; @@ -66,7 +68,7 @@ public class ResourceServlet extends HttpServlet { private static Set<String> knownParameters = new HashSet<>( Arrays.asList(("type,running,search,lookup,fingerprint,country,as," + "flag,first_seen_days,last_seen_days,contact,order,limit," - + "offset,fields,family").split(","))); + + "offset,fields,family,version").split(","))); private static Set<String> illegalSearchQualifiers = new HashSet<>(Arrays.asList(("search,fingerprint,order,limit," @@ -275,6 +277,15 @@ public class ResourceServlet extends HttpServlet { } rh.setContact(contactParts); } + if (parameterMap.containsKey("version")) { + String versionParameter = this.parseVersionParameter( + parameterMap.get("version")); + if (null == versionParameter) { + response.sendError(HttpServletResponse.SC_BAD_REQUEST); + return; + } + rh.setVersion(versionParameter); + } if (parameterMap.containsKey("order")) { String[] order = this.parseOrderParameter(parameterMap.get("order")); if (order == null) { @@ -532,5 +543,16 @@ public class ResourceServlet extends HttpServlet { } return parameter.toLowerCase().split(","); } + + private static Pattern versionParameterPattern = + Pattern.compile("^[0-9a-zA-Z\\.-]+$"); + + private String parseVersionParameter(String parameter) { + if (!versionParameterPattern.matcher(parameter).matches()) { + /* Version contains illegal character(s). */ + return null; + } + return parameter; + } } diff --git a/src/main/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdater.java b/src/main/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdater.java index cc50f4b..9fcff35 100644 --- a/src/main/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdater.java +++ b/src/main/java/org/torproject/onionoo/updater/NodeDetailsStatusUpdater.java @@ -283,6 +283,7 @@ public class NodeDetailsStatusUpdater implements DescriptorListener, nodeStatus.setRecommendedVersion((recommendedVersions == null || entry.getVersion() == null) ? null : recommendedVersions.contains(entry.getVersion())); + nodeStatus.setVersion(entry.getVersion()); } if (entry.getUnmeasured()) { if (!this.lastSeenUnmeasured.containsKey(fingerprint) @@ -484,6 +485,7 @@ public class NodeDetailsStatusUpdater implements DescriptorListener, updatedNodeStatus.setAsNumber(nodeStatus.getAsNumber()); updatedNodeStatus.setRecommendedVersion( nodeStatus.getRecommendedVersion()); + updatedNodeStatus.setVersion(nodeStatus.getVersion()); } if (nodeStatus.getFirstSeenMillis() < updatedNodeStatus.getFirstSeenMillis() diff --git a/src/main/java/org/torproject/onionoo/writer/SummaryDocumentWriter.java b/src/main/java/org/torproject/onionoo/writer/SummaryDocumentWriter.java index b2f678d..4364aad 100644 --- a/src/main/java/org/torproject/onionoo/writer/SummaryDocumentWriter.java +++ b/src/main/java/org/torproject/onionoo/writer/SummaryDocumentWriter.java @@ -91,10 +91,11 @@ public class SummaryDocumentWriter implements DocumentWriter { SortedSet<String> declaredFamily = nodeStatus.getDeclaredFamily(); SortedSet<String> effectiveFamily = nodeStatus.getEffectiveFamily(); String nickname = nodeStatus.getNickname(); + String version = nodeStatus.getVersion(); SummaryDocument summaryDocument = new SummaryDocument(isRelay, nickname, fingerprint, addresses, lastSeenMillis, running, relayFlags, consensusWeight, countryCode, firstSeenMillis, - asNumber, contact, declaredFamily, effectiveFamily); + asNumber, contact, declaredFamily, effectiveFamily, version); if (this.documentStore.store(summaryDocument, fingerprint)) { this.writtenDocuments++; } diff --git a/src/test/java/org/torproject/onionoo/docs/SummaryDocumentTest.java b/src/test/java/org/torproject/onionoo/docs/SummaryDocumentTest.java index 413ca83..9d7625a 100644 --- a/src/test/java/org/torproject/onionoo/docs/SummaryDocumentTest.java +++ b/src/test/java/org/torproject/onionoo/docs/SummaryDocumentTest.java @@ -26,7 +26,7 @@ public class SummaryDocumentTest { new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC", "0025C136C1F3A9EEFE2AE3F918F03BFA21B5070B" })), new TreeSet<>(Arrays.asList( - new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" }))); + new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" })), null); } @Test() diff --git a/src/test/java/org/torproject/onionoo/server/ResourceServletTest.java b/src/test/java/org/torproject/onionoo/server/ResourceServletTest.java index 04c5d7d..4ebf0d2 100644 --- a/src/test/java/org/torproject/onionoo/server/ResourceServletTest.java +++ b/src/test/java/org/torproject/onionoo/server/ResourceServletTest.java @@ -142,7 +142,8 @@ public class ResourceServletTest { new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC", "0025C136C1F3A9EEFE2AE3F918F03BFA21B5070B" })), new TreeSet<>(Arrays.asList( - new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" }))); + new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" })), + "0.2.3.25"); org.torproject.onionoo.docs.SummaryDocument relayFerrari458 = new org.torproject.onionoo.docs.SummaryDocument(true, "Ferrari458", "001C13B3A55A71B977CA65EC85539D79C653A3FC", Arrays.asList( @@ -154,7 +155,7 @@ public class ResourceServletTest { new TreeSet<String>(Arrays.asList(new String[] { "000C5F55BD4814B917CC474BD537F1A3B33CCE2A" })), new TreeSet<>(Arrays.asList(new String[] { - "000C5F55BD4814B917CC474BD537F1A3B33CCE2A" }))); + "000C5F55BD4814B917CC474BD537F1A3B33CCE2A" })), "0.2.3.24-rc-dev"); this.relays = new TreeMap<>(); this.relays.put("000C5F55BD4814B917CC474BD537F1A3B33CCE2A", relayTorkaZ); @@ -170,7 +171,7 @@ public class ResourceServletTest { DateTimeHelper.parse("2013-04-16 18:00:00"), "AS6830", "1024D/51E2A1C7 steven j. murdoch " + "<tor+steven.murdoch(a)cl.cam.ac.uk> <fb-token:5sr_k_zs2wm=>", - new TreeSet<String>(), new TreeSet<String>()); + new TreeSet<String>(), new TreeSet<String>(), "0.2.3.25"); this.relays.put("0025C136C1F3A9EEFE2AE3F918F03BFA21B5070B", relayTimMayTribute); this.bridges = new TreeMap<>(); @@ -181,7 +182,7 @@ public class ResourceServletTest { DateTimeHelper.parse("2013-04-21 18:07:03"), false, new TreeSet<>(Arrays.asList(new String[] { "Valid" })), -1L, null, DateTimeHelper.parse("2013-04-20 15:37:04"), null, null, - null, null); + null, null, null); this.bridges.put("0000831B236DFF73D409AD17B40E2A728A53994F", bridgeec2bridgercc7f31fe); org.torproject.onionoo.docs.SummaryDocument bridgeUnnamed = @@ -191,7 +192,7 @@ public class ResourceServletTest { DateTimeHelper.parse("2013-04-20 17:37:04"), false, new TreeSet<>(Arrays.asList(new String[] { "Valid" })), -1L, null, DateTimeHelper.parse("2013-04-14 07:07:05"), null, null, - null, null); + null, null, null); this.bridges.put("0002D9BDBBC230BD9C78FF502A16E0033EF87E0C", bridgeUnnamed); org.torproject.onionoo.docs.SummaryDocument bridgegummy = @@ -202,7 +203,7 @@ public class ResourceServletTest { new TreeSet<>(Arrays.asList(new String[] { "Running", "Valid" })), -1L, null, DateTimeHelper.parse("2013-01-16 21:07:04"), null, null, null, - null); + null, null); this.bridges.put("1FEDE50ED8DBA1DD9F9165F78C8131E4A44AB756", bridgegummy); } @@ -279,7 +280,8 @@ public class ResourceServletTest { int expectedRelaysNumber, String[] expectedRelaysNicknames, int expectedBridgesNumber, String[] expectedBridgesNicknames) { this.runTest(request); - assertNotNull(this.summaryDocument); + assertNotNull("Summary document is null, status code is " + + this.response.errorStatusCode, this.summaryDocument); assertEquals(expectedRelaysNumber, this.summaryDocument.relays.length); if (expectedRelaysNicknames != null) { @@ -1531,5 +1533,61 @@ public class ResourceServletTest { this.assertErrorStatusCode( "/summary?family=00000000000000000000000000000000000000", 400); } + + @Test + public void testVersion02325() { + this.assertSummaryDocument("/summary?version=0.2.3.25", 2, + new String[] { "TorkaZ", "TimMayTribute" }, 0, null); + } + + @Test + public void testVersion02324() { + this.assertSummaryDocument("/summary?version=0.2.3.24-rc-dev", 1, + new String[] { "Ferrari458" }, 0, null); + } + + @Test + public void testVersion12345() { + this.assertSummaryDocument("/summary?version=1.2.3.4.5", 0, null, 0, null); + } + + @Test + public void testVersionBlaBlaBla() { + this.assertSummaryDocument("/summary?version=bla-bla-bla", 0, null, 0, + null); + } + + @Test + public void testVersion0() { + this.assertSummaryDocument("/summary?version=0", 3, null, 0, null); + } + + @Test + public void testVersion02() { + this.assertSummaryDocument("/summary?version=0.2", 3, null, 0, null); + } + + @Test + public void testVersion023() { + this.assertSummaryDocument("/summary?version=0.2.3", 3, null, 0, null); + } + + @Test + public void testVersion0232() { + /* This is correct when comparing strings. */ + this.assertSummaryDocument("/summary?version=0.2.3.2", 3, null, 0, null); + } + + @Test + public void testVersion023Dot() { + /* This is also correct when comparing strings. */ + this.assertSummaryDocument("/summary?version=0.2.3.", 3, null, 0, null); + } + + @Test + public void testVersionStart() { + /* This is also correct when comparing strings. */ + this.assertErrorStatusCode("/summary?version=*", 400); + } } diff --git a/src/test/java/org/torproject/onionoo/server/SummaryDocumentComparatorTest.java b/src/test/java/org/torproject/onionoo/server/SummaryDocumentComparatorTest.java index c1d909f..fb3f5b3 100644 --- a/src/test/java/org/torproject/onionoo/server/SummaryDocumentComparatorTest.java +++ b/src/test/java/org/torproject/onionoo/server/SummaryDocumentComparatorTest.java @@ -40,7 +40,7 @@ public class SummaryDocumentComparatorTest { new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC", "0025C136C1F3A9EEFE2AE3F918F03BFA21B5070B" })), new TreeSet<>(Arrays.asList( - new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" }))); + new String[] { "001C13B3A55A71B977CA65EC85539D79C653A3FC" })), null); } /** Some values for running all comparison types. */ [View Less]

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits August 2017