March 2014 - tor-commits - lists.torproject.org

[onionoo/master] Be more precise about leaving out non-null values.
by karsten＠torproject.org 11 Mar '14

11 Mar '14

commit da05cefbb3f360d00d20cf5becbd6f06963f7d7b Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 6 16:10:35 2014 +0100 Be more precise about leaving out non-null values. The specification said that we're only including non-null values for series of at least two subsequent data points. But that's not true. We're not including histories if there are not at least two subsequent non-null values. But those histories may contain single non-null values with null values previous and next to them. --- web/index.html | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/web/index.html b/web/index.html index f6e749a..88745af 100644 --- a/web/index.html +++ b/web/index.html @@ -1391,8 +1391,8 @@ Array of normalized bandwidth values in bytes per second. May contain null values if the relay did not provide any bandwidth data or only data for less than 20% of a given time period. -Only includes non-null values for series of at least two subsequent data -points to enable drawing of line graphs. +Contains at least two subsequent non-null values to enable drawing of line +graphs. </li> @@ -1618,8 +1618,8 @@ Can also be derived from the number of elements in the subsequent array. Array of normalized weights values. May contain null values if the relay was running less than 20% of a given time period. -Only includes non-null values for series of at least two subsequent data -points to enable drawing of line graphs. +Contains at least two subsequent non-null values to enable drawing of line +graphs. </li>

1 0

[onionoo/master] Fix compressing bandwidth and weights histories.
by karsten＠torproject.org 11 Mar '14

11 Mar '14

commit b8db05c28eeaec8b69a4bc63c1df12d2191b36ae Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 6 16:13:21 2014 +0100 Fix compressing bandwidth and weights histories. We compress histories by merging adjacent intervals to save disk space, unless we want intervals to stay distinct. For example, we might want to compress all intervals on the same UTC day but not want to merge intervals with the previous or next UTC day. However, we had an off-by-one error which made us merge the wrong intervals. For example, we merged intervals from 23:00:00 on one day to 23:00:00 the next day. --- src/org/torproject/onionoo/BandwidthDataWriter.java | 4 ++-- src/org/torproject/onionoo/WeightsDataWriter.java | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/org/torproject/onionoo/BandwidthDataWriter.java b/src/org/torproject/onionoo/BandwidthDataWriter.java index 4291bdd..68bf59a 100644 --- a/src/org/torproject/onionoo/BandwidthDataWriter.java +++ b/src/org/torproject/onionoo/BandwidthDataWriter.java @@ -196,8 +196,8 @@ public class BandwidthDataWriter implements DataWriter, intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; } if (lastEndMillis == startMillis && - (lastEndMillis / intervalLengthMillis) == - (endMillis / intervalLengthMillis)) { + ((lastEndMillis - 1L) / intervalLengthMillis) == + ((endMillis - 1L) / intervalLengthMillis)) { lastEndMillis = endMillis; lastBandwidth += bandwidth; } else { diff --git a/src/org/torproject/onionoo/WeightsDataWriter.java b/src/org/torproject/onionoo/WeightsDataWriter.java index 40c85ed..81b412c 100644 --- a/src/org/torproject/onionoo/WeightsDataWriter.java +++ b/src/org/torproject/onionoo/WeightsDataWriter.java @@ -387,8 +387,8 @@ public class WeightsDataWriter implements DataWriter, DescriptorListener { intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; } if (lastEndMillis == startMillis && - (lastEndMillis / intervalLengthMillis) == - (endMillis / intervalLengthMillis)) { + ((lastEndMillis - 1L) / intervalLengthMillis) == + ((endMillis - 1L) / intervalLengthMillis)) { double lastIntervalInHours = (double) ((lastEndMillis - lastStartMillis) / 60L * 60L * 1000L); double currentIntervalInHours = (double) ((endMillis

1 0

[onionoo/master] Add new document with fractional uptimes.
by karsten＠torproject.org 11 Mar '14

11 Mar '14

commit 7c911367930d6aa4183090c13f8c2a9de4842f56 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Mar 10 14:24:48 2014 +0100 Add new document with fractional uptimes. Prepares for providing monthly relay and bridge statistics (#11041). --- etc/web.xml.template | 4 + src/org/torproject/onionoo/DocumentStore.java | 31 +- src/org/torproject/onionoo/Main.java | 4 +- src/org/torproject/onionoo/ResourceServlet.java | 2 + src/org/torproject/onionoo/ResponseBuilder.java | 28 ++ src/org/torproject/onionoo/UptimeDataWriter.java | 533 ++++++++++++++++++++++ src/org/torproject/onionoo/UptimeDocument.java | 8 + src/org/torproject/onionoo/UptimeStatus.java | 5 + web/index.html | 192 ++++++++ 9 files changed, 800 insertions(+), 7 deletions(-) diff --git a/etc/web.xml.template b/etc/web.xml.template index 53a1878..f69314f 100644 --- a/etc/web.xml.template +++ b/etc/web.xml.template @@ -40,6 +40,10 @@ <servlet-name>Resource</servlet-name> <url-pattern>/clients</url-pattern> </servlet-mapping> + <servlet-mapping> + <servlet-name>Resource</servlet-name> + <url-pattern>/uptime</url-pattern> + </servlet-mapping> </web-app> diff --git a/src/org/torproject/onionoo/DocumentStore.java b/src/org/torproject/onionoo/DocumentStore.java index 1092e25..519b4f2 100644 --- a/src/org/torproject/onionoo/DocumentStore.java +++ b/src/org/torproject/onionoo/DocumentStore.java @@ -124,6 +124,9 @@ public class DocumentStore { } else if (documentType.equals(ClientsStatus.class)) { directory = this.statusDir; subdirectory = "clients"; + } else if (documentType.equals(UptimeStatus.class)) { + directory = this.statusDir; + subdirectory = "uptimes"; } else if (documentType.equals(DetailsDocument.class)) { directory = this.outDir; subdirectory = "details"; @@ -136,6 +139,9 @@ public class DocumentStore { } else if (documentType.equals(ClientsDocument.class)) { directory = this.outDir; subdirectory = "clients"; + } else if (documentType.equals(UptimeDocument.class)) { + directory = this.outDir; + subdirectory = "uptimes"; } if (directory != null && subdirectory != null) { Stack<File> files = new Stack<File>(); @@ -186,7 +192,8 @@ public class DocumentStore { } else if (document instanceof DetailsDocument || document instanceof BandwidthDocument || document instanceof WeightsDocument || - document instanceof ClientsDocument) { + document instanceof ClientsDocument || + document instanceof UptimeDocument) { Gson gson = new Gson(); documentString = gson.toJson(this); } else { @@ -275,7 +282,8 @@ public class DocumentStore { } else if (documentType.equals(DetailsDocument.class) || documentType.equals(BandwidthDocument.class) || documentType.equals(WeightsDocument.class) || - documentType.equals(ClientsDocument.class)) { + documentType.equals(ClientsDocument.class) || + documentType.equals(UptimeDocument.class)) { return this.retrieveParsedDocumentFile(documentType, documentString); } else { @@ -351,12 +359,11 @@ public class DocumentStore { private <T extends Document> File getDocumentFile(Class<T> documentType, String fingerprint) { File documentFile = null; - if (fingerprint == null && - !documentType.equals(UpdateStatus.class)) { + if (fingerprint == null && !documentType.equals(UpdateStatus.class) && + !documentType.equals(UptimeStatus.class)) { // TODO Instead of using the update file workaround, add new method // lastModified(Class<T> documentType) that serves a similar - // purpose. Once that's implemented, make fingerprint mandatory for - // all methods. + // purpose. return null; } File directory = null; @@ -381,6 +388,15 @@ public class DocumentStore { fileName = String.format("clients/%s/%s/%s", fingerprint.substring(0, 1), fingerprint.substring(1, 2), fingerprint); + } else if (documentType.equals(UptimeStatus.class)) { + directory = this.statusDir; + if (fingerprint == null) { + fileName = "uptime"; + } else { + fileName = String.format("uptimes/%s/%s/%s", + fingerprint.substring(0, 1), fingerprint.substring(1, 2), + fingerprint); + } } else if (documentType.equals(UpdateStatus.class)) { directory = this.outDir; fileName = "update"; @@ -401,6 +417,9 @@ public class DocumentStore { } else if (documentType.equals(ClientsDocument.class)) { directory = this.outDir; fileName = String.format("clients/%s", fingerprint); + } else if (documentType.equals(UptimeDocument.class)) { + directory = this.outDir; + fileName = String.format("uptimes/%s", fingerprint); } if (directory != null && fileName != null) { documentFile = new File(directory, fileName); diff --git a/src/org/torproject/onionoo/Main.java b/src/org/torproject/onionoo/Main.java index 87e7c9a..5631a5e 100644 --- a/src/org/torproject/onionoo/Main.java +++ b/src/org/torproject/onionoo/Main.java @@ -38,7 +38,9 @@ public class Main { Logger.printStatusTime("Initialized weights data writer"); ClientsDataWriter cdw = new ClientsDataWriter(dso, ds, t); Logger.printStatusTime("Initialized clients data writer"); - DataWriter[] dws = new DataWriter[] { ndw, bdw, wdw, cdw }; + UptimeDataWriter udw = new UptimeDataWriter(dso, ds, t); + Logger.printStatusTime("Initialized uptime data writer"); + DataWriter[] dws = new DataWriter[] { ndw, bdw, wdw, cdw, udw }; Logger.printStatus("Reading descriptors."); dso.readRelayNetworkConsensuses(); diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index d04828f..de47d44 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -130,6 +130,8 @@ public class ResourceServlet extends HttpServlet { resourceType = "weights"; } else if (uri.startsWith("/clients")) { resourceType = "clients"; + } else if (uri.startsWith("/uptime")) { + resourceType = "uptime"; } else { response.sendError(HttpServletResponse.SC_BAD_REQUEST); return; diff --git a/src/org/torproject/onionoo/ResponseBuilder.java b/src/org/torproject/onionoo/ResponseBuilder.java index b7231db..b0775b1 100644 --- a/src/org/torproject/onionoo/ResponseBuilder.java +++ b/src/org/torproject/onionoo/ResponseBuilder.java @@ -776,6 +776,8 @@ public class ResponseBuilder { return this.writeWeightsLines(summaryLine); } else if (this.resourceType.equals("clients")) { return this.writeClientsLines(summaryLine); + } else if (this.resourceType.equals("uptime")) { + return this.writeUptimeLines(summaryLine); } else { return ""; } @@ -928,4 +930,30 @@ public class ResponseBuilder { return ""; } } + + private String writeUptimeLines(String summaryLine) { + String fingerprint = null; + if (summaryLine.contains("\"f\":\"")) { + fingerprint = summaryLine.substring(summaryLine.indexOf( + "\"f\":\"") + "\"f\":\"".length()); + } else if (summaryLine.contains("\"h\":\"")) { + fingerprint = summaryLine.substring(summaryLine.indexOf( + "\"h\":\"") + "\"h\":\"".length()); + } else { + return ""; + } + fingerprint = fingerprint.substring(0, 40); + UptimeDocument uptimeDocument = documentStore.retrieve( + UptimeDocument.class, false, fingerprint); + if (uptimeDocument != null && + uptimeDocument.documentString != null) { + String uptimeLines = uptimeDocument.documentString; + uptimeLines = uptimeLines.substring(0, uptimeLines.length() - 1); + return uptimeLines; + } else { + // TODO We should probably log that we didn't find an uptime + // document that we expected to exist. + return ""; + } + } } diff --git a/src/org/torproject/onionoo/UptimeDataWriter.java b/src/org/torproject/onionoo/UptimeDataWriter.java new file mode 100644 index 0000000..7a8b3af --- /dev/null +++ b/src/org/torproject/onionoo/UptimeDataWriter.java @@ -0,0 +1,533 @@ +/* Copyright 2014 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.onionoo; + +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Scanner; +import java.util.SortedMap; +import java.util.SortedSet; +import java.util.TimeZone; +import java.util.TreeMap; +import java.util.TreeSet; + +import org.torproject.descriptor.BridgeNetworkStatus; +import org.torproject.descriptor.Descriptor; +import org.torproject.descriptor.NetworkStatusEntry; +import org.torproject.descriptor.RelayNetworkStatusConsensus; + +public class UptimeDataWriter implements DataWriter, DescriptorListener { + + private DescriptorSource descriptorSource; + + private DocumentStore documentStore; + + private long now; + + public UptimeDataWriter(DescriptorSource descriptorSource, + DocumentStore documentStore, Time time) { + this.descriptorSource = descriptorSource; + this.documentStore = documentStore; + this.now = time.currentTimeMillis(); + this.registerDescriptorListeners(); + } + + private void registerDescriptorListeners() { + this.descriptorSource.registerListener(this, + DescriptorType.RELAY_CONSENSUSES); + this.descriptorSource.registerListener(this, + DescriptorType.BRIDGE_STATUSES); + } + + public void processDescriptor(Descriptor descriptor, boolean relay) { + if (descriptor instanceof RelayNetworkStatusConsensus) { + this.processRelayNetworkStatusConsensus( + (RelayNetworkStatusConsensus) descriptor); + } else if (descriptor instanceof BridgeNetworkStatus) { + this.processBridgeNetworkStatus( + (BridgeNetworkStatus) descriptor); + } + } + + private SortedSet<Long> newRelayStatuses = new TreeSet<Long>(), + newBridgeStatuses = new TreeSet<Long>(); + private SortedMap<String, SortedSet<Long>> + newRunningRelays = new TreeMap<String, SortedSet<Long>>(), + newRunningBridges = new TreeMap<String, SortedSet<Long>>(); + + private static final long ONE_HOUR_MILLIS = 60L * 60L * 1000L; + + private void processRelayNetworkStatusConsensus( + RelayNetworkStatusConsensus consensus) { + SortedSet<String> fingerprints = new TreeSet<String>(); + for (NetworkStatusEntry entry : + consensus.getStatusEntries().values()) { + if (entry.getFlags().contains("Running")) { + fingerprints.add(entry.getFingerprint()); + } + } + if (!fingerprints.isEmpty()) { + long dateHourMillis = (consensus.getValidAfterMillis() + / ONE_HOUR_MILLIS) * ONE_HOUR_MILLIS; + for (String fingerprint : fingerprints) { + if (!this.newRunningRelays.containsKey(fingerprint)) { + this.newRunningRelays.put(fingerprint, new TreeSet<Long>()); + } + this.newRunningRelays.get(fingerprint).add(dateHourMillis); + } + this.newRelayStatuses.add(dateHourMillis); + } + } + + private void processBridgeNetworkStatus(BridgeNetworkStatus status) { + SortedSet<String> fingerprints = new TreeSet<String>(); + for (NetworkStatusEntry entry : + status.getStatusEntries().values()) { + if (entry.getFlags().contains("Running")) { + fingerprints.add(entry.getFingerprint()); + } + } + if (!fingerprints.isEmpty()) { + long dateHourMillis = (status.getPublishedMillis() + / ONE_HOUR_MILLIS) * ONE_HOUR_MILLIS; + for (String fingerprint : fingerprints) { + if (!this.newRunningBridges.containsKey(fingerprint)) { + this.newRunningBridges.put(fingerprint, new TreeSet<Long>()); + } + this.newRunningBridges.get(fingerprint).add(dateHourMillis); + } + this.newBridgeStatuses.add(dateHourMillis); + } + } + + public void updateStatuses() { + for (Map.Entry<String, SortedSet<Long>> e : + this.newRunningRelays.entrySet()) { + this.updateStatus(true, e.getKey(), e.getValue()); + } + this.updateStatus(true, null, this.newRelayStatuses); + for (Map.Entry<String, SortedSet<Long>> e : + this.newRunningBridges.entrySet()) { + this.updateStatus(false, e.getKey(), e.getValue()); + } + this.updateStatus(false, null, this.newBridgeStatuses); + Logger.printStatusTime("Updated uptime status files"); + } + + private static class UptimeHistory + implements Comparable<UptimeHistory> { + private boolean relay; + private long startMillis; + private int uptimeHours; + private UptimeHistory(boolean relay, long startMillis, + int uptimeHours) { + this.relay = relay; + this.startMillis = startMillis; + this.uptimeHours = uptimeHours; + } + public static UptimeHistory fromString(String uptimeHistoryString) { + String[] parts = uptimeHistoryString.split(" ", 3); + if (parts.length != 3) { + return null; + } + boolean relay = false; + if (parts[0].equals("r")) { + relay = true; + } else if (!parts[0].equals("b")) { + return null; + } + long startMillis = -1L; + SimpleDateFormat dateHourFormat = new SimpleDateFormat( + "yyyy-MM-dd-HH"); + dateHourFormat.setLenient(false); + dateHourFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + try { + startMillis = dateHourFormat.parse(parts[1]).getTime(); + } catch (ParseException e) { + return null; + } + int uptimeHours = -1; + try { + uptimeHours = Integer.parseInt(parts[2]); + } catch (NumberFormatException e) { + return null; + } + return new UptimeHistory(relay, startMillis, uptimeHours); + } + public String toString() { + StringBuilder sb = new StringBuilder(); + SimpleDateFormat dateHourFormat = new SimpleDateFormat( + "yyyy-MM-dd-HH"); + dateHourFormat.setLenient(false); + dateHourFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + sb.append(this.relay ? "r" : "b"); + sb.append(" " + dateHourFormat.format(this.startMillis)); + sb.append(" " + String.format("%d", this.uptimeHours)); + return sb.toString(); + } + public void addUptime(UptimeHistory other) { + this.uptimeHours += other.uptimeHours; + if (this.startMillis > other.startMillis) { + this.startMillis = other.startMillis; + } + } + public int compareTo(UptimeHistory other) { + if (this.relay && !other.relay) { + return -1; + } else if (!this.relay && other.relay) { + return 1; + } + return this.startMillis < other.startMillis ? -1 : + this.startMillis > other.startMillis ? 1 : 0; + } + public boolean equals(Object other) { + return other instanceof UptimeHistory && + this.relay == ((UptimeHistory) other).relay && + this.startMillis == ((UptimeHistory) other).startMillis; + } + } + + private void updateStatus(boolean relay, String fingerprint, + SortedSet<Long> newUptimeHours) { + SortedSet<UptimeHistory> history = this.readHistory(fingerprint); + this.addToHistory(history, relay, newUptimeHours); + history = this.compressHistory(history); + this.writeHistory(fingerprint, history); + } + + private SortedSet<UptimeHistory> readHistory(String fingerprint) { + SortedSet<UptimeHistory> history = new TreeSet<UptimeHistory>(); + UptimeStatus uptimeStatus = fingerprint == null ? + documentStore.retrieve(UptimeStatus.class, false) : + documentStore.retrieve(UptimeStatus.class, false, fingerprint); + if (uptimeStatus != null) { + Scanner s = new Scanner(uptimeStatus.documentString); + while (s.hasNextLine()) { + String line = s.nextLine(); + UptimeHistory parsedLine = UptimeHistory.fromString(line); + if (parsedLine != null) { + history.add(parsedLine); + } else { + System.err.println("Could not parse uptime history line '" + + line + "' for fingerprint '" + fingerprint + + "'. Skipping."); + } + } + s.close(); + } + return history; + } + + private void addToHistory(SortedSet<UptimeHistory> history, + boolean relay, SortedSet<Long> newIntervals) { + for (long startMillis : newIntervals) { + UptimeHistory interval = new UptimeHistory(relay, startMillis, 1); + if (!history.headSet(interval).isEmpty()) { + UptimeHistory prev = history.headSet(interval).last(); + if (prev.relay == interval.relay && + prev.startMillis + ONE_HOUR_MILLIS * prev.uptimeHours > + interval.startMillis) { + continue; + } + } + if (!history.tailSet(interval).isEmpty()) { + UptimeHistory next = history.tailSet(interval).first(); + if (next.relay == interval.relay && + next.startMillis < interval.startMillis + ONE_HOUR_MILLIS) { + continue; + } + } + history.add(interval); + } + } + + private SortedSet<UptimeHistory> compressHistory( + SortedSet<UptimeHistory> history) { + SortedSet<UptimeHistory> compressedHistory = + new TreeSet<UptimeHistory>(); + UptimeHistory lastInterval = null; + for (UptimeHistory interval : history) { + if (lastInterval != null && + lastInterval.startMillis + ONE_HOUR_MILLIS + * lastInterval.uptimeHours == interval.startMillis && + lastInterval.relay == interval.relay) { + lastInterval.addUptime(interval); + } else { + if (lastInterval != null) { + compressedHistory.add(lastInterval); + } + lastInterval = interval; + } + } + if (lastInterval != null) { + compressedHistory.add(lastInterval); + } + return compressedHistory; + } + + private void writeHistory(String fingerprint, + SortedSet<UptimeHistory> history) { + StringBuilder sb = new StringBuilder(); + for (UptimeHistory interval : history) { + sb.append(interval.toString() + "\n"); + } + UptimeStatus uptimeStatus = new UptimeStatus(); + uptimeStatus.documentString = sb.toString(); + if (fingerprint == null) { + this.documentStore.store(uptimeStatus); + } else { + this.documentStore.store(uptimeStatus, fingerprint); + } + } + + public void updateDocuments() { + SortedSet<UptimeHistory> + knownRelayStatuses = new TreeSet<UptimeHistory>(), + knownBridgeStatuses = new TreeSet<UptimeHistory>(); + SortedSet<UptimeHistory> knownStatuses = this.readHistory(null); + for (UptimeHistory status : knownStatuses) { + if (status.relay) { + knownRelayStatuses.add(status); + } else { + knownBridgeStatuses.add(status); + } + } + for (String fingerprint : this.newRunningRelays.keySet()) { + this.updateDocument(true, fingerprint, knownRelayStatuses); + } + for (String fingerprint : this.newRunningBridges.keySet()) { + this.updateDocument(false, fingerprint, knownBridgeStatuses); + } + Logger.printStatusTime("Wrote uptime document files"); + } + + private void updateDocument(boolean relay, String fingerprint, + SortedSet<UptimeHistory> knownStatuses) { + SortedSet<UptimeHistory> history = this.readHistory(fingerprint); + UptimeDocument uptimeDocument = new UptimeDocument(); + uptimeDocument.documentString = this.formatHistoryString(relay, + fingerprint, history, knownStatuses); + this.documentStore.store(uptimeDocument, fingerprint); + } + + private String[] graphNames = new String[] { + "1_week", + "1_month", + "3_months", + "1_year", + "5_years" }; + + private long[] graphIntervals = new long[] { + 7L * 24L * 60L * 60L * 1000L, + 31L * 24L * 60L * 60L * 1000L, + 92L * 24L * 60L * 60L * 1000L, + 366L * 24L * 60L * 60L * 1000L, + 5L * 366L * 24L * 60L * 60L * 1000L }; + + private long[] dataPointIntervals = new long[] { + 60L * 60L * 1000L, + 4L * 60L * 60L * 1000L, + 12L * 60L * 60L * 1000L, + 2L * 24L * 60L * 60L * 1000L, + 10L * 24L * 60L * 60L * 1000L }; + + private String formatHistoryString(boolean relay, String fingerprint, + SortedSet<UptimeHistory> history, + SortedSet<UptimeHistory> knownStatuses) { + StringBuilder sb = new StringBuilder(); + sb.append("{\"fingerprint\":\"" + fingerprint + "\""); + sb.append(",\n\"uptime\":{"); + int graphIntervalsWritten = 0; + for (int graphIntervalIndex = 0; graphIntervalIndex < + this.graphIntervals.length; graphIntervalIndex++) { + String timeline = this.formatTimeline(graphIntervalIndex, relay, + history, knownStatuses); + if (timeline != null) { + sb.append((graphIntervalsWritten++ > 0 ? "," : "") + "\n" + + timeline); + } + } + sb.append("}"); + sb.append("\n}\n"); + return sb.toString(); + } + + private String formatTimeline(int graphIntervalIndex, boolean relay, + SortedSet<UptimeHistory> history, + SortedSet<UptimeHistory> knownStatuses) { + String graphName = this.graphNames[graphIntervalIndex]; + long graphInterval = this.graphIntervals[graphIntervalIndex]; + long dataPointInterval = + this.dataPointIntervals[graphIntervalIndex]; + int dataPointIntervalHours = (int) (dataPointInterval + / ONE_HOUR_MILLIS); + List<Integer> statusDataPoints = new ArrayList<Integer>(); + long intervalStartMillis = ((this.now - graphInterval) + / dataPointInterval) * dataPointInterval; + int statusHours = 0; + for (UptimeHistory hist : knownStatuses) { + if (hist.relay != relay) { + continue; + } + long histEndMillis = hist.startMillis + ONE_HOUR_MILLIS + * hist.uptimeHours; + if (histEndMillis < intervalStartMillis) { + continue; + } + while (hist.startMillis >= intervalStartMillis + + dataPointInterval) { + statusDataPoints.add(statusHours * 5 > dataPointIntervalHours + ? statusHours : -1); + statusHours = 0; + intervalStartMillis += dataPointInterval; + } + while (histEndMillis >= intervalStartMillis + dataPointInterval) { + statusHours += (int) ((intervalStartMillis + dataPointInterval + - Math.max(hist.startMillis, intervalStartMillis)) + / ONE_HOUR_MILLIS); + statusDataPoints.add(statusHours * 5 > dataPointIntervalHours + ? statusHours : -1); + statusHours = 0; + intervalStartMillis += dataPointInterval; + } + statusHours += (int) ((histEndMillis - Math.max(hist.startMillis, + intervalStartMillis)) / ONE_HOUR_MILLIS); + } + statusDataPoints.add(statusHours * 5 > dataPointIntervalHours + ? statusHours : -1); + List<Integer> uptimeDataPoints = new ArrayList<Integer>(); + intervalStartMillis = ((this.now - graphInterval) + / dataPointInterval) * dataPointInterval; + int uptimeHours = 0; + long firstStatusStartMillis = -1L; + for (UptimeHistory hist : history) { + if (hist.relay != relay) { + continue; + } + if (firstStatusStartMillis < 0L) { + firstStatusStartMillis = hist.startMillis; + } + long histEndMillis = hist.startMillis + ONE_HOUR_MILLIS + * hist.uptimeHours; + if (histEndMillis < intervalStartMillis) { + continue; + } + while (hist.startMillis >= intervalStartMillis + + dataPointInterval) { + if (firstStatusStartMillis < intervalStartMillis + + dataPointInterval) { + uptimeDataPoints.add(uptimeHours); + } else { + uptimeDataPoints.add(-1); + } + uptimeHours = 0; + intervalStartMillis += dataPointInterval; + } + while (histEndMillis >= intervalStartMillis + dataPointInterval) { + uptimeHours += (int) ((intervalStartMillis + dataPointInterval + - Math.max(hist.startMillis, intervalStartMillis)) + / ONE_HOUR_MILLIS); + uptimeDataPoints.add(uptimeHours); + uptimeHours = 0; + intervalStartMillis += dataPointInterval; + } + uptimeHours += (int) ((histEndMillis - Math.max(hist.startMillis, + intervalStartMillis)) / ONE_HOUR_MILLIS); + } + uptimeDataPoints.add(uptimeHours); + List<Double> dataPoints = new ArrayList<Double>(); + for (int dataPointIndex = 0; dataPointIndex < statusDataPoints.size(); + dataPointIndex++) { + if (dataPointIndex >= uptimeDataPoints.size()) { + dataPoints.add(0.0); + } else if (uptimeDataPoints.get(dataPointIndex) >= 0 && + statusDataPoints.get(dataPointIndex) > 0) { + dataPoints.add(((double) uptimeDataPoints.get(dataPointIndex)) + / ((double) statusDataPoints.get(dataPointIndex))); + } else { + dataPoints.add(-1.0); + } + } + int firstNonNullIndex = -1, lastNonNullIndex = -1; + for (int dataPointIndex = 0; dataPointIndex < dataPoints.size(); + dataPointIndex++) { + double dataPoint = dataPoints.get(dataPointIndex); + if (dataPoint >= 0.0) { + if (firstNonNullIndex < 0) { + firstNonNullIndex = dataPointIndex; + } + lastNonNullIndex = dataPointIndex; + } + } + if (firstNonNullIndex < 0) { + return null; + } + long firstDataPointMillis = (((this.now - graphInterval) + / dataPointInterval) + firstNonNullIndex) + * dataPointInterval + dataPointInterval / 2L; + if (graphIntervalIndex > 0 && firstDataPointMillis >= + this.now - graphIntervals[graphIntervalIndex - 1]) { + /* Skip uptime history object, because it doesn't contain + * anything new that wasn't already contained in the last + * uptime history object(s). */ + return null; + } + long lastDataPointMillis = firstDataPointMillis + + (lastNonNullIndex - firstNonNullIndex) * dataPointInterval; + double factor = 1.0 / 999.0; + int count = lastNonNullIndex - firstNonNullIndex + 1; + StringBuilder sb = new StringBuilder(); + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + sb.append("\"" + graphName + "\":{" + + "\"first\":\"" + dateTimeFormat.format(firstDataPointMillis) + + "\",\"last\":\"" + dateTimeFormat.format(lastDataPointMillis) + + "\",\"interval\":" + String.valueOf(dataPointInterval / 1000L) + + ",\"factor\":" + String.format(Locale.US, "%.9f", factor) + + ",\"count\":" + String.valueOf(count) + ",\"values\":["); + int dataPointsWritten = 0, previousNonNullIndex = -2; + boolean foundTwoAdjacentDataPoints = false; + for (int dataPointIndex = firstNonNullIndex; dataPointIndex <= + lastNonNullIndex; dataPointIndex++) { + double dataPoint = dataPoints.get(dataPointIndex); + if (dataPoint >= 0.0) { + if (dataPointIndex - previousNonNullIndex == 1) { + foundTwoAdjacentDataPoints = true; + } + previousNonNullIndex = dataPointIndex; + } + sb.append((dataPointsWritten++ > 0 ? "," : "") + + (dataPoint < -0.5 ? "null" : + String.valueOf((long) (dataPoint * 999.0)))); + } + sb.append("]}"); + if (foundTwoAdjacentDataPoints) { + return sb.toString(); + } else { + return null; + } + } + + public String getStatsString() { + StringBuilder sb = new StringBuilder(); + sb.append(" " + Logger.formatDecimalNumber( + this.newRelayStatuses.size()) + " hours of relay uptimes " + + "processed\n"); + sb.append(" " + Logger.formatDecimalNumber( + this.newBridgeStatuses.size()) + " hours of bridge uptimes " + + "processed\n"); + sb.append(" " + Logger.formatDecimalNumber( + this.newRunningRelays.size() + this.newRunningBridges.size()) + + " uptime status files updated\n"); + sb.append(" " + Logger.formatDecimalNumber( + this.newRunningRelays.size() + this.newRunningBridges.size()) + + " uptime document files updated\n"); + return sb.toString(); + } +} + diff --git a/src/org/torproject/onionoo/UptimeDocument.java b/src/org/torproject/onionoo/UptimeDocument.java new file mode 100644 index 0000000..f71cb87 --- /dev/null +++ b/src/org/torproject/onionoo/UptimeDocument.java @@ -0,0 +1,8 @@ +/* Copyright 2014 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.onionoo; + +class UptimeDocument extends Document { + +} + diff --git a/src/org/torproject/onionoo/UptimeStatus.java b/src/org/torproject/onionoo/UptimeStatus.java new file mode 100644 index 0000000..d16c8fc --- /dev/null +++ b/src/org/torproject/onionoo/UptimeStatus.java @@ -0,0 +1,5 @@ +package org.torproject.onionoo; + +class UptimeStatus extends Document { +} + diff --git a/web/index.html b/web/index.html index c07ca86..732ff8f 100644 --- a/web/index.html +++ b/web/index.html @@ -58,6 +58,7 @@ h3 .request-response { padding: 0 !important; } <li><a href="#bandwidth">Bandwidth documents</a></li> <li><a href="#weights">Weights documents</a></li> <li><a href="#clients">Clients documents</a></li> + <li><a href="#uptime">Uptime documents</a></li> </ul> </div> @@ -1955,6 +1956,197 @@ It might be removed in the future without notice. </div>  +<div class="box"> +<a name="uptime"></a> +<h3>Uptime documents <a href="#uptime">#</a> + +<a href="uptime?limit=4">example request</a> + +</h3> + + +Added on March 10, 2014. +Uptime documents contain fractional uptimes of relays and bridges. +Uptime documents contain different time intervals and are available for +all relays and bridges that have been running in the past week. +Uptime documents contain the following fields: + + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when +the last known relay network status consensus started being valid. +Indicates how recent the relay uptime documents in this document are. + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Array of objects representing relay uptime documents. +Each array object contains the following key-value pairs: + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +Relay fingerprint consisting of 40 upper-case +hexadecimal characters. + +</li> + +<li> +uptime +<code class="typeof">object</code> +optional + +History object containing the fractional uptime of this relay. +Keys are string representation of the time period covered by the uptime +history object. +Keys are fixed strings "1_week", +"1_month", "3_months", +"1_year", and "5_years". +Keys refer to the last known uptime history of a relay, not to the time +when the uptime document was published. +An uptime history object is only contained if the time period it covers +is not already contained in another uptime history object with shorter +time period and higher data resolution. +Each uptime history object contains the following key-value pairs: + + +<ul class="properties"> + +<li> +first +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data +data point in the uptime history. + +</li> + +<li> +last +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data +data point in the uptime history. + +</li> + +<li> +interval +<code class="typeof">number</code> +required + +Time interval between two data points in seconds. + +</li> + +<li> +factor +<code class="typeof">number</code> +required + +Factor by which subsequent uptime values need to +be multiplied to get the fractional uptime. +This is only done for compatibility reasons with the other document types. + +</li> + +<li> +count +<code class="typeof">number</code> +optional + +Number of provided data points, included mostly for debugging purposes. +Can also be derived from the number of elements in the subsequent array. + +</li> + +<li> +values +<code class="typeof">array of numbers</code> +required + +Array of normalized uptime values. +May contain null values if less than 20% of network statuses have been +processed for a given time period. +Contains at least two subsequent non-null values to enable drawing of line +graphs. + +</li> + +</ul> + +</li> + +</ul> + +</li> + +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when +the last known bridge network status was published. +Indicates how recent the bridge uptime documents in this document are. + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Array of objects representing bridge uptime documents. +Each array object contains the following key-value pairs: + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +SHA-1 hash of the bridge fingerprint consisting +of 40 upper-case hexadecimal characters. + +</li> + +<li> +uptime +<code class="typeof">object</code> +required + +Object containing uptime history objects for different time periods. +The specification of uptime history objects is similar to those in the +uptime field of relays. + +</li> + +</li> + +</ul> + +</div>  + </body> </html>

1 0

[onionoo/master] Add new document with per-bridge usage statistics.
by karsten＠torproject.org 11 Mar '14

11 Mar '14

commit 985f9bf7a5b1301ef1db6217906e2808f01c0dc7 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 6 16:46:10 2014 +0100 Add new document with per-bridge usage statistics. Implements #10331. --- etc/web.xml.template | 4 + src/org/torproject/onionoo/ClientsDataWriter.java | 633 +++++++++++++++++++++ src/org/torproject/onionoo/ClientsDocument.java | 8 + src/org/torproject/onionoo/ClientsStatus.java | 5 + src/org/torproject/onionoo/DocumentStore.java | 20 +- src/org/torproject/onionoo/Main.java | 4 +- src/org/torproject/onionoo/ResourceServlet.java | 2 + src/org/torproject/onionoo/ResponseBuilder.java | 38 +- web/index.html | 236 ++++++++ 9 files changed, 946 insertions(+), 4 deletions(-) diff --git a/etc/web.xml.template b/etc/web.xml.template index 25314aa..53a1878 100644 --- a/etc/web.xml.template +++ b/etc/web.xml.template @@ -36,6 +36,10 @@ <servlet-name>Resource</servlet-name> <url-pattern>/weights</url-pattern> </servlet-mapping> + <servlet-mapping> + <servlet-name>Resource</servlet-name> + <url-pattern>/clients</url-pattern> + </servlet-mapping> </web-app> diff --git a/src/org/torproject/onionoo/ClientsDataWriter.java b/src/org/torproject/onionoo/ClientsDataWriter.java new file mode 100644 index 0000000..9e868a4 --- /dev/null +++ b/src/org/torproject/onionoo/ClientsDataWriter.java @@ -0,0 +1,633 @@ +/* Copyright 2014 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.onionoo; + +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Scanner; +import java.util.SortedMap; +import java.util.SortedSet; +import java.util.TimeZone; +import java.util.TreeMap; +import java.util.TreeSet; + +import org.torproject.descriptor.Descriptor; +import org.torproject.descriptor.ExtraInfoDescriptor; + +/* + * Example extra-info descriptor used as input: + * + * extra-info ndnop2 DE6397A047ABE5F78B4C87AF725047831B221AAB + * dirreq-stats-end 2014-02-16 16:42:11 (86400 s) + * dirreq-v3-resp ok=856,not-enough-sigs=0,unavailable=0,not-found=0, + * not-modified=40,busy=0 + * bridge-stats-end 2014-02-16 16:42:17 (86400 s) + * bridge-ips ??=8,in=8,se=8 + * bridge-ip-versions v4=8,v6=0 + * + * Clients status file produced as intermediate output: + * + * 2014-02-15 16:42:11 2014-02-16 00:00:00 + * 259.042 in=86.347,se=86.347 v4=259.042 + * 2014-02-16 00:00:00 2014-02-16 16:42:11 + * 592.958 in=197.653,se=197.653 v4=592.958 + * + * Clients document file produced as output: + * + * "1_month":{ + * "first":"2014-02-03 12:00:00", + * "last":"2014-02-28 12:00:00", + * "interval":86400, + * "factor":0.139049349, + * "count":26, + * "values":[371,354,349,374,432,null,485,458,493,536,null,null,524,576, + * 607,622,null,635,null,566,774,999,945,690,656,681], + * "countries":{"cn":0.0192,"in":0.1768,"ir":0.2487,"ru":0.0104, + * "se":0.1698,"sy":0.0325,"us":0.0406}, + * "transports":{"obfs2":0.4581}, + * "versions":{"v4":1.0000}} + */ +public class ClientsDataWriter implements DataWriter, DescriptorListener { + + private static class ResponseHistory + implements Comparable<ResponseHistory> { + private long startMillis; + private long endMillis; + private double totalResponses; + private SortedMap<String, Double> responsesByCountry; + private SortedMap<String, Double> responsesByTransport; + private SortedMap<String, Double> responsesByVersion; + private ResponseHistory(long startMillis, long endMillis, + double totalResponses, + SortedMap<String, Double> responsesByCountry, + SortedMap<String, Double> responsesByTransport, + SortedMap<String, Double> responsesByVersion) { + this.startMillis = startMillis; + this.endMillis = endMillis; + this.totalResponses = totalResponses; + this.responsesByCountry = responsesByCountry; + this.responsesByTransport = responsesByTransport; + this.responsesByVersion = responsesByVersion; + } + public static ResponseHistory fromString( + String responseHistoryString) { + String[] parts = responseHistoryString.split(" ", 8); + if (parts.length != 8) { + return null; + } + long startMillis = -1L, endMillis = -1L; + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setLenient(false); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + try { + startMillis = dateTimeFormat.parse(parts[0] + " " + parts[1]). + getTime(); + endMillis = dateTimeFormat.parse(parts[2] + " " + parts[3]). + getTime(); + } catch (ParseException e) { + return null; + } + if (startMillis >= endMillis) { + return null; + } + double totalResponses = 0.0; + try { + totalResponses = Double.parseDouble(parts[4]); + } catch (NumberFormatException e) { + return null; + } + SortedMap<String, Double> responsesByCountry = + parseResponses(parts[5]); + SortedMap<String, Double> responsesByTransport = + parseResponses(parts[6]); + SortedMap<String, Double> responsesByVersion = + parseResponses(parts[7]); + if (responsesByCountry == null || responsesByTransport == null || + responsesByVersion == null) { + return null; + } + return new ResponseHistory(startMillis, endMillis, totalResponses, + responsesByCountry, responsesByTransport, responsesByVersion); + } + private static SortedMap<String, Double> parseResponses( + String responsesString) { + SortedMap<String, Double> responses = new TreeMap<String, Double>(); + if (responsesString.length() > 0) { + for (String pair : responsesString.split(",")) { + String[] keyValue = pair.split("="); + if (keyValue.length != 2) { + return null; + } + double value = 0.0; + try { + value = Double.parseDouble(keyValue[1]); + } catch (NumberFormatException e) { + return null; + } + responses.put(keyValue[0], value); + } + } + return responses; + } + public String toString() { + StringBuilder sb = new StringBuilder(); + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setLenient(false); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + sb.append(dateTimeFormat.format(startMillis)); + sb.append(" " + dateTimeFormat.format(endMillis)); + sb.append(" " + String.format("%.3f", this.totalResponses)); + this.appendResponses(sb, this.responsesByCountry); + this.appendResponses(sb, this.responsesByTransport); + this.appendResponses(sb, this.responsesByVersion); + return sb.toString(); + } + private void appendResponses(StringBuilder sb, + SortedMap<String, Double> responses) { + sb.append(" "); + int written = 0; + for (Map.Entry<String, Double> e : responses.entrySet()) { + sb.append((written++ > 0 ? "," : "") + e.getKey() + "=" + + String.format("%.3f", e.getValue())); + } + } + public void addResponses(ResponseHistory other) { + this.totalResponses += other.totalResponses; + this.addResponsesByCategory(this.responsesByCountry, + other.responsesByCountry); + this.addResponsesByCategory(this.responsesByTransport, + other.responsesByTransport); + this.addResponsesByCategory(this.responsesByVersion, + other.responsesByVersion); + if (this.startMillis > other.startMillis) { + this.startMillis = other.startMillis; + } + if (this.endMillis < other.endMillis) { + this.endMillis = other.endMillis; + } + } + private void addResponsesByCategory( + SortedMap<String, Double> thisResponses, + SortedMap<String, Double> otherResponses) { + for (Map.Entry<String, Double> e : otherResponses.entrySet()) { + if (thisResponses.containsKey(e.getKey())) { + thisResponses.put(e.getKey(), thisResponses.get(e.getKey()) + + e.getValue()); + } else { + thisResponses.put(e.getKey(), e.getValue()); + } + } + } + public int compareTo(ResponseHistory other) { + return this.startMillis < other.startMillis ? -1 : + this.startMillis > other.startMillis ? 1 : 0; + } + public boolean equals(Object other) { + return other instanceof ResponseHistory && + this.startMillis == ((ResponseHistory) other).startMillis; + } + } + + private DescriptorSource descriptorSource; + + private DocumentStore documentStore; + + private long now; + + public ClientsDataWriter(DescriptorSource descriptorSource, + DocumentStore documentStore, Time time) { + this.descriptorSource = descriptorSource; + this.documentStore = documentStore; + this.now = time.currentTimeMillis(); + this.registerDescriptorListeners(); + } + + private void registerDescriptorListeners() { + this.descriptorSource.registerListener(this, + DescriptorType.BRIDGE_EXTRA_INFOS); + } + + public void processDescriptor(Descriptor descriptor, boolean relay) { + if (descriptor instanceof ExtraInfoDescriptor && !relay) { + this.processBridgeExtraInfoDescriptor( + (ExtraInfoDescriptor) descriptor); + } + } + + private static final long ONE_HOUR_MILLIS = 60L * 60L * 1000L, + ONE_DAY_MILLIS = 24L * ONE_HOUR_MILLIS; + + private SortedMap<String, SortedSet<ResponseHistory>> newResponses = + new TreeMap<String, SortedSet<ResponseHistory>>(); + + private void processBridgeExtraInfoDescriptor( + ExtraInfoDescriptor descriptor) { + long dirreqStatsEndMillis = descriptor.getDirreqStatsEndMillis(); + long dirreqStatsIntervalLengthMillis = + descriptor.getDirreqStatsIntervalLength() * 1000L; + SortedMap<String, Integer> responses = descriptor.getDirreqV3Resp(); + if (dirreqStatsEndMillis < 0L || + dirreqStatsIntervalLengthMillis != ONE_DAY_MILLIS || + responses == null || !responses.containsKey("ok")) { + return; + } + double okResponses = (double) (responses.get("ok") - 4); + if (okResponses < 0.0) { + return; + } + String hashedFingerprint = descriptor.getFingerprint().toUpperCase(); + long dirreqStatsStartMillis = dirreqStatsEndMillis + - dirreqStatsIntervalLengthMillis; + long utcBreakMillis = (dirreqStatsEndMillis / ONE_DAY_MILLIS) + * ONE_DAY_MILLIS; + for (int i = 0; i < 2; i++) { + long startMillis = i == 0 ? dirreqStatsStartMillis : utcBreakMillis; + long endMillis = i == 0 ? utcBreakMillis : dirreqStatsEndMillis; + if (startMillis >= endMillis) { + continue; + } + double totalResponses = okResponses + * ((double) (endMillis - startMillis)) + / ((double) ONE_DAY_MILLIS); + SortedMap<String, Double> responsesByCountry = + this.weightResponsesWithUniqueIps(totalResponses, + descriptor.getBridgeIps(), "??"); + SortedMap<String, Double> responsesByTransport = + this.weightResponsesWithUniqueIps(totalResponses, + descriptor.getBridgeIpTransports(), "<??>"); + SortedMap<String, Double> responsesByVersion = + this.weightResponsesWithUniqueIps(totalResponses, + descriptor.getBridgeIpVersions(), ""); + ResponseHistory newResponseHistory = new ResponseHistory( + startMillis, endMillis, totalResponses, responsesByCountry, + responsesByTransport, responsesByVersion); + if (!this.newResponses.containsKey(hashedFingerprint)) { + this.newResponses.put(hashedFingerprint, + new TreeSet<ResponseHistory>()); + } + this.newResponses.get(hashedFingerprint).add( + newResponseHistory); + } + } + + private SortedMap<String, Double> weightResponsesWithUniqueIps( + double totalResponses, SortedMap<String, Integer> uniqueIps, + String omitString) { + SortedMap<String, Double> weightedResponses = + new TreeMap<String, Double>(); + int totalUniqueIps = 0; + if (uniqueIps != null) { + for (Map.Entry<String, Integer> e : uniqueIps.entrySet()) { + if (e.getValue() > 4) { + totalUniqueIps += e.getValue() - 4; + } + } + } + if (totalUniqueIps > 0) { + for (Map.Entry<String, Integer> e : uniqueIps.entrySet()) { + if (!e.getKey().equals(omitString) && e.getValue() > 4) { + weightedResponses.put(e.getKey(), + (((double) (e.getValue() - 4)) * totalResponses) + / ((double) totalUniqueIps)); + } + } + } + return weightedResponses; + } + + public void updateStatuses() { + for (Map.Entry<String, SortedSet<ResponseHistory>> e : + this.newResponses.entrySet()) { + String hashedFingerprint = e.getKey(); + SortedSet<ResponseHistory> history = + this.readHistory(hashedFingerprint); + this.addToHistory(history, e.getValue()); + history = this.compressHistory(history); + this.writeHistory(hashedFingerprint, history); + } + Logger.printStatusTime("Updated clients status files"); + } + + private SortedSet<ResponseHistory> readHistory( + String hashedFingerprint) { + SortedSet<ResponseHistory> history = new TreeSet<ResponseHistory>(); + ClientsStatus clientsStatus = this.documentStore.retrieve( + ClientsStatus.class, false, hashedFingerprint); + if (clientsStatus != null) { + Scanner s = new Scanner(clientsStatus.documentString); + while (s.hasNextLine()) { + String line = s.nextLine(); + ResponseHistory parsedLine = ResponseHistory.fromString(line); + if (parsedLine != null) { + history.add(parsedLine); + } else { + System.err.println("Could not parse clients history line '" + + line + "' for fingerprint '" + hashedFingerprint + + "'. Skipping."); + } + } + s.close(); + } + return history; + } + + private void addToHistory(SortedSet<ResponseHistory> history, + SortedSet<ResponseHistory> newIntervals) { + for (ResponseHistory interval : newIntervals) { + if ((history.headSet(interval).isEmpty() || + history.headSet(interval).last().endMillis <= + interval.startMillis) && + (history.tailSet(interval).isEmpty() || + history.tailSet(interval).first().startMillis >= + interval.endMillis)) { + history.add(interval); + } + } + } + + private SortedSet<ResponseHistory> compressHistory( + SortedSet<ResponseHistory> history) { + SortedSet<ResponseHistory> compressedHistory = + new TreeSet<ResponseHistory>(); + ResponseHistory lastResponses = null; + for (ResponseHistory responses : history) { + long intervalLengthMillis; + if (this.now - responses.endMillis <= + 92L * 24L * 60L * 60L * 1000L) { + intervalLengthMillis = 24L * 60L * 60L * 1000L; + } else if (this.now - responses.endMillis <= + 366L * 24L * 60L * 60L * 1000L) { + intervalLengthMillis = 2L * 24L * 60L * 60L * 1000L; + } else { + intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; + } + if (lastResponses != null && + lastResponses.endMillis == responses.startMillis && + ((lastResponses.endMillis - 1L) / intervalLengthMillis) == + ((responses.endMillis - 1L) / intervalLengthMillis)) { + lastResponses.addResponses(responses); + } else { + if (lastResponses != null) { + compressedHistory.add(lastResponses); + } + lastResponses = responses; + } + } + if (lastResponses != null) { + compressedHistory.add(lastResponses); + } + return compressedHistory; + } + + private void writeHistory(String hashedFingerprint, + SortedSet<ResponseHistory> history) { + StringBuilder sb = new StringBuilder(); + for (ResponseHistory responses : history) { + sb.append(responses.toString() + "\n"); + } + ClientsStatus clientsStatus = new ClientsStatus(); + clientsStatus.documentString = sb.toString(); + this.documentStore.store(clientsStatus, hashedFingerprint); + } + + public void updateDocuments() { + for (String hashedFingerprint : this.newResponses.keySet()) { + SortedSet<ResponseHistory> history = + this.readHistory(hashedFingerprint); + ClientsDocument clientsDocument = new ClientsDocument(); + clientsDocument.documentString = this.formatHistoryString( + hashedFingerprint, history); + this.documentStore.store(clientsDocument, hashedFingerprint); + } + Logger.printStatusTime("Wrote clients document files"); + } + + private String[] graphNames = new String[] { + "1_week", + "1_month", + "3_months", + "1_year", + "5_years" }; + + private long[] graphIntervals = new long[] { + 7L * 24L * 60L * 60L * 1000L, + 31L * 24L * 60L * 60L * 1000L, + 92L * 24L * 60L * 60L * 1000L, + 366L * 24L * 60L * 60L * 1000L, + 5L * 366L * 24L * 60L * 60L * 1000L }; + + private long[] dataPointIntervals = new long[] { + 24L * 60L * 60L * 1000L, + 24L * 60L * 60L * 1000L, + 24L * 60L * 60L * 1000L, + 2L * 24L * 60L * 60L * 1000L, + 10L * 24L * 60L * 60L * 1000L }; + + private String formatHistoryString(String hashedFingerprint, + SortedSet<ResponseHistory> history) { + StringBuilder sb = new StringBuilder(); + sb.append("{\"fingerprint\":\"" + hashedFingerprint + "\""); + sb.append(",\n\"average_clients\":{"); + int graphIntervalsWritten = 0; + for (int graphIntervalIndex = 0; graphIntervalIndex < + this.graphIntervals.length; graphIntervalIndex++) { + String timeline = this.formatTimeline(graphIntervalIndex, history); + if (timeline != null) { + sb.append((graphIntervalsWritten++ > 0 ? "," : "") + "\n" + + timeline); + } + } + sb.append("}"); + sb.append("\n}\n"); + return sb.toString(); + } + + private String formatTimeline(int graphIntervalIndex, + SortedSet<ResponseHistory> history) { + String graphName = this.graphNames[graphIntervalIndex]; + long graphInterval = this.graphIntervals[graphIntervalIndex]; + long dataPointInterval = + this.dataPointIntervals[graphIntervalIndex]; + List<Double> dataPoints = new ArrayList<Double>(); + long intervalStartMillis = ((this.now - graphInterval) + / dataPointInterval) * dataPointInterval; + long millis = 0L; + double responses = 0.0, totalResponses = 0.0; + SortedMap<String, Double> + totalResponsesByCountry = new TreeMap<String, Double>(), + totalResponsesByTransport = new TreeMap<String, Double>(), + totalResponsesByVersion = new TreeMap<String, Double>(); + for (ResponseHistory hist : history) { + if (hist.endMillis < intervalStartMillis) { + continue; + } + while ((intervalStartMillis / dataPointInterval) != + (hist.endMillis / dataPointInterval)) { + dataPoints.add(millis * 2L < dataPointInterval + ? -1.0 : responses * ((double) ONE_DAY_MILLIS) + / (((double) millis) * 10.0)); + responses = 0.0; + millis = 0L; + intervalStartMillis += dataPointInterval; + } + responses += hist.totalResponses; + totalResponses += hist.totalResponses; + for (Map.Entry<String, Double> e : + hist.responsesByCountry.entrySet()) { + if (!totalResponsesByCountry.containsKey(e.getKey())) { + totalResponsesByCountry.put(e.getKey(), 0.0); + } + totalResponsesByCountry.put(e.getKey(), e.getValue() + + totalResponsesByCountry.get(e.getKey())); + } + for (Map.Entry<String, Double> e : + hist.responsesByTransport.entrySet()) { + if (!totalResponsesByTransport.containsKey(e.getKey())) { + totalResponsesByTransport.put(e.getKey(), 0.0); + } + totalResponsesByTransport.put(e.getKey(), e.getValue() + + totalResponsesByTransport.get(e.getKey())); + } + for (Map.Entry<String, Double> e : + hist.responsesByVersion.entrySet()) { + if (!totalResponsesByVersion.containsKey(e.getKey())) { + totalResponsesByVersion.put(e.getKey(), 0.0); + } + totalResponsesByVersion.put(e.getKey(), e.getValue() + + totalResponsesByVersion.get(e.getKey())); + } + millis += (hist.endMillis - hist.startMillis); + } + dataPoints.add(millis * 2L < dataPointInterval + ? -1.0 : responses * ((double) ONE_DAY_MILLIS) + / (((double) millis) * 10.0)); + double maxValue = 0.0; + int firstNonNullIndex = -1, lastNonNullIndex = -1; + for (int dataPointIndex = 0; dataPointIndex < dataPoints.size(); + dataPointIndex++) { + double dataPoint = dataPoints.get(dataPointIndex); + if (dataPoint >= 0.0) { + if (firstNonNullIndex < 0) { + firstNonNullIndex = dataPointIndex; + } + lastNonNullIndex = dataPointIndex; + if (dataPoint > maxValue) { + maxValue = dataPoint; + } + } + } + if (firstNonNullIndex < 0) { + return null; + } + long firstDataPointMillis = (((this.now - graphInterval) + / dataPointInterval) + firstNonNullIndex) * dataPointInterval + + dataPointInterval / 2L; + if (graphIntervalIndex > 0 && firstDataPointMillis >= + this.now - graphIntervals[graphIntervalIndex - 1]) { + /* Skip clients history object, because it doesn't contain + * anything new that wasn't already contained in the last + * clients history object(s). */ + return null; + } + long lastDataPointMillis = firstDataPointMillis + + (lastNonNullIndex - firstNonNullIndex) * dataPointInterval; + double factor = ((double) maxValue) / 999.0; + int count = lastNonNullIndex - firstNonNullIndex + 1; + StringBuilder sb = new StringBuilder(); + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + sb.append("\"" + graphName + "\":{" + + "\"first\":\"" + dateTimeFormat.format(firstDataPointMillis) + + "\",\"last\":\"" + dateTimeFormat.format(lastDataPointMillis) + + "\",\"interval\":" + String.valueOf(dataPointInterval / 1000L) + + ",\"factor\":" + String.format(Locale.US, "%.9f", factor) + + ",\"count\":" + String.valueOf(count) + ",\"values\":["); + int dataPointsWritten = 0, previousNonNullIndex = -2; + boolean foundTwoAdjacentDataPoints = false; + for (int dataPointIndex = firstNonNullIndex; dataPointIndex <= + lastNonNullIndex; dataPointIndex++) { + double dataPoint = dataPoints.get(dataPointIndex); + if (dataPoint >= 0.0) { + if (dataPointIndex - previousNonNullIndex == 1) { + foundTwoAdjacentDataPoints = true; + } + previousNonNullIndex = dataPointIndex; + } + sb.append((dataPointsWritten++ > 0 ? "," : "") + + (dataPoint < 0.0 ? "null" : + String.valueOf((long) ((dataPoint * 999.0) / maxValue)))); + } + sb.append("]"); + if (!totalResponsesByCountry.isEmpty()) { + sb.append(",\"countries\":{"); + int written = 0; + for (Map.Entry<String, Double> e : + totalResponsesByCountry.entrySet()) { + if (e.getValue() > totalResponses / 100.0) { + sb.append((written++ > 0 ? "," : "") + "\"" + e.getKey() + + "\":" + String.format(Locale.US, "%.4f", + e.getValue() / totalResponses)); + } + } + sb.append("}"); + } + if (!totalResponsesByTransport.isEmpty()) { + sb.append(",\"transports\":{"); + int written = 0; + for (Map.Entry<String, Double> e : + totalResponsesByTransport.entrySet()) { + if (e.getValue() > totalResponses / 100.0) { + sb.append((written++ > 0 ? "," : "") + "\"" + e.getKey() + + "\":" + String.format(Locale.US, "%.4f", + e.getValue() / totalResponses)); + } + } + sb.append("}"); + } + if (!totalResponsesByVersion.isEmpty()) { + sb.append(",\"versions\":{"); + int written = 0; + for (Map.Entry<String, Double> e : + totalResponsesByVersion.entrySet()) { + if (e.getValue() > totalResponses / 100.0) { + sb.append((written++ > 0 ? "," : "") + "\"" + e.getKey() + + "\":" + String.format(Locale.US, "%.4f", + e.getValue() / totalResponses)); + } + } + sb.append("}"); + } + sb.append("}"); + if (foundTwoAdjacentDataPoints) { + return sb.toString(); + } else { + return null; + } + } + + public String getStatsString() { + int newIntervals = 0; + for (SortedSet<ResponseHistory> hist : this.newResponses.values()) { + newIntervals += hist.size(); + } + StringBuilder sb = new StringBuilder(); + sb.append(" " + + Logger.formatDecimalNumber(newIntervals / 2) + + " client statistics processed from extra-info descriptors\n"); + sb.append(" " + + Logger.formatDecimalNumber(this.newResponses.size()) + + " client status files updated\n"); + sb.append(" " + + Logger.formatDecimalNumber(this.newResponses.size()) + + " client document files updated\n"); + return sb.toString(); + } +} + diff --git a/src/org/torproject/onionoo/ClientsDocument.java b/src/org/torproject/onionoo/ClientsDocument.java new file mode 100644 index 0000000..c8679fc --- /dev/null +++ b/src/org/torproject/onionoo/ClientsDocument.java @@ -0,0 +1,8 @@ +/* Copyright 2014 The Tor Project + * See LICENSE for licensing information */ +package org.torproject.onionoo; + +class ClientsDocument extends Document { + +} + diff --git a/src/org/torproject/onionoo/ClientsStatus.java b/src/org/torproject/onionoo/ClientsStatus.java new file mode 100644 index 0000000..65fb341 --- /dev/null +++ b/src/org/torproject/onionoo/ClientsStatus.java @@ -0,0 +1,5 @@ +package org.torproject.onionoo; + +class ClientsStatus extends Document { +} + diff --git a/src/org/torproject/onionoo/DocumentStore.java b/src/org/torproject/onionoo/DocumentStore.java index b78f5ff..1092e25 100644 --- a/src/org/torproject/onionoo/DocumentStore.java +++ b/src/org/torproject/onionoo/DocumentStore.java @@ -121,6 +121,9 @@ public class DocumentStore { } else if (documentType.equals(WeightsStatus.class)) { directory = this.statusDir; subdirectory = "weights"; + } else if (documentType.equals(ClientsStatus.class)) { + directory = this.statusDir; + subdirectory = "clients"; } else if (documentType.equals(DetailsDocument.class)) { directory = this.outDir; subdirectory = "details"; @@ -130,6 +133,9 @@ public class DocumentStore { } else if (documentType.equals(WeightsDocument.class)) { directory = this.outDir; subdirectory = "weights"; + } else if (documentType.equals(ClientsDocument.class)) { + directory = this.outDir; + subdirectory = "clients"; } if (directory != null && subdirectory != null) { Stack<File> files = new Stack<File>(); @@ -179,7 +185,8 @@ public class DocumentStore { documentString = document.documentString; } else if (document instanceof DetailsDocument || document instanceof BandwidthDocument || - document instanceof WeightsDocument) { + document instanceof WeightsDocument || + document instanceof ClientsDocument) { Gson gson = new Gson(); documentString = gson.toJson(this); } else { @@ -267,7 +274,8 @@ public class DocumentStore { documentString); } else if (documentType.equals(DetailsDocument.class) || documentType.equals(BandwidthDocument.class) || - documentType.equals(WeightsDocument.class)) { + documentType.equals(WeightsDocument.class) || + documentType.equals(ClientsDocument.class)) { return this.retrieveParsedDocumentFile(documentType, documentString); } else { @@ -368,6 +376,11 @@ public class DocumentStore { fileName = String.format("weights/%s/%s/%s", fingerprint.substring(0, 1), fingerprint.substring(1, 2), fingerprint); + } else if (documentType.equals(ClientsStatus.class)) { + directory = this.statusDir; + fileName = String.format("clients/%s/%s/%s", + fingerprint.substring(0, 1), fingerprint.substring(1, 2), + fingerprint); } else if (documentType.equals(UpdateStatus.class)) { directory = this.outDir; fileName = "update"; @@ -385,6 +398,9 @@ public class DocumentStore { } else if (documentType.equals(WeightsDocument.class)) { directory = this.outDir; fileName = String.format("weights/%s", fingerprint); + } else if (documentType.equals(ClientsDocument.class)) { + directory = this.outDir; + fileName = String.format("clients/%s", fingerprint); } if (directory != null && fileName != null) { documentFile = new File(directory, fileName); diff --git a/src/org/torproject/onionoo/Main.java b/src/org/torproject/onionoo/Main.java index a7e9461..87e7c9a 100644 --- a/src/org/torproject/onionoo/Main.java +++ b/src/org/torproject/onionoo/Main.java @@ -36,7 +36,9 @@ public class Main { Logger.printStatusTime("Initialized bandwidth data writer"); WeightsDataWriter wdw = new WeightsDataWriter(dso, ds, t); Logger.printStatusTime("Initialized weights data writer"); - DataWriter[] dws = new DataWriter[] { ndw, bdw, wdw }; + ClientsDataWriter cdw = new ClientsDataWriter(dso, ds, t); + Logger.printStatusTime("Initialized clients data writer"); + DataWriter[] dws = new DataWriter[] { ndw, bdw, wdw, cdw }; Logger.printStatus("Reading descriptors."); dso.readRelayNetworkConsensuses(); diff --git a/src/org/torproject/onionoo/ResourceServlet.java b/src/org/torproject/onionoo/ResourceServlet.java index 165bc01..d04828f 100644 --- a/src/org/torproject/onionoo/ResourceServlet.java +++ b/src/org/torproject/onionoo/ResourceServlet.java @@ -128,6 +128,8 @@ public class ResourceServlet extends HttpServlet { resourceType = "bandwidth"; } else if (uri.startsWith("/weights")) { resourceType = "weights"; + } else if (uri.startsWith("/clients")) { + resourceType = "clients"; } else { response.sendError(HttpServletResponse.SC_BAD_REQUEST); return; diff --git a/src/org/torproject/onionoo/ResponseBuilder.java b/src/org/torproject/onionoo/ResponseBuilder.java index 502b928..b7231db 100644 --- a/src/org/torproject/onionoo/ResponseBuilder.java +++ b/src/org/torproject/onionoo/ResponseBuilder.java @@ -348,6 +348,7 @@ public class ResponseBuilder { relayFingerprintSummaryLines); Map<String, String> filteredBridges = new HashMap<String, String>( bridgeFingerprintSummaryLines); + filterByResourceType(filteredRelays, filteredBridges); filterByType(filteredRelays, filteredBridges); filterByRunning(filteredRelays, filteredBridges); filterBySearchTerms(filteredRelays, filteredBridges); @@ -371,6 +372,16 @@ public class ResponseBuilder { writeBridges(orderedBridges, pw); } + private void filterByResourceType(Map<String, String> filteredRelays, + Map<String, String> filteredBridges) { + if (this.resourceType.equals("clients")) { + filteredRelays.clear(); + } + if (this.resourceType.equals("weights")) { + filteredBridges.clear(); + } + } + private void filterByType(Map<String, String> filteredRelays, Map<String, String> filteredBridges) { if (this.type == null) { @@ -580,7 +591,7 @@ public class ResponseBuilder { filteredRelays.remove(fingerprint); } } - if (!this.bridgesByFlag.containsKey(flag)) { + if (!bridgesByFlag.containsKey(flag)) { filteredBridges.clear(); } else { Set<String> bridgesWithFlag = bridgesByFlag.get(flag); @@ -763,6 +774,8 @@ public class ResponseBuilder { return this.writeBandwidthLines(summaryLine); } else if (this.resourceType.equals("weights")) { return this.writeWeightsLines(summaryLine); + } else if (this.resourceType.equals("clients")) { + return this.writeClientsLines(summaryLine); } else { return ""; } @@ -892,4 +905,27 @@ public class ResponseBuilder { return ""; } } + + private String writeClientsLines(String summaryLine) { + String fingerprint = null; + if (summaryLine.contains("\"h\":\"")) { + fingerprint = summaryLine.substring(summaryLine.indexOf( + "\"h\":\"") + "\"h\":\"".length()); + } else { + return ""; + } + fingerprint = fingerprint.substring(0, 40); + ClientsDocument clientsDocument = documentStore.retrieve( + ClientsDocument.class, false, fingerprint); + if (clientsDocument != null && + clientsDocument.documentString != null) { + String clientsLines = clientsDocument.documentString; + clientsLines = clientsLines.substring(0, clientsLines.length() - 1); + return clientsLines; + } else { + // TODO We should probably log that we didn't find a clients + // document that we expected to exist. + return ""; + } + } } diff --git a/web/index.html b/web/index.html index 88745af..c07ca86 100644 --- a/web/index.html +++ b/web/index.html @@ -57,6 +57,7 @@ h3 .request-response { padding: 0 !important; } <li><a href="#details">Details documents</a></li> <li><a href="#bandwidth">Bandwidth documents</a></li> <li><a href="#weights">Weights documents</a></li> + <li><a href="#clients">Clients documents</a></li> </ul> </div> @@ -205,6 +206,13 @@ document</a> document</a> </li> +<li class="api-request"> +GET +https://onionoo.torproject.org/clients +returns a <a href="#clients">clients +document</a> +</li> + </ul> <h4>Parameters</h4> @@ -1719,6 +1727,234 @@ Only included for compatibility reasons with the other document types. </div>  +<div class="box"> +<a name="clients"></a> +<h3>Clients documents <a href="#clients">#</a> + +<a href="clients?limit=4">example request</a> + +</h3> + + +Added on March 10, 2014. +Clients documents contain estimates of the average number of clients +connecting to a bridge every day. +There are no clients documents available for relays, just for bridges. +Clients documents contain different time intervals and are available for +all bridges that have been running in the past week. +Clients documents contain the following fields: + + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when +the last known relay network status consensus started being valid. +Only included for compatibility reasons with the other document types. + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Empty array of objects that would represent relay clients documents. +Only included for compatibility reasons with the other document types. + +</li> + +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when +the last known bridge network status was published. +Indicates how recent the bridge clients documents in this document are. + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Array of objects representing bridge clients documents. +Each array object contains the following key-value pairs: + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +SHA-1 hash of the bridge fingerprint consisting +of 40 upper-case hexadecimal characters. + +</li> + +<li> +average_clients +<code class="typeof">object</code> +optional + +History object containing the average number of clients connecting to +this bridge. +Keys are string representation of the time period covered by the clients +history object. +Keys are fixed strings "1_week", +"1_month", "3_months", +"1_year", and "5_years". +Keys refer to the last known clients history of a bridge, not to the time +when the clients document was published. +A clients history object is only contained if the time period it covers +is not already contained in another clients history object with shorter +time period and higher data resolution. +Each clients history object contains the following key-value pairs: + + +<ul class="properties"> + +<li> +first +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data +data point in the clients history. + +</li> + +<li> +last +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data +data point in the clients history. + +</li> + +<li> +interval +<code class="typeof">number</code> +required + +Time interval between two data points in seconds. + +</li> + +<li> +factor +<code class="typeof">number</code> +required + +Factor by which subsequent clients values need to +be multiplied to get the average number of clients. +The idea is that contained clients values are normalized to a range from 0 +to 999 to reduce document size while still providing sufficient detail for +both heavily used and mostly unused bridges. + +</li> + +<li> +count +<code class="typeof">number</code> +optional + +Number of provided data points, included mostly for +debugging purposes. +Can also be derived from the number of elements in the subsequent array. + +</li> + +<li> +values +<code class="typeof">array of numbers</code> +required + +Array of normalized clients values. +May contain null values if the bridge did not report client statistics for +at least 50% of a given time period. +Contains at least two subsequent non-null values to enable drawing of line +graphs. + +</li> + +<li> +countries +<code class="typeof">object</code> +optional + +Object containing fractions of clients by country in the considered time +period. +Keys are two-letter lower-case country codes as found in a GeoIP database. +Values are numbers between 0 and 1 standing for the fraction of clients by +country. +A country is only included if at least 1% of clients came from this +country. +Omitted if the bridge did not report client statistics by country. +BETA: This field breaks compatibility +with the history objects contained in other documents pretty badly. +It might be removed in the future without notice. + +</li> + +<li> +transports +<code class="typeof">object</code> +optional + +Object containing fractions of clients by transport in the considered time +period. +Keys are transport names, or "<OR>" for the default +onion-routing transport protocol. +Values are numbers between 0 and 1 standing for the fraction of clients by +transport. +Omitted if the bridge did not report client statistics by transport. +BETA: This field breaks compatibility +with the history objects contained in other documents pretty badly. +It might be removed in the future without notice. + +</li> + +<li> +versions +<code class="typeof">object</code> +optional + +Object containing fractions of clients by IP version in the considered +time period. +Keys are either "v4" for IPv4 or "v6" +for IPv6. +Values are numbers between 0 and 1 standing for the fraction of clients by +version. +Omitted if the bridge did not report client statistics by IP version. +BETA: This field breaks compatibility +with the history objects contained in other documents pretty badly. +It might be removed in the future without notice. + +</li> + +</ul> + +</li> + +</ul> + +</li> + +</ul> + +</div>  + </body> </html>

1 0

[onionoo/master] Don't merge intervals across month ends.
by karsten＠torproject.org 11 Mar '14

11 Mar '14

commit 26d20b3451a069157e2f8ec962007588fe3b5344 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Mar 6 17:56:35 2014 +0100 Don't merge intervals across month ends. Prepares for providing monthly relay and bridge statistics (#11041). --- src/org/torproject/onionoo/BandwidthDataWriter.java | 8 +++++++- src/org/torproject/onionoo/ClientsDataWriter.java | 8 +++++++- src/org/torproject/onionoo/WeightsDataWriter.java | 8 +++++++- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/src/org/torproject/onionoo/BandwidthDataWriter.java b/src/org/torproject/onionoo/BandwidthDataWriter.java index 68bf59a..9f2f97e 100644 --- a/src/org/torproject/onionoo/BandwidthDataWriter.java +++ b/src/org/torproject/onionoo/BandwidthDataWriter.java @@ -179,6 +179,9 @@ public class BandwidthDataWriter implements DataWriter, new TreeMap<Long, long[]>(history); history.clear(); long lastStartMillis = 0L, lastEndMillis = 0L, lastBandwidth = 0L; + SimpleDateFormat dateTimeFormat = new SimpleDateFormat("yyyy-MM"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + String lastMonthString = "1970-01"; for (long[] v : uncompressedHistory.values()) { long startMillis = v[0], endMillis = v[1], bandwidth = v[2]; long intervalLengthMillis; @@ -195,9 +198,11 @@ public class BandwidthDataWriter implements DataWriter, } else { intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; } + String monthString = dateTimeFormat.format(startMillis); if (lastEndMillis == startMillis && ((lastEndMillis - 1L) / intervalLengthMillis) == - ((endMillis - 1L) / intervalLengthMillis)) { + ((endMillis - 1L) / intervalLengthMillis) && + lastMonthString.equals(monthString)) { lastEndMillis = endMillis; lastBandwidth += bandwidth; } else { @@ -209,6 +214,7 @@ public class BandwidthDataWriter implements DataWriter, lastEndMillis = endMillis; lastBandwidth = bandwidth; } + lastMonthString = monthString; } if (lastStartMillis > 0L) { history.put(lastStartMillis, new long[] { lastStartMillis, diff --git a/src/org/torproject/onionoo/ClientsDataWriter.java b/src/org/torproject/onionoo/ClientsDataWriter.java index 9e868a4..b956f59 100644 --- a/src/org/torproject/onionoo/ClientsDataWriter.java +++ b/src/org/torproject/onionoo/ClientsDataWriter.java @@ -356,6 +356,9 @@ public class ClientsDataWriter implements DataWriter, DescriptorListener { SortedSet<ResponseHistory> compressedHistory = new TreeSet<ResponseHistory>(); ResponseHistory lastResponses = null; + SimpleDateFormat dateTimeFormat = new SimpleDateFormat("yyyy-MM"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + String lastMonthString = "1970-01"; for (ResponseHistory responses : history) { long intervalLengthMillis; if (this.now - responses.endMillis <= @@ -367,10 +370,12 @@ public class ClientsDataWriter implements DataWriter, DescriptorListener { } else { intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; } + String monthString = dateTimeFormat.format(responses.startMillis); if (lastResponses != null && lastResponses.endMillis == responses.startMillis && ((lastResponses.endMillis - 1L) / intervalLengthMillis) == - ((responses.endMillis - 1L) / intervalLengthMillis)) { + ((responses.endMillis - 1L) / intervalLengthMillis) && + lastMonthString.equals(monthString)) { lastResponses.addResponses(responses); } else { if (lastResponses != null) { @@ -378,6 +383,7 @@ public class ClientsDataWriter implements DataWriter, DescriptorListener { } lastResponses = responses; } + lastMonthString = monthString; } if (lastResponses != null) { compressedHistory.add(lastResponses); diff --git a/src/org/torproject/onionoo/WeightsDataWriter.java b/src/org/torproject/onionoo/WeightsDataWriter.java index 81b412c..855e3e9 100644 --- a/src/org/torproject/onionoo/WeightsDataWriter.java +++ b/src/org/torproject/onionoo/WeightsDataWriter.java @@ -371,6 +371,9 @@ public class WeightsDataWriter implements DataWriter, DescriptorListener { new TreeMap<long[], double[]>(history.comparator()); long lastStartMillis = 0L, lastEndMillis = 0L; double[] lastWeights = null; + SimpleDateFormat dateTimeFormat = new SimpleDateFormat("yyyy-MM"); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + String lastMonthString = "1970-01"; for (Map.Entry<long[], double[]> e : history.entrySet()) { long startMillis = e.getKey()[0], endMillis = e.getKey()[1]; double[] weights = e.getValue(); @@ -386,9 +389,11 @@ public class WeightsDataWriter implements DataWriter, DescriptorListener { } else { intervalLengthMillis = 10L * 24L * 60L * 60L * 1000L; } + String monthString = dateTimeFormat.format(startMillis); if (lastEndMillis == startMillis && ((lastEndMillis - 1L) / intervalLengthMillis) == - ((endMillis - 1L) / intervalLengthMillis)) { + ((endMillis - 1L) / intervalLengthMillis) && + lastMonthString.equals(monthString)) { double lastIntervalInHours = (double) ((lastEndMillis - lastStartMillis) / 60L * 60L * 1000L); double currentIntervalInHours = (double) ((endMillis @@ -410,6 +415,7 @@ public class WeightsDataWriter implements DataWriter, DescriptorListener { lastEndMillis = endMillis; lastWeights = weights; } + lastMonthString = monthString; } if (lastStartMillis > 0L) { compressedHistory.put(new long[] { lastStartMillis, lastEndMillis },

1 0

[translation/tails-misc_completed] Update translations for tails-misc_completed
by translation＠torproject.org 11 Mar '14

11 Mar '14

commit 465d10a9dba1641264c95b894f0071b1bd317e0c Author: Translation commit bot <translation(a)torproject.org> Date: Tue Mar 11 07:45:45 2014 +0000 Update translations for tails-misc_completed --- km.po | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/km.po b/km.po index d52ad1d..432de04 100644 --- a/km.po +++ b/km.po @@ -3,14 +3,15 @@ # This file is distributed under the same license as the PACKAGE package. # # Translators: +# khoemsokhem <sokhem(a)open.org.kh>, 2014 # soksophea <sksophea(a)gmail.com>, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2014-01-10 12:23+0100\n" -"PO-Revision-Date: 2014-01-29 09:40+0000\n" -"Last-Translator: soksophea <sksophea(a)gmail.com>\n" +"POT-Creation-Date: 2014-03-06 14:52+0100\n" +"PO-Revision-Date: 2014-03-11 07:40+0000\n" +"Last-Translator: khoemsokhem <sokhem(a)open.org.kh>\n" "Language-Team: Khmer (http://www.transifex.com/projects/p/torproject/language/km/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -246,21 +247,21 @@ msgstr "ព័ត៌មានស្ថាបនា៖\n%s" msgid "About Tails" msgstr "អំពី Tails" -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:115 -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:121 -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:125 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:118 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:124 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:128 msgid "Your additional software" msgstr "កម្មវិធីរបស់អ្នកបន្ថែម" -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:116 -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:126 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:119 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:129 msgid "" "The upgrade failed. This might be due to a network problem. Please check " "your network connection, try to restart Tails, or read the system log to " "understand better the problem." msgstr "ការធ្វើបច្ចុប្បន្នភាពបានបរាជ័យ។ វាអាចដោយសារបញ្ហាបណ្ដាញ។ សូមពិនិត្យការតភ្ជាប់បណ្ដាញរបស់អ្នក រួចសាកល្បងចាប់ផ្ដើម Tails ឡើងវិញ ឬអានកំណត់ហេតុប្រព័ន្ធដើម្បីដឹងច្បាស់អំពីបញ្ហា។" -#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:122 +#: config/chroot_local-includes/usr/local/sbin/tails-additional-software:125 msgid "The upgrade was successful." msgstr "បានធ្វើបច្ចុប្បន្នភាពដោយជោគជ័យ" @@ -278,10 +279,45 @@ msgstr "Tor ត្រូវការនាឡិកាដែលទៀ msgid "Failed to synchronize the clock!" msgstr "ការធ្វើសមកាលកម្មនាឡិកាបានបរាជ័យ!" +#: config/chroot_local-includes/usr/local/sbin/tails-restricted-network-detector:38 +msgid "Network connection blocked?" +msgstr "បានទប់ស្កាត់ការភ្ជាប់បណ្ដាញ?" + +#: config/chroot_local-includes/usr/local/sbin/tails-restricted-network-detector:40 +msgid "" +"It looks like you are blocked from the network. This may be related to the " +"MAC spoofing feature. For more information, see the <a " +"href=\\\"file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html#blocked\\\">MAC" +" spoofing documentation</a>." +msgstr "វាទំនងជាអ្នកត្រូវបានទប់ស្កាត់ពីបណ្ដាញ។ វាប្រហែលជាទាក់ទងនឹងលក្ខណៈក្លែង MAC ។ ចំពោះព័ត៌មានបន្ថែម សូមមើល <a href=\\\"file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html#blocked\\\">ឯកសារពីការក្លែង MAC</a> ។" + #: config/chroot_local-includes/usr/local/bin/tails-security-check:145 msgid "This version of Tails has known security issues:" msgstr "កំណែរបស់ Tails នេះគឺស្គាល់បញ្ហាសុវត្ថិភាព៖" +#: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:29 +#, sh-format +msgid "Network card ${nic} disabled" +msgstr "បានបិទកាតបណ្ដាញ ${nic}" + +#: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:30 +#, sh-format +msgid "" +"MAC spoofing failed for network card ${nic_name} (${nic}) so it is temporarily disabled.\n" +"You might prefer to restart Tails and disable MAC spoofing. See the <a href='file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html'>documentation</a>." +msgstr "បានបរាជ័យក្នុងការក្លែង MAC សម្រាប់កាតបណ្ដាញ ${nic_name} (${nic}) ដូច្នេះវាត្រូវបានបិទជាបណ្ដោះអាសន្ន។\nអ្នកអាចចាប់ផ្ដើម Tails ឡើងវិញ និងបិទការក្លែង MAC ។ សូមមើល <a href='file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html'>ឯកសារ</a> ។" + +#: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:39 +msgid "All networking disabled" +msgstr "បានបិទការភ្ជាប់បណ្ដាញទាំងអស់" + +#: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:40 +#, sh-format +msgid "" +"MAC spoofing failed for network card ${nic_name} (${nic}). The error recovery also failed so all networking is disabled.\n" +"You might prefer to restart Tails and disable MAC spoofing. See the <a href='file:///usr/share/doc/first_steps/startup_options/mac_spoofing.en.html'>documentation</a>." +msgstr "បានបរាជ័យក្នុងក្លែង MAC សម្រាប់កាតបណ្ដាញ ${nic_name} (${nic}) ។ ការសង្គ្រោះកំហុសក៏បរាជ័យដែរ ដូច្នេះការភ្ជាប់បណ្ដាញទាំងអស់ត្រូវបានបិទ។\nអ្នកអាចចាប់ផ្ដើម Tails ឡើងវិញ និងបិទការក្លែង MAC ។ សូមមើល <a href='file:///usr/share/doc/first_steps/startup_options/mac_spoofing.en.html'>ឯកសារ</a> ។" + #: config/chroot_local-includes/usr/local/bin/tails-start-i2p:62 msgid "Starting I2P..." msgstr "កំពុងចាប់ផ្ដើម I2P..." @@ -408,10 +444,15 @@ msgstr "TrueCrypt នឹងត្រូវបានលុបចេញ msgid "Report an error" msgstr "រាយការណ៍កំហុស" -#: ../config/chroot_local-includes/etc/skel/Desktop/Tails_documentation.desktop.in.h:1 +#: ../config/chroot_local-includes/etc/skel/Desktop/tails-documentation.desktop.in.h:1 +#: ../config/chroot_local-includes/usr/share/applications/tails-documentation.desktop.in.h:1 msgid "Tails documentation" msgstr "ឯកសារ Tails" +#: ../config/chroot_local-includes/usr/share/applications/tails-documentation.desktop.in.h:2 +msgid "Learn how to use Tails" +msgstr "សិក្សាពីវិធីប្រើ Tails" + #: ../config/chroot_local-includes/usr/share/applications/i2p.desktop.in.h:1 msgid "Anonymous overlay network " msgstr "អនាមិកនៅលើបណ្ដាញ"

1 0

[translation/tails-misc] Update translations for tails-misc
by translation＠torproject.org 11 Mar '14

11 Mar '14

commit 924705e0abd5d0338080c712c77dc9cd4cce35df Author: Translation commit bot <translation(a)torproject.org> Date: Tue Mar 11 07:45:43 2014 +0000 Update translations for tails-misc --- km.po | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/km.po b/km.po index 23aff84..432de04 100644 --- a/km.po +++ b/km.po @@ -3,14 +3,15 @@ # This file is distributed under the same license as the PACKAGE package. # # Translators: +# khoemsokhem <sokhem(a)open.org.kh>, 2014 # soksophea <sksophea(a)gmail.com>, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2014-03-06 14:52+0100\n" -"PO-Revision-Date: 2014-03-07 08:49+0000\n" -"Last-Translator: runasand <runa.sandvik(a)gmail.com>\n" +"PO-Revision-Date: 2014-03-11 07:40+0000\n" +"Last-Translator: khoemsokhem <sokhem(a)open.org.kh>\n" "Language-Team: Khmer (http://www.transifex.com/projects/p/torproject/language/km/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -280,7 +281,7 @@ msgstr "ការធ្វើសមកាលកម្មនាឡិក #: config/chroot_local-includes/usr/local/sbin/tails-restricted-network-detector:38 msgid "Network connection blocked?" -msgstr "" +msgstr "បានទប់ស្កាត់ការភ្ជាប់បណ្ដាញ?" #: config/chroot_local-includes/usr/local/sbin/tails-restricted-network-detector:40 msgid "" @@ -288,7 +289,7 @@ msgid "" "MAC spoofing feature. For more information, see the <a " "href=\\\"file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html#blocked\\\">MAC" " spoofing documentation</a>." -msgstr "" +msgstr "វាទំនងជាអ្នកត្រូវបានទប់ស្កាត់ពីបណ្ដាញ។ វាប្រហែលជាទាក់ទងនឹងលក្ខណៈក្លែង MAC ។ ចំពោះព័ត៌មានបន្ថែម សូមមើល <a href=\\\"file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html#blocked\\\">ឯកសារពីការក្លែង MAC</a> ។" #: config/chroot_local-includes/usr/local/bin/tails-security-check:145 msgid "This version of Tails has known security issues:" @@ -297,25 +298,25 @@ msgstr "កំណែរបស់ Tails នេះគឺស្គាល #: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:29 #, sh-format msgid "Network card ${nic} disabled" -msgstr "" +msgstr "បានបិទកាតបណ្ដាញ ${nic}" #: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:30 #, sh-format msgid "" "MAC spoofing failed for network card ${nic_name} (${nic}) so it is temporarily disabled.\n" "You might prefer to restart Tails and disable MAC spoofing. See the <a href='file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html'>documentation</a>." -msgstr "" +msgstr "បានបរាជ័យក្នុងការក្លែង MAC សម្រាប់កាតបណ្ដាញ ${nic_name} (${nic}) ដូច្នេះវាត្រូវបានបិទជាបណ្ដោះអាសន្ន។\nអ្នកអាចចាប់ផ្ដើម Tails ឡើងវិញ និងបិទការក្លែង MAC ។ សូមមើល <a href='file:///usr/share/doc/tails/website/doc/first_steps/startup_options/mac_spoofing.en.html'>ឯកសារ</a> ។" #: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:39 msgid "All networking disabled" -msgstr "" +msgstr "បានបិទការភ្ជាប់បណ្ដាញទាំងអស់" #: config/chroot_local-includes/usr/local/sbin/tails-spoof-mac:40 #, sh-format msgid "" "MAC spoofing failed for network card ${nic_name} (${nic}). The error recovery also failed so all networking is disabled.\n" "You might prefer to restart Tails and disable MAC spoofing. See the <a href='file:///usr/share/doc/first_steps/startup_options/mac_spoofing.en.html'>documentation</a>." -msgstr "" +msgstr "បានបរាជ័យក្នុងក្លែង MAC សម្រាប់កាតបណ្ដាញ ${nic_name} (${nic}) ។ ការសង្គ្រោះកំហុសក៏បរាជ័យដែរ ដូច្នេះការភ្ជាប់បណ្ដាញទាំងអស់ត្រូវបានបិទ។\nអ្នកអាចចាប់ផ្ដើម Tails ឡើងវិញ និងបិទការក្លែង MAC ។ សូមមើល <a href='file:///usr/share/doc/first_steps/startup_options/mac_spoofing.en.html'>ឯកសារ</a> ។" #: config/chroot_local-includes/usr/local/bin/tails-start-i2p:62 msgid "Starting I2P..." @@ -450,7 +451,7 @@ msgstr "ឯកសារ Tails" #: ../config/chroot_local-includes/usr/share/applications/tails-documentation.desktop.in.h:2 msgid "Learn how to use Tails" -msgstr "" +msgstr "សិក្សាពីវិធីប្រើ Tails" #: ../config/chroot_local-includes/usr/share/applications/i2p.desktop.in.h:1 msgid "Anonymous overlay network "

1 0

[translation/tails-greeter_completed] Update translations for tails-greeter_completed
by translation＠torproject.org 11 Mar '14

11 Mar '14

commit 3e3f94682c4176a8c3ac1663d8ef883bec48f3ac Author: Translation commit bot <translation(a)torproject.org> Date: Tue Mar 11 07:45:29 2014 +0000 Update translations for tails-greeter_completed --- km/km.po | 48 ++++++++++++++++++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 12 deletions(-) diff --git a/km/km.po b/km/km.po index cb71130..6cd6020 100644 --- a/km/km.po +++ b/km/km.po @@ -9,8 +9,8 @@ msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2014-03-04 11:11+0100\n" -"PO-Revision-Date: 2014-03-07 04:30+0000\n" +"POT-Creation-Date: 2014-03-06 17:16+0100\n" +"PO-Revision-Date: 2014-03-11 07:44+0000\n" "Last-Translator: khoemsokhem <sokhem(a)open.org.kh>\n" "Language-Team: Khmer (http://www.transifex.com/projects/p/torproject/language/km/)\n" "MIME-Version: 1.0\n" @@ -36,8 +36,8 @@ msgid "Use persistence?" msgstr "ប្រើ persistence?" #: ../glade/persistencewindow.glade.h:5 -msgid "<a href=\"doc/first_steps/persistence/use.en.html\">Help</a>" -msgstr "<a href=\"doc/first_steps/persistence/use.en.html\">ជំនួយ</a>" +msgid "<a href=\"doc/first_steps/persistence/use.en.html\">Documentation</a>" +msgstr "<a href=\"doc/first_steps/persistence/use.en.html\">ឯកសារ</a>" #: ../glade/persistencewindow.glade.h:6 msgid "Yes" @@ -70,8 +70,8 @@ msgstr "ពាក្យសម្ងាត់សម្រាប់កា #: ../glade/optionswindow.glade.h:4 msgid "" "<a " -"href=\"doc/first_steps/startup_options/administration_password.en.html\">Help</a>" -msgstr "<a href=\"doc/first_steps/startup_options/administration_password.en.html\">ជំនួយ</a>" +"href=\"doc/first_steps/startup_options/administration_password.en.html\">Documentation</a>" +msgstr "<a href=\"doc/first_steps/startup_options/administration_password.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:5 msgid "" @@ -92,14 +92,14 @@ msgid "Passwords do not match" msgstr "ពាក្យសម្ងាត់មិនផ្គូផ្គង" #: ../glade/optionswindow.glade.h:10 -msgid "Windows Camouflage" -msgstr "Windows Camouflage" +msgid "Windows camouflage" +msgstr "Windows camouflage" #: ../glade/optionswindow.glade.h:11 msgid "" "<a " -"href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">Help</a>" -msgstr "<a href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">ជំនួយ</a>" +"href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">Documentation</a>" +msgstr "<a href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:12 msgid "" @@ -116,8 +116,10 @@ msgid "MAC address spoofing" msgstr "បញ្ឆោតអាសយដ្ឋាន MAC" #: ../glade/optionswindow.glade.h:15 -msgid "<a href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">Help</a>" -msgstr "<a href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">ជំនួយ</a>" +msgid "" +"<a " +"href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">Documentation</a>" +msgstr "<a href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:16 msgid "" @@ -135,6 +137,28 @@ msgstr "ជាទូទៅវាកាន់តែមានសុ msgid "Spoof all MAC addresses" msgstr "បញ្ឆោតអាសយដ្ឋាន MAC ទាំងអស់" +#: ../glade/optionswindow.glade.h:19 +msgid "Network configuration" +msgstr "ការកំណត់រចនាសម្ព័ន្ធបណ្ដាញ" + +#: ../glade/optionswindow.glade.h:20 +msgid "" +"<a " +"href=\"doc/first_steps/startup_options/network_configuration.en.html\">Documentation</a>" +msgstr "<a href=\"doc/first_steps/startup_options/network_configuration.en.html\">ឯកសារ</a>" + +#: ../glade/optionswindow.glade.h:21 +msgid "" +"This computer's Internet connection is clear of obstacles. You would like to" +" connect directly to the Tor network." +msgstr "ការភ្ជាប់អ៊ីនធឺណិតកុំព្យូទ័រនេះគ្មានបញ្ហាទេ។ អ្នកចង់ភ្ជាប់ដោយផ្ទាល់ទៅបណ្ដាញ Tor ។" + +#: ../glade/optionswindow.glade.h:22 +msgid "" +"This computer's Internet connection is censored, filtered, or proxied. You " +"need to configure bridge, firewall, or proxy settings." +msgstr "ការភ្ជាប់អ៊ីនធឺណិតកុំព្យូទ័រនេះត្រូវបានពិនិត្យ ច្រោះ ឬកំណត់ប្រូកស៊ី។ អ្នកត្រូវកំណត់រចនាសម្ព័ន្ធប៊្រីត ជញ្ជាំងភ្លើង ឬការកំណត់ប្រូកស៊ី។" + #: ../glade/langpanel.glade.h:1 msgid " " msgstr " "

1 0

[translation/tails-greeter] Update translations for tails-greeter
by translation＠torproject.org 11 Mar '14

11 Mar '14

commit 06c8525aadd9a54826fe553407a4860370d703a1 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Mar 11 07:45:27 2014 +0000 Update translations for tails-greeter --- km/km.po | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/km/km.po b/km/km.po index 349826d..6cd6020 100644 --- a/km/km.po +++ b/km/km.po @@ -10,8 +10,8 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2014-03-06 17:16+0100\n" -"PO-Revision-Date: 2014-03-07 08:52+0000\n" -"Last-Translator: runasand <runa.sandvik(a)gmail.com>\n" +"PO-Revision-Date: 2014-03-11 07:44+0000\n" +"Last-Translator: khoemsokhem <sokhem(a)open.org.kh>\n" "Language-Team: Khmer (http://www.transifex.com/projects/p/torproject/language/km/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -37,7 +37,7 @@ msgstr "ប្រើ persistence?" #: ../glade/persistencewindow.glade.h:5 msgid "<a href=\"doc/first_steps/persistence/use.en.html\">Documentation</a>" -msgstr "" +msgstr "<a href=\"doc/first_steps/persistence/use.en.html\">ឯកសារ</a>" #: ../glade/persistencewindow.glade.h:6 msgid "Yes" @@ -71,7 +71,7 @@ msgstr "ពាក្យសម្ងាត់សម្រាប់កា msgid "" "<a " "href=\"doc/first_steps/startup_options/administration_password.en.html\">Documentation</a>" -msgstr "" +msgstr "<a href=\"doc/first_steps/startup_options/administration_password.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:5 msgid "" @@ -93,13 +93,13 @@ msgstr "ពាក្យសម្ងាត់មិនផ្គូផ្ #: ../glade/optionswindow.glade.h:10 msgid "Windows camouflage" -msgstr "" +msgstr "Windows camouflage" #: ../glade/optionswindow.glade.h:11 msgid "" "<a " "href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">Documentation</a>" -msgstr "" +msgstr "<a href=\"doc/first_steps/startup_options/windows_camouflage.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:12 msgid "" @@ -119,7 +119,7 @@ msgstr "បញ្ឆោតអាសយដ្ឋាន MAC" msgid "" "<a " "href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">Documentation</a>" -msgstr "" +msgstr "<a href=\"doc/first_steps/startup_options/mac_spoofing.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:16 msgid "" @@ -139,25 +139,25 @@ msgstr "បញ្ឆោតអាសយដ្ឋាន MAC ទាំងអស #: ../glade/optionswindow.glade.h:19 msgid "Network configuration" -msgstr "" +msgstr "ការកំណត់រចនាសម្ព័ន្ធបណ្ដាញ" #: ../glade/optionswindow.glade.h:20 msgid "" "<a " "href=\"doc/first_steps/startup_options/network_configuration.en.html\">Documentation</a>" -msgstr "" +msgstr "<a href=\"doc/first_steps/startup_options/network_configuration.en.html\">ឯកសារ</a>" #: ../glade/optionswindow.glade.h:21 msgid "" "This computer's Internet connection is clear of obstacles. You would like to" " connect directly to the Tor network." -msgstr "" +msgstr "ការភ្ជាប់អ៊ីនធឺណិតកុំព្យូទ័រនេះគ្មានបញ្ហាទេ។ អ្នកចង់ភ្ជាប់ដោយផ្ទាល់ទៅបណ្ដាញ Tor ។" #: ../glade/optionswindow.glade.h:22 msgid "" "This computer's Internet connection is censored, filtered, or proxied. You " "need to configure bridge, firewall, or proxy settings." -msgstr "" +msgstr "ការភ្ជាប់អ៊ីនធឺណិតកុំព្យូទ័រនេះត្រូវបានពិនិត្យ ច្រោះ ឬកំណត់ប្រូកស៊ី។ អ្នកត្រូវកំណត់រចនាសម្ព័ន្ធប៊្រីត ជញ្ជាំងភ្លើង ឬការកំណត់ប្រូកស៊ី។" #: ../glade/langpanel.glade.h:1 msgid " "

1 0

[translation/liveusb-creator_completed] Update translations for liveusb-creator_completed
by translation＠torproject.org 11 Mar '14

11 Mar '14

commit bba5e0893830e0f7e33463f38be738a2eb8bcfa6 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Mar 11 07:45:22 2014 +0000 Update translations for liveusb-creator_completed --- km/km.po | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/km/km.po b/km/km.po index 114ec16..fced4a6 100644 --- a/km/km.po +++ b/km/km.po @@ -3,14 +3,15 @@ # This file is distributed under the same license as the PACKAGE package. # # Translators: +# khoemsokhem <sokhem(a)open.org.kh>, 2014 # soksophea <sksophea(a)gmail.com>, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2014-01-29 14:16+0100\n" -"PO-Revision-Date: 2014-02-04 03:30+0000\n" -"Last-Translator: soksophea <sksophea(a)gmail.com>\n" +"POT-Creation-Date: 2014-03-06 18:42+0100\n" +"PO-Revision-Date: 2014-03-11 07:30+0000\n" +"Last-Translator: khoemsokhem <sokhem(a)open.org.kh>\n" "Language-Team: Khmer (http://www.transifex.com/projects/p/torproject/language/km/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -250,11 +251,8 @@ msgid "No mount points found" msgstr "រកមិនឃើញចំណុចម៉ោន" #: ../liveusb/creator.py:393 -#, python-format -msgid "" -"Not enough free space on device.\n" -"%dMB ISO + %dMB overlay > %dMB free space" -msgstr "មិនមានទំហំទំនេរគ្រប់គ្រាន់នៅលើឧបករណ៍។\n%dMB ISO + %dMB overlay > ទំហំទំនេរ %dMB" +msgid "Not enough free space on device." +msgstr "មិនមានទំហំទំនេរគ្រប់គ្រាន់លើឧបករណ៍។" #: ../liveusb/gui.py:554 msgid "Partition is FAT16; Restricting overlay size to 2G"

1 0