August 2020 - tor-commits - lists.torproject.org

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 07 Aug '20

07 Aug '20

commit 37a07f9f92292506976ec14595bc48bce98588cb Author: Translation commit bot <translation(a)torproject.org> Date: Fri Aug 7 19:47:58 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+ar.po | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/contents+ar.po b/contents+ar.po index 9e3417c5ea..d028cc833a 100644 --- a/contents+ar.po +++ b/contents+ar.po @@ -3531,6 +3531,9 @@ msgid "" " import the key using the … [View More]**Workaround (using a public key)** section " "instead." msgstr "" +"إذا تلقيت رسالة خطأ ، حدث خطأ ما ولا يمكنك المتابعة حتى تكتشف سبب عدم نجاح " +"ذلك. قد تتمكن من استيراد المفتاح باستخدام قسم ** الحل البديل (باستخدام مفتاح" +" عام) ** بدلاً من ذلك." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3556,6 +3559,8 @@ msgid "" "gone wrong and you cannot continue until you've figured out why this didn't " "work." msgstr "" +"إذا لم يكن \". / tor.keyring\" موجودًا بعد تشغيل هذا الأمر ، فقد حدث خطأ ما " +"ولا يمكنك المتابعة حتى تكتشف سبب عدم نجاح ذلك." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3589,6 +3594,9 @@ msgid "" " you will have downloaded a different version than 9.0 and you may not have " "chosen the English (en-US) version." msgstr "" +"لاحظ أن هذه الأوامر تستخدم أمثلة لأسماء الملفات وأن أسمائك ستكون مختلفة: " +"ستكون قد قمت بتنزيل إصدار مختلف عن 9.0 وربما لم تختر الإصدار الإنجليزي (en-" +"US)." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3605,6 +3613,8 @@ msgid "" "gpgv --keyring ./tor.keyring ~/Downloads/TorBrowser-9.0-osx64_en-US.dmg.asc " "~/Downloads/TorBrowser-9.0-osx64_en-US.dmg" msgstr "" +"gpgv --keyring ./tor.keyring ~/Downloads/TorBrowser-9.0-osx64_en-US.dmg.asc " +"~/Downloads/TorBrowser-9.0-osx64_en-US.dmg" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3618,6 +3628,8 @@ msgid "" "gpgv --keyring ./tor.keyring ~/Downloads/tor-browser-linux64-9.0_en-" "US.tar.xz.asc ~/Downloads/tor-browser-linux64-9.0_en-US.tar.xz" msgstr "" +"gpgv --keyring ./tor.keyring ~/Downloads/tor-browser-linux64-9.0_en-" +"US.tar.xz.asc ~/Downloads/tor-browser-linux64-9.0_en-US.tar.xz" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3650,6 +3662,9 @@ msgid "" "something went wrong with one of the previous steps, or you forgot that " "these commands use example file names and yours will be a little different." msgstr "" +"إذا تلقيت رسائل خطأ تحتوي على \"لا يوجد مثل هذا الملف أو الدليل\" ، فإما أن " +"هناك خطأ ما في إحدى الخطوات السابقة ، أو نسيت أن هذه الأوامر تستخدم أمثلة " +"لأسماء الملفات وستكون أوامرك مختلفة قليلاً." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -3730,6 +3745,11 @@ msgid "" "2019](https://www.zdnet.com/article/firefox-to-add-tor-browser-anti-" "fingerprinting-technique-called-letterboxing/)." msgstr "" +"يأتي متصفح Tor 9 مزودًا بالدفاع عن البصمات لتلك السيناريوهات أيضًا ، وهو ما " +"يُسمى [Letterboxing] (https://en.wikipedia.org/wiki/Letterboxing_ (تصوير)) ،" +" وهي تقنية طورتها Mozilla و [تم تقديمها في عام 2019] " +"(https://www.zdnet.com/article/firefox-to-add-tor-browser-anti-" +"fingerprinting-technique-called-letterboxing/)." #: https//support.torproject.org/tbb/maximized-torbrowser-window/ #: (content/tbb/maximized-torbrowser-window/contents+en.lrquestion.description) @@ -4286,6 +4306,9 @@ msgid "" "(\"≡\")](../../glossary/hamburger-menu), then click on \"Preferences\", and " "finally on \"Tor\" in the side bar." msgstr "" +"عندما يكون متصفح Tor مفتوحًا ، يمكنك الانتقال إلى [قائمة الهامبرغر (\"≡\")] " +"(../../ المسرد / قائمة الهامبرغر) ، ثم النقر على \"التفضيلات\" ، وأخيراً على" +" \"Tor\" في شريط جانبي." #: https//support.torproject.org/tbb/tbb-21/ #: (content/tbb/tbb-21/contents+en.lrquestion.description) @@ -4479,6 +4502,8 @@ msgid "" "Both options are located in the [hamburger menu (\"≡\")](../../glossary" "/hamburger-menu)." msgstr "" +"يوجد كلا الخيارين في [قائمة الهامبرغر (\"\")] (../../ المسرد / قائمة " +"الهامبرغر)." #: https//support.torproject.org/tbb/tbb-29/ #: (content/tbb/tbb-29/contents+en.lrquestion.description) @@ -4706,6 +4731,8 @@ msgid "" "There are methods for setting Tor Browser as your default browser, but those" " methods may not work always or in every operating system." msgstr "" +"هناك طرق لتعيين متصفح Tor كمتصفحك الافتراضي ، ولكن هذه الطرق قد لا تعمل " +"دائمًا أو في كل نظام تشغيل." #: https//support.torproject.org/tbb/tbb-32/ #: (content/tbb/tbb-32/contents+en.lrquestion.description) [View Less]

1 0

[metrics-web/master] Update to latest metrics-lib version 2.14.0.
by karsten＠torproject.org 07 Aug '20

07 Aug '20

commit 09d94661dbca84eb9a29b2e6863e797a0b18f351 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Aug 7 21:07:50 2020 +0200 Update to latest metrics-lib version 2.14.0. As a result, we'll be able to process OnionPerf analysis JSON file version 3.0. --- CHANGELOG.md | 2 +- build.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 132f269..f8fe202 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,… [View More]

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 07 Aug '20

07 Aug '20

commit d0cbdf9605dda8aaa632a11221392dbec45940e1 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Aug 7 19:18:04 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+ar.po | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/contents+ar.po b/contents+ar.po index 63ad0ddcbc..9e3417c5ea 100644 --- a/contents+ar.po +++ b/contents+ar.po @@ -2543,6 +2543,8 @@ msgid "" "However, by default,… [View More] it does not prevent somebody watching your Internet " "traffic from learning that you're using Tor." msgstr "" +"ومع ذلك ، بشكل افتراضي ، فإنه لا يمنع أي شخص يراقب حركة المرور الخاصة بك على" +" الإنترنت من معرفة أنك تستخدم Tor." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2551,6 +2553,9 @@ msgid "" " [bridge](https://tb-manual.torproject.org/bridges/) rather than connecting " "directly to the Tor network." msgstr "" +"إذا كان هذا يهمك ، فيمكنك تقليل هذه المخاطر عن طريق تكوين Tor لاستخدام [جسر]" +" (https://tb-manual.torproject.org/bridges/) بدلاً من الاتصال مباشرة بشبكة " +"Tor." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2559,21 +2564,23 @@ msgid "" "there are near you and the more diverse their interests, the less dangerous " "it will be that you are one of them." msgstr "" +"في نهاية المطاف ، فإن أفضل حماية هي النهج الاجتماعي: فكلما زاد عدد مستخدمي " +"Tor بالقرب منك وكلما تنوع اهتماماتهم ، قل خطر كونك واحدًا منهم." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "Convince other people to use Tor, too!" -msgstr "" +msgstr "إقناع الآخرين باستخدام Tor أيضًا!" #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "Be smart and learn more." -msgstr "" +msgstr "كن ذكيا وتعلم المزيد." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "Understand what Tor does and does not offer." -msgstr "" +msgstr "افهم ما يقدمه Tor وما لا يقدمه." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2581,6 +2588,8 @@ msgid "" "This list of pitfalls isn't complete, and we need your help [identifying and" " documenting all the issues](https://community.torproject.org/)." msgstr "" +"قائمة المخاطر هذه ليست كاملة ، ونحتاج إلى مساعدتك [تحديد جميع المشكلات وتوثيقها\n" +"(https://community.torproject.org/)" #: https//support.torproject.org/about/backdoor/ #: (content/about/backdoor/contents+en.lrquestion.title) @@ -3322,6 +3331,8 @@ msgid "" "These are example file names and will not exactly match the file names that " "you download." msgstr "" +"هذه أمثلة لأسماء الملفات ولن تتطابق تمامًا مع أسماء الملفات التي تقوم " +"بتنزيلها." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) [View Less]

1 0

[metrics-lib/master] Bump version to 2.14.0-dev.
by karsten＠torproject.org 07 Aug '20

07 Aug '20

commit cf4830cac377503697c7e688e995a3f3cea5225e Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Aug 7 21:01:10 2020 +0200 Bump version to 2.14.0-dev. --- CHANGELOG.md | 3 +++ build.xml | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 06c9bd7..c6886e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +# Changes in version 2.??.? - 2020-??-?? + + # Changes in version 2.14.0 - 2020-08-07 * Medium … [View More]

1 0

[metrics-lib/master] Prepare for 2.14.0 release.
by karsten＠torproject.org 07 Aug '20

07 Aug '20

commit 750f6471a42c6cbdbfc3755076ae3a50dc822e67 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Aug 7 20:54:48 2020 +0200 Prepare for 2.14.0 release. --- CHANGELOG.md | 2 +- build.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bb55312..06c9bd7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -# Changes in version 2.??.? - 2020-??-?? +# Changes in version 2.14.0 - 2020-08-07 * Medium … [View More]

1 0

[metrics-lib/master] Parse OnionPerf analysis results format v3.0.
by karsten＠torproject.org 07 Aug '20

07 Aug '20

commit b44c6bccb46ca1eb9c861a84dfcd7d5d2047dee4 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jul 16 21:37:19 2020 +0200 Parse OnionPerf analysis results format v3.0. Implements tpo/metrics/library#40001. --- CHANGELOG.md | 1 + .../onionperf/OnionPerfAnalysisConverter.java | 235 ++++++++++++++++----- .../onionperf/ParsedOnionPerfAnalysis.java | 206 ++++++++++++++++++ .../onionperf/… [View More]TorperfResultsBuilder.java | 38 ++++ .../onionperf/OnionPerfAnalysisConverterTest.java | 80 ++++++- ...20-07-13.op-nl-test1.onionperf.analysis.json.xz | Bin 0 -> 1736 bytes 6 files changed, 499 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7bb22c4..4679688 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ * Medium changes - Extend Torperf results to provide error codes. + - Parse OnionPerf analysis results format version 3.0. # Changes in version 2.13.0 - 2020-05-16 diff --git a/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java b/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java index 61fd173..f1b9d11 100644 --- a/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java +++ b/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java @@ -69,25 +69,32 @@ public class OnionPerfAnalysisConverter { * Torperf results. */ public List<Descriptor> asTorperfResults() throws DescriptorParseException { - ParsedOnionPerfAnalysis parsedOnionPerfAnalysis; + ParsedOnionPerfAnalysis parsedOnionPerfAnalysis + = this.parseOnionPerfAnalysis(); + this.verifyDocumentTypeAndVersion(parsedOnionPerfAnalysis); + StringBuilder formattedTorperfResults + = this.formatTorperfResults(parsedOnionPerfAnalysis); + this.parseFormattedTorperfResults(formattedTorperfResults); + return this.convertedTorperfResults; + } + + /** + * Parse the OnionPerf analysis JSON document. + */ + private ParsedOnionPerfAnalysis parseOnionPerfAnalysis() + throws DescriptorParseException { try { InputStream compressedInputStream = new ByteArrayInputStream( this.rawDescriptorBytes); InputStream decompressedInputStream = new XZCompressorInputStream( compressedInputStream); byte[] decompressedBytes = IOUtils.toByteArray(decompressedInputStream); - parsedOnionPerfAnalysis = ParsedOnionPerfAnalysis.fromBytes( - decompressedBytes); + return ParsedOnionPerfAnalysis.fromBytes(decompressedBytes); } catch (IOException ioException) { throw new DescriptorParseException("Ran into an I/O error while " + "attempting to parse an OnionPerf analysis document.", ioException); } - this.verifyDocumentTypeAndVersion(parsedOnionPerfAnalysis); - StringBuilder formattedTorperfResults - = this.formatTorperfResults(parsedOnionPerfAnalysis); - this.parseFormattedTorperfResults(formattedTorperfResults); - return this.convertedTorperfResults; } /** @@ -109,9 +116,9 @@ public class OnionPerfAnalysisConverter { throw new DescriptorParseException("Parsed OnionPerf analysis file does " + "not contain version information."); } else if ((parsedOnionPerfAnalysis.version instanceof Double - && (double) parsedOnionPerfAnalysis.version > 2.999) + && (double) parsedOnionPerfAnalysis.version > 3.999) || (parsedOnionPerfAnalysis.version instanceof String - && ((String) parsedOnionPerfAnalysis.version).compareTo("3.") >= 0)) { + && ((String) parsedOnionPerfAnalysis.version).compareTo("4.") >= 0)) { throw new DescriptorParseException("Parsed OnionPerf analysis file " + "contains unsupported version " + parsedOnionPerfAnalysis.version + "."); @@ -131,8 +138,7 @@ public class OnionPerfAnalysisConverter { : parsedOnionPerfAnalysis.data.entrySet()) { String nickname = data.getKey(); ParsedOnionPerfAnalysis.MeasurementData measurements = data.getValue(); - if (null == measurements.measurementIp || null == measurements.tgen - || null == measurements.tgen.transfers) { + if (null == measurements.tgen) { continue; } String measurementIp = measurements.measurementIp; @@ -153,57 +159,69 @@ public class OnionPerfAnalysisConverter { } } } - for (ParsedOnionPerfAnalysis.Transfer transfer - : measurements.tgen.transfers.values()) { - if (null == transfer.endpointLocal) { - continue; - } - String[] endpointLocalParts = transfer.endpointLocal.split(":"); - if (endpointLocalParts.length < 3) { - continue; - } - TorperfResultsBuilder torperfResultsBuilder - = new TorperfResultsBuilder(); - - torperfResultsBuilder.addString("SOURCE", nickname); - torperfResultsBuilder.addString("SOURCEADDRESS", measurementIp); - this.formatTransferParts(torperfResultsBuilder, transfer); - List<String> errorCodeParts = null; - if (transfer.isError) { - errorCodeParts = new ArrayList<>(); - if ("PROXY".equals(transfer.errorCode)) { - errorCodeParts.add("TOR"); - } else { - errorCodeParts.add("TGEN"); - errorCodeParts.add(transfer.errorCode); + if (null != measurements.tgen.transfers) { + for (ParsedOnionPerfAnalysis.Transfer transfer + : measurements.tgen.transfers.values()) { + TorperfResultsBuilder torperfResultsBuilder + = new TorperfResultsBuilder(); + torperfResultsBuilder.addString("SOURCE", nickname); + torperfResultsBuilder.addString("SOURCEADDRESS", measurementIp); + this.formatTransferParts(torperfResultsBuilder, transfer); + if (null != transfer.endpointLocal) { + String[] endpointLocalParts = transfer.endpointLocal.split(":"); + if (endpointLocalParts.length >= 3) { + String sourcePort = endpointLocalParts[2]; + if (streamsBySourcePort.containsKey(sourcePort)) { + for (ParsedOnionPerfAnalysis.Stream stream + : streamsBySourcePort.get(sourcePort)) { + if (Math.abs(transfer.unixTsEnd - stream.unixTsEnd) < 150.0) { + this.formatStreamParts(torperfResultsBuilder, stream); + if (null != stream.circuitId + && circuitsByCircuitId.containsKey(stream.circuitId)) { + ParsedOnionPerfAnalysis.Circuit circuit + = circuitsByCircuitId.get(stream.circuitId); + this.formatCircuitParts(torperfResultsBuilder, circuit); + } + } + } + } + } } + formattedTorperfResults.append(torperfResultsBuilder.build()); } - String sourcePort = endpointLocalParts[2]; - if (streamsBySourcePort.containsKey(sourcePort)) { - for (ParsedOnionPerfAnalysis.Stream stream - : streamsBySourcePort.get(sourcePort)) { - if (Math.abs(transfer.unixTsEnd - stream.unixTsEnd) < 150.0) { - if (null != errorCodeParts && null != stream.failureReasonLocal) { - errorCodeParts.add(stream.failureReasonLocal); - if (null != stream.failureReasonRemote) { - errorCodeParts.add(stream.failureReasonRemote); + } + if (null != measurements.tgen.streams) { + for (ParsedOnionPerfAnalysis.TgenStream stream + : measurements.tgen.streams.values()) { + TorperfResultsBuilder torperfResultsBuilder + = new TorperfResultsBuilder(); + torperfResultsBuilder.addString("SOURCE", nickname); + torperfResultsBuilder.addString("SOURCEADDRESS", measurementIp); + this.formatTgenStreamParts(torperfResultsBuilder, stream); + if (null != stream.transportInfo + && null != stream.transportInfo.local) { + String[] endpointLocalParts = stream.transportInfo.local.split(":"); + if (endpointLocalParts.length >= 3) { + String sourcePort = endpointLocalParts[2]; + if (streamsBySourcePort.containsKey(sourcePort)) { + for (ParsedOnionPerfAnalysis.Stream torStream + : streamsBySourcePort.get(sourcePort)) { + if (Math.abs(stream.unixTsEnd + - torStream.unixTsEnd) < 150.0) { + this.formatStreamParts(torperfResultsBuilder, torStream); + if (null != torStream.circuitId && circuitsByCircuitId + .containsKey(torStream.circuitId)) { + ParsedOnionPerfAnalysis.Circuit circuit + = circuitsByCircuitId.get(torStream.circuitId); + this.formatCircuitParts(torperfResultsBuilder, circuit); + } + } } } - if (null != stream.circuitId - && circuitsByCircuitId.containsKey(stream.circuitId)) { - ParsedOnionPerfAnalysis.Circuit circuit - = circuitsByCircuitId.get(stream.circuitId); - this.formatStreamParts(torperfResultsBuilder, stream); - this.formatCircuitParts(torperfResultsBuilder, circuit); - } } } + formattedTorperfResults.append(torperfResultsBuilder.build()); } - if (null != errorCodeParts) { - String errorCode = String.join("/", errorCodeParts); - torperfResultsBuilder.addString("ERRORCODE", errorCode); - } - formattedTorperfResults.append(torperfResultsBuilder.build()); } } return formattedTorperfResults; @@ -288,6 +306,105 @@ public class OnionPerfAnalysisConverter { transfer.elapsedSeconds.lastByte); if (transfer.isError) { torperfResultsBuilder.addInteger("DIDTIMEOUT", 1); + if ("PROXY".equals(transfer.errorCode)) { + torperfResultsBuilder.addErrorCodePart("TOR"); + } else { + torperfResultsBuilder.addErrorCodePart("TGEN"); + torperfResultsBuilder.addErrorCodePart(transfer.errorCode); + } + } + } + } + + /** + * Format relevant tgen stream data as Torperf result key-value pairs. + * + * @param torperfResultsBuilder Torperf results builder to add key-value pairs + * to. + * @param stream Stream data obtained from the parsed OnionPerf analysis file. + */ + private void formatTgenStreamParts( + TorperfResultsBuilder torperfResultsBuilder, + ParsedOnionPerfAnalysis.TgenStream stream) { + torperfResultsBuilder.addString("READBYTES", "0"); + torperfResultsBuilder.addString("WRITEBYTES", "0"); + if (null != stream.byteInfo) { + torperfResultsBuilder.addString("READBYTES", + stream.byteInfo.totalBytesRecv); + torperfResultsBuilder.addString("WRITEBYTES", + stream.byteInfo.totalBytesSend); + } + if (null != stream.streamInfo) { + torperfResultsBuilder.addString("FILESIZE", stream.streamInfo.recvsize); + torperfResultsBuilder.addString("HOSTNAMELOCAL", stream.streamInfo.name); + torperfResultsBuilder.addString("HOSTNAMEREMOTE", + stream.streamInfo.peername); + } + if (null != stream.transportInfo) { + torperfResultsBuilder.addString("ENDPOINTLOCAL", + stream.transportInfo.local); + torperfResultsBuilder.addString("ENDPOINTPROXY", + stream.transportInfo.proxy); + torperfResultsBuilder.addString("ENDPOINTREMOTE", + stream.transportInfo.remote); + } + torperfResultsBuilder.addInteger("DIDTIMEOUT", 0); + + for (String key : new String[] { "START", "SOCKET", "CONNECT", "NEGOTIATE", + "REQUEST", "RESPONSE", "DATAREQUEST", "DATARESPONSE", + "DATACOMPLETE" }) { + torperfResultsBuilder.addString(key, "0.0"); + } + torperfResultsBuilder.addTimestamp("START", stream.unixTsStart, 0.0); + if (null != stream.unixTsStart) { + if (null != stream.timeInfo) { + torperfResultsBuilder.addTimestamp("SOCKET", stream.unixTsStart, + stream.timeInfo.usecsToSocketCreate); + torperfResultsBuilder.addTimestamp("CONNECT", stream.unixTsStart, + stream.timeInfo.usecsToSocketConnect); + torperfResultsBuilder.addTimestamp("NEGOTIATE", stream.unixTsStart, + stream.timeInfo.usecsToProxyChoice); + torperfResultsBuilder.addTimestamp("REQUEST", stream.unixTsStart, + stream.timeInfo.usecsToProxyRequest); + torperfResultsBuilder.addTimestamp("RESPONSE", stream.unixTsStart, + stream.timeInfo.usecsToProxyResponse); + torperfResultsBuilder.addTimestamp("DATAREQUEST", stream.unixTsStart, + stream.timeInfo.usecsToCommand); + torperfResultsBuilder.addTimestamp("DATARESPONSE", stream.unixTsStart, + stream.timeInfo.usecsToResponse); + torperfResultsBuilder.addTimestamp("DATACOMPLETE", stream.unixTsStart, + stream.timeInfo.usecsToLastByteRecv); + } + if (null != stream.elapsedSeconds) { + if (null != stream.elapsedSeconds.payloadBytesRecv) { + for (Map.Entry<String, Double> payloadBytesRecvEntry + : stream.elapsedSeconds.payloadBytesRecv.entrySet()) { + String key = String.format("PARTIAL%s", + payloadBytesRecvEntry.getKey()); + Double elapsedSeconds = payloadBytesRecvEntry.getValue(); + torperfResultsBuilder.addTimestamp(key, stream.unixTsStart, + elapsedSeconds); + } + } + if (null != stream.elapsedSeconds.payloadProgressRecv) { + for (Map.Entry<String, Double> payloadProgressRecvEntry + : stream.elapsedSeconds.payloadProgressRecv.entrySet()) { + String key = String.format("DATAPERC%.0f", + Double.parseDouble(payloadProgressRecvEntry.getKey()) * 100.0); + Double elapsedSeconds = payloadProgressRecvEntry.getValue(); + torperfResultsBuilder.addTimestamp(key, stream.unixTsStart, + elapsedSeconds); + } + } + } + if (null != stream.isError && stream.isError) { + torperfResultsBuilder.addInteger("DIDTIMEOUT", 1); + if ("PROXY".equals(stream.streamInfo.error)) { + torperfResultsBuilder.addErrorCodePart("TOR"); + } else { + torperfResultsBuilder.addErrorCodePart("TGEN"); + torperfResultsBuilder.addErrorCodePart(stream.streamInfo.error); + } } } } @@ -301,6 +418,12 @@ public class OnionPerfAnalysisConverter { */ private void formatStreamParts(TorperfResultsBuilder torperfResultsBuilder, ParsedOnionPerfAnalysis.Stream stream) { + if (null != stream.failureReasonLocal) { + torperfResultsBuilder.addErrorCodePart(stream.failureReasonLocal); + if (null != stream.failureReasonRemote) { + torperfResultsBuilder.addErrorCodePart(stream.failureReasonRemote); + } + } torperfResultsBuilder.addTimestamp("USED_AT", stream.unixTsEnd, 0.0); torperfResultsBuilder.addInteger("USED_BY", stream.streamId); } diff --git a/src/main/java/org/torproject/descriptor/onionperf/ParsedOnionPerfAnalysis.java b/src/main/java/org/torproject/descriptor/onionperf/ParsedOnionPerfAnalysis.java index 4eca0ff..e94fdb4 100644 --- a/src/main/java/org/torproject/descriptor/onionperf/ParsedOnionPerfAnalysis.java +++ b/src/main/java/org/torproject/descriptor/onionperf/ParsedOnionPerfAnalysis.java @@ -4,6 +4,7 @@ package org.torproject.descriptor.onionperf; import com.fasterxml.jackson.annotation.JsonAutoDetect; +import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.annotation.PropertyAccessor; import com.fasterxml.jackson.databind.DeserializationFeature; import com.fasterxml.jackson.databind.ObjectMapper; @@ -95,6 +96,11 @@ public class ParsedOnionPerfAnalysis { * Measurement data by transfer identifier. */ Map<String, Transfer> transfers; + + /** + * Measurement data by stream identifier. + */ + Map<String, TgenStream> streams; } /** @@ -240,6 +246,206 @@ public class ParsedOnionPerfAnalysis { Double socketCreate; } + /** + * Measurement data related to a single stream obtained from client-side + * {@code tgen} logs. + */ + static class TgenStream { + + /** + * Information on sent and received bytes. + */ + ByteInfo byteInfo; + + /** + * Elapsed seconds until a given number or fraction of payload bytes have + * been received or sent, obtained from {@code [stream-status]}, + * {@code [stream-success]}, and {@code [stream-error]} log messages, only + * included if the measurement was a success. + */ + ElapsedSecondsPayload elapsedSeconds; + + /** + * Whether an error occurred. + */ + Boolean isError; + + /** + * Information about the TGen stream. + */ + StreamInfo streamInfo; + + /** + * Elapsed time until reaching given substeps in a measurement. + */ + TimeInfo timeInfo; + + /** + * Information about the TGen transport. + */ + TransportInfo transportInfo; + + /** + * Initial start time of the measurement, obtained by subtracting the + * largest number of elapsed microseconds in {@code time_info} from + * {@code unix_ts_end}, given in seconds since the epoch. + */ + Double unixTsStart; + + /** + * Final end time of the measurement, obtained from the log time of the + * {@code [stream-success]} or {@code [stream-error]} log message, given in + * seconds since the epoch. + */ + Double unixTsEnd; + } + + /** + * Information on sent and received bytes. + */ + static class ByteInfo { + + /** + * Total number of bytes received. + */ + @JsonProperty("total-bytes-recv") + String totalBytesRecv; + + /** + * Total number of bytes sent. + */ + @JsonProperty("total-bytes-send") + String totalBytesSend; + } + + /** + * Elapsed seconds until a given number or fraction of payload bytes have been + * received or sent, obtained from {@code [stream-status]}, + * {@code [stream-success]}, and {@code [stream-error]} log messages, only + * included if the measurement was a success. + */ + static class ElapsedSecondsPayload { + + /** + * Number of received payload bytes. + */ + Map<String, Double> payloadBytesRecv; + + /** + * Fraction of received payload bytes. + */ + Map<String, Double> payloadProgressRecv; + } + + /** + * Information about the TGen stream. + */ + static class StreamInfo { + + /** + * Error code, or {@code NONE} if no error occurred. + */ + String error; + + /** + * Hostname of the TGen client. + */ + String name; + + /** + * Hostname of the TGen server. + */ + String peername; + + /** + * Number of expected payload bytes in the response. + */ + String recvsize; + } + + /** + * Elapsed time until reaching given substeps in a measurement. + */ + static class TimeInfo { + + /** + * Elapsed microseconds until the TGen client has sent the command to the + * TGen server, or -1 if missing (step 7). + */ + @JsonProperty("usecs-to-command") + String usecsToCommand; + + /** + * Elapsed microseconds until the TGen client has received the last payload + * byte, or -1 if missing (step 10). + */ + @JsonProperty("usecs-to-last-byte-recv") + String usecsToLastByteRecv; + + /** + * Elapsed microseconds until the TGen client has received the SOCKS choice + * from the Tor client, or -1 if missing (step 4). + */ + @JsonProperty("usecs-to-proxy-choice") + String usecsToProxyChoice; + + /** + * Elapsed microseconds until the TGen client has sent the SOCKS request to + * the Tor client, or -1 if missing (step 5). + */ + @JsonProperty("usecs-to-proxy-request") + String usecsToProxyRequest; + + /** + * Elapsed microseconds until the TGen client has received the SOCKS + * response from the Tor client, or -1 if missing (step 6). + */ + @JsonProperty("usecs-to-proxy-response") + String usecsToProxyResponse; + + /** + * Elapsed microseconds until the TGen client has received the command from + * the TGen server, or -1 if missing (step 8). + */ + @JsonProperty("usecs-to-response") + String usecsToResponse; + + /** + * Elapsed microseconds until the TGen client has connected to the Tor + * client's SOCKS port, or -1 if missing (step 2). + */ + @JsonProperty("usecs-to-socket-connect") + String usecsToSocketConnect; + + /** + * Elapsed microseconds until the TGen client has opened a TCP connection + * to the Tor client's SOCKS port, or -1 if missing (step 1). + */ + @JsonProperty("usecs-to-socket-create") + String usecsToSocketCreate; + } + + /** + * Information about the TGen transport. + */ + static class TransportInfo { + + /** + * Local host name, IP address, and TCP port. + */ + String local; + + /** + * Proxy host name, IP address, and TCP port. + */ + String proxy; + + /** + * Remote host name, IP address, and TCP port. + */ + String remote; + } + /** * Measurement data obtained from client-side {@code tor} controller event * logs. diff --git a/src/main/java/org/torproject/descriptor/onionperf/TorperfResultsBuilder.java b/src/main/java/org/torproject/descriptor/onionperf/TorperfResultsBuilder.java index a99257a..766f7ad 100644 --- a/src/main/java/org/torproject/descriptor/onionperf/TorperfResultsBuilder.java +++ b/src/main/java/org/torproject/descriptor/onionperf/TorperfResultsBuilder.java @@ -15,6 +15,19 @@ import java.util.TreeMap; */ public class TorperfResultsBuilder { + /** + * Error code parts, to be formatted as + * {@code ERRORCODE=part_1/part_2/.../part_n}. + */ + private List<String> errorCodeParts = null; + + void addErrorCodePart(String errorCodePart) { + if (null == errorCodeParts) { + this.errorCodeParts = new ArrayList<>(); + } + this.errorCodeParts.add(errorCodePart); + } + /** * Key-value pairs to be formatted as Torperf results line. */ @@ -77,6 +90,27 @@ public class TorperfResultsBuilder { } } + /** + * Add a timestamp value as the sum of a double value, formatted as seconds + * since the epoch with two decimal places, and a string value, formatted as + * microseconds since the first value, unless either of the two summands is + * {@code null} or negative. + * + * @param key Key. + * @param unixTsStart First summand representing seconds since the epoch. + * @param elapsedMicroseconds Second summand representing microseconds + * elapsed since the first summand. + */ + void addTimestamp(String key, Double unixTsStart, + String elapsedMicroseconds) { + if (null != unixTsStart && unixTsStart >= 0.0 && null != elapsedMicroseconds + && !elapsedMicroseconds.startsWith("-")) { + double elapsedSeconds = Double.parseDouble(elapsedMicroseconds) + / 1000000.0; + this.addTimestamp(key, unixTsStart, elapsedSeconds); + } + } + /** * Build the Torperf results line by putting together all key-value pairs as * {@code "key=value"}, separated by spaces, prefixed by an annotation line @@ -89,6 +123,10 @@ public class TorperfResultsBuilder { StringBuilder result = new StringBuilder(); result.append("@type torperf 1.1\r\n"); List<String> torperfResultsParts = new ArrayList<>(); + if (null != this.errorCodeParts) { + String errorCode = String.join("/", errorCodeParts); + this.addString("ERRORCODE", errorCode); + } for (Map.Entry<String, String> keyValuePairsEntry : this.keyValuePairs.entrySet()) { torperfResultsParts.add(String.format("%s=%s", diff --git a/src/test/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverterTest.java b/src/test/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverterTest.java index 51e0896..f8d9f90 100644 --- a/src/test/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverterTest.java +++ b/src/test/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverterTest.java @@ -3,6 +3,7 @@ package org.torproject.descriptor.onionperf; +import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertTrue; @@ -16,6 +17,8 @@ import org.junit.Test; import java.io.IOException; import java.io.InputStream; import java.net.URL; +import java.util.ArrayList; +import java.util.List; public class OnionPerfAnalysisConverterTest { @@ -89,6 +92,10 @@ public class OnionPerfAnalysisConverterTest { @Test public void testAsTorperfResults() throws IOException, DescriptorParseException { + List<String> expectedTorperfResults = new ArrayList<>(); + expectedTorperfResults.add(torperfResultTransfer1m1); + expectedTorperfResults.add(torperfResultTransfer1m3); + expectedTorperfResults.add(torperfResultTransfer50k2); URL resouce = getClass().getClassLoader().getResource( "onionperf/onionperf.analysis.json.xz"); assertNotNull(resouce); @@ -103,12 +110,75 @@ public class OnionPerfAnalysisConverterTest { String formattedTorperfResult = new String(descriptor.getRawDescriptorBytes()).trim(); assertNotNull(formattedTorperfResult); - assertTrue(String.format("Unrecognized formatted Torperf result: %s", - formattedTorperfResult), - formattedTorperfResult.equals(torperfResultTransfer1m1) - || formattedTorperfResult.equals(torperfResultTransfer1m3) - || formattedTorperfResult.equals(torperfResultTransfer50k2)); + String expectedTorperfResult = expectedTorperfResults.remove(0); + assertEquals(expectedTorperfResult, formattedTorperfResult); } + assertTrue(expectedTorperfResults.isEmpty()); + } + + private final String torperfResultStream155 + = "BUILDTIMES=0.62,0.90,1.26 CIRC_ID=1008 CONNECT=1594633118.41 " + + "DATACOMPLETE=1594633145.46 DATAPERC0=1594633123.64 " + + "DATAPERC10=1594633125.57 DATAPERC100=1594633145.46 " + + "DATAPERC20=1594633127.56 DATAPERC30=1594633129.56 " + + "DATAPERC40=1594633132.20 DATAPERC50=1594633134.29 " + + "DATAPERC60=1594633136.41 DATAPERC70=1594633138.84 " + + "DATAPERC80=1594633140.76 DATAPERC90=1594633142.88 " + + "DATAREQUEST=1594633123.01 DATARESPONSE=1594633123.64 DIDTIMEOUT=0 " + + "ENDPOINTLOCAL=localhost:127.0.0.1:46222 " + + "ENDPOINTPROXY=localhost:127.0.0.1:29849 " + + "ENDPOINTREMOTE=jjsvldkkjd3lxy6gljy6xmhrn5mnirzgj2mrftrx2bf3n4bbx53fxza" + + "d.onion:0.0.0.0:8080 FILESIZE=5242880 HOSTNAMELOCAL=op-nl-test1 " + + "HOSTNAMEREMOTE=op-nl-test1 LAUNCH=1594633119.06 " + + "NEGOTIATE=1594633118.41 PARTIAL10240=1594633123.80 " + + "PARTIAL102400=1594633124.11 PARTIAL1048576=1594633127.56 " + + "PARTIAL20480=1594633123.85 PARTIAL204800=1594633124.26 " + + "PARTIAL2097152=1594633132.20 PARTIAL51200=1594633123.96 " + + "PARTIAL512000=1594633125.50 PARTIAL5242880=1594633145.46 " + + "PATH=$65888719E2F619E6198F1045A93AF0176C05354D," + + "$3043C1A6DF23AFEDC8FD3C0671FADCEDFF6D3429," + + "$8E5F4EE45E0631A60E59CAA42E1464FD7120459D QUANTILE=0.8 " + + "READBYTES=5242990 REQUEST=1594633118.42 RESPONSE=1594633123.01 " + + "SOCKET=1594633118.41 SOURCE=op-nl-test1 SOURCEADDRESS=unknown " + + "START=1594633118.41 TIMEOUT=1500 USED_AT=1594633145.46 USED_BY=1498 " + + "WRITEBYTES=2174"; + + private final String torperfResultStream156 + = "CONNECT=1594633539.83 DATACOMPLETE=0.0 DATAREQUEST=0.0 " + + "DATARESPONSE=0.0 DIDTIMEOUT=1 ENDPOINTLOCAL=localhost:127.0.0.1:46246 " + + "ENDPOINTPROXY=localhost:127.0.0.1:29849 " + + "ENDPOINTREMOTE=jjsvldkkjd3lxy6gljy6xmhrn5mnirzgj2mrftrx2bf3n4bbx53fxza" + + "d.onion:0.0.0.0:8080 ERRORCODE=TOR/TIMEOUT FILESIZE=5242880 " + + "HOSTNAMELOCAL=op-nl-test1 HOSTNAMEREMOTE=(null) " + + "NEGOTIATE=1594633539.83 READBYTES=0 REQUEST=1594633539.83 " + + "RESPONSE=0.0 SOCKET=1594633539.83 SOURCE=op-nl-test1 " + + "SOURCEADDRESS=unknown START=1594633539.83 USED_AT=1594633539.83 " + + "USED_BY=1510 WRITEBYTES=0"; + + @Test + public void testAsTorperfResultsVersion3() throws IOException, + DescriptorParseException { + List<String> expectedTorperfResults = new ArrayList<>(); + expectedTorperfResults.add(torperfResultStream155); + expectedTorperfResults.add(torperfResultStream156); + URL resouce = getClass().getClassLoader().getResource( + "onionperf/2020-07-13.op-nl-test1.onionperf.analysis.json.xz"); + assertNotNull(resouce); + InputStream compressedInputStream = resouce.openStream(); + assertNotNull(compressedInputStream); + byte[] rawDescriptorBytes = IOUtils.toByteArray(compressedInputStream); + OnionPerfAnalysisConverter onionPerfAnalysisConverter + = new OnionPerfAnalysisConverter(rawDescriptorBytes, null); + for (Descriptor descriptor + : onionPerfAnalysisConverter.asTorperfResults()) { + assertTrue(descriptor instanceof TorperfResult); + String formattedTorperfResult + = new String(descriptor.getRawDescriptorBytes()).trim(); + assertNotNull(formattedTorperfResult); + String expectedTorperfResult = expectedTorperfResults.remove(0); + assertEquals(expectedTorperfResult, formattedTorperfResult); + } + assertTrue(expectedTorperfResults.isEmpty()); } } diff --git a/src/test/resources/onionperf/2020-07-13.op-nl-test1.onionperf.analysis.json.xz b/src/test/resources/onionperf/2020-07-13.op-nl-test1.onionperf.analysis.json.xz new file mode 100644 index 0000000..78be8c5 Binary files /dev/null and b/src/test/resources/onionperf/2020-07-13.op-nl-test1.onionperf.analysis.json.xz differ [View Less]

1 0

[metrics-lib/master] Parse new ipv6-* lines in extra-info descriptors.
by karsten＠torproject.org 07 Aug '20

07 Aug '20

commit a7850a2dc0e8be8a7d9bc289c96dbeed6f204b00 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Aug 7 20:48:57 2020 +0200 Parse new ipv6-* lines in extra-info descriptors. --- CHANGELOG.md | 2 + .../torproject/descriptor/ExtraInfoDescriptor.java | 72 ++++++++++++++++ .../descriptor/impl/ExtraInfoDescriptorImpl.java | 95 ++++++++++++++++++++++ .../java/org/torproject/descriptor/impl/Key.java | 3 + .../impl/… [View More]ExtraInfoDescriptorImplTest.java | 74 +++++++++++++++++ 5 files changed, 246 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4679688..bb55312 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,8 @@ * Medium changes - Extend Torperf results to provide error codes. - Parse OnionPerf analysis results format version 3.0. + - Parse new ipv6-{write,read}-history and ipv6-conn-bi-direct + lines in extra-info descriptors. # Changes in version 2.13.0 - 2020-05-16 diff --git a/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java b/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java index a2c893b..e094fed 100644 --- a/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java +++ b/src/main/java/org/torproject/descriptor/ExtraInfoDescriptor.java @@ -106,6 +106,22 @@ public interface ExtraInfoDescriptor extends Descriptor { */ BandwidthHistory getWriteHistory(); + /** + * Return the server's history of written IPv6 bytes, or {@code null} if the + * descriptor does not contain a bandwidth history. + * + * @since 2.14.0 + */ + BandwidthHistory getIpv6WriteHistory(); + + /** + * Return the server's history of read IPv6 bytes, or {@code null} if the + * descriptor does not contain a bandwidth history. + * + * @since 2.14.0 + */ + BandwidthHistory getIpv6ReadHistory(); + /** * Return a SHA-1 digest of the GeoIP database file used by this server * to resolve client IP addresses to country codes, encoded as 40 @@ -419,6 +435,62 @@ public interface ExtraInfoDescriptor extends Descriptor { */ int getConnBiDirectBoth(); + /** + * Return the time in milliseconds since the epoch when the included + * statistics on bi-directional IPv6 connection usage ended, or -1 if no such + * statistics are included. + * + * @since 2.14.0 + */ + long getIpv6ConnBiDirectStatsEndMillis(); + + /** + * Return the interval length of the included statistics on + * bi-directional IPv6 connection usage in seconds, or -1 if no such + * statistics are included. + * + * @since 2.14.0 + */ + long getIpv6ConnBiDirectStatsIntervalLength(); + + /** + * Return the number of IPv6 connections on which this server read and wrote + * less than 2 KiB/s in a 10-second interval, or -1 if no such + * statistics are included. + * + * @since 2.14.0 + */ + int getIpv6ConnBiDirectBelow(); + + /** + * Return the number of IPv6 connections on which this server read and wrote + * at least 2 KiB/s in a 10-second interval and at least 10 times more + * in read direction than in write direction, or -1 if no such + * statistics are included. + * + * @since 2.14.0 + */ + int getIpv6ConnBiDirectRead(); + + /** + * Return the number of IPv6 connections on which this server read and wrote + * at least 2 KiB/s in a 10-second interval and at least 10 times more + * in write direction than in read direction, or -1 if no such + * statistics are included. + * + * @since 2.14.0 + */ + int getIpv6ConnBiDirectWrite(); + + /** + * Return the number of IPv6 connections on which this server read and wrote + * at least 2 KiB/s in a 10-second interval but not 10 times more in + * either direction, or -1 if no such statistics are included. + * + * @since 2.14.0 + */ + int getIpv6ConnBiDirectBoth(); + /** * Return the time in milliseconds since the epoch when the included * exit statistics interval ended, or -1 if no such statistics are diff --git a/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java index f02b540..5cab6ab 100644 --- a/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java +++ b/src/main/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImpl.java @@ -97,6 +97,12 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl case WRITE_HISTORY: this.parseWriteHistoryLine(line, partsNoOpt); break; + case IPV6_READ_HISTORY: + this.parseIpv6ReadHistoryLine(line, partsNoOpt); + break; + case IPV6_WRITE_HISTORY: + this.parseIpv6WriteHistoryLine(line, partsNoOpt); + break; case GEOIP_DB_DIGEST: this.parseGeoipDbDigestLine(line, partsNoOpt); break; @@ -179,6 +185,9 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl case CONN_BI_DIRECT: this.parseConnBiDirectLine(line, partsNoOpt); break; + case IPV6_CONN_BI_DIRECT: + this.parseIpv6ConnBiDirectLine(line, partsNoOpt); + break; case EXIT_STATS_END: this.parseExitStatsEndLine(line, partsNoOpt); break; @@ -314,6 +323,18 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl partsNoOpt); } + private void parseIpv6ReadHistoryLine(String line, + String[] partsNoOpt) throws DescriptorParseException { + this.ipv6ReadHistory = new BandwidthHistoryImpl(line, + partsNoOpt); + } + + private void parseIpv6WriteHistoryLine(String line, + String[] partsNoOpt) throws DescriptorParseException { + this.ipv6WriteHistory = new BandwidthHistoryImpl(line, + partsNoOpt); + } + private void parseGeoipDbDigestLine(String line, String[] partsNoOpt) throws DescriptorParseException { if (partsNoOpt.length < 2) { @@ -573,6 +594,24 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl this.connBiDirectBoth = parsedConnBiDirectStats[3]; } + private void parseIpv6ConnBiDirectLine(String line, + String[] partsNoOpt) throws DescriptorParseException { + long[] parsedStatsEndData = this.parseStatsEndLine(line, partsNoOpt, + 6); + this.ipv6ConnBiDirectStatsEndMillis = parsedStatsEndData[0]; + this.ipv6ConnBiDirectStatsIntervalLength = parsedStatsEndData[1]; + Integer[] parsedIpv6ConnBiDirectStats = ParseHelper + .parseCommaSeparatedIntegerValueList(line, partsNoOpt, 5); + if (parsedIpv6ConnBiDirectStats.length != 4) { + throw new DescriptorParseException("Illegal line '" + line + "' in " + + "extra-info descriptor."); + } + this.ipv6ConnBiDirectBelow = parsedIpv6ConnBiDirectStats[0]; + this.ipv6ConnBiDirectRead = parsedIpv6ConnBiDirectStats[1]; + this.ipv6ConnBiDirectWrite = parsedIpv6ConnBiDirectStats[2]; + this.ipv6ConnBiDirectBoth = parsedIpv6ConnBiDirectStats[3]; + } + private void parseExitStatsEndLine(String line, String[] partsNoOpt) throws DescriptorParseException { long[] parsedStatsEndData = this.parseStatsEndLine(line, partsNoOpt, @@ -837,6 +876,20 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl return this.writeHistory; } + private BandwidthHistory ipv6ReadHistory; + + @Override + public BandwidthHistory getIpv6ReadHistory() { + return this.ipv6ReadHistory; + } + + private BandwidthHistory ipv6WriteHistory; + + @Override + public BandwidthHistory getIpv6WriteHistory() { + return this.ipv6WriteHistory; + } + private String geoipDbDigest; @Override @@ -1082,6 +1135,48 @@ public abstract class ExtraInfoDescriptorImpl extends DescriptorImpl return this.connBiDirectBoth; } + private long ipv6ConnBiDirectStatsEndMillis = -1L; + + @Override + public long getIpv6ConnBiDirectStatsEndMillis() { + return this.ipv6ConnBiDirectStatsEndMillis; + } + + private long ipv6ConnBiDirectStatsIntervalLength = -1L; + + @Override + public long getIpv6ConnBiDirectStatsIntervalLength() { + return this.ipv6ConnBiDirectStatsIntervalLength; + } + + private int ipv6ConnBiDirectBelow = -1; + + @Override + public int getIpv6ConnBiDirectBelow() { + return this.ipv6ConnBiDirectBelow; + } + + private int ipv6ConnBiDirectRead = -1; + + @Override + public int getIpv6ConnBiDirectRead() { + return this.ipv6ConnBiDirectRead; + } + + private int ipv6ConnBiDirectWrite = -1; + + @Override + public int getIpv6ConnBiDirectWrite() { + return this.ipv6ConnBiDirectWrite; + } + + private int ipv6ConnBiDirectBoth = -1; + + @Override + public int getIpv6ConnBiDirectBoth() { + return this.ipv6ConnBiDirectBoth; + } + private long exitStatsEndMillis = -1L; @Override diff --git a/src/main/java/org/torproject/descriptor/impl/Key.java b/src/main/java/org/torproject/descriptor/impl/Key.java index a6202e4..70db350 100644 --- a/src/main/java/org/torproject/descriptor/impl/Key.java +++ b/src/main/java/org/torproject/descriptor/impl/Key.java @@ -94,7 +94,10 @@ public enum Key { HIDSERV_STATS_END("hidserv-stats-end"), ID("id"), IDENTITY_ED25519("identity-ed25519"), + IPV6_CONN_BI_DIRECT("ipv6-conn-bi-direct"), IPV6_POLICY("ipv6-policy"), + IPV6_READ_HISTORY("ipv6-read-history"), + IPV6_WRITE_HISTORY("ipv6-write-history"), KNOWN_FLAGS("known-flags"), LEGACY_DIR_KEY("legacy-dir-key"), LEGACY_KEY("legacy-key"), diff --git a/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java index 9750046..1233bd5 100644 --- a/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java +++ b/src/test/java/org/torproject/descriptor/impl/ExtraInfoDescriptorImplTest.java @@ -75,6 +75,24 @@ public class ExtraInfoDescriptorImplTest { return db.buildDescriptor(); } + private String ipv6WriteHistoryLine = null; + + private static ExtraInfoDescriptor createWithIpv6WriteHistoryLine( + String line) throws DescriptorParseException { + DescriptorBuilder db = new DescriptorBuilder(); + db.ipv6WriteHistoryLine = line; + return db.buildDescriptor(); + } + + private String ipv6ReadHistoryLine = null; + + private static ExtraInfoDescriptor createWithIpv6ReadHistoryLine( + String line) throws DescriptorParseException { + DescriptorBuilder db = new DescriptorBuilder(); + db.ipv6ReadHistoryLine = line; + return db.buildDescriptor(); + } + private String dirreqWriteHistoryLine = "dirreq-write-history " + "2012-02-11 09:03:39 (900 s) 81281024,64996352,60625920," + "67922944"; @@ -160,6 +178,15 @@ public class ExtraInfoDescriptorImplTest { return db.buildDescriptor(); } + private String ipv6ConnBiDirectLine = null; + + private static ExtraInfoDescriptor createWithIpv6ConnBiDirectLine( + String line) throws DescriptorParseException { + DescriptorBuilder db = new DescriptorBuilder(); + db.ipv6ConnBiDirectLine = line; + return db.buildDescriptor(); + } + private String exitStatsLines = null; private static ExtraInfoDescriptor createWithExitStatsLines( @@ -266,6 +293,12 @@ public class ExtraInfoDescriptorImplTest { if (this.readHistoryLine != null) { sb.append(this.readHistoryLine).append("\n"); } + if (this.ipv6WriteHistoryLine != null) { + sb.append(this.ipv6WriteHistoryLine).append("\n"); + } + if (this.ipv6ReadHistoryLine != null) { + sb.append(this.ipv6ReadHistoryLine).append("\n"); + } if (this.dirreqWriteHistoryLine != null) { sb.append(this.dirreqWriteHistoryLine).append("\n"); } @@ -293,6 +326,9 @@ public class ExtraInfoDescriptorImplTest { if (this.connBiDirectLine != null) { sb.append(this.connBiDirectLine).append("\n"); } + if (this.ipv6ConnBiDirectLine != null) { + sb.append(this.ipv6ConnBiDirectLine).append("\n"); + } if (this.exitStatsLines != null) { sb.append(this.exitStatsLines).append("\n"); } @@ -1137,6 +1173,29 @@ public class ExtraInfoDescriptorImplTest { + "4707695616,4699666432,4650004480,4489718784"); } + @Test + public void testIpv6WriteHistory() + throws DescriptorParseException { + ExtraInfoDescriptor descriptor = DescriptorBuilder + .createWithIpv6WriteHistoryLine("ipv6-write-history 2020-07-15 " + + "03:54:50 (86400 s) 1290676224,2538212352,6013477888"); + assertNotNull(descriptor.getIpv6WriteHistory()); + assertNull(descriptor.getIpv6ReadHistory()); + assertEquals(86400L, descriptor.getIpv6WriteHistory().getIntervalLength()); + } + + @Test + public void testIpv6ReadHistory() + throws DescriptorParseException { + ExtraInfoDescriptor descriptor = DescriptorBuilder + .createWithIpv6ReadHistoryLine("ipv6-read-history 2020-07-15 03:54:50 " + + "(86400 s) 55123875840,62352131072,69582769152"); + assertNull(descriptor.getIpv6WriteHistory()); + assertNotNull(descriptor.getIpv6ReadHistory()); + assertEquals(3, + descriptor.getIpv6ReadHistory().getBandwidthValues().size()); + } + @Test public void testDirreqWriteHistoryMissingBytesBegin() throws DescriptorParseException { @@ -1724,6 +1783,21 @@ public class ExtraInfoDescriptorImplTest { + "2012-02-11 01:59:39 (86400 s) 42173,1591,1310,1744 +1"); } + @Test + public void testIpv6ConnBiDirectValid() + throws DescriptorParseException { + ExtraInfoDescriptor descriptor = DescriptorBuilder + .createWithIpv6ConnBiDirectLine("ipv6-conn-bi-direct 2020-02-11 " + + "01:59:39 (86400 s) 42173,1591,1310,1744"); + assertEquals(1581386379000L, + descriptor.getIpv6ConnBiDirectStatsEndMillis()); + assertEquals(86400L, descriptor.getIpv6ConnBiDirectStatsIntervalLength()); + assertEquals(42173, descriptor.getIpv6ConnBiDirectBelow()); + assertEquals(1591, descriptor.getIpv6ConnBiDirectRead()); + assertEquals(1310, descriptor.getIpv6ConnBiDirectWrite()); + assertEquals(1744, descriptor.getIpv6ConnBiDirectBoth()); + } + @Test public void testExitStatsValid() throws DescriptorParseException { ExtraInfoDescriptor descriptor = ExitStatsBuilder [View Less]

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 07 Aug '20

07 Aug '20

commit 2e2a2a3403fb7420ecdfd75c41c1ba8aea27d1d0 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Aug 7 18:47:53 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+ar.po | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/contents+ar.po b/contents+ar.po index e0e22ca7d3..63ad0ddcbc 100644 --- a/contents+ar.po +++ b/contents+ar.po @@ -2468,7 +2468,7 @@ msgstr "" #: https//… [View More]support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "### Don't open documents downloaded through Tor while online" -msgstr "" +msgstr "### لا تفتح المستندات التي تم تنزيلها عبر Tor أثناء الاتصال بالإنترنت" #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2476,11 +2476,13 @@ msgid "" "Tor Browser will warn you before automatically opening documents that are " "handled by external applications." msgstr "" +"سيحذرك متصفح Tor قبل فتح المستندات التي تتعامل معها التطبيقات الخارجية " +"تلقائيًا." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "**DO NOT IGNORE THIS WARNING**." -msgstr "" +msgstr "** لا تتجاهل هذا التحذير **." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2490,11 +2492,14 @@ msgid "" "Browser) as these documents can contain Internet resources that will be " "downloaded outside of Tor by the application that opens them." msgstr "" +"يجب أن تكون حذرًا جدًا عند تنزيل المستندات عبر Tor (خاصة ملفات DOC و PDF ، " +"ما لم تستخدم عارض PDF المدمج في متصفح Tor) حيث يمكن أن تحتوي هذه المستندات " +"على موارد الإنترنت التي سيتم تنزيلها خارج Tor بواسطة التطبيق الذي يفتحها." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "This will reveal your non-Tor IP address." -msgstr "" +msgstr "سيكشف هذا عن عنوان IP الخاص بك بخلاف Tor." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2504,6 +2509,10 @@ msgid "" "[dangerzone](https://github.com/firstlookmedia/dangerzone) to create safe " "PDF files that you can open." msgstr "" +"إذا كان لا بد من العمل مع الملفات التي تم تنزيلها عبر Tor ، فنحن نوصي بشدة " +"إما باستخدام جهاز كمبيوتر غير متصل ، أو استخدام [dangerzone] " +"(https://github.com/firstlookmedia/dangerzone) لإنشاء ملفات PDF آمنة يمكنك " +"فتحها." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) @@ -2512,18 +2521,21 @@ msgid "" "Tor](https://blog.torproject.org/blog/bittorrent-over-tor-isnt-good-idea) " "together, however." msgstr "" +"ومع ذلك ، لا يعتبر استخدام [BitTorrent و Tor] " +"(https://blog.torproject.org/blog/bittorrent-over-tor-isnt-good-idea) معًا " +"آمنًا تحت أي ظرف من الظروف." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "### Use bridges and/or find company" -msgstr "" +msgstr "### استخدم الجسور و / أو ابحث عن شركة" #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) msgid "" "Tor tries to prevent attackers from learning what destination websites you " "connect to." -msgstr "" +msgstr "يحاول Tor منع المهاجمين من معرفة مواقع الويب الوجهة التي تتصل بها." #: https//support.torproject.org/faq/staying-anonymous/ #: (content/faq/staying-anonymous/contents+en.lrquestion.description) [View Less]

1 0

[tor-browser-build/master] Merge remote-tracking branch 'gitlab/merge-requests/11' into master
by gk＠torproject.org 07 Aug '20

07 Aug '20

commit de185dd7b4e5188905ed972496e69dd537bd0534 Merge: eeb2153 251a6f7 Author: Georg Koppen <gk(a)torproject.org> Date: Fri Aug 7 18:27:14 2020 +0000 Merge remote-tracking branch 'gitlab/merge-requests/11' into master projects/pion-stun/config | 2 +- projects/snowflake/config | 2 +- .../tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix | 2 +- projects/tor-browser/… [View More]

1 0

[tor-browser-build/master] Bug 40016: update Snowflake to discover NAT type
by gk＠torproject.org 07 Aug '20

07 Aug '20

commit 251a6f767679b14b4e13b09920fe909d41462814 Author: Cecylia Bocovich <cohosh(a)torproject.org> Date: Thu Jul 9 13:03:30 2020 +0000 Bug 40016: update Snowflake to discover NAT type This updates Snowflake and the set of default stun servers used by the Snowflake client. --- projects/pion-stun/config | 2 +- projects/snowflake/config | 2 +- .../tor-browser/Bundle-Data/… [View More]PTConfigs/linux/torrc-defaults-appendix | 2 +- projects/tor-browser/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix | 2 +- .../tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix | 2 +- .../tor-onion-proxy-library/0001-Bug-30318-Add-snowflake-support.patch | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/projects/pion-stun/config b/projects/pion-stun/config index 37405eb..546e379 100644 --- a/projects/pion-stun/config +++ b/projects/pion-stun/config @@ -1,7 +1,7 @@ # vim: filetype=yaml sw=2 version: '[% c("abbrev") %]' git_url: https://github.com/pion/stun -git_hash: 51fef3542ed793994ff3da9c9b897fb23d744503 # v0.3.3 +git_hash: 4fa7a9d658b6037114fd35b6cb257979ed474a9b # v0.3.5 filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' build: '[% c("projects/go/var/build_go_lib") %]' diff --git a/projects/snowflake/config b/projects/snowflake/config index 898fe70..10fe1cc 100644 --- a/projects/snowflake/config +++ b/projects/snowflake/config @@ -1,7 +1,7 @@ # vim: filetype=yaml sw=2 version: '[% c("abbrev") %]' git_url: https://git.torproject.org/pluggable-transports/snowflake.git -git_hash: 7043a055f9fb0680281ecffd7d458a43f2ce65b5 +git_hash: 82031289a3362ff3fa199628276d76e840491431 filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' var: diff --git a/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix b/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix index ac89698..411c6e1 100644 --- a/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix +++ b/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix @@ -2,4 +2,4 @@ ClientTransportPlugin meek_lite,obfs2,obfs3,obfs4,scramblesuit exec ./TorBrowser/Tor/PluggableTransports/obfs4proxy ## snowflake configuration -ClientTransportPlugin snowflake exec ./TorBrowser/Tor/PluggableTransports/snowflake-client -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302 +ClientTransportPlugin snowflake exec ./TorBrowser/Tor/PluggableTransports/snowflake-client -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302,stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478 diff --git a/projects/tor-browser/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix b/projects/tor-browser/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix index 154bda4..baa0a6e 100644 --- a/projects/tor-browser/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix +++ b/projects/tor-browser/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix @@ -2,4 +2,4 @@ ClientTransportPlugin meek_lite,obfs2,obfs3,obfs4,scramblesuit exec PluggableTransports/obfs4proxy ## snowflake configuration -ClientTransportPlugin snowflake exec PluggableTransports/snowflake-client -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302 +ClientTransportPlugin snowflake exec PluggableTransports/snowflake-client -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302,stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478 diff --git a/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix b/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix index af16b3c..42502f4 100644 --- a/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix +++ b/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix @@ -2,4 +2,4 @@ ClientTransportPlugin meek_lite,obfs2,obfs3,obfs4,scramblesuit exec TorBrowser\Tor\PluggableTransports\obfs4proxy.exe ## snowflake configuration -ClientTransportPlugin snowflake exec TorBrowser\Tor\PluggableTransports\snowflake-client.exe -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302 +ClientTransportPlugin snowflake exec TorBrowser\Tor\PluggableTransports\snowflake-client.exe -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302,stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478 diff --git a/projects/tor-onion-proxy-library/0001-Bug-30318-Add-snowflake-support.patch b/projects/tor-onion-proxy-library/0001-Bug-30318-Add-snowflake-support.patch index 03e06aa..f168eed 100644 --- a/projects/tor-onion-proxy-library/0001-Bug-30318-Add-snowflake-support.patch +++ b/projects/tor-onion-proxy-library/0001-Bug-30318-Add-snowflake-support.patch @@ -71,7 +71,7 @@ index 2405097..bcb6a37 100644 - buffer.append("ClientTransportPlugin meek_lite,obfs3,obfs4 exec ").append(clientPath).append('\n'); + public TorConfigBuilder transportPlugin(String obfsPath, String snowPath) { + buffer.append("ClientTransportPlugin meek_lite,obfs3,obfs4 exec ").append(obfsPath).append('\n'); -+ buffer.append("ClientTransportPlugin snowflake exec ").append(snowPath).append(" -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302\n"); ++ buffer.append("ClientTransportPlugin snowflake exec ").append(snowPath).append(" -url https://snowflake-broker.azureedge.net/ -front ajax.aspnetcdn.com -ice stun:stun.l.google.com:19302,stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478\n"); return this; } [View Less]

1 0