September 2020 - tor-commits - lists.torproject.org

[onionperf/master] Order analysis and filter keys, authored by Karsten
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit 7b0c91f1983ab21f39f37b83d3a58db5a4d11a8f Author: Ana Custura <ana(a)netstat.org.uk> Date: Thu Sep 10 12:37:00 2020 +0100 Order analysis and filter keys, authored by Karsten --- onionperf/filtering.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/onionperf/filtering.py b/onionperf/filtering.py index c008c03..7ef6168 100644 --- a/onionperf/filtering.py +++ b/onionperf/filtering.py @@ -65,10 +65,12 @@ class Filtering(object): break if not keep: tor_circuits[circuit_id]["filtered"] = True + tor_circuits[circuit_id] = dict(sorted(tor_circuit.items())) def apply_filters(self, input_path, output_dir, output_file): self.analysis = OPAnalysis.load(filename=input_path) self.filter_tor_circuits(self.analysis) self.analysis.json_db["filters"] = self.filters + self.analysis.json_db = dict(sorted(self.analysis.json_db.items())) self.analysis.save(filename=output_file, output_prefix=output_dir, sort_keys=False)

1 0

[onionperf/master] Bump analysis version to 4.0
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit d0660c1847b46863783b7006efdac4df8103f123 Author: Ana Custura <ana(a)netstat.org.uk> Date: Thu Sep 10 12:54:41 2020 +0100 Bump analysis version to 4.0 --- onionperf/analysis.py | 4 ++-- onionperf/filtering.py | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/onionperf/analysis.py b/onionperf/analysis.py index 49c109c..907d451 100644 --- a/onionperf/analysis.py +++ b/onionperf/analysis.py @@ -24,7 +24,7 @@ class OPAnalysis(Analysis): def __init__(self, nickname=None, ip_address=None): super().__init__(nickname, ip_address) - self.json_db = {'type': 'onionperf', 'version': '3.0', 'data': {}} + self.json_db = {'type': 'onionperf', 'version': '4.0', 'data': {}} self.torctl_filepaths = [] def add_torctl_file(self, filepath): @@ -133,7 +133,7 @@ class OPAnalysis(Analysis): if 'type' not in db or 'version' not in db: logging.warning("'type' or 'version' not present in database") return None - elif db['type'] != 'onionperf' or str(db['version']) >= '4.': + elif db['type'] != 'onionperf' or str(db['version']) >= '5.': logging.warning("type or version not supported (type={0}, version={1})".format(db['type'], db['version'])) return None else: diff --git a/onionperf/filtering.py b/onionperf/filtering.py index 0ba9ebc..e8ea164 100644 --- a/onionperf/filtering.py +++ b/onionperf/filtering.py @@ -71,6 +71,7 @@ class Filtering(object): self.analysis = OPAnalysis.load(filename=input_path) self.filter_tor_circuits(self.analysis) self.analysis.json_db["filters"] = self.filters + self.analysis.json_db["version"] = '4.0' self.analysis.json_db = dict(sorted(self.analysis.json_db.items())) self.analysis.save(filename=output_file, output_prefix=output_dir, sort_keys=False)

1 0

[onionperf/master] Attempt to clean up CHANGELOG.md and README.md.
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit f063b636fd26a938a52508424d7abe62cd45ccf0 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 16 12:26:56 2020 +0200 Attempt to clean up CHANGELOG.md and README.md. --- CHANGELOG.md | 12 +++++------- README.md | 14 +++++--------- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a7d5005..13fb202 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,13 +1,11 @@ -# Changes in version 0.7 - 2020-??-?? +# Changes in version 0.8 - 2020-09-16 - Add a new `onionperf filter` mode that takes an OnionPerf analysis results file or directory as input, applies filters, and produces - new OnionPerf analysis results file(s) as output. - - Bump the analysis version number to 4.0 for new analyses and analyses - produced by the `onionperf filter` mode - - Analyses produced by the `onionperf filter` mode have additional filter - metadata defined in a new 'filters' field, and an optional 'filtered\_out' - field per tor circuit + new OnionPerf analysis results file(s) as output. Bump the analysis + version number to 4.0 to include additional filter metadata defined + in a 'filters' field and an optional 'filtered\_out' field per Tor + circuit. Implements #33260. # Changes in version 0.6 - 2020-08-08 diff --git a/README.md b/README.md index 13a9eee..6e7684f 100644 --- a/README.md +++ b/README.md @@ -252,31 +252,27 @@ OnionPerf's `analyze` mode has several command-line parameters for customizing t ```shell onionperf analyze --help ``` + ### Filtering measurement results -The `filter` subcommand is typically used in combination with the `visualize` subcommand. The workflow is to filter out any Tor streams/circuits that are not desired then visualize only those measurements with an existing mapping between TGen transfers/streams and Tor streams/circuits. +The `filter` subcommand can be used to filter out measurement results based on given criteria. This subcommand is typically used in combination with the `visualize` subcommand. The workflow is to apply one or more filters and then visualize only those measurements with an existing mapping between TGen transfers/streams and Tor streams/circuits. -Currently, OnionPerf measurement results can be filtered based on Tor relay fingerprints, although support for filtering TGen based on transfers/streams may be added in the future. +Currently, OnionPerf measurement results can be filtered based on Tor relay fingerprints found in Tor circuits, although support for filtering based on Tor streams and/or TGen transfers/streams may be added in the future. -The `filter` mode takes a list of fingerprints and one or more existing analysis files as inputs, and outputs new analysis files which include, unchanged, the Tor results obtained over a Tor circuit path which includes or excludes fingerprints in the input list. All other Tor results are also included in the file, but are marked as 'filtered\_out'. -Filter metadata detailing the filter type and path to the input list used is also included in the analysis file. +The `filter` mode takes a list of fingerprints and one or more existing analysis files as inputs and outputs new analysis files with the same contents as the input analysis files plus annotations on those Tor circuits that have been filtered out. If a directory of analysis files is given to '-i', the structure and filenames of that directory are preserved under the path specified with '-o'. -For example, the analysis file produced above can be filtered with the following command, which retains measurements based on fingerprints contained in the file 'fingerprints.txt': +For example, the analysis file produced above can be filtered with the following command, which retains only those Tor circuits with fingerprints contained in the file 'fingerprints.txt': ```shell onionperf filter -i onionperf.analysis.json.xz -o filtered.onionperf.analysis.json.xz --include-fingerprints fingerprints.txt ``` -The output analysis file is written to the path specified with `-o`. If processing a directory of analysis files, its structure and filenames are preserved under the path specified with '-o'. -Note that while the subcommand filters `tgen` measurements, it leaves `tgen` and `tor` summaries in the original analysis file unchanged. - OnionPerf's `filter` command usage can be inspected with: ```shell onionperf filter --help ``` - ### Visualizing measurement results Step two in the analysis is to process analysis files with OnionPerf's `visualize` mode which produces CSV and PDF files as output.

1 0

[onionperf/master] Append to existing "filters" field.
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit c41c0b74c47442dc6f6d23f49a012729f71aee49 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 16 12:01:07 2020 +0200 Append to existing "filters" field. Before this change we would have replaced an existing "filters" field with the filters applied in this run. But if the user runs `onionperf filter` more than once to apply different filters, we'll want to list all filters in the "filters" field. This change also removes a couple of `self.`s, in particular `self.analysis` and `self.filters`, because one Filtering object can apply filters to more than one analysis. --- onionperf/filtering.py | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/onionperf/filtering.py b/onionperf/filtering.py index e8ea164..15d9c19 100644 --- a/onionperf/filtering.py +++ b/onionperf/filtering.py @@ -7,7 +7,6 @@ import re from onionperf.analysis import OPAnalysis -from collections import defaultdict class Filtering(object): @@ -15,7 +14,6 @@ class Filtering(object): self.fingerprints_to_include = None self.fingerprints_to_exclude = None self.fingerprint_pattern = re.compile("\$?([0-9a-fA-F]{40})") - self.filters = defaultdict(list) def include_fingerprints(self, path): self.fingerprints_to_include = [] @@ -40,11 +38,12 @@ class Filtering(object): def filter_tor_circuits(self, analysis): if self.fingerprints_to_include is None and self.fingerprints_to_exclude is None: return - self.filters["tor/circuits"] = [] + filters = analysis.json_db.setdefault("filters", {}) + tor_circuits_filters = filters.setdefault("tor/circuits", []) if self.fingerprints_to_include: - self.filters["tor/circuits"].append({"name": "include_fingerprints", "filepath": self.fingerprints_to_include_path }) + tor_circuits_filters.append({"name": "include_fingerprints", "filepath": self.fingerprints_to_include_path }) if self.fingerprints_to_exclude: - self.filters["tor/circuits"].append({"name": "exclude_fingerprints", "filepath": self.fingerprints_to_exclude_path }) + tor_circuits_filters.append({"name": "exclude_fingerprints", "filepath": self.fingerprints_to_exclude_path }) for source in analysis.get_nodes(): tor_circuits = analysis.get_tor_circuits(source) filtered_circuit_ids = [] @@ -68,10 +67,9 @@ class Filtering(object): tor_circuits[circuit_id] = dict(sorted(tor_circuit.items())) def apply_filters(self, input_path, output_dir, output_file): - self.analysis = OPAnalysis.load(filename=input_path) - self.filter_tor_circuits(self.analysis) - self.analysis.json_db["filters"] = self.filters - self.analysis.json_db["version"] = '4.0' - self.analysis.json_db = dict(sorted(self.analysis.json_db.items())) - self.analysis.save(filename=output_file, output_prefix=output_dir, sort_keys=False) + analysis = OPAnalysis.load(filename=input_path) + self.filter_tor_circuits(analysis) + analysis.json_db["version"] = '4.0' + analysis.json_db = dict(sorted(analysis.json_db.items())) + analysis.save(filename=output_file, output_prefix=output_dir, sort_keys=False)

1 0

[onionperf/master] Bump version to 0.8.
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit 118d14990ce4821232ed6751c6a3d2b05e872dbe Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 16 12:29:43 2020 +0200 Bump version to 0.8. --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index d72b27a..b7b00d0 100644 --- a/setup.py +++ b/setup.py @@ -6,7 +6,7 @@ with open('requirements.txt') as f: install_requires = f.readlines() setup(name='OnionPerf', - version='0.7', + version='0.8', description='A utility to monitor, measure, analyze, and visualize the performance of Tor and Onion Services', author='Rob Jansen', url='https://github.com/robgjansen/onionperf/',

1 0

[onionperf/master] Merge branch 'acute-task-33260-4' into develop
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit 76517ac669f3e7575965662507ab970b41395324 Merge: c8275b2 f063b63 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 16 12:28:19 2020 +0200 Merge branch 'acute-task-33260-4' into develop CHANGELOG.md | 9 + README.md | 23 ++ onionperf/analysis.py | 18 +- onionperf/filtering.py | 75 ++++++ onionperf/onionperf | 73 ++++++ onionperf/visualization.py | 31 ++- schema/onionperf-4.0.json | 577 +++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 793 insertions(+), 13 deletions(-) diff --cc CHANGELOG.md index 80fb14d,13fb202..2c1f5ca --- a/CHANGELOG.md +++ b/CHANGELOG.md @@@ -1,13 -1,12 +1,22 @@@ + # Changes in version 0.8 - 2020-09-16 + + - Add a new `onionperf filter` mode that takes an OnionPerf analysis + results file or directory as input, applies filters, and produces + new OnionPerf analysis results file(s) as output. Bump the analysis + version number to 4.0 to include additional filter metadata defined + in a 'filters' field and an optional 'filtered\_out' field per Tor + circuit. Implements #33260. + +# Changes in version 0.7 - 2020-09-01 + + - Add `onionperf measure --drop-guards` parameter to use and drop + guards and circuit build timeouts every given number of hours, if + supported by the Tor version. Implements #33399. + - Remove the `onionperf measure --oneshot` switch and replace it with + new switches `--tgen-pause-initial`, `--tgen-pause-between`, + `--tgen-transfer-size`, and `--tgen-num-transfers ` to further + configure the generated TGen model. Implemets #33432. + # Changes in version 0.6 - 2020-08-08 - Update to TGen 1.0.0, use TGenTools for parsing TGen log files, and

1 0

[onionperf/develop] Bump version to 0.8.
by karsten＠torproject.org 18 Sep '20

18 Sep '20

commit 118d14990ce4821232ed6751c6a3d2b05e872dbe Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 16 12:29:43 2020 +0200 Bump version to 0.8. --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index d72b27a..b7b00d0 100644 --- a/setup.py +++ b/setup.py @@ -6,7 +6,7 @@ with open('requirements.txt') as f: install_requires = f.readlines() setup(name='OnionPerf', - version='0.7', + version='0.8', description='A utility to monitor, measure, analyze, and visualize the performance of Tor and Onion Services', author='Rob Jansen', url='https://github.com/robgjansen/onionperf/',

1 0

[translation/torbutton-torbuttonproperties] https://gitweb.torproject.org/translation.git/commit/?h=torbutton-torbuttonproperties
by translation＠torproject.org 18 Sep '20

18 Sep '20

commit b8cb99385b04aa3dc5a688c8773d981bc9fe9978 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Sep 18 04:47:19 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=torbutton-torbutton… --- ro/torbutton.properties | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ro/torbutton.properties b/ro/torbutton.properties index cb9c846f60..01b9ae0e06 100644 --- a/ro/torbutton.properties +++ b/ro/torbutton.properties @@ -60,7 +60,7 @@ updateDownloadingPanelUILabel=Se descarcă actualizarea %S # .Onion Page Info prompt. Strings are kept here for ease of translation. pageInfo_OnionEncryptionWithBitsAndProtocol=Conexiune Criptată (Serviciu Onion, %1$S, cheie pe %2$S biți, %3$S) pageInfo_OnionEncryption=Conexiune Criptată (Serviciu Onion) -pageInfo_OnionName=Onion Name: +pageInfo_OnionName=Nume Onion: # Onion services strings. Strings are kept here for ease of translation. onionServices.learnMore=Află mai mult @@ -111,7 +111,7 @@ onionServices.introTimedOut=Eroare la conectarea la Oniosite, posibil din cauza onionServices.introTimedOut.longDescription=Detalii: %S — Conexiunea la serviciul Onion solicitat a expirat în timp ce încercați să construiți circuitul de întâlnire. # # LOCALIZATION NOTE: %S will be replaced with the .onion address. -onionServices.authPrompt.description2=%S is requesting that you authenticate. +onionServices.authPrompt.description2=%S vă cere să vă autentificați. onionServices.authPrompt.keyPlaceholder=Introduceți cheia privată pentru acest serviciu de ceapă onionServices.authPrompt.done=Gata onionServices.authPrompt.doneAccessKey=d

1 0

[translation/tails-misc_release] https://gitweb.torproject.org/translation.git/commit/?h=tails-misc_release
by translation＠torproject.org 18 Sep '20

18 Sep '20

commit 87dc818e9a09ad68516c6ab85cf725deac6ca8ee Author: Translation commit bot <translation(a)torproject.org> Date: Fri Sep 18 04:46:38 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=tails-misc_release --- ro.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ro.po b/ro.po index 0e210f7111..754f6c09cf 100644 --- a/ro.po +++ b/ro.po @@ -27,7 +27,7 @@ msgstr "" "Project-Id-Version: Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2020-09-12 17:58+0200\n" -"PO-Revision-Date: 2020-09-18 04:15+0000\n" +"PO-Revision-Date: 2020-09-18 04:25+0000\n" "Last-Translator: eduard pintilie <eduard.pintilie(a)gmail.com>\n" "Language-Team: Romanian (http://www.transifex.com/otf/torproject/language/ro/)\n" "MIME-Version: 1.0\n"

1 0

[translation/tails-misc] https://gitweb.torproject.org/translation.git/commit/?h=tails-misc
by translation＠torproject.org 18 Sep '20

18 Sep '20

commit c6c7bf4b569b99d974f8743a61c708762a76b7d4 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Sep 18 04:45:59 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=tails-misc --- ro.po | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ro.po b/ro.po index be7bb402f1..e360457266 100644 --- a/ro.po +++ b/ro.po @@ -27,7 +27,7 @@ msgstr "" "Project-Id-Version: Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2020-09-12 17:58+0200\n" -"PO-Revision-Date: 2020-09-18 04:15+0000\n" +"PO-Revision-Date: 2020-09-18 04:25+0000\n" "Last-Translator: eduard pintilie <eduard.pintilie(a)gmail.com>\n" "Language-Team: Romanian (http://www.transifex.com/otf/torproject/language/ro/)\n" "MIME-Version: 1.0\n" @@ -1455,7 +1455,7 @@ msgstr "Navigatorul neprotejat Unsafe Browser" msgid "" "The Unsafe Browser was not enabled in the Welcome Screen.\\n\\nTo use the Unsafe Browser, restart Tails and enable the Unsafe Browser in the additional settings of the Welcome Screen.\\n\n" "To always enable the Unsafe Browser, turn on the Welcome Screen feature of the Persistent Storage." -msgstr "" +msgstr "Unsafe Browser nu a fost activat în ecranul de întâmpinare.\\n\\nPentru a utiliza Unsafe Browser, reinițializați Tails și activați Unsafe Browser în setările suplimentare ale ecranului de întâmpinare.\\n\nPentru a activa întotdeauna Unsafe Browser, activați caracteristica Ecran de întâmpinare a stocării persistente." #: config/chroot_local-includes/usr/local/sbin/unsafe-browser:99 msgid "" @@ -1645,19 +1645,19 @@ msgid "" "A captive portal is a web page that is displayed before you can access the Internet. Captive portals usually require to log in to the network or enter information such as an email address.\n" "\n" "The Unsafe Browser is not anonymous and can deanonymize you. Use it only to log in to captive portals." -msgstr "" +msgstr "Unsafe Browser vă permite să vă conectați la un portal captiv.\nPortalul captiv este o pagină web care este afișată înainte de a putea accesa Internetul. Portalurile captive necesită, de obicei, conectarea la rețea sau introducerea de informații, cum ar fi o adresă de e-mail.\nUnsafe Browser nu este anonim și vă poate deanonimiza. Utilizați-l numai pentru a vă conecta la portaluri captive." #: ../config/chroot_local-includes/usr/share/tails/greeter/additional_settings.ui.in:572 msgid "Disable the Unsafe Browser (default)" -msgstr "" +msgstr "Dezactivare Unsafe Browser (implicit)" #: ../config/chroot_local-includes/usr/share/tails/greeter/additional_settings.ui.in:617 msgid "Enable the Unsafe Browser" -msgstr "" +msgstr "Activare Unsafe Browser" #: ../config/chroot_local-includes/usr/share/tails/greeter/main.ui.in:100 msgid "Settings were loaded from the persistent storage." -msgstr "" +msgstr "Setările au fost încărcate din stocarea persistentă." #: ../config/chroot_local-includes/usr/share/tails/greeter/main.ui.in:168 msgid ""

1 0