February 2019 - tor-commits - lists.torproject.org

[sbws/master] docs: Add Tor bandwidth files documentation
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit b01ce3ddb367b78e0328168c815750baa402a6a1 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 11:20:35 2019 +0000 docs: Add Tor bandwidth files documentation --- docs/source/tor_bandwidth_files.rst | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/source/tor_bandwidth_files.rst b/docs/source/tor_bandwidth_files.rst new file mode 100644 index 0000000..94aaae7 --- /dev/null +++ b/docs/source/tor_bandwidth_files.rst @@ -0,0 +1,32 @@ +How bandwidth files are shown in the Tor network +================================================= + +Directory authorities' votes +----------------------------- + +moria, using Tor 0.3.5.7: + +.. code:: text + + bandwidth-file-headers timestamp=1548181637 + +https://collector.torproject.org/recent/relay-descriptors/votes/ + +To appear in Tor v0.4.1.x: + +.. code:: text + + bandwidth-file-digest sha256=01234567890123456789abcdefghijkl + +https://trac.torproject.org/projects/tor/ticket/26698 + +Directory authorities' bandwidth file URL +----------------------------------------- + +To appear in Tor v0.4.1.x: + +.. code:: text + + /tor/status-vote/next/bandwidth.z + +https://trac.torproject.org/projects/tor/ticket/21377

1 0

[sbws/master] docs: Add metrics bandwidth documentation
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit d9bcfca1f3f502cffb85c4fe5013710cb4e8d44f Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:02:52 2019 +0000 docs: Add metrics bandwidth documentation --- docs/source/images/advertised_bandwidth.png | Bin 0 -> 58213 bytes docs/source/images/torperf.png | Bin 0 -> 99045 bytes docs/source/images/totalcw.png | Bin 0 -> 81651 bytes docs/source/monitoring_bandwidth.rst | 35 ++++++++++++++++++++++++++++ 4 files changed, 35 insertions(+) diff --git a/docs/source/images/advertised_bandwidth.png b/docs/source/images/advertised_bandwidth.png new file mode 100644 index 0000000..9e85e06 Binary files /dev/null and b/docs/source/images/advertised_bandwidth.png differ diff --git a/docs/source/images/torperf.png b/docs/source/images/torperf.png new file mode 100644 index 0000000..acd33db Binary files /dev/null and b/docs/source/images/torperf.png differ diff --git a/docs/source/images/totalcw.png b/docs/source/images/totalcw.png new file mode 100644 index 0000000..ac8d8b7 Binary files /dev/null and b/docs/source/images/totalcw.png differ diff --git a/docs/source/monitoring_bandwidth.rst b/docs/source/monitoring_bandwidth.rst new file mode 100644 index 0000000..888d544 --- /dev/null +++ b/docs/source/monitoring_bandwidth.rst @@ -0,0 +1,35 @@ +Monitoring bandwidth changes in the Tor Network +================================================ + +Total bandwidth +--------------- + +Should not decrease. + +.. image:: images/advertised_bandwidth.png + :alt: advertised bandwidth + + +https://metrics.torproject.org/bandwidth-flags.html + + +Time to download a file +----------------------- + +Should not increase. + +.. image:: images/torperf.png + :alt: torperf + + +https://metrics.torproject.org/torperf.html + + +Total consensus weights across bandwidth authorities +---------------------------------------------------- + +.. image:: images/totalcw.png + :alt: total consensus weight + + +https://metrics.torproject.org/totalcw.html

1 0

[sbws/master] docs: Update internal Tor configuration
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 1a1a97faef6f554c32a3d948346094a875d3704a Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:24:23 2019 +0000 docs: Update internal Tor configuration and change titles. --- docs/source/config_tor.rst | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/docs/source/config_tor.rst b/docs/source/config_tor.rst index 1ddde81..175a9ff 100644 --- a/docs/source/config_tor.rst +++ b/docs/source/config_tor.rst @@ -1,18 +1,37 @@ .. _config_tor: -sbws scanner tor configuration -------------------------------- +Internal Tor configuration for the scanner +------------------------------------------ -At the time of writing, sbws sets the following torrc options for the following -reasons when it launches Tor. You can find them in ``sbws/globals.py`` and -``sbws/util/stem.py``. +The scanner needs an specific Tor configuration. +The following options are either set when launching Tor or required when +connection to an existing Tor daemon. + +Default configuration: - ``SocksPort auto``: To proxy requests over Tor. - ``CookieAuthentication 1``: The easiest way to authenticate to Tor. -- ``LearnCircuitBuildTimeout 0``: To keep circuit build timeouts static. -- ``CircuitBuildTimeout 10``: To give up on struggling circuits sooner. - ``UseEntryGuards 0``: To avoid path bias warnings. -- ``DataDirectory ...``: To set Tor's datadirectory to be inside sbws's. -- ``PidFile ...``: To make it easier to tell if Tor is running. -- ``ControlSocket ...``: To control Tor. -- ``Log notice ...``: To know what the heck is going on. +- ``UseMicrodescriptors 0``: Because full server descriptors are needed. +- ``SafeLogging 0``: Useful for logging, since there's no need for anonymity. +- ``LogTimeGranularity 1`` +- ``ProtocolWarnings 1`` +- ``LearnCircuitBuildTimeout 0``: To keep circuit build timeouts static. + +Configuration that depends on the user configuration file: + +- ``CircuitBuildTimeout ...``: The timeout trying to build a circuit. +- ``DataDirectory ...``: The Tor data directory path. +- ``PidFile ...``: The Tor PID file path. +- ``ControlSocket ...``: The Tor control socket path. +- ``Log notice ...``: The Tor log level and path. + +Configuration that needs to be set on runtime: + +- ``__DisablePredictedCircuits 1``: To build custom circuits. +- ``__LeaveStreamsUnattached 1`` + +Currently most of the code that sets this configuration is in :func:`sbws.util.stem.launch_tor` +and the default configuration is ``sbws/globals.py``. + +.. note:: the location of these code is being refactored. \ No newline at end of file

1 0

[sbws/master] docs: Change title to API page
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 0160a6bc3628095dcb824b5b8a8df09c3bac0559 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:25:16 2019 +0000 docs: Change title to API page --- docs/source/sbws.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/sbws.rst b/docs/source/sbws.rst index 3c78a63..0495138 100644 --- a/docs/source/sbws.rst +++ b/docs/source/sbws.rst @@ -1,4 +1,4 @@ -sbws package API +Package API ================= Subpackages

1 0

[sbws/master] docs: Update and change order in index
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit f27d8dcc4c0be2e54c698eacde204d72a774b244 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:39:19 2019 +0000 docs: Update and change order in index Closes: #29149. Bugfix v0.1.0. --- docs/source/index.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index a41a8de..e6c5d6e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -37,17 +37,21 @@ Included in the .. toctree:: :maxdepth: 1 - specification - faq - glossary - roadmap - config_tor testing documenting + how_works + code_design state config - diagrams + config_tor sbws + bandwidth_distribution + tor_bandwidth_files + bandwidth_authorities + monitoring_bandwidth + roadmap + glossary + faq Proposals:

1 0

[sbws/master] docs: Add viewcode extension
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 127609f5053ae31dee74dd775e37503b52553514 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:40:02 2019 +0000 docs: Add viewcode extension and numerate figures. --- docs/source/conf.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index aa8ffca..3dc8660 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -49,7 +49,8 @@ extensions = [ 'sphinx.ext.coverage', 'sphinx.ext.githubpages', 'sphinx.ext.imgmath', - 'sphinx.ext.intersphinx' + 'sphinx.ext.intersphinx', + 'sphinx.ext.viewcode' ] # Add any paths that contain templates here, relative to this directory. @@ -199,3 +200,5 @@ todo_include_todos = True source_parsers = { '.md': 'recommonmark.parser.CommonMarkParser', } + +numfig = True

1 0

[sbws/master] docs: Change titles to internal configuration page
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 3a59f5d02fdc9243ad061032a18110e98254ec75 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:05:31 2019 +0000 docs: Change titles to internal configuration page --- docs/source/config.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/config.rst b/docs/source/config.rst index 5aabc29..328ce69 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -1,7 +1,7 @@ .. _config_internal: -How sbws configuration works internally ----------------------------------------- +Internal code configuration files +================================== Sbws has two default config files it reads: on general, and one specific to logging. They all get combined internally to the same ``conf`` structure. @@ -34,8 +34,8 @@ The user example config file provided by ``sbws`` might look like this. .. _default-config: -Default Config --------------- +Default Configuration +---------------------- .. literalinclude:: config.default.ini :caption: config.default.ini

1 0

[sbws/master] docs: add link to State class
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 8a2fa4391f95efc6388321f79b7f079fc31fd367 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 10:27:41 2019 +0000 docs: add link to State class --- docs/source/state.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/state.rst b/docs/source/state.rst index b61b69e..6a692a5 100644 --- a/docs/source/state.rst +++ b/docs/source/state.rst @@ -18,3 +18,5 @@ The last time ``sbws scanner`` was started. - **Producer**: ``sbws scanner``, once at startup. - **Consumer**: ``sbws generate``, once each time it is ran. + +Code: :class:`sbws.util.state.State` \ No newline at end of file

1 0

[sbws/master] docs: replace http to https
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 0fb01281435dd2661b5f62f64852630d46cccd92 Author: juga0 <juga(a)riseup.net> Date: Tue Feb 12 11:36:40 2019 +0000 docs: replace http to https --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2bbe7ff..70508da 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ of the Tor bandwidth authorities, to avoid creating unnecessary traffic. **ADVICE**: It is recommended to read this documentation at [Read the Docs](https://sbws.rtfd.io). In -[Github](http://github.com/torproject/sbws) some links won't be properly +[Github](https://github.com/torproject/sbws) some links won't be properly rendered. It can also be read after installing the Debian package ``sbws-doc`` in ``/usr/share/doc/sbws`` or after building it locally as explained in

1 0

[sbws/master] generate: Load all results, including error ones
by juga＠torproject.org 23 Feb '19

23 Feb '19

commit 9e31dcfb5662993a285e9b625e24255ef6f1e54d Author: juga0 <juga(a)riseup.net> Date: Sat Feb 23 10:28:46 2019 +0000 generate: Load all results, including error ones The bandwidth file is generated only with relays that have some successful measurements, but we forgot to load all results (including the relays with failed measurements) after we started to include the number of failure measurements in each relay bandwidth line. Closes #29568. Bugfix v0.4.0 (line introduced in v0.1.0). --- sbws/core/generate.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbws/core/generate.py b/sbws/core/generate.py index 353bea0..df86029 100644 --- a/sbws/core/generate.py +++ b/sbws/core/generate.py @@ -94,7 +94,7 @@ def main(args, conf): reset_bw_ipv4_changes = conf.getboolean('general', 'reset_bw_ipv4_changes') reset_bw_ipv6_changes = conf.getboolean('general', 'reset_bw_ipv6_changes') results = load_recent_results_in_datadir( - fresh_days, datadir, success_only=True, + fresh_days, datadir, on_changed_ipv4=reset_bw_ipv4_changes, on_changed_ipv6=reset_bw_ipv6_changes) if len(results) < 1:

1 0