[tor-commits] [sbws/maint-1.1] fix: doc: Separate how scanner and generator work
juga at torproject.org
juga at torproject.org
Fri Feb 26 10:17:25 UTC 2021
commit ece2e2d60a7d52175940a5275c45e9fdd09ccdae
Author: juga0 <juga at riseup.net>
Date: Wed Feb 17 16:47:25 2021 +0000
fix: doc: Separate how scanner and generator work
in different files and link to each other.
---
docs/source/generator.rst | 50 ++++++++++++++++++++++++++++++++++++
docs/source/how_works.rst | 65 ++++++++---------------------------------------
docs/source/index.rst | 1 +
3 files changed, 61 insertions(+), 55 deletions(-)
diff --git a/docs/source/generator.rst b/docs/source/generator.rst
new file mode 100644
index 0000000..ef94703
--- /dev/null
+++ b/docs/source/generator.rst
@@ -0,0 +1,50 @@
+.. _generator:
+
+How the Bandwidth File is generated
+===================================
+
+.. seealso:: :ref:`scanner`.
+
+Every hour, the generator:
+
+#. Aggregate all the measurements (not older than 6 six days) for every relay.
+#. Filter the measurements
+#. Scale the measurements
+#. Write the bandwidth file
+
+Source code: :func:`sbws.lib.v3bwfile.V3BWFile.from_results`
+
+Filtering the bandwidth measurements
+-------------------------------------
+
+Each relay bandwidth measurements are selected in the following way:
+
+#. At least two bandwidth measurements (``Result`` s) MUST have been obtained
+ within an arbitrary number of seconds (currently one day).
+ If they are not, the relay MUST NOT be included in the Bandwith File.
+#. The measurements than are are older than an arbitrary number of senconds
+ in the past MUST be discarded.
+ Currently this number is the same as ``data_period`` (5 days) when not
+ scaling as Torflow and 28 days when scaling as Torflow.
+
+If the number of relays to include in the Bandwidth File are less than
+a percententage (currently 60%) than the number of relays in the consensus,
+additional Header Lines MUST be added (see XXX) to the Bandwith File and the
+relays SHOULD NOT be included.
+
+Scaling the bandwidth measurements
+------------------------------------
+
+Consensus bandwidth obtained by new implementations MUST be comparable to the
+consensus bandwidth, therefore they MUST implement torflow_scaling_.
+The bandwidth_file_spec_ appendix B describes torflow scaling and a linear
+scaling method.
+
+Writing the bandwidth file
+---------------------------
+
+The bandwidth file format is defined in the bandwidth_file_spec_.
+
+
+.. _torflow_scaling: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n298
+.. _bandwidth_file_spec: https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt
diff --git a/docs/source/how_works.rst b/docs/source/how_works.rst
index ff9b999..ebe4cf9 100644
--- a/docs/source/how_works.rst
+++ b/docs/source/how_works.rst
@@ -1,11 +1,10 @@
-What the scanner and the generator do
-======================================
+.. _scanner:
-Running the scanner
------------------------
+How sbws works
+==============
Overview
-~~~~~~~~~
+---------
.. The following text is part of the introduction in the README, but rst
formatted.
@@ -28,7 +27,7 @@ by a :term:`directory authority` to report relays’ bandwidth in its vote.
.. image:: ./images/dirauths_bwauths.png
Intialization
-~~~~~~~~~~~~~~
+--------------
.. At some point it should be able to get environment variables
@@ -58,7 +57,7 @@ Classes used in the initialization:
Source code: :func:`sbws.core.scanner.run_speedtest`
Measuring relays
-~~~~~~~~~~~~~~~~~
+-----------------
#. For every relay:
#. Select a second relay to build a Tor circuit.
@@ -84,7 +83,7 @@ Measuring a relay
Source code: :func:`sbws.core.scanner.measure_relay`
Selecting a second relay
-~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------
#. If the relay to measure is an exit, use it as an exit and obtain the
non-exits.
@@ -102,7 +101,7 @@ Selecting a second relay
Source code: :func:`sbws.core.scanner.measure_relay`
Selecting the data to download
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-------------------------------
#. While the downloaded data is smaller than 1GB or the number of download
is minor than 5:
@@ -116,7 +115,7 @@ Selecting the data to download
Source code: :func:`sbws.core.scanner._should_keep_result`
Writing the measurements to the filesystem
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-------------------------------------------
For every measured relay, the measurement result is put in a queue.
There's an independent thread getting measurements from the queue every second.
@@ -126,54 +125,10 @@ The file is named with the current date. Every day a new file is created.
Source code: :func:`sbws.lib.resultdump.ResultDump.enter`
-Running the generator
------------------------
-
-Every hour, the generator:
-
-#. Aggregate all the measurements (not older than 6 six days) for every relay.
-#. Filter the measurements
-#. Scale the measurements
-#. Write the bandwidth file
-
-Source code: :func:`sbws.lib.v3bwfile.V3BWFile.from_results`
-
-Filtering the bandwidth measurements
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Each relay bandwidth measurements are selected in the following way:
-
-#. At least two bandwidth measurements (``Result`` s) MUST have been obtained
- within an arbitrary number of seconds (currently one day).
- If they are not, the relay MUST NOT be included in the Bandwith File.
-#. The measurements than are are older than an arbitrary number of senconds
- in the past MUST be discarded.
- Currently this number is the same as ``data_period`` (5 days) when not
- scaling as Torflow and 28 days when scaling as Torflow.
-
-If the number of relays to include in the Bandwidth File are less than
-a percententage (currently 60%) than the number of relays in the consensus,
-additional Header Lines MUST be added (see XXX) to the Bandwith File and the
-relays SHOULD NOT be included.
-
-Scaling the bandwidth measurements
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Consensus bandwidth obtained by new implementations MUST be comparable to the
-consensus bandwidth, therefore they MUST implement torflow_scaling_.
-The bandwidth_file_spec_ appendix B describes torflow scaling and a linear
-scaling method.
-
-Writing the bandwidth file
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The bandwidth file format is defined in the bandwidth_file_spec_.
-
+.. seealso:: :ref:`generator`.
.. _torflow: https://gitweb.torproject.org/torflow.git
.. _stem: https://stem.torproject.org
.. https://github.com/requests/requests/issues/4885
.. _requests: http://docs.python-requests.org/
.. _peerflow: https://www.nrl.navy.mil/itd/chacs/sites/www.nrl.navy.mil.itd.chacs/files/pdfs/16-1231-4353.pdf
-.. _torflow_scaling: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n298
-.. _bandwidth_file_spec: https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 58c3911..53c3886 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -40,6 +40,7 @@ Included in the
testing
documenting
how_works
+ generator
code_design
state
config
More information about the tor-commits
mailing list