September 2018 - tor-commits - lists.torproject.org

[sbws/master] Remove unneed steps, include system dependencies example
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit d42c53080b7ab35bc60d823e5fbc8565f62c66c5 Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:04:20 2018 +0000 Remove unneed steps, include system dependencies example --- INSTALL.rst | 30 ++---------------------------- 1 file changed, 2 insertions(+), 28 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index 1c6775e..4122e4a 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -16,35 +16,9 @@ System requirements - virtualenv_ (while there is not ``stem`` release > 1.6.0, it is recommended to install the required python dependencies in a virtualenv) +In Debian:: -.. _Stem: https://stem.torproject.org/ -.. _Requests: http://docs.python-requests.org/ - -.. _release page: https://github.com/pastly/simple-bw-scanner/releases - -Virtualenv - Development ------------------------------------------------------------------------------- - -Choose a directory to store code in. I might choose ``~/src``. - -:: - - cd ~/src - -Get stem -~~~~~~~~~~~~~ - -:: - - git clone https://git.torproject.org/stem.git - -Get sbws -~~~~~~~~~~~~~~~~~ - -:: - - git clone https://github.com/pastly/simple-bw-scanner.git - cd simple-bw-scanner + sudo apt install tor python3 virtualenv Create and enter virtualenv

1 0

[sbws/master] Add refernce to deploy
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit 18d992abbfdf4a0110bd9c36950c52470d2f5f3c Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:08:01 2018 +0000 Add refernce to deploy --- INSTALL.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index 21e8fbd..2fccc46 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -45,8 +45,8 @@ Install the python dependencies:: install it. It's deprecated, but it won't be needed as soon as there is an ``stem`` release > 1.6.0 -Run sbws -~~~~~~~~~~ +``sbws`` needs :term:`destination` s to request files from. +Please, see :ref:`deploy` to know how to configure, deploy and run ``sbws``. If you would like to use a custom configuration file you can create it in ``~/.sbws.ini`` or provide ``sbws`` with it via ``-c`` ``--config`` option.

1 0

[sbws/master] Add extra dependencies for tests. Remove obsolete
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit b199c3afa122f77201299c05b23be501cc0cd16b Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:09:08 2018 +0000 Add extra dependencies for tests. Remove obsolete --- INSTALL.rst | 34 +++++++++------------------------- 1 file changed, 9 insertions(+), 25 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index 2fccc46..4041223 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -48,35 +48,19 @@ Install the python dependencies:: ``sbws`` needs :term:`destination` s to request files from. Please, see :ref:`deploy` to know how to configure, deploy and run ``sbws``. -If you would like to use a custom configuration file you can create it in -``~/.sbws.ini`` or provide ``sbws`` with it via ``-c`` ``--config`` option. +Installing tests dependencies and running them +------------------------------------------------ -See the documentation section about configuration files for more information -about how to create a configuration file. +To run the tests, extra Python depenencies are needed: -Tor run the scanner run -:: - - sbws scanner - - -[OBSOLETE DO NOT FOLLOW] Virtualenv - Production ------------------------------------------------------------------------------- - -Installing -~~~~~~~~~~ - -:: +- Flake8_ +- tox_ +- pytest_ +- coverage_ - git clone https://github.com/pastly/simple-bw-scanner.git - cd simple-bw-scanner - git checkout v1.5.3 - virtualenv -p python3 venv - source venv/bin/activate - pip install --process-dependency-links . - sbws init +To install them from ``sbws`` :: -.. note:: + pip install .[dev] && pip install .[doc] Because we relay on a ``-dev`` version of stem, we need to fetch it from git.torproject.org. Thus ``--process-dependency-links`` is necessary.

1 0

[sbws/master] Remove paragraph about versions
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit a0f750566b85458ab92bb3250efca12afd246899 Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:02:27 2018 +0000 Remove paragraph about versions --- INSTALL.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index b61c642..1c6775e 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -16,11 +16,6 @@ System requirements - virtualenv_ (while there is not ``stem`` release > 1.6.0, it is recommended to install the required python dependencies in a virtualenv) -Read all the information for the installation method of your choice before -beginning. Often you will want to be armed with the knowledge of the latest -released version of sbws. Determine that by examining its git tags, or visiting -its `release page`_. In the remainder of this document, we assume the latest -version is 1.5.3, which would be tagged as ``v1.5.3``. .. _Stem: https://stem.torproject.org/ .. _Requests: http://docs.python-requests.org/

1 0

[sbws/master] And remove it from README
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit 2303b06399edb4a976872b491d0cde6d62c35dbe Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:12:46 2018 +0000 And remove it from README --- README.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/README.md b/README.md index ddf4d20..6febd3a 100644 --- a/README.md +++ b/README.md @@ -111,14 +111,6 @@ how its code is laid out. Therefore the code may change drastically without a major version bump as long as the way users interact with it does not change in a backward incompatible way. -### Build HTML documentation - - pip install -e .[doc] - cd docs - make html - -The generated HTML will be in `docs/build/`. - ## The `.sbws` directory By default is `~/.sbws`.

1 0

[sbws/master] Update links to the dependencies
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit bd17b20dde344bb64e5956afb8665979ecdd9efe Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:13:19 2018 +0000 Update links to the dependencies --- INSTALL.rst | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/INSTALL.rst b/INSTALL.rst index b774ae2..8a5807b 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -98,4 +98,14 @@ To build the documentation diagrams:: The generated diagrams will be in `docs/build/images/`. - This doesn't update dependencies and needs to be fixed. +.. _virtualenv: https://virtualenv.pypa.io/en/stable/installation/ +.. _Stem: https://stem.torproject.org/ +.. _socks: http://docs.python-requests.org/en/master/user/advanced/#socks +.. _Requests: http://docs.python-requests.org/ +.. _Flake8: http://flake8.pycqa.org/ +.. _pytest: https://docs.pytest.org/ +.. _tox: https://tox.readthedocs.io +.. _Coverage: https://coverage.readthedocs.io/ +.. _Sphinx: http://www.sphinx-doc.org +.. _recommonmark: https://recommonmark.readthedocs.io/ +.. _Pylint_: https://www.pylint.org/ \ No newline at end of file

1 0

[sbws/master] And remove it from README
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit 59b42b085931c44fb474753e04cead8bc8945de8 Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:10:23 2018 +0000 And remove it from README --- README.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/README.md b/README.md index 7072f8b..ddf4d20 100644 --- a/README.md +++ b/README.md @@ -133,12 +133,3 @@ In this directory you will find - `v3bw/` This directory stores the v3bw files created with `sbws generate`. - `state.dat` A file for storing state needed between sbws commands. See its documentation for more information. - -## Running tests - -Make sure you have test dependencies installed. From within the top level -repository directory: - - pip install -e .[test] - -This should install tox and pytest. Then simply run `tox`.

1 0

[sbws/master] Explain how to build docs. Remove obsolete
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit b16d5c0113c7bfe299336ddc2515e5f1c5ff8247 Author: juga0 <juga(a)riseup.net> Date: Tue Aug 28 15:11:56 2018 +0000 Explain how to build docs. Remove obsolete --- INSTALL.rst | 31 +++++++++---------------------- 1 file changed, 9 insertions(+), 22 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index f83489d..b774ae2 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -80,35 +80,22 @@ To install them from ``sbws``:: pip install .[doc] -:: - - git clone https://github.com/pastly/simple-bw-scanner.git - cd simple-bw-scanner - git checkout v1.5.3 - virtualenv -p python3 venv-editable - source venv-editable/bin/activate - pip install --process-dependency-links --editable . - sbws init +To build the documentation as HTML:: -.. note:: + cd docs/ && make html - Because we relay on a ``-dev`` version of stem, we need to fetch it from - git.torproject.org. Thus ``--process-dependency-links`` is necessary. +The generated HTML will be in `docs/build/`. -.. warning:: +To build the manual (``man``) pages:: - Run these commands one at a time and check for errors before continuing. + cd docs/ && make man -Updating -~~~~~~~~ +The generated man pages will be in `docs/man/`. -:: +To build the documentation diagrams:: - cd simple-bw-scanner - git pull - # Determine the newest released version. Assuming it is v1.5.3 ... - git checkout v1.5.3 + cd docs/ && make umlsvg -.. todo:: +The generated diagrams will be in `docs/build/images/`. This doesn't update dependencies and needs to be fixed.

1 0

[sbws/master] Fix broken envvar in config.default.ini
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit c0885dcb4e77c58eb0afe9635172a5a93557dd0e Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Tue Aug 28 14:20:27 2018 -0400 Fix broken envvar in config.default.ini --- CHANGELOG.md | 5 +++++ sbws/config.default.ini | 2 +- sbws/util/config.py | 3 ++- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e1a6d1a..b760644 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ## [Unreleased] +### Fixed + +- Broken environment variable in default sbws config. To use envvar $FOO, write + $$FOO in the config. + ## [0.7.0] - 2018-08-09 **Important changes**: diff --git a/sbws/config.default.ini b/sbws/config.default.ini index 0ad0ebf..e7950b1 100644 --- a/sbws/config.default.ini +++ b/sbws/config.default.ini @@ -59,7 +59,7 @@ max_download_size = 1073741824 [tor] datadir = ${paths:sbws_home}/tor # create runtime dir for data that does not need to be persistent -run_dpath = $XDG_RUNTIME_DIR/sbws +run_dpath = $$XDG_RUNTIME_DIR/sbws control_socket = ${tor:run_dpath}/control pid = ${tor:run_dpath}/tor.pid # note this is a directory diff --git a/sbws/util/config.py b/sbws/util/config.py index 9532bdc..3577c5a 100644 --- a/sbws/util/config.py +++ b/sbws/util/config.py @@ -26,7 +26,8 @@ log = logging.getLogger(__name__) def _expand_path(path): """Expand path string containing shell variables and ~ constructions - into their values. + into their values. Environment variables have to have their $ escaped by + another $. For example: $$XDG_RUNTIME_DIR/foo.bar """ return os.path.expanduser(os.path.expandvars(path))

1 0

[sbws/master] Fix a missed conf['path'] -> conf.getpath() transition
by pastly＠torproject.org 05 Sep '18

05 Sep '18

commit 43812bed068fd1933022ccae8e83d67f9e84cb1e Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Tue Aug 28 14:18:59 2018 -0400 Fix a missed conf['path'] -> conf.getpath() transition --- sbws/core/scanner.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbws/core/scanner.py b/sbws/core/scanner.py index 3e24d22..4e59498 100644 --- a/sbws/core/scanner.py +++ b/sbws/core/scanner.py @@ -379,7 +379,7 @@ def main(args, conf): os.makedirs(conf.getpath('paths', 'datadir'), exist_ok=True) - state = State(conf['paths']['state_fname']) + state = State(conf.getpath('paths', 'state_fname')) state['scanner_started'] = now_isodt_str() try:

1 0