July 2018 - tor-commits - lists.torproject.org

[sbws/master] Remove unused config option log_filepath
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit fa105e14697d30c624fe79cdd2c2d9fe2efbe139 Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Mon Jun 25 11:22:41 2018 -0400 Remove unused config option log_filepath --- sbws/config.default.ini | 1 - sbws/util/config.py | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/sbws/config.default.ini b/sbws/config.default.ini index 3e49052..dbe46a2 100644 --- a/sbws/config.default.ini +++ b/sbws/config.default.ini @@ -3,7 +3,6 @@ datadir = ${sbws_home}/datadir v3bw_dname = ${sbws_home}/v3bw v3bw_fname = ${v3bw_dname}/{}.v3bw started_filepath = ${sbws_home}/started_at -log_filepath = ${sbws_home}/sbws.log log_dname = ${sbws_home}/log [destinations] diff --git a/sbws/util/config.py b/sbws/util/config.py index 0e6f308..bf73540 100644 --- a/sbws/util/config.py +++ b/sbws/util/config.py @@ -198,7 +198,7 @@ def _validate_paths(conf): err_tmpl = Template('$sec/$key ($val): $e') unvalidated_keys = [ 'datadir', 'sbws_home', 'v3bw_fname', 'v3bw_dname', - 'started_filepath', 'log_filepath', 'log_dname'] + 'started_filepath', 'log_dname'] all_valid_keys = unvalidated_keys allow_missing = ['sbws_home'] errors.extend(_validate_section_keys(conf, sec, all_valid_keys, err_tmpl,

1 0

[sbws/master] Update docs about the config files that are read
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit 65978ba49b1e91c776fbb52e2d1a8d026d38ebe9 Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Wed Jul 4 11:52:34 2018 -0400 Update docs about the config files that are read --- docs/source/config.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/source/config.rst b/docs/source/config.rst index 69bc012..723949d 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -1,7 +1,7 @@ Simple Bandwidth Scanner Configuration Files ============================================ -Sbws has four config files it reads: two general, and two specific to logging. +Sbws has three config files it reads: two general, and one specific to logging. They all get combined internally to the same ``conf`` structure, so technically you can put any option in any file, but you need to pay attention to the order in which they are read. Options specified in multiple files will take the @@ -25,12 +25,7 @@ Sbws then reads your custom config file. By default, after running ``sbws init``, it is located in ``~/.sbws/config.ini``. Options in this file overwrite options set in previously read config files. -Finally, sbws reads your custom logging config file. By default, after running -``sbws init``, it is located in ``~/.sbws/config.log.ini``. Options set in this file -overwrite options set in all other config files. - -After running ``sbws init``, your ``~/.sbws/config.ini`` might look like this. Your -``~/.sbws/config.log.ini`` will be empty. +After running ``sbws init``, your ``~/.sbws/config.ini`` might look like this. .. _init-config: @@ -49,5 +44,11 @@ Default Config .. _default-log-config: +If you know how to use +`Python's logging configuration file format`_, +then you can override or add to what is listed here by editing your config.ini. + .. literalinclude:: config.log.default.ini :caption: config.log.default.ini + +.. _Python's logging configuration file format: https://docs.python.org/3.5/library/logging.config.html#logging-config-file…

1 0

[sbws/master] Add 2 conf opts for logging to file, but not used yet
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit d111af98e18a176590bffd9f80cb8c0b7ca3601b Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Wed Jul 4 11:32:18 2018 -0400 Add 2 conf opts for logging to file, but not used yet --- sbws/config.default.ini | 7 +++++++ sbws/util/config.py | 7 ++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/sbws/config.default.ini b/sbws/config.default.ini index dbe46a2..d9e2622 100644 --- a/sbws/config.default.ini +++ b/sbws/config.default.ini @@ -87,6 +87,13 @@ min_relays = 50 to_file = yes # Whether or not to log to stdout to_stdout = no +# If logging to file, how large (in bytes) should the file be allowed to get +# before rotating to a new one. 10485760 is 10 MiB. If zero or number of +# backups is zero, never rotate the log file. +to_file_max_bytes = 10485760 +# If logging to file, how many backups to keep. If zero or max bytes is zero, +# never rotate the log file. +to_file_num_backups = 50 # Level to log at. Debug, info, warning, error, critical. level = info to_file_level = ${level} diff --git a/sbws/util/config.py b/sbws/util/config.py index bf73540..8953d92 100644 --- a/sbws/util/config.py +++ b/sbws/util/config.py @@ -282,8 +282,13 @@ def _validate_logging(conf): 'to_file': {}, 'to_stdout': {}, } + ints = { + 'to_file_max_bytes': {'minimum': 0, 'maximum': None}, + 'to_file_num_backups': {'minimum': 0, 'maximum': None}, + } unvalidated = ['format', 'to_file_format', 'to_stdout_format'] - all_valid_keys = list(bools.keys()) + list(enums.keys()) + unvalidated + all_valid_keys = list(bools.keys()) + list(enums.keys()) + \ + list(ints.keys()) + unvalidated errors.extend(_validate_section_keys(conf, sec, all_valid_keys, err_tmpl)) errors.extend(_validate_section_bools(conf, sec, bools, err_tmpl)) errors.extend(_validate_section_enums(conf, sec, enums, err_tmpl))

1 0

[sbws/master] s/DEBUG/debug/
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit 9bb2913035c1c4e786a5cdd76c46a1c03191642a Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Wed Jul 4 12:02:23 2018 -0400 s/DEBUG/debug/ --- tests/integration/conftest.py | 2 +- tests/integration/core/test_scanner.py | 2 +- tests/integration/lib/test_relayprioritizer.py | 2 +- tests/unit/conftest.py | 2 +- tests/unit/core/test_cleanup.py | 20 ++++++++++---------- tests/unit/core/test_stats.py | 12 ++++++------ 6 files changed, 20 insertions(+), 20 deletions(-) diff --git a/tests/integration/conftest.py b/tests/integration/conftest.py index 94b2433..cfd0e28 100644 --- a/tests/integration/conftest.py +++ b/tests/integration/conftest.py @@ -19,7 +19,7 @@ def persistent_empty_dotsbws(parser): ''' d = TemporaryDirectory() args = parser.parse_args( - '-d {} --log-level DEBUG init'.format(d.name).split()) + '-d {} --log-level debug init'.format(d.name).split()) conf = get_config(args) sbws.core.init.main(args, conf) os.makedirs(os.path.join(d.name, 'datadir')) diff --git a/tests/integration/core/test_scanner.py b/tests/integration/core/test_scanner.py index 152e939..e7e0d86 100644 --- a/tests/integration/core/test_scanner.py +++ b/tests/integration/core/test_scanner.py @@ -22,7 +22,7 @@ def assert_within(value, target, radius): def get_everything_to_measure(dotsbws, cont, parser): args = parser.parse_args( - '-d {} --log-level DEBUG scanner'.format(dotsbws).split()) + '-d {} --log-level debug scanner'.format(dotsbws).split()) conf = get_config(args) conf['destinations']['foo'] = 'on' conf['destinations.foo'] = {} diff --git a/tests/integration/lib/test_relayprioritizer.py b/tests/integration/lib/test_relayprioritizer.py index 237a139..d626fe0 100644 --- a/tests/integration/lib/test_relayprioritizer.py +++ b/tests/integration/lib/test_relayprioritizer.py @@ -14,7 +14,7 @@ def static_time(value): def get_global_stuff(dotsbws, cont, parser): args = parser.parse_args( - '-d {} --log-level DEBUG'.format(dotsbws).split()) + '-d {} --log-level debug'.format(dotsbws).split()) conf = get_config(args) rl = RelayList(args, conf, cont) return { diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index 002987c..8915669 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -95,7 +95,7 @@ def empty_dotsbws(parser): ''' d = TemporaryDirectory() args = parser.parse_args( - '-d {} --log-level DEBUG init'.format(d.name).split()) + '-d {} --log-level debug init'.format(d.name).split()) conf = get_config(args) sbws.core.init.main(args, conf) return d diff --git a/tests/unit/core/test_cleanup.py b/tests/unit/core/test_cleanup.py index 0f6fc41..3278e00 100644 --- a/tests/unit/core/test_cleanup.py +++ b/tests/unit/core/test_cleanup.py @@ -13,7 +13,7 @@ def test_cleanup_no_dotsbws(tmpdir, caplog, parser): caplog.set_level(logging.DEBUG) dotsbws = tmpdir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws).split()) + '-d {} --log-level debug cleanup'.format(dotsbws).split()) conf = get_config(args) try: sbws.core.cleanup.main(args, conf) @@ -27,7 +27,7 @@ def test_cleanup_no_dotsbws(tmpdir, caplog, parser): def test_cleanup_no_datadir(empty_dotsbws, caplog, parser): dotsbws = empty_dotsbws args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) try: sbws.core.cleanup.main(args, conf) @@ -42,7 +42,7 @@ def test_cleanup_no_datadir(empty_dotsbws, caplog, parser): def test_cleanup_small_stale(empty_dotsbws_datadir, caplog, parser): dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '2' @@ -60,7 +60,7 @@ def test_cleanup_small_stale(empty_dotsbws_datadir, caplog, parser): def test_cleanup_small_rotten(empty_dotsbws_datadir, caplog, parser): dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '5' @@ -78,7 +78,7 @@ def test_cleanup_small_rotten(empty_dotsbws_datadir, caplog, parser): def test_cleanup_medium_stale(empty_dotsbws_datadir, caplog, parser): dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '10' conf['cleanup']['stale_days'] = '19' @@ -94,7 +94,7 @@ def test_cleanup_only_compress_stale(time_mock, empty_dotsbws_datadir, caplog, caplog.set_level(logging.DEBUG) dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '10' @@ -140,7 +140,7 @@ def test_cleanup_only_delete_rotten(time_mock, empty_dotsbws_datadir, caplog, caplog.set_level(logging.DEBUG) dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '10' @@ -188,7 +188,7 @@ def test_cleanup_nothing_to_do(empty_dotsbws_datadir, caplog, parser): caplog.set_level(logging.DEBUG) dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) try: sbws.core.cleanup.main(args, conf) @@ -203,7 +203,7 @@ def test_cleanup_compress_barely_stale(time_mock, empty_dotsbws_datadir, caplog.set_level(logging.DEBUG) dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '10' @@ -239,7 +239,7 @@ def test_cleanup_delete_barely_rotten(time_mock, empty_dotsbws_datadir, caplog.set_level(logging.DEBUG) dotsbws = empty_dotsbws_datadir args = parser.parse_args( - '-d {} --log-level DEBUG cleanup'.format(dotsbws.name).split()) + '-d {} --log-level debug cleanup'.format(dotsbws.name).split()) conf = get_config(args) conf['general']['data_period'] = '1' conf['cleanup']['stale_days'] = '5' diff --git a/tests/unit/core/test_stats.py b/tests/unit/core/test_stats.py index 9554b16..4bac862 100644 --- a/tests/unit/core/test_stats.py +++ b/tests/unit/core/test_stats.py @@ -18,7 +18,7 @@ import logging def init_directory(dname): p = create_parser() - args = p.parse_args('-d {} --log-level DEBUG init'.format(dname).split()) + args = p.parse_args('-d {} --log-level debug init'.format(dname).split()) conf = get_config(args) sbws.core.init.main(args, conf) @@ -64,7 +64,7 @@ def test_stats_uninitted(tmpdir, caplog): An un-initialized .sbws directory should fail hard and exit immediately ''' p = create_parser() - args = p.parse_args('-d {} --log-level DEBUG stats'.format(tmpdir).split()) + args = p.parse_args('-d {} --log-level debug stats'.format(tmpdir).split()) conf = get_config(args) try: sbws.core.stats.main(args, conf) @@ -83,7 +83,7 @@ def test_stats_initted(tmpdir, caplog): ''' init_directory(tmpdir) p = create_parser() - args = p.parse_args('-d {} --log-level DEBUG stats'.format(tmpdir).split()) + args = p.parse_args('-d {} --log-level debug stats'.format(tmpdir).split()) conf = get_config(args) try: sbws.core.stats.main(args, conf) @@ -103,7 +103,7 @@ def test_stats_stale_result(tmpdir, caplog): init_directory(tmpdir) add_single_stale_result(tmpdir) p = create_parser() - args = p.parse_args('-d {} --log-level DEBUG stats'.format(tmpdir).split()) + args = p.parse_args('-d {} --log-level debug stats'.format(tmpdir).split()) conf = get_config(args) sbws.core.stats.main(args, conf) assert 'No fresh results' == caplog.records[-1].getMessage() @@ -121,7 +121,7 @@ def test_stats_fresh_result(tmpdir, capsys, caplog): add_single_fresh_result(tmpdir) p = create_parser() args = p.parse_args( - '-d {} --log-level DEBUG stats --error-types'.format(tmpdir).split()) + '-d {} --log-level debug stats --error-types'.format(tmpdir).split()) conf = get_config(args) sbws.core.stats.main(args, conf) captured = capsys.readouterr() @@ -158,7 +158,7 @@ def test_stats_fresh_results(time_mock, tmpdir, capsys, caplog): add_two_fresh_results(tmpdir, start-1) p = create_parser() args = p.parse_args( - '-d {} --log-level DEBUG stats --error-types'.format(tmpdir).split()) + '-d {} --log-level debug stats --error-types'.format(tmpdir).split()) conf = get_config(args) sbws.core.stats.main(args, conf) needed_output_lines = [

1 0

[sbws/master] Add changelog entry for logging overhaul
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit ae7f5d8c65a14f6361eddb63907a697ac57e2cb2 Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Wed Jul 4 12:24:33 2018 -0400 Add changelog entry for logging overhaul --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c3e372..a8acfb9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. - Measure relays that have both Exit and BadExit as non-exits, which is how clients would use them. (GH#217) +- Could not init sbws because of a catch-22 related to logging configuration. + Overhaul how logging is configured. (GH#186 GHPR#224) ## [0.5.0] - 2018-06-26

1 0

[sbws/master] Log exception and exit when launching tor fails
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit 2fce34d0adc0bd3704f35339f6b75d8171920b45 Author: juga0 <juga(a)riseup.net> Date: Tue Jul 3 08:56:50 2018 +0000 Log exception and exit when launching tor fails --- sbws/util/stem.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sbws/util/stem.py b/sbws/util/stem.py index 54779fd..7224c10 100644 --- a/sbws/util/stem.py +++ b/sbws/util/stem.py @@ -181,8 +181,11 @@ def launch_tor(conf): existing_val.append(value) torrc.update({key: existing_val}) # Finally launch Tor - stem.process.launch_tor_with_config( - torrc, init_msg_handler=log.debug, take_ownership=True) + try: + stem.process.launch_tor_with_config( + torrc, init_msg_handler=log.debug, take_ownership=True) + except Exception as e: + fail_hard('Error trying to launch tor: %s', e) # And return a controller to it cont = _init_controller_socket(conf['tor']['control_socket']) # Because we build things by hand and can't set these before Tor bootstraps

1 0

[sbws/master] Remove is_controller_ok and log exceptions
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit b5d3fb4fc8d1d2edb9c48a7a6d9996f7b37912b9 Author: juga0 <juga(a)riseup.net> Date: Tue Jul 3 08:26:40 2018 +0000 Remove is_controller_ok and log exceptions --- sbws/core/scanner.py | 1 - sbws/lib/circuitbuilder.py | 32 ++++++++++++--------- sbws/lib/relaylist.py | 21 +++++++++----- sbws/util/stem.py | 57 +++++++++++++++++++++---------------- tests/integration/util/test_stem.py | 1 - 5 files changed, 66 insertions(+), 46 deletions(-) diff --git a/sbws/core/scanner.py b/sbws/core/scanner.py index 249d8d4..5bcf276 100644 --- a/sbws/core/scanner.py +++ b/sbws/core/scanner.py @@ -347,7 +347,6 @@ def run_speedtest(args, conf): 'exists for sbws developers. It is expected to be broken and may ' 'even lead to messed up results.', conf['tor']['control_socket']) time.sleep(15) - assert stem_utils.is_controller_okay(controller) rl = RelayList(args, conf, controller) cb = CB(args, conf, controller, rl) rd = ResultDump(args, conf, end_event) diff --git a/sbws/lib/circuitbuilder.py b/sbws/lib/circuitbuilder.py index d55b929..b8d4db7 100644 --- a/sbws/lib/circuitbuilder.py +++ b/sbws/lib/circuitbuilder.py @@ -1,7 +1,6 @@ from stem import CircuitExtensionFailed, InvalidRequest, ProtocolError, Timeout from stem import InvalidArguments import random -import sbws.util.stem as stem_utils from .relaylist import Relay import logging @@ -59,28 +58,30 @@ class CircuitBuilder: def get_circuit_path(self, circ_id): c = self.controller - assert stem_utils.is_controller_okay(c) - circ = c.get_circuit(circ_id, default=None) - if circ is None: - return None - return [relay[0] for relay in circ.path] + try: + circ = c.get_circuit(circ_id, default=None) + except Exception as e: + log.exception("Exception trying to get circuit: %s.", e) + else: + return [relay[0] for relay in circ.path] + return None def close_circuit(self, circ_id): c = self.controller - if not stem_utils.is_controller_okay(c): - return - if c.get_circuit(circ_id, default=None): + try: + c.get_circuit(circ_id, default=None) try: c.close_circuit(circ_id) except InvalidArguments: pass self.built_circuits.discard(circ_id) + except Exception as e: + log.exception("Error trying to get circuit to close it: %s.", e) def _build_circuit_impl(self, path): if not valid_circuit_length(path): raise PathLengthException() c = self.controller - assert stem_utils.is_controller_okay(c) timeout = self.circuit_timeout fp_path = '[' + ' -> '.join([p[0:8] for p in path]) + ']' log.debug('Building %s', fp_path) @@ -92,6 +93,9 @@ class CircuitBuilder: ProtocolError, Timeout) as e: log.warning(e) continue + except Exception as e: + log.exception("Exception trying to build circuit: %s.", e) + continue else: return circ_id return None @@ -100,14 +104,16 @@ class CircuitBuilder: c = self.controller if not self.close_circuits_on_exit: return - if not stem_utils.is_controller_okay(c): - return for circ_id in self.built_circuits: - if c.get_circuit(circ_id, default=None): + try: + c.get_circuit(circ_id, default=None) try: c.close_circuit(circ_id) except InvalidArguments: pass + except Exception as e: + log.exception("Exception trying to get circuit to delete: %s", + e) self.built_circuits.clear() diff --git a/sbws/lib/relaylist.py b/sbws/lib/relaylist.py index 11390de..bafd7b7 100644 --- a/sbws/lib/relaylist.py +++ b/sbws/lib/relaylist.py @@ -1,6 +1,5 @@ from stem.descriptor.router_status_entry import RouterStatusEntryV3 from stem.descriptor.server_descriptor import ServerDescriptor -import sbws.util.stem as stem_utils from stem import Flag from stem.util.connection import is_valid_ipv4_address from stem.util.connection import is_valid_ipv6_address @@ -25,17 +24,22 @@ class Relay: ''' assert isinstance(fp, str) assert len(fp) == 40 - assert stem_utils.is_controller_okay(cont) if ns is not None: assert isinstance(ns, RouterStatusEntryV3) self._ns = ns else: - self._ns = cont.get_network_status(fp, default=None) + try: + self._ns = cont.get_network_status(fp, default=None) + except Exception as e: + log.exception("Exception trying to get ns %s", e) if desc is not None: assert isinstance(desc, ServerDescriptor) self._desc = desc else: - self._desc = cont.get_server_descriptor(fp, default=None) + try: + self._desc = cont.get_server_descriptor(fp, default=None) + except Exception as e: + log.exception("Exception trying to get ns %s", e) def _from_desc(self, attr): if not self._desc: @@ -190,9 +194,12 @@ class RelayList: def _init_relays(self): c = self._controller - assert stem_utils.is_controller_okay(c) - relays = [Relay(ns.fingerprint, c, ns=ns) - for ns in c.get_network_statuses()] + try: + relays = [Relay(ns.fingerprint, c, ns=ns) + for ns in c.get_network_statuses()] + except Exception as e: + log.exception("Exception trying to init relays %s", e) + return [] return relays def _refresh(self): diff --git a/sbws/util/stem.py b/sbws/util/stem.py index 8a92673..54779fd 100644 --- a/sbws/util/stem.py +++ b/sbws/util/stem.py @@ -17,7 +17,6 @@ stream_building_lock = RLock() def attach_stream_to_circuit_listener(controller, circ_id): ''' Returns a function that should be given to add_event_listener(). It looks for newly created streams and attaches them to the given circ_id ''' - assert is_controller_okay(controller) def closure_stream_event_listener(st): if st.status == 'NEW' and st.purpose == 'USER': @@ -28,21 +27,25 @@ def attach_stream_to_circuit_listener(controller, circ_id): except (UnsatisfiableRequest, InvalidRequest) as e: log.warning('Couldn\'t attach stream to circ %s: %s', circ_id, e) + except Exception as e: + log.exception("Exception trying to get ns %s", e) else: pass return closure_stream_event_listener def add_event_listener(controller, func, event): - assert is_controller_okay(controller) - controller.add_event_listener(func, event) + try: + controller.add_event_listener(func, event) + except Exception as e: + log.exception("Exception trying to add event listener %s", e) def remove_event_listener(controller, func): - if not is_controller_okay(controller): - log.warning('Controller not okay so not trying to remove event') - return - controller.remove_event_listener(func) + try: + controller.remove_event_listener(func) + except Exception as e: + log.exception("Exception trying to remove event %s", e) def init_controller(port=None, path=None, set_custom_stream_settings=True): @@ -72,9 +75,11 @@ def init_controller(port=None, path=None, set_custom_stream_settings=True): def is_bootstrapped(c): - if not is_controller_okay(c): + try: + line = c.get_info('status/bootstrap-phase') + except Exception as e: + log.exception("Exception bootstrapping %s", e) return False - line = c.get_info('status/bootstrap-phase') state, _, progress, *_ = line.split() progress = int(progress.split('=')[1]) if state == 'NOTICE' and progress == 100: @@ -83,12 +88,6 @@ def is_bootstrapped(c): return False -def is_controller_okay(c): - if not c: - return False - return c.is_alive() and c.is_authenticated() - - def _init_controller_port(port): assert isinstance(port, int) try: @@ -106,6 +105,10 @@ def _init_controller_socket(socket): c = Controller.from_socket_file(path=socket) c.authenticate() except (IncorrectSocketType, SocketError): + log.debug("Error initting controller socket: socket error.") + return None + except Exception as e: + log.exception("Error initting controller socket: %s", e) return None # TODO: Allow for auth via more than just CookieAuthentication return c @@ -182,21 +185,26 @@ def launch_tor(conf): torrc, init_msg_handler=log.debug, take_ownership=True) # And return a controller to it cont = _init_controller_socket(conf['tor']['control_socket']) - assert is_controller_okay(cont) # Because we build things by hand and can't set these before Tor bootstraps cont.set_conf('__DisablePredictedCircuits', '1') cont.set_conf('__LeaveStreamsUnattached', '1') - log.info('Started and connected to Tor %s via %s', cont.get_version(), - conf['tor']['control_socket']) - return cont + try: + log.info('Started and connected to Tor %s via %s', cont.get_version(), + conf['tor']['control_socket']) + return cont + except Exception as e: + log.exception("Exception trying to launch tor %s", e) def get_socks_info(controller): ''' Returns the first SocksPort Tor is configured to listen on, in the form of an (address, port) tuple ''' - assert is_controller_okay(controller) - socks_ports = controller.get_listeners(Listener.SOCKS) - return socks_ports[0] + try: + socks_ports = controller.get_listeners(Listener.SOCKS) + return socks_ports[0] + except Exception as e: + log.exception("Exception trying to get socks info: %e.", e) + exit(1) def only_relays_with_bandwidth(controller, relays, min_bw=None, max_bw=None): @@ -206,7 +214,6 @@ def only_relays_with_bandwidth(controller, relays, min_bw=None, max_bw=None): min_bw nor max_bw are given, essentially just returns the input list of relays. ''' - assert is_controller_okay(controller) assert min_bw is None or min_bw >= 0 assert max_bw is None or max_bw >= 0 ret = [] @@ -221,7 +228,6 @@ def only_relays_with_bandwidth(controller, relays, min_bw=None, max_bw=None): def circuit_str(controller, circ_id): - assert is_controller_okay(controller) assert isinstance(circ_id, str) int(circ_id) try: @@ -230,6 +236,9 @@ def circuit_str(controller, circ_id): log.warning('Circuit %s no longer seems to exist so can\'t return ' 'a valid circuit string for it: %s', circ_id, e) return None + except Exception as e: + log.exception("Exception trying to get circuit string %s", e) + return None return '[' +\ ' -> '.join(['{} ({})'.format(n, fp[0:8]) for fp, n in circ.path]) +\ ']' diff --git a/tests/integration/util/test_stem.py b/tests/integration/util/test_stem.py index 203cf5d..291f485 100644 --- a/tests/integration/util/test_stem.py +++ b/tests/integration/util/test_stem.py @@ -3,5 +3,4 @@ import sbws.util.stem as stem_utils def test_launch_and_okay(persistent_launch_tor): cont = persistent_launch_tor - assert stem_utils.is_controller_okay(cont) assert stem_utils.is_bootstrapped(cont)

1 0

[sbws/master] Add test that calls sbws init
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit 43856fdbe3d242396b3c9f6533bd20b9d1c5ec27 Author: Matt Traudt <sirmatt(a)ksu.edu> Date: Wed Jul 4 12:33:13 2018 -0400 Add test that calls sbws init --- tests/integration/test_sbws.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/integration/test_sbws.py b/tests/integration/test_sbws.py new file mode 100644 index 0000000..5122f38 --- /dev/null +++ b/tests/integration/test_sbws.py @@ -0,0 +1,22 @@ +from tempfile import TemporaryFile +import subprocess +import os + + +def test_simple_init(tmpdir): + # out = None + err = None + with TemporaryFile('w+t') as stdout, TemporaryFile('w+t') as stderr: + retcode = subprocess.call( + 'sbws -d {} --log-level debug init'.format(tmpdir).split(), + stdout=stdout, stderr=stderr) + stdout.seek(0, 0) + stderr.seek(0, 0) + # out = stdout.read() + err = stderr.read() + assert retcode == 0 + assert len(err) == 0 + conf_fname = os.path.join(str(tmpdir), 'config.ini') + assert os.path.exists(conf_fname) + with open(conf_fname, 'rt') as fd: + assert len(fd.read()) > 0

1 0

[sbws/master] Add changelog for is_controller_ok changes
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit bfea21d8e5aeffc0157bea667026bdc9d96dff35 Author: juga0 <juga(a)riseup.net> Date: Tue Jul 3 08:56:08 2018 +0000 Add changelog for is_controller_ok changes --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a8acfb9..360d76a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,6 +64,8 @@ generated ones are kept. A `latest.v3bw` symlink is updated. (GH#179 GHPR#190) - Code refactoring in the v3bw classes and generation area - Replace v3bw-into-xy bash script with python script to handle a more complex v3bw file format (GH#182) +- Remove is_controller_ok. Instead catch possible controller exceptions and + log them ## [0.4.1] - 2018-06-14

1 0

[sbws/master] Move is_controller_ok changelog entry to unreleased
by pastly＠torproject.org 11 Jul '18

11 Jul '18

commit 2cf477274125a18b6a6b1140b51e4e9074dbfea8 Author: juga0 <juga(a)riseup.net> Date: Tue Jul 3 17:28:39 2018 +0000 Move is_controller_ok changelog entry to unreleased --- CHANGELOG.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 360d76a..feaff96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. - Could not init sbws because of a catch-22 related to logging configuration. Overhaul how logging is configured. (GH#186 GHPR#224) +### Changed + +- Remove is_controller_ok. Instead catch possible controller exceptions and +log them + ## [0.5.0] - 2018-06-26 @@ -64,8 +69,6 @@ generated ones are kept. A `latest.v3bw` symlink is updated. (GH#179 GHPR#190) - Code refactoring in the v3bw classes and generation area - Replace v3bw-into-xy bash script with python script to handle a more complex v3bw file format (GH#182) -- Remove is_controller_ok. Instead catch possible controller exceptions and - log them ## [0.4.1] - 2018-06-14

1 0