May 2013 - tor-commits - lists.torproject.org

[oonib/master] Update TravisCI config with unattended SSL cert generation commands.
by isis＠torproject.org 04 May '13

04 May '13

commit 5808bfd2503eab85dc2fa0899b41feda0015fdef Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 03:18:35 2013 +0000 Update TravisCI config with unattended SSL cert generation commands. --- .travis.yml | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index e534630..dd07dcc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -16,8 +16,10 @@ install: script: - cp oonib.conf.example oonib.conf - openssl genrsa -des3 -passout pass:oonib -out private.key 4096 - - openssl req -new -key private.key -passin pass:oonib -out server.csr + - openssl req -batch -passin pass:oonib -new -key private.key -out server.csr - cp private.key private.key.org - - openssl rsa -in private.key.org private.key - - openssl x509 -passin pass:oonib -req -days365 -in server.csr -signkey private.key -out certificate.crt + - openssl rsa -passin pass:oonib -in private.key.org private.key + - openssl x509 -req -days 365 -in server.csr -signkey private.key -out certificate.crt + - test -f "private.key.org" && rm -f private.key.org + - test -f "server.csr" && rm -f server.csr - bin/oonib

1 0

[oonib/master] Add openssl self-signed cert generation commands to .travis.yml.
by isis＠torproject.org 04 May '13

04 May '13

commit 9dbedbe5a5fad2696c939628f00524613e744417 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Apr 20 01:38:00 2013 +0000 Add openssl self-signed cert generation commands to .travis.yml. --- .travis.yml | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/.travis.yml b/.travis.yml index c34fb86..e534630 100644 --- a/.travis.yml +++ b/.travis.yml @@ -15,4 +15,9 @@ install: # command to run tests, e.g. python setup.py test script: - cp oonib.conf.example oonib.conf + - openssl genrsa -des3 -passout pass:oonib -out private.key 4096 + - openssl req -new -key private.key -passin pass:oonib -out server.csr + - cp private.key private.key.org + - openssl rsa -in private.key.org private.key + - openssl x509 -passin pass:oonib -req -days365 -in server.csr -signkey private.key -out certificate.crt - bin/oonib

1 0

[oonib/master] Add pyOpenSSL dependency back into requirements.txt.
by isis＠torproject.org 04 May '13

04 May '13

commit b87b8f2eae0bc209293cbaa20c19d2f8edda56ff Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Apr 20 00:59:52 2013 +0000 Add pyOpenSSL dependency back into requirements.txt. --- requirements.txt | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/requirements.txt b/requirements.txt index 00aac7b..fd6fb4b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -18,6 +18,7 @@ transaction>=1.4.1 storm>=0.19 #https://pypi.python.org/packages/source/s/storm/storm-0.19.tar.gz#md5=61d1ee4cd2a08639ab917e43fa2c9265#egg=storm txtorcon>=0.7 +pyOpenSSL>=0.13 zope.component>=4.0.0 zope.event>=4.0.0 zope.interface>=4.0.1

1 0

[oonib/master] Remove CI tests for python3.x, add openssl to requirements.
by isis＠torproject.org 04 May '13

04 May '13

commit 58d6b479926b969f15d35e724dfc7aaceb9786c7 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Apr 20 00:50:41 2013 +0000 Remove CI tests for python3.x, add openssl to requirements. --- .travis.yml | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 733c2af..c34fb86 100644 --- a/.travis.yml +++ b/.travis.yml @@ -4,13 +4,13 @@ before_install: - gpg --keyserver keys.gnupg.net --recv 886DDD89 - gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - - sudo apt-get update - - sudo apt-get install deb.torproject.org-keyring tor tor-geoipdb build-essential python-dev python-setuptools + - sudo apt-get install deb.torproject.org-keyring tor tor-geoipdb build-essential python-dev python-setuptools openssl python: - - "3.3" + #- "3.3" - "2.7" - "2.6" install: - - pip install --use-mirrors -r requirements.txt + - pip install -r requirements.txt --use-mirrors - python setup.py install # command to run tests, e.g. python setup.py test script:

1 0

[oonib/master] Add missing OpenSSL cert generation parameter to .travis.yml.
by isis＠torproject.org 04 May '13

04 May '13

commit 059cb07e2e6bf0c945616cd6f2d5b05a99b9315e Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 03:26:30 2013 +0000 Add missing OpenSSL cert generation parameter to .travis.yml. * (oonib)∃!isisⒶwintermute:~/code/torproject/ooni-backend ∴ date Sun Apr 21 03:18:17 UTC 2013 (oonib)∃!isisⒶwintermute:~/code/torproject/ooni-backend ∴ travis-lint Hooray, /home/isis/code/torproject/ooni-backend/.travis.yml seems to be solid! --- .travis.yml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/.travis.yml b/.travis.yml index dd07dcc..adc1117 100644 --- a/.travis.yml +++ b/.travis.yml @@ -18,7 +18,7 @@ script: - openssl genrsa -des3 -passout pass:oonib -out private.key 4096 - openssl req -batch -passin pass:oonib -new -key private.key -out server.csr - cp private.key private.key.org - - openssl rsa -passin pass:oonib -in private.key.org private.key + - openssl rsa -passin pass:oonib -in private.key.org -out private.key - openssl x509 -req -days 365 -in server.csr -signkey private.key -out certificate.crt - test -f "private.key.org" && rm -f private.key.org - test -f "server.csr" && rm -f server.csr

1 0

[oonib/master] Don't override the pip PI URI for TravisCI; use alternate ".requirements.travis".
by isis＠torproject.org 04 May '13

04 May '13

commit 74cee6ab8ccf9081817f136f442fe1ff451776e4 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 03:30:09 2013 +0000 Don't override the pip PI URI for TravisCI; use alternate ".requirements.travis". * (oonib)∃!isisⒶwintermute:~/code/torproject/ooni-backend ∴ date Sun Apr 21 03:29:49 UTC 2013 (oonib)∃!isisⒶwintermute:~/code/torproject/ooni-backend ∴ travis-lint Hooray, /home/isis/code/torproject/ooni-backend/.travis.yml seems to be solid! --- .requirements.travis | 25 +++++++++++++++++++++++++ .travis.yml | 2 +- 2 files changed, 26 insertions(+), 1 deletions(-) diff --git a/.requirements.travis b/.requirements.travis new file mode 100644 index 0000000..e2929a6 --- /dev/null +++ b/.requirements.travis @@ -0,0 +1,25 @@ +## we do not want to override the PI URI for TravisCI: +#-i https://pypi.python.org/packages +PyYAML>=3.10 +#https://pypi.python.org/packages/source/P/PyYAML/PyYAML-3.10.tar.gz#md5=74c94a383886519e9e7b3dd1ee540247#egg=PyYAML +Twisted>=12.2.0 +#https://pypi.python.org/packages/source/T/Twisted/Twisted-13.0.0.tar.bz2#md5=68afff4e1efd3757d934e39f70c99f57#egg=Twisted +cyclone>=1.1 +#https://pypi.python.org/packages/source/c/cyclone/cyclone-1.1.tar.gz#md5=477c5ef9cf8902e37105fe450c4d8c5d#egg=cyclone +ipaddr>=2.1.10 +#https://ipaddr-py.googlecode.com/files/ipaddr-2.1.10.tar.gz#sha1=c608450b077b19773d4f1b5f1ef88b26f6650ce0#egg=ipaddr +pygeoip>=0.2.6 +#https://pypi.python.org/packages/source/p/pygeoip/pygeoip-0.2.6.zip#md5=b3ac1bfcd535782bc59af78e722cf5c1#egg=pygeoip +# Originally fetched from the hg repo on secdev.org: +# https://hg.secdev.org/scapy/archive/tip.zip#egg=scapy +# This is a Tor Project mirror with valid SSL/TLS certs that is stable and fast: +https://people.torproject.org/~ioerror/src/mirrors/ooniprobe/scapy-02-25-2013-tip.zip#egg=scapy +transaction>=1.4.1 +#https://pypi.python.org/packages/source/t/transaction/transaction-1.4.1.zip#md5=8db2680bc0f999219861a67b8f335a88#egg=transaction +storm>=0.19 +#https://pypi.python.org/packages/source/s/storm/storm-0.19.tar.gz#md5=61d1ee4cd2a08639ab917e43fa2c9265#egg=storm +txtorcon>=0.7 +pyOpenSSL>=0.13 +zope.component>=4.0.0 +zope.event>=4.0.0 +zope.interface>=4.0.1 diff --git a/.travis.yml b/.travis.yml index adc1117..f347600 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,7 +10,7 @@ python: - "2.7" - "2.6" install: - - pip install -r requirements.txt --use-mirrors + - pip install -r .requirements.travis --use-mirrors - python setup.py install # command to run tests, e.g. python setup.py test script:

1 0

[oonib/master] Add file headers and cleanup module docstring in oonib/runner.py.
by isis＠torproject.org 04 May '13

04 May '13

commit e77aad66039aa5dfc167f507ffb28e00a48413d1 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 05:53:45 2013 +0000 Add file headers and cleanup module docstring in oonib/runner.py. --- oonib/runner.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/oonib/runner.py b/oonib/runner.py index b5bc5dc..e7b3db9 100644 --- a/oonib/runner.py +++ b/oonib/runner.py @@ -1,7 +1,9 @@ +# -*- encoding: utf-8 -*- +# +# :authors: Arturo Filastò, Isis Lovecruft +# :licence: see LICENSE for details """ In here we define a runner for the oonib backend system. -We are just extending the - """ from twisted.internet import reactor

1 0

[oonib/master] Add logfile as a config option and fix several bugs in the log.py.
by isis＠torproject.org 04 May '13

04 May '13

commit 6bfbf1d788377126eb7505ee05864eb7fa58ba66 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 05:40:14 2013 +0000 Add logfile as a config option and fix several bugs in the log.py. * Fixed bugs due to several log functions attempting to use "config.advanced.debug" and "config.basic.logfile". These are both located under the config.main section. * Add LICENSE header and encoding specification to log.py. (Yes, we need these.) * Add log.warn() function back in. --- oonib.conf.example | 1 + oonib/__init__.py | 4 ++-- oonib/log.py | 21 ++++++++++++++++----- 3 files changed, 19 insertions(+), 7 deletions(-) diff --git a/oonib.conf.example b/oonib.conf.example index fb04f19..a4cb707 100644 --- a/oonib.conf.example +++ b/oonib.conf.example @@ -1,5 +1,6 @@ main: report_dir: Null + logfile: Null tor_datadir: Null database_uri: 'sqlite://oonib_test_db.db' db_threadpool_size: 10 diff --git a/oonib/__init__.py b/oonib/__init__.py index 30ebd69..8ddad00 100644 --- a/oonib/__init__.py +++ b/oonib/__init__.py @@ -1,7 +1,7 @@ # -*- encoding: utf-8 -*- # -# :authors: Arturo Filastò -# :licence: see LICENSE +# :authors: Arturo Filastò, Isis Lovecruft +# :licence: see LICENSE for details """ In here we shall keep track of all variables and objects that should be instantiated only once and be common to pieces of GLBackend code. diff --git a/oonib/log.py b/oonib/log.py index 64d82f1..14b203d 100644 --- a/oonib/log.py +++ b/oonib/log.py @@ -1,3 +1,11 @@ +# -*- encoding: utf-8 -*- +# +# :authors: Arturo Filastò, Isis Lovecruft +# :licence: see LICENSE for details +""" +Twisted logger for the ooni backend system. +""" + import sys import os import logging @@ -28,16 +36,16 @@ def start(logfile=None, application_name="oonib"): daily_logfile = None if not logfile: - logfile = config.basic.logfile + logfile = config.main.logfile log_folder = os.path.dirname(logfile) log_filename = os.path.basename(logfile) daily_logfile = DailyLogFile(log_filename, log_folder) - txlog.msg("Starting %s on %s (%s UTC)" % (application_name, otime.prettyDateNow(), - otime.utcPrettyDateNow())) - + txlog.msg("Starting %s on %s (%s UTC)" % (application_name, + otime.prettyDateNow(), + otime.utcPrettyDateNow())) txlog.startLoggingWithObserver(LogWithNoPrefix(sys.stdout).emit) txlog.addObserver(txlog.FileLogObserver(daily_logfile).emit) @@ -48,9 +56,12 @@ def msg(msg, *arg, **kw): print "%s" % msg def debug(msg, *arg, **kw): - if config.advanced.debug: + if config.main.debug: print "[D] %s" % msg +def warn(msg, *arg, **kw): + print "[W] %s" % msg + def err(msg, *arg, **kw): print "[!] %s" % msg

1 0

[oonib/master] Fix the "got non-string for process arg" errors resulting from blank config lines.
by isis＠torproject.org 04 May '13

04 May '13

commit 66aa12e086e2da80ea29ccf8a7d0c79582f8f057 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 05:37:25 2013 +0000 Fix the "got non-string for process arg" errors resulting from blank config lines. --- oonib.conf.example | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/oonib.conf.example b/oonib.conf.example index 3afdd38..fb04f19 100644 --- a/oonib.conf.example +++ b/oonib.conf.example @@ -1,9 +1,10 @@ main: - report_dir: - tor_datadir: + report_dir: Null + tor_datadir: Null database_uri: 'sqlite://oonib_test_db.db' db_threadpool_size: 10 - tor_binary: '/usr/bin/tor' + tor_binary: Null + socks_port: 9055 tor2webmode: false pidfile: 'oonib.pid' nodaemon: true

1 0

[oonib/master] Cleanup log/print statements and docstrings in oonib/runner.py.
by isis＠torproject.org 04 May '13

04 May '13

commit 568d05b4276d6a9cd0d0f4a8e93f4b84022f65f5 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 21 05:56:12 2013 +0000 Cleanup log/print statements and docstrings in oonib/runner.py. --- oonib/runner.py | 19 ++++++++++--------- 1 files changed, 10 insertions(+), 9 deletions(-) diff --git a/oonib/runner.py b/oonib/runner.py index e7b3db9..4b2ebb5 100644 --- a/oonib/runner.py +++ b/oonib/runner.py @@ -6,6 +6,8 @@ In here we define a runner for the oonib backend system. """ +from __future__ import print_function + from twisted.internet import reactor from twisted.application import service, internet, app from twisted.python.runtime import platformType @@ -24,7 +26,8 @@ def txSetupFailed(failure): def setupCollector(tor_process_protocol): def setup_complete(port): - print "Exposed collector Tor hidden service on httpo://%s" % port.onion_uri + print("Exposed collector Tor hidden service on httpo://%s" + % port.onion_uri) torconfig = txtorcon.TorConfig(tor_process_protocol.tor_protocol) public_port = 80 @@ -39,7 +42,7 @@ def setupCollector(tor_process_protocol): def startTor(): def updates(prog, tag, summary): - print "%d%%: %s" % (prog, summary) + print("%d%%: %s" % (prog, summary)) torconfig = txtorcon.TorConfig() torconfig.SocksPort = 9055 @@ -60,16 +63,16 @@ if platformType == "win32": from twisted.scripts._twistw import WindowsApplicationRunner OBaseRunner = WindowsApplicationRunner - # XXX Current we don't support windows for the starting of Tor Hidden Service + # XXX Currently we don't support windows for starting a Tor Hidden Service + log.warn( + "Apologies! We don't support starting a Tor Hidden Service on Windows.") else: from twisted.scripts._twistd_unix import UnixApplicationRunner class OBaseRunner(UnixApplicationRunner): def postApplication(self): - """ - To be called after the application is created: start the - application and run the reactor. After the reactor stops, - clean up PID files and such. + """After the application is created, start the application and run + the reactor. After the reactor stops, clean up PID files and such. """ self.startApplication(self.application) # This is our addition. The rest is taken from @@ -82,5 +85,3 @@ else: return oonibackend.application OBaseRunner.loggerFactory = log.LoggerFactory - -

1 0