September 2012 - tor-commits - lists.torproject.org

[tor-cloud/master] Use apt-get and not aptitude
by runa＠torproject.org 15 Sep '12

15 Sep '12

commit b7d62fefc67178dc3beaa31f5c6d05c62898db3e Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Sat Sep 15 22:27:36 2012 +0100 Use apt-get and not aptitude --- rc.local | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/rc.local b/rc.local index a2af486..8593f96 100644 --- a/rc.local +++ b/rc.local @@ -1,4 +1,5 @@ #!/bin/sh -e -sudo aptitude -y safe-upgrade +sudo apt-get -y update +sudo apt-get -y upgrade sudo /etc/ec2-prep.sh type exit 0

1 0

[tor-cloud/master] Use upgrade and not safe-upgrade with apt-get
by runa＠torproject.org 15 Sep '12

15 Sep '12

commit 7d4e7b3383ba6e3a1f2cc8842d549d3c9bdc6d75 Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Sat Sep 15 22:27:54 2012 +0100 Use upgrade and not safe-upgrade with apt-get --- ec2-prep.sh | 8 +------- 1 files changed, 1 insertions(+), 7 deletions(-) diff --git a/ec2-prep.sh b/ec2-prep.sh index a32c148..9236527 100644 --- a/ec2-prep.sh +++ b/ec2-prep.sh @@ -25,7 +25,7 @@ fi # Get the latest package updates echo "Updating the system..." apt-get update -apt-get -y safe-upgrade +apt-get -y upgrade # Configure unattended-upgrades. The system will automatically download, # install and configure all packages, and reboot if necessary. @@ -267,9 +267,3 @@ echo "Done configuring the system, will reboot" echo "Your system has been configured for blocking diagnostics" > /etc/ec2-prep.sh reboot fi - -# XXX TODO -# Generally, we'll want to rm /var/lib/tor/* and remove all state from the system -# -# We're done; tell the user and then reboot the system -#reboot

1 0

[tor-cloud/master] Use the correct device label when creating and mounting
by runa＠torproject.org 15 Sep '12

15 Sep '12

commit b72e124ea7d40c02135cb4c9ea8762a609dd2a93 Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Sat Sep 15 22:26:21 2012 +0100 Use the correct device label when creating and mounting --- build.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/build.sh b/build.sh index 2534cde..cf4367a 100755 --- a/build.sh +++ b/build.sh @@ -96,7 +96,7 @@ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ub # Continue the build process echo "Verified the signature, continue with the build process" -ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "sudo chown ubuntu:ubuntu /mnt && cd /mnt && tar -Sxvzf /mnt/ubuntu-12.04-server-cloudimg-i386.tar.gz && sudo mkdir /mnt/src /mnt/target && sudo mount -o loop,rw /mnt/precise-server-cloudimg-i386.img /mnt/src && sudo mkfs.ext4 -F -L cloudimg-rootfs /dev/sdh && sudo mount /dev/sdh /mnt/target" +ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "sudo chown ubuntu:ubuntu /mnt && cd /mnt && tar -Sxvzf /mnt/ubuntu-12.04-server-cloudimg-i386.tar.gz && sudo mkdir /mnt/src /mnt/target && sudo mount -o lo op,rw /mnt/precise-server-cloudimg-i386.img /mnt/src && sudo mkfs.ext4 -F -L cloudimg-rootfs /dev/xvdh && sudo mount /dev/xvdh /mnt/target" # this is our startup file that loads tor-prep.sh on first boot ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -v -t "sudo wget https://gitweb.torproject.org/tor-cloud.git/blob_plain/HEAD:/rc.local -O /mnt/src/etc/rc.local"

1 0

[tor/master] fold in changes files so far
by arma＠torproject.org 15 Sep '12

15 Sep '12

commit 2e342df7cdde5236ce2bdf2db457d193bdbffa5e Author: Roger Dingledine <arma(a)torproject.org> Date: Sat Sep 15 16:38:47 2012 -0400 fold in changes files so far --- ChangeLog | 57 +++++++++++++++++++++++++++++++++++++++++++++ changes/bug6341 | 5 ---- changes/bug6538 | 16 ------------ changes/bug6811 | 5 ---- changes/bug6815 | 6 ---- changes/bug6819 | 3 -- changes/bug6827 | 9 ------- changes/bug6831 | 4 --- changes/bug6833 | 4 --- changes/bug6844 | 4 --- changes/old_gcc_macro_fix | 5 ---- changes/refactor_config | 3 -- 12 files changed, 57 insertions(+), 64 deletions(-) diff --git a/ChangeLog b/ChangeLog index 495ddf2..c517207 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,60 @@ +Changes in version 0.2.4.3-alpha - 2012-09-?? + o Security fixes: + - Fix an assertion failure in tor_timegm() that could be triggered + by a badly formatted directory object. Bug found by fuzzing with + Radamsa. Fixes bug 6811; bugfix on 0.2.0.20-rc. + + o Major bugfixes: + - Fix a possible crash bug when checking for deactivated circuits + in connection_or_flush_from_first_active_circuit(). Fixes bug 6341; + bugfix on 0.2.2.7-alpha. Bug report and fix received pseudonymously. + - Allow routers to detect that their own DirPorts are running. When + we removed support for versions_supports_begindir, we also + accidentally removed the mechanism we used to self-test our + DirPort. Diagnosed with help from kargig. Fixes bugs 6814 and 6815; + bugfix on 0.2.4.2-alpha. + + o Security features: + - Switch to a completely time-invariant approach for picking nodes + weighted by bandwidth. Our old approach would run through the + part of the loop after it had made its choice slightly slower + than it ran through the part of the loop before it had made its + choice. Addresses ticket 6538. + + o Minor bugfixes: + - Correct file sizes when reading binary files on Cygwin, to avoid + a bug where Tor would fail to read its state file. Fixes bug 6844; + bugfix on 0.1.2.7-alpha. + - Fix round_to_power_of_2() so it doesn't invoke undefined behavior + with large values. This situation was untriggered, but nevertheless + incorrect. Fixes bug 6831; bugfix on 0.2.0.1-alpha. + - Reject consensus votes with more than 64 known-flags. We aren't even + close to that limit yet, and our code doesn't handle it correctly. + Fixes bug 6833; bugfix on 0.2.0.1-alpha. + - Avoid undefined behaviour when parsing the list of supported + rendezvous/introduction protocols in a hidden service descriptor. + Previously, Tor would have confused (as-yet-unused) protocol version + numbers greater than 32 with lower ones on many platforms. Fixes + bug 6827; bugfix on 0.2.0.10-alpha. Found by George Kadianakis. + - Fix building with older versions of GCC that don't like preprocessor + directives inside macro arguments. Found by grarpamp. Fixes bug + 6842; bugfix on 0.2.4.2-alpha. + - Switch weighted node selection rule from using a list of doubles + to using a list of int64_t. This change should make the process + slightly easier to debug and maintain. Needed to finish ticket 6538. + + o Code simplification and refactoring: + - Move the generic "config" code into a new file, and have "config.c" + hold only torrc- and state-related code. Resolves ticket 6823. + - Move the core of our "choose a weighted element at random" logic + into its own function, and give it unit tests. Now the logic is + testable, and a little less fragile too. + + o Documentation fixes: + - Resolve a typo in torrc.sample.in. Fixes bug 6819; bugfix on + 0.2.3.14-alpha. + + Changes in version 0.2.3.22-rc - 2012-09-11 Tor 0.2.3.22-rc fixes another opportunity for a remotely triggerable assertion. diff --git a/changes/bug6341 b/changes/bug6341 deleted file mode 100644 index 04e52c7..0000000 --- a/changes/bug6341 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Fix a possible crash bug when checking for deactivated circuits - in connection_or_flush_from_first_active_circuit(). Fixes bug - 6341; bugfix on 0.2.2.7-alpha. Bug report and fix received - pseudonymously. diff --git a/changes/bug6538 b/changes/bug6538 deleted file mode 100644 index 03c168b..0000000 --- a/changes/bug6538 +++ /dev/null @@ -1,16 +0,0 @@ - o Minor bugfixes: - - Switch weighted node selection rule from using a list of doubles - to using a list of int64_t. This should make the process slightly - easier to debug and maintain. Needed for fix for bug 6538. - - o Security features: - - Switch to a completely time-invariant approach for picking nodes - weighted by bandwidth. Our old approach would run through the - part of the loop after it had made its choice slightly slower - than it ran through the part of the loop before it had made its - choice. Fix for bug 6538. - - o Code simplifications and refactoring: - - Move the core of our "choose a weighted element at random" logic - into its own function, and give it unit tests. Now the logic is - testable, and a little less fragile too. diff --git a/changes/bug6811 b/changes/bug6811 deleted file mode 100644 index 841ec1c..0000000 --- a/changes/bug6811 +++ /dev/null @@ -1,5 +0,0 @@ - o Major security fixes: - - Fix an assertion failure in tor_timegm that could be triggered - by a badly formatted directory object. Bug found by fuzzing with - Radamsa. Fixes bug 6811; bugfix on 0.2.0.20-rc. - diff --git a/changes/bug6815 b/changes/bug6815 deleted file mode 100644 index d6a1233..0000000 --- a/changes/bug6815 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Allow routers to correctly detect their own DirPorts as running. - When we removed support for versions_supports_begindir, we also - accidentally removed the mechanism we used to self-test our - DirPort. Diagnosed with help from kargig. Fixes bugs 6814 and - 6815; bugfix on 0.2.4.2-alpha. diff --git a/changes/bug6819 b/changes/bug6819 deleted file mode 100644 index 8b5c224..0000000 --- a/changes/bug6819 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation fixes: - - Resolve a typo in torrc.sample.in. Fixes bug 6819; bugfix on - 0.2.3.14-alpha. diff --git a/changes/bug6827 b/changes/bug6827 deleted file mode 100644 index bf71d2b..0000000 --- a/changes/bug6827 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes: - - - Avoid undefined behaviour when parsing the list of supported - rendezvous/introduction protocols in a hidden service - descriptor. Previously, Tor would have confused (as-yet-unused) - protocol version numbers greater than 32 with lower ones on many - platforms. Fixes bug 6827; bugfix on 0.2.0.10-alpha; found by - George Kadianakis. - diff --git a/changes/bug6831 b/changes/bug6831 deleted file mode 100644 index ac4775b..0000000 --- a/changes/bug6831 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Fix round_to_power_of_2 so it doesn't invoke undefined behavior - with large values. This was untriggered, but nevertheless incorrect. - Fixes bug 6831; bugfix on 0.2.0.1-alpha. diff --git a/changes/bug6833 b/changes/bug6833 deleted file mode 100644 index 4a6a5d3..0000000 --- a/changes/bug6833 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (directory authority): - - Reject consensus votes with more than 64 known-flags. We aren't even - close to that limit yet, and our code doesn't handle it - correctly. Fixes bug 6833; bugfix on 0.2.0.1-alpha. diff --git a/changes/bug6844 b/changes/bug6844 deleted file mode 100644 index 338e19d..0000000 --- a/changes/bug6844 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Correct file sizes when reading binary files on - Cygwin, to avoid a bug where Tor would fail to read its state file. - Fixes bug 6844; bugfix on 0.1.2.7-alpha. diff --git a/changes/old_gcc_macro_fix b/changes/old_gcc_macro_fix deleted file mode 100644 index 5fc0a47..0000000 --- a/changes/old_gcc_macro_fix +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Fix building with older versions of GCC that don't like preprocessor - directives inside macro arguments. Found by grarpamp. Fixes bug 6842; - bugfix on 0.2.4.2-alpha. - diff --git a/changes/refactor_config b/changes/refactor_config deleted file mode 100644 index 8d272e7..0000000 --- a/changes/refactor_config +++ /dev/null @@ -1,3 +0,0 @@ - o Code simplification and refactoring: - - Move the generic "config" code into a new file, and have "config.c" - hold only torrc- and state-related code.

1 0

[tor-cloud/master] Close connecting if we cannot fetch the gpg key
by runa＠torproject.org 15 Sep '12

15 Sep '12

commit e6822aabbcd7ffc8e1bd6081a234e99af1e5d3e6 Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Sat Sep 15 21:18:26 2012 +0100 Close connecting if we cannot fetch the gpg key --- build.sh | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/build.sh b/build.sh index e8cfa37..2534cde 100755 --- a/build.sh +++ b/build.sh @@ -88,6 +88,7 @@ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ub # Verify the signature echo "Verify the signature" ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "sudo gpg --keyserver keys.gnupg.net --recv-key 7DB87C81" +ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "if [ `echo $?` -eq "1" ]; then echo 'Could not get key, will lock you out of the instance' ; sudo rm /home/ubuntu/.ssh/authorized_keys ; fi" ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "cd /mnt ; sudo gpg --verify SHA256SUMS.gpg SHA256SUMS" ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "cd /mnt ; sudo sha256sum -c SHA256SUMS 2>&1 | grep OK" ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${sshkey} ubuntu@${host} -q -t "echo $?"

1 0

[ooni-probe/master] * Fixed bug in the assets parser.
by isis＠torproject.org 15 Sep '12

15 Sep '12

commit 21505f84aef5d60c7e138590a1a40e3df773d680 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Sep 15 07:49:29 2012 +0000 * Fixed bug in the assets parser. --- .gitignore | 1 + ooni/assets/bridgetests.txt | 5 ----- ooni/plugins/bridget.py | 13 ++++++++++--- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/.gitignore b/.gitignore index 7f270bb..686e319 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ proxy-lists/italy-http-ips.txt private/* /ooni/plugins/dropin.cache oonib/oonibackend.conf +ooni/lib/txtorcon \ No newline at end of file diff --git a/ooni/assets/bridgetests.txt b/ooni/assets/bridgetests.txt deleted file mode 100644 index 7bba841..0000000 --- a/ooni/assets/bridgetests.txt +++ /dev/null @@ -1,5 +0,0 @@ -#213.151.89.102:9001 -#108.166.106.156:443 -#217.150.224.213:443 -85.193.252.111:443 -#68.98.41.14:9001 diff --git a/ooni/plugins/bridget.py b/ooni/plugins/bridget.py index 745815f..49565d8 100644 --- a/ooni/plugins/bridget.py +++ b/ooni/plugins/bridget.py @@ -200,11 +200,11 @@ class BridgetTest(OONITest): ## we should probably find a more memory nice way to load addresses, ## in case the files are really large if self.local_options: - @staticmethod + def make_asset_list(opt, lst): log.msg("Loading information from %s ..." % opt) with open(opt) as opt_file: - for line in opt_file.readline(): + for line in opt_file.readlines(): if line.startswith('#'): continue else: @@ -389,7 +389,7 @@ class BridgetTest(OONITest): if len(self.bridge_list) >= 1: for bridge in self.bridge_list: try: - print "BRIDGE IS %s" % bridge + log.msg("Current Bridge: %s" % bridge) reconf_controller(self.config, bridge) except: reconf_fail(bridge) @@ -421,6 +421,10 @@ bridget = BridgetTest(None, None, None) ## ----------- ## self.config.save() only needs to be called if Tor is already running. ## +## to test gid, uid, and euid: +## with open('/proc/self/state') as uidfile: +## print uidfile.read(1000) +## ## TODO: ## o add option for any kwarg=arg self.config setting ## o cleanup documentation @@ -428,6 +432,9 @@ bridget = BridgetTest(None, None, None) ## o check if bridges are public relays ## o take bridge_desc file as input, also be able to give same ## format as output +## o Add assychronous timout for deferred, so that we don't wait +## forever for bridges that don't work. +## o Add mechanism for testing through another host ## ## FIX: ## o DataDirectory is not found, or permissions aren't right

1 0

[atlas/master] Make exitpolicies scroll
by karsten＠torproject.org 15 Sep '12

15 Sep '12

commit 3234789e8170ae3f7d27c7463cdf84b9424b33e1 Author: Sathyanarayanan Gunasekaran <gsathya.ceg(a)gmail.com> Date: Sat Sep 15 08:31:37 2012 +0530 Make exitpolicies scroll --- css/bootstrap.css | 6 ++++++ templates/details/main.html | 6 +++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/css/bootstrap.css b/css/bootstrap.css index 250ee39..e60bb20 100644 --- a/css/bootstrap.css +++ b/css/bootstrap.css @@ -492,6 +492,12 @@ pre { pre.prettyprint { margin-bottom: 18px; } + +pre.pre-scrollable { + max-height: 340px; + overflow-y: scroll; +} + pre code { padding: 0; background-color: transparent; diff --git a/templates/details/main.html b/templates/details/main.html index 088b6b8..b8b5421 100644 --- a/templates/details/main.html +++ b/templates/details/main.html @@ -33,15 +33,15 @@ <dt>Exit Policy Summary</dt> <dd> <% if ('accept' in relay.get('exit_policy_summary')) { %> - <pre>accept<% _.each(relay.get('exit_policy_summary')['accept'], + <pre class="pre-scrollable">accept<% _.each(relay.get('exit_policy_summary')['accept'], function(ports) { %> <%= $.trim(ports) %><% });} %></pre> <% if ('reject' in relay.get('exit_policy_summary')) { %> - <pre>reject<% _.each(relay.get('exit_policy_summary')['reject'], + <pre class="pre-scrollable">reject<% _.each(relay.get('exit_policy_summary')['reject'], function(ports) { %> <%= $.trim(ports) %><% });} %></pre> </dd> <dt>Exit Policy</dt> <dd> - <pre><% _.each(relay.get('exit_policy'), function(line) { %> + <pre class="pre-scrollable"><% _.each(relay.get('exit_policy'), function(line) { %> <%= $.trim(line) %><% }); %></pre> </dd> </dl>

1 0

[ooni-probe/master] * Modified exception handling in bridget so that ImportErrors from missing
by isis＠torproject.org 15 Sep '12

15 Sep '12

commit 803e24021cc9a1d48eaa4c24b3e19d39bc8be11b Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Sep 15 03:09:22 2012 +0000 * Modified exception handling in bridget so that ImportErrors from missing OONI libraries don't clutter the logs. * Removed txtorcon directory and modified the Makefile so that git clones dependencies correctly. * Modified class and import structures in bridget so that txtorcon doesn't get imported until used, and ImportErrors and missing libraries give instructions on how to fix any problems. --- ooni/lib/Makefile | 18 ++- ooni/lib/txtorcon | 1 - ooni/plugins/bridget.py | 268 +++++++++++++++++++++++++---------------------- 3 files changed, 153 insertions(+), 134 deletions(-) diff --git a/ooni/lib/Makefile b/ooni/lib/Makefile index 3b0c922..c40b8d2 100644 --- a/ooni/lib/Makefile +++ b/ooni/lib/Makefile @@ -1,4 +1,4 @@ -all: txtorcon txtraceroute +all: txtorcon txtraceroute: echo "Processing dependency txtraceroute..." @@ -14,13 +14,19 @@ txtorcon: clean: rm -rf txtorcon +# rm -rf txtraceroute.py +# rm -rf txscapy.py + +cleanall: + rm -rf txtorcon rm -rf txtraceroute.py + rm -rf txscapy.py -#txscapy: -# echo "Processing dependency txscapy" -# git clone https://github.com/hellais/txscapy.git txscapy.git -# mv txscapy.git/txscapy.py txscapy.py -# rm -rf txscapy.git +txscapy: + echo "Processing dependency txscapy" + git clone https://github.com/hellais/txscapy.git txscapy.git + mv txscapy.git/txscapy.py txscapy.py + rm -rf txscapy.git #rfc3339: # echo "Processing RFC3339 dependency" diff --git a/ooni/lib/txtorcon b/ooni/lib/txtorcon deleted file mode 160000 index def1551..0000000 --- a/ooni/lib/txtorcon +++ /dev/null @@ -1 +0,0 @@ -Subproject commit def155177f4ba6fd0728922e5b6f3c0066006a94 diff --git a/ooni/plugins/bridget.py b/ooni/plugins/bridget.py index 95f7453..be39371 100644 --- a/ooni/plugins/bridget.py +++ b/ooni/plugins/bridget.py @@ -13,33 +13,28 @@ # :version: 0.1.0-alpha from __future__ import with_statement -from zope.interface import implements +from os import getcwd +from os.path import isfile +from os.path import join as pj from twisted.python import usage from twisted.plugin import IPlugin from twisted.internet import defer, error, reactor +from zope.interface import implements import random import sys -try: - from ooni.lib.txtorcon import CircuitListenerMixin, IStreamAttacher -except: - print "BridgeT requires txtorcon: https://github.com/meejah/txtorcon.git" - print "Your copy of OONI should have it included, if you're seeing this" - print "message, please file a bug report." - log.msg ("Bridget: Unable to import from ooni.lib.txtorcon") - from ooni.utils import log from ooni.plugoo.tests import ITest, OONITest from ooni.plugoo.assets import Asset class BridgetArgs(usage.Options): - def portCheck(number): number = int(number) if number not in range(1024, 65535): raise ValueError("Port out of range") + portCheck.coerceDoc = "Ports must be between 1024 and 65535" optParameters = [ @@ -72,93 +67,6 @@ class BridgetArgs(usage.Options): e = "Unable to use random and specific ports simultaneously" raise usageError, e -class CustomCircuit(CircuitListenerMixin): - implements(IStreamAttacher) - - from txtorcon.interface import IRouterContainer, ICircuitContainer - - def __init__(self, state): - self.state = state - self.waiting_circuits = [] - - def waiting_on(self, circuit): - for (circid, d) in self.waiting_circuits: - if circuit.id == circid: - return true - return False - - def circuit_extend(self, circuit, router): - "ICircuitListener" - if circuit.purpose != 'GENERAL': - return - if self.waiting_on(circuit): - log.msg("Circuit %d (%s)" % (circuit.id, router.id_hex)) - - def circuit_built(self, circuit): - "ICircuitListener" - if circuit.purpose != 'GENERAL': - return - - log.msg("Circuit %s built ..." % circuit.id) - log.msg("Full path of %s: %s" % (circuit.id, circuit.path)) - - for (circid, d) in self.waiting_circuits: - if circid == circuit.id: - self.waiting_circuits.remove(circid, d) - d.callback(circuit) - - def circuit_failed(self, circuit, reason): - if self.waiting_on(circuit): - log.msg("A circuit we requested %s failed for reason %s" % - (circuit.id, reason)) - circid, d = None, None - for x in self.waiting_circuits: - if x[0] == circuit.id: - circid, d, stream_cc = x - if d is None: - raise Exception("Expected to find circuit.") - - self.waiting_circuits.remove((circid, d)) - log.msg("Trying to build a circuit for %s" % circid) - self.request_circuit_build(d) - - def check_circuit_route(self, circuit, router): - if router in circuit.path: - #router.update() ## XXX can i use without args? no. - TorInfo.dump(self) - - def request_circuit_build(self, deferred): - entries = self.state.entry_guards.value() - relays = self.state.routers.values() - - log.msg("We have these nodes listed as entry guards:") - log.msg("%s" % entries) - log.msg("We have these nodes listed as relays:") - log.msg("%s" % relays) - - path = [random.choice(entries), - random.choice(relays), - random.choice(relays)] - - log.msg("Requesting a circuit: %s" - % '-->'.join(map(lambda x: x.location.countrycode, path))) - - class AppendWaiting: - def __init__(self, attacher, deferred): - self.attacher = attacher - self.d = deferred - - def __call__(self, circuit): - """ - Return from build_circuit is a Circuit, however, we want to - wait until it is built before we can issue an attach on it and - callback to the Deferred we issue here. - """ - log.msg("Circuit %s is in progress ..." % circuit.id) - self.attacher.waiting_circuits.append((circuit.id, self.d)) - - return self.state.build_circuit(path).addCallback(AppendWaiting(self, deferred_to_callback)).addErrback(log.err) - class BridgetAsset(Asset): """ Class for parsing bridge assets so that they can be commented out. @@ -214,18 +122,25 @@ class BridgetTest(OONITest): self.data_directory = None if self.local_options: - try: - from ooni.lib.txtorcon import TorConfig - except: - e = "Could not import TorConfig class from txtorcon!" - raise ImportError, e - - options = self.local_options - self.config = TorConfig() + options = self.local_options - ## Don't run the experiment if we don't have anything to test if not options['bridges'] and not options['relays']: self.suicide = True + return + + try: + from ooni.lib.txtorcon import TorConfig + except ImportError: + log.msg ("Bridget: Unable to import from ooni.lib.txtorcon") + wd, tx = getcwd(), 'lib/txtorcon/torconfig.py' + chk = pj(wd,tx) if wd.endswith('ooni') else pj(wd,'ooni/'+tx) + try: + assert isfile(chk) + except: + log.msg("Error: Some OONI libraries are missing!") + log.msg("Please go to /ooni/lib/ and do \"make all\"") + + self.config = TorConfig() if options['bridges']: self.config.UseBridges = 1 @@ -284,26 +199,23 @@ class BridgetTest(OONITest): ## we should probably find a more memory nice way to load addresses, ## in case the files are really large if self.local_options: - if self.local_options['bridges']: - log.msg("Loading bridge information from %s ..." - % self.local_options['bridges']) - with open(self.local_options['bridges']) as bridge_file: - for line in bridge_file.readlines(): + @staticmethod + def make_asset_list(opt, lst): + log.msg("Loading information from %s ..." % opt) + with open(opt) as opt_file: + for line in opt_file.readline(): if line.startswith('#'): continue else: - self.bridge_list.append(line.replace('\n','')) - assets.update({'bridges': self.bridge_list}) + lst.append(line.replace('\n','')) + if self.local_options['bridges']: + make_asset_list(self.local_options['bridges'], + self.bridge_list) + assets.update({'bridges': self.bridge_list}) if self.local_options['relays']: - log.msg("Loading relay information from %s ..." - % self.local_options['relays']) - with open(options['relays']) as relay_file: - for line in relay_file.readlines(): - if line.startswith('#'): - continue - else: - self.relay_list.append(line.replace('\n','')) + make_asset_list(self.local_options['relays'], + self.relay_list) assets.update({'relays': self.relay_list}) return assets @@ -321,8 +233,20 @@ class BridgetTest(OONITest): this has a :class:`ooni.lib.txtorcon.torcontol.TorControlProtocol <TorControlProtocol>` instance as .protocol. """ - from ooni.lib.txtorcon import TorProtocolFactory, TorConfig, TorState - from ooni.lib.txtorcon import DEFAULT_VALUE, launch_tor + try: + from ooni.lib.txtorcon import CircuitListenerMixin, IStreamAttacher + from ooni.lib.txtorcon import TorProtocolFactory, TorConfig, TorState + from ooni.lib.txtorcon import DEFAULT_VALUE, launch_tor + except ImportError: + log.msg("Error: Unable to import from ooni.lib.txtorcon") + wd, tx = getcwd(), 'lib/txtorcon/torconfig.py' + chk = pj(wd,tx) if wd.endswith('ooni') else pj(wd,'ooni/'+tx) + try: + assert isfile(chk) + except: + log.msg("Error: Some OONI libraries are missing!") + log.msg(" Please go to /ooni/lib/ and do \"make all\"") + return sys.exit() def bootstrap(ctrl): """ @@ -365,12 +289,102 @@ class BridgetTest(OONITest): log.msg("%d%%: %s" % (prog, summary)) if len(args) == 0: - log.msg("Bridget needs lists of bridges and/or relays to test!") + log.msg("Bridget can't run without bridges or relays to test!") log.msg("Exiting ...") - d = sys.exit() - return d - + return sys.exit() else: + + class CustomCircuit(CircuitListenerMixin): + implements(IStreamAttacher) + + from txtorcon.interface import IRouterContainer + from txtorcon.interface import ICircuitContainer + + def __init__(self, state): + self.state = state + self.waiting_circuits = [] + + def waiting_on(self, circuit): + for (circid, d) in self.waiting_circuits: + if circuit.id == circid: + return true + return False + + def circuit_extend(self, circuit, router): + "ICircuitListener" + if circuit.purpose != 'GENERAL': + return + if self.waiting_on(circuit): + log.msg("Circuit %d (%s)" + % (circuit.id, router.id_hex)) + + def circuit_built(self, circuit): + "ICircuitListener" + if circuit.purpose != 'GENERAL': + return + log.msg("Circuit %s built ..." % circuit.id) + log.msg("Full path of %s: %s" % (circuit.id, circuit.path)) + for (circid, d) in self.waiting_circuits: + if circid == circuit.id: + self.waiting_circuits.remove(circid, d) + d.callback(circuit) + + def circuit_failed(self, circuit, reason): + if self.waiting_on(circuit): + log.msg("A circuit we requested %s failed for reason %s" + % (circuit.id, reason)) + circid, d = None, None + for x in self.waiting_circuits: + if x[0] == circuit.id: + circid, d, stream_cc = x + if d is None: + raise Exception("Expected to find circuit.") + + self.waiting_circuits.remove((circid, d)) + log.msg("Trying to build a circuit for %s" % circid) + self.request_circuit_build(d) + + def check_circuit_route(self, circuit, router): + if router in circuit.path: + #router.update() ## XXX can i use without args? no. + TorInfo.dump(self) + + def request_circuit_build(self, deferred): + entries = self.state.entry_guards.value() + relays = self.state.routers.values() + log.msg("We have these nodes listed as entry guards:") + log.msg("%s" % entries) + log.msg("We have these nodes listed as relays:") + log.msg("%s" % relays) + path = [random.choice(entries), + random.choice(relays), + random.choice(relays)] + log.msg("Requesting a circuit: %s" + % '-->'.join(map(lambda x: x.location.countrycode, + path))) + + class AppendWaiting: + def __init__(self, attacher, deferred): + self.attacher = attacher + self.d = deferred + + def __call__(self, circuit): + """ + Return from build_circuit is a Circuit, however, + we want to wait until it is built before we can + issue an attach on it and callback to the Deferred + we issue here. + """ + log.msg("Circuit %s is in progress ..." % circuit.id) + self.attacher.waiting_circuits.append((circuit.id, + self.d)) + + fin = self.state.build_circuit(path) + fin.addCallback(AppendWaiting(self, deferred_to_callback)) + fin.addErrback(log.err) + return fin + + if len(self.bridge_list) >= 1: for bridge in self.bridge_list: try:

1 0

[ooni-probe/master] * Fixed the help menu in bridget because the portCheck type enforcement
by isis＠torproject.org 15 Sep '12

15 Sep '12

commit ec583eab1515aa545f371f255fffff2f84c4bbb7 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Sep 15 05:23:01 2012 +0000 * Fixed the help menu in bridget because the portCheck type enforcement function wasn't working. * Added documentation at /docs/bridget.md on using bridget. --- docs/bridget.md | 102 +++++++++++++++++++++++++++++++++++++++++++++++ ooni/plugins/bridget.py | 47 +++++++++++----------- 2 files changed, 126 insertions(+), 23 deletions(-) diff --git a/docs/bridget.md b/docs/bridget.md new file mode 100644 index 0000000..05ba65b --- /dev/null +++ b/docs/bridget.md @@ -0,0 +1,102 @@ += ooniprobe Bridget Plugin = + +== Who's that? == + +Bridget is an OONI plugin for testing the reachability of bridges and relays +in the Tor network. + +== Dependencies == + +In addition to the regular OONI dependencies, bridget requires +[https://github.com/meejah/txtorcon.git txtorcon]. If you don't already have +it installed on your system, there is a Makefile in ooni/lib/ which will git +clone it for you, and copy the API portions to ooni/lib/ and discard the +rest. If you're seeing errors saying "Bridget requires txtorcon!" or "Unable +to import libraries from txtorcon!", then you should use the Makefile by +doing: + + $ cd ooni/lib/ && make txtorcon + +== Using Bridget == + +=== Defaults === + +'''Tor Binary Path''' +By default, Bridget will start a new Tor process from the binary at +/usr/sbin/tor. If you have a different Tor binary that you'd like to use, +you can tell Bridget where it is by using the '--torpath' option. + +'''Tor SocksPort and ControlPort''' +Also, Bridget is configured to start Tor with ControlPort=9052 and +SocksPort=9049. If you're running a firewall, you should make sure that +localhost is allowed incoming access to those ports. To use different ports, +see the '--socks' and '--control' options. + +If you're a true Entropist, you can use the '--random' option to randomize +which ports are used for the SocksPort and ControlPort. + +=== Running Tests === + +You'll need some assets first. + +OONI assets are simple text files describing what it is that you want +tested. Normally, one thing-to-be-tested per line, sometimes with comma +separated values. The test you're using should tell you what it wants (if it +doesn't, you should complain to one of the OONI developers). Normally, assets +look like this: + + [file:assets/things.txt] + torproject.org + ooni.nu + eff.org + riseup.net + [...] + +==== Testing Relays with Bridget ==== + +This doesn't work all the way yet. Eventually, you'll need to give bridget +an asset with one relay IP address per line, like this: + + [file:assets/relays.txt] + 123.123.123.123 + 11.22.33.44 + 111.1.111.1 + [...] + +and you'll use it by doing: + + path-to-ooni-probe/ooni/$ ./ooniprobe.py bridget -f assets/relays.txt + +Once the relays feature is functional, it will be possible to test bridges and +relays at the same time: Bridget will try to connect to a bridge, and, if that +worked, it will try to build a circuit from the provided relays, noting which +bridges/relays fail and which ones succeed. This will continue until all the +bridges and relays have been tested. + +==== Testing Bridges with Bridget ==== + +Bridget will need an asset file with bridge IPs and ORPorts, one per +line. Your asset file should look like this: + + [file:assets/bridges.txt] + 11.22.33.44:443 + 2.3.5.7:9001 + 222.111.0.111:443 + [...] + +and you'll call Bridget like this: + + path-to-ooni-probe/ooni/$ ./ooniprobe.py bridget -b assets/bridges.txt + +==== Testing Bridge Pluggable Transports ==== + +Bridget can test whether a bridge with a pluggable transport, like obfsproxy, +is reachable. To reach these bridges, you need to have the pluggable transport +that you want to test installed. You should look at the documentation for that +transport to figure out how to do that. + +To test a pluggable transport, do this (the quotemarks are important): + + $ ./ooniprobe.py bridget -t "/path/to/PT/binary [options]" \ + -b assets/bridges.txt + diff --git a/ooni/plugins/bridget.py b/ooni/plugins/bridget.py index be39371..745815f 100644 --- a/ooni/plugins/bridget.py +++ b/ooni/plugins/bridget.py @@ -29,47 +29,48 @@ from ooni.plugoo.tests import ITest, OONITest from ooni.plugoo.assets import Asset -class BridgetArgs(usage.Options): - def portCheck(number): - number = int(number) - if number not in range(1024, 65535): - raise ValueError("Port out of range") +def portCheck(number): + number = int(number) + if number not in range(1024, 65535): + raise ValueError("Port out of range") + +portCheckAllowed = "must be between 1024 and 65535." +sockCheck, ctrlCheck = portCheck, portCheck +sockCheck.coerceDoc = "Port to use for Tor's SocksPort, " + portCheckAllowed +ctrlCheck.coerceDoc = "Port to use for Tor's ControlPort, " + portCheckAllowed - portCheck.coerceDoc = "Ports must be between 1024 and 65535" +class BridgetArgs(usage.Options): optParameters = [ ['bridges', 'b', None, - 'List of bridges to scan <IP>:<ORport>'], + 'File listing bridge IP:ORPorts to test'], ['relays', 'f', None, - 'List of relays to scan <IP>'], - ['socks', 's', 9049, portCheck, - 'Tor SocksPort to use'], - ['control', 'c', 9052, portCheck, - 'Tor ControlPort to use'], - ['tor-path', 'p', None, + 'File listing relay IPs to test'], + ['socks', 's', 9049, None, portCheck], + ['control', 'c', 9052, None, portCheck], + ['torpath', 'p', None, 'Path to the Tor binary to use'], - ['data-dir', 'd', None, + ['datadir', 'd', None, 'Tor DataDirectory to use'], ['transport', 't', None, 'Tor ClientTransportPlugin'], ['resume', 'r', 0, 'Resume at this index']] - optFlags = [['random', 'x', 'Randomize control and socks ports']] + optFlags = [ + ['random', 'x', 'Use random ControlPort and SocksPort']] def postOptions(self): - ## We can't test pluggable transports without bridges if self['transport'] and not self['bridges']: e = "Pluggable transport requires the bridges option" raise usage.UsageError, e - ## We can't use random and static port simultaneously if self['socks'] and self['control']: if self['random']: e = "Unable to use random and specific ports simultaneously" - raise usageError, e + raise usage.usageError, e class BridgetAsset(Asset): """ - Class for parsing bridge assets so that they can be commented out. + Class for parsing bridget Assets ignoring commented out lines. """ def __init__(self, file=None): self = Asset.__init__(self, file) @@ -163,11 +164,11 @@ class BridgetTest(OONITest): self.socks_port = random.randint(1024, 2**16) self.control_port = random.randint(1024, 2**16) - if options['tor-path']: - self.tor_binary = options['tor-path'] + if options['torpath']: + self.tor_binary = options['torpath'] - if options['data-dir']: - self.config.DataDirectory = options['data-dir'] + if options['datadir']: + self.config.DataDirectory = options['datadir'] if options['transport']: ## ClientTransportPlugin transport socks4|socks5 IP:PORT

1 0

[flashproxy/master] Don't disable for Safari 6.0 and later.
by dcf＠torproject.org 15 Sep '12

15 Sep '12

commit 03d253168bb6fce49bcbd0dc77b5fe930dece8b7 Author: David Fifield <david(a)bamsoftware.com> Date: Fri Sep 14 22:01:09 2012 -0700 Don't disable for Safari 6.0 and later. Safari 6 now has the newer hybi/RFC 6455 type of WebSocket. I just tested and was able to bootstrap through Safari for the first time. I did not find a nice in-browser way to test for the websocket type, short of User-Agent sniffing. --- flashproxy.js | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/flashproxy.js b/flashproxy.js index 32eb534..cd31f60 100644 --- a/flashproxy.js +++ b/flashproxy.js @@ -843,10 +843,11 @@ function flashproxy_should_disable() { } } - if (ua.match(/\bsafari\b/i) && !ua.match(/\bchrome\b/i)) { - /* Disable on Safari because it doesn't have the hybi/RFC type of - WebSockets. */ - puts("Disable because User-Agent is Safari."); + if (ua.match(/\bsafari\b/i) && !ua.match(/\bchrome\b/i) + && !ua.match(/\bversion\/[6789]\./i)) { + /* Disable before Safari 6.0 because it doesn't have the hybi/RFC type + of WebSockets. */ + puts("Disable because User-Agent is Safari before 6.0."); return true; }

1 0