November 2012 - tor-commits - lists.torproject.org

[torspec/master] Clean up and finish 216-ntor-handshake.txt
by nickm＠torproject.org 27 Nov '12

27 Nov '12

commit 08b4c89058e7704f9cdb1e11407c7d1c8186c0cd Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 27 15:42:00 2012 -0500 Clean up and finish 216-ntor-handshake.txt Remove the PARAMS/PARAMSLEN business I had added: nobody really vetted it. Refer to proposal 200 by number. Add a section at the end to explain where to put keys and everything. --- proposals/000-index.txt | 2 + proposals/216-ntor-handshake.txt | 59 +++++++++++++++++++++++--------------- 2 files changed, 38 insertions(+), 23 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 4a641a2..72bec71 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -136,6 +136,7 @@ Proposals by number: 213 Remove stream-level sendmes from the design [OPEN] 214 Allow 4-byte circuit IDs in a new link protocol [OPEN] 215 Let the minimum consensus method change with time [OPEN] +216 Improved circuit-creation key exchange [OPEN] Proposals by status: @@ -187,6 +188,7 @@ Proposals by status: 213 Remove stream-level sendmes from the design 214 Allow 4-byte circuit IDs in a new link protocol 215 Let the minimum consensus method change with time + 216 Improved circuit-creation key exchange ACCEPTED: 140 Provide diffs between consensuses 147 Eliminate the need for v2 directories in generating v3 directories [for 0.2.4.x] diff --git a/proposals/216-ntor-handshake.txt b/proposals/216-ntor-handshake.txt index 081d6af..c6c6579 100644 --- a/proposals/216-ntor-handshake.txt +++ b/proposals/216-ntor-handshake.txt @@ -1,17 +1,17 @@ -Filename: xxx-ntor-handshake.txt +Filename: 216-ntor-handshake.txt Title: Improved circuit-creation key exchange Author: Nick Mathewson Created: 11-May-2011 -Status: Draft +Status: Open +Summary: -This is an attempt to translate the proposed circuit handshake from -"Anonymity and one-way authentication in key-exchange protocols" by -Goldberg, Stebila, and Ustaoglu, into a Tor proposal format. + This is an attempt to translate the proposed circuit handshake from + "Anonymity and one-way authentication in key-exchange protocols" by + Goldberg, Stebila, and Ustaoglu, into a Tor proposal format. -It assumes something like Robert Ransom's proposal draft is in place to -provide an extended CREATE cell format that can indicate what type of -handshake is in use. + It assumes that proposal 200 is implemented, to provide an extended CREATE + cell format that can indicate what type of handshake is in use. Notation: @@ -67,35 +67,25 @@ Protocol: NODEID: ID -- H_LENGTH bytes KEYID: KEYID(B) -- H_LENGTH bytes CLIENT_PK: X -- G_LENGTH bytes - PARAMSLEN: -- 2 bytes - PARMS: -- PARAMSLEN byets - - (The "PARAMS" component is used to encode any additional authenticated - information that's needed for establishing the right kind of circuit. - XXXXX nickm added it; we need to ask if it's safe!!!!) The server generates a keypair of y,Y = KEYGEN(), and computes - secret_input = EXP(X,y) | EXP(X,b) | ID | B | X | Y | PARAMSLEN | PARAMS - | PROTOID + secret_input = EXP(X,y) | EXP(X,b) | ID | B | X | Y | PROTOID KEY_SEED = H(secret_input, t_key) verify = H(secret_input, t_verify) - auth_input = verify | ID | B | Y | X | PARAMSLEN | PARAMS | PROTOID - | "Server" + auth_input = verify | ID | B | Y | X | PROTOID | "Server" The server sends a CREATED cell containing: SERVER_PK: Y -- G_LENGTH bytes AUTH: H(auth_input, t_mac) -- H_LENGTH byets - The client then checks Y is in G^* [see below], and computes + The client then checks Y is in G^* [see NOTE below], and computes - secret_input = EXP(Y,x) | EXP(B,x) | ID | B | X | Y | PARAMSLEN | PARAMS - | PROTOID + secret_input = EXP(Y,x) | EXP(B,x) | ID | B | X | Y | PROTOID KEY_SEED = H(secret_input, t_key) verify = H(secret_input, t_verify) - auth_input = verify | ID | B | Y | X | PARAMLENS | PARAMS | PROTOID - | "Server" + auth_input = verify | ID | B | Y | X | PROTOID | "Server" The client verifies that AUTH == H(auth_input, t_mac). @@ -125,6 +115,8 @@ Key expansion: Ian says this is due to a construction from Krawczyk at http://eprint.iacr.org/2010/264 . + Let m_expand be PROTOID | ":key_expand" + Performance notes: In Tor's current circuit creation handshake, the client does: @@ -149,3 +141,24 @@ Performance notes: 3 H operations for the handshake 3 H operations for the key expansion +Integrating with the rest of Tor: + + Add a new optional entry to router descriptors and microdescriptors: + + "onion-key-ntor" SP Base64Key NL + + where Base64Key is a base-64 encoded 32-byte value, with padding + omitted. + + Add a new consensus method to tell servers to copy "onion-key-ntor" + entries to from router descriptors to microdescriptors. + + Add a "UseNTorHandshake" configuration option and a corresponding + consensus parameter to control whether clients use the ntor + handshake. If the configuration option is "auto", clients should + obey the consensus parameter. Have the configuration default be + "auto" and the consensus value initially be "0". + + Reserve the handshake type [00 01] for this handshake in CREATE2 and + EXTEND2 cells. +

1 0

[torspec/master] Turn xxx-ntor-handshake into a real proposal
by nickm＠torproject.org 27 Nov '12

27 Nov '12

commit f8c392b65751908ecb7be7f10d41aa3bf0fbe86c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 27 15:39:54 2012 -0500 Turn xxx-ntor-handshake into a real proposal --- proposals/216-ntor-handshake.txt | 151 ++++++++++++++++++++++++++++++++ proposals/ideas/xxx-ntor-handshake.txt | 151 -------------------------------- 2 files changed, 151 insertions(+), 151 deletions(-) diff --git a/proposals/216-ntor-handshake.txt b/proposals/216-ntor-handshake.txt new file mode 100644 index 0000000..081d6af --- /dev/null +++ b/proposals/216-ntor-handshake.txt @@ -0,0 +1,151 @@ +Filename: xxx-ntor-handshake.txt +Title: Improved circuit-creation key exchange +Author: Nick Mathewson +Created: 11-May-2011 +Status: Draft + + +This is an attempt to translate the proposed circuit handshake from +"Anonymity and one-way authentication in key-exchange protocols" by +Goldberg, Stebila, and Ustaoglu, into a Tor proposal format. + +It assumes something like Robert Ransom's proposal draft is in place to +provide an extended CREATE cell format that can indicate what type of +handshake is in use. + +Notation: + + Let a|b be the concatenation of a with b. + + Let H(x,t) be a tweakable hash function of output width H_LENGTH bytes. + + Let t_mac, t_key, and t_verify be a set of arbitrarily-chosen tweaks + for the hash function. + + Let EXP(a,b) be a^b in some appropriate group G where the appropriate DH + parameters hold. Let's say elements of this group, when represented as + byte strings, are all G_LENGTH bytes long. Let's say we are using a + generator g for this group. + + Let a,A=KEYGEN() yield a new private-public keypair in G, where a is the + secret key and A = EXP(g,a). If additional checks are needed to insure + a valid keypair, they should be performed. + + Let PROTOID be a string designating this variant of the protocol. + + Let KEYID be a collision-resistant (but not necessarily preimage-resistant) + hash function on members of G, of output length H_LENGTH bytes. + +Instantiation: + + Let's call this PROTOID "ntor-curve25519-sha256-1" (We might want to make + this shorter if it turns out to save us a block of hashing somewhere.) + + Set H(x,t) == HMAC_SHA256 with message x and key t. So H_LENGTH == 32. + Set t_mac == PROTOID | ":mac" + t_key == PROTOID | ":key" + t_verify == PROTOID | ":verify" + + Set EXP(a,b) == curve25519(.,b,a), and g == 9 . Let KEYGEN() do the + appropriate manipulations when generating the secret key (clearing the + low bits, twiddling the high bits). + + Set KEYID(B) == B. (We don't need to use a hash function here, since our + keys are already very short. It is trivially collision-resistant, since + KEYID(A)==KEYID(B) iff A==B.) + +Protocol: + + Take a router with identity key digest ID. + + As setup, the router generates a secret key b, and a public onion key + B with b, B = KEYGEN(). The router publishes B in its server descriptor. + + To send a create cell, the client generates a keypair x,X = KEYGEN(), and + sends a CREATE cell with contents: + + NODEID: ID -- H_LENGTH bytes + KEYID: KEYID(B) -- H_LENGTH bytes + CLIENT_PK: X -- G_LENGTH bytes + PARAMSLEN: -- 2 bytes + PARMS: -- PARAMSLEN byets + + (The "PARAMS" component is used to encode any additional authenticated + information that's needed for establishing the right kind of circuit. + XXXXX nickm added it; we need to ask if it's safe!!!!) + + The server generates a keypair of y,Y = KEYGEN(), and computes + + secret_input = EXP(X,y) | EXP(X,b) | ID | B | X | Y | PARAMSLEN | PARAMS + | PROTOID + KEY_SEED = H(secret_input, t_key) + verify = H(secret_input, t_verify) + auth_input = verify | ID | B | Y | X | PARAMSLEN | PARAMS | PROTOID + | "Server" + + The server sends a CREATED cell containing: + + SERVER_PK: Y -- G_LENGTH bytes + AUTH: H(auth_input, t_mac) -- H_LENGTH byets + + The client then checks Y is in G^* [see below], and computes + + secret_input = EXP(Y,x) | EXP(B,x) | ID | B | X | Y | PARAMSLEN | PARAMS + | PROTOID + KEY_SEED = H(secret_input, t_key) + verify = H(secret_input, t_verify) + auth_input = verify | ID | B | Y | X | PARAMLENS | PARAMS | PROTOID + | "Server" + + The client verifies that AUTH == H(auth_input, t_mac). + + [NOTE: It may be adequate to check that EXP(Y,x) is not the point at + infinity. See tor-dev thread.] + + Both parties now have a shared value for KEY_SEED. They expand this into + the keys needed for the Tor relay protocol. + +Key expansion: + + Currently, the key expansion formula used by Tor here is + + K = SHA(K0 | [00]) | SHA(K0 | [01]) | SHA(K0 | [02]) | ... + + where K0==g^xy, and K is divvied up into Df, Db, Kf, and Kb portions. + + Instead, let's have it be + + K = K_0 | K_1 | K_2 | K_3 | ... + + Where K_0 = H(m_expand | INT8(i) , KEY_SEED ) + and K_(i+1) = H(K_i | m_expand | INT8(i) , KEY_SEED ) + and m_expend is an arbitrarily chosen value, + and INT8(i) is a octet with the value "i". + + Ian says this is due to a construction from Krawczyk at + http://eprint.iacr.org/2010/264 . + +Performance notes: + + In Tor's current circuit creation handshake, the client does: + One RSA public-key encryption + A full DH handshake in Z_p + A short AES encryption + Five SHA1s for key expansion + And the server does: + One RSA private-key decryption + A full DH handshake in Z_p + A short AES decryption + Five SHA1s for key expansion + + While in the revised handshake, the client does: + A full DH handshake + A public-half of a DH handshake + 3 H operations for the handshake + 3 H operations for the key expansion + and the server does: + A full DH handshake + A private-half of a DH handshake + 3 H operations for the handshake + 3 H operations for the key expansion + diff --git a/proposals/ideas/xxx-ntor-handshake.txt b/proposals/ideas/xxx-ntor-handshake.txt deleted file mode 100644 index 081d6af..0000000 --- a/proposals/ideas/xxx-ntor-handshake.txt +++ /dev/null @@ -1,151 +0,0 @@ -Filename: xxx-ntor-handshake.txt -Title: Improved circuit-creation key exchange -Author: Nick Mathewson -Created: 11-May-2011 -Status: Draft - - -This is an attempt to translate the proposed circuit handshake from -"Anonymity and one-way authentication in key-exchange protocols" by -Goldberg, Stebila, and Ustaoglu, into a Tor proposal format. - -It assumes something like Robert Ransom's proposal draft is in place to -provide an extended CREATE cell format that can indicate what type of -handshake is in use. - -Notation: - - Let a|b be the concatenation of a with b. - - Let H(x,t) be a tweakable hash function of output width H_LENGTH bytes. - - Let t_mac, t_key, and t_verify be a set of arbitrarily-chosen tweaks - for the hash function. - - Let EXP(a,b) be a^b in some appropriate group G where the appropriate DH - parameters hold. Let's say elements of this group, when represented as - byte strings, are all G_LENGTH bytes long. Let's say we are using a - generator g for this group. - - Let a,A=KEYGEN() yield a new private-public keypair in G, where a is the - secret key and A = EXP(g,a). If additional checks are needed to insure - a valid keypair, they should be performed. - - Let PROTOID be a string designating this variant of the protocol. - - Let KEYID be a collision-resistant (but not necessarily preimage-resistant) - hash function on members of G, of output length H_LENGTH bytes. - -Instantiation: - - Let's call this PROTOID "ntor-curve25519-sha256-1" (We might want to make - this shorter if it turns out to save us a block of hashing somewhere.) - - Set H(x,t) == HMAC_SHA256 with message x and key t. So H_LENGTH == 32. - Set t_mac == PROTOID | ":mac" - t_key == PROTOID | ":key" - t_verify == PROTOID | ":verify" - - Set EXP(a,b) == curve25519(.,b,a), and g == 9 . Let KEYGEN() do the - appropriate manipulations when generating the secret key (clearing the - low bits, twiddling the high bits). - - Set KEYID(B) == B. (We don't need to use a hash function here, since our - keys are already very short. It is trivially collision-resistant, since - KEYID(A)==KEYID(B) iff A==B.) - -Protocol: - - Take a router with identity key digest ID. - - As setup, the router generates a secret key b, and a public onion key - B with b, B = KEYGEN(). The router publishes B in its server descriptor. - - To send a create cell, the client generates a keypair x,X = KEYGEN(), and - sends a CREATE cell with contents: - - NODEID: ID -- H_LENGTH bytes - KEYID: KEYID(B) -- H_LENGTH bytes - CLIENT_PK: X -- G_LENGTH bytes - PARAMSLEN: -- 2 bytes - PARMS: -- PARAMSLEN byets - - (The "PARAMS" component is used to encode any additional authenticated - information that's needed for establishing the right kind of circuit. - XXXXX nickm added it; we need to ask if it's safe!!!!) - - The server generates a keypair of y,Y = KEYGEN(), and computes - - secret_input = EXP(X,y) | EXP(X,b) | ID | B | X | Y | PARAMSLEN | PARAMS - | PROTOID - KEY_SEED = H(secret_input, t_key) - verify = H(secret_input, t_verify) - auth_input = verify | ID | B | Y | X | PARAMSLEN | PARAMS | PROTOID - | "Server" - - The server sends a CREATED cell containing: - - SERVER_PK: Y -- G_LENGTH bytes - AUTH: H(auth_input, t_mac) -- H_LENGTH byets - - The client then checks Y is in G^* [see below], and computes - - secret_input = EXP(Y,x) | EXP(B,x) | ID | B | X | Y | PARAMSLEN | PARAMS - | PROTOID - KEY_SEED = H(secret_input, t_key) - verify = H(secret_input, t_verify) - auth_input = verify | ID | B | Y | X | PARAMLENS | PARAMS | PROTOID - | "Server" - - The client verifies that AUTH == H(auth_input, t_mac). - - [NOTE: It may be adequate to check that EXP(Y,x) is not the point at - infinity. See tor-dev thread.] - - Both parties now have a shared value for KEY_SEED. They expand this into - the keys needed for the Tor relay protocol. - -Key expansion: - - Currently, the key expansion formula used by Tor here is - - K = SHA(K0 | [00]) | SHA(K0 | [01]) | SHA(K0 | [02]) | ... - - where K0==g^xy, and K is divvied up into Df, Db, Kf, and Kb portions. - - Instead, let's have it be - - K = K_0 | K_1 | K_2 | K_3 | ... - - Where K_0 = H(m_expand | INT8(i) , KEY_SEED ) - and K_(i+1) = H(K_i | m_expand | INT8(i) , KEY_SEED ) - and m_expend is an arbitrarily chosen value, - and INT8(i) is a octet with the value "i". - - Ian says this is due to a construction from Krawczyk at - http://eprint.iacr.org/2010/264 . - -Performance notes: - - In Tor's current circuit creation handshake, the client does: - One RSA public-key encryption - A full DH handshake in Z_p - A short AES encryption - Five SHA1s for key expansion - And the server does: - One RSA private-key decryption - A full DH handshake in Z_p - A short AES decryption - Five SHA1s for key expansion - - While in the revised handshake, the client does: - A full DH handshake - A public-half of a DH handshake - 3 H operations for the handshake - 3 H operations for the key expansion - and the server does: - A full DH handshake - A private-half of a DH handshake - 3 H operations for the handshake - 3 H operations for the key expansion -

1 0

[torspec/master] Fix 6963 in proposal 198 too.
by nickm＠torproject.org 27 Nov '12

27 Nov '12

commit 58809be4a72d6b81b3a4d9b48c14dc8139ab26b4 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Nov 23 10:14:49 2012 -0500 Fix 6963 in proposal 198 too. In the list of v1 ciphers, we had listed SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, which didn't belong there, in a likely copy-and-paste issue from tor-spec.txt Fixes #6963; reported by cced. --- proposals/198-restore-clienthello-semantics.txt | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/proposals/198-restore-clienthello-semantics.txt b/proposals/198-restore-clienthello-semantics.txt index 3e336b0..c3b74c6 100644 --- a/proposals/198-restore-clienthello-semantics.txt +++ b/proposals/198-restore-clienthello-semantics.txt @@ -75,7 +75,6 @@ Proposed spec changes: TLS_DHE_RSA_WITH_AES_256_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA - SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA If the advertised ciphersuites in the ClientHello are _exactly_[*] the following, they indicate that the Tor v2+ link protocol is in

1 0

[tor-cloud/master] Use sudo bash when running the ec2-prep.sh script, thanks john_
by runa＠torproject.org 27 Nov '12

27 Nov '12

commit aa572c0ff9ddf7ca3f2500c59e50922b59d87530 Author: Runa A. Sandvik <runa.sandvik(a)gmail.com> Date: Tue Nov 27 20:16:52 2012 +0000 Use sudo bash when running the ec2-prep.sh script, thanks john_ --- rc.local | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/rc.local b/rc.local index 8593f96..b6c0644 100644 --- a/rc.local +++ b/rc.local @@ -1,5 +1,5 @@ #!/bin/sh -e sudo apt-get -y update sudo apt-get -y upgrade -sudo /etc/ec2-prep.sh type +sudo bash /etc/ec2-prep.sh type exit 0

1 0

[metrics-tasks/master] Add a graph with relative probabilities (#1854).
by karsten＠torproject.org 27 Nov '12

27 Nov '12

commit bd4e73627eebfe7332e125c772b368c86cd07ce8 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Nov 27 12:44:17 2012 -0500 Add a graph with relative probabilities (#1854). --- task-1854/plot-entropy.R | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/task-1854/plot-entropy.R b/task-1854/plot-entropy.R index cc963e4..ac02782 100644 --- a/task-1854/plot-entropy.R +++ b/task-1854/plot-entropy.R @@ -45,6 +45,19 @@ scale_colour_hue(name = "Advertised bandwidth cutoff in B/s") + opts(title = paste("Consensus with valid-after time", max(p$validafter)), legend.position = "top") +c <- p[p$advbw >= 1048576, ] +c <- data.frame(advbw = c$advbw, prob = c$prob, + minadvbw = paste("c", c$minadvbw, sep = "")) +c <- cast(c, advbw ~ minadvbw, value = "prob") +c <- data.frame(advbw = c$advbw, rel = c$c1048576 / c$c20480) +ggplot(c, aes(x = advbw, y = rel)) + +geom_point(alpha = 0.25) + +scale_x_log10(name = "\nAdvertised bandwidth in B/s (log scale)") + +scale_y_continuous(name = paste("Ratio of probaility at 1 MiB/s cutoff", + "to probability at 20 KiB/s cutoff\n")) + +opts(title = paste("Consensus with valid-after time ", max(p$validafter), + "\n", sep = ""), legend.position = "top") + e <- read.csv("linf-extracted.csv", header = FALSE, col.names = c("validafter", "min_adv_bw", "relays", "linf", "excl_adv_bw", "graph"), stringsAsFactor = FALSE)

1 0

[metrics-tasks/master] Remove unused GeoIP code (#1854).
by karsten＠torproject.org 27 Nov '12

27 Nov '12

commit 767e93c8b3639e0e06f8e17e3f176e625efde9c5 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Nov 27 09:17:03 2012 -0500 Remove unused GeoIP code (#1854). --- task-1854/pylinf.py | 18 ------------------ 1 files changed, 0 insertions(+), 18 deletions(-) diff --git a/task-1854/pylinf.py b/task-1854/pylinf.py index 3bdd8a2..9885de5 100644 --- a/task-1854/pylinf.py +++ b/task-1854/pylinf.py @@ -11,7 +11,6 @@ rsync -arz --delete metrics.torproject.org::metrics-recent/relay-descriptors/con import sys import math import os -import pygeoip import tarfile import pickle import StringIO @@ -28,8 +27,6 @@ class Router: self.bandwidth = None self.advertised_bw = None self.hex_digest = None - self.country = None - self.as_no = None self.is_exit = None self.is_guard = None @@ -37,8 +34,6 @@ class Router: self.hex_digest = b2a_hex(a2b_base64(values[2]+"=")) self.advertised_bw = self.get_advertised_bw(self.hex_digest) ip = values[5] - #self.country = gi_db.country_code_by_addr(ip) - #self.as_no = self.get_as_details(ip) def add_weights(self, values): self.bandwidth = int(values[0].split('=')[1]) @@ -49,13 +44,6 @@ class Router: if "Guard" in values: self.is_guard = True - def get_as_details(self, ip): - try: - value = as_db.org_by_addr(str(ip)).split() - return value[0] - except: - return "" - def get_advertised_bw(self, hex_digest): try: return descriptors[self.hex_digest.upper()] @@ -213,10 +201,6 @@ def parse_args(): usage = "Usage - python pyentropy.py [options]" parser = OptionParser(usage) - parser.add_option("-g", "--geoip", dest="gi_db", default="GeoIP.dat", - help="Input GeoIP database") - parser.add_option("-a", "--as", dest="as_db", default="GeoIPASNum.dat", - help="Input AS GeoIP database") parser.add_option("-s", "--server_desc", dest="server_desc", default=False, help="Server descriptors directory") parser.add_option("-l", "--linf-output", dest="linf", default="linf.csv", @@ -234,8 +218,6 @@ def parse_args(): if __name__ == "__main__": options = parse_args() - gi_db = pygeoip.GeoIP(options.gi_db) - as_db = pygeoip.GeoIP(options.as_db) server_desc_files = [] global descriptors

1 0

[metrics-tasks/master] Add graph with single relay probabilities (#1854).
by karsten＠torproject.org 27 Nov '12

27 Nov '12

commit 7d94e96a9989342b7dd3a78705cfd41a9611853f Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Nov 27 10:41:14 2012 -0500 Add graph with single relay probabilities (#1854). --- task-1854/plot-entropy.R | 29 ++++++++++++++++++++++++++--- task-1854/pylinf.py | 13 ++++++++----- 2 files changed, 34 insertions(+), 8 deletions(-) diff --git a/task-1854/plot-entropy.R b/task-1854/plot-entropy.R index 5d30583..cc963e4 100644 --- a/task-1854/plot-entropy.R +++ b/task-1854/plot-entropy.R @@ -3,7 +3,7 @@ library(reshape) library(scales) p <- read.csv("prob-extracted.csv", header = FALSE, - col.names = c("validafter", "minadvbw", "advbw", "cumprob"), + col.names = c("validafter", "minadvbw", "advbw", "cumprob", "prob"), stringsAsFactor = FALSE) p <- p[p$minadvbw >= 20480, ] c <- data.frame(x = p$advbw, y = p$cumprob, @@ -13,14 +13,37 @@ geom_line() + scale_x_log10(name = "\nAdvertised bandwidth in B/s (log scale)") + scale_y_continuous(name = "Cumulative probability\n") + scale_colour_hue(name = "Adv. bw. cutoff in B/s") + -opts(legend.position = "top") +opts(title = paste("Consensus with valid-after time", max(p$validafter)), + legend.position = "top") ggplot(c, aes(x = x, y = y, colour = colour)) + geom_line() + scale_x_log10(name = "\nAdvertised bandwidth in B/s (log scale)") + scale_y_log10(name = "Cumulative probability (log scale)\n") + scale_colour_hue(name = "Adv. bw. cutoff in B/s") + -opts(legend.position = "top") +opts(title = paste("Consensus with valid-after time", max(p$validafter)), + legend.position = "top") + +p <- p[p$minadvbw == 20480 | p$minadvbw == 1048576, ] +c <- data.frame(x = p$advbw, y = p$prob, + colour = as.factor(p$minadvbw)) +ggplot(c, aes(x = x, y = y, colour = colour)) + +geom_point(alpha = 0.25) + +scale_x_log10(name = "\nAdvertised bandwidth in B/s (log scale)") + +scale_y_continuous(name = paste("Single relay probability, *not*", + "probability distribution function\n")) + +scale_colour_hue(name = "Advertised bandwidth cutoff in B/s") + +opts(title = paste("Consensus with valid-after time", max(p$validafter)), + legend.position = "top") + +ggplot(c, aes(x = x, y = y, colour = colour)) + +geom_point(alpha = 0.25) + +scale_x_log10(name = "\nAdvertised bandwidth in B/s (log scale)") + +scale_y_log10(name = paste("Single relay probability, *not*", + "probability distribution function (log scale)\n")) + +scale_colour_hue(name = "Advertised bandwidth cutoff in B/s") + +opts(title = paste("Consensus with valid-after time", max(p$validafter)), + legend.position = "top") e <- read.csv("linf-extracted.csv", header = FALSE, col.names = c("validafter", "min_adv_bw", "relays", "linf", diff --git a/task-1854/pylinf.py b/task-1854/pylinf.py index 9885de5..15833e6 100644 --- a/task-1854/pylinf.py +++ b/task-1854/pylinf.py @@ -168,8 +168,8 @@ def run(data): str(len(routers)-omitted_routers), str(max(prob_diff))])) - while len(cutoffs) > 0 and min_adv_bw > cutoffs[0]: - cumulated_prob = 0.0 + while len(cutoffs) > 0 and min_adv_bw >= cutoffs[0]: + prev_prob, cumulated_prob = 0.0, 0.0 prev_advertised_bw = 0 for router in routers: if router.advertised_bw > cutoffs[0] and \ @@ -177,13 +177,16 @@ def run(data): prob_string.append(','.join([valid_after, str(cutoffs[0]), str(prev_advertised_bw), - str(cumulated_prob)])) + str(cumulated_prob), + str(prev_prob)])) prev_advertised_bw = router.advertised_bw - cumulated_prob += float(router.bandwidth)/float(total_bw) + prev_prob = float(router.bandwidth)/float(total_bw) + cumulated_prob += prev_prob prob_string.append(','.join([valid_after, str(cutoffs[0]), str(prev_advertised_bw), - str(cumulated_prob)])) + str(cumulated_prob), + str(prev_prob)])) cutoffs.pop(0) # remove routers with min adv_bw

1 0

[ooni-probe/master] Parse the test specific command line options when running a test or tests
by art＠torproject.org 27 Nov '12

27 Nov '12

commit 5cbd79a53c6e9d2612b270677ed7d3f5c9401225 Author: Arturo Filastò <art(a)fuffa.org> Date: Tue Nov 27 11:54:55 2012 +0100 Parse the test specific command line options when running a test or tests * Read the SOCKS port and Control port via the control port when starting with Tor --- ooni/oonicli.py | 57 ++++++++++++++++++++++++++++++++---------------------- ooni/runner.py | 20 ++++++++++++++---- 2 files changed, 49 insertions(+), 28 deletions(-) diff --git a/ooni/oonicli.py b/ooni/oonicli.py index 26cfdab..f706ee3 100644 --- a/ooni/oonicli.py +++ b/ooni/oonicli.py @@ -111,26 +111,18 @@ def startSniffing(): sniffer = ScapySniffer(config.reports.pcap) config.scapyFactory.registerProtocol(sniffer) +def runTestList(none, test_list): + """ + none: is always None. -def runTestDeckOrTest(result, cmd_line_options): + test_list (list): a list of tuples containing (test_cases, options, + cmd_line_options) + """ deck_dl = [] - resume = cmd_line_options['resume'] - if cmd_line_options['testdeck']: - test_deck = yaml.safe_load(open(cmd_line_options['testdeck'])) - for test in test_deck: - del cmd_line_options - cmd_line_options = test['options'] - if resume: - cmd_line_options['resume'] = True - else: - cmd_line_options['resume'] = False - d1 = runner.runTest(cmd_line_options) - deck_dl.append(d1) - else: - log.msg("No test deck detected") - del cmd_line_options['testdeck'] - d1 = runner.runTest(cmd_line_options) + for test in test_list: + test_cases, options, cmd_line_options = test + d1 = runner.runTestCases(test_cases, options, cmd_line_options) deck_dl.append(d1) d2 = defer.DeferredList(deck_dl) @@ -146,9 +138,8 @@ def errorRunningTests(failure): def run(): """ - Call me to begin testing from a file. + Parses command line arguments of test. """ - cmd_line_options = Options() if len(sys.argv) == 1: cmd_line_options.getUsage() @@ -163,15 +154,35 @@ def run(): log.msg("Starting") runner.startSniffing() + resume = cmd_line_options['resume'] + + # contains (test_cases, options, cmd_line_options) + test_list = [] + + if cmd_line_options['testdeck']: + test_deck = yaml.safe_load(open(cmd_line_options['testdeck'])) + for test in test_deck: + del cmd_line_options + cmd_line_options = test['options'] + if resume: + cmd_line_options['resume'] = True + else: + cmd_line_options['resume'] = False + test_list.append(runner.loadTest(cmd_line_options)) + else: + log.msg("No test deck detected") + del cmd_line_options['testdeck'] + test_list.append(runner.loadTest(cmd_line_options)) + if config.advanced.start_tor: log.msg("Starting Tor...") d = runner.startTor() - d.addCallback(runTestDeckOrTest, cmd_line_options) + d.addCallback(runTestList, test_list) d.addErrback(errorRunningTests) else: - # We need to pass None as first argument because when the callback - # is fired it will pass it's result to runTestCase. - d = runTestDeckOrTest(None, cmd_line_options) + # We need to pass None as first argument because when the callback is + # fired it will pass it's result to runTestCase. + d = runTestList(None, test_list) d.addErrback(errorRunningTests) reactor.run() diff --git a/ooni/runner.py b/ooni/runner.py index 30682e9..d5cc5fb 100644 --- a/ooni/runner.py +++ b/ooni/runner.py @@ -44,9 +44,11 @@ def processTest(obj): :param obj: An uninstantiated old test, which should be a subclass of :class:`ooni.plugoo.tests.OONITest`. + :param cmd_line_options: A configured and instantiated :class:`twisted.python.usage.Options` class. + """ if not hasattr(obj.usageOptions, 'optParameters'): obj.usageOptions.optParameters = [] @@ -440,8 +442,12 @@ def startTor(): log.debug("We now have the following circuits: ") for circuit in state.circuits.values(): log.debug(" * %s" % circuit) - config.tor.socks_port = yield state.protocol.get_conf("SocksPort") - config.tor.control_port = yield state.protocol.get_conf("ControlPort") + + socks_port = yield state.protocol.get_conf("SocksPort") + control_port = yield state.protocol.get_conf("ControlPort") + + config.tor.socks_port = int(socks_port.values()[0]) + config.tor.control_port = int(control_port.values()[0]) def setup_failed(failure): log.exception(failure) @@ -458,7 +464,7 @@ def startTor(): return state.post_bootstrap def updates(prog, tag, summary): - print "%d%%: %s" % (prog, summary) + log.msg("%d%%: %s" % (prog, summary)) tor_config = TorConfig() if config.tor.control_port: @@ -501,7 +507,11 @@ def startSniffing(): sniffer = ScapySniffer(config.reports.pcap) config.scapyFactory.registerProtocol(sniffer) -def runTest(cmd_line_options): +def loadTest(cmd_line_options): + """ + Takes care of parsing test command line arguments and loading their + options. + """ config.cmd_line_options = cmd_line_options config.generateReportFilenames() @@ -517,4 +527,4 @@ def runTest(cmd_line_options): classes = findTestClassesFromFile(cmd_line_options['test']) test_cases, options = loadTestsAndOptions(classes, cmd_line_options) - return runTestCases(test_cases, options, cmd_line_options) + return test_cases, options, cmd_line_options

1 0

[ooni-probe/master] Make debug messages a bit more useful
by art＠torproject.org 27 Nov '12

27 Nov '12

commit c1ae6afc95e8fdc23b6ce3d75a117a902be543d2 Author: Arturo Filastò <art(a)fuffa.org> Date: Tue Nov 27 15:15:40 2012 +0100 Make debug messages a bit more useful --- ooni/oonicli.py | 2 -- ooni/runner.py | 4 +++- ooni/templates/httpt.py | 3 +-- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/ooni/oonicli.py b/ooni/oonicli.py index f706ee3..ca80ad7 100644 --- a/ooni/oonicli.py +++ b/ooni/oonicli.py @@ -166,8 +166,6 @@ def run(): cmd_line_options = test['options'] if resume: cmd_line_options['resume'] = True - else: - cmd_line_options['resume'] = False test_list.append(runner.loadTest(cmd_line_options)) else: log.msg("No test deck detected") diff --git a/ooni/runner.py b/ooni/runner.py index d5cc5fb..97d4d7e 100644 --- a/ooni/runner.py +++ b/ooni/runner.py @@ -412,15 +412,17 @@ def runTestCases(test_cases, options, cmd_line_options): test_filename = os.path.basename(cmd_line_options['test']) if cmd_line_options['resume']: + log.debug("Resuming %s" % test_filename) resumeTest(test_filename, input_unit_factory) else: + log.debug("Nothing to resume for %s" % test_filename) config.stateDict[test_filename] = 0 updateProgressMeters(test_filename, input_unit_factory, len(test_cases)) try: for input_unit in input_unit_factory: - log.debug("Running this input unit %s" % input_unit) + log.debug("Running %s with input unit %s" % (test_filename, input_unit)) yield runTestCasesWithInputUnit(test_cases, input_unit, oreporter) diff --git a/ooni/templates/httpt.py b/ooni/templates/httpt.py index e3c8efe..dcd8ff9 100644 --- a/ooni/templates/httpt.py +++ b/ooni/templates/httpt.py @@ -215,7 +215,6 @@ class HTTPTest(NetTestCase): dict containing the response headers. This will lead self.headerProcessor to not be called. - body_processor (func): a function to be called with as argument the body of the response. This will lead self.bodyProcessor to not be called. @@ -281,7 +280,7 @@ class HTTPTest(NetTestCase): # We prefix the URL with 's' to make the connection go over the # configured socks proxy if use_tor: - log.debug("Using control agent for the request") + log.debug("Using Tor for the request to %s" % url) url = 's'+url agent = self.control_agent else:

1 0

[translation/tsum] Update translations for tsum
by translation＠torproject.org 27 Nov '12

27 Nov '12

commit f6ca3b0ccecdf226948a0ed4738f3c66f2eebf45 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Nov 27 04:45:14 2012 +0000 Update translations for tsum --- id/short-user-manual_id_noimg.xhtml | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/id/short-user-manual_id_noimg.xhtml b/id/short-user-manual_id_noimg.xhtml index 38012cc..28b4b7e 100644 --- a/id/short-user-manual_id_noimg.xhtml +++ b/id/short-user-manual_id_noimg.xhtml @@ -53,7 +53,7 @@ <p>After downloading the Tor Browser Bundle, extract the package onto your desktop or a USB stick. You should have a directory containing a few files. One of the files is an executable called "Start Tor Browser" (or "start-tor-browser", depending on your operating system).</p> <p>When you start the Tor Browser Bundle, you will first see Vidalia start up and connect you to the Tor network. After that, you will see a browser confirming that you are now using Tor. This is done by displaying <a href="https://check.torproject.org/">https://check.torproject.org/</a>. You can now browse the Internet through Tor.</p> <p> - <em>Please note that it is important that you use the browser that comes with the bundle, and not your own browser.</em> + <em>Harap di ingat penting untuk anda menggunakan peramban yang bersama dalam satu bundel, bukan peramban anda sendiri.</em> </p> <h3 id="what-to-do-when-tor-does-not-connect">Apa yang harus dilakukan ketika Tor tidak terhubung</h3> <p>Some users will notice that Vidalia gets stuck when trying to connect to the Tor network. If this happens, make sure that you are connected to the Internet. If you need to connect to a proxy server, see <em>How to use an open proxy</em> below.</p> @@ -68,7 +68,7 @@ <p>Configuring more than one bridge address will make your Tor connection more stable, in case some of the bridges become unreachable. There is no guarantee that the bridge you are using now will work tomorrow, so you should make a habit of updating your list of bridges every so often.</p> <h3 id="how-to-use-a-bridge">How to use a bridge</h3> <p>Once you have a set of bridges to use, open the Vidalia control panel, click on <em>Settings</em>, <em>Network</em> and tick the box that says <em>My ISP blocks connections to the Tor network</em>. Enter the bridges in the box below, hit <em>OK</em> and start Tor again.</p> - <h3 id="how-to-use-an-open-proxy">How to use an open proxy</h3> + <h3 id="how-to-use-an-open-proxy">Bagaiman cara untuk menggunakan proxy terbuka</h3> <p>If using a bridge does not work, try configuring Tor to use any HTTPS or SOCKS proxy to get access to the Tor network. This means even if Tor is blocked by your local network, open proxies can be safely used to connect to the Tor Network and on to the uncensored Internet.</p> <p>The steps below assume you have a functional Tor/Vidalia configuration, and you have found a list of HTTPS, SOCKS4, or SOCKS5 proxies.</p> <ol style="list-style-type: decimal">

1 0