tor-commits October 2013

tor-commits@lists.torproject.org

17 participants
1284 discussions

[flashproxy/master] Move gmail-setup.txt to gmail-howto.txt to match other docs.
by dcf＠torproject.org 14 Oct '13

14 Oct '13

commit a935e70d3697023a6d14453bfb7fd0470a8ddcd7 Author: David Fifield <david(a)bamsoftware.com> Date: Sun Oct 13 19:10:12 2013 -0700 Move gmail-setup.txt to gmail-howto.txt to match other docs. appengine-howto.txt already referred to gmail-howto.txt. --- facilitator/doc/facilitator-howto.txt | 2 +- facilitator/doc/gmail-howto.txt | 61 +++++++++++++++++++++++++++++++++ facilitator/doc/gmail-setup.txt | 61 --------------------------------- 3 files changed, 62 insertions(+), 62 deletions(-) diff --git a/facilitator/doc/facilitator-howto.txt b/facilitator/doc/facilitator-howto.txt index 8bafce9..a5cadb6 100644 --- a/facilitator/doc/facilitator-howto.txt +++ b/facilitator/doc/facilitator-howto.txt @@ -185,7 +185,7 @@ Copy the new fp-facilitator.pem to the facilitator server as === Email poller setup The facilitator-email-poller program requires a password that is used to -log in to the designated Gmail account. See the file gmail-setup.txt for +log in to the designated Gmail account. See the file gmail-howto.txt for instructions on setting up a Gmail account. After you've set up the account and have the password, save it to a file reg-email.pass and make it not readable or writable by anyone but its owner. diff --git a/facilitator/doc/gmail-howto.txt b/facilitator/doc/gmail-howto.txt new file mode 100644 index 0000000..b51ce90 --- /dev/null +++ b/facilitator/doc/gmail-howto.txt @@ -0,0 +1,61 @@ +These are instructions for setting up a Gmail account for use with the +email-based rendezvous and flashproxy-reg-email. These instructions were +current as of May 2013. + +You may have trouble if you are using Tor to create the account, for two +reasons. The first is that exit nodes are a source of abuse and Google +is more suspicious of them. The second is that Gmail is suspicious and +can lock you out of the account when your IP address is changing. While +setting up the account, use a single node in your torrc ExitNodes +configuration. Choose a U.S. exit node, one with low bandwidth. + +Go to https://mail.google.com/. Allow JavaScript to run (even from +youtube.com; it seems to be necessary). Click the "CREATE AN ACCOUNT" +button. + +Enter the account details. You don't need to fill in "Your current email +address". Enter a mobile phone number for later activation of two-factor +authentication. Solve the captcha. Click "Next Step". You may have to do +a phone SMS verification here. + +At this point the Gmail account is created. If you are pushed into +joining Google+, close everything out and go back to +https://mail.google.com/. + +Log out of the account and then back in again. There will be new text in +the lower right reading "Last account activity". Click "Details" and +turn off the unusual activity alerts. This will keep you from getting +locked out when you come from different IP addresses. At this point you +should remove the temporary ExitNodes configuration from torrc. + +Add a filter to prevent registrations from being marked as spam. Click +on the gear icon and select "Settings". Select "Filters" then "Create a +new filter". For "Has the words" type "in:spam", then "Create filter +with this search". There will be a warning that filters using "in:" will +never match incoming mail; this appears to be false and you can just +click OK. Check "Never send it to Spam" and click "Create filter". + +Enable IMAP. Click the gear icon, then "Settings", then "Forwarding and +POP/IMAP". + * Disable POP + * Enable IMAP + * Auto-Expunge on +Click "Save Changes". + +Enable two-factor authentication. We do this not so much for the +two-factor, but because it allows creating an independent password that +is used only for IMAP and does not have access to the web interface of +Gmail. Click the email address in the upper right, then "Account". Click +"Security". By "2-step verification" click "Edit". Click through until +it lets you set up. The phone number you provided when the account was +created will be automatically filled in. Choose "Text message (SMS)" +then click "Send code". Get your text message, type it in, and hit +"Verify". Uncheck "Trust this computer" on the next screen. Finally +"Confirm". On the following summary page, click "Show backup codes" and +save the codes to encrypted storage. Future codes can be generated at +https://www.google.com/accounts/SmsAuthConfig. + +Still on the 2-step summary page, click "Manage application-specific +passwords". Enter "IMAP" for the name and click "Generate password". +Save the password to encrypted storage. This is the password that will +be used with the --pass option of facilitator-email-poller. diff --git a/facilitator/doc/gmail-setup.txt b/facilitator/doc/gmail-setup.txt deleted file mode 100644 index b51ce90..0000000 --- a/facilitator/doc/gmail-setup.txt +++ /dev/null @@ -1,61 +0,0 @@ -These are instructions for setting up a Gmail account for use with the -email-based rendezvous and flashproxy-reg-email. These instructions were -current as of May 2013. - -You may have trouble if you are using Tor to create the account, for two -reasons. The first is that exit nodes are a source of abuse and Google -is more suspicious of them. The second is that Gmail is suspicious and -can lock you out of the account when your IP address is changing. While -setting up the account, use a single node in your torrc ExitNodes -configuration. Choose a U.S. exit node, one with low bandwidth. - -Go to https://mail.google.com/. Allow JavaScript to run (even from -youtube.com; it seems to be necessary). Click the "CREATE AN ACCOUNT" -button. - -Enter the account details. You don't need to fill in "Your current email -address". Enter a mobile phone number for later activation of two-factor -authentication. Solve the captcha. Click "Next Step". You may have to do -a phone SMS verification here. - -At this point the Gmail account is created. If you are pushed into -joining Google+, close everything out and go back to -https://mail.google.com/. - -Log out of the account and then back in again. There will be new text in -the lower right reading "Last account activity". Click "Details" and -turn off the unusual activity alerts. This will keep you from getting -locked out when you come from different IP addresses. At this point you -should remove the temporary ExitNodes configuration from torrc. - -Add a filter to prevent registrations from being marked as spam. Click -on the gear icon and select "Settings". Select "Filters" then "Create a -new filter". For "Has the words" type "in:spam", then "Create filter -with this search". There will be a warning that filters using "in:" will -never match incoming mail; this appears to be false and you can just -click OK. Check "Never send it to Spam" and click "Create filter". - -Enable IMAP. Click the gear icon, then "Settings", then "Forwarding and -POP/IMAP". - * Disable POP - * Enable IMAP - * Auto-Expunge on -Click "Save Changes". - -Enable two-factor authentication. We do this not so much for the -two-factor, but because it allows creating an independent password that -is used only for IMAP and does not have access to the web interface of -Gmail. Click the email address in the upper right, then "Account". Click -"Security". By "2-step verification" click "Edit". Click through until -it lets you set up. The phone number you provided when the account was -created will be automatically filled in. Choose "Text message (SMS)" -then click "Send code". Get your text message, type it in, and hit -"Verify". Uncheck "Trust this computer" on the next screen. Finally -"Confirm". On the following summary page, click "Show backup codes" and -save the codes to encrypted storage. Future codes can be generated at -https://www.google.com/accounts/SmsAuthConfig. - -Still on the 2-step summary page, click "Manage application-specific -passwords". Enter "IMAP" for the name and click "Generate password". -Save the password to encrypted storage. This is the password that will -be used with the --pass option of facilitator-email-poller.

1 0

[flashproxy/master] move appengine and facilitator docs to facilitator subtree
by dcf＠torproject.org 14 Oct '13

14 Oct '13

commit e59943493584cb0c704763eadffb8802c36aebdd Author: Ximin Luo <infinity0(a)gmx.com> Date: Wed Sep 11 17:21:09 2013 +0100 move appengine and facilitator docs to facilitator subtree --- appengine/README | 11 -- appengine/app.yaml | 9 -- appengine/fp-reg.go | 54 --------- doc/appengine-howto.txt | 56 ---------- doc/facilitator-howto.txt | 199 --------------------------------- doc/gmail-setup.txt | 61 ---------- facilitator/appengine/README | 11 ++ facilitator/appengine/app.yaml | 9 ++ facilitator/appengine/fp-reg.go | 54 +++++++++ facilitator/doc/appengine-howto.txt | 56 ++++++++++ facilitator/doc/facilitator-howto.txt | 199 +++++++++++++++++++++++++++++++++ facilitator/doc/gmail-setup.txt | 61 ++++++++++ 12 files changed, 390 insertions(+), 390 deletions(-) diff --git a/appengine/README b/appengine/README deleted file mode 100644 index 00d0611..0000000 --- a/appengine/README +++ /dev/null @@ -1,11 +0,0 @@ -This is the server-side code that runs on Google App Engine for the -"appspot" registration method. - -See doc/appengine-howto.txt for information about setting up an -application. - -To run with the development server: -$ ~/google_appengine/dev_appserver.py appengine/ - -To upload a new version: -$ torify ~/google_appengine/appcfg.py update appengine/ diff --git a/appengine/app.yaml b/appengine/app.yaml deleted file mode 100644 index 3b99ada..0000000 --- a/appengine/app.yaml +++ /dev/null @@ -1,9 +0,0 @@ -application: fp-reg-a -version: 1 -runtime: go -api_version: go1 - -handlers: -- url: /.* - script: _go_app - secure: always diff --git a/appengine/fp-reg.go b/appengine/fp-reg.go deleted file mode 100644 index d4f84c2..0000000 --- a/appengine/fp-reg.go +++ /dev/null @@ -1,54 +0,0 @@ -package fp_reg - -import ( - "io" - "net" - "net/http" - "path" - - "appengine" - "appengine/urlfetch" -) - -const BASE = "https://fp-facilitator.org/reg/" - -func robotsTxtHandler(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - w.Write([]byte("User-agent: *\nDisallow:\n")) -} - -func ipHandler(w http.ResponseWriter, r *http.Request) { - remoteAddr := r.RemoteAddr - if net.ParseIP(remoteAddr).To4() == nil { - remoteAddr = "[" + remoteAddr + "]" - } - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - w.Write([]byte(remoteAddr)) -} - -func regHandler(w http.ResponseWriter, r *http.Request) { - dir, blob := path.Split(path.Clean(r.URL.Path)) - if dir != "/reg/" { - http.NotFound(w, r) - return - } - client := urlfetch.Client(appengine.NewContext(r)) - resp, err := client.Get(BASE + blob) - if err != nil { - http.Error(w, err.Error(), http.StatusInternalServerError) - return - } - for key, values := range resp.Header { - for _, value := range values { - w.Header().Add(key, value) - } - } - w.WriteHeader(resp.StatusCode) - io.Copy(w, resp.Body) -} - -func init() { - http.HandleFunc("/robots.txt", robotsTxtHandler) - http.HandleFunc("/ip", ipHandler) - http.HandleFunc("/reg/", regHandler) -} diff --git a/doc/appengine-howto.txt b/doc/appengine-howto.txt deleted file mode 100644 index 44c5324..0000000 --- a/doc/appengine-howto.txt +++ /dev/null @@ -1,56 +0,0 @@ -These are instructions for how to set up a Google App Engine application -to run at appspot.com. - -General links: -https://developers.google.com/appengine/ -https://developers.google.com/appengine/docs/whatisgoogleappengine -https://developers.google.com/appengine/docs/go/gettingstarted/ - -You first need to create a Google account. See gmail-howto.txt for how -to do that. - -Download the SDK: -https://developers.google.com/appengine/docs/go/gettingstarted/devenvironment - -Write your program and app.yaml file: -https://developers.google.com/appengine/docs/go/gettingstarted/helloworld - -When you're ready to upload, log in with your Google account and follow -the directions: -https://appengine.google.com/ -https://developers.google.com/appengine/docs/go/gettingstarted/uploading - -Enter an application ID and create the application. - -Use the appcfg.py program to upload the program. It should look -something like this: - -$ torify ./google_appengine/appcfg.py update myapp/ -07:25 PM Host: appengine.google.com -07:25 PM Application: application-id; version: 1 -07:25 PM -Starting update of app: application-id, version: 1 -07:25 PM Getting current resource limits. -Email: xxx(a)gmail.com -Password for xxx(a)gmail.com: -07:26 PM Scanning files on local disk. -07:26 PM Cloning 2 application files. -07:26 PM Uploading 1 files and blobs. -07:26 PM Uploaded 1 files and blobs -07:26 PM Compilation starting. -07:26 PM Compilation: 1 files left. -07:26 PM Compilation completed. -07:26 PM Starting deployment. -07:26 PM Checking if deployment succeeded. -07:26 PM Deployment successful. -07:26 PM Checking if updated app version is serving. -07:26 PM Completed update of app: application-id, version: 1 - -Uploading the program in this way seems to create the files -~/.appcfg_nag and ~/.appcfg_cookies. Running the update command again -doesn't require you to enter your password again. - -Once logged in, you can disable logging for the application. Click -"Logs" on the left panel. Under "Total Logs Storage", click "Change -Settings". Enter "0" in the "days of logs" box and click "Save -Settings". diff --git a/doc/facilitator-howto.txt b/doc/facilitator-howto.txt deleted file mode 100644 index 8bafce9..0000000 --- a/doc/facilitator-howto.txt +++ /dev/null @@ -1,199 +0,0 @@ -This document describes how to run a flash proxy facilitator on Debian 7. -We will use the domain name fp-facilitator.example.com. - -== Overview - -The facilitator consists of a backend server that is essentially a -dynamic database of client addresses, as well as helper programs that -receive client registrations from the Internet over various means and -pass them to the backend. There are three supported helper rendezvous -methods: HTTP, URL, and email. - -The HTTP rendezvous uses an HTTP server and a CGI program. The HTTP -server is responsible for speaking TLS and invoking the CGI program. The -CGI program receives client registrations and proxy requests for -clients, parses them, and forwards them to the backend. We use Apache 2 -as the HTTP server. The CGI script is facilitator.cgi. The backend is -facilitator. - -The email rendezvous uses the helper program facilitator-email-poller. -Clients use the flashproxy-reg-email program to send an encrypted -message to a Gmail address. The poller constantly checks for new -messages and forwards them to facilitator-reg. - -facilitator-reg is a simple program that forwards its standard input to -a locally running facilitator-reg-daemon process. - -facilitator-reg-daemon accepts connections containing encrypted client -registrations and forwards them to the facilitator. It exists as a -process of its own so that only one program requires access to the -facilitator's private key. - -The URL rendezvous uses the helper program flashproxy-reg-email. The -helper program doesn't actually make a registration; rather, it prints -out a URL which, when retrieved, makes the registration. - -fac.py is a Python module containing code common to the various -facilitator programs. - -== Basic and security setup - -Install some essential packages and configure a firewall. - - # cat >/etc/apt/apt.conf.d/90suggests<<EOF -APT::Install-Recommends "0"; -APT::Install-Suggests "0"; -EOF - # apt-get remove portmap - # apt-get update - # apt-get upgrade - # apt-get install shorewall shorewall6 - -Away from the facilitator, generate an SSH key for authentication: - - $ ssh-keygen -f ~/.ssh/fp-facilitator - $ ssh-copy-id -i ~/.ssh/fp-facilitator.pub root(a)fp-facilitator.example.com - -Then log in and edit /etc/ssh/sshd_config to disable password -authentication: - - PasswordAuthentication no - -Configure the firewall to allow only SSH and HTTPS. - - # cd /etc/shorewall - # cp /usr/share/doc/shorewall/examples/Universal/{interfaces,policy,rules,zones} . - Edit /etc/shorewall/rules: -SECTION NEW -SSH(ACCEPT) net $FW -HTTPS(ACCEPT) net $FW - - # cd /etc/shorewall6 - # cp /usr/share/doc/shorewall6/examples/Universal/{interfaces,policy,rules,zones} . - Edit /etc/shorewall6/rules: -SECTION NEW -SSH(ACCEPT) all $FW -HTTPS(ACCEPT) all $FW - -Edit /etc/default/shorewall and /etc/default/shorewall6 and set - - startup=1 - -Restart servers. - - # /etc/init.d/ssh restart - # /etc/init.d/shorewall start - # /etc/init.d/shorewall6 start - -== Facilitator program installation - -Create the user the daemons will run as. - - # useradd --shell /usr/sbin/nologin --system --home /nonexistent -M facilitator-nobody - -Install the programs. - - # cd - # apt-get install git python-m2crypto make - # git clone https://git.torproject.org/flashproxy.git - # cd flashproxy/facilitator - # make install - -This installs facilitator.cgi, facilitator, facilitator-email-poller, -facilitator-reg-daemon, facilitator-reg, and fac.py to /usr/local/bin. -It also installs System V init files to /etc/init.d/. - -=== Registration daemon setup - -The facilitator-reg-daemon program requires a private RSA key (used to -decrypt encrypted client registrations). - - # mkdir /etc/flashproxy - # openssl genrsa -out /etc/flashproxy/reg-daemon.key 2048 - # chmod 600 /etc/flashproxy/reg-daemon.key - # openssl rsa -pubout < /etc/flashproxy/reg-daemon.key > reg-daemon.pub - -You will have to edit flashproxy-reg-email and copy the contents of -reg-daemon.pub into the appropriate place. - -Install reg-daemon.key /etc/flashproxy to match what the init script -expects. - -=== Init script setup - - # update-rc.d facilitator defaults - # /etc/init.d/facilitator start - # update-rc.d facilitator-reg-daemon defaults - # /etc/init.d/facilitator-reg-daemon start - -== HTTP server setup - -Apache is the web server that runs the CGI program. - - # apt-get install apache2 libapache2-mod-evasive - # a2enmod ssl headers - -Edit /etc/apache2/ports.conf and comment out the port 80 configuration. - - # NameVirtualHost *:80 - # Listen 80 - -Create a new file /etc/apache2/sites-available/fp-facilitator with -these contents: - MaxClients 256 - <VirtualHost *:443> - ServerName fp-facilitator.example.com - DocumentRoot /dev/null - ScriptAliasMatch ^(.*) /usr/local/bin/facilitator.cgi$1 - CustomLog /dev/null common - ErrorLog ${APACHE_LOG_DIR}/error.log - LogLevel warn - SSLEngine on - SSLCertificateFile /etc/apache2/fp-facilitator.pem - SSLCertificateChainFile /etc/apache2/intermediate.pem - Header add Strict-Transport-Security "max-age=15768000" - </VirtualHost> -intermediate.pem is a possible intermediate certificate file provided by -the CA; if you did not get one, then leave off the -SSLCertificateChainFile line. - -Link the configured site into sites-enabled. - # a2ensite fp-facilitator - -=== HTTPS setup - -The HTTP server should serve only over HTTPS and not unencrypted HTTP. -You will need a certificate and private key from a certificate -authority. An article on making a certificate signing request and -getting it signed is here: - http://www.debian-administration.org/articles/284 -This is the basic command to generate a CSR. - $ openssl req -new -nodes -out fp-facilitator.csr.pem -The instructions below assume you have an offline private key -in fp-facilitator.key.pem and a certificate in fp-facilitator.crt.pem. - -Make a file containing both the private key and a certificate. - - $ cat fp-facilitator.key.pem fp-facilitator.crt.pem > fp-facilitator.pem - $ chmod 400 fp-facilitator.pem - -Copy the new fp-facilitator.pem to the facilitator server as -/etc/apache2/fp-facilitator.pem. - - # /etc/init.d/apache2 restart - -=== Email poller setup - -The facilitator-email-poller program requires a password that is used to -log in to the designated Gmail account. See the file gmail-setup.txt for -instructions on setting up a Gmail account. After you've set up the -account and have the password, save it to a file reg-email.pass and make -it not readable or writable by anyone but its owner. - - # chmod 600 /etc/flashproxy/reg-email.pass - -Install reg-email.pass to /etc/flashproxy to match what the init script -expects. - - # update-rc.d facilitator-email-poller defaults - # /etc/init.d/facilitator-email-poller start diff --git a/doc/gmail-setup.txt b/doc/gmail-setup.txt deleted file mode 100644 index b51ce90..0000000 --- a/doc/gmail-setup.txt +++ /dev/null @@ -1,61 +0,0 @@ -These are instructions for setting up a Gmail account for use with the -email-based rendezvous and flashproxy-reg-email. These instructions were -current as of May 2013. - -You may have trouble if you are using Tor to create the account, for two -reasons. The first is that exit nodes are a source of abuse and Google -is more suspicious of them. The second is that Gmail is suspicious and -can lock you out of the account when your IP address is changing. While -setting up the account, use a single node in your torrc ExitNodes -configuration. Choose a U.S. exit node, one with low bandwidth. - -Go to https://mail.google.com/. Allow JavaScript to run (even from -youtube.com; it seems to be necessary). Click the "CREATE AN ACCOUNT" -button. - -Enter the account details. You don't need to fill in "Your current email -address". Enter a mobile phone number for later activation of two-factor -authentication. Solve the captcha. Click "Next Step". You may have to do -a phone SMS verification here. - -At this point the Gmail account is created. If you are pushed into -joining Google+, close everything out and go back to -https://mail.google.com/. - -Log out of the account and then back in again. There will be new text in -the lower right reading "Last account activity". Click "Details" and -turn off the unusual activity alerts. This will keep you from getting -locked out when you come from different IP addresses. At this point you -should remove the temporary ExitNodes configuration from torrc. - -Add a filter to prevent registrations from being marked as spam. Click -on the gear icon and select "Settings". Select "Filters" then "Create a -new filter". For "Has the words" type "in:spam", then "Create filter -with this search". There will be a warning that filters using "in:" will -never match incoming mail; this appears to be false and you can just -click OK. Check "Never send it to Spam" and click "Create filter". - -Enable IMAP. Click the gear icon, then "Settings", then "Forwarding and -POP/IMAP". - * Disable POP - * Enable IMAP - * Auto-Expunge on -Click "Save Changes". - -Enable two-factor authentication. We do this not so much for the -two-factor, but because it allows creating an independent password that -is used only for IMAP and does not have access to the web interface of -Gmail. Click the email address in the upper right, then "Account". Click -"Security". By "2-step verification" click "Edit". Click through until -it lets you set up. The phone number you provided when the account was -created will be automatically filled in. Choose "Text message (SMS)" -then click "Send code". Get your text message, type it in, and hit -"Verify". Uncheck "Trust this computer" on the next screen. Finally -"Confirm". On the following summary page, click "Show backup codes" and -save the codes to encrypted storage. Future codes can be generated at -https://www.google.com/accounts/SmsAuthConfig. - -Still on the 2-step summary page, click "Manage application-specific -passwords". Enter "IMAP" for the name and click "Generate password". -Save the password to encrypted storage. This is the password that will -be used with the --pass option of facilitator-email-poller. diff --git a/facilitator/appengine/README b/facilitator/appengine/README new file mode 100644 index 0000000..00d0611 --- /dev/null +++ b/facilitator/appengine/README @@ -0,0 +1,11 @@ +This is the server-side code that runs on Google App Engine for the +"appspot" registration method. + +See doc/appengine-howto.txt for information about setting up an +application. + +To run with the development server: +$ ~/google_appengine/dev_appserver.py appengine/ + +To upload a new version: +$ torify ~/google_appengine/appcfg.py update appengine/ diff --git a/facilitator/appengine/app.yaml b/facilitator/appengine/app.yaml new file mode 100644 index 0000000..3b99ada --- /dev/null +++ b/facilitator/appengine/app.yaml @@ -0,0 +1,9 @@ +application: fp-reg-a +version: 1 +runtime: go +api_version: go1 + +handlers: +- url: /.* + script: _go_app + secure: always diff --git a/facilitator/appengine/fp-reg.go b/facilitator/appengine/fp-reg.go new file mode 100644 index 0000000..d4f84c2 --- /dev/null +++ b/facilitator/appengine/fp-reg.go @@ -0,0 +1,54 @@ +package fp_reg + +import ( + "io" + "net" + "net/http" + "path" + + "appengine" + "appengine/urlfetch" +) + +const BASE = "https://fp-facilitator.org/reg/" + +func robotsTxtHandler(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + w.Write([]byte("User-agent: *\nDisallow:\n")) +} + +func ipHandler(w http.ResponseWriter, r *http.Request) { + remoteAddr := r.RemoteAddr + if net.ParseIP(remoteAddr).To4() == nil { + remoteAddr = "[" + remoteAddr + "]" + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + w.Write([]byte(remoteAddr)) +} + +func regHandler(w http.ResponseWriter, r *http.Request) { + dir, blob := path.Split(path.Clean(r.URL.Path)) + if dir != "/reg/" { + http.NotFound(w, r) + return + } + client := urlfetch.Client(appengine.NewContext(r)) + resp, err := client.Get(BASE + blob) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + for key, values := range resp.Header { + for _, value := range values { + w.Header().Add(key, value) + } + } + w.WriteHeader(resp.StatusCode) + io.Copy(w, resp.Body) +} + +func init() { + http.HandleFunc("/robots.txt", robotsTxtHandler) + http.HandleFunc("/ip", ipHandler) + http.HandleFunc("/reg/", regHandler) +} diff --git a/facilitator/doc/appengine-howto.txt b/facilitator/doc/appengine-howto.txt new file mode 100644 index 0000000..44c5324 --- /dev/null +++ b/facilitator/doc/appengine-howto.txt @@ -0,0 +1,56 @@ +These are instructions for how to set up a Google App Engine application +to run at appspot.com. + +General links: +https://developers.google.com/appengine/ +https://developers.google.com/appengine/docs/whatisgoogleappengine +https://developers.google.com/appengine/docs/go/gettingstarted/ + +You first need to create a Google account. See gmail-howto.txt for how +to do that. + +Download the SDK: +https://developers.google.com/appengine/docs/go/gettingstarted/devenvironment + +Write your program and app.yaml file: +https://developers.google.com/appengine/docs/go/gettingstarted/helloworld + +When you're ready to upload, log in with your Google account and follow +the directions: +https://appengine.google.com/ +https://developers.google.com/appengine/docs/go/gettingstarted/uploading + +Enter an application ID and create the application. + +Use the appcfg.py program to upload the program. It should look +something like this: + +$ torify ./google_appengine/appcfg.py update myapp/ +07:25 PM Host: appengine.google.com +07:25 PM Application: application-id; version: 1 +07:25 PM +Starting update of app: application-id, version: 1 +07:25 PM Getting current resource limits. +Email: xxx(a)gmail.com +Password for xxx(a)gmail.com: +07:26 PM Scanning files on local disk. +07:26 PM Cloning 2 application files. +07:26 PM Uploading 1 files and blobs. +07:26 PM Uploaded 1 files and blobs +07:26 PM Compilation starting. +07:26 PM Compilation: 1 files left. +07:26 PM Compilation completed. +07:26 PM Starting deployment. +07:26 PM Checking if deployment succeeded. +07:26 PM Deployment successful. +07:26 PM Checking if updated app version is serving. +07:26 PM Completed update of app: application-id, version: 1 + +Uploading the program in this way seems to create the files +~/.appcfg_nag and ~/.appcfg_cookies. Running the update command again +doesn't require you to enter your password again. + +Once logged in, you can disable logging for the application. Click +"Logs" on the left panel. Under "Total Logs Storage", click "Change +Settings". Enter "0" in the "days of logs" box and click "Save +Settings". diff --git a/facilitator/doc/facilitator-howto.txt b/facilitator/doc/facilitator-howto.txt new file mode 100644 index 0000000..8bafce9 --- /dev/null +++ b/facilitator/doc/facilitator-howto.txt @@ -0,0 +1,199 @@ +This document describes how to run a flash proxy facilitator on Debian 7. +We will use the domain name fp-facilitator.example.com. + +== Overview + +The facilitator consists of a backend server that is essentially a +dynamic database of client addresses, as well as helper programs that +receive client registrations from the Internet over various means and +pass them to the backend. There are three supported helper rendezvous +methods: HTTP, URL, and email. + +The HTTP rendezvous uses an HTTP server and a CGI program. The HTTP +server is responsible for speaking TLS and invoking the CGI program. The +CGI program receives client registrations and proxy requests for +clients, parses them, and forwards them to the backend. We use Apache 2 +as the HTTP server. The CGI script is facilitator.cgi. The backend is +facilitator. + +The email rendezvous uses the helper program facilitator-email-poller. +Clients use the flashproxy-reg-email program to send an encrypted +message to a Gmail address. The poller constantly checks for new +messages and forwards them to facilitator-reg. + +facilitator-reg is a simple program that forwards its standard input to +a locally running facilitator-reg-daemon process. + +facilitator-reg-daemon accepts connections containing encrypted client +registrations and forwards them to the facilitator. It exists as a +process of its own so that only one program requires access to the +facilitator's private key. + +The URL rendezvous uses the helper program flashproxy-reg-email. The +helper program doesn't actually make a registration; rather, it prints +out a URL which, when retrieved, makes the registration. + +fac.py is a Python module containing code common to the various +facilitator programs. + +== Basic and security setup + +Install some essential packages and configure a firewall. + + # cat >/etc/apt/apt.conf.d/90suggests<<EOF +APT::Install-Recommends "0"; +APT::Install-Suggests "0"; +EOF + # apt-get remove portmap + # apt-get update + # apt-get upgrade + # apt-get install shorewall shorewall6 + +Away from the facilitator, generate an SSH key for authentication: + + $ ssh-keygen -f ~/.ssh/fp-facilitator + $ ssh-copy-id -i ~/.ssh/fp-facilitator.pub root(a)fp-facilitator.example.com + +Then log in and edit /etc/ssh/sshd_config to disable password +authentication: + + PasswordAuthentication no + +Configure the firewall to allow only SSH and HTTPS. + + # cd /etc/shorewall + # cp /usr/share/doc/shorewall/examples/Universal/{interfaces,policy,rules,zones} . + Edit /etc/shorewall/rules: +SECTION NEW +SSH(ACCEPT) net $FW +HTTPS(ACCEPT) net $FW + + # cd /etc/shorewall6 + # cp /usr/share/doc/shorewall6/examples/Universal/{interfaces,policy,rules,zones} . + Edit /etc/shorewall6/rules: +SECTION NEW +SSH(ACCEPT) all $FW +HTTPS(ACCEPT) all $FW + +Edit /etc/default/shorewall and /etc/default/shorewall6 and set + + startup=1 + +Restart servers. + + # /etc/init.d/ssh restart + # /etc/init.d/shorewall start + # /etc/init.d/shorewall6 start + +== Facilitator program installation + +Create the user the daemons will run as. + + # useradd --shell /usr/sbin/nologin --system --home /nonexistent -M facilitator-nobody + +Install the programs. + + # cd + # apt-get install git python-m2crypto make + # git clone https://git.torproject.org/flashproxy.git + # cd flashproxy/facilitator + # make install + +This installs facilitator.cgi, facilitator, facilitator-email-poller, +facilitator-reg-daemon, facilitator-reg, and fac.py to /usr/local/bin. +It also installs System V init files to /etc/init.d/. + +=== Registration daemon setup + +The facilitator-reg-daemon program requires a private RSA key (used to +decrypt encrypted client registrations). + + # mkdir /etc/flashproxy + # openssl genrsa -out /etc/flashproxy/reg-daemon.key 2048 + # chmod 600 /etc/flashproxy/reg-daemon.key + # openssl rsa -pubout < /etc/flashproxy/reg-daemon.key > reg-daemon.pub + +You will have to edit flashproxy-reg-email and copy the contents of +reg-daemon.pub into the appropriate place. + +Install reg-daemon.key /etc/flashproxy to match what the init script +expects. + +=== Init script setup + + # update-rc.d facilitator defaults + # /etc/init.d/facilitator start + # update-rc.d facilitator-reg-daemon defaults + # /etc/init.d/facilitator-reg-daemon start + +== HTTP server setup + +Apache is the web server that runs the CGI program. + + # apt-get install apache2 libapache2-mod-evasive + # a2enmod ssl headers + +Edit /etc/apache2/ports.conf and comment out the port 80 configuration. + + # NameVirtualHost *:80 + # Listen 80 + +Create a new file /etc/apache2/sites-available/fp-facilitator with +these contents: + MaxClients 256 + <VirtualHost *:443> + ServerName fp-facilitator.example.com + DocumentRoot /dev/null + ScriptAliasMatch ^(.*) /usr/local/bin/facilitator.cgi$1 + CustomLog /dev/null common + ErrorLog ${APACHE_LOG_DIR}/error.log + LogLevel warn + SSLEngine on + SSLCertificateFile /etc/apache2/fp-facilitator.pem + SSLCertificateChainFile /etc/apache2/intermediate.pem + Header add Strict-Transport-Security "max-age=15768000" + </VirtualHost> +intermediate.pem is a possible intermediate certificate file provided by +the CA; if you did not get one, then leave off the +SSLCertificateChainFile line. + +Link the configured site into sites-enabled. + # a2ensite fp-facilitator + +=== HTTPS setup + +The HTTP server should serve only over HTTPS and not unencrypted HTTP. +You will need a certificate and private key from a certificate +authority. An article on making a certificate signing request and +getting it signed is here: + http://www.debian-administration.org/articles/284 +This is the basic command to generate a CSR. + $ openssl req -new -nodes -out fp-facilitator.csr.pem +The instructions below assume you have an offline private key +in fp-facilitator.key.pem and a certificate in fp-facilitator.crt.pem. + +Make a file containing both the private key and a certificate. + + $ cat fp-facilitator.key.pem fp-facilitator.crt.pem > fp-facilitator.pem + $ chmod 400 fp-facilitator.pem + +Copy the new fp-facilitator.pem to the facilitator server as +/etc/apache2/fp-facilitator.pem. + + # /etc/init.d/apache2 restart + +=== Email poller setup + +The facilitator-email-poller program requires a password that is used to +log in to the designated Gmail account. See the file gmail-setup.txt for +instructions on setting up a Gmail account. After you've set up the +account and have the password, save it to a file reg-email.pass and make +it not readable or writable by anyone but its owner. + + # chmod 600 /etc/flashproxy/reg-email.pass + +Install reg-email.pass to /etc/flashproxy to match what the init script +expects. + + # update-rc.d facilitator-email-poller defaults + # /etc/init.d/facilitator-email-poller start diff --git a/facilitator/doc/gmail-setup.txt b/facilitator/doc/gmail-setup.txt new file mode 100644 index 0000000..b51ce90 --- /dev/null +++ b/facilitator/doc/gmail-setup.txt @@ -0,0 +1,61 @@ +These are instructions for setting up a Gmail account for use with the +email-based rendezvous and flashproxy-reg-email. These instructions were +current as of May 2013. + +You may have trouble if you are using Tor to create the account, for two +reasons. The first is that exit nodes are a source of abuse and Google +is more suspicious of them. The second is that Gmail is suspicious and +can lock you out of the account when your IP address is changing. While +setting up the account, use a single node in your torrc ExitNodes +configuration. Choose a U.S. exit node, one with low bandwidth. + +Go to https://mail.google.com/. Allow JavaScript to run (even from +youtube.com; it seems to be necessary). Click the "CREATE AN ACCOUNT" +button. + +Enter the account details. You don't need to fill in "Your current email +address". Enter a mobile phone number for later activation of two-factor +authentication. Solve the captcha. Click "Next Step". You may have to do +a phone SMS verification here. + +At this point the Gmail account is created. If you are pushed into +joining Google+, close everything out and go back to +https://mail.google.com/. + +Log out of the account and then back in again. There will be new text in +the lower right reading "Last account activity". Click "Details" and +turn off the unusual activity alerts. This will keep you from getting +locked out when you come from different IP addresses. At this point you +should remove the temporary ExitNodes configuration from torrc. + +Add a filter to prevent registrations from being marked as spam. Click +on the gear icon and select "Settings". Select "Filters" then "Create a +new filter". For "Has the words" type "in:spam", then "Create filter +with this search". There will be a warning that filters using "in:" will +never match incoming mail; this appears to be false and you can just +click OK. Check "Never send it to Spam" and click "Create filter". + +Enable IMAP. Click the gear icon, then "Settings", then "Forwarding and +POP/IMAP". + * Disable POP + * Enable IMAP + * Auto-Expunge on +Click "Save Changes". + +Enable two-factor authentication. We do this not so much for the +two-factor, but because it allows creating an independent password that +is used only for IMAP and does not have access to the web interface of +Gmail. Click the email address in the upper right, then "Account". Click +"Security". By "2-step verification" click "Edit". Click through until +it lets you set up. The phone number you provided when the account was +created will be automatically filled in. Choose "Text message (SMS)" +then click "Send code". Get your text message, type it in, and hit +"Verify". Uncheck "Trust this computer" on the next screen. Finally +"Confirm". On the following summary page, click "Show backup codes" and +save the codes to encrypted storage. Future codes can be generated at +https://www.google.com/accounts/SmsAuthConfig. + +Still on the 2-step summary page, click "Manage application-specific +passwords". Enter "IMAP" for the name and click "Generate password". +Save the password to encrypted storage. This is the password that will +be used with the --pass option of facilitator-email-poller.

1 0

[stem/master] Tagging version 1.1 changelog
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit a5392292c829190940d43c34690b0d08e6eaaf1c Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Oct 13 18:37:04 2013 -0700 Tagging version 1.1 changelog Revising our changelog to list the changes in version 1.1. Also re-adding the '-dev' suffix to our git version. --- docs/change_log.rst | 5 +++++ stem/__init__.py | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/change_log.rst b/docs/change_log.rst index 334f995..62c9ea3 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -38,6 +38,11 @@ Unreleased The following are only available within stem's `git repository <download.html>`_. +.. _version_1.1: + +Version 1.1 +----------- + * **Controller** * :func:`~stem.control.Controller.get_network_status` and :func:`~stem.control.Controller.get_network_statuses` now provide v3 rather than v2 directory information (:trac:`7953`, :spec:`d2b7ebb`) diff --git a/stem/__init__.py b/stem/__init__.py index 5abe6a7..e87bd9f 100644 --- a/stem/__init__.py +++ b/stem/__init__.py @@ -368,7 +368,7 @@ Library for working with the tor process. =============== =========== """ -__version__ = '1.1.0' +__version__ = '1.1.0-dev' __author__ = 'Damian Johnson' __contact__ = 'atagar(a)torproject.org' __url__ = 'https://stem.torproject.org/'

1 0

[stem/master] Stem release 1.1.0
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit 2ef492467d13a121a76ac8ebfdc2dfbaa4b20e1d Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Oct 13 18:15:41 2013 -0700 Stem release 1.1.0 --- stem/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stem/__init__.py b/stem/__init__.py index 5b9a975..5abe6a7 100644 --- a/stem/__init__.py +++ b/stem/__init__.py @@ -368,7 +368,7 @@ Library for working with the tor process. =============== =========== """ -__version__ = '1.0.1-dev' +__version__ = '1.1.0' __author__ = 'Damian Johnson' __contact__ = 'atagar(a)torproject.org' __url__ = 'https://stem.torproject.org/'

1 0

[flashproxy/master] Change the client transport method name from "websocket" to "flashproxy".
by dcf＠torproject.org 14 Oct '13

14 Oct '13

commit c47025cd9260a77d65a7ca41f5f806a87c1baf27 Author: David Fifield <david(a)bamsoftware.com> Date: Sun Oct 13 17:21:11 2013 -0700 Change the client transport method name from "websocket" to "flashproxy". Both names still work and are equivalent. This only affects what string appears in the configuration files we ship. --- ChangeLog | 10 ++++++++++ README | 6 +++--- doc/design.txt | 4 ++-- experiments/exercise/exercise.sh | 2 +- torrc | 4 ++-- 5 files changed, 18 insertions(+), 8 deletions(-) diff --git a/ChangeLog b/ChangeLog index ff115cf..8707b89 100644 --- a/ChangeLog +++ b/ChangeLog @@ -16,6 +16,16 @@ assign proxies to clients that use matching transports. Patch by George Kadianakis. Part of bug 9349. + o Changed to use the pluggable transport method name "flashproxy" + rather than "websocket". Both names are equivalent and "websocket" + continues to work. The reason for this change is to reduce confusion + with a transport that simply makes a WebSocket connection to a + "websocket" bridge, without receiving an inbound connection from a + flash proxy. The default argument to the --transport option + continues to be "websocket", because that option controls which + particular protocol flash proxies should use to connect to you, and + is distinct from the transport method name used by Tor. + Changes in version 1.3 o Added a new observed Google public key pin. diff --git a/README b/README index ca93ce8..aa1dc6a 100644 --- a/README +++ b/README @@ -19,11 +19,11 @@ By default the transport plugin listens on Internet-facing TCP port 9000. If you have to use a different port (to get through a firewall, for example), edit the ClientTransportPlugin line of the torrc to give a different port number: - ClientTransportPlugin websocket exec ./flashproxy-client --register :0 :8888 + ClientTransportPlugin flashproxy exec ./flashproxy-client --register :0 :8888 If the flashproxy-client program is in a different directoy (after being installed, for example), use the full path in the ClientTransportPlugin line: - ClientTransportPlugin websocket exec /usr/local/bin/flashproxy-client --register + ClientTransportPlugin flashproxy exec /usr/local/bin/flashproxy-client --register You should receive a flash proxy connection within about 60 seconds. See "Troubleshooting" below if it doesn't work. @@ -69,7 +69,7 @@ public instance of it. This way is not as realistic because all your Tor traffic will first go to a fixed address and can be easily blocked. However this is an easy way to try out the system without having to do port forwarding. - $ tor ClientTransportPlugin "websocket socks4 fp-facilitator.org:9999" UseBridges 1 Bridge "websocket 0.0.1.0:1" LearnCircuitBuildTimeout 0 CircuitBuildTimeout 60 + $ tor ClientTransportPlugin "flashproxy socks4 fp-facilitator.org:9999" UseBridges 1 Bridge "flashproxy 0.0.1.0:1" LearnCircuitBuildTimeout 0 CircuitBuildTimeout 60 == Troubleshooting diff --git a/doc/design.txt b/doc/design.txt index bc0f377..20b63b7 100644 --- a/doc/design.txt +++ b/doc/design.txt @@ -100,9 +100,9 @@ Design of flash proxies The Tor client must be configured to make its connections through a local proxy (the client transport plugin). This configuration is sufficient: - ClientTransportPlugin websocket socks4 127.0.0.1:9001 + ClientTransportPlugin flashproxy socks4 127.0.0.1:9001 UseBridges 1 - Bridge websocket 0.0.1.0:1 + Bridge flashproxy 0.0.1.0:1 LearnCircuitBuildTimeout 0 The address given for the "Bridge" option is actually irrelevant. The client transport plugin will ignore it and connect (through the flash diff --git a/experiments/exercise/exercise.sh b/experiments/exercise/exercise.sh index 0bc49c0..efefde7 100755 --- a/experiments/exercise/exercise.sh +++ b/experiments/exercise/exercise.sh @@ -26,7 +26,7 @@ PIDS_TO_KILL+=($!) sleep 20 -"$TOR" ClientTransportPlugin "websocket socks4 127.0.0.1:$LOCAL_PORT" UseBridges 1 Bridge "websocket 0.0.1.0:1" LearnCircuitBuildTimeout 0 CircuitBuildTimeout 60 & +"$TOR" ClientTransportPlugin "flashproxy socks4 127.0.0.1:$LOCAL_PORT" UseBridges 1 Bridge "flashproxy 0.0.1.0:1" LearnCircuitBuildTimeout 0 CircuitBuildTimeout 60 & PIDS_TO_KILL+=($!) sleep 60 diff --git a/torrc b/torrc index e047cd0..4a66162 100644 --- a/torrc +++ b/torrc @@ -4,10 +4,10 @@ UseBridges 1 # The address and port are ignored by the client transport plugin. -Bridge websocket 0.0.1.0:1 +Bridge flashproxy 0.0.1.0:1 LearnCircuitBuildTimeout 0 CircuitBuildTimeout 60 # Change the second number here (9000) to the number of a port that can # receive connections from the Internet (the port for which you # configured port forwarding). -ClientTransportPlugin websocket exec ./flashproxy-client --register :0 :9000 +ClientTransportPlugin flashproxy exec ./flashproxy-client --register :0 :9000

1 0

[stem/master] Using @lru_cache for exit policy's str()
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit 1dfa353ecfbef39ee606231e745b9b6318fd3a31 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Oct 12 17:35:55 2013 -0700 Using @lru_cache for exit policy's str() Oops, missed one of our instances of manual caching. --- stem/exit_policy.py | 65 +++++++++++++++++++++++---------------------------- 1 file changed, 29 insertions(+), 36 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index ee13308..a00665e 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -499,10 +499,6 @@ class ExitPolicyRule(object): self._apply_addrspec(rule, addrspec) self._apply_portspec(rule, portspec) - # Lazily loaded string representation of our policy. - - self._str_representation = None - # If true then a submask of /0 is treated by is_address_wildcard() as being # a wildcard. @@ -642,6 +638,7 @@ class ExitPolicyRule(object): return self._masked_bits + @lru_cache() def __str__(self): """ Provides the string representation of our policy. This does not @@ -651,42 +648,39 @@ class ExitPolicyRule(object): to re-create this rule. """ - if self._str_representation is None: - label = "accept " if self.is_accept else "reject " + label = "accept " if self.is_accept else "reject " - if self.is_address_wildcard(): - label += "*:" - else: - address_type = self.get_address_type() - - if address_type == AddressType.IPv4: - label += self.address - else: - label += "[%s]" % self.address - - # Including our mask label as follows... - # - exclude our mask if it doesn't do anything - # - use our masked bit count if we can - # - use the mask itself otherwise - - if (address_type == AddressType.IPv4 and self._masked_bits == 32) or \ - (address_type == AddressType.IPv6 and self._masked_bits == 128): - label += ":" - elif self._masked_bits is not None: - label += "/%i:" % self._masked_bits - else: - label += "/%s:" % self.get_mask() + if self.is_address_wildcard(): + label += "*:" + else: + address_type = self.get_address_type() - if self.is_port_wildcard(): - label += "*" - elif self.min_port == self.max_port: - label += str(self.min_port) + if address_type == AddressType.IPv4: + label += self.address + else: + label += "[%s]" % self.address + + # Including our mask label as follows... + # - exclude our mask if it doesn't do anything + # - use our masked bit count if we can + # - use the mask itself otherwise + + if (address_type == AddressType.IPv4 and self._masked_bits == 32) or \ + (address_type == AddressType.IPv6 and self._masked_bits == 128): + label += ":" + elif self._masked_bits is not None: + label += "/%i:" % self._masked_bits else: - label += "%i-%i" % (self.min_port, self.max_port) + label += "/%s:" % self.get_mask() - self._str_representation = label + if self.is_port_wildcard(): + label += "*" + elif self.min_port == self.max_port: + label += str(self.min_port) + else: + label += "%i-%i" % (self.min_port, self.max_port) - return self._str_representation + return label @lru_cache() def _get_mask_bin(self): @@ -821,7 +815,6 @@ class MicroExitPolicyRule(ExitPolicyRule): self.address = None # wildcard address self.min_port = min_port self.max_port = max_port - self._str_representation = None def is_address_wildcard(self): return True

1 0

[stem/master] Adding hash functions to exit policy classes
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit 4ef5ee1ee8fbecabeaee872f2681cdda1fb86584 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Oct 12 20:18:12 2013 -0700 Adding hash functions to exit policy classes Huh, didn't expect that. To work the @lru_cache requires that all arguments (including the class in the ase of methods) are hashable. This makes perfectly sense, after all the cache is a 'argument => cached value' dictionary. Under python 2.x all seemed to be well, but under python 3.x the @lru_cache complained that our classes weren't hashable (which... well, they aren't). Why, then, did it work under python 2.x? Turns out there's a subtle difference where our object parent provides a __hash__ method for our id, but in python 3.x it's a little different (object still does, but for reasons I haven't been able to figure out it doesn't for our classes). The ExitPolicyRule can certainly be hashable. The ExitPolicy, however, is a lot trickier due to its lazy loading of rules. For now just opting for the same behavior as python 2.x and using the address. --- stem/exit_policy.py | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index 9366a54..a358908 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -349,6 +349,14 @@ class ExitPolicy(object): def __str__(self): return ', '.join([str(rule) for rule in self._get_rules()]) + def __hash__(self): + # TODO: It would be nice to provide a real hash function, but doing so is + # tricky due to how we lazily load the rules. Like equality checks a proper + # hash function would need to call _get_rules(), but that's behind + # @lru_cache which calls hash() forming a circular dependency. + + return id(self) + def __eq__(self, other): if isinstance(other, ExitPolicy): return self._get_rules() == list(other) @@ -429,6 +437,9 @@ class MicroExitPolicy(ExitPolicy): def __str__(self): return self._policy + def __hash__(self): + return id(self) + def __eq__(self, other): if isinstance(other, MicroExitPolicy): return str(self) == str(other) @@ -682,6 +693,22 @@ class ExitPolicyRule(object): return label + def __hash__(self): + my_hash = 0 + + for attr in ("is_accept", "address", "min_port", "max_port"): + my_hash *= 1024 + + attr_value = getattr(self, attr) + + if attr_value is not None: + my_hash += hash(attr_value) + + my_hash *= 1024 + my_hash += hash(self.get_mask(False)) + + return my_hash + @lru_cache() def _get_mask_bin(self): # provides an integer representation of our mask @@ -792,7 +819,7 @@ class ExitPolicyRule(object): # 0.0.0.0/0" == "accept 0.0.0.0/0.0.0.0" will be True), but these # policies are effectively equivalent. - return str(self) == str(other) + return hash(self) == hash(other) else: return False @@ -827,3 +854,16 @@ class MicroExitPolicyRule(ExitPolicyRule): def get_masked_bits(self): return None + + def __hash__(self): + my_hash = 0 + + for attr in ("is_accept", "min_port", "max_port"): + my_hash *= 1024 + + attr_value = getattr(self, attr) + + if attr_value is not None: + my_hash += hash(attr_value) + + return my_hash

1 0

[stem/master] Python 3 lru_cache isn't a collections annotation
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit 677ee1570cc1378b8abadb4071627ff53510cb8a Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Oct 12 19:23:04 2013 -0700 Python 3 lru_cache isn't a collections annotation Oops, it's in functools, not collections. --- stem/descriptor/extrainfo_descriptor.py | 2 +- stem/descriptor/microdescriptor.py | 2 +- stem/descriptor/server_descriptor.py | 2 +- stem/exit_policy.py | 2 +- stem/prereq.py | 2 +- stem/util/proc.py | 2 +- stem/version.py | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/stem/descriptor/extrainfo_descriptor.py b/stem/descriptor/extrainfo_descriptor.py index 0d50f75..b13efff 100644 --- a/stem/descriptor/extrainfo_descriptor.py +++ b/stem/descriptor/extrainfo_descriptor.py @@ -84,7 +84,7 @@ from stem.descriptor import ( try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/descriptor/microdescriptor.py b/stem/descriptor/microdescriptor.py index c48419a..1f9e8cb 100644 --- a/stem/descriptor/microdescriptor.py +++ b/stem/descriptor/microdescriptor.py @@ -77,7 +77,7 @@ from stem.descriptor import ( try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/descriptor/server_descriptor.py b/stem/descriptor/server_descriptor.py index 51b382f..160dab7 100644 --- a/stem/descriptor/server_descriptor.py +++ b/stem/descriptor/server_descriptor.py @@ -54,7 +54,7 @@ from stem.descriptor import ( try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/exit_policy.py b/stem/exit_policy.py index a00665e..9366a54 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -65,7 +65,7 @@ import stem.util.str_tools try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/prereq.py b/stem/prereq.py index 22b5819..df2e10d 100644 --- a/stem/prereq.py +++ b/stem/prereq.py @@ -24,7 +24,7 @@ import sys try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/util/proc.py b/stem/util/proc.py index fbff14c..95e75de 100644 --- a/stem/util/proc.py +++ b/stem/util/proc.py @@ -55,7 +55,7 @@ from stem.util import log try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache diff --git a/stem/version.py b/stem/version.py index bf41898..eca03cf 100644 --- a/stem/version.py +++ b/stem/version.py @@ -64,7 +64,7 @@ import stem.util.system try: # added in python 3.2 - from collections import lru_cache + from functools import lru_cache except ImportError: from stem.util.lru_cache import lru_cache

1 0

[stem/master] Fixing a couple python 3 network status document TypeErrors
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit e1bf7cbacb8797845c1bc2a778baa93d9748f4c9 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Oct 13 12:10:38 2013 -0700 Fixing a couple python 3 network status document TypeErrors ====================================================================== ERROR: test_duplicate_fields ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/data/python3/test/unit/descriptor/networkstatus/document_v3.py", line 247, in test_duplicate_fields if not is_consensus and lines[index].startswith('dir-source'): TypeError: startswith first arg must be bytes or a tuple of bytes, not str ====================================================================== ERROR: test_misordered_fields ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/data/python3/test/unit/descriptor/networkstatus/document_v3.py", line 225, in test_misordered_fields elif not is_consensus and lines[index].startswith('dir-source'): TypeError: startswith first arg must be bytes or a tuple of bytes, not str ---------------------------------------------------------------------- Ran 35 tests in 0.234s --- test/unit/descriptor/networkstatus/document_v3.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/unit/descriptor/networkstatus/document_v3.py b/test/unit/descriptor/networkstatus/document_v3.py index 71ef8b6..4c82617 100644 --- a/test/unit/descriptor/networkstatus/document_v3.py +++ b/test/unit/descriptor/networkstatus/document_v3.py @@ -222,7 +222,7 @@ class TestNetworkStatusDocument(unittest.TestCase): if is_consensus and lines[index].startswith(stem.util.str_tools._to_bytes(CRYPTO_BLOB[1:10])): break - elif not is_consensus and lines[index].startswith('dir-source'): + elif not is_consensus and lines[index].startswith(b'dir-source'): break # swaps this line with the one after it @@ -244,7 +244,7 @@ class TestNetworkStatusDocument(unittest.TestCase): lines = get_network_status_document_v3(attr, content = True).split(b"\n") for index, line in enumerate(lines): - if not is_consensus and lines[index].startswith('dir-source'): + if not is_consensus and lines[index].startswith(b'dir-source'): break # Stop when we hit the 'directory-signature' for a couple reasons...

1 0

[stem/master] Fixing python 3 issue concerning exit policy compression
by atagar＠torproject.org 14 Oct '13

14 Oct '13

commit 208c03cded57507f835265d7e4d2b4677d0e30e5 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Oct 12 20:07:52 2013 -0700 Fixing python 3 issue concerning exit policy compression Oops, forgot about the byte vs unicode differences when doing compression for the ExitPolicy. --- stem/exit_policy.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index a358908..9c60bda 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -300,13 +300,16 @@ class ExitPolicy(object): rules = [] is_all_accept, is_all_reject = True, True - if isinstance(self._input_rules, str): - decompressed_rules = zlib.decompress(self._input_rules).split(',') + if isinstance(self._input_rules, bytes): + decompressed_rules = zlib.decompress(self._input_rules).split(b',') else: decompressed_rules = self._input_rules for rule in decompressed_rules: - if isinstance(rule, (bytes, unicode)): + if isinstance(rule, bytes): + rule = stem.util.str_tools._to_unicode(rule) + + if isinstance(rule, unicode): rule = ExitPolicyRule(rule.strip()) if rule.is_accept:

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits October 2013