- tor-dev - lists.torproject.org

[Measurement Team] Next IRC meeting happens on Wednesday, Aug 12, 14:00 UTC in #tor-project
by Karsten Loesing 11 Aug '15

11 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, August 12, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150812T14 Suggested agenda is: - 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) - Priorities: should we focus more on a certain aspect of measurements in the Tor network, while temporarily suspending or even permanently dropping other aspects? Let's talk about how we can add such priorities to the roadmap draft we have been working on [0]. - Misc/discussion: whatever comes up during the meeting If you think this can be fun, please join us at the meeting! Talk to you tomorrow! All the best, Karsten [0] https://people.torproject.org/~karsten/volatile/measurement-roadmap.pdf -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJVyhwAAAoJEJD5dJfVqbCrjkMIAMCUpfVBIw2GWaPLMgW9MSFo 1ANENAwisD86f2r+ypRtBtK28TDN4d3cauACkqY8zLHs4ZzghO0f58Ob9ZhGgHei jkOM2i/M/yBEZehAit2kYoeND8gfWdW2vj5HufFF2khw+xFsL1+oQgVyurfdMQ3/ TfI1iWBuv5NSTQx4h/W64DJ9vagwYiLNvPrI2TbO2hhOiEKS1nIMZ3vYjI2V3Yff 0rjGsTd3WMm2AIarbJtVoeFW2jgqeweSFR5M+aAYdsRRKH/nRdcl6ToVPro+8uih I3GRCFJpEIh1CHJSGvfLuazJ88W1INYf/LQiOkEAx8CiSshBHar+0JpWOMpsrfc= =ZC0R -----END PGP SIGNATURE-----

1 0

Tor's default behavior for ed25519 identities
by s7r 11 Aug '15

11 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Tor 0.2.7.x will support Ed25519 router identities along with the traditional 1024-bit RSA ones which will be used simultaneously for some time, until we will completely deprecate RSA router identities. I would like to document what Tor needs to do (or don't do) by default in some cases to prevent mistakes which confused operators could do. Please review and add/edit/remove if something is wrong, doesn't make sense or is incomplete. When finished, I will split it in more parts and open tickets on trac based on priorities. Some cases are already covered (Tor does what it should by default). We also need to compose the messages Tor will print in the log files in a way that they will be of a reasonable length and expressive/explicit/polite/whatever. Take in consideration the design: For an Ed25519 identity, Tor will generate these files in $datadirectory/keys: - - ed25519_master_id_secret_key - the master identity key. Optionally this can be kept offline or encrypted. This is the file one needs in case he wants to restore a relay. Never expires. Sensitive (if reached it will compromise the identity of the relay forever). - - ed25519_master_id_public_key - the public key. This is the actual router identity which Tor publishes in the descriptors. It is computed from the secret key and it is not sensitive. Never expires. - - ed25519_signing_secret_key - a temporary signing key. This will be used for circuits (onionskin) encryption. It is valid only temporary for as long as ed25519_signing_cert says so. - - ed25519_signing_cert - a cert signed by the master ID key (ed25519_master_id_secret_key) attesting that the ed25519_signing_secret_key 'x' is valid for 'n' days/months. The validity period can be configured with the torrc option SigningKeyLifetime 'n' days/weeks/months (default is 30 days). Also take in consideration that the directory authorities will pin relays with paired RSA-Ed25519 identities and reject them if one changes. For example, if relay A has ID RSA_A & Ed25519_A, and later changes to RSA_B & Ed25519_A or RSA_A & Ed25519_B, it will be rejected. The text is also available on Etherpad: https://etherpad.mozilla.org/efIBq6nDwt - ------------------------------------------------------ Ed25519 router identity - default behavior - ------------------------------------------------------ We don't touch the RSA ID key behavior. We let it as it was, use it if secret ID file is there, generate new one if not. The behavior below should be implemented by taking in consideration only the ed25519_* files in $datadirectory/keys (should not care or notice the RSA related files). We don't say anything about offline Ed25519 ID keys or encrypted Ed25519 ID keys. Whoever wants to use these features will have to read the FAQ first and understand some things. The default is: generate ed25519_master_id_secret_key (unencrypted) in $datadirectory/keys and use that forever, by automatically renewing ed25519_signing_secret_key and ed25519_signing_cert with the default validity period of 30 days, unless otherwise specified in torrc. This is already implemented and it is how Tor works now. At every start, restart, reload, server reboot, signal sent by user or another app, etc. Tor should: 1) check if file ed25519_master_id_secret_key exists in $datadirectory/keys and it's a valid curve25519 private key file; 2) check if file ed25519_master_id_secret_key_encrypted exists in $datadirectory/keys; 3) check if file ed25519_master_id_public_key exists in $datadirectory/keys and it's a valid curve25519 public key file; 4) check if file ed25519_signing_cert exists in $datadirectory/keys; 5) check if ed25519_signing_cert in $datadirectory/keys is expired; 6) check if ed25519_signing_cert correlates with ed25519_master_id_public_key; 7) check if file ed25519_signing_secret_key exists in $datadirectory/keys; 8) check if ed25519_signing_secret_key correlates with ed25519_signing_cert (non-expired cert refers to that signing key); 1. If ed25519_master_id_secret_key && ed25519_master_id_public_key && ed25519_signing_secret_key && ed25519_signing_cert are all missing from $datadirectory/keys: - -> We assume it's a new relay or a relay which just upgraded to a Tor that supports Ed25519 router identities. Generate a new Ed25519 identity unencrypted (ed25519_master_id_secret_key, ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert with a validity for 30 days unless specified otherwise in torrc) and publish a descriptor using that, proceed as normal after. 2. If ed25519_master_id_secret_key is missing or exists encrypted && ed25519_master_id_public_key exists in $datadirectory/keys and ed25519_signing_secret_key and ed25519_signing_cert are expired: - -> We assume this can happen suddenly, while running unattended. Disable relay functionality and stop publishing descriptors (close ORPort and (if open) DirPort). Leave client functionality since that could be used by something else on that machine and it shouldn't be affected. Print warnings in the log file that action is required (generate new ed25519_signing_secret_key and ed25519_signing_cert or move the ed25519_master_id_secret_key unencrypted in $datadirectory/keys so Tor can do this by itself). The operator needs to signal Tor with a reload signal (HUP) or restart/stop & start after completing the action. Tor will check for the valid ed25519_signing_secret_key and ed25519_signing_cert by itself every time server restarts (along with Tor process) or at every automated reload signal (HUP). 3. If ed25519_master_id_secret_key exists unencrypted in $datadirectory/keys and there is no ed25519_master_id_public_key, ed25519_signing_secret_key or ed25519_signing_cert (or ed25519_signing_secret_key and ed25519_signing_cert are expired): - -> We assume the relay has just been restored and the Ed25519 identity was backed up. Automatically generate ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert with a validity period of 30 days (unless otherwise specified in torrc). - -> Since we have the ed25519_master_id_secret_key unencrypted, and at startup we always check for availability/validity of the other ed25519* files by default anyway, we should generate automatically in this case: ed25519_master_id_public_key (if missing), ed25519_signing_secret_key (if missing or expired) and ed25519_signing_cert (if missing or expired). 4. If ed25519_master_id_secret_key exists encrypted in $datadirectory/keys && there is no ed25519_master_id_public_key, ed25519_signing_secret_key or ed25519_signing_cert (even if just one of these 3 is missing): - -> We assume the relay identity existed previously and ed25519_master_id_secret_key_encrypted was restored form a backup. Do not start Tor at all and print explicit warnings that action is required - Tor cannot decrypt the ed25519_master_id_secret_key_encrypted file. Generate ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert or decrypt the ed25519_master_id_secret_key_encrypted in $datadirectory/keys so Tor can do it by itself. 5. If the chain validation fails (ed25519_signing_cert doesn't correlate to ed25519_master_id_public_key AND/OR ed25519_signing_secret_key doesn't correlate to ed25519_signing_cert): - -> We assume the operator probably runs many relays and holds many ed25519_master_id_secret_key files maybe accidentally mixes them and generates temporary signing key and key cert using the master key from another relay. Do no start Tor service at all and print warnings: the ed25519_signing_secret_key and ed25519_signing_cert are generated with the ed25519_master_id_secret_key of another relay. These files should be removed and replaced with the correct ones (search for the ed25519_master_id_secret_key which belongs to this relay). Otherwise it's useless (since we already reject RSA-Ed25519 which don't match), if the ed25519_master_id_secret_key is lost permanently, nothing else for the operator to do but to manually generate fresh identity both RSA and Ed25519 so the relay can be part of the consensus (start from scratch). 6. If ed25519_master_id_secret_key is missing or _encrypted AND ed25519_master_id_public_key is also missing from $datadirectory/keys AND a) we have an ed25519_signing_secret_key and ed25519_signing_cert which are not expired: - ->Automatically compute ed25519_master_public_key from the ed25519_signing_cert and save it in $datadirectory/keys. b) if ed25519_signing_secret_key and ed25519_signing_cert are expired: - ->Do no start Tor service at all and print warnings that user action is required (generate new ed25519_signing_secret_key and ed25519_signing_cert or move the ed25519_master_id_secret_key unencrypted in $datadirectory/keys so Tor can do this by itself). The operator needs to signal Tor with a reload signal (HUP) or restart/stop & start after completing the action. Tor will check for the valid ed25519_signing_secret_key and ed25519_signing_cert by itself every time server restarts (along with Tor process) or at every automated reload signal (HUP). - ------------------------------------------------------ Commands related to Ed25519 keys - ------------------------------------------------------ 1. tor --keygen should be replaced with tor --ecckeygen (keygen is not explicit enough since we are talking about 2 router identities which are used simultaneously [RSA and Ed25519]). - - Ask if user wants to encrypt the key with a passphrase and if yes ask 2 passphrase confirmations. Only allow ASCII, check and return invalid passphrase otherwise (local language keyboards mess things up). I suggest the Encrypt key? question [y/N] should be a No by default. - - Unless user specifies --out /path/to/dir to save the output files, save them in working directory (where the command is run). Do not create the folder and save them in $HOME/.tor. - - Only generate an ed25519_master_id_secret_key with this command (no ed25519_master_id_public_key, ed25519_signing_secret_key or ed25519_signing_cert). Print a message confirming this: ed25519_master_id_secret_key was successfully generated and saved encrypted/unencrypted (depending how it was generated) in $location ($location = either working directory where the command was run either the location specified separately with --out argument included in the - --ecckeygen command). Append _encrypted at the end of the file name in case a passphrase was provided. 2. tor --encryptkey <ed25519_master_id_secret_key file> or /path/to/<key file> - - check if it's a valid ed25519_master_id_secret_key unencrypted, ask for 2 passphrase confirmations and encrypt it. Only allow ASCII, check and return invalid passphrase otherwise (local language keyboards mess things up). - - ask if the user wants to delete the old plaintext ed25519_master_id_secret_key and only remain with the encrypted copy. [Y/n], Yes by default. Delete it in a secure way if yes, obviously. - - save the encrypted key in the working directory where the command is run or with an argument --out /path/to/dir. Print a message confirming this: ed25519_master_id_secret_key was successfully encrypted and saved in $location ($location = either working directory where the command was run either the location specified separately with --out argument included in the --encryptkey command). 3. tor --decryptkey <ed25519_master_id_secret_key file> or /path/to/<key file> - - ask for confirmation: Are you sure you wan to decrypt your ed25519_master_id_secret_key? This will save the master ID key in plaintext, not protected by a passphrase. [Y/n], Yes by default. - - ask for the passphrase and decrypt. - - ask if the user wants to delete the old encrypted ed25519_master_id_secret_key and only remain with the unencrypted copy of it. [y/N], No by default. - - save the plaintext key in the working directory where the command is run or with an argument --out /path/to/dir. Print a message confirming this: ed25519_master_id_secret_key was successfully decrypted permanently and saved in $location ($location = either working directory where the command was run either the location specified separately with --out argument included in the --decryptkey command). 4. tor --newpasskey <ed25519_master_id_secret_key file> or /path/to/<key file> - - ask for the current passphrase. - - ask for the new passphrase 2 times. Only allow ASCII, check and return invalid passphrase otherwise (local language keyboards mess things up). - - update the ed25519_master_id_secret_key file with then new passphrase in place, don't save a separate new copy with this command. - - print a message confirming that the passphrase has been successfully and permanently changed. 5. tor --gensignkey <ed25519_master_id_secret_key file> or /path/to/<key file> - - ask for the passphrase if key file is encrypted. - - generate new ed25519_signing_secret_key and ed25519_signing_cert valid for 30 days unless specified otherwise in torrc. If there is a SigningKeyLifetime in torrc, read it and ask the user: The signing key lifetime in your torrc is n days/weeks/months, use this validity period? [Y/n], yes default. If user enters N, ask the user to manually enter the lifetime. - - save the output files in the working directory where the command is run or with an argument --out /path/to/dir. Print a message which explains that these files need to be moved in Tor's $datadirectory/keys. - - Print a message confirming this: ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert with a validity period of $validity have been successfully created and saved in $location. Please move them to your Tor $datadirectory/keys folder and make sure Tor can access/read them. ($validity = SigningKeyLifetime in torrc or lifetime manually specified by the user or 30 days, the default if there is no torrc setting for this and no input from the user; $location = either working directory where the command was run either the location specified separately with --out argument included in the --gensignkey command). 6. tor --getpubkey <ed25519_master_id_secret_key file> or /path/to/<key file> - - ask for the passphrase if key file is encrypted - - compute the ed2551_master_id_public_key - - save the output file ed25519_master_id_public_key in working directory where the command is run or with an argument --out /path/to/dir. - - Print a message confirming that the ed25519_master_id_public_key has been saved in $location ($location = either working directory where the command was run either the location specified separately with - --out argument included in the --getpubkey command). - ------------------------------------------------------ Permissions - a thing many users don't know/care about: - - If Tor cannot complete a command from above because it does not have permission to read the ed25519_master_id_secret_key print an error message and instruct the user that he should run the tor command as the rightful user or change permissions over the ed25519_master_id_secret_key file. - - If Tor cannot save the output files resulted from a command because it does not have write permissions in the working directory where the command is run or the destination directory manually specified with - --out argument included in command print an error message and instruct the user that he should either run the tor command as the rightful user or select a destination directory where current user has write permissions. - - Always save the output files when using above commands with the correct permissions (I guess these are chmod 0600?). - - Can Tor work if the files in $datadirectory/keys folder are owned by other user than the one running Tor? If not, we should try to detect what user runs Tor on that system (not everywhere Tor runs under debian-tor, in FreeBSD it is _tor for example) and change the owner of the files after they are generated and saved. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBCAAGBQJVv/FNAAoJEIN/pSyBJlsRQJYIAJvO10nrCyGR7ImHptZS+GGN DQS424+drGF5uwMunADN/m9MQ2sqXBfa9xA74irA9KU4y1HtY4NUTJZWC9mUrlZv xb8nkp1jml6NMk7M8RQ/A330Ca04bTNcMOzvaBwqDodlG4GDuiNBMnia2SIqzaBD WSLzVQ1wz5zVdF5CPblQXIzS8jhLYQU4wjQf3Z7nnmPgjjn4HNKcd9Q9x/DWRCL7 TsdgE05ZCHpjCzep9xi0RhPV4tdY77Yti7n9b81ty4O+ctYSU8kzvYhiY4nglnDH YQvJSvDH2QnNqR6QXcC+cHSLAkoJMM56OLKoZmhU6cJkbZ3aC/9I5EcFyAZs62E= =X/f3 -----END PGP SIGNATURE-----

2 8

please can haz code review -> Tor-ified Twisted Python HTTP proxy server - txtorhttpproxy
by David Stainton 11 Aug '15

11 Aug '15

Dear Twisted/Tor developers, I am motivated by language security concerns. I think Twisted Python can often implement excellent replacements for our old crufty C implementations. In this case I'd like to stop using C programs running as root (or not root) to torify http connections... like privoxy, polipo, apt tor transport etc. Last year I wrote this little tor http proxy; yes it also handles RFC 2817 proxy CONNECT method as well: https://github.com/david415/txtorhttpproxy I wrote an AgentProxy that can proxy any sort of twisted http agent: https://github.com/david415/txtorhttpproxy/blob/master/txtorhttpproxy/proxy… It used to use more terrible underscore imports but Leif fixed the bit-rot and made it work with Twisted 15: https://github.com/david415/txtorhttpproxy/commit/38f1a0cfab586f7a8965a47e9… Is this project worthy of your precious 10 minutes to review it... so I can improve the code quality? If I get rid of this last terrible import: from twisted.web._newclient import ResponseDone then I'm not sure how to detect when to call the finish() method here: https://github.com/david415/txtorhttpproxy/blob/master/txtorhttpproxy/proxy… Sincerely, David Stainton

1 0

Re: [tor-dev] First release of OnioNS for beta testing
by Jesse V 10 Aug '15

10 Aug '15

Thanks Dr. Fu, I look forward to your comments. From the server log, apparently so far no one has tested the client, though someone has set up a server. I just want to highlight that I have servers up and running so the client and the HS functionality should work out of the box. For the client, install the software then follow the instructions in the README to integrate it into the Tor Browser. My current approach is to rename the Tor binary from "tor" to "torbin" and then insert a symlink to the onions-tbb executable under the name "tor". Then the onions-tbb software lauches "torbin", "onions-client", and the Stem script in that order, thus allowing all the necessary software to start with the Tor Browser. I have also added some functionality to wait until Tor is fully bootstrapped before launching onions-client and the Stem script, thus this approach is fully compatible with Tor bridges. Jesse V. > Subject: > Re: [tor-dev] First release of OnioNS for beta testing > From: > Xinwen Fu > Date: > 08/08/2015 04:25 AM > > To: > "tor-dev(a)lists.torproject.org" <tor-dev(a)lists.torproject.org> > > > Fantastic work. Will test it. > > Xinwen Fu >

1 0

Invitation to participate in a survey about Tor's future
by Sue Gardner 09 Aug '15

09 Aug '15

Hello people of tor-dev, You may have heard that I'm working with the Tor Project to help it develop a strategic plan [1]. To kick off, we're running a survey asking Tor-related people their opinions about Tor and its future. I'd like to invite you to participate: https://www.surveymonkey.com/r/tor_core The survey's pretty short and should take about 15 minutes to fill out. If you have time to do it within the next few days, that would be great. We'd really appreciate your input :) If you've got questions, there's an FAQ here (or you can just ask me here on this list). https://trac.torproject.org/projects/tor/wiki/org/Strategy/SurveyFAQ Thanks, Sue [1] http://suegardner.org/2015/05/20/why-im-working-with-tor-and-first-look/

2 1

collector problems since 2015-08-07 18:00?
by nusenu 08 Aug '15

08 Aug '15

Hi, I was wondering why onionoo reported empty platform strings for a growing number of relays. I wanted to confirm that by looking at collector data, but then I noticed that there is a problem with collector data itself. Files in [1] usually have a size of around 1 MB, currently they are at 1KB (basically empty). @thomas: Is that the cause for your onionoo instance issue? [1] https://collector.torproject.org/recent/relay-descriptors/server-descriptor…

3 3

First release of OnioNS for beta testing
by Jesse V 08 Aug '15

08 Aug '15

Happy Saturday everyone, At long last, 310 commits later, I am pleased to present a release of the Onion Name System (OnioNS), a DNS for Tor hidden services. This release is a usability test; it offers reliable behind-the-scenes integration with the Tor Browser, a friendly command-line dialog for claiming domain names and subdomains, and many options for hosting and configuring a server. The system utilizes two servers: a single Quorum node which hidden services upload their claims to and another server which clients query against. I am looking for feedback as to how usable the system is and areas where it could be improved. Most of the changes going forward will be behind-the-scenes. The software is divided into three primary pieces, OnioNS-client, OnioNS-HS, and OnioNS-server. These all have OnioNS-common (a shared library) as a dependency. You can install whichever one you'd like, or all of them. This software is currently Linux-only, and Debian 7 and 8, Ubuntu 14.04 - 15.10, Mint 17 - 17.2, and Fedora 21 - 23 are supported. I provide packages for Debian 7 and a software repository for currently-supported versions of Ubuntu and Mint on 32-bit, 62-bit, and ARM systems. If possible, please use the repository. Please see the READMEs in the following repositories for more information, including installation, initialization, and configuration procedures. Manpages are also included for your convenience. https://github.com/Jesse-V/OnioNS-common https://github.com/Jesse-V/OnioNS-client https://github.com/Jesse-V/OnioNS-HS https://github.com/Jesse-V/OnioNS-server Please star the repository if it works well for you. I have claimed "example.tor" for my project's HS and claimed the "arma.example.tor" subdomain that points to Roger's site, so you can test this from the client. Please open a ticket if you find a new bug, or contact me if you don't have a Github account. A brief FAQ: Q: How does one pronounce "OnioNS"? A: As one would pronounce the lowercase form: "onions", the plural of "onion". Q: It only takes a couple of minutes to claim a domain name, isn't that too easy? A: In this release, I have set a very small difficulty level. It will certainly be harder in the future and more counter-measures are being considered. Also, since the claims are not yet saved to disk, I offer no guarantee of their long-term survival. Q: Should I use this on production hidden services? A: No, this software is not ready. This release introduces _features_, not security. Tor circuits are used on both the client and HS side, but I can't guarantee that my SOCKS use is leak-proof, for example. I'm asking for help with finding bugs that may compromise anonymity. Q: I'm running Windows/OSX/BSD/Arch/Gentoo/LFS/etc, can I test OnioNS? A: Yes, but I'm not currently supporting that environment, so you're mostly on your own. However, if you can give me compilation instructions, I may be able to. I am also looking to coordinate with anyone familiar with RPM or Windows development. Q: Is there security on your network communications? A: Client and HS communication occurs over Tor circuits, and there are some integrity checks, but simply getting everything to work is the priority here. Most of the infrastructure is set up so adding signatures and such will be easy, but that is next on the list. Once that occurs, the name server (Mirror) the client uses can be malicious with no significant impact. Q: Where can I learn more about this project? A: The normal project page, onions55e7yam27n.onion, is currently being rewritten. A simple page is in its place, so "example.tor" is still there. Literature on this project may be found at https://github.com/Jesse-V/OnioNS-literature. Please see the PDFs under the respective folders. Note that the distributed design will be changing to use George's commit-and-reveal scheme. Q: Are the servers reliable enough to run under Comcast? A: I have not tested them in production or otherwise under https://github.com/tylertreat/comcast, but I may in the future. I welcome help in this area. Enjoy, Jesse V.

2 1

simple onionoo patch
by firebrand＠ruggedinbox.com 07 Aug '15

07 Aug '15

Hi, I have submitted a small patch some time ago to fix some issues in onionoo. Could someone have a look? https://trac.torproject.org/projects/tor/ticket/16626 ** firebrand

2 1

Relay Dashboard Code Review
by Damian Johnson 05 Aug '15

05 Aug '15

Hi Cristobal, again apologies for the delay in getting you feedback. Just gave RWSD a whirl and worked great - nice work! >From a UI perspective only feedback on what we have so far is... * Both graphs just have the title of 'Graph component'. Better would be to say if it's inboud or outbound. * Graph's y-axis should indicate its units. * Personally I kinda like Vidalia's line graph better than bar graphs for this, but obviously just something to experiment with. Lots of options. https://upload.wikimedia.org/wikipedia/commons/e/ef/Tor-non-exit-relay-band… * Only having fifteen seconds of information is pretty short. Actually, if you raise it to something high (like several minutes of data) then you might want to consider prepopulating bandwidth information from before RWSD started. Here's how Nyx does it... https://gitweb.torproject.org/nyx.git/tree/nyx/graph_panel.py#n244 Unfortunately I'm still a bit of an Angular noob so I can't comment much on those bits. Arturo: do you have any thoughts? Otherwise, here's some feedback on the code! ============================================================ run_client.py ============================================================ 23 settings = dict( 24 template_path=os.path.join(os.path.dirname(__file__), "client/templates"), 25 static_path=os.path.join(os.path.dirname(__file__), "client/static"), 26 autoescape=None, 27 ) 28 handlers = [ 29 (r"/", MainHandler), 30 ] 31 cyclone.web.Application.__init__(self, handlers, **settings) Very minor thing, but little odd using dict() that way. This could also be... handlers = [ (r"/", MainHandler), ] cyclone.web.Application.__init__(self, handlers, { template_path: os.path.join(os.path.dirname(__file__), "client/templates"), static_path: os.path.join(os.path.dirname(__file__), "client/static"), autoescape: None, }) Again, minor and up to you. run_server.py has the same. This is similar to what I do all throughout my codebases... https://gitweb.torproject.org/nyx.git/tree/nyx/graph_panel.py#n71 ------------------------------------------------------------ 37 reactor.listenTCP(int(config.get('client.port')), Application()) Actually, you don't need the int() if you provide a default. If you call... config.get('client.port', 9051) ... then the config will give you an int, and emit a warning if the config has something else... https://stem.torproject.org/api/util/conf.html#stem.util.conf.Config.get Same in run_server.py. ============================================================ run_server.py ============================================================ 35 if(dual_mode()): Good idea on this config option. In python you don't use parentheses for conditionals, unless you're doing some grouping. Speaking of which, might I suggest baking pyflakes and pep8 checks into your project? Even if you don't have unit tests yet these can speed development by catching minor issues, and help the codebase have a more uniform style. Stem provides a couple utilities that should make this easy. Here's an example of using them... https://gitweb.torproject.org/nyx.git/tree/run_tests.py Oh, and it'll catch the trailing whitespaces that's slipped in. ;) % grep -R ' $' rwsd/* | grep -v 'Binary file' | wc -l 17 ------------------------------------------------------------ 45 @uses_settings 46 def main(config, dual = True): The 'dual = True' argument is unused. You use the dual_mode() util function instead. ------------------------------------------------------------ 87 except: 88 log.msg("unable to attach listeners, controller off") 89 except: 90 log.msg("unable to start tor controller from rwsd") _conn_listener() is a neat approach. I like it. Generally it's good idea to both only catch the exception types you need, and include the exception in the message so there's some hint for troubleshooting. The second, for instance, is in effect just calling stem.connection.connect(), which only raises ValueErrors (for invalid arguments)... https://stem.torproject.org/api/connection.html#stem.connection.connect Seeing how you use it here you probably want Controller.from_port() instead. As we discussed earlier connect() prints errors to stdout and returns None if it can't connect. However, you're not checking if it returns None so the second try/catch will hit an AttributeError. I suspect there *might* be a bug around double attaching listeners when tor is connected then reconnected, but not sure. ============================================================ util/__init__.py ============================================================ 49 def msg(message, config, **attr): Unused, but feel free to leave it alone if you plan on using it later. ============================================================ server/handlers/main.py ============================================================ Your run_client.py defines a MainHandler but then it seems you have an unused copy of it here too. Since run_server.py uses it too maybe drop it from run_client.py and use this instead? ============================================================ server/handlers/status.py ============================================================ 14 def _handle_status_event(controller, state, timestamp): 15 if state == State.CLOSED: 16 output = {'status': 'off'} 17 else: 18 output = {'status': 'on'} 19 20 send_data(WebSocketType.STATUS, output) Certainly fine, but can condense this if you'd like. def _handle_status_event(controller, state, timestamp): send_data(WebSocketType.STATUS, {'status': 'off' if state == State.CLOSED else 'on'}) ============================================================ server/handlers/graph.py ============================================================ 24 if read_total and write_total: 25 output.update({'read_total': read_total}) 26 output.update({'write_total': write_total}) No need to call update() for single values. This is the same as... if read_total and write_total: output.update['read_total'] = read_total output.update['write_total'] = write_total Same for more calls below. ============================================================ server/handlers/base.py ============================================================ 33 def init_websockets(): 34 global WEBSOCKETS 35 WEBSOCKETS = {} 36 for ws_type in WebSocketType: 37 WEBSOCKETS.update({ws_type: []}) Could also be... def init_websockets(): global WEBSOCKETS WEBSOCKETS = dict([(ws_type, []) for ws_type in WebSocketType]) ------------------------------------------------------------ 40 def get_websockets(ws_type = None): 41 global WEBSOCKETS 42 if ws_type: 43 if WEBSOCKETS[ws_type]: 44 return WEBSOCKETS[ws_type] 45 else: 46 return None 47 else: 48 return WEBSOCKETS The inner conditional made me pause for a second until I realized it's to convert empty lists to None. Are you familiar with dictionary's get() method? It allows you to specify a default, and might allow this to be improved a bit. def get_websockets(ws_type = None): websocket = WEBSOCKETS.get(ws_type, []) return websocket if websocket else None Also, you only need the 'global' keyword if you're doing assignment. Your init_websockets() is the only function that needs it. ============================================================ client/static/css/* client/static/font/* ============================================================ Perfectly fine, but our readme should say we're bundling a copy of angular, bootstrap, and awesome. Also, please note their licenses. I assume bundling is the usual approach for this? ============================================================ client/static/img/body-bg.png ============================================================ Is this just a solid color? If so, any reason it's an image rather than a css background color property? ============================================================ client/static/js/* ============================================================ Please separate our custom javascript from bundled modules. This way it's clear what is ours verses a copy. ============================================================ client/static/js/bandwidthGraph.controller.js ============================================================ 9 function bandwidthGraph($scope, bandwidthWebsocket) { 10 activate(); 11 $scope.readData.data = bandwidthWebsocket.read_data; 12 $scope.writtenData.data = bandwidthWebsocket.written_data; Ahh, so we're only using read_data and written_data. Our bandwidth handler is doing a bit of extra work each second to fetch additional data from tor. Fine for now, but we should later drop those GETINFO calls if they remain unused. Is the activate() getting called on every bandwidth event? On first glance looks like $scope.readData and $scope.writtenData are getting reset each time. ============================================================ client/templates/index.html ============================================================ 10 <link href="http://fonts.googleapis.com/css?family=Open+Sans:400italic,600italic,400,600" rel="stylesheet"> ... 15  16  Fine for now, but cross domain fetches to google (googleapis.com and googlecode.com) are likely a no-go for our final version. Also, it might also be unwise to link to Google Code since it's being phased out... https://code.google.com/p/support/wiki/ReadOnlyTransition Doesn't matter in the short term (being read-only is, of course, fine) but sounds like that may go away eventually. ------------------------------------------------------------ 19 <style> 20 .chart { 21 width: 100%; 22 height: 300px; 23 } 24 </style> Should we start our own application.css in static for things like this? No doubt this'll just be the first of many. ------------------------------------------------------------ 68 <div class="main"> 69 <div class="main-inner"> 70 <div class="container"> 71 <div class="row" ng-controller="bandwidthGraph"> ... Angular supports templates which store html snippets. Essentially partials. Maybe that would help break this up? It would be nice if our base index.html was dead simple, and pulled in modular components which do the actual work.

4 6

Summary of meek's costs, July 2015
by David Fifield 05 Aug '15

05 Aug '15

Here's the summary of meek's CDN fees for July 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 -- total by CDN $5497.73 + $6346.26 + $0.00 = $11843.99 grand total The big event last month was an outage of meek-azure that was caused by a mistake during an account migration: https://lists.torproject.org/pipermail/tor-talk/2015-July/038487.html The outage is unfortunately still ongoing, although there's been progress in getting it fixed (I opened a ticket with Microsoft support and they have to migrate things manually). In the meantime, you can use Tor Browser 5.0a4, which uses a newly configured CDN endpoint. The number of users dropped below 3000, perhaps because of the meek-azure outage. https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Instance hours were up sharply, over 40%, I don't know why. Here is how the Google costs broke down: 2934 GB $352.11 7598 instance hours $379.90 Compared to the previous month: 2807 GB $336.87 5299 instance hours $264.96 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Number of requests was up 8% while bandwidth was down 11%. Asia Pacific (Singapore) 155M requests $187.04 1034 GB $137.11 Asia Pacific (Sydney) 1M requests $2.46 5 GB $0.66 Asia Pacific (Tokyo) 28M requests $34.08 157 GB $20.10 EU (Ireland) 198M requests $238.14 1124 GB $86.91 South America (Sao Paulo) 7M requests $16.02 50 GB $11.94 US East (Northern Virginia) 71M requests $71.71 395 GB $30.90 -- total 463M requests $549.45 2768 GB $287.62 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == The meek-azure outage started on July 20. You can see it clearly in the attached graph, where bandwidth dropped almost to zero. If you are affected by this outage, you should try Tor Browser 5.0a4, which uses a working CDN endpoint for meek-azure. You can also manually change the configuration in other versions of Tor Browser: https://lists.torproject.org/pipermail/tor-talk/2015-July/038496.html https://trac.torproject.org/projects/tor/ticket/16634 https://onionoo.torproject.org/bandwidth?fingerprint=AA033EEB61601B2B7312D8… Estimated bandwidth use this month: 2015-07 1974 GB Compared to last month: 2015-06 3038 GB https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html

1 0