June 2014 - tor-dev - lists.torproject.org

Re: [tor-dev] [HTTPS-Everywhere] GSoC report - Zack Mullaly - HTTPS Everywhere secure ruleset update mechanism
by Yan Zhu 15 Jun '14

15 Jun '14

(Trying again with my tp.o email address) On 06/15/2014 02:05 PM, Yan Zhu wrote: > It's unclear whether this message went through to tor-dev (can't find it > in the archives), but I've added this update to > https://trac.torproject.org/projects/tor/wiki/doc/gsoc. > > On 06/13/2014 05:06 PM, Red wrote: >> Hello, everyone! >> I apologize for the fact that this is coming in late, but here is a >> summary of my progress and plans thus far in developing a secure ruleset >> update mechanism for the HTTPS Everywhere browser extension. >> >> The specification document detailing how the ruleset updater will >> function has been perhaps the greatest focus for me until now. The >> document is currently hosted on Github as a gist[1], and currently >> details the format for the JSON document the extension will fetch to >> determine whether the update information it receives is authentic and >> relevant. >> >> A second task I have been working on is the creation of a utility[2] >> used to automate much of the process of building the update.json file >> contents outlined by [1]. A lot of the work done here so far has been >> experimental, but it is already providing some utility for composing >> data that can be used for testing purposes. >> >> The third thing I have been working on is the actual implementation of >> the ruleset updater[3]. There are to be some changes to the spec that >> will be reflected in this code in the coming week, but the >> implementation so far is very close to being ready to test. >> >> In the last week, a lot of discussion has occurred centered around >> improving the specification for the ruleset update mechanism and how the >> update.json file and signing thereof should function and be written. I >> have posted my weekly meeting notes to another gist[4] which I will from >> today onwards be keeping up to date with my weekly notes so that they >> will be publicly available and well-formatted. In summary, my upcoming >> work will involve updating the update.json spec to reflect the >> discussion being had on the https-everywhere mailing list and between >> myself and my mentor, Yan. I will then focus on updating the extension >> code as well as the utility I have been working on to reflect the >> changes to the spec. I will then move on to testing the signature >> verification method locally by creating example documents and a Python >> script to verify the signature. I will also be setting up a testing >> environment to properly test my work on the ruleset update mechanism. >> >> My work can be more closely followed on Github- specifically, my fork of >> the official HTTPS-Everywhere repository[5]. The code I have been >> working on resides in my "makeJSONManifest" and "rulesetUpdating" >> branches. You can also follow the discussion on the https-everywhere >> mailing list, and are welcome to join in mine and Yan's weekly meetings >> in #https-everywhere on irc.oftc.net at 11:00AM Pacific Time on >> Fridays. We're happy to have people chime in with ideas, and commentary >> in IRC, the mailing list, and on Github is welcome! >> >> [1]: https://gist.github.com/redwire/2e1d8377ea58e43edb40 >> [2]: >> https://github.com/redwire/https-everywhere/blob/makeJSONManifest/utils/rul… >> [3]: >> https://github.com/redwire/https-everywhere/blob/rulesetUpdating/src/chrome… >> [4]: https://gist.github.com/redwire/b62f03905a826e79947a >> [5]: https://github.com/redwire/https-everywhere >> >> >> >> _______________________________________________ >> HTTPS-Everywhere mailing list >> HTTPS-Everywhere(a)lists.eff.org >> https://lists.eff.org/mailman/listinfo/https-everywhere >> > > -- Yan Zhu <yan(a)eff.org>, <yan(a)torproject.org> Staff Technologist Electronic Frontier Foundation https://www.eff.org 815 Eddy Street, San Francisco, CA 94109 +1 415 436 9333 x134

1 0

Shutting down relay-search service shrinks metrics database by almost 97%
by Karsten Loesing 15 Jun '14

15 Jun '14

Hi devs, I have been planning to shut down the relay-search service, that was part of the metrics website, for about half a year, and now it's gone: https://lists.torproject.org/pipermail/tor-dev/2013-December/005948.html https://metrics.torproject.org/relay-search.html It was a good service. The main reason for shutting it down was that it required us to keep a full database of relays. And the Tor network has grown quite a bit over time. Now that the service is gone I was finally able to clean up the metrics database. Here are some numbers from before and after the cleanup with all sizes being rounded GiB: table | before | after --------------+---------------- bwhist | 13 | 0 descriptor | 6 | 0 dirreq_stats | 10 | 0 statusentry | 67 | 2 ... | | ------------------------------- total | 95 | 3 So, 92 of 95 GiB are gone, simply because we don't need that data anymore. That's much more than I had expected. Almost 97%! Just in case, and I'm going to watch over this, too: if the metrics website shows you funny numbers in the next couple of days, please let me know. All the best, Karsten

1 0

ANN: txtorcon 0.10.0
by meejah 15 Jun '14

15 Jun '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm excited to announce txtorcon 0.10.0 which adds support for Twisted's endpoint strings. This means that ANY Twisted program that uses endpoints can accept "onion:" strings to bring up a hidden services easily (by launching a new Tor instance). Typically, no code changes to the application should be needed (just "pip install txtorcon"). "twistd" supports endpoints, so for example to serve some Web content from ~/public_html as a hidden-serivce, we can do this (with txtorcon installed): twistd web --port onion:80 --path ~/public_html Some examples of other valid "onion:" endpoint strings: onion:80:hiddenServiceDir=/dev/shm/hidsrv onion:80:controlPort=9050:hiddenServiceDir=/srv/hidden The first allows specifying existing hidden service keys and the second says to connect to an already-running Tor instance. Thanks to David Stainton (dawuud) for the initial pull-request (and continued collaboration) that made this happen. There is a complete demonstration of the power of this Fully Operational endpoint-station here: https://txtorcon.readthedocs.org/en/latest/howtos.html#endpoints-enable-tor… You can download the release from PyPI or GitHub (or of course use "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.10.0 https://github.com/meejah/txtorcon/releases/tag/v0.10.0 sha256sum reports: 910ff3216035de0a779cfc167c0545266ff1f26687b163fc4655f298aca52d74 txtorcon-0.10.0-py2-none-any.whl c93f3d0f21d53c6b4c1521fc8d9dc2c9aff4a9f60497becea207d1738fa78279 txtorcon-0.10.0.tar.gz thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJTnTM5AAoJEMJgKAMSgGmnB5EIAIJACHhd7PEWYfCZmB8lL1pr J4rrhUuXD1iI7zJJL/rmS/SZLL+34JoKke6iQsBRGYzU0GRK2JQkFAeMq++AMdMu QvoTg745OhnMSbZyCyS6buN/NhAVcmD3GYI9h8TK60jfUPEFn7+sDstMG4OePdgq X0QQHBuJq9XtDwEmRWFHq2Aht//7J2DUliPPjtwT3C4FhPT5pXxasvfQA4jOYxsO jTja6UE4/4TnonV5gB/chijp60cKvVxFQi86mKuNnVaKTZu+QCZFJtXkr/DbRSxv CJ+z7jJ9s8xhfQ5Nkv9pkVAreH6w9bemc/iijKlZmQGYvLBTsvISPATyWGtn1f4= =wIVZ -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] Tor Cloud maintenance - Question
by Sina Rabbani 13 Jun '14

13 Jun '14

Dear Team, I started building new images for Amazon EC2, as of last night. I noticed that obf2 and obf3 ports are hard-coded and the same for all Tor instances on Amazon. Is there a specific reason for this? If not, can I randomize the ports? All the best, SiNA On 2014-06-11 23:50, Karsten Loesing wrote: > Hi Sina, > > please find the following conversation below from #tor-project. Any > thoughts? > > Thanks, > Karsten > > > 06:43:15 < Lunar^> We really need a maintainer for Tor Cloud. Thre's > some of the support tickets we are getting I have > no idea how to answer… > 06:45:20 < karsten> > > https://trac.torproject.org/projects/tor/wiki/org/operations/Infrastructure > says inf0 and Runa are the maintainers. > 06:45:55 < karsten> is that not the case? should we mark it as > unmaintained and threaten to turn it off three > months later? > 06:47:01 < arma1> sina said he would pick it up > 06:47:04 < arma1> but i don't know that he ever did > 06:47:13 < karsten> sina = inf0? > 06:47:18 < arma1> perhaps karsten could mail him and ask if he is > maintaining it or if we should turn it off? > 06:47:19 < arma1> yes > 06:47:28 < karsten> sure, I can do that. > 06:47:57 < arma1> thanks! -- "be the change we wish to see in the world." Gandhi

2 3

Re: [tor-dev] Proposal for improving social incentives for relay operators
by Virgil Griffith 11 Jun '14

11 Jun '14

> Also, theconcept of naming authorities is about to be phased out [1], so > better not build new services that rely on nicknames. Karsten I love you. Not only do you have fine ideas, you are the greatest feedback provider in the world. Agreed 100%. Replace key-by-nickname with key-by-fingerprint. Not crazy about having a a separate Torati name space, but meh, you do what you have to. Maybe make the nicknames a version 2.0 feature. (Could always make a namespace integration with google+! Har.) > Okay. See Andrew's concerns about avoiding words having "Tor" in them. I will find a name that doesn't explicitly invoke Tor. -V

1 0

Typo in torspec.git/control-spec.txt
by Frithjof 11 Jun '14

11 Jun '14

Hi, the Tor control protocol spec says in §4.1 about asyncrhronous replies: > These replies can be sent after a corresponding SETEVENTS command has been > received. They will not be interleaved with other Reply elements, but they > can not appear between a command and its corresponding reply. > > C: SETEVENTS CIRC > S: 250 OK > C: GETCONF SOCKSPORT ORPORT > S: 650 CIRC 1000 EXTENDED moria1,moria2 > S: 250-SOCKSPORT=9050 > S: 250 ORPORT=0 If I am not reading the example wrongly, the negation in the last sentence is wrong, e.g. diff --git a/control-spec.txt b/control-spec.txt index 2dde043..70ddf3b 100644 --- a/control-spec.txt +++ b/control-spec.txt @@ -1253,7 +1253,7 @@ These replies can be sent after a corresponding SETEVENTS command has been received. They will not be interleaved with other Reply elements, but they - can not appear between a command and its corresponding reply. For example, + can appear between a command and its corresponding reply. For example, this sequence is possible: Cheers, Frithjof

2 1

Reminder: tor development meetings this (and every) Wednesday, 19:00 UTC
by Nick Mathewson 11 Jun '14

11 Jun '14

Here's your regular reminder for the weekly IRC meeting for people working on the program "tor". (This won't cover all the other programs developed under the Tor umbrella.) The next meeting time will be: Wednesday June 11, 19:00 UTC. (That's 3pm EDT and 12:00 noon PDT.) As usual, we'll do it on the #tor-dev IRC channel. I've seen a drop-off in the number of people coming since I stopped sending these out weekly, so I'm sending them again for a while. Among other stuff, I expect we'll be wrapping up (or planning to wrap up) 0.2.5.5-alpha and maybe doing some initial strategizing on 0.2.6. cheers, -- Nick

1 0

GSoC: BridgeDB Twitter Distributor report
by Kostas Jakeliunas 11 Jun '14

11 Jun '14

Hey all, in the past weeks I've been working on understanding what can be done using Twitter APIs and its media support in its CDN (for a later captcha implementation), as well as on improving my existing Twitter bridge distributor bot PoC. I've written some broken code, but it's alright. More details below. Distributor bot improvements included working on adding a churn rate control mechanism which securely stores Twitter user IDs (with code and design ideas from BridgeDB's HMAC approach to remembering e.g. email addresses in the EmailDistributor), and implementing a (mostly) bogus text-based challenge-response system (this is mostly so that we have a generic design for doing challenge-responses in this distributor - we'll be able to later on replace it with a decent CAPTCHA, for example. It's just nice to have a generic system and a thing for testing out the bot, etc.) I've also looked into using isis' new and shiny BridgeRequest objects to process user (well) 'bridge requests' in a non-hacky way; this should also eventually result in a bridge request syntax compatible with (a subset of) GetTor commands. But I still need to figure out the best way to use BridgeRequests, so nothing interesting to show yet. TODO * (still yet to) summarize a nice meeting i've had with sysrqb and isis. No definite conclusions were made, but there were (iirc) some nice ideas about a generic BridgeDB API that could be used by third party components, etc. (i.e. it might be worth pursuing it even if the Social Distributor is to be implemented at some later point.) * clean up my mess, test new code not to fail, and push new things onto https://github.com/wfn/twidibot/ (current (old) code there does work, if anyone's curious to run it) * figure out BridgeRequests, the new IRequestBridges (ha!) interface, and use these in the twitter bot * be able to 'serve' the bot fake bridge data so it could process it in a way that may be compatible with a future BridgeDB API (i.e., hopefully this bot will be able to run as a third-party-thing, separate from core bridgedb. This is hopefully how future distributors will/should work.) This way the bot will be more/actually 'realistic' in the way it serves current bogus bridge lines to users. (I thought I'd have this by now, but I don't. Hrm.) * continue looking into captcha systems modulo what can be used in the twitter context * look into bridgedb buckets and what I can help re: them, so the bridgedb API could happen sooner than later. (Old todo list item, did not yet touch it.) All in all, need to write more non-broken code, fewer words, and just continue with the current bot. Have a nice day/night/thing! -- Kostas. 0x0e5dce45 @ pgp.mit.edu

1 0

[GSoC 2014] Revamp GetTor: Send HTTP links for downloading TBB
by Israel Leiva 10 Jun '14

10 Jun '14

Hi! I've been discussing with my mentors (Sukhbir and Nima) the option of sending HTTP links to download TBB via GetTor. The main reason for this would be that in some censor countries is very difficult to download a "big file" (>= 10 Mb) over SSL, so HTTP links would be needed. Obviously, the user would be warned about the lack of SSL and that he/she *must* verify the integrity of the bundle. On the other side, bundles downloaded over HTTPS should also be verified if you need maximum security, since we can't trust in the cloud services neither. (e.g. Dropbox). Since this is not an easy decision, we've decided to discuss it publicly. Any thoughts on this? What's your experience regarding this matter? All feedback is welcomed. In the meanwhile we'll keep considering HTTPS links only. Regards. -- israel

2 2

GSoC: Multicore daemon status report
by Белоус Михаил 10 Jun '14

10 Jun '14

Hi all, As start of my project, I made patch to work queue, making thread pool more efficient and wrote some benchmarks. Future work 1.Rewrite architecture of tor circuit processing, to use thread pool 2.Write benchmarks showing, that multi core implementation efficient, or not? Most of my work can be found at https://github.com/towelenee/tor. Please feel free to provide feedback of any sort. Cheers, Mikhail

1 0