- tor-dev - lists.torproject.org

Tor project automation work
by Nicolas Vigier 12 Dec '13

12 Dec '13

Hello, If you don't know me yet, I am the new person working on automation : https://www.torproject.org/about/jobs-lead-automation.html.en To get started working on tor automation, I am currently looking at all components that are part of tor project, to find what is the current status regarding build, packaging, testing, and what should be done. I'm going to write a summary of this, with some proposals and a general plan for the work to be done during the coming months, and it will be available at the following URLs : https://people.torproject.org/~boklm/automation/tor-automation-review.html https://people.torproject.org/~boklm/automation/tor-automation-review.pdf (you can find an early version there) In order to help me doing that, I'm very interested to receive from developers of any tor components : - a description or ticket number of bugs that you wish could have been detected earlier with automated tests - any wish or specific needs that you may have - anything else that you think might be useful for me to know If you want to discuss this on IRC, I'm 'boklm' on #tor-dev. Thanks! Nicolas

3 7

Proposal 225: Strawman proposal: commit-and-reveal shared rng
by Nick Mathewson 11 Dec '13

11 Dec '13

This used to be an appendix in my drafts for proposal 224, but I split it into its own document since it's orthogonal to the rest of 224. See section 1 for appropriate provisos and disclaimers. I'm writing this as a proposal mainly so other researchers can write better proposals for this kind of thing. Filename: 225-strawman-shared-rand.txt Title: Strawman proposal: commit-and-reveal shared rng Author: Nick Mathewson Created: 2013-11-29 Status: Draft 1. Introduction This is a strawman proposal: I don't think we should actually build it. It's just a simple writeup of the more trivial commit-then-reveal protocol for generating a shared random value. It's insecure to the extent that an adversary who controls b of the authorities gets to choose among 2^b outcomes for the result of the protocol. See proposal 224, section HASHRING for some motivation of why we want one of these in Tor. Let's do better! [TODO: Are we really stuck with Tor's nasty metaformat here?] 2. The protocol Here's a protocol for producing a shared random value. It should run less frequently than the directory consensus algorithm. It runs in these phases. 1. COMMITMENT 2. REVEAL 3. COMPUTE SHARED RANDOM It should be implemented by software other than Tor, which should be okay for authorities. Note: This is not a great protocol. It has a number of failure modes. Better protocols seem hard to implement, though, and it ought to be possible to drop in a replacement here, if we do it right. At the start of phase 1, each participating authority publishes a statement of the form: shared-random 1 shared-random-type commit signing-key-certification (certification here; see proposal 220) commitment-key-certification (certification here; see proposal 220) published YYYY-MM-DD HH:MM:SS period-start YYYY-MM-DD HH:MM:SS attempt INT commitment sha512 C signature (made with commitment key; see proposal 220) The signing key is the one used for consensus votes, signed by the directory authority identity key. The commitment key is used for this protocol only. The signature is made with the commitment key. The period-start value is the start of the period for which the shared random value should be in use. The attempt value starts at 1, and increments by 1 for each time that the protocol fails. The other fields should be self-explanatory. The commitment value C is a base64-encoded SHA-512 hash of a 256-bit random value R. During the rest of phase 1, every authority collects the commitments from other authorities, and publishes them to other authorities, as they do today with directory votes. At the start of phase 2, each participating authority publishes: shared-random 1 shared-random-type reveal signing-key-certification (certification here; see proposal 220) commitment-key-certification (certification here; see proposal 220) received-commitment ID sig received-commitment ID sig published YYYY-MM-DD HH:MM:SS period-start YYYY-MM-DD HH:MM:SS attempt INT commitment sha512 C reveal R signature (made with commitment key; see proposal 220) The R value is the one used to generate C. The received-commitment lines are the signatures on the documents from other authorities in phase 1. All other fields are as in the commitments. During the rest of phase 2, every authority collects the reveals from other authorities, as above with commitments. At the start of phase 3, each participating authority either has a reveal from every authority that it received a commitment from, or it does not. Each participating authority then says shared-random 1 shared-random-type finish signing-key-certification (certification here; see proposal 220) commitment-key-certification (certification here; see proposal 220) received-commitment ID sig R received-commitment ID sig R ... published YYYY-MM-DD HH:MM:SS period-start YYYY-MM-DD HH:MM:SS attempt INT consensus C signature (made with commitment key; see proposal 220) Where C = SHA256(ID | R | ID | R | ID | R | ...) where the ID values appear in ascending order and the R values appear after their corresponding ID values. See [SHAREDRANDOM-REFS] for more discussion here. (TODO: should this be its own spec? If so, does it have to use our regular metaformat or can it use something less sucky?)

3 4

Debug-assert in 'test util/logging/sigsafe_err'
by Gisle Vanem 11 Dec '13

11 Dec '13

Running "test --verbose util/logging/sigsafe_err" gives me an debug-assert in write(STDERR_FILENO,...). Assert that failed is: "(_osfile(h) & FOPEN)" Here's the call-stack in WinDbg: MSVCR100D!_write+0x126 test!tor_log_err_sigsafe_write(char * s = 0x00707290 ".====================== ====================================== T=")+0x66 [g:\mingw32\src\inet\tor-project\src\common\log.c @ 463] test!tor_log_err_sigsafe(char * m = 0x006bae50 "Minimal..")+0xdf [g:\mingw32\src\inet\tor-project\src\common\log.c @ 491] test!test_sigsafe_err(void * arg = 0x00000000)+0xb0 [g:\mingw32\src\inet\tor-project\src\test\test_logging.c @ 98] ... It makes sense that "(_osfile(h) & FOPEN)" is FALSE, since in test_logging.c I see: close(STDERR_FILENO); <<< !!! what good is this? log_err(LD_BUG, "Say, this isn't too cool."); tor_log_err_sigsafe("Minimal.\n", NULL); If you close stderr, shouldn't handle 2 point to something else? I cannot see it does. All this is on Win-XP SP3, MSVC v16 (debug). --gv

1 0

fteproxy v0.2.1
by Kevin P Dyer 11 Dec '13

11 Dec '13

Hi dcf1/asn, Yesterday I tagged release v0.2.1 of fteproxy. In this release I focused on breaking away from heavyweight dependencies: OpenFST and boost. Cross-platform PTTBB binaries are available on the fteproxy website: https://fteproxy.org/download. asn - Please have a look at the code and confirm that you're happy. I had to make a few changes to the Python modules written in C++, as I can no longer rely upon boost::python. I've also added additional information to the function-level descriptions, which should help explain the significance of the language theoretical algorithms. It is also worth noting that the pseudocode is available in Appendix A of the full version of our paper [0]. dcf1 - I finally have the gitian build process producing vanilla binaries. However, I don't see anything specific to pluggable transports in [1] or [2]. Is there a fork of this code that has logic for building the pluggable transports? Or is that still an open task? -Kevin [0] http://eprint.iacr.org/2012/494 [1] https://gitweb.torproject.org/builders/tor-browser-bundle.git [2] https://gitweb.torproject.org/builders/gitian-builder.git

2 1

txtorcon 0.8.2
by meejah 10 Dec '13

10 Dec '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 First, sorry this announcement is a little late, this was in my drafts :( There was a bug in txtorcon 0.8.1 where the TCP4HiddenServiceEndpoint wasn't ensuring that only 127.0.0.1 was being listened on. GitHub's master and the tag v0.8.2 fix this issue, and (thanks to Lunar) the Debian packages are also updated. PyPI packages are also updated (via Twine this time, which is really nice). If you don't use TCP4HiddenServiceEndpoint, there is no need to upgrade. If you *do* use it, you should upgrade. Thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iQEcBAEBAgAGBQJSp03QAAoJEJ0UOBRRgEVQWPcIAMV0aupijakh3GUBqF2s0zEp TxPr658w0KDZuWH9cON9h/Apkss0BQnW2LuoPKP4spAXh8qOlUd6sBzj+fat2RwE 09qkJPKuyt6kGZyxD5jRL3pRNEBU8qs0C6AIcdysfpfWRGOx53ZOjJgBoanzzmRA JmdFQBtAVBCW0IzImFpf8rCIQpx/8SXtA898RqQODANdtxucwqRVqC/Y5uX+7a29 xdtgFpr7NcqW2KC0WMInubQ6lo6KfPgCfnrSU2FBcafwdXlh97cmdovzUX0/raSM bHJvgruPyF1hJBDuTHXzKbnb41Xb0Vk6MMALw3ad3t505jFDhZT1g8kBw0mo1Qk= =PM/Z -----END PGP SIGNATURE-----

1 0

Apple App Store Redux
by Griffin Boyce 10 Dec '13

10 Dec '13

It's been a while since there's been a discussion on-list about getting the TBB into Apple's app store [1]. Interest hasn't really gone away in the intervening 13 months, so I just want to open up discussion about it. Here are the issues as I see them: - Apple has traditionally been at odds with GPL-licensed stuff [3], though of late it seems to have relaxed a bit with dual-licensed material [2]. - If the TBB is added to the app store by Tor, it requires review of and agreement to Apple's terms and also agreeing not to reveal DRM sekrits [4]. - It requires time and energy to keep the app store listing maintained. Here are some possible solutions: - Submit Apple agreements to Wendy for review and rejection/acceptance. The last mention of this was a year ago on #6540. Status? - A volunteer who doesn't work for Tor maintaining the app store version of TBB. This would also free Tor as an organization from having to sign agreements. (Though this may contravene Apple's terms). - Actively decide to continue without being blessed by Apple, but focusing instead on educating Mac users about their application security options. Thoughts? ~Griffin [1] https://trac.torproject.org/projects/tor/ticket/6540 [2] https://www.opensource.apple.com/license/gpl-with-exception/ [3] http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/ [4] https://www.eff.org/deeplinks/2012/05/apples-crystal-prison-and-future-open… -- Be kind, for everyone you meet is fighting a hard battle. PGP: 0xD9D4CADEE3B67E7AB2C05717E331FD29AE792C97 OTR: saint(a)jabber.ccc.de

12 16

Damian's Status Report - November 2013
by Damian Johnson 09 Dec '13

09 Dec '13

Hi all. Work and life is conspiring to keep me busier than usual. What time was left this November mostly went toward non-development activities. I miss coding... ====================================================================== Tor Ecosystem Presentation ====================================================================== On 11/18 I gave a presentation at Seattle's TA3M overviewing projects in the Tor ecosystem, with an eye toward getting more developers involved... Presentation: https://archive.org/download/t3am-seattle-nov2013/damian_johnson.mp4 Slides: https://www.atagar.com/transfer/tor_ecosystem/ Preparing for this gobbled up a surprising amount of time (I don't do much public speaking), but I'm really happy with how it turned out. I'll likely give this presentation again at other local venues, such as LinuxFest Northwest in the spring. ====================================================================== Email List Management ====================================================================== Much of this last month went toward managing our lists and leading a discussion about what we'd like to do with our non-public lists. Work on this is ongoing, but at the very least we'll finally get a public, documented process for adding folks... https://trac.torproject.org/projects/tor/wiki/doc/internalLists I also sunk several hours into a frustrating GMail issue where it bounces Mailman's moderator queue as spam... https://productforums.google.com/forum/#!category-topic/gmail/ask-a-how-to-… At one point this seems to have been responsible for unsubscribing everyone using GMail from tor-assistants@. I've changed Mailman's configuration so this hopefully won't happen again, but I'm out of ideas for the GMail bounces. ====================================================================== Other news this month includes... * Fixed an important Stem ImportError under Windows. This resulted in a hotfix release of Stem (version 1.1.1). [1] * Added Stem support for the new event types from proposal 218. [2] * Code reviewed Philipp's ExitMap project. [3] * Read over tbbscraper and added to Stem's examples page. [4] * Helped provide stats for the impact of changing tor's default exit policy. [5] Cheers! -Damian [1] https://trac.torproject.org/10072 [2] https://gitweb.torproject.org/stem.git/commitdiff/e8c916b?hp=ae6d978 [3] https://lists.torproject.org/pipermail/tor-dev/2013-December/005889.html [4] https://github.com/zackw/tbbscraper/blob/master/controller/controller.py [5] https://trac.torproject.org/10067#comment:6

2 4

Statistics on fraction of connections used uni-/bidirectionally
by Karsten Loesing 09 Dec '13

09 Dec '13

Björn, Florian, a few years back (in 2010, to be precise) we added statistics to little-t-tor reporting what fraction of connections is used uni-/bidirectionally. Quoting dir-spec.txt: "conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL [At most once] Number of connections, split into 10-second intervals, that are used uni-directionally or bi-directionally as observed in the NSEC seconds (usually 86400 seconds) before YYYY-MM-DD HH:MM:SS. Every 10 seconds, we determine for every connection whether we read and wrote less than a threshold of 20 KiB (BELOW), read at least 10 times more than we wrote (READ), wrote at least 10 times more than we read (WRITE), or read and wrote more than the threshold, but not 10 times more in either direction (BOTH). After classifying a connection, read and write counters are reset for the next 10-second interval. These statistics are disabled by default, but when they are enabled, relays publish them in their extra-info descriptors. And quite a few relays do that. Here's a (bad) visualization (that used to be slightly less bad when fewer relays published these statistics): https://metrics.torproject.org/performance.html#connbidirect Here's the question: Is there still value in having these statistics? I recall that they were useful in 2010, but will that still be the case in 2013? If the answer is "yes", never mind. If the answer is "no", I'd create a ticket and submit a patch to remove code parts from little-t-tor, and I'd remove the not-really-useful graph from the metrics website. Cc'ing Rob, Aaron, and Roger as the people who typically have an interest in these kinds of statistics. If other tor-dev@ people have an opinion on this, please raise your voice! All the best, Karsten

1 0

Merge of Gitian PT build with 3.0-rc-1; next steps
by David Fifield 08 Dec '13

08 Dec '13

I recently merged the Gitian-based pluggable transports bundle build with tag tbb-3.0rc1-build1, with suprisingly little difficulty. I'd like you to take a look at what the pt branch is doing and help me decide what to do next. Some background: The ticket for building the PT bundles is https://trac.torproject.org/projects/tor/ticket/9444 The pt branch is at https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/shortlog/refs… To see what's different in the pt branch with respect to master, in an existing tor-browser-bundle.git checkout, do this: git remote add dcf https://git.torproject.org/user/dcf/tor-browser-bundle.git git fetch dcf Then, for example, git diff master...dcf/pt --stat git diff master...dcf/pt gitk master dcf/pt A recent build of 00789049873753d931fec3803c9ccae75a5c5267 is here, also containing the full complement of usual build outputs. https://people.torproject.org/~dcf/pt-bundle/3.0-rc-1-pt20131207-1/ These bundles are up to date and seem to be working well. I would be comfortable shipping them to users. I'd like your advice on how to make the pluggable transports browser part of the usual build. The diff between descriptors gitian-bundle.yml and gitian-ptbundle.yml is small; in fact I can paste it here: --- descriptors/linux/gitian-bundle.yml 2013-12-06 19:38:40.000000000 -0800 +++ descriptors/linux/gitian-ptbundle.yml 2013-12-06 19:38:40.000000000 -0800 @@ -1,3 +1,3 @@ --- -name: "bundle-linux" +name: "ptbundle-linux" suites: @@ -29,2 +29,4 @@ - "tor-linux64-gbuilt.zip" +- "pluggable-transports-linux32-gbuilt.zip" +- "pluggable-transports-linux64-gbuilt.zip" - "relativelink-src.zip" @@ -39,2 +41,4 @@ - "tbb-docs.zip" +- "pttbb-docs.zip" +- "linux-pt-torrc" script: | @@ -98,4 +102,8 @@ unzip ~/build/tor-linux$GBUILD_BITS-gbuilt.zip + unzip ~/build/pluggable-transports-linux$GBUILD_BITS-gbuilt.zip unzip ~/build/linux-skeleton.zip unzip ~/build/tbb-docs.zip + mkdir -p Docs/PluggableTransports + unzip -d Docs/PluggableTransports ~/build/pttbb-docs.zip + cat ~/build/linux-pt-torrc >> Data/Tor/torrc-defaults chmod 700 Data/Browser @@ -122,3 +130,3 @@ #mv tor-browser.exe $OUTDIR/tor-browser-linux${GBUILD_BITS}_en-US.7z-selfextract - ~/build/dtar.sh $OUTDIR/tor-browser-linux${GBUILD_BITS}-${TORBROWSER_VERSION}_en-US.tar.xz tor-browser_en-US + ~/build/dtar.sh $OUTDIR/tor-pluggable-transports-browser-linux${GBUILD_BITS}-${TORBROWSER_VERSION}_en-US.tar.xz tor-browser_en-US rm -rf tor-browser_en-US @@ -149,3 +157,3 @@ #mv tor-browser.exe $OUTDIR/tor-browser-linux${GBUILD_BITS}_$LANG.7z-selfextract - ~/build/dtar.sh $OUTDIR/tor-browser-linux${GBUILD_BITS}-${TORBROWSER_VERSION}_$LANG.tar.xz tor-browser_$LANG + ~/build/dtar.sh $OUTDIR/tor-pluggable-transports-browser-linux${GBUILD_BITS}-${TORBROWSER_VERSION}_$LANG.tar.xz tor-browser_$LANG I see a couple of options for getting rid of gitian-ptbundle.yml: 1. Add a conditional variable, something like the '[ ${TORBROWSER_VERSION::3} != "3.0" ]' in linux/gitian-firefox.yml, that controls whether a vanilla or a PT build is underway. When the variable is set, it would just do the few small changes in the diff above. 2. Build the pluggable transports bundles unconditionally, and just do all the dtars in gitian-bundle.yml twice: once for tor-browser and once for tor-pluggable-transports-browser. Either option, I think, puts us in a good place to step to a unified bundle in the future. What do you think? David Fifield

1 0

Torbirdy - IMAP issue
by arkmd 07 Dec '13

07 Dec '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Accessing an email server via IMAP may leak data by saving a draft on the remote server. Using Thunderbird+Enigmail+Torbirdy. While writing a message on Thunderbird, it is automatically saved as a draft, which by default is sent to the IMAP server. So the server will be able to read that message. That's a big problem when the message should be encrypted before sent. So the email provider will be able to read sensitive data on those drafts in cleartext and the user probably won't notice. To solve this the user need to manually set the account drafts settings (in Copies & Folders) to keep drafts on Local Folders. I think Torbirdy should do it by default. This info should be added to known issues on Torbirdy wiki. I know Torbirdy developers recommend POP over IMAP, but as a mailtor.net user I don't have any other option. - -- arkmd DeepBlog | A verdade nua crua e distorcida http://xzzpowtjlobho6kd.onion/ OpenPGP Public Key: http://xzzpowtjlobho6kd.onion/arkmd.asc 4096R: 0461 DF2C B6B7 6059 7529 77E0 04CD FE83 766B 8DA6 -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJSoUwEAAoJEATN/oN2a42mW/AQAI7IITFPTZEe4X2UBNS3kovK UHJIe9j/K9FA63ubGxu8TotdkIK5TvnkPP/DGDg6NrnILIx/KeK6XAWTDhv6vfgY hIBA+v+9mjAW64fQ31tTSXEVKUofKUyazN984QnVcb/Qyj297GujgrI0vdDFU0c9 KIS7z15Lk3QRx+Qtb+VboXH3ES59oRPuutTSQ+z0RyLToMmFs41BDei01WtLfy/i aLcIYTWUZb+yVmspkOuOKAe9enlGNTXeFGaR6uspA8ImedsCUYq5iySBhm5CVs0B 0sF6Mq18rXTVDF4jwoNLYqkgyANSI9IzgDbxlzgBaJwJjnN9OAeF8vJI+qkZUElS 56pVIzayyEMJXhvsqotuBYH+X+aRt2+Hw2prIxuUcRDFP3iOrupElARpqvDqYa4U IrQ/8lUl5jEnPNNJFklBwTjzHOVtxmCymEDh+zSXlIzL4WQJwYPDj1+uqwOenwO4 FCy1ADFXyMDQso3bXwPqD3VSGdYGqmMc+34vIVA28vTSFYyxiBwjHZLF16Us+uD/ sMbMGEOg+za8MSrgLVwcBAkbRG14fE50litW+bnSWl3QltGOt0rV9INKLKSGvEy4 DMCL2mc9aU9Rago1j/FJlXAOCctuFlTOS9MkCzF+iSJIoa3IxwUmnrvWwOuEuFxa zrSdjkyzMJKgnd+b+Vm4 =oF2B -----END PGP SIGNATURE-----

5 6