- tor-dev - lists.torproject.org

Re: [tor-dev] bridgdb automation
by isis agora lovecruft 03 Jan '14

03 Jan '14

Nicolas Vigier transcribed 1.4K bytes: > Hi, Hey Nicolas, Thanks for all the work you're doing, and my apologies that I hadn't responded to your tor-dev@ call yet. Just in case you haven't seen it, Lunar made a wiki page which has quite a bit of info on it, and I filled in some more on BridgeDB. [0] aabgsn maintained BridgeDB for a year or so, but no longer works on it (though they are more than welcome to do so, if they wish to). sysrqb has been helping me maintain BridgeDB quite a bit (feel free to CC them on BridgeDB topics). > I am currently looking at the status and list of things to be done > regarding automation on tor project. I have been looking at bridgedb : > https://people.torproject.org/~boklm/automation/tor-automation-review.html#… From that page: > Continuous Build > BridgeDB is not currently built and tested by Jenkins. > > However, Isis Lovecruft has a personnal development fork on github that is > built and tested by travis-ci.org: > https://travis-ci.org/isislovecruft/bridgedb/ > > Packaging > BridgeDB does not have packages. It is currently deployed using a Python virtualenv. > To my knowledge, BridgeDB is not currently deployed in a virtualenv (sysrqb was the last to redeploy it). I recently refactored the main loop and scripts so that it *can* run in a virtualenv, and it *should* be run in one, because: 1. We won't need to nag weasel/Sebastian to update/install BridgeDB dependencies. 2. Dependencies will not be installed via sudo. I've been considering creating packages for BridgeDB on PyPI. Pros: * Even if we manually download the bundle, verify the hash, and then install it, this seems potentially easier and less error-prone than checking out a git tag, verifying it, and then building. * Packaging it now reserves the 'bridgedb' Python namespace for our use. Cons: * I don't want to make people think that this thing is a polished distribution system for people who wish to run their own BridgeAuths. If proper packaging is helpful for Jenkins, however, I can easily do so. > Testing > Some unit tests are implemented in lib/bridgedb/Tests.py and can be run with > the command python setup.py test. Actually, the tests in lib/bridgedb/Tests.py are old tests. Running them with `python setup.py test` or `make test` will run them via the Python stdlib unittest module (which doesn't play nicely with Twisted's asynchronicity). See #9865, #9872. [1] [2] There are new tests in lib/bridgedb/test/test_*.py [3] and they can be run with `[sudo] make [re]install && bridgedb test`. I began setting up a system which will run the old lib/bridgedb/Tests.py unittests with Twisted's trial runner (#9873). [4] The old unittests will get run twice, once with removed/deprecated classes and functions which have been taken out of BridgeDB's codebase, and again with new/refactored code; this way, the old unittests function as a (partial) regression test suite. The way I designed it, the removed/deprecated code (various classes/functions before refactoring) will go into lib/bridgedb/test/deprecated.py, and they are `twisted.python.monkey.MonkeyPatch`ed into place for a run of the old unitests in lib/bridgedb/Tests.py. Then, the old unittests are run a second time with the newly refactored code, so that the difference between the two can be clearly seen, and bugs introduced by new code can (hopefully) be caught immediately. > > Proposals > Add BridgeDB build and test to Jenkins Created ticket #10417: BridgeDB should be built and tested on Jenkins https://trac.torproject.org/projects/tor/ticket/10417 > The main thing to be done that I have seen is running the unit tests > with Jenkins when there are new commits. You can let me know if I missed > something important, or if you have other ideas / needs. Needs: 1. We need a lot more unittests, but this is perhaps not a task for volunteers (or, rather, people who aren't very familiar with BridgeDB's code). 2. BridgeDB needs *a lot* more documentation. It had almost none when I started working on it 6 months ago; it has a few bits now. [8] Questions: 1. Does it help if I use tox? [5] [6] 2. If not, I believe you'll need a shell script which Jenkins can use to install BridgeDB in a virtualenv. [7] Or do you need some sort of Maven thing, or both? 3. Is there somewhere I should put that documentation on torproject.org (other than people.tpo/~isis)? [0]: https://trac.torproject.org/projects/tor/wiki/AutomationInventory [1]: https://trac.torproject.org/projects/tor/ticket/9865 [2]: https://trac.torproject.org/projects/tor/ticket/9872 [3]: https://gitweb.torproject.org/user/isis/bridgedb.git/tree/refs/heads/develo… [4]: https://trac.torproject.org/projects/tor/ticket/9873 [5]: http://tox.readthedocs.org/en/latest/ [6]: http://alexgaynor.net/2010/dec/17/getting-most-out-tox/ [7]: http://www.alexconrad.org/2011/10/jenkins-and-python.html [8]: https://para.noid.cat/bridgedb/ Thanks, -- ♥Ⓐ isis agora lovecruft _________________________________________________________ GPG: 4096R/A3ADB67A2CDB8B35 Current Keys: https://blog.patternsinthevoid.net/isis.txt

3 5

Damian's Status Report - December 2013
by Damian Johnson 02 Jan '14

02 Jan '14

Hi all, hope you've enjoyed the holidays (and for a few of you CCC)! This December I finally swapped back to focusing on arm in earnest. Good news is that the codebase is rapidly improving [1], the focus for this month being... * Finished overhauling the starter module. * Expanded test coverage as more of arm gets rewritten. * Integrated pyflakes and pep8 with our tests. * Moved user facing strings to the internal configuration. I did this to improve our code, but it could also allow us to add internationalization support to arm in the future if folks want it. [2] Bad news is unfortunately that none of these improvements are user facing, and even by the rosiest estimates arm's next release is still many months off. This December I also finished with my efforts to revise our internal lists. None of the more substantial changes got much support, but this still resulted in some worthwhile improvements... * A wiki with our lists, maintainers, and a brief summary of each... https://trac.torproject.org/projects/tor/wiki/doc/emailLists * We now have a public, documented policy for adding folks to our internal lists. * The tor-assistants list is now a proper subset of tor-internal. * Identified some unused lists we can drop. [3] Cheers! -Damian [1] https://gitweb.torproject.org/arm.git/shortlog/5eebd00?hp=ff27d2a [2] https://gitweb.torproject.org/arm.git/blob/HEAD:/arm/config/strings.cfg [3] https://trac.torproject.org/10463

1 0

exit-node block bypassing
by Ximin Luo 01 Jan '14

01 Jan '14

Hey all, Flashproxy[1] helps to bypass entry-node blocks. But we could apply the general idea to exit-nodes as well - have the exit-node connect to the destination via an ephemeral proxy. The actual technology probably needs to be different since we can't assume the destination has a flashproxy (websocket/webrtc) PT server running, but we could probably find a technical solution to that. However, I talked this over with a few people and there might be legal and security issues. A few points: - running an exit node carries a great risk, it would be bad/unethical to let ephemeral proxy runners take this risk - (for security reasons we don't fully understand) there is a process for trusting exit nodes and/or detecting misbehaviour (I see badexit emails from time to time). this would be made much harder if exits were ephemeral. - someone could create a massive number of ephemeral exit nodes and capture a lot of exit traffic, giving them extra data to de-anonymise people. I was wondering if any of these have been discussed in depth before already, or if the general topic of exit-node block bypassing is something to be explored. X [1] http://crypto.stanford.edu/flashproxy -- GPG: 4096R/1318EFAC5FBBDBCE git://github.com/infinity0/pubkeys.git

3 3

Otter Buoyant Meeting Today - Delay
by Kelley Misata 01 Jan '14

01 Jan '14

Team - Today's meeting of the Otter Buoyant project needs to be postponed until next Tuesday, 1/7. My laptop is having problems and I will be offline this afternoon to repair it. (hopefully) See you all on IRC next Tuesday 1/7 - 3:30 pm (EST) - IRC #tor-dev. Happy New Years! Kelley -- *Kelley Misata* *Outreach and Communications* *The Tor Project* *www.torproject.org <http://www.torproject.org>*

1 0

Fwd: [guardian-dev] Building OpenSSL and Xtables for Orbot
by Nathan Freitas 29 Dec '13

29 Dec '13

-------- Original Message -------- Subject: [guardian-dev] Building OpenSSL and Xtables for Orbot Date: Sat, 28 Dec 2013 09:28:57 -0500 From: Nathan of Guardian <nathan(a)guardianproject.info> To: Guardian Dev <guardian-dev(a)lists.mayfirst.org> For a long time, building an up-to-date OpenSSL and iptables binary was a challenge for apps like Orbot which need it. Figuring out all the various flags and patches necessary was quite tricky, and the version bundles in the AOSP code were quite out of date. I am happy to say that both through my own persistence, recent patches by a few others, and improvements in the Android NDK toolchain, we can down build the latest OpenSSL and XTables/Iptables code from the original, direct git source. This means we have all the latest ciphers for Tor to use in order to better obscure traffic, and we have a bundled xtables binary in Orbot that can handle all of the transproxying rules we need, no matter what the built-in rooted firmware provides. You can find the updates build instructions in the Orbot external Makefile here: https://gitweb.torproject.org/n8fr8/orbot.git/blob/HEAD:/external/Makefile We will have beta of the new release, based on Tor 0.2.4.20 shortly. +n _______________________________________________ Guardian-dev mailing list Post: Guardian-dev(a)lists.mayfirst.org List info: https://lists.mayfirst.org/mailman/listinfo/guardian-dev To Unsubscribe Send email to: Guardian-dev-unsubscribe(a)lists.mayfirst.org Or visit: https://lists.mayfirst.org/mailman/options/guardian-dev/nathan%40guardianpr… You are subscribed as: nathan(a)guardianproject.info

1 0

Does TLS round-trip optimization apply do Tor?
by Fabio Pietrosanti (naif) 25 Dec '13

25 Dec '13

Hi, i've been reading the article "Optimizing NGIX TLS time for first byte" below: http://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-byte I've been thinking whenever that kind of optimization does apply also to Tor or not? -- Fabio Pietrosanti (naif) HERMES - Center for Transparency and Digital Human Rights http://logioshermes.org - http://globaleaks.org - http://tor2web.org

5 4

[Question to sysadmins and HS operators:] How should Hidden Services scale?
by George Kadianakis 25 Dec '13

25 Dec '13

Hello people, you might have noticed that there is a redesign of hidden services going on. For a while we've been told that "hidden services don't scale" and "there is a max number of clients that a hidden service can handle" so we decided to also consider hidden service scalability as part of the upcoming redesign. Unfortunately, we are not experienced in maintaining busy hidden services so we need some help here. For example, a load balancing technique that hidden services are missing is DNS round-robin; that is, something that load balances on the IP layer (you send some clients to one IP and the rest to another IP). To do the equivalent in hidden services you have to do the load distribution in the Introduction Point layer. Some people have been thinking about this: https://lists.torproject.org/pipermail/tor-dev/2013-October/005556.html https://lists.torproject.org/pipermail/tor-dev/2013-October/005674.html Unfortunately, the above designs are not trivial to implement. They require the hidden service peers to communicate with each other so that they can generate and recycle keys, and it also puts Introduction Points in a position where they can find out the status or number of hidden service peers. For this reason we started wondering whether DNS-round-robin-like scalability is actually worth such trouble. AFAIK most big websites use DNS round-robin, but is it necessary? What about application-layer solutions like HAProxy? Do application-layer load balancing solutions exist for other (stateful) protocols (IRC, XMPP, etc.)? What should we do to make Hidden Services scale to a large number of clients? Also, is "scalability" the correct phrase here? Should we replace it with a combination of "load balancing", "high availability", or something else? [I sent this mail twice because the first did not get posted in tor-dev.]

3 5

OnionMail 1.0.48B is Out.
by Liste 23 Dec '13

23 Dec '13

OnionMail 1.0.48B is now Ready. http://onionmail.onfo Now OnionMail can use GPG (We are upgrading some OnionMail Servers). The source code is on github: https://github.com/onionmail/onionmail If you don't want to connect only via tor, yo ca use an hybrid solution: The NTU LocalProxy https://github.com/onionmail/ntu It can hide more hiddenservices.

1 0

Re: [tor-dev] Statistics on fraction of connections used uni-/bidirectionally
by Florian Tschorsch 22 Dec '13

22 Dec '13

Hi Karsten, we still believe that the statistics are useful. However we also agree with Rob that since more and more relays report data the scatter plot becomes confusing. I think some kind of aggregation would be helpful. In one of our previous papers we assessed the performance impact of simultaneous TCP connections in overlay networks [1]. There we found that the occurring performance degradation is hard(er) to solve when connections are used bidirectionally -- which is the motivation for these statistics. From the current results I would presume that this is the case in Tor most of the time. In Future developments the statistics could become helpful. Cheers, Florian. [1] D. Marks, F. Tschorsch, and B. Scheuermann, "Unleashing Tor, BitTorrent Co.: How to relieve TCP deficiencies in overlays," in 35th IEEE Conference on Local Computer Networks (LCN'10), 2010, pp. 320–323. On 16/12/13 05:34, Rob Jansen wrote: > Hey Karsten, > > I think the statistics could be useful, though I don't currently utilize them. I think the current presentation is somewhat confusing. Perhaps we can try to brainstorm some alternative ways to present the data if the decision is that we should keep it around. > > Best, > Rob > > On Dec 9, 2013, at 12:43 PM, Karsten Loesing wrote: > >> Björn, Florian, >> >> a few years back (in 2010, to be precise) we added statistics to >> little-t-tor reporting what fraction of connections is used >> uni-/bidirectionally. Quoting dir-spec.txt: >> >> "conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL >> [At most once] >> >> Number of connections, split into 10-second intervals, that are >> used uni-directionally or bi-directionally as observed in the NSEC >> seconds (usually 86400 seconds) before YYYY-MM-DD HH:MM:SS. Every >> 10 seconds, we determine for every connection whether we read and >> wrote less than a threshold of 20 KiB (BELOW), read at least 10 >> times more than we wrote (READ), wrote at least 10 times more than >> we read (WRITE), or read and wrote more than the threshold, but >> not 10 times more in either direction (BOTH). After classifying a >> connection, read and write counters are reset for the next >> 10-second interval. >> >> These statistics are disabled by default, but when they are enabled, >> relays publish them in their extra-info descriptors. And quite a few >> relays do that. Here's a (bad) visualization (that used to be slightly >> less bad when fewer relays published these statistics): >> >> https://metrics.torproject.org/performance.html#connbidirect >> >> Here's the question: Is there still value in having these statistics? I >> recall that they were useful in 2010, but will that still be the case in >> 2013? >> >> If the answer is "yes", never mind. >> >> If the answer is "no", I'd create a ticket and submit a patch to remove >> code parts from little-t-tor, and I'd remove the not-really-useful graph >> from the metrics website. >> >> Cc'ing Rob, Aaron, and Roger as the people who typically have an >> interest in these kinds of statistics. If other tor-dev@ people have an >> opinion on this, please raise your voice! >> >> All the best, >> Karsten >

3 5

Trivial patch: avoid redundant calls to ENGINE_register_all_complete
by coderman 20 Dec '13

20 Dec '13

hello all, Nick, per the other thread in tor-talk about RDRAND, this is the minor fix for OpenSSL 1.0.1+ mentioned. i don't know that this is useful, and i am still giving the engine code a thorough review per Nick's other feedback: "Above all, do not assume that you understand how OpenSSL works until you have investigated with a debugger, the source code, and a pot of coffee." :) best regards, --- diff --git a/src/common/crypto.c b/src/common/crypto.c index 5afb98e..7c02ea4 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -282,7 +282,10 @@ crypto_global_init(int useAccel, const char *accelName, const char *accelDir) log_info(LD_CRYPTO, "Initializing OpenSSL engine support."); ENGINE_load_builtin_engines(); - ENGINE_register_all_complete(); + /* OpenSSL 1.0.1 and newer register complete when engines loaded. */ + if (SSLeay() < OPENSSL_V_SERIES(1,0,1)) { + ENGINE_register_all_complete(); + } if (accelName) { if (accelDir) {

2 3