- tor-dev - lists.torproject.org

Summary of meek's costs, July 2015
by David Fifield 05 Aug '15

05 Aug '15

Here's the summary of meek's CDN fees for July 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 -- total by CDN $5497.73 + $6346.26 + $0.00 = $11843.99 grand total The big event last month was an outage of meek-azure that was caused by a mistake during an account migration: https://lists.torproject.org/pipermail/tor-talk/2015-July/038487.html The outage is unfortunately still ongoing, although there's been progress in getting it fixed (I opened a ticket with Microsoft support and they have to migrate things manually). In the meantime, you can use Tor Browser 5.0a4, which uses a newly configured CDN endpoint. The number of users dropped below 3000, perhaps because of the meek-azure outage. https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Instance hours were up sharply, over 40%, I don't know why. Here is how the Google costs broke down: 2934 GB $352.11 7598 instance hours $379.90 Compared to the previous month: 2807 GB $336.87 5299 instance hours $264.96 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Number of requests was up 8% while bandwidth was down 11%. Asia Pacific (Singapore) 155M requests $187.04 1034 GB $137.11 Asia Pacific (Sydney) 1M requests $2.46 5 GB $0.66 Asia Pacific (Tokyo) 28M requests $34.08 157 GB $20.10 EU (Ireland) 198M requests $238.14 1124 GB $86.91 South America (Sao Paulo) 7M requests $16.02 50 GB $11.94 US East (Northern Virginia) 71M requests $71.71 395 GB $30.90 -- total 463M requests $549.45 2768 GB $287.62 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == The meek-azure outage started on July 20. You can see it clearly in the attached graph, where bandwidth dropped almost to zero. If you are affected by this outage, you should try Tor Browser 5.0a4, which uses a working CDN endpoint for meek-azure. You can also manually change the configuration in other versions of Tor Browser: https://lists.torproject.org/pipermail/tor-talk/2015-July/038496.html https://trac.torproject.org/projects/tor/ticket/16634 https://onionoo.torproject.org/bandwidth?fingerprint=AA033EEB61601B2B7312D8… Estimated bandwidth use this month: 2015-07 1974 GB Compared to last month: 2015-06 3038 GB https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html

1 0

Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
by Nick Mathewson 05 Aug '15

05 Aug '15

Yawning's mail below reminds me: I am considering removing the C implementation of tor-fw-helper from the tor distribution, and recommending Yawning's pure-Go implementation instead. But before I do this, I'd like to get some sense of whether folks are shipping tor-fw-helper today, or using it in practice. On Jul 21, 2015 11:26 AM, "Yawning Angel" <yawning(a)schwanenlied.me> wrote: > > On Wed, 22 Jul 2015 01:06:41 +1000 > teor <teor2345(a)gmail.com> wrote: > > > > > > On 20 Jul 2015, at 11:11 , Serg <std.serg(a)gmail.com> wrote: > > > > > >> How do you plan to map ports on NAT devices? > > > > > > If it can't be done automatically using UPnP, This must be done > > > manually. No alternative cases. > > > > Our experience is that most routers' UPnP / NAT-PMP implementations > > don't work well with (our) automated tools. So this would have to be > > done manually, significantly reducing the pool of available > > volunteers. > > Just chiming in here. This may well work for a good number of users, > but the support overhead for when it fails is utterly gigantic because > certain brands of consumer routers have extremely poor UPnP/NAT-PMP > implementations. > > The usual symptom of a poor implementation is "the router crashes" but > certain other behaviors have been documented in the past by people > trying to use UPnP in ways that are spec compliant such as "the router > crashes and requires a NVRAM reset", "random port mappings get > obliterated", "the UPnP/NAT-PMP stack on the router crashes" etc. I wonder how commercial software handles these cruddy routers. Do they restrict themselves to a tiny part of the spec? Do they probe for problematic firmware, and maintain a list of unreliable versions? [I am very glad it is not our job to maintain such a list.] -- Nick

8 20

curve25519_donna vs. crypto_scalarmult_curve25519?
by Steve Snyder 05 Aug '15

05 Aug '15

Given a contemporary release of Tor with a contemporary version of OpenSSL, under what circumstances is the intrinsic curve25519_donna() preferred over the libsodium/NaCl crypto_scalarmult_curve25519(), or vice versa? Does it come down to 32-bit vs. 64-bit? Or CPU instruction sets detected at build time? Or has libsodium/NaCl been deprecated altogether and just not removed from the configure script? I'm trying to determine if installation of libsodium in a Tor build environment will yield a "better" Tor binary. Thanks.

2 1

5th status report for OnioNS
by Jesse V 04 Aug '15

04 Aug '15

Hi everyone, As I stated in my last status report, I promised to work on server-to-server communication in preparation for a beta test. Quite a bit of work was done in this area, with the result that the server-server protocols are now working as intended and fairly reliably. The changes include: * Ability to bind to a specific IPv4 address. This is useful not only in servers with multiple interfaces, but allows two instances of OnioNS-server to run simultaniously on localhost, which is great for testing and debugging. * Improved server-side logging and diagnostic information. Each network session has it's own identification number, which makes it easy to keep track of events and conversations. * Rewrote the networking code responsible for retransmitting information such as new Records. The servers now correctly sends Records to subscribers. * A Mirror server now reestablishes the network connection with a Quorum server if the Quorum server drops from the network. * Many bug fixes, addressing numerous issues with crashes, networking issues, and serialization faults. With these changes, the OnioNS project is in a pretty good state. Here's an overview of the system's order of operations: 1) A single Quorum server starts up and listens on a specific IP and port. It's a TCP server and waits for JSON-encoded messages. 2) A Mirror server starts up, connects with and subscribes to the Quorum server. 3) A hidden service operator generates a claim on a domain name (say "example.tor") and optionally some subdomains. This claim is represented in the Record data structure. 4) The operator sends the Record over a Tor circuit to the Quorum server, who inspects it, caches it, and sends it to all subscribers, which is the single Mirror node. 5) The Mirror inspects the Record, caches it, and sends it to all subscribers, but here it doesn't have any. 6) A client starts up the Tor Browser with the OnioNS software installed. This means that the starting of the Tor Browser also triggers the launching of the OnioNS software: a binary and a Python script. 7) The user types in "example.tor". Their Python script detects the pseudo-TLD and performs a query lookup to the Mirror node, who replies with the Record. 8) The OnioNS-client software inspects the Record, extracts the hidden service address, and passes the address back to the Stem script. 9) The Stem script rewrites the original stream request from step 7 to this HS address and tells Tor to attach the stream to some circuit. 10) Tor attaches the stream with the effect that the hidden service loads in the Tor Browser. All of these pieces are now in place and generally work as intended without requiring any user intervention. The step of separating the responsibility into two different servers is now behind me, which when coupled with the working subscriber-based server-server communication, gives me many options to move forward towards a distributed system. I have a couple of annoyance-level bugs to fix but a prototype is very near. I will be shortly resume building packages and will currently be supporting Ubuntu, Mint, Debian, and Fedora on i386, amd64, and armhf. As a final note, a quick thanks to George for the well-written commit-reveal-consensus proposal. As he stated in his introduction, I certainly need that in order to integrate OnioNS with the Tor network. Jesse V.

1 0

CompCert C compiler
by Mansour Moufid 04 Aug '15

04 Aug '15

Hi everyone, This is just to let you know that Tor compiles without modification using the CompCert C compiler. This week I'll run it in the "C interpreter" to see what happens. http://compcert.inria.fr/man/manual004.html#cinterp Mansour

1 0

[Measurement Team] Next IRC meeting happens on Wednesday, Aug 5, 14:00 UTC in #tor-project
by Karsten Loesing 03 Aug '15

03 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, August 5, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150805T14 Last week's meeting only had two participants, and while I had a really good time chatting with Philipp, I think it's safe to suggest the same agenda for this week: - Introductions, problems, products, priorities: any feedback, remarks, comments on what we did in the previous four weeks - 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) - Misc/discussion: whatever comes up during the meeting If you think this can be fun, please join us at the meeting! Talk to you on Wednesday! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJVv8ZVAAoJEJD5dJfVqbCrJuAH/2wSlJn8v3m7SDy4kzBEyA5Y lSItMFAn8Q+p6zd4aCkeBxMFUaEJZ8qpNPuq7m8Nv+L5lDAMISpjn8D9z7cH4qLb HGM+Y6oHZwX+e/o/Xj6iVEG8MvJVXB2l1QhXynohGDpJoMX8Lj/lff1bDdt19n2f 5cAULUH5xtFKf4l9XZ7gRgRW+iIZZ3NdI08A/8uZT6PU/rvoKHN9j/52gUIAM+mB j0Vr6xIW38fC3O3N8CnlVmoXglTiuLXJ0HC56VPbGlKRU/Al1pM2M1vk+6ddsi7q 9zcB3qqd485Oj8GFeOUS238GEBawrQk61prKh1xS58Vdji/XfPB/i7vuDgs0Pms= =L7j1 -----END PGP SIGNATURE-----

1 0

Relay Web Dashboard - Summer of Privacy
by Cristobal 31 Jul '15

31 Jul '15

Hello everyone, This summer I'll be working on a Relay Web Dashboard as part of Tor's Summer of Privacy [1]. The goal of this project is to develop a web application to monitor Tor relays. A good part of the back-end code will be based on what nyx [2] currently does (using stem [3]). You can take a look at my SOP proposal here [4] to get a better idea of what I'm planning to do. TL;DR : use Javascript to fetch information from a 'Relay API'. This 'Relay API' will use stem to collect data from the relay, keeping in mind that the idea is to follow a 'read-only' approach, in other words: just monitoring (at least for now). On the first half of SOP I'll be working on making this app based on nyx features. On the second half I'm planning to add new features if necessary, and for that I'd be very interested in hearing your feedback and ideas, specially if you are a relay operator and you think there are features that will help you with the task of running a relay. I hope this project would be of help to relay operators community by giving them a secure, useful and easy-to-use tool for their relays. I'll be informing you about my progress with bi-weekly reports to this list. I'm also on IRC by the nickname @clv Looking forward to hear from you! Best regards, Cristobal GPG: 2ACA 8DD4 444E 358F B72E 8C3C 7196 690D 9E03 CDFD [1] https://trac.torproject.org/projects/tor/wiki/doc/gsoc [2] https://gitweb.torproject.org/nyx.git [3] https://stem.torproject.org/ [4] https://leivaburto.github.io/sop-proposal

4 4

The Shadow Simulator will soon use cooperative threading to better support plug-ins
by Rob Jansen 31 Jul '15

31 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello Shadow! Hello Tor! For the last couple of weeks, I have been overhauling the way Shadow interacts with plug-ins, and I want to make sure you (and others developing/using Shadow plug-ins) are aware of the upcoming changes. I am enhancing Shadow to support the following two capabilities: 1. Run plug-ins that make blocking system calls 2. Run plug-ins that create and use pthreads To achieve this, I am using a user-space cooperative threading library called pth: http://www.gnu.org/software/pth/ Pth works by storing and swapping stack states using makecontext and swapcontext (see man pages), so that we can intelligently switch back to Shadow's execution stack when the plug-in makes blocking libc function calls, and switch back to the plug-in stack only when its file descriptors are waiting for I/O or timers have expired. This same technique allows us to support would-be plug-in pthreads as application layer pth threads instead, which will help us avoid expensive kernel context switches. This will be especially useful/efficient when running thousands of nodes simultaneously, which normally would consume thousands of pthreads which are not Shadow aware. I have modified the pth library to be reentrant and thread-safe using caller-supplied states and thread-specific data, which I needed because Shadow itself runs with multiple simulator worker threads. Cool, huh!? So what does this mean? With this fancy new approach, we must still build each plug-in using LLVM/Clang in order to make use of Shadow's cool state-swapping trick, but we may no longer need to do other customization, particularly those historically needed for programs that block or use threads. Also, with these changes, Shadow's "API" has been removed, and Shadow now just calls the "main" function symbol in each plug-in directly, and the plug-in need not know anything about Shadow. I'm excited about these changes, and hope they will improve support for new plug-ins while also reducing maintenance in the long term. So far I have Shadow's built-in traffic generator running correctly with the new approach, and am now working on Tor. None of this code is public yet, but if you are interested in an example of how this would work or want some early access to the code, drop me a message off-list. Cheers! Rob - -- Rob Jansen, Ph.D. Computer Scientist Center for High Assurance Computer Systems U.S. Naval Research Laboratory rob.g.jansen(a)nrl.navy.mil robgjansen.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJVu4lYAAoJEO3Z5w0UVGXoASAP/3qjW0551xxnrLwWQcrXhbiN cf7hryzU7vDzKRkdzzNbEive0JFZhDONbruJ5Lcx1BW4/uvFZ1emW0FuZFri1m7b xWW6Pp43cMDGeIaMzmWZ9UAea59wirFk/C47R/jDBkXWnZpz5Arc1DjryPbj2LqN oPcfd9JDyE9zk2Y0sSytYt3IulKA62ZOV7+WbIwhHUTj70OgXFvVLhLMMVgOBEDO rHWxDg/+5mC8WqEg4L7lS8/ivAjr1YboA4PV0uSPzgZ+208iQBbsF3DoCG/8D9kY 83xpX/+3aGdWl7XjrncpA5x7gt392fX/3EL9k8lUYu2PKPaOWmJ0/xWbGoXGJOhA epAWPr95X7DFzpcsS59sq6usDeiltFllzfxgA5TbDsUgVwOa9227ygYtK/MVifi6 gYkpZzC6YEMDR6id0ZOwwqYKTsx1Sxomyz30w5iAp1l+Grb10bhUZlfqsNx9PkuI HdBMfvNziApSOQBge36QwskGFlJobfGdpL5w0yep1MSmBJCLVSCopCllLVcbDzQX ipbjmHjJXHNaIL7DHm073VJe2XAjcoMPHHs/nhq0vTgrMFGMDsNvMJ7+66xz3iwZ V3WUjEP0fPjgGZ8MTWj2HjGOQyENa5yAmAiBJUosg1fzgeinushWfE+Zm4m0sVEJ BJttAL/8SnI4DpbIqq6e =c1wf -----END PGP SIGNATURE-----

1 0

Unreachable hidden service by some clients
by Liste 31 Jul '15

31 Jul '15

I was open some hidden service to do an experiment: All work fine. After a day all hidden service on a single server are unreachable from another server used as client. After some hous all hidden server become reachable. Some time when the hidden services are unreachable, they return to work when tor restart. What's going on? We are under attack?

2 1

onionoo.tpo stuck at 2015-07-29 07:00
by nusenu 30 Jul '15

30 Jul '15

2 1