- tor-dev - lists.torproject.org

Retiring old user number estimates
by Karsten Loesing 28 Oct '13

28 Oct '13

Hi everyone, some of you may already know our new approach to estimating daily Tor users: https://metrics.torproject.org/users.html#userstats This new approach is in beta since April, and I'm quite happy with it. I trust the new numbers more than the old ones, both for direct users and bridge users. The new code for direct users is quite similar to the old one, but much cleaner. The approach for bridge users is a much better idea than the old hack. Today I added the missing features like the top-10 lists and the censorship detector. Why do I tell you this? Because the old approach uses resources on our poor, already overloaded metrics machine, and I'm planning to shut down the old approach in the very near future. Here's the plan: - Compute user numbers for 2012 and before; the current numbers start on January 1, 2013. This is going to take at least until September 23. - Take out the "BETA" labels and throw out everything above "New approach to estimating daily Tor users (BETA)". This could happen on October 1. Thoughts? Did I miss anything that's worth keeping? Anyone want to create an archive of their favorite graphs before I pull the plug? All the best, Karsten

5 15

Damian's Status Report - October 2013
by Damian Johnson 28 Oct '13

28 Oct '13

Hi all. Here's a quick update of what I was up to this October... ====================================================================== Stem Release 1.1.0 ====================================================================== After seven months of work Stem version 1.1.0 is finally out! https://blog.torproject.org/blog/stem-release-11 This month was primarily focused on pre-release performance and compatibility improvements in preparation for this... * Handful of improvements that reduced the memory usage of our Descriptor classes by 20%. [1] * Replaced Stem's custom caching with Python's @lru_cache. This was added to the standard library in Python 3.2, so Stem includes a Python 2.x backport to our utilities. [2][3] * Dozens of compatibility fixes for Python 2.6 and 3.x. * Added test coverage for Stem's interpretor examples, courtesy of Python's doctest module. [4][5] ====================================================================== GSoC Mentor Summit ====================================================================== Nick, Moritz and I attended the GSoC Mentor Summit at Google's main campus. By and large it was very similar to previous years, with many of the same faces and discussion topics. Unlike the prior couple years however I did not focus on keeping notes which, on reflection, was a mistake. Here's the small bit I remember... * Had a couple fun language discussions with developers from SciRuby and Scala. The former mostly concerned the advantages and disadvantages of Ruby with respect to Python (I use the former at work, so I've had some time to form an opinion on this). The later concerned Scala support for issuing database queries via list comprehensions. * Couple discussions with Wikimedia developers regarding Tor abuse. Unfortunately this is a perennial discussion that never seems to go anywhere despite having plenty of technical solutions. * Nick gave me a tour of Chutney's codebase and tried to persuade me to take on development of it. Nearly worked too. But while Chutney, SoaT, TorBEL, and a dozen other projects would be a lot of fun to work on I only have so much I can squeeze in alongside a full-time job. * A hackathon organizer with Random Hacks of Kindness discussed what they've found most successfully attracts contributors. The two most memorable tips were momentum and recognition. Regularly organizing events is necessary to maintain a core contributor group, and engaging local media doesn't hurt as it provides recognition for the contributors. * Shadowed Nick during discussions with a Mozilla developer concerning the upcoming Tor IM bundle. They certainly seem excited for OTR support... * Visited with Terry and Arc from the PSF quite a bit. Still no ETA on when distributions will switch to Python 3, and Mailman 3 is still a ways off. Momentum on lots of projects though. Evidently they acted as an umbrella org for roughly sixty students. I would weep bitter, bitter tears if I had to be the admin for that... ====================================================================== Other news this month includes... * Karsten, Andrew and I are now all moderators for the tor email lists. I'm taking point in this area, making daily sweeps through the moderator queue and tweaking Mailman to reduce friction. * Handful of DocTor fixes and improvements. Karsten shut down the Java DocTor early in the month so our shiny, new Python DocTor is now flying solo! * Attended events in the Seattle area including 'Go for Python Developers' at Google Fremont and the second TA3M meetup. * Worked with authority operators to resolve brief outages with dizum and urras. Cheers! -Damian

1 1

Using Stem to monitor TBB usage.
by remi py 27 Oct '13

27 Oct '13

Hy, I am having trouble using Stem to connect to the tor controller, while at the same time browsing the web using the TBB. The tor control settings can be edited in Vidalia -> settings -> advanced. There it is configured by default to use port 9150 and a random password for authentication, but I can not see what password. If I change any of these settings, then I cannot browse anymore, and TB displays "The proxy server is refusing connections". I am using this snippet: import stem from stem.control import Controller with Controller.from_port(port = 9151) as controller: controller.authenticate('') # provide the password here if you set one bytes_read = controller.get_info("traffic/read") bytes_written = controller.get_info("traffic/written") print "My Tor relay has read %s bytes and written %s." % (bytes_read, bytes_written) from https://stem.torproject.org/tutorials/the_little_relay_that_could.html So my problem is that I can not connect to the tor controller while at the same time browsing using the tbb. How do I fix this? Kind regards, Rémi.

3 3

This is my new OnionMail
by enfn 26 Oct '13

26 Oct '13

The first OnionMail server is working. OnionMail + Tails is simple!

2 1

test
by enfn 26 Oct '13

26 Oct '13

Test

1 0

Browser extension identification/fingerprinting mitigation?
by Griffin Boyce 26 Oct '13

26 Oct '13

Hi all, I'm looking at possibly replacing the images used by Cupcake with inline SVG XML, to reduce the possibility of fingerprinting/identifying Cupcake users who use Chrome [1]. One of the more talked-about methods of identifying a user's browser extensions is to look for images used by the extension (in Chrome at least), so this seems to make some amount of sense. [2] Does anyone have any thoughts on this? ~Griffin [1] https://chrome.google.com/webstore/detail/cupcake/dajjbehmbnbppjkcnpdkaniap… [2] http://blog.kotowicz.net/2012/02/intro-to-chrome-addons-hacking.html -- "Cypherpunks write code not flame wars." --Jurre van Bergen #Foucault / PGP: 0xAE792C97 / OTR: saint(a)jabber.ccc.de My posts are my own, not my employer's.

1 0

Re: [tor-dev] Incorporating your torsearch changes into Onionoo
by Kostas Jakeliunas 25 Oct '13

25 Oct '13

On Fri, Oct 11, 2013 at 12:00 PM, Karsten Loesing <karsten(a)torproject.org>wrote: > Hi Kostas, > > should we move this thread to tor-dev@? > Hi Karsten! sure. >From our earlier conversation about your GSoC project: > > In particular, we should discuss how to integrate your project into > > Onionoo. I could imagine that we: > > > > - create a database on the Onionoo machine; > > - run your database importer cronjob right after the current Onionoo > > cronjob; > > - make your code produce statuses documents and store them on disk, > > similar to details/weights/bandwidth documents; > > - let the ResourceServlet use your database to return the > > fingerprints to return documents for; and > > - extend the ResourceServlet to support the new statuses documents. > > > > Maybe I'm overlooking something and you have a better plan? In any > > case, we should take the path that implies writing as little code as > > possible to integrate your code in Onionoo. > > Let me know what you think! > Sounds good. Responding to particular points: > - create a database on the Onionoo machine; > - run your database importer cronjob right after the current Onionoo > cronjob; These should be no problem and make perfect sense. It's always best to use raw SQL table creation routines to make sure the database looks exactly like the one on the dev machine I guess (cf. using SQLAlchemy abstractions to do that (I did that before)). Current SQL script to do that is at [1]. I'll look over it. For example, I'd (still) like to generate some plots showing the chances of two fingerprints having the same substring (this is for the intermediate fingerprint table.) (One axis would be substring length, another would be the possibility in (portions of) %.) As of now, we still use substr(fingerprint, 0, 12), and it is reflected in the schema. Overall though, no particular snags here. > - make your code produce statuses documents and store them on disk, > similar to details/weights/bandwidth documents; Right, so if we are planning to support all V3 network statuses for all fingerprints, how are we to store all the status documents? The idea is to preprocess and serve static JSON documents, correct (as in the current Onionoo)? (cf. the idea of simply caching documents: if we serve a particular status document, it gets cached, and depending on the query parameters (date range restriction, e.g.) it may be set not to expire at all.) Or should we try and actually store all the statuses (the condensed status document version [2], of course)? > - let the ResourceServlet use your database to return the > fingerprints to return documents for; and > - extend the ResourceServlet to support the new statuses documents. Sounds good. I assume you are very busy with other things as well, so ideally maybe you had in mind that I could try and do the Java part? :) Though, since you are much more familiar with (your own) code, you could probably do it faster than me. Not sure. Any particular technical issues/nuances here (re: ResourceServlet)? cheerio Kostas. [1]: https://github.com/wfn/torsearch/blob/master/db/db_create.sql [2]: https://github.com/wfn/torsearch/blob/master/docs/onionoo_api.md#network-st… (e.g. http://ts.mkj.lt:5555/statuses?lookup=9695DFC35FFEB861329B9F1AB04C46397020C… )

2 2

Guard-security projects extracted from Roger's blog post
by George Kadianakis 25 Oct '13

25 Oct '13

Hey Nick, I made a pad with some of the tasks that Roger mentioned in his recent blog post [0]. The pad can be found here: https://pad.riseup.net/p/BQl2W58RLurU_guard It's probably not an exhaustive list and needs more work. Unfortunately I won't have time to work on it during the weekend so I'll post it here in case you find it useful. Cheers! [0]: https://blog.torproject.org/blog/improving-tors-anonymity-changing-guard-pa…

1 0

Re: [tor-dev] [tor-assistants] Finding a maintainer for Weather
by Damian Johnson 23 Oct '13

23 Oct '13

For those on tor-dev@, this is a thread where Karsten and I have been talking about the future of Tor Weather. Present thoughts are to do one of the following... a. Attempt to tap the community to find a maintainer (the service has been unmaintained for years). b. Hire a developer from odesk to address its present issues. Karsten and I aren't especially thrilled about this idea since it would mean continuing to have an unmaintained service linger on. c. Replace the functionality Weather provides with something else (that's most of what our recent discussion has been about). d. Simply deprecate and later shut down the service. >>> 3. We redesign Weather based on stem. >> >> As mentioned earlier this is already done to the extent that a patch >> exists. Weather's use of TorCtl was pretty simple so it was easy to >> map to stem, making some general improvements to the codebase in the >> process. If we opt for any plan involving the continuation of Weather >> than I'd like to see us do this since it's both easy, and will let >> Weather move off a deprecated dependency. We can then shift the >> service to Onionoo or rewrite it afterwards. > > There's a patch for this? Sounds like a new Weather maintainer should > apply this patch as their first action. (However, I'm not sure if we > should apply it without having a maintainer, because if it breaks, who's > going to pick up the pieces? Unless you'd want to do that?) Yup, a patch but untested (I didn't invest time into starting up a new Weather instance). So we definitely do need a maintainer before applying it. >>> 5. (Even smarter ideas go here) >> >> What about my earlier suggestion? I'd like to replace weather with a >> new torrc parameter, say... >> >> SendNotifications me(a)some_address.com >> >> ... which is then included in the extrainfo descriptors. DocTor (or a >> new Weather if we really don't want to merge services) could then send >> notifications when a relay that previously set this goes down, or has >> reached a point of deserving a t-shirt. The only trouble with this is >> that it would expose real addresses in descriptor data (and hence, >> possibly spam). >> >> Maybe there's something smart we can do to overcome this issue? >> Providing relay operators with a method for including real contact >> information would be helpful (as mentioned with the upgrade scenario >> earlier in the thread). > > But we already have ContactInfo lines. What's the difference between > the two? I'd find this rather confusing as a relay operator. Differences are... * This option would be an opt-in for receiving notifications about their relay. * It resides in extrainfo descriptors rather than server descriptors (not sure if that'll lessen spam, but ContactInfo's definitely in the wrong spot). * We've already trained users to throw garbage information into the ContactInfo. Since SendNotifications would be for automated systems like DocTor we could clearly state that it requires un-munged addresses, lest it doesn't work (and hence is pointless to set). This said, without a solution to the spam issue I don't think this is a good idea. I'm hoping others on this list will have some ideas around this since a solution for the spam problem is the only thing standing in our way for having a good replacement for Weather. > Also, I dislike the fact that we're adding a config option to tor that > is mostly used by the Weather service. What if we want to discontinue > the Weather service? The config option will be around for another two > years after that, but it won't do anything, which will make people think > the tor program is broken. Actually, DocTor rather than Weather - I want to shut Weather down. In theory this is a more general field for 'hey, automated services may send me notices regarding my relay', maybe also with 'here's tags for the notices I want'. Anyway, food for thought - an opt-in flag and email address is really most of what Weather provides us. > Also also, should this discussion happen on tor-dev@? :) Perfectly fine with me - added. Cheers! -Damian

3 4

Attentive Otter: Usability issues with existing OTR clients
by Derek Moss 23 Oct '13

23 Oct '13

>As you point out, you get a "Unverified/Verified" when using OTR. Verifying >fingerprints of anonymous chat is practically impossible -- every side >channel I can think of (telephone, meeting in person, email - ha) either >leaks the act of contacting, isn't practical or convenient, or requires >some kind of Internet reputation. I'd bet that most OTR users don't verify >fingerprints. >TOFU is a must, along with a dialogue that explains in simple terms what is >going on, with a way to dismiss the dialogue (e.g. by correctly answering a >security question). >Perhaps replacing "Unverified" with "In use for: x days" ? Changing fingerprints seems like a big problem, because it's reasonable for users to verify them when they first add a user to their contacts (via phone or whatever) but they're not going to keep doing that if the contact keeps changing their fingerprint and will probably just click OK when warned that the fingerprint has changed. I have no idea if this exposes a risk from a MITM attack or some other compromise but it would seem like it might. Certainly it should be easy to bring up a contact's fingerprint for verifying, by right-clicking on their name for example and not buried away somewhere in the preferences. > > We should have per-message icons for > > both sent and received messages that indicate encryption status for that > > message in the chat scrollback window. >If the Attentive Otter client is going to be "Force OTR", then you'd >only need a special flag for "received insecurely" (much as pidgin-otr >has). With Jitsi, I think it's not clear enough whether chats are secured or not and there seems to be some bugs that cause it send unsecured messages despite the "Force OTR" setting. I've had problems with it sending unencrypted messages when starting a conversation from one end but not in the reverse direction. Obviously this needs to be prevented and I like the idea of just not sending any text until a secure connection has been established. For Attentive Otter, hopefully the "Force OTR" can be made rock-solid and so it should only need to indicate unsecured incoming message (or maybe even block those but perhaps there's not much benefit from doing so). >Is "skull and crossbones" an easily understandable icon for "message >received insecurely"? It would certainly need a tooltip hint, or >something like that. I think a padlock icon is much more understandable for most users. > > 7. Logging should be off by default when OTR is on. >Agree (as with pidgin-otr 4.x). >> I guess people like logging for some reason, and I'll admit that pidgins >>"History" mode really enables asynchronous chat. Rather than require you >> *turn off* OTR to get this feature, an alternative would be to require a >> passphrase and automatically expunge encrypted logs after a configurable >> period of time. And, unlike other clients, perhaps telling the conversation >> counterparty that logging is enabled (e.g. the opposite of gchat "OTR" >> notifications ) at the start of the session would provide informed consent. >Unfortunately, such counterparty notifications cannot be relied on, as >you could have hacked your client. On the issue of logging, I'd suggest it would be best to have no logging features at all. That way, they can't be enabled accidentally or deliberately and other than the other user running a hacked version (either on purpose or unknown to them) both users can have more confidence that their chat isn't being logged. Of course, this won't prevent the other user cutting and pasting the conversation to manually log it but that comes down to trusting them, whereas logging features can cause a security breach without either party intending to do so. This was a big issue for a number of users on the Jitsi mailing list and we were rather disappointed that the devs seemed more focused on the convenience some users get from being able to refer back to old conversations than the security from not having conversations logged. Although the devs didn't really accept that logging could be a security problem and felt that Jitsi's purpose was to secure chats in transit and nothing else. If you must have logging, at least encrypt the logs so that they're not sitting around in plaintext for anyone to grab. Despite the issues I've mentioned with Jitsi, I wonder if it might not be an alternative starting point to InstantBird? I see that it's been decided you don't want to use Pidgin/libpurple but wonder if other alternatives, like Jitsi which I understand is free to be forked, have been considered, where much of the required OTR work might already have been done?

1 0