- tor-dev - lists.torproject.org

Spinning off Directory Archive from Metrics Portal
by Karsten Loesing 07 Jul '14

07 Jul '14

Hello devs, I'm continuously tweaking the Metrics Portal [0] in the attempt to make it more useful. My latest idea is to finally spin off the Directory Archive part from it, which is the part that serves descriptor tarballs. I'd like to hear what people think about that. Let me give you some more context. The Metrics Portal serves three main purposes: 1. Graphs [1]: there are graphs on network size, network diversity, user number estimates, and performance measurements. This is probably what most visitors are interested in. In addition to graphs, there are also tables and .csv files for download. 2. Research [2]: we offer descriptor tarballs for download and explain the data formats. This is mostly interesting for researchers and developers. (This is the part that I'd like to spin off and move to a separate place.) 3. Status [3]: we provided (and still provide) services that are based on archived descriptors. This includes ExoneraTor and Consensus Health, both of which have moved to their own sites, and it includes Relay Search which is high on the list of endangered services. While we (did) provide these services on the Metrics Portal, they're almost gone. So, my plan is to move away the Research part, number 2 above, and re-organize the remaining Graphs part to address visitors who are not necessarily researchers or developers. The new Directory Archive website would contain the following content: - Start page: possibly re-using parts from the current start page [0]. - Data: How to obtain the data, possibly re-using parts from the current Data page [4], but without separate pages for file lists. - Formats: similar to the current Data Formats page [5]. - Tools: similar to the current Tools page [6]. What would be a good name for the new website holding the Tor Directory Archive? How about: - CollecTor, collector.torproject.org (not available yet) or - AggregaTor, aggregator.torproject.org (not available yet) On the software side, I'd like to remove all dynamic (Java) parts from the new website and have it served by Apache alone instead of Tomcat. The only parts that still need to be dynamically generated would be file lists, and I'd like to solve that by using Apache directory listings or some other Apache module. The re-organized Metrics Portal is going to have the following content: - Start page: possibly re-using parts from the current start page [0]. - Graphs: all four sub pages from the current Graphs page [1], so Network, Bubbles, Users, and Performance. - Aggregated data: the processed data behind graphs that is currently available on the Statistics page [7]. Speaking of, if anybody wants to help design the new website (or help re-design the existing Metrics Portal website once the cruft is gone), your help would be much appreciated. Bonus points if no JavaScript is required, at least for the Directory Archive website. Please contact me if you're interested. Any feedback welcome! Thanks! All the best, Karsten [0] https://metrics.torproject.org/ [1] https://metrics.torproject.org/graphs.html [2] https://metrics.torproject.org/research.html [3] https://metrics.torproject.org/status.html [4] https://metrics.torproject.org/data.html [5] https://metrics.torproject.org/formats.html [6] https://metrics.torproject.org/tools.html [7] https://metrics.torproject.org/stats.html

3 6

7 Dir Servers Dropping - Doctor Error?
by Tom Ritter 07 Jul '14

07 Jul '14

On 6 July 2014 18:59, doctor role account <doctor(a)cappadocicum.torproject.org> wrote: > ERROR: Unable to retrieve the consensus from maatuska (http://171.25.193.9:443/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from tor26 (http://86.59.21.38:80/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from urras (http://208.83.223.34:443/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from gabelmoo (http://212.112.245.170:80/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from moria1 (http://128.31.0.39:9131/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from dannenberg (http://193.23.244.244:80/tor/status-vote/current/consensus) timed out > ERROR: Unable to retrieve the consensus from Faravahar (http://154.35.32.5:80/tor/status-vote/current/consensus) timed out I've been keeping an eye on consensus health, it seems to be pretty reliable, but this is an obvious outlier. I assume it was a doctor error of some sort? -tom

2 1

Revised spec for on adding extra torrc fields
by Virgil Griffith 06 Jul '14

06 Jul '14

Damian and I worked through this off-list and this is the output of our consensus. URL: https://dl.dropboxusercontent.com/u/3308162/ExtraRelayDescriptorFields.v3.t… Any further revisions to be made before adding this to torspec? -V

1 0

GSoC Stegotorus Security Enhancement
by Noah Rahman 05 Jul '14

05 Jul '14

Greetings all, I've been refining the Elligator handshake to work on the circuit level for now as opposed to the connection level[1], merging in some SRI changes[2], especially those with respect to configuration files[3], and studying resources in order to prepare for a possible implementation of Elligator^2 [4][5](with parts in Lua [6]) as well as understanding the new SRI JumpBox alternative [7] to the transparent proxy that vmon developed. Happy fireworks (for you US-based readers!) [1] https://github.com/TheTorProject/stegotorus/tree/f--chop-handshaker [2] https://github.com/SRI-CSL/stegotorus [3] https://github.com/TheTorProject/stegotorus/tree/f--modus-operandi [4] http://www.hackersdelight.org/divcMore.pdf [5] http://www.hackersdelight.org/MontgomeryMultiplication.pdf [6] I also can probably implement concurrency in Lua faster than in C++ or Go, and Lua bignum libraries do exist. Also, why this numbering begins at 1 instead of zero ;p [7] Vinod Yegneswaran, private communcation.

1 0

GSOC Pluggable Transport Combiner Status Update 3
by quinn jarrell 05 Jul '14

05 Jul '14

Hi everyone, I've been working on: - Making tor connect to the pt combiner directly instead of connecting to the first pt as it did before. Now when tor sends a socks request to the combiner, the socks request is intercepted and a connection is made to the first pt instead of the requested bridge. The last pt has its destination set as the original requested bridge address so when the last pt is finally given the data to obfuscate, it will forward it to the actual bridge. - Added a configuration file reader to the pt combiner server - Turned the obfs3_flashproxy method into a configuration file and set it as the default configuration file if one is not specified. - Worked on rearranging the project structure into a python and go project [0] You can see my progress at [1] [0] https://trac.torproject.org/projects/tor/ticket/12215 [1] https://github.com/infinity0/obfs-flash

1 0

Re: [tor-dev] Response on adding extra torrc fields
by Damian Johnson 05 Jul '14

05 Jul '14

> Hi Damian! I'm trying to make this work under your suggestions, but I'm not > entirely clear how. Can you help me get this in shape to pass your muster? Hi Virgil, I'd be glad to! Adding tor-dev@ back in case others have thoughts on the details. I like your idea, what I was trying to hint by asking about the motivation was that the proposal should clearly explain why we're doing this (with example use cases). As for detail, I mean the proposal should have text we can copy and paste into the dir-spec. Maybe something like... ============================================================ The following would be added to section 2.1.2 of the dir-spec (Extra-info document format): "X-" CustomKey SP CustomValue NL CustomKeyChar = ALPHA / DIGIT / "_" / "-" CustomKey = 1*20 CustomKeyChar CustomValueChar = atext / SP CustomValue = 1*200 CustomValueChar Custom field that can appear multiple times, for example... X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-favoritequote Be excellent to each other. Party on dudes! X-foo bar A CustomKey cannot appear more than once, and there cannot be more than ten of these entries. ============================================================ As a controller library author what I want from proposals is enough detail so I know *exactly* how to parse new additions. There's lot of potential edge cases. Can values be an empty string? How about keys? What should we do if a key appears multiple times? Is that valid? You mentioned a 5 KB limit, is that per option or for all custom fields? Rather than a size threshold, in the above I'm proposing a limit of ten entries each of which can be at most 220 characters. That would only be 2 KB so feel free to tweak these limitations. I'm not entirely sure 'atext' is what we want here. This would exclude tabs, but maybe that's not such a bad thing. You can look it up on... http://www.ietf.org/rfc/rfc2822.txt Cheers! -Damian

1 0

GSoC : Rewrite Tor Weather (Status Update 3)
by Sreenatha Bhatlapenumarthi 04 Jul '14

04 Jul '14

Hi all, I've written most of the new weather's backend a.k.a the notification scripts that run hourly(LowBandwidth, Nodedown) and daily(Welcome, Tshirt) as cron jobs and this is how they essentially operate: 1. Fetch and parse required documents from Onionoo. 2. Filter out required relays(e.g 'running', 'stable' flagged for welcome email). 3. For each filtered relay: 3.a. For each notification type: 3.a.a. Query the corresponding subscription model to get subscribers' info. 3.a.b. Collect the email(to, from, subject, content) to be sent. 4. Send out the collected mass email. 5. Delete old entries and/or update the django models. I happened to make some changes the onionoo-wrapper module in this process by modifying the response objects, utilities etc. The current version with all the changes is available at https://github.com/lucyd/tor-weather/tree/backend. After fixing a couple of issues, I setup the vagrant+puppet environment that was previously configured(#11417). I tested out the notification scripts and although the daily script is working fine, the hourly script is giving a couple of errors for now. I plan on fixing these pretty soon and testing the overall weather to make sure everything's working fine. Cheers, Sreenatha

1 0

[GSoC] Consensus diffs - Third report
by Daniel Martí 04 Jul '14

04 Jul '14

Hello everyone, This is the thid status report of my Google Summer of Code project, which is to implement consensus diffs for Tor. We hold weekly meetings every Wednesday at 14h UTC with the project mentors Nick and Sebastian. Two weeks ago I sent the second report, which explained how I had working versions of both the ed diff generation and application code, and that I would move on to writing tests, fixing bugs and starting the integration into Tor. And that's what I've done these past two weeks. I have restrured my codebase to make it fit into the main repository (I came up with the 'consdiff' name, so consdiff.c and consdiff.h - criticism or better names welcome). I have also started writing tests in test_consdiff.c, which are extensive but still have room for improvement. Most importantly, while writing the tests and adding assertions I found a handful of issues which are now solved. I'll try to come up with as many edge cases as I can to get rid of any remaining bugs. All the code is up on github: https://github.com/mvdan/tor_sample Our schedules this week have been difficult in part due to the tor-dev summer meeting, so not much else has happened other than my coding. Hopefully we'll be back at full speed soon! -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

1 0

Gsoc tor daemon status report
by Белоус Михаил 04 Jul '14

04 Jul '14

As I said before applying for Gsoc,until 19 July I'm in military camp. After my return I'm going to rewrite code.

1 0

XKeyscore rules probably are from Snowden, after all
by Maxim Kammerer 04 Jul '14

04 Jul '14

There has been some speculation that the recent XKeyscore rule leaks [1] do not come from Snowden — particularly, by Schneier [2]. I believe that there is a good case that the leaks do come from Snowden, since it is possible to pinpoint the date range when the rule sources [3] have been last updated. The earliest possible date is 2011-08-08, when the Linux Journal writeup about Tails [4], referenced by the glob pattern "linuxjournal.com/content/linux*" has been published. The pattern is not a generic Linux Journal filter, as implied in [1]. The likely latest possible date is 2012-02-28, when "maatuska" directory authority has changed its IP [5]. A less likely upper bound is 2012-09-21, when "Faravahar" directory authority has been added [6]. NSA either took the 8 authorities from the actual consensus, or picked them from Tor's sources [7]. However, Tor sources list more than 8 authorities, and are not properly maintained (e.g., see entry for "moria1" wrt. its last .34/.39 octet tweaks), so I doubt NSA would use that. Moreover, it is hard to miss the port number in the sources, whereas NSA did miss that some authorities do not (and did not) use ports 80/443. E.g., "moria1" (the MIT campus server mentioned in [1]) would not be matched as a Tor authority by the rules. Snowden most likely tried to contact Greenwald at the end of 2012 [8], which is entirely consistent with the above. Another NSA employee leaking XKeyscore rules after being inspired by Snowden's leaks, would have probably downloaded a more up-to-date rules file. Cross-posting to tor-dev, in case I got any historical directory authority changes wrong. [1] http://daserste.ndr.de/panorama/aktuell/nsa230_page-1.html [2] https://www.schneier.com/blog/archives/2014/07/nsa_targets_pri.html [3] http://daserste.ndr.de/panorama/xkeyscorerules100.txt [4] http://www.linuxjournal.com/content/linux-distro-tales-you-can-never-be-too… [5] https://lists.torproject.org/pipermail/tor-dev/2012-February/003312.html [6] https://trac.torproject.org/projects/tor/ticket/5749 [7] https://gitweb.torproject.org/tor.git/blob/HEAD:/src/or/config.c [8] http://www.nytimes.com/2013/08/18/magazine/laura-poitras-snowden.html -- Maxim Kammerer Liberté Linux: http://dee.su/liberte

1 0