- tor-dev - lists.torproject.org

Revised spec for on adding extra torrc fields
by Virgil Griffith 06 Jul '14

06 Jul '14

Damian and I worked through this off-list and this is the output of our consensus. URL: https://dl.dropboxusercontent.com/u/3308162/ExtraRelayDescriptorFields.v3.t… Any further revisions to be made before adding this to torspec? -V

1 0

GSoC Stegotorus Security Enhancement
by Noah Rahman 05 Jul '14

05 Jul '14

Greetings all, I've been refining the Elligator handshake to work on the circuit level for now as opposed to the connection level[1], merging in some SRI changes[2], especially those with respect to configuration files[3], and studying resources in order to prepare for a possible implementation of Elligator^2 [4][5](with parts in Lua [6]) as well as understanding the new SRI JumpBox alternative [7] to the transparent proxy that vmon developed. Happy fireworks (for you US-based readers!) [1] https://github.com/TheTorProject/stegotorus/tree/f--chop-handshaker [2] https://github.com/SRI-CSL/stegotorus [3] https://github.com/TheTorProject/stegotorus/tree/f--modus-operandi [4] http://www.hackersdelight.org/divcMore.pdf [5] http://www.hackersdelight.org/MontgomeryMultiplication.pdf [6] I also can probably implement concurrency in Lua faster than in C++ or Go, and Lua bignum libraries do exist. Also, why this numbering begins at 1 instead of zero ;p [7] Vinod Yegneswaran, private communcation.

1 0

GSOC Pluggable Transport Combiner Status Update 3
by quinn jarrell 05 Jul '14

05 Jul '14

Hi everyone, I've been working on: - Making tor connect to the pt combiner directly instead of connecting to the first pt as it did before. Now when tor sends a socks request to the combiner, the socks request is intercepted and a connection is made to the first pt instead of the requested bridge. The last pt has its destination set as the original requested bridge address so when the last pt is finally given the data to obfuscate, it will forward it to the actual bridge. - Added a configuration file reader to the pt combiner server - Turned the obfs3_flashproxy method into a configuration file and set it as the default configuration file if one is not specified. - Worked on rearranging the project structure into a python and go project [0] You can see my progress at [1] [0] https://trac.torproject.org/projects/tor/ticket/12215 [1] https://github.com/infinity0/obfs-flash

1 0

Re: [tor-dev] Response on adding extra torrc fields
by Damian Johnson 05 Jul '14

05 Jul '14

> Hi Damian! I'm trying to make this work under your suggestions, but I'm not > entirely clear how. Can you help me get this in shape to pass your muster? Hi Virgil, I'd be glad to! Adding tor-dev@ back in case others have thoughts on the details. I like your idea, what I was trying to hint by asking about the motivation was that the proposal should clearly explain why we're doing this (with example use cases). As for detail, I mean the proposal should have text we can copy and paste into the dir-spec. Maybe something like... ============================================================ The following would be added to section 2.1.2 of the dir-spec (Extra-info document format): "X-" CustomKey SP CustomValue NL CustomKeyChar = ALPHA / DIGIT / "_" / "-" CustomKey = 1*20 CustomKeyChar CustomValueChar = atext / SP CustomValue = 1*200 CustomValueChar Custom field that can appear multiple times, for example... X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-favoritequote Be excellent to each other. Party on dudes! X-foo bar A CustomKey cannot appear more than once, and there cannot be more than ten of these entries. ============================================================ As a controller library author what I want from proposals is enough detail so I know *exactly* how to parse new additions. There's lot of potential edge cases. Can values be an empty string? How about keys? What should we do if a key appears multiple times? Is that valid? You mentioned a 5 KB limit, is that per option or for all custom fields? Rather than a size threshold, in the above I'm proposing a limit of ten entries each of which can be at most 220 characters. That would only be 2 KB so feel free to tweak these limitations. I'm not entirely sure 'atext' is what we want here. This would exclude tabs, but maybe that's not such a bad thing. You can look it up on... http://www.ietf.org/rfc/rfc2822.txt Cheers! -Damian

1 0

GSoC : Rewrite Tor Weather (Status Update 3)
by Sreenatha Bhatlapenumarthi 04 Jul '14

04 Jul '14

Hi all, I've written most of the new weather's backend a.k.a the notification scripts that run hourly(LowBandwidth, Nodedown) and daily(Welcome, Tshirt) as cron jobs and this is how they essentially operate: 1. Fetch and parse required documents from Onionoo. 2. Filter out required relays(e.g 'running', 'stable' flagged for welcome email). 3. For each filtered relay: 3.a. For each notification type: 3.a.a. Query the corresponding subscription model to get subscribers' info. 3.a.b. Collect the email(to, from, subject, content) to be sent. 4. Send out the collected mass email. 5. Delete old entries and/or update the django models. I happened to make some changes the onionoo-wrapper module in this process by modifying the response objects, utilities etc. The current version with all the changes is available at https://github.com/lucyd/tor-weather/tree/backend. After fixing a couple of issues, I setup the vagrant+puppet environment that was previously configured(#11417). I tested out the notification scripts and although the daily script is working fine, the hourly script is giving a couple of errors for now. I plan on fixing these pretty soon and testing the overall weather to make sure everything's working fine. Cheers, Sreenatha

1 0

[GSoC] Consensus diffs - Third report
by Daniel Martí 04 Jul '14

04 Jul '14

Hello everyone, This is the thid status report of my Google Summer of Code project, which is to implement consensus diffs for Tor. We hold weekly meetings every Wednesday at 14h UTC with the project mentors Nick and Sebastian. Two weeks ago I sent the second report, which explained how I had working versions of both the ed diff generation and application code, and that I would move on to writing tests, fixing bugs and starting the integration into Tor. And that's what I've done these past two weeks. I have restrured my codebase to make it fit into the main repository (I came up with the 'consdiff' name, so consdiff.c and consdiff.h - criticism or better names welcome). I have also started writing tests in test_consdiff.c, which are extensive but still have room for improvement. Most importantly, while writing the tests and adding assertions I found a handful of issues which are now solved. I'll try to come up with as many edge cases as I can to get rid of any remaining bugs. All the code is up on github: https://github.com/mvdan/tor_sample Our schedules this week have been difficult in part due to the tor-dev summer meeting, so not much else has happened other than my coding. Hopefully we'll be back at full speed soon! -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

1 0

Gsoc tor daemon status report
by Белоус Михаил 04 Jul '14

04 Jul '14

As I said before applying for Gsoc,until 19 July I'm in military camp. After my return I'm going to rewrite code.

1 0

XKeyscore rules probably are from Snowden, after all
by Maxim Kammerer 04 Jul '14

04 Jul '14

There has been some speculation that the recent XKeyscore rule leaks [1] do not come from Snowden — particularly, by Schneier [2]. I believe that there is a good case that the leaks do come from Snowden, since it is possible to pinpoint the date range when the rule sources [3] have been last updated. The earliest possible date is 2011-08-08, when the Linux Journal writeup about Tails [4], referenced by the glob pattern "linuxjournal.com/content/linux*" has been published. The pattern is not a generic Linux Journal filter, as implied in [1]. The likely latest possible date is 2012-02-28, when "maatuska" directory authority has changed its IP [5]. A less likely upper bound is 2012-09-21, when "Faravahar" directory authority has been added [6]. NSA either took the 8 authorities from the actual consensus, or picked them from Tor's sources [7]. However, Tor sources list more than 8 authorities, and are not properly maintained (e.g., see entry for "moria1" wrt. its last .34/.39 octet tweaks), so I doubt NSA would use that. Moreover, it is hard to miss the port number in the sources, whereas NSA did miss that some authorities do not (and did not) use ports 80/443. E.g., "moria1" (the MIT campus server mentioned in [1]) would not be matched as a Tor authority by the rules. Snowden most likely tried to contact Greenwald at the end of 2012 [8], which is entirely consistent with the above. Another NSA employee leaking XKeyscore rules after being inspired by Snowden's leaks, would have probably downloaded a more up-to-date rules file. Cross-posting to tor-dev, in case I got any historical directory authority changes wrong. [1] http://daserste.ndr.de/panorama/aktuell/nsa230_page-1.html [2] https://www.schneier.com/blog/archives/2014/07/nsa_targets_pri.html [3] http://daserste.ndr.de/panorama/xkeyscorerules100.txt [4] http://www.linuxjournal.com/content/linux-distro-tales-you-can-never-be-too… [5] https://lists.torproject.org/pipermail/tor-dev/2012-February/003312.html [6] https://trac.torproject.org/projects/tor/ticket/5749 [7] https://gitweb.torproject.org/tor.git/blob/HEAD:/src/or/config.c [8] http://www.nytimes.com/2013/08/18/magazine/laura-poitras-snowden.html -- Maxim Kammerer Liberté Linux: http://dee.su/liberte

1 0

Orbot & Orfox - GSoC bi-weekly report 3
by Amogh Pradeep 04 Jul '14

04 Jul '14

Status Report 3 July 4th, 2014. Things I have done: 1)Building Fennec: Finally built Fennec and applied the patches to provide anonymity to some extent. The apk can be found at https://guardianproject.info/releases/FennecForTor_GSoC_prototype.apk . This is just a prototype and requires the original browser code to be a fully anonymous browser. 2)Building tor-browser: I downloaded the source for tor-browser which is based on firefox 24esr. Built it, and it runs, makes an apk which works to some extent but I was informed that the browser won't function unless certain plugins are added to it. This means that I'm going to have to port the plugins to work on Fennec or find a usable alternative.

1 0

Torspec proposal for adding new X- fields to relay descriptor
by Virgil Griffith 03 Jul '14

03 Jul '14

URL: https://dl.dropboxusercontent.com/u/3308162/ExtraRelayDescriptorFields.txt Fulltext below. Comments appreciated. -V ======================= Filename: ExtraRelayDescriptorFields.txt Title: Adding new X- fields to relay descriptor Author: Virgil Griffith, Nick Mathewson Created: 2014-06-03 Status: Open 1. Motivation We wish to allow developers to build new applications atop relays. Towards this end, we wish to add the ability for users to specify arbitrary new key-value entries under the "X-" namespace. 2. Proposal Allow optional key-value lines in the relay's torrc file. These lines will be mirrored in the relay's descriptor which is then published in the directory consensus. The format is: X-key value For example: X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-nameid virgil.gr X-favoritequote Be excellent to each other. Party on dudes! The value field must be printable ASCII (characters 32-126). The value must not under any condition contain a newline. The key may contain lowercase ASCII letters (a-z), digits, underscore, or dash. In regex, [-_0-9a-z]. There may need to be a maximum sum length of the X- entries. This is left to the developers. I propose a maximum sum length of 5 kilobytes.

4 4

Re: [tor-dev] I have a group at internet archive that are interested in buying a lot of OnionPi's
by Virgil Griffith 02 Jul '14

02 Jul '14

Roger et al, I'm interested in something like onion-pi to be a Tor relay. Is there something with enough COU to be viable? I know nothing about this embedded scene. -V

2 1

[GSoC 2014] Revamp GetTor status report #3
by Israel Leiva 02 Jul '14

02 Jul '14

Hi all! This is my third report for the Revamp GetTor GSoC project. It's not much, since I'm finishing my semester and I haven't had much time to do complex stuff. Achievements: [*] Fixed minor issues pointed out by my mentor on the Core module (still one remaining). The biggest thing was to eliminate unnecessary inspection of the stack to know which service was calling. I implemented a rather simple solution: the service should be specified when calling the Core module. This will help for statistics. [*] Create the basis of the SMTP module. For now, it has the main classes and methods to be used, with basic documentation and logging. There are some methods with working code, but nothing definitive yet. [*] Created a dummy script to test the SMTP module. For now, it just reads emails from stdin (eml files). Goals until next bi-weekly report: [+] Fix the one remaining issue on the Core module. [+] Have a functional SMTP module, hopefully tested on a real server. As always, you can check what's been done on [0]. As I mentioned earlier, I'm finishing my semester and I'll be free by mid of the next week. After that, I'll considerably increase my productivity in order to achieve the timeline proposed. [0]: https://github.com/ileiva/gettor/tree/master/src Best, -- israel

1 0

(no subject)
by Gareth Owen 01 Jul '14

01 Jul '14

Dear all I'm working on a tor research project and am having difficulty understanding the SENDME cells. The tor spec acknowledges that it isn't particularly clear so I would welcome some clarification please. The spec says that *all* nodes in a circuit maintain a send and receive window, and that this window is decremented on each RELAY_DATA and incremented on each SENDME. Cells that are neither of these do not affect the window size. The problem I have understanding is, that only edge nodes will know whether a cell is a RELAY_DATA, the intermediate nodes only know that its a RELAY but not what type. So, if only RELAY_DATA decrements the window size, and intermediate nodes cannot spot these, what point is there in intermediate nodes having a window? Any help greatly appreciated. Gareth --- Dr Gareth Owen Senior Lecturer School of Computing, University of Portsmouth W: ghowen.me

1 0

Re: [tor-dev] I have a group at internet archive that are, interested in buying a lot of OnionPi's
by Virgil Griffith 01 Jul '14

01 Jul '14

It's already established that, for clients, onion-pi's are discouraged---onion-pi wifi doesn't protect enough (I.e., at all) from browser-based attacks. Given that, The question is now, "Are onion-pi's are good enough to be useful relays?" Roger said no. Is there a more informed opinion on this matter---particularly from someone who has actually tried this? Are there any relays that are known to run on onion-pi? If an onion-pi is insufficient for a useful Tor relay, what is the limiting reagent? What more does it need to be useful? -V

2 1

Disk full under .tor
by grarpamp 30 Jun '14

30 Jun '14

Tor seems to have kept running ok through at least one disk full instance here. Posting in case such rare edge conditions haven't yet been fully 'do the right thinged' in design. cached-consensus": No space left on device cached-descriptors.new": No space left on device Unable to store router descriptor

1 0

I have a group at internet archive that are interested in buying a lot of OnionPi's
by Virgil Griffith 29 Jun '14

29 Jun '14

What is the current state of the art on this, and if it is ready for larger deployment want to buy about 50-100 of them. -V

5 4

Re: [tor-dev] I have a group at internet archive that are, interested in buying a lot of OnionPi's
by Rémi 29 Jun '14

29 Jun '14

Roger wrote: > And the onionpi boxes don't have enough cpu to be a useful relay. I'm not sure what the definition of 'useful relay' is, but I am running an exit relay with 900KB/s and between 1000-1500 consensus weight. This is the limit for the pi, but definitively above the 100KB/s I read somewhere else. Also, although I know it is nowhere near decent security, I do get a warm feeling from the fact that all my non-tor home traffic gets mixed/emitted along with tor exit traffic. I'm not running an onionpi because I prefer TorBrowser for all the good reasons, but the pi makes a nice little relay I think. R.

1 0

Proposal for improving social incentives for relay operators
by Virgil Griffith 29 Jun '14

29 Jun '14

For a while I've been seeking to grow the Tor network in both size and goodput. Towards this end, I've explored various avenues such as increasing user-awareness via tor2web. More recently, I've been exploring financial incentives like TorCoin. Not wanting to strictly limit ourselves to financial incentives, I began reading the literature on incentivizing volunteers. The most relevant papers I found are: * http://www-2.rotman.utoronto.ca/facbios/file/LMS2_ManSci-Paper-Final.pdf * http://pareto.uab.es/~prey/gneezy_254.pdf * https://dl.dropboxusercontent.com/u/3308162/Slonim%202013.pdf The most relevant of these papers (Lacetera 2013) cites the major motivations for volunteer labor are: "pure altruism, warm glow, self-image, and reputation". Upon reading this I realized TorCoin's technical interestingness had blinded me to much easier to leverage motivations of "warm glow" and "reputation". I propose the following system for harnessing "warm glow" and "reputation" for Tor relay operators. I am willing to fund this in its entirety. I propose establishing a subdomain on torproject.org giving each Tor relay operator (hereafter affectionately called "Torati") his/her own page using the information her machines provide to the Tor Directory Consensus. The fields to show on her "Torati profile page" would be things like: ContactInfo, PGP fingerprint, list of server nicknames, date the Directory Authorities first saw her contact info, etc. You can also imagine a receiving special "special recognition stars" for operating an exit or bridge node. Moreover, some bandwidth measurement like EigenSpeed or TorCoin gain traction, the Torati page could recognize contributors with by listing the sum total she has relayed to the Tor network. Naturally a node can opt-out of Torati recognition by setting a parameter in the torrc file. I argue this would be a cheap and easy way to motivate operators to volunteer more bandwidth for the Tor network. As mentioned before, I am willing to fund this in its entirety. -Virgil

8 10

[PATCH] remove test for remote SOCKS Port at config validation
by Martin Kepplinger 28 Jun '14

28 Jun '14

remove the check for a non-local SOCKS Port in the config by removing the flag when calling parse_port_config() for it. this should only prevent parse_port_config() from writing a warning that is written when the listener is opened anyways, according to https://trac.torproject.org/projects/tor/ticket/4019 --- this is UNTESTED and more of a question if that may suffice for closing ticket 4019 for now. src/or/config.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/or/config.c b/src/or/config.c index 10df839..7c22171 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -6002,8 +6002,7 @@ parse_ports(or_options_t *options, int validate_only, options->SocksPort_lines, options->SocksListenAddress, "Socks", CONN_TYPE_AP_LISTENER, "127.0.0.1", 9050, - CL_PORT_WARN_NONLOCAL|CL_PORT_ALLOW_EXTRA_LISTENADDR| - CL_PORT_TAKES_HOSTNAMES) < 0) { + CL_PORT_ALLOW_EXTRA_LISTENADDR|CL_PORT_TAKES_HOSTNAMES) < 0) { *msg = tor_strdup("Invalid SocksPort/SocksListenAddress configuration"); goto err; } -- 1.7.10.4

1 0

PrivEX - Privacy
by Tariq Elahi 27 Jun '14

27 Jun '14

tl;dr: We propose collecting data from exit nodes to improve the Tor network, using differential privacy and secure multiparty computation to do it in a privacy-sensitive manner. Hi tor-dev, In the ongoing effort to make Tor faster, secure and more resilient, network data plays an important role. If we know how the network is being used, what its clients' needs are and the threats that it faces we can deal with these in an intelligent manner. While the Tor Project does collect some statistics from guards, it does not currently collect and share potentially sensitive exit statistics. This data includes destination statistics and client timing behaviour, among many other potentially interesting, but privacy sensitive, data points. This reticence to collect data is due to the (well-founded) risk to clients and OR operators that this data could pose, such as correlation and coercion attacks. This is unfortunate since, as we observe above, in order to make improvements to the Tor network and its feature set, it would be beneficial to know what is going on inside it and with its users. To that end, it would be great if we were able to learn about network and client trend data. Some concrete examples include circuit-level data volumes, guard traffic usage, lengths of internal buffers, and latencies at relays. Indeed, if it can be counted then we should be able to collect and report it in a privacy-preserving manner. Which brings me to the reason for this email; I have had the good fortune to work with George Danezis at UCL and my supervisor Ian Goldberg at the University of Waterloo on coming up with a solution to this private data collection problem. We have created a system, PrivEx, that uses modern privacy-preserving techniques such as differential privacy and secure multiparty computation to address this thorny set of challenges; we have written up the details in a tech report that can be found here:http://cacr.uwaterloo.ca/techreports/2014/cacr2014-08.pdf . We have also created implementations of the two variants of PrivEx as described in the tech report. We are currently putting in the finishing touches and will be releasing them soon as open source in a git repo. We would like to start by rolling out our own PrivEx-enabled exits in the Tor network and begin collecting destination visit statistics. We expect that PrivEx will be generally useful to all exit operators and the Tor network in general but there is no requirement to deploy it everywhere. We hope to deploy PrivEx on a handful of exits during the June-August timeframe. What we would really like in order of importance is 1) a design review of our proposal, 2) an implementation review would be nice (once we release it). We hope that these reviews will address the main concerns of the community at large as well as give it, and us, a measure of confidence that collecting data with PrivEx is inherently good and is being done in a responsible and intelligent manner. We anticipate that this would make PrivEx an attractive addition for the Tor Project and their data collection needs. Please don't hesitate to give us your feedback, either to the list or to me via email. Cheers, Tariq

3 4

Who shall we send to the GSoC reunion?
by Amogh Pradeep 25 Jun '14

25 Jun '14

Well, questions pretty straightforward, and I didn't get selected via the lucky draw, so I was thinking of alternative ways to get there!

1 0

Fwd: Run a Small Tor Network for a Research
by Soroosh Sardari 25 Jun '14

25 Jun '14

Hi I want to run a small tor network with only 3 node, entry, router, and exit node. Also I want to set this three nodes in the client node without using directory server. Any idea appreciated. Regards, Soroosh

3 2

Damian's Status Report - June 2014
by Damian Johnson 25 Jun '14

25 Jun '14

Oooh summer. Pollen scratching my eyes and a fiery ball of death hellbent on frying me. We oughta nuke the sun! Teach that lazy, freeloading ball of plasma who's boss. Just think of all the savings on air conditioning! In those rare moments not staving off heat stroke I spent much of this month visiting family, and just a tad on code... * Stem 1.2.0 introduced a couple version compatibility regressions, the first with python 3.x [1] and another with python 2.6 [2]. Both received hotfixes, bumping us now to version 1.2.2. * Karsten and I discussed a Python rewrite of ExoneraTor at length [3]. While this would be a pretty fun and interesting project, its relative impact compared to other things I have in the works right now isn't especially high. * Reviewed Stem usage in Kai's DetecTor project. [4] Recently I shifted my focus back to arm. I've been truly impressed at the level of interest in it on IRC! New arm users are appearing almost daily, and I owe them a shiny new release! [1] https://trac.torproject.org/projects/tor/ticket/12185 [2] https://trac.torproject.org/projects/tor/ticket/12223 [3] https://lists.torproject.org/pipermail/tor-dev/2014-June/006970.html [4] https://lists.torproject.org/pipermail/tor-dev/2014-June/006965.html

1 0

Python ExoneraTor
by Damian Johnson 25 Jun '14

25 Jun '14

Hi Karsten. This is diving into enough detail that we might as well move this over to tor-dev@. For the list's benefit, Karsten and I are discussing a Python rewrite of ExoneraTor... https://exonerator.torproject.org/ https://gitweb.torproject.org/exonerator.git First I think I need to take a step back to figure out exactly what we're after. From a quick peek at ExoneraTor it looks like it behaves as follows... a. User enters an address (IPv4 or IPv6) and a date (either for a day or an hour). b. ExoneraTor lists router status entries for all relays that match the criteria. These entries link to the consensus they came from and server descriptors they reference. c. The user can then enter a destination address and port to search exit policies in TorDNSEL entres. Step 'a' and 'b' make sense to me. Step 'c' however I'm having a little difficulty groking. Ignoring TorDNSEL entries for a moment, we already have all the ingredients to provide the user with three fields to start with... * Source Address (required) * Timestamp (required) * Destination Address and/or Port (optional) The source address and timestamp come from the consensus, and an optional 'can it exit to destination X' consults the server descriptor's exit policy. So what is TorDNSEL providing us and why is it a separate search on the page? As I understand it the value of TorDNSEL is that we can't trust the address in the router status entries. If that's the case then our present search fields don't make sense to me... * Our initial search consults consensus information for the address and timestamp but not the exit policy. This is weird both because the address this has is faulty, and we have the exit policy so we could trivially include that in our search criteria. * Our second search gives the impression that we're using the earlier consensus results to query exit criteria from TorDNSEL. As I understand it though that's not what it's doing. TorDNSEL is completely independent from the consensus information. I could understand a search that just consults consensus information (ignoring address accuracy, it has everything we need). I could also understand a search that just consults TorDNSEL information (ignoring its inconsistent poll rate, it has everything we need). However, this hybrid approach and how it's presented really confuses me. Unless I'm mistaken with something above what I'd expect from ExoneraTor is... * The three search fields mentioned above. * It shows results based on the consensus information like we presently do. * If we have TorDNSEL entries that either indicate that a relay we're presenting had a different external address or another relay had the address we're searching for then note that. That is to say, the base search is based on consensus information (using server descriptor exit policies if we want to filter by that), and the TorDNSEL results are just appended notes since we can't rely on its poll rate. Thoughts? Cheers! -Damian PS. Congratulations on getting me invested. I just spent the last three hours in front of a whiteboard trying to puzzle out why ExoneraTor works the way it presently does. ;) PPS. Stem's ExitPolicy class has a can_eixt_to() method that would be really handy for this... https://stem.torproject.org/api/exit_policy.html#stem.exit_policy.ExitPolic… PPPS. I'm still hesitant about actually tackling this project. Arm is midway through being rewritten, and considering its sudden uptick in usage probably the most important project on my plate right now. That said, I'm happy to discuss this. Even if we don't implement it right now this thread will be useful so we know where we're going with ticket #8260. Concerning the earlier discussion of 'work with Karsten on a python project' I have a personal bias toward collaborating when the project has few unknowns for me, but working alone when *I'm* learning something. That is to say, I'd love to work with you on a straightforward Stem project and I'd also like to discuss ExoneraTor's design. But when it comes to coding, this has enough unknowns that if I take it on I'd prefer to experiment alone for a while - at least until I know enough about the APIs involved that I can avoid embarrassing myself. :) On Sun, Jun 8, 2014 at 2:56 AM, Karsten Loesing <karsten(a)torproject.org> wrote: > On 08/06/14 06:27, Damian Johnson wrote: >>>> Here's a quick overview of the codebase to facilitate reading through it: >>> >>> Ahhh, very useful - thanks. >> >> Hmmm. Just took a quick peek at the ExoneraTor codebase and, unless >> I'm mistaken, it doesn't actually use metrics-lib, does it? > > You're right, looks like it doesn't. > >> Honestly >> looking over the code is making me a little hesitant to take this on >> after all. I was anticipating a small, quick project of DocTor's scope >> but I've never touched SQLAlchemy or Posgress before. > > I don't think we'll even have to touch the Postgres for moving from Java > to Python. The Python code would simply do SQL calls via its SQL > library just like Java does. > > I just copied all SQL statements that the Python part would have to > prepare and execute: > > CALL insert_descriptor(?, ?); > CALL insert_statusentry(?, ?, ?, ?, ?, ?, ?); > CALL insert_consensus(?, ?); > CALL insert_exitlistentry(?, ?, ?, ?, ?); > SELECT MIN(validafter) AS first, MAX(validafter) AS last FROM consensus; > SELECT validafter FROM consensus WHERE validafter >= ? AND validafter <= ?; > CALL search_statusentries_by_address_date(?, ?); > CALL search_addresses_in_same_24 (?, ?); > CALL search_addresses_in_same_48 (?, ?); > SELECT rawdescriptor FROM descriptor WHERE descriptor = ?; > SELECT descriptor, rawdescriptor FROM descriptor WHERE descriptor LIKE ?; > SELECT rawconsensus FROM consensus WHERE validafter = ?; > > That's it. No further knowledge about Postgres required. > >> Once I wrote this I realized I'm being a damn hypocrite. Here I was >> saying "Karsten, learn Python so we can leverage each other's >> codebases!" but then I hightail it once the project delves into areas >> new to me. New arm users are showing up almost daily on irc and I'm >> anxious to give them a new release... but then this is exactly the >> issue, isn't it? Deliverables you'd like to focus on crowding out time >> to learn new things. >> >> So TL;DR I'm gonna eat my own words and suggest we focus on our >> separate domains for now. I really would like to work on some small >> metrics projects with you. Each month I eyeball your status reports >> asking myself "Is there anything here I can work with Karsten on to >> draw our spaces closer together?" so please let me know if you run >> across anything in Metrics we can collaborate on. > > (Replying below, first replying to the DynamoDB part.) > >> Your hypocritical friend, ~Damian >> >> PS. When we next meet I'd like to discuss ExoneraTor's design a bit. >> First thought I had when looking at the code was 'huh... I wonder if >> this would be a good use case for DynamoDB'. > > I'm wary about moving to another database, especially NoSQL ones and/or > cloud-based ones. They don't magically make things faster, and Postgres > is something I understand quite well by now. And again, I think that we > keep the Postgres part entirely unchanged when moving to Python. Not > saying that DymanoDB can't be the better choice, but switching the > database is not a priority for me. > > > So, regarding the rewrite: rather than canceling the project before it > starts, how about we find a role for you that you're more comfortable with? > > For example, I'd want to try rewriting it step by step based on your > suggestion of frameworks/libraries and with some code review of yours. > > If you're interested, which framework would I use for the new Python > ExoneraTor? It's supposed to do the following tasks: > > - Provide a simple web site with a web form, backed by the PostgreSQL > database. > - Maybe offer a simple RESTful API for lookups that the web form could > use to compose responses, but that could also be used by other > applications directly. > - Return documents from the database by identifier, so without > providing a search functionality. > - Run a scheduled task once per hour that fetches data from CollecTor > and puts it in a database. > > Bonus points if the result is as easy to deploy on Debian Wheezy as > possible. Like, install these few Debian packages, run the setup > script, done. > > Of course, if you'd prefer to focus on other things and not discuss > ExoneraTor stuff, that's perfectly fine, too. :) > > All the best, > Karsten >

4 13

Reminder: tor dev IRC meeting each Wednesday (but not next week)
by Nick Mathewson 25 Jun '14

25 Jun '14

Hi, all! The next tor dev meeting is at the usual time: 19:00 UTC this Wednesday (June 25). This is the weekly IRC meeting for people working on the program "tor". (It won't cover all the other programs developed under the Tor umbrella.) (That's 3pm EDT and 12:00 noon PDT.) As usual, we'll do it on the #tor-dev IRC channel. (Also: we're not going to have one on Wednesday July 3: too many tor developers will be in France.) cheers, -- Nick

1 0