- tor-dev - lists.torproject.org

Status of Open Hidden Service Proposals (October 2015)
by George Kadianakis 22 Oct '15

22 Oct '15

Greetings, it's well known that hidden services need some love: https://blog.torproject.org/blog/hidden-services-need-some-love For the past 2 years we've been busy designing the upcoming hidden service protocol with improved cryptography, security, and performance. During this time we've written a good amount of improvement proposals and specifications, that have now been floating around our git repositories. In this mail I aim to collect and briefly explain all these proposals in one place so that researchers and developers have easier access to them. Ideally we would also make a wiki page tracking them. Similar efforts have been done for the set of all Tor proposals by Nick: https://blog.torproject.org/blog/tor-design-proposals-how-we-make-changes-o… https://gitweb.torproject.org/torspec.git/tree/proposals/proposal-status.txt This might also make for an informative blog post if I clean it up a bit. Please let me know if I should try to get it posted on the blog so that it reaches a greater audience. Let's start walking over each proposal in a hopefully reasonable order: ======================================================================== == Proposal 250: Random Number Generation During Tor Voting == [Prerequisite proposal] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/250-commit-reveal-… Status: [Under development - https://trac.torproject.org/projects/tor/ticket/16943] This is a prerequisite for the proposals that follow. It specifies how the Tor directory authorities can produce a fresh and unpredictable random value every day. We plan to use this value to randomize the responsible HSDirs of hidden services and make them unpredictable. This will help defend against attacks that require the attacker to become the HSDir of a hidden service. == Proposal 224: Next-Generation Hidden Services in Tor == [Main proposal!] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t… Status: [Under development - https://trac.torproject.org/projects/tor/ticket/12424] This is the master proposal of the "Next Generation Hidden Services" project. It outlines a more or less completely revised version of the Tor hidden services protocol, improved to accomodate better cryptography and defenses for several attacks we'd never considered when we did the original design! The following proposals plug into the protocol specified by this proposal. == Proposal 246: Merging Hidden Service Directories and Introduction Points == [Performance improvement] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/246-merge-hsdir-an… Status: [Research Phase - https://lists.torproject.org/pipermail/tor-dev/2015-July/009079.html] This document describes a modification to proposal 224, which simplifies and improves the architecture by combining hidden service directories and introduction points at the same relays. It will speed up the initial connection to hidden services considerably since only two circuit establishments will be needed instead of three. == Proposal 247: Defending Against Guard Discovery Attacks using Vanguards == [Security improvement] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/247-hs-guard-disco… Status: [Research Phase - https://lists.torproject.org/pipermail/tor-dev/2015-July/009066.html] This document describes a modification to the path selection for hidden service circuits. It aims to defend against attacks where clients try to discover the hidden service's guard relay(s). This proposal also depends on having better and more robust algorithms for guard node selection. This requires another mini-proposal: https://lists.torproject.org/pipermail/tor-dev/2015-August/009297.html == Proposal 255: Controller features to allow for load-balancing hidden services == [Scalability improvement] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/255-hs-load-balanc… Discussion thread: https://lists.torproject.org/pipermail/tor-dev/2015-September/009597.html Status: [Under Development - https://trac.torproject.org/projects/tor/ticket/17254] We have plans for bringing hidden services to the next level. We are talking hidden services with 100x the clients they can currently handle, and with mechanisms that allow operators to load balance and achieve high availability. This proposal defines a way for hidden services to _load balance_ their clients by allowing *multiple hosts* to do the actual rendezvous with the clients. This is something that busy hidden service operators need currently. On the scaling front, we also worked on onionbalance which allows operators to have _high availability_ by allowing multiple hosts that handle introductions. Onionbalance is already usable by operators, and we have various improvements that we want to do in the future: https://github.com/DonnchaC/onionbalance == Proposal 252: Single Onion Services == [Optional Performance improvement] URL: https://gitweb.torproject.org/torspec.git/tree/proposals/252-single-onion.t… Status: [Research Phase - https://lists.torproject.org/pipermail/tor-dev/2015-September/009408.html] Websites like blockchain.info and Facebook are starting to offer hidden services to their clients. They do so to protect their clients from the fundamental exit-node attacks and also to provide them with Tor-specific features. Using hidden services in this context is also good news for the whole Tor, since hidden service circuits don't require exit relays who are the current bottleneck of the network. However, services like blockchain.info don't care about their anonymity; they only care about the anonymity of their clients. For services with this threat model, there are protocol modification that we can do to provide greater performance and load balancing options, since they don't need the 3-hop anonymizing circuits of Tor. Proposal 252 specifies how we can modify the Tor protocol to better accomodate services with this use case. Of course this would be an *opt-in setting* only for the services that want it. == Proposal: Direct Onion Services == [Optional Performance Improvement] URL: https://lists.torproject.org/pipermail/tor-dev/2015-April/008625.html Status: [Under Development - https://trac.torproject.org/projects/tor/ticket/17178] Proposal 252 "Single Onion Services" requires some protocol modifications that render it backwards _incompatible_. This means that Tor clients need to be updated to use these "single onion services". In the meanwhile services with the blockchain.info threat model that want to enjoy greater performance even with the current protocol can simply use 1-hop circuits for their server-side circuits. This should grant better performance with no cost to client anonymity while remaining backwards compatible. The "Direct Onion Services" proposal specifies how this should be done. I hear a newer version of the proposal will soon come out! ========================================================================

1 0

Load Balancing in 2.7 series - incompatible with OnionBalance ?
by Alec Muffett 22 Oct '15

22 Oct '15

So I’ve just had a conversation with dgoulet on IRC, which I will reformat and subedit here as a conversation regarding OnionBalance and issues in 2.6 and 2.7 when a recently rebooted HS publishes a fresh descriptor: […] alecm: consider OnionBalance which - being a bunch of daemons on a bunch of servers - will be a lot more prone to intermittent failures of 1+ daemons yielding a lot of republishing alecm: we tend to move services around, and daemons will be killed in one place and resurrected elsewhere, and then we'll have to bundle up a new descriptor and ship it out dgoulet: hrm so with that new 027 cache behavior, as long as the IP are usable, the descriptor will be kept, if they all become unusable, a new descriptor fetch is triggered and then those IPs will be tried alecm: There's a mandatory refresh [of the descriptor] after N minutes? dgoulet: we'll retry 3 times and after that all HSDir are in timeout for 15 minutes (I think, I'll have to validate) before retrying any HSDirs alecm: I wonder if descriptors should publish a recommended TTL - [number of seconds to live before refresh] dgoulet: yeah we have an idea for a "revision-counter" in the descriptor being incremented at each new version for the 24 hours period dgoulet: a TTL could be useful for load balancing though! alecm: so, here's a scenario: imagine that we run 10 daemons, alecm: call these daemons: A B C D E F G H I J - they all have random onion addresses alecm: we steal one IP from each daemon, and bundle the 10 stolen IPs together to make an onionbalance site descriptor and publish it alecm: people pull that descriptor, it's quite popular alecm: we then lose power in a datacentre, which takes out half of our onions - say, A through E alecm: we reboot the datacentre and restart A-E merely 10 minutes later alecm: everyone who has already loaded our onionbalance site descriptor tests A B C D E and finds them all dead, because the old IPs for A-E are invalid alecm: so they all move to F G H I J - which get overloaded even though (new) A B C D E are back up alecm: and this persists for up to 244, even though the outage was only 10 minutes alecm: net result: large chunks of the world (anyone with an old descriptor + anyone randomly choosing F-J) have a shitty experience, which is not what high-availability is all about :-) dgoulet: that will be what's going to happen - having a TTL in the desc. would help here indeed, I see the issue dgoulet: TTL would be one thing to add, here we could also add a mechanism for a client retrying IPs that failed in the situation where some of the IPs are still working, or making client balance themself randomly could be also an idea dgoulet: definitely there is some content here for tor-dev - I don't have a good answer but it should definitely be addressed alecm: proper random selection of IP would be beneficial for load-balancing; not perfect, but in the long run, helpful — Alec Muffett Security Infrastructure Facebook Engineering London

4 4

First Tor-wide 1-1-1 task exchange meeting happens on Thursday, Oct 22, 15:00 UTC in #tor-dev
by Karsten Loesing 21 Oct '15

21 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello devs, The first Tor-wide 1-1-1 task exchange meeting happens on Thursday, October 22, 2015, 15:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20151022T15 The idea of the 1-1-1 task exchange meeting is that participants get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do within 1 week. It could be to review a document, write some analysis code, fix a small bug, and so on. It's best to come prepared with a task or two to get the most out of this. In a perfect world people would give one task for each task they take, but that is not required. The Measurement Team has been playing the 1-1-1 task exchange game for a few weeks as part of team meetings. But we realized at the Berlin dev meeting that this doesn't have to be team specific. That's why we're opening this up to anyone in Tor who wants to come. For reference, here are our past tasks: https://pad.riseup.net/p/1-1-1-task-exchange We're also planning to put tasks that were not taken on Tor Weekly News to invite new volunteers who wouldn't attend our IRC meetings. Though the general approach will still be to contact the task owner before working on the task to avoid duplicating efforts. Talk to you tomorrow. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWJ6eEAAoJEJD5dJfVqbCrGoMH/2X0RBY/r/+Ndx+gHTyJtWjM lnjkC1KMckJeUsFI5jCHfgvm8BJJphiV9EHkvR8rIHi3PNlVufNOZywKAKM9j1Kh Gm3sQ2reBJ3BcNhEdi4f+q3/iaANCQG0ePOWLFPg3Vwsfra3jqyurz8C6WBsKyaN sJxPi5SzkHZFA2FkBE88GbJySIR8WKNAzoDpJEVqlxas+VfNuic499J1c/ufTI4R IG2UmLQjeWJOwKBlaSxY7RivTyGEWpJNOAs3xKIho0bZbDqa986ZLe+kHI34QHVJ cfS/wHFPnYIhiloUUuvwyRwrk4NfS3pUY3c5C/Xmzm+80mAHKT8bUFhxsMExqHE= =3O3D -----END PGP SIGNATURE-----

1 0

[Measurement Team] Next IRC meeting happens on Thursday, Oct 22, 14:00 UTC in #tor-dev
by Karsten Loesing 21 Oct '15

21 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Thursday, October 22, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20151022T14 As usual, we'll decide on the agenda at the beginning of the meeting. Here are some suggestions for topics: - team renaming - analytics server - team mailing list - parsing lib comparison - Roster update - CollecTor website redesign - metrics data visualization If you want to suggest something else, just add it to the pad: https://pad.riseup.net/p/zUNzEIFRq5S4 Talk to you in 24 hours. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWJ5p7AAoJEJD5dJfVqbCrD6cH/1jU21PDjoseeD1DSy2tHwFr gBtgNf4nNiqa6EFd8xuZPezChXZJeIPQ+COUUII0xERHTgwq3aBfcLq1vNixqyKl WMNbsGWBcjvS6G+kd1DritmnWWM3IiihdTvLCZjgLl4KDkT6qB7Do2ZNhZ7qhUUh NOyTY5Yd2ipjXO3PJJzEPoPjWdofcLlnX7rDerGxfewt+EnkEm1tvdVS4Efjo6pc 5udfuojhUQI7kIiAHyjUSyRSt+gWmLFEz01kj52gdG9Q+YFdZ/Pg0vH7+TCzQlLF 8r4Df594CWUAFUvY5X42ge+hfy7ebAkY7oqzFg0P4ukxqgE7f4sRgPTNlrwuxbk= =XYYl -----END PGP SIGNATURE-----

1 0

Request for comment: AEZ for relay cryptography
by Nick Mathewson 21 Oct '15

21 Oct '15

[This is a draft proposal; I'm not giving it a number yet. I'm hoping it'll receive some good advice.] Filename: xxx-aez-relay.txt Title: AEZ for relay cryptography Author: Nick Mathewson Created: 13 Oct 2015 Status: Draft 1. Summary and preliminaries This proposal describes an improved algorithm for circuit encryption, based on the wide-block SPRP AEZ. I also describe the attendant bookkeeping, including CREATE cells, and several variants of the proposal. For more information about AEZ, see http://web.cs.ucdavis.edu/~rogaway/aez/ For motivations, see proposal 202. 2. Specifications 2.1. New CREATE cell types. We add a new CREATE cell type that behaves as an ntor cell but which specifies that the circuit will be created to use this mode of encryption. [TODO: Can/should we make this unobservable?] The ntor handshake is performed as usual, but a different PROTOID is used: "ntor-curve25519-sha256-aez-1" To derive keys under this handshake, we still use HKDF_SHA256, but we produce 96 bytes of output: struct hkdf_output { u8 key_forward[48]; u8 key_backward[48]; }; These two fields are constant for the lifetime of the circuit. (But see section 4.3 below.) (Also see 4.1 for a variant that uses less storage key material.) 2.2. New relay cell payload We specify the following relay cell payload format, to be used when the exit node circuit hop was created with the CREATE format in 2.1 above: struct relay_cell_payload { u32 zero_1; u16 zero_2; u16 stream_id; u16 length IN [0..498]; u8 command; u8 data[498]; // payload_len - 11 }; Note that the payload length is unchanged. The fields are now rearranged to be aligned. The 'recognized' and 'length' fields are replaced with zero_1, zero_2, and the high 7 bits of length, for a minimum of 55 bits of unambigious verification. (Additional verification can be done by checking the other fields for correctness; AEZ users can exploit plaintext redundancy for additional cryptographic checking.) When encrypting a cell for a hop that was created using one of these circuits, clients and relays encrypt them using the AEZ algorithm with the following parameters: Key = Kf for forward cells, Kb for backward cells. # In theory, we are allowed to use a single key here, but I'm # tau = 0 # We want no per-hop ciphertext expansion. Instead we use # redundancy in the plaintext to authenticate the data. Nonce = struct { u64 cell_number; u8 is_forward; u8 is_early; } # The cell number is the number of relay cells that have # traveled in this direction on this circuit before this cell. # ie, it's zero for the first cell, two for the second, etc. # # is_forward is 1 for outbound cells, 0 for inbound cells. # is_early is 1 for cells packaged as RELAY_EARLY, 0 for # cells packaged as RELAY. # # Technically these two values would be more at home in AD # than in Nonce; but AEZ doesn't actually distinguish N and AD # internally. AD = [ The last 32 bytes of the previous cell's plaintext, if this is not the first cell sent in this direction on this circuit ] # Using this as additional data guarantees that any corrupt # ciphertext received will corrupt the plaintext, which will # corrupt all future plaintexts. Using the last 32 bytes of the # ciphertext would not have the same property. This instantiates a wide-block cipher, tweaked based on the cell index and direction. It authenticates part of the previous cell's plaintext, thereby ensuring that if the previous cell was corrupted, this cell will be unrecoverable. 3. Design considerations 3.1. Wide-block pros and cons? See proposal 202, section 4. 3.2. Given wide-block, why AEZ? It's a reasonably fast probably secure wide-block cipher. In particular, it's performance-competitive with AES_CTR. (How fast is it? To encrypt a 509-byte relay cell with a 16 byte nonce and 32 bytes of additional data, AEZ only uses 360 aes rounds. This is the same number of aes rounds as we'd need to CTR encrypt a 512-byte cell with 11.25 rounds per block. AES128 uses 10 rounds per block; AES256 uses 14 rounds per block. We could chop out 4 of the AES rounds by optimizing the code for the tau=0 case, or with AD shenenegans, but that's probably unwise. Additionally, we would no longer need to maintain a running SHA-1 of cells.) It seems secure-ish too. Several cryptographers I know seem to think it's likely secure enough, and almost surely at least as good as AES. [There are many other competing wide-block SPRP constructions if you like. Many require blocks be an integer number of blocks, or aren't tweakable. Some are slow. Do you know a good one?] 3.3. Why _not_ AEZ? There are also some reasons to consider avoiding AEZ, even if we do decide to use a wide-block cipher. FIRST it is complicated to implement. As the specification says, "The easiness claim for AEZ is with respect to ease and versatility of use, not implementation." SECOND, it's still more complicated to implement well (fast, side-channel-free) on systems without AES acceleration. We'll need to pull the round functions out of fast assembly AES, which is everybody's favorite hobby. THIRD, it's really horrible to try to do it in hardware. FOURTH, it is comparatively new. Although several cryptographers like it, and it is closely related to a system with a security proof, you never know. FIFTH, something better may come along. 4. Alternative designs 4.1. Only one key We already use different nonces for the forward and reverse direction; according to the AEZ design, this is sufficient to give security, even if K_b and K_f are the same. We could generate and store only half as much key material by using only a single key per circuit. 4.2. Authenticating things differently Adding only _a part of the plaintext_ of the previous cell seems a little screwy: that's usually easy information to predict. I believe this is secure, however, since the only purpose here is to ensure that _if_ the previous cell was corrupted, subsequent cells will be corrupted too. We could authenticate more stuff, however. We could, for example, authenticate the _entire_ previous ciphertext cell. Or we could authenticate the last 8 bytes of ciphertext and the last 24 bytes of plaintext. (Another thing we might dislike about the current proposal is that it appears to requires us to remember 32 bytes of plaintext until we get another cell. But that part is fixable: note that in the structure of AEZ, the AD is processed in the AEZ-hash() function, and then no longer used. We can compute the AEZ-hash() to be used for the next cell after each cell is en/de crypted.) 4.3. A forward-secure variant. We might want the property that after every cell, we can forget some secret that would enable us to decrypt that cell if we saw it again. One way to do this, at a little extra expense, is to keep a 16 or 32 byte 'chaining' value that changes after each cell. The initial chaining value in each direction would be another output of the HKDF. We could use it as an extra AD for the AEZ encryption. To update the chaining value, we need a one-way function. One option would be your-favorite-hash-function; blake2b isn't _that_ bad, right? We could also try to XOR it with a function of some hidden value from AEZ: E(S,-1,?) is promising, but it would require that we get our hands inside of our AEZ implementation. Also it would require a real cryptographer to come up with it. :) A more severe option is to update the entire key after each cell. This would conflict with 4.1 above, and cost us a bit more. A positively silly option would be to reserve the last X bytes of each relay cell's plaintext for random bytes, if they are not used for payload. This would help forward secrecy a little, in a really doofy way. Any other ideas? 4.4. SHA256 is stupid We could update the ntor definition used in this to use blake2b as its tweakable hash and for its KDF as well. (This would be faster _and_ more secure, not only because blake2b is lots faster than SHA256, but also because we could use the personalization and salt and key features of blake2b to avoid HMAC.) Or there's sha3 I guess if you want to do that.

5 4

Effect of padding on end to end correlation false positive rate
by s7r 20 Oct '15

20 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello, I have an idea which I want to put into a proposal, but need a clarification first so I won't be working on something which doesn't make a big difference. I am describing something like a Sybil attack where the adversary runs relays, gets lucky and is selected in a certain position of a certain rendezvous circuit and can do an end to end correlation of the traffic (both ends). This is different from one end correlation or website fingerprinting attack. For this scenario: HS -> Guard -> Middle1 -> Middle2 -> RDV -> MiddleC -> GuardC -> Client If the client is the attacker, and Middle1 is an evil relay colluding with him, and it was selected 2nd hop in a rendezvous circuit that connects the HS with the attacker (rendezvous is not colluding), he can learn the guard of the hidden service. The client (attacker) can send traffic over that circuit until he gathers enough data to make a fair assumption about where the traffic is coming from (identify the guard of that HS). I am not sure if there will even be a nonzero false positive rate in this context. Confirmation needed for these assumptions, this is what I think after reading: [0],[1]. Does this change with padding? If yes, how? For the same scenario described above, if padding is used, will the attacker have to own the entire path to the guard (Middle1 and Middle2 in the same circuit, since the rest of the path is under his control to select anyway), in order to make a reasonable guess on which might be the guard he is interested in? Or can he make the difference between the real traffic and the padding / dummy traffic and have the same results as with no padding? [0]: https://blog.torproject.org/blog/one-cell-enough [1]: https://blog.torproject.org/blog/traffic-correlation-using-netflows -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBCAAGBQJWIU5yAAoJEIN/pSyBJlsRlj4H+wcAc958SazmoXgYTtJisK5R dgcWZILgnYTYg17ub8Y3Hsgh0YjJyEgwgXV/r6ftNNir8kMSn3t+KMFAt9OpVhOJ 4W0f3E/X18viJNn3Pcb3FDc41sXxpwMVdUd337wGzqaNb8wsCARspmkDv9gdsLje GTTrz2MUa6lWyZHEuJMp02W2pkPVr48UnOsd0S0lyiDM3uxAAQyEtS9XwQyd+hBE KgLG7DHgwOnNzpBeQYJ5KlTzWi6j/NGxXcnHWcuWkq32ZG/gQsw6I43BNV9DMs6r 3UJbrdHsYJKDq8I16Ir0fblaOjsxmU7cEZZioOVngmw8wG6KevGI7fYKsEcqAhk= =CJly -----END PGP SIGNATURE-----

2 1

ResearchEthics
by grarpamp 20 Oct '15

20 Oct '15

https://trac.torproject.org/projects/tor/wiki/doc/ResearchEthics Any number of problems and obstacles to legitimate research areas exist with this... " Examples of unacceptable research activity It is not acceptable to run an HSDir, harvest onion addresses, and do a Web crawl of those onion services. Don't set up exit relays to sniff, or tamper with exit traffic. " Assuming such bans, how is one supposed to legitimately research and report on the real makeup of onion or exit space? These are significant unanswered questions regarding tor use. Concentrate not on bans, which will be ignored by both legit and illegit researchers anyways, but on proper design for data handling and minimization, particularly being sensitive to the tor users and operators involved lest that data become compromised at any stage before final anonymization and wiping.

5 7

Time-to-first-byte on trac.torproject.org
by Virgil Griffith 19 Oct '15

19 Oct '15

I started using Trac a bit more and the slowness is a little unpleasant. Here are some stats: http://www.webpagetest.org/result/151019_RW_387/ The time-to-first-byte is *painful* on both the first load as well as reload. Are there some ways we could improve this? If it's sever-power I'm willing to contribute to a fund for it, but my first thought is that more internal caching needs to be turned on within Trac. Common queries like the frontpage shouldn't be this difficult. But they are inherently this difficult, then we should at least spread it out among multiple machines. -V

1 0

Another Virtual Network Environment needs your I/O
by Rob van der Hoeven 18 Oct '15

18 Oct '15

Hi folks, During this hot summer I did some cool programming which resulted in... Another Virtual Network Environment (avne) AVNE is a small Linux program that runs other programs inside a virtual network environment. All network traffic from this environment is intercepted and forwarded to the Tor network. Interception is transparent and is guaranteed to be 100% effective. To forward network traffic you can simply type avne followed by a program name. Examples: avne bash avne ip link avne iceweasel -P avne -no-remote avne chromium --disable-cache avne wget https://blog.torproject.org/ Because the network interception takes place at the IP-level, no special Tor configuration is needed. In theory all TCP-clients should now be able to use the Tor network. AVNE is currently in the alpha phase of development. It works quite well, but for the for the first beta version I need input from this community. First of all I want to know if there are programs that do not work with avne. I tested with firefox, chromium, wget etc. and encountered no problems. The second question I have is: are there programs / protocols which should be blocked by avne? For example: I think its a bad idea to support unencrypted POP3. Of course I am also interested in bug-reports and your ideas for improvement. On my blog I wrote an article about avne that explains the technologies it uses (User Space Networking and Lightweight virtualization), and also gives an overview of its implementation. You can find the article here: https://hoevenstein.nl/another-virtual-network-environment The source code can be downloaded at: https://hoevenstein.nl/downloads An on-line viewable version of the code is at: https://hoevenstein.nl/source-code-avne-c Hope you like the ideas and code. And of course, your feedback is much appreciated! Rob. https://hoevenstein.nl

2 5

Re: [tor-dev] Crux: Privacy-preserving statistics for Tor
by Vasilios Mavroudis 18 Oct '15

18 Oct '15

Hi, so you were right the databases were corrupt, but they shouldn't have been there in the first place. :-) I didn't want to include large files in the git repo (~120mb in total), so there is a generation script in the tools directory (now added). I added some instructions on the readme file too. Of course, this procedure is done only once. I tried it and it should work now, but let me know if there is still a problem. There is a plan to optimize the way the pre-computed values are used, and this could potentially shrink the size of the databases by a lot. Vasilis On 17 October 2015 at 15:56, str4d <str4d(a)i2pmail.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > str4d wrote: > > Vasilios Mavroudis wrote: > >> Hello, > > > >> I would like to introduce our project "Crux", which enables the > >> computation of privacy preserving statistics on sensitive data. > >> The project was developed at University College London (UCL) by > >> me, in close collaboration with George Danezis. > > > > This is fantastic work. Haven't read all of your thesis yet, but > > it looks similar to the stats collection system that we ran for a > > while inside I2P, but with the privacy protections that I wanted it > > to have. Well done! > > > > > >> "Crux" was designed with Tor hidden services in mind (in > >> accordance with this > >> <https://research.torproject.org/techreports/hidden-service-stats-201 > 5 > > > >> > - -04-28.pdf> > > > > > > Tor > >> Tech Report), but it can be applied on various kinds of data > >> (given an appropriate parser). > > > > Excellent. I want to use this in I2P :) > > > > > >> It supports three statistics (i.e. mean, median, variance) and > >> its threat model is compatible Tor's (see technical document > >> below). > > > > I will look into this, but I doubt there will be any > > insurmountable incompatibilities with I2P's threat model. > > > >> Currently, we don't have a parser specific for the data > >> collected by the Tor relays, but we plan to keep adding > >> functionality. > > > >> The source code can be found here: > >> https://github.com/mavroudisv/Crux The theoretical background, > >> threat model, security argument and technical details can be > >> found here: link <http://mavroudisv.eu/files/thesis.pdf> > > > > > >> Ideas, comments, feedback much appreciated! > > > > Firstly: needs more docs. I shouldn't have to read your entire > > thesis in order to figure out how to run it :P The README is a good > > start, but the command parameters need clarification/definitions, > > and it is not at all obvious how the relays work. An XLS file is > > also mentioned without explanation. > > I have now tried running the code, but the "unit test" run by > authority.py fails because bsbdb.btopen() returns an empty dict. I > don't know if this is because the table files are broken or because of > a local issue with my setup (I am using a virtualenv, but I get the > same empty dict when I try to open the tables manually outside the > virtualenv). I am not going to try and fix the latter on my own, > because bsddb is deprecated since Python 2.6, so you shouldn't really > be using it :) (But it must be something else, because I tried using > bsddb3 instead and that also returns an empty dict). > > str4d > > > > > Secondly, you could probably simplify configuration by managing > > hidden service connections yourself. I believe the Stem library can > > do this? I2P has a python library that can be used to listen on and > > connect to I2P Destinations, with a socket-like interface. > > > > It's not clear to me from your thesis whether you plan to merge > > the relay component into the Tor relay code (and have the Tor > > relays talk directly to the query processors), or have your relays > > running separately and interfacing with the Tor relays to fetch > > stats. This will potentially alter how it would be used with other > > networks. In I2P's case I'd imagine it would be easiest (in terms > > of usability by those setting up relays) to write an I2P plugin in > > Java that performs the relay role, but I wouldn't want to be > > duplicating a lot of logic that subsequently needs to be kept > > in-sync with upstream :) > > > > str4d > > > > > >> Vasilis > > > > > > > >> _______________________________________________ tor-dev mailing > >> list tor-dev(a)lists.torproject.org > >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > > _______________________________________________ tor-dev mailing > > list tor-dev(a)lists.torproject.org > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > -----BEGIN PGP SIGNATURE----- > > iQIcBAEBCgAGBQJWItIgAAoJEBO17ljAn7PglsQP/3tOrL9EZfYMb5fxxVI6/wJm > qDvA8RpbEJDhoMjOc9sLNyMVgvGGPxEdma8I/R8H37yIPOp2GJg82MoXfUk+AGFe > 5ZIIXwf2EzHH0RFLRI5xaQ4DW3Ru1yIzH+E3vhuGclL87YRKnAM0xkMQcueR5xo5 > /ETe9TI7oPCdETI4J6iFoM+Ob9qsRiEVeuT5HEFFKwQhBvl0NSpfMHV9ftuxQHuu > I2B9jomnBdzF9OXnSQrYXg3ho2rxX4hO1TZfQHg5DgHvkt8TQYQM3cwdGE7/rsqC > d4+kBW2RuAz68wq+fLPeeZ2U2ff2QqmDLqVyUJvpP5RqeUcv66J/iYm7md886tTc > palwVGodKiTJ6DBM0z1K97x6Sl1jJXEAkZQAr2yQ8jrC/BJqQRPu2vHeZILqbMEQ > UU3JLjIk/Jc2RGK6zitvj4kTrLdfACEQMYy3brQUawZhaM42QNQgb3LQq1bJXzUo > vISZckX/CVfF+R63oO085LYOO7oTLyY3EhA1PCXiV09Usl/To66Oh4AVYtz/lPLH > 6uoWR/ic5X9gFIXt1t+617+i/L/ILMfqkqN7GzoSbQ8QfyBJrjIikuj575M7AH3o > RuhRYHUTmaCRL+9niDFG/mSVGBxBqiSfW1eS5pGE3hIAcLQd0eFT/38HRdonkfLP > Ug5ihMFSKfKmCIljQ6mm > =jDmS > -----END PGP SIGNATURE----- >

1 0