- tor-dev - lists.torproject.org

Re: [tor-dev] #15060: Decide the fate of MyFamily / prop242 better families
by teor 23 Mar '15

23 Mar '15

> Date: Mon, 23 Mar 2015 09:32:50 -0400 > From: Nick Mathewson <nickm(a)alum.mit.edu> > >> On Mon, Mar 23, 2015 at 8:56 AM, Nusenu <nusenu(a)openmailbox.org> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> Hi, >> >> I like prop242 [1] because it doesn't "Nuke MyFamily" [2] but replaces >> it with a better approach. "Better" in terms of scalability and >> maintainability. >> >> I don't know of any current 'tools' (except the tor client itself) >> that use MyFamily data but Virgil's [3] gamification website and >> compass group-by family feature [4] would depend on it. >> >> So before spending time on features that depends on an uncertain >> descriptor field, it is probably better to wait till the fate of that >> property (MyFamily) has been decided. >> >> The reason for this email is to find out when that fate will be >> decided upon? (expected answer: before tor 0.2.7.x-final is released) > > I don't know that we have a date here, or a consensus that it ought to > be done. Whether any changes happen in 0.2.7 here is going to depend > on whether we've got a solid answer one way or another on the question > of MyFamily. > > So, what do we think? I'd say that MyFamily is likely to continue to > serve a useful purpose, and that removing it entirely would be > premature given how often people come up with fun new routing and path > selection and guard selection ideas. > > On the other hand, prop242 is a fair bit of work to do, and I don't > know whether it will rise to a high enough priority over all the other > stuff we need to do. And who knows, maybe somebody will come up with > a solid argument in favor of removing MyFamily entirely? It's worth > thinking about. I can think of one good social/technical reason to keep MyFamily or similar feature(s): It provides a useful mechanism for the Tor relay community to distinguish between Sybils, technically capable enthusiastic new operators, and operators who can't/won't maintain their relays. I've seen this sort of communication a few times, but I'm not sure how frequently it happens: Someone says: "Thanks for signing up N Tor relays, please add MyFamily to each of them so we don't choose any of them in the same path." If the operator complies, they become part of the Tor network. If they decline or ignore, they are excluded from the Tor network. It would be a shame to lose such a useful technique for weeding out the bad onions, even if it relies on us finding them in the first place. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED

1 0

onionoo: grouping families in the backend?
by Nusenu 23 Mar '15

23 Mar '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I figured it might make sense to discuss this in a separate thread. > And just in case you were wondering, if you ever have a feature > request for Onionoo, I'd be happy to discuss that. In contrast to > some of its clients, Onionoo itself is actively maintained. Actually I came to the conclusion that it would make sense to move the myfamily grouping to the backend since any other compass grouping is based on onionoo data and not on data "generated" by compass itself (grouping key). The MyFamily grouping takes also a bit CPU time. So "computing" families once per generated details document make a lot more sense than recalculating them in every compass query again. Since the MyFamily is subject to change [1] is also relevant here. [1] https://lists.torproject.org/pipermail/tor-dev/2015-March/008516.html -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVEBuLAAoJEFv7XvVCELh05MMP/0Gy6zYQLbZtavXL1wdISLMg X/zdJM9+ZysuFkDj2hl0fP1iXfrNCCCvjCOdCYQJXzD5axrvgzPK/Jql1b2J1tO5 m+fbkLla9/KRCukJKYOjmkeoKAuadRDiZvI9LvePuGPrP53ZmlJmh4PT/Iss8Qh+ ndTBIbz1Kl04cBVS8RW08veFIKyzblkd2bJVj6AIlsvRvNybLJdEatgNAJ0YX6Vk ERuWuI65qP40ATFMAmL+IMsrdJ2WVdhpbbFLNkrZfloXt/UlNvFddMKlJ0k3YUkb ZQsFmZUPPv9+gMGx7Rkca94XmleBxMidjAbDVT0Y+SVyC0HxEOa5UsZ099cVu52F Hf2ZHt2E6vSxICoO4lxFEx82BxuaJLvG/+pQcNmUQk+SeBlcYn/W6PIyYV1bs9sh /pLjikLnMEMpNCls5XNc/wtFJtbnfljmsDhYL+R5jCHfUk5s3XmyEuDitenwZ8Qb 3u4qtnusQqLzW0R2bcnA8OV5RU5N12kBmFnki2lZ+m7VKtHCQVgJP8Rpvfg7G0/6 Js36UnNL2ijMNuDJED70PZEPop8kiuSZq6ePI3lYsPaLLRGLmNyHCgFQFRBox4BK ntAiePf2HtQ1kSLM2j5GLrQK5p59ko9xXrQkRBy60M5oVJkl+Rtf/+79W0Nf9f8Y /rcCkyU4s7XqW0UoHMYQ =ly6d -----END PGP SIGNATURE-----

2 5

#15060: Decide the fate of MyFamily / prop242 better families
by Nusenu 23 Mar '15

23 Mar '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, I like prop242 [1] because it doesn't "Nuke MyFamily" [2] but replaces it with a better approach. "Better" in terms of scalability and maintainability. I don't know of any current 'tools' (except the tor client itself) that use MyFamily data but Virgil's [3] gamification website and compass group-by family feature [4] would depend on it. So before spending time on features that depends on an uncertain descriptor field, it is probably better to wait till the fate of that property (MyFamily) has been decided. The reason for this email is to find out when that fate will be decided upon? (expected answer: before tor 0.2.7.x-final is released) Nick's statement: > Having some paths with all three nodes in the same family scares > me gives me some hope that it won't go away completely and will allow me to group relays to an operator (as long as the operator decides to configure it). Even in the unlikely event of MyFamily being nuked completely (with no replacement) - I guess it will be some years(?) till you deprecate pre tor 0.2.7 clients? [1] https://trac.torproject.org/projects/tor/ticket/6676 [2] https://gitweb.torproject.org/torspec.git/tree/proposals/242-better-familie… [3] https://lists.torproject.org/pipermail/tor-dev/2014-June/006975.html [4] https://raw.githubusercontent.com/nusenu/misc-files/master/biggest_tor_rela… -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVEA1hAAoJEFv7XvVCELh0cUYP/jl5EknR8EC9A5KNS9rWR/4J PEOz9u/6ySu4Pn1oDaWw7zjZK3IAzaqTdDenHq4et4U4GmrLG7JaKTdhL3pAmc8W mBNzwgE7FvYDgxKAwbh7fdVS2mD13QyZ6e9SPqdYQiR7WkKF1Ep5omp1HGoyGe4M mPo413K21SvkXGBcIhf62+R20renwTTNQ6G2dWB2wwlcjKoUJ385p7TYeNBbA/Jd NjCSmfAFFX7OeC8XuP/EHdf7+5CZI/SVn5Dq3oxZIpABNl4UdDzHdjRUoXSCEvxh hxpFbp7nozV7++73QUD5ECAFjUe98siKYODSFGb/xBMA9rCm93JKLKPB8iJPy8RO 8+D+qKsK0PFQVpEbbu/Uix2gcg68lTmv/zhHt1vVtT0h13LLtHyTQGQQ1SihcNlS Jv0mSxL3wDCs/yVRVR8r76MOdMAh891rPUzrW0/TxCc6W/LWXsA2ST4xt7zE8RQf LDWqlmSO6813M5vuEXZYFBQ+GsOquTUssPRKc/l1Ik8V628LYDvyfkN8QAJ72MDR HPkc6RGmHUVejpTUa9pcSbol2F96pEZ8XVYCyX3pIRoPiqlq0phnOlctaQFeAyGN 75FfN/GhEW2JDpqmIsD0BQbgcYTHa+qMiRZ8mWE9NRkQKIcDXyOTY7nIt810wbp9 0idgIP01pqKYPzhhaO5h =81G7 -----END PGP SIGNATURE-----

4 3

RFC: Ephemeral Hidden Services via the Control Port
by Yawning Angel 23 Mar '15

23 Mar '15

Hi, The Warning: DO NOT USE MY BRANCH YET, IT HAS HAD MINIMAL TESTING AND REVIEW. IT IS NOT DONE. IT WILL BROADCAST YOUR SECRETS TO THE NSA'S ORBITAL SPACE STATION. Trac Ticket: https://trac.torproject.org/projects/tor/ticket/6411 Branch: https://github.com/Yawning/tor/compare/feature6411 With that out of the way, I've been poking at implementing a way to add hidden services at runtime via the control port. I'm calling this "ephemeral hidden services" because the HS state is not written out to disk, and they disappear when the tor damon is shutdown. Yes, this means that if you run "kittensomgmewmew.onion" and are scared of the NSA's persistent attempts to extract your hidden service key, via ultrasonic laser beamed from their satellites, you could run your tor instance entirely in a ram disk, and load the HS key manually each time from a USB token you wear around your neck. What I've come up with looks like this: ADD_EPH_HS - Adds a new ephemeral hidden service. The syntax is: "ADD_EPH_HS" keytype:keyblob SP 1*(port SP addrPort) CRLF keytype = "RSA1024", "NEW" keyblob = opaque ascii serialized key material, or an algorithm specifier, in the case of "keytype" == "NEW". port = the "VIRTPORT" addrPort = the "TARGET" A hidden service is created using the key and list of port/targets, that will persist till configuration reload or the termination of the tor process. The currently supported keytypes and keyblobs are: "RSA1024" - The keyblob contains the Base64 encoding of the PKCS#1 layout DER representation of a 1024 bit RSA private key. "NEW" - The tor instance should generate a key with the keytype specified in the keyblob parameter. A type of "BEST" will generate a key using the best supported algorithm. Returns: "250-ServiceID=serviceID" - The hidden service's onion address without the ".onion" suffix. "250-PrivateKey=keytype:keyblob" - The newly generated hidden service's private key, if a keytype of "NEW" was specified. "250 OK" - Everything went well. Proper key management, is left as an exercise for the application invoking the controller command. DEL_EPH_HS - Removes an existing ephemeral hidden service. The syntax is: "DEL_EPH_HS" serviceID CRLF serviceID = the hidden service identifier, without the trailing ".onion" suffix The hidden service specified by the given service ID is removed, and the intro points are closed. Existing connections are left as is, and it is up to the application to close them if it wishes to do so. Examples: * Adding a new ephemeral hidden service, with a pre-generated RSA key: > ADD_EPH_HS RSA1024:[Base64 blob] 80 127.0.0.1:8080 < 250-ServiceID=blahblahblahblah < 250 OK Hidden service is reachable at blahblahblahblah.onion:80 * Adding a new ephemeral hidden service, tor generates a new RSA key: > ADD_EPH_HS NEW:RSA1024 22 127.0.0.1:22 < 250-ServiceID=hogehogehogehoge < 250-PrivateKey=RSA1024:[Base64 blob] < 250 OK Hidden service is reachable at hogehogehogehoge.onion:22. At a later date, the value passed back as "PrivateKey" can be used to re-create the hidden service. * Add a new ephemeral hidden service, tor picks the algorithm: > ADD_EPH_HS NEW:BEST 1245 192.168.1.5:23 < 250-ServiceID=hemohemohemohemo < 250-PrivateKey=Ed25519:[Base64 blob] < 250 OK Hidden service is reachable at hemohemohemohemo:23. (nb: Ed25519 key support is not actually implemented yet, and is used as an example of a better algorithm.) * Remove the hidden service configured in the first example: > DEL_EPH_HS blahblahblahblah < 250 OK Note: ADD_EPH_HS can take more than one VIRTPORT/TARGET pair at once, the examples only show one pair for brevity. This does not support `HiddenServiceAuthorizeClient` yet in any way (neither basic nor stealth). It may in the future, but coming up with a sane interface for it is way more pain than I am willing to self-inflict at the moment. Design question that I want feedback on, primarily from control port library maintainers, and authors of applications that use the control port: * meejah suggested that ephemeral hidden services should have their lifetime tied to the originating control port connection. I think this is a good idea, but this would be the only control port command that does this sort of thing. My motivations for strongly considering implementing this is that, I don't think that adding a command to enumerate hidden services is a good idea (SketchORChat has no business in seeing what hidden services JankORFileSharing is using), and that this will automate cleanup in the event of an application crashing, or not removing it's hidden services before terminating. * Does this interface expose enough functionality, in a way that is intuitive and easy to use? (The initial version I was planning on proposing made key generation entirely the application's problem, but nickm convinced me otherwise). Questions, comments, feedback appreciated, -- Yawning Angel

11 28

compass: new group by options: by-contact, by-OS, by-version
by Nusenu 23 Mar '15

23 Mar '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Karsten, I added a few new grouping options to compass: - -T --by-contact (#6675) (Maybe we should truncate contacts and remove the undefined) - -O --by-os - -V --by-version (#6855) +expose -N options in html (was implemented but not exposed in gui) https://github.com/nusenu/tor-compass For all output I (mis)used the nick column as a quick solution since it is not used otherwise when grouping. Advertised bw is broken for some time. There is no 'advertised_bandwidth_fraction' (This field was removed from onionoo in November 2014.). Is it ok to change this to observed_bandwidth? (which would be a non fraction item)* ? (The MyFamily lookup is also broken) short commit log: - - add new group by ContactInfo option: by_contact (-T, --by-contact) (quick 'n dirty) - - added new grouping -O, --by-os (all BSDs are merged into one group) - - added new grouping by tor version (-V, --by-version) - - tell app.py about new options (by_version, by_os, by_contact) - - expose -N option in html and fix a missing label tag - - include network data in html output when using -N (uses nick column) - - expose by_version, by_os and by_contact options in HTML - - if it is just one item, display the actual value for AS and CC instead of saying (1) What do you think about it? thanks, Nusenu *) while playing with that I found 3 relays from JP that have a suspiciosly high adv bw: https://atlas.torproject.org/#details/F528DED21EACD2E4E9301EC0AABD370EDCAD2… https://atlas.torproject.org/#details/8901B1D2D4C0D3398C3F8363247B5AABF3136… https://atlas.torproject.org/#details/4EA0464A1B8D4231F176BA2FA1BCBF0A26F12… reminded me of SKKU43: https://lists.torproject.org/pipermail/tor-talk/2014-February/032094.html -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVCtlTAAoJEFv7XvVCELh0bY0P/1aJyZncqdy47WZM9I7Y1E4a Sg1luXTqi9VlFZhbUJeFrhidLVPSqQYbMw79sKufU2uw/Jcqr0Ro2O48q9Gj7sL8 lDcM4AgFzm1yTedoXqyNAA5+bBeQTAjEgQU1ADYUZvDx2ULmGG4aD6qCuaFoP+3/ kjJ6r92jLOSYpOVdt8FyNc8IZYpbEwQNMKc4A0uZhQV/WVv6vFiOaXHliPdR6m2+ SU+KTqldWppfeezWIalGl3t7HEnz87JOaWrdnQ8fjLKSsqkex2T5E1MClguM4u2o 5ABPxg6uF/izjPv2VEy/hl42RQbLM5SDazM4CLMFN3mQu6Xa4iZ5R8A6iLDgZBj6 zK2wExcxScc58r4Briv5EKUpC9LTgE52DkPkX+TM3i9ktniYQbb/k/cy85nn+ENa WrZ2WDmaMyQXy6kXDCS4U/3gbv1+Dwq82hg372N2fga6hOXsZPFrxV1TwWlMTzp9 MvpaF6/GZsivrjNklteHxvxgcxTyywaKW5QUdFJdeCZnOrDy29iGKIql10xQ7TXa gTC7OTsDCAPiJn8P5xKhEAlVeJmz0zZN7QhbY/ybIc5KWkOoKyYwkT9ysmpS8rOK mapKbuMXvuA3Sg1F1sK9PEUy+D8ZAe+sBjOEHzqILARA6VXsly0rSJkcnhNKNpmL yxSwWe4uWOVXDmVEl7hQ =jGQe -----END PGP SIGNATURE-----

3 7

#if 0 unused functions?
by Sebastian Hahn 23 Mar '15

23 Mar '15

Hi there, we have some functions which we never call anywhere. In many cases, it appears we shouldn't delete them from the source because they "belong" there - the thing I initially stumbled across was ed25519_seckey_write_to_file(), for example. But I also don't see why compiling it and potentially including it in our binary makes any sense. Thoughts? Cheers Sebastian

2 1

Re: [tor-dev] #if 0 unused functions?
by teor 22 Mar '15

22 Mar '15

> Date: Sun, 22 Mar 2015 05:12:03 +0100 > From: Sebastian Hahn <sebastian(a)torproject.org> > To: tor-dev(a)lists.torproject.org > > Hi there, > > we have some functions which we never call anywhere. In many cases, it > appears we shouldn't delete them from the source because they "belong" > there - the thing I initially stumbled across was > ed25519_seckey_write_to_file(), for example. But I also don't see why > compiling it and potentially including it in our binary makes any sense. > > Thoughts? > > Cheers > Sebastian I'm think that removing / disabling unused code is a great idea, but I wonder what happens over the long term: [How] do we prevent bit-rot on the disabled code? Does the disabled code have unit tests? Should we add unit tests or keep them active to prevent bit-rot? Or should any unit tests be disabled too? I wonder if we could unit test unused code, but not link it into the tor binary. This would make sure it works if we ever want to use it in future, but without the risks and performance impacts of including unused code in the binary. There could be issues with this strategy that I haven't thought of. Like the added maintenance cost involved in keeping unused code and unit tests up-to-date. Perhaps we could: * remove entirely if we're unlikely to ever use the code, * disable if we want to keep it around for future reference, and * unit test it, but not link it in to the tor binary, if we think we'll use it soon. Which category do most of the unused functions fall into? teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED

2 1

Directory Server Consensus Status Flag
by Matthew Finkel 21 Mar '15

21 Mar '15

Hi all, I'd like some opinions. Currently authorities give relays the V2Dir if they are useful directory servers. Clients don't actually care about this flag in any way, but it's a useful visual indicator. With proposal 237 (All relays are directory servers) and #12538 as its implementation, the current plan is to continue using the V2Dir flag, but now clients will actually care if a relay has the V2Dir flag and choose directory servers based on it. Tickets #11103 and #1338 actually wanted to get rid of the V2Dir flag, for good reasons. If we're going to rename the flag, we should do it before #12538 is merged, and probably make the change in the same branch. The V2Dir flag needlessly specifies an unuseful protocol version. #1338 suggested switching to a V3Dir flag, but it was decided this was not such a good idea. Proposal 185 chose a "DirCache" Consensus flag. I think either "DirServer" or "DirCache" are good choices for this, but DirServer is more in line with the wording used in Prop 237. Are there any objections to changing the name of the flag? (There will be another email in a few minutes with an updated version of prop 237.) Thanks! Matt

1 0

Call for a big fast bridge (to be the meek backend)
by David Fifield 21 Mar '15

21 Mar '15

The meek pluggable transport is currently running on the bridge I run, which also happens to be the backend bridge for flash proxy. I'd like to move it to a fast relay run by an experienced operator. I want to do this both to diffuse trust, so that I don't run all the infrastructure, and because my bridge is not especially fast and I'm not especially adept at performance tuning. All you will need to do is run the meek-server program, add some lines to your torrc, and update the software when I ask you to. The more CPU, memory, and bandwidth you have, the better, though at this point usage is low enough that you won't even notice it if you are already running a fast relay. I think it will help if your bridge is located in the U.S., because that reduces latency from Google App Engine. The meek-server plugin is basically just a little web server: https://gitweb.torproject.org/pluggable-transports/meek.git/tree/HEAD:/meek… Since meek works differently than obfs3, for example, it doesn't help us to have hundreds of medium-fast bridges. We need one (or maybe two or three) big fat fast relays, because all the traffic that is bounced through App Engine or Amazon will be pointed at it. My PGP key is at https://www.bamsoftware.com/david/david.asc if you want to talk about it. David Fifield

6 11

Re: [tor-dev] Fwd: [guardian-dev] Orbot v15 alpha 5 is out: MeekObfs4VPNQRCodez!
by Nathan Freitas 20 Mar '15

20 Mar '15

On Fri, Mar 20, 2015, at 04:05 AM, isis wrote: > Nathan Freitas transcribed 0.8K bytes: > > On Thu, Mar 19, 2015, at 06:58 PM, David Fifield wrote: > > > What would it take to get some screenshots that show how to turn on > > > pluggable transports? I would like to add a guide to > > > https://trac.torproject.org/projects/tor/wiki/doc/meek#Quickstart and > > > instructions to > > > https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports#Howto… > > > . > > > > > > Or do you have such a guide I can just link to? > > > > > > > We're still finalizing the UI for the new features (QR code scanning, > > meek bridge type selection, etc) so once that is completed, we'll do a > > round of screenshots, guide updates, etc. This will most likely happen > > in early April. > > Wow, this is awesome! Great work, to all of you! > > Let me know if there's anything more I can do to help. In thinking about how users on phones/tablets will actually get obfs3/4/scramblesuit bridges, I realized that going to https://bridges.torproject.org on their mobile device will not be a great experience. The QR code scanning if they or a friend also has a PC, but in a pure mobile environment, it doesn't help so much. I can create a ticket for this, but just to lay out the idea, I think we need to have the bridge:// (or the improved proposed obfs4:// obfs3:// type specific scheme URLs) as a clickable link in both the bridges.tp.o site and the gettor email response messages. +n

2 1