- tor-dev - lists.torproject.org

Using Let's Encrypt for meek bridges
by David Fifield 09 Apr '16

09 Apr '16

I'm looking for ideas of good ways to handle TLS certificates and their renewal for meek bridges. I want to use Let's Encrypt for this process, and I hope that someone who knows Let's Encrypt well can contribute some ideas. All three of the meek bridges use HTTPS to receive connections from the CDN, so they need TLS certificates. For example, when you use meek-azure, your traffic is forwarded to the bridge at https://meek.bamsoftware.com/. How it works now is I do the usual domain validation procedure with a CA, receive an email to show that I control the domain, install the cert and key, and then run the server like this: meek-server --cert /etc/meek/cert.pem --key /etc/meek/key.pem When I used Let's Encrypt in the past, using the --webroot option, it wanted to write a file to the URL path "/.well-known/acme-challenge/{token}". That won't work for meek-server as it exists today, because meek-server never serves files from the filesystem. But it could. Perhaps we could add an option like --acme-webroot that would allow serving files from a single whitelisted directory. I notice that there are other ways of proving domain ownership (HTTP, TLS SNI, DNS). Maybe we could use one of those? https://ietf-wg-acme.github.io/acme/#identifier-validation-challenges I also note that there are third-party plugins: https://github.com/letsencrypt/letsencrypt/wiki/Plugins Maybe there could be a plugin for meek-server (ideally not requiring new complicated code in meek-server itself). Currently you have to restart meek-server in order to make it notice a changed certificate and key file. It would be better if that were not necessary--maybe we could periodically stat the files, and re-load them if they have changed? This is going to be an issue for Snowflake as well, because we will want to use WebSocket over TLS for the server component.

2 2

Introduction: CollecTor Improvement Project
by iwakeh 08 Apr '16

08 Apr '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! With the first nice spring days in some parts of the Northern Hemisphere the CollecTor Improvement Project started. You probably know that CollecTor [0] is the friendly data-collecting service in the Tor network. It fetches data from various nodes and services in the public Tor network and makes it available to the world. The current project aims at making CollecTor more robust against data loss and improve its source base, development structure, as well as its installation procedure and configuration. More details can be found in the project's wiki page [1]. Further general discussion of the CollecTor Improvement Project will take place in the metrics-team mailing list [2]. So, if you're interested your can subscribe there, too. Cheers, iwakeh [0]: https://collector.torproject.org/ [1]: https://trac.torproject.org/projects/tor/wiki/doc/CollecTor/Improvements [2]: https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlcHjlgACgkQsANAxTOj/aYIVQD+KwlYx5FFXy+uiCtF8wCIAfeA SJQ3Fnk4YQC598/jo9QA/RUaZ/Z99Yjey8DsniBZTOJ/lTNlCOcKpXyXLIaOezOB =CSaG -----END PGP SIGNATURE-----

2 1

Research on Breaking reCAPTCHA
by bancfc＠openmailbox.org 08 Apr '16

08 Apr '16

While I don't understand why researchers are doing free labor for Google its still an interesting read on the innards of reCAPTCHAs and its weaknesses. Some tests were done with Tor: http://www.cs.columbia.edu/~polakis/papers/sivakorn_eurosp16.pdf

1 0

Re: [tor-dev] Update on 259
by Ola Bini 08 Apr '16

08 Apr '16

Hey, > > - OrPort vs DirPort > > ORPort is used for regular circuits, while DirPort is used when getting directory information. We need to interpret reachable stuff > > differently depending on the purpose. > > > > I'm not actually sure what the comment means here. This was more for our own benefit. The OrPort vs DirPort distinction has been a bit complicated so far. The comment basically means, when we are looking up directory information, we should use the DirPort to decide reachability and so on instead, correct? > Ensuring a min percentage of dirguards in our sampled set could work. Then, > when we need a directory guard, we could filter the sampled set and only > examine guards that can do directory requests. Yeah, we talked about this yesterday and our current thinking is to have a sampled set that contains every kind of thing, and then we dynamically filter it based on config and so on during START. > Hm, are you talking about the guardlists here? What's the question? > > BTW, if we have the ability to do "ensure a min percentage of X in our sampled > set", couldn't we just ensure a min percentage of dystopic guards in our > sampled set? And forget about the two separate guardlists? > > In this case we can have the percentage value be the actual portion of the > network that is dystopic guards. So if 20% of the total guard bandwidth is > dystopic, we could ensure that at least 20% of our sampled set is > dystopic". Well, the problem is really that the idea of dystopic doesn't necessarily make sense, since it's so dependent on the current network position of the client. Our current thinking is to do away with that concept as well. =) > > - DYSTOPIC - is there value in trying 80 and 443? > > Probably not. > > > > What does "trying" mean in this case? Falling back to guards with 80 and 443. > Restart pending guard selection algorithms on a SIGHUP? Plausible, but I don't > know how hard it would be to implement this. Well, the alternative is to just finish the running guard selections with the old settings, but use the new settings for new algorithm instances. > That's not very nice because the USED_GUARDS set that was created when > ClientsUseIPv6 or FascistFirewall were on will have reduced diversity. Then > even if we switch off those options, we are still stuck with reduced diversity. > > I'm not sure what's the right way to do this here! > > We could imagine having multiple USED_GUARDS sets, where we make a new set for > each possible filter. This might be worth considering, but I imagine there will > be technical difficulties. e.g. when a guard goes down, you need to update its > state in all the USED_GUARDS sets that it's in. Also, a person who toggles the > FascistFirewall option frequently, will end up using two different sets of > guards all the time which is suboptimal. Well, one thing you could do is hash the settings (and maybe also reachable ports) and use that as a key to differentiate the different USED_GUARDS. That would solve the problem, but might lead to a single client using lots of different guards in different locations. Might that be OK? > > - Can we make the lists smaller? > > Probably. Maybe a sampled set of 30 guards? Or 1.5%? > > > > Plausible. However, if we take the filtering approach but use a small sampled > guards list, it could happen that the list is not able to satisfy some of our > filtering restrictions. > > e.g. maybe in our 30 guards there are no IPv6 guards at all, and the user just > turned on ClientUseIPv6. What to do now? > > This is important to understand, because currently there is no mechanism to add > stuff to the sampled guards list if a restriction cannot be satisfied. So what > will Tor do, if a user enables ClientsUseIPv6 _and_ FascistFirewall but there > are no IPv6+80/443 guards in our sampled guards list? Yeah, we talked about that yesterday. Our suggestion is to do something like this: - if the filtered/reduced sample-set contains less than X (5?) guards, expand SAMPLED guards using the regular process. - If SAMPLE guards reach SAMPLED_MAX (50?) size, we fail closed with an error saying something like "your current network settings make it impossible for us to safely choose an entry guard. If you really need to connect under these circumstances, consider explicitly setting the EntryGuards configuration option" > I think it asks "What happens when guards in our sampled set drop out of the > consensus and get marked as bad?" (see bad_since in entrynodes.c) . > > This is also a great question. Especially when combined with the planned > "ensure a min percentage of X in our sampled set" logic. > > Like, what happens if suddenly most of our sampled IPv6 guards drop out of the > consensus when we have ClientsUseIPv6 on? Should we replace them? And if yes, > don't we need to replace them with other IPv6 guards to maintain the minimum > percentage? Well, I think if we replace, we should just replace randomly just like we always expand the sampled set. If most ipv6 guards drop out, we will have fewer ipv6 guards in our sampled set, but that also reflects the Tor network. I suspect a plausible thing to do is to wait a few consensus rounds with expanding the sampled set to replace "bad" guards - they might come back, and under most circumstances we shouldn't need to use the sampled set anyway. > The current proposal says the following about SAMPLED_UTOPIC_GUARDS: > > It will be filled in by the algorithm if it's empty, or if it contains > less than SAMPLE_SET_THRESHOLD guards after winnowing out older guards. > > which I think is a good suggestion. However, what should we do if we end up > going with the "ensure minimum percentage" logic? Yeah, my suggestion is not have minimum percentage of X type guards - instead just fill it randomly, and use the expanding process if we can't find enough guards for a specific purpose. > > - EntryNodes > > If this is set, never use the algorithm for regular circuits - we should still use it for directory server connections though. > > If this is set we should not use our algorithm, but we should instead pick one > of the guards in the EntryNodes list. This is for people who want to hardcode > their guard. It's used a lot by people currently. Yeah. Is the guard picked randomly from this list, or using something more complicated? > > - UseEntryGuardsAsDirGuards > > I don't understand exactly what this settings does. > > I'm not sure either. I'd just let it keep the exact same semantics it currently has. Yeah, except we don't exactly understand what does semantics are, and if we need to change something in our code to match it. =) Thanks for all the feedback! Hopefully we're getting close to the final iteration of this spec. =) Cheers -- Ola Bini (https://olabini.se) "Yields falsehood when quined" yields falsehood when quined.

5 10

Tor debugging
by SMTP Test 07 Apr '16

07 Apr '16

Hi all, I am working on a project that needs to do some modifications on the source code of Tor, to add functionalities to Tor relay. To debug I used chutney to setup Tor network locally, and checked the logs of the relay. But it is still painful and slow: I need to restart the script, and wait the network to be up every time I make changes. My question is: is there any fast way to test/debug the functionalities of Tor relay? Thanks!

2 1

extension for chrome and mozilla
by Mukul Gupta 07 Apr '16

07 Apr '16

hello tor-dev I want to make extension which can control network in my browser. so that i can connect to my some other computer which has VPN . And after that the data is exchanged between VPN server and the user. And the intermediate nodes won't be able to identify whats going on. This is my project proposal. Can you help me in this how to initiate this project.

1 0

Re: [tor-dev] Update on 259
by tdruiva＠riseup.net 06 Apr '16

06 Apr '16

Adding tor-dev. On 04/06/2016 08:53 AM, Tania Silva wrote: > Hey, > >>>>>> Yeah, we talked about that yesterday. Our suggestion is to do >>>>>> something like this: >>>>>> - if the filtered/reduced sample-set contains less than X (5?) guards, >>>>>> expand SAMPLED guards using the regular process. >>>>>> - If SAMPLE guards reach SAMPLED_MAX (50?) size, we fail closed with >>>>>> an error saying something like "your current network settings make it >>>>>> impossible for us to safely choose an entry guard. If you really need >>>>>> to connect under these circumstances, consider explicitly setting the >>>>>> EntryGuards configuration option" >>>> >>>> Oh, wow, I don't think failing closed is a good idea. >>>> It means users that move around a lot (and clients which have a longer state history) could fail at some arbitrary time. >>>> Why not simply continue to add guards that satisfy the restrictions? >> Well, users that move around a lot will only have an expanded sampled >> set if they move between several different networks that have severe >> restrictions - but mutually exclusive such restrictions. And we would >> only ever hit this fail closed if we can't find anything in the >> sampled set that matches the current needed restrictions. If we keep >> adding guards, the idea of the sampled set as a measure to minimize >> exposure to too many guards fly out the window. >> >> The problem really comes down to this - if you have a network that is >> actively firewalling every guard that is not under their control, if >> we keep expanding we will sooner or later be forced to use a guard >> under adversary control. By failing closed, we can avoid that >> eventuality. > > We thought in to have a minimum size for this set after been filtered [1]. > > ``` > MINIMUM_FILTERED_SAMPLE_SIZE > The minimum size of the sampled set after filtering out nodes based on > client configuration (FILTERED_SAMPLED). Proposed value is ???. > ``` > > In order to expose less guards when we are expanding sample set, we > thought in to have a maximum size before to fail. > What do you think if this is a configurable value by user? (we can have > as default if not provided) > > ``` > MAXIMUM_SAMPLE_SIZE_THRESHOLD > In order to guarantee a minimum size of guards after filtering, we > expand SAMPLED_GUARDS until a limit. This fraction of GUARDS will be > used as an upper bound when expanding SAMPLED_GUARDS. Proposed value is > 0.03. > ``` > > 1 - > https://github.com/twstrike/torspec/blob/review/proposals/259-guard-selecti… >

1 0

Adding one more (optional) step to Trac's ticket work-flow
by Isabela 06 Apr '16

06 Apr '16

Hello tor-dev! tl;dr; we want to add one more step to Trac ticket work-flow called 'merge_ready' - if you don't want to use it you don't need to During Tor's Dev Meeting in Valencia, the Network Team discussed a code review process[1] for the team and we concluded that for us a two stage review process would make things much better for the team. In order to organize that on our ticket work-flow on Trac, we decided to add the step 'merge_ready'[2] that would come after either 'needs_review' or 'needs_revision' steps. For our team, what that will mean is that we have reviewed the code and believe is ready for Nick to do a second review and merge. The first review will remove load from Nick and we want to eventually have more folks helping Nick on the second review as well. But this change alone will be already a big step. What does that means for someone working on a ticket that is not part of Core Tor development flow? Well, if you don't want to use it in your team or your project flow, you can just ignore it and keep using Trac as normal. We are sending this note to make sure folks knows about it before it happens and of course, to hear any concerns or feedback. Cheers, Isabela [1] https://trac.torproject.org/projects/tor/wiki/org/meetings/2016WinterDevMee… [2] https://storm.torproject.org/shared/pt9FtdJNG_gB_afqayg9dXTJwdDFncgU0HVXJEq… -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

3 2

Re: [tor-dev] [TOR+QUIC] Progress Update
by Xiaofan Li 05 Apr '16

05 Apr '16

Tim: Thank you for the detailed email. I've also included tor-dev on CC. Anyone is welcome to comment. To reply to your concerns: 1. All the documentation we have is basically in email correspondences as well as in the generic simple-quic codebase here: https://github.com/kku1993/simple-quic. The quicksock_t interface is in the above repo under /quicsock. There isn't much documentation because: a) our designs are still under test and large portions are not stable yet, b) we haven't gotten much time to properly document designs and changes to the Tor codebase. But the emails should be somewhat clear on some key design aspects. 2. About providing an alternative for QUIC connections: - Great idea about keeping both type of sockets in connection. I think this will give us a cleaner design. - Exit relay will speak normal TCP. This is part of our design from the start but we haven't gotten around to look into the exit connection code. - Connections to the directory authorities should be normal TCP. - While compatibility with the current system is very important, our main goal is to compare performances of the OR connections with TCP vs. QUIC. So we haven't spent much effort making things beautiful yet. After all, it is possible that QUIC is a bad idea for Tor.. 3. I was looking at chutney the other day but I couldn't find much documentation on how it works. I will play with it more and then I might ask for more help. If anyone else is expert on the SSL part of the codebase, please provide some insights about the security aspect of the design, specifically: "About SSL, we want to adopt the design where QUIC handles SSL by not using the ssl_buffer in TOR code. Any concerns at all?" (not using read_to_buf_tls() and other similar ssl functions) Another concern is the assumption on TOR_SOCKET_T in the Tor codebase. In particular, there are many call sites of fcntl, getsockname, setsockopt etc made directly on this object, assuming that it is a UNIX socket. For compatibility, after replacing UNIX socket with quicsock_t, we would use "qs_get_fd" for all those low-level calls. Note that quicsock_t holds a UDP socket internally. Without the testing system ready, we don't know the effect of these calls on a UDP socket instead of the normal TCP socket. Any other feedback also welcomed! Li. On Mon, Apr 4, 2016 at 10:43 PM, Tim Wilson-Brown - teor <teor2345(a)gmail.com > wrote: > Hi Li, > > If you're ok with emailing the tor-dev mailing list, you will likely get > better answers to some of these questions there. > All of these questions could benefit from the wider Tor developer > community. > When I know I can't fully answer a question, I'll say so below. > > Please feel free to forward my response to tor-dev. > > > On 5 Apr 2016, at 05:38, Xiaofan Li <xli2(a)andrew.cmu.edu> wrote: > > > > Hi Tim: > > I think maybe now is a good time to bother you again. In the following > email, we outlined some current progress on the TOR+QUIC project. This > weekend we booted up our first TOR node running QUIC (then of course it > failed to contact the directory authorities and hanged). We hope to move on > to testing our design while clean up some uncertain "loose-ends" in the > integration process, with your help. In particular: > > • We decided to implement a QUIC interface similar to the UNIX > socket interface. We have replaced TOR_SOCKET_T with our quicksock_t. Any > issues about this design? > > I'm not particularly familiar with this part of the tor codebase, you > might get better answers from tor-dev. > Also, without knowing the definition of quicksock_t, I can't say much more. > > Overall, I think your design has two major flaws: > * you need to provide QUIC as an alternative, not a replacement, because > Exits need to speak TCP, and so might Clients and Onion Services > * you've chosen to modify all tor sockets to speak QUIC, rather than > replacing SSL over TCP with QUIC > > Why not put your quicksock_t in connection_t (in or.h), and then choose > whether to use QUIC or a plain socket based on whether you're using an OR > connection (or_connection_t) or not? > You'd also need some logic in OR listener connections > (listener_connection_t) that creates a QUIC or plain socket from the > listener depending on whether it's an OR listener or not. > > See "Kinds of Connections" in > https://gitweb.torproject.org/user/nickm/torguts.git/tree/02-dataflow.md > And > https://gitweb.torproject.org/user/nickm/torguts.git/plain/diagrams/02/02-c… > And maybe > https://gitweb.torproject.org/user/nickm/torguts.git/plain/diagrams/02/02-d… > > Here are my concerns in detail: > > I think you need a design that enables QUIC over UDP as an alternative to > SSL over TCP sockets, like the way Tor's IPv6 support provides IPv6 as an > alternative to IPv4. (Of course, IPv6 is not the same level of the protocol > stack as QUIC.) > > If you completely replace TCP with QUIC, then Exits will also speak QUIC > to sites on the wider Internet. (As you describe below.) > Also, relays with speak QUIC to the directory authorities when downloading > descriptors, rather than using HTTP over TCP. > And you'll force the browser and Tor controllers to speak QUIC to the Tor > Client. > And Tor Onion Services to speak QUIC to the backend servers they connect > to. > (I think some parts of Tor's DNS might also be affected, but it depends on > the level of abstraction they're at.) > > This is unlikely to be what you want. And there could also be other > impacts on connections that should stay TCP. > > > • About SSL, we want to adopt the design where QUIC handles SSL by > not using the ssl_buffer in TOR code. Any concerns at all? > > You'll need to ask tor-dev for this one, I'm not familiar with this code > at all. > > > • During integration, I found many instances where TOR assumes > TOR_SOCKET_T is INT. Why can TOR make this assumption? Is the type > TOR_SOCKET_T only created to accommodate windows dev? > > I think the assumption was already there when Windows support was added. > It would appear that tor_socket_t exists for Windows compatibility. > > From the definition of tor_socket_t in compat.h: > > /* XXX Actually, this should arguably be SOCKET; we use intptr_t here so > that > * any inadvertent checks for the socket being <= 0 or > 0 will probably > * still work. */ > > > • For testing, what's the most correct (and easy enough to do in a > few weeks) way to simulate the TOR network? Right now I'm thinking using > the microdesc cache to avoid contacting any directory authorities. I will > create a cluster of about 5 nodes in server mode, all of which only know > the other nodes running QUIC. Do you think this will work? > > Tor has a TestingTorNetwork config option, which modifies a significant > number of settings to enable local testing. > If you define your own authorities, they can provide a consensus to your > clients. > This could be much more reliable than using the microdesc cache. > (Please read the TestingTorNetwork entry in the tor manual for more > details.) > > You may find this easiest to set up using chutney, which is how we test a > full tor network on a local machine. > Chutney will configure authorities, relays, and clients for you, then > launch the network. > The easiest way to set it up is to: > 1. Clone https://gitweb.torproject.org/chutney.git > 2. Clone your tor repository next to the chutney clone > 3. Run tor's ./configure > 4. Run tor's make test-network > > If this doesn't work, you might find it easiest to start with the chutney > "basic" template: > 5. Run src/test/test-network.sh --flavor basic > > If you start by running chutney on tor master, you'll get an idea of how > it works. > Then you can see what breaks when you try it with your QUIC-based tor. > > > • Anything else bad about the implementation? > > I haven't seen the implementation. > Have you sent me a link, or did you want me to give advice based on the > design issues you describe in the email you forwarded? > > As far as I can see, your major issues are getting a Tor test network > running (I strongly recommend chutney), and dealing with Exits needing > QUIC, TCP, and DNS/UDP. If you want interoperability with non-QUIC browsers > and servers, you'll need to work out similar issues with Clients and Onion > Services as well. > > Tim > > > Thank you! > > Li. > > > > > > > > > > ---------- Forwarded message ---------- > > From: Xiaofan Li <xli2(a)andrew.cmu.edu> > > Date: Mon, Apr 4, 2016 at 3:20 PM > > Subject: [TOR+QUIC] Progress Update > > To: Mashael AlSabah <malsabah(a)gmail.com> > > Cc: Srinivasan Seshan <srini(a)cs.cmu.edu>, Kevin Ku <kku(a)andrew.cmu.edu> > > > > > > Hi all, > > Before the project meetings, we want to debrief you on the recent > progress of the project so that our discussions can be more efficient. In > general, this week we made reasonable progress and now we are in the > debugging/testing phase. Particularly, we: > > • Designed and implemented the interaction between QUIC and the > libevent system inside TOR. > > • Integrating QUIC with TOR causes mismatches of > abstractions (see below). We worked hard to come up with a clean design > while not blocking development. > > • We have to pass the event_base to QUIC for it to > register an alarm callback that is required by QUIC. > > • libevent -> TOR-register callbacks -> QUIC interface -> > data -> QUIC processing -> only application data gets passed up to TOR > > • Minimum code change in TOR: we still use all the > callback mechanisms (timer + signal + async) in TOR but we modify those > callbacks. > > • Modified code paths inside TOR to accommodate QUIC. > > • We turned off Tor's SSL code path in the read/write > callbacks. We should probably also turn off the handshake inside TOR as > well. > > • TOR has a lot of implementation shortcut that make > assumptions about the TOR_SOCKET_T type. We cleaned this up. > > • Some remaining issues about direct low-level calls on > the socket (e.g. fcntl or setsockopt). The effects of these issues are > unclear right now. > > • Small hacks to make sure things ran. > > • We hardcoded the certificate and private key used by > QUIC. > > • QUIC is compiled into a shared library and it seems to > be working well. (Don't know about the SSL things right now.) > > • Tweaked the our QUIC interface to allow generality and smooth > integration with TOR. > > Some challenging problems we encountered: > > • Mismatched abstractions: This is obviously the most challenging > implementation problem. We hope our experience can perhaps suggestion some > designs for the ongoing QUIC interface design: > > • SSL/TLS: Fundamentally, QUIC is providing TLS security > on a lower level than what TOR is expecting and this is causing problems. > We should take away the TOR's SSL/TLS code to achieve the benefits of QUIC. > > • Other Implementations: Compared to the UNIX socket type, > our QUIC "socket" provides different interface and functionalities and it > is possible that there are assumptions made by TOR about the socket > interface. For example, the fact that QUIC requires an alarm system is > destroying the abstraction separation between TOR and QUIC. The alarms are > used for path discovery, handshakes, retransmission etc. We believe that > those should be internal to the lower level transport and thus hidden by > QUIC. We are unable to hide them right now because QUIC does not have the > necessary OS support to do so. The performance implication is unclear. > > • Testing: We are in the phase of testing. We are investigating > various testing methods. The key problem is that we have to basically scrap > all the systems that TOR already provides because we speak a different > protocol. Particularly, we need to: > > • Replace or disable the directory authorities. Disabling > is easier because we can hardcode the bootstrapping process and make the OP > contact our QUIC node, but this way we might not have the most accurate > (and comparable) data on the bootstrapping process. > > • We need to set up some number of circuits and > differentiate normal nodes from exit nodes. Right now the exit nodes will > also speak QUIC which is not good. > > • We will probably need 3-5 machines at least. (Some > Amazon AWS credit maybe??) > > Right now, our simple_quic interface is relatively stable and it is > located at: https://github.com/kku1993/simple-quic . It offers a purely > generic interface that does not specifically tie to TOR. Feel free to take > a look! We are thinking about giving it to the QUIC people at Google after > we polish and document it better. > > Kevin, feel free to add anything. > > > > Looking forward to the meetings! > > Li. > >

2 1

Re: [tor-dev] Yawning's CFC, web caching, and PETs
by leep＠sigaint.org 04 Apr '16

04 Apr '16

Check out CENO. Right now it runs Freenet as a back-end for distributed caching but it can gain Tahoe-LAFS support. https://censorship.no/

1 0