July 2012 - tor-commits - lists.torproject.org

[torspec/master] Add proposal 205: Remove global client-side DNS caching
by nickm＠torproject.org 20 Jul '12

20 Jul '12

commit 610f21f2f6c4efda748d95c37234c63344b3037f Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 20 16:26:56 2012 -0400 Add proposal 205: Remove global client-side DNS caching --- proposals/205-local-dnscache.txt | 146 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 146 insertions(+), 0 deletions(-) diff --git a/proposals/205-local-dnscache.txt b/proposals/205-local-dnscache.txt new file mode 100644 index 0000000..c0e19e5 --- /dev/null +++ b/proposals/205-local-dnscache.txt @@ -0,0 +1,146 @@ +Filename: 205-local-dnscache.txt +Title: Remove global client-side DNS caching +Author: Nick Mathewson +Created: 20 July 2012 +Status: Open + + +0. Overview + + This proposal suggests that, for reasons of security, we move + client-side DNS caching from a global cache to a set of per-circuit + caches. + + This will break some things that used to work. I'll explain how to + fix them. + +1. Background and Motivation + + Since the earliest Tor releases, we've kept a client-side DNS + cache. This lets us implement exit policies and exit enclaves -- + if we remember that www.mit.edu is 18.9.22.169 the first time we + see it, then we can avoid making future requests for www.mit.edu + via any node that blocks net 18. Also, if there happened to be a + Tor node at 18.9.22.169, we could use that node as an exit enclave. + + But there are security issues with DNS caches. A malicious exit + node or DNS server can lie. And unlike other traffic, where the + effect of a lie is confined to the request in question, a malicious + exit node can affect the behavior of future circuits when it gives + a false DNS reply. This false reply could be used to keep a client + connecting to an MITM'd target, or to make a client use a chosen + node as an exit enclave for that node, or so on. + + With IPv6, tracking attacks will become even more possible: A + hostile exit node can give every client a different IPv6 address + for every hostname they want to resolve, such that every one of + those addresses is under the attacker's control. + + And even if the exit node is honest, having a cached DNS result can + cause Tor clients to build their future circuits distinguishably: + the exit on any subsequent circuit can tell whether the client knew + the IP for the address yet or not. + + So client-side DNS caching needs to go away. + +2. Design + +2.1. The basic idea + + I propose that clients should cache DNS results in per-circuit DNS + caches, not in the global address map. + +2.2. What about exit policies? + + Microdescriptor-based clients have already dropped the ability to + track which nodes declare which exit policies, without much ill + effect. As we go forward, I think that remembering the IP address + of each request so that we can match it to exit policies will even + less effective, especially if proposals to allow AS-based exit + policies can succeed. + +2.3. What about exit enclaves? + + Exit enclaves are already borken. They need to move towards a + cross-certification solution where a node advertises that it can + exit to a hostname or domain X.Y.Z, and a signed record at X.Y.Z + advertises that . That's out-of-scope for this proposal, except to + note that nothing proposed here keeps that design from working. + +2.4. What about address mapping? + + Our current address map algorithm is, more or less: + + N = 0 + while N < MAX_MAPPING && exists map[address]: + address = map[address] + N = N + 1 + if N == MAX_MAPPING: + Give up, it's a loop. + + Where 'map' is the union of all mapping entries derived from the + controller, the configuration file, trackhostexits maps, + virtual-address mps, DNS replies, and so on. + + With this design, the DNS cache will not be part of the address + map. That means that entries in the address map which relied on + happening after the DNS cache entries can no longer work so well. + These would include: + + A) Mappings from an IP address to a particular exit, either + manually declared or inserted by TrackHostExits. + B) Mappings from IP addresses to other IP addresses. + C) Mappings from IP addresses to hostnames. + + We can try to solve these by introducing an extra step of address + mapping after the DNS cache is applied. In other words, we should + apply the address map, then see if we can attach to a circuit. If + we can, we try to apply that circuit's dns cache, then apply the + address map again. + + +2.5. What about the performance impact? + + That all depends on application behavior. + + If the application continues to make all of its requests with the + hostname, there shouldn't be much trouble. Exit-side DNS caches and + exit-side DNS will avoid any additional round trips across the Tor + network; compared to that, the time to do a DNS resolution at the + exit node *should* be small. + + That said, this will hurt performance a little in the case where + the exit node and its resolver don't have the answer cached, and it + takes a long time to resolve the hostname. + + + If the application is doing "resolve, then connect to an IP", see + 2.6 below. + +2.6. What about DNSPort? + + If the application is doing its own DNS caching, they won't get + much security benefit from here. + + If the application is doing a resolve before each connect, there + will be a performance hit when the resolver is using a circuit that + hadn't previously resolved the address. + + Also, DNSPort users: AutomapHostsOnResolve is your friend. + +3. Alternate designs and future directions + +3.1. Why keep client-side DNS caching at all? + + A fine question! I am not sure it actually buys us anything any + longer, since exits also have DNS caching. Shall we discuss that? + It would sure simplify matters. + +3.2. The impact of DNSSec + + Once we get DNSSec support, clients will be able to verify whether + an exit's answers are correctly signed or not. When that happens, + we could get most of the benefits of global DNS caching back, + without most of the security issues, if we restrict it to + DNSSec-signed answers. +

1 0

[torspec/master] Cleanups of typos found by arma
by nickm＠torproject.org 20 Jul '12

20 Jul '12

commit 60cec021a3ec4fa79c14dbe087578dc95c225f09 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 20 18:26:22 2012 -0400 Cleanups of typos found by arma --- proposals/205-local-dnscache.txt | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-) diff --git a/proposals/205-local-dnscache.txt b/proposals/205-local-dnscache.txt index c0e19e5..e25e456 100644 --- a/proposals/205-local-dnscache.txt +++ b/proposals/205-local-dnscache.txt @@ -39,7 +39,10 @@ Status: Open And even if the exit node is honest, having a cached DNS result can cause Tor clients to build their future circuits distinguishably: the exit on any subsequent circuit can tell whether the client knew - the IP for the address yet or not. + the IP for the address yet or not. Further, if the site's DNS + provides different answers to clients from different parts of the + world, then the client's cached choice of IP will reveal where it + first learned about the website. So client-side DNS caching needs to go away. @@ -55,8 +58,8 @@ Status: Open Microdescriptor-based clients have already dropped the ability to track which nodes declare which exit policies, without much ill effect. As we go forward, I think that remembering the IP address - of each request so that we can match it to exit policies will even - less effective, especially if proposals to allow AS-based exit + of each request so that we can match it to exit policies will be + even less effective, especially if proposals to allow AS-based exit policies can succeed. 2.3. What about exit enclaves? @@ -64,8 +67,9 @@ Status: Open Exit enclaves are already borken. They need to move towards a cross-certification solution where a node advertises that it can exit to a hostname or domain X.Y.Z, and a signed record at X.Y.Z - advertises that . That's out-of-scope for this proposal, except to - note that nothing proposed here keeps that design from working. + advertises that the node is an enclave exit for X.Y.Z. That's + out-of-scope for this proposal, except to note that nothing + proposed here keeps that design from working. 2.4. What about address mapping? @@ -80,7 +84,7 @@ Status: Open Where 'map' is the union of all mapping entries derived from the controller, the configuration file, trackhostexits maps, - virtual-address mps, DNS replies, and so on. + virtual-address maps, DNS replies, and so on. With this design, the DNS cache will not be part of the address map. That means that entries in the address map which relied on

1 0

[torspec/master] Add proposal 205 to the index
by nickm＠torproject.org 20 Jul '12

20 Jul '12

commit ec9ca811f71b36d2f3cf498da68fafc250d863b5 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 20 18:26:34 2012 -0400 Add proposal 205 to the index --- proposals/000-index.txt | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 78de7dc..7671ce5 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -125,6 +125,7 @@ Proposals by number: 202 Two improved relay encryption protocols for Tor cells [OPEN] 203 Avoiding censorship by impersonating an HTTPS server [DRAFT] 204 Subdomain support for Hidden Service addresses [OPEN] +205 Remove global client-side DNS caching [OPEN] Proposals by status: @@ -166,6 +167,7 @@ Proposals by status: 201 Make bridges report statistics on daily v3 network status requests [for 0.2.4.x] 202 Two improved relay encryption protocols for Tor cells 204 Subdomain support for Hidden Service addresses + 205 Remove global client-side DNS caching ACCEPTED: 117 IPv6 exits [for 0.2.4.x] 140 Provide diffs between consensuses

1 0

[translation/orbot_completed] Update translations for orbot_completed
by translation＠torproject.org 20 Jul '12

20 Jul '12

commit b845769293a179e14cf19d81c97743fad32dbac6 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Jul 20 21:45:09 2012 +0000 Update translations for orbot_completed --- values-es/strings.xml | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/values-es/strings.xml b/values-es/strings.xml index 98bc2cf..876434e 100644 --- a/values-es/strings.xml +++ b/values-es/strings.xml @@ -119,8 +119,9 @@ <string name="pref_entrance_node">Los nodos de entrada</string> <string name="pref_entrance_node_summary">Las llaves publicas, apodos, países y direcciones para el primer salto, o nodo (primer servidor que se conectara para acceder a la red Tor)</string> <string name="pref_entrance_node_dialog">Introduzca los nodos de entrada</string> - <string name="pref_use_whispercore">Utilizar plataforma de seguridad WhisperCore</string> - <string name="pref_use_whispercore_summary">Utilizar las API de NetFilter proporcionada por WhisperSystems (es necesario que su dispositivo tenga WhisperCore instalado)</string> +  + <string name="pref_proxy_title">Proxy de Red Saliente</string> <string name="pref_proxy_type_title">Tipo de proxy</string> <string name="pref_proxy_type_summary">Protocolo que se utilizará para la configuracion del servidor proxy: HTTP, HTTPS, SOCKS4, SOCKS5</string> <string name="pref_proxy_type_dialog">Introduzca el Tipo de Proxy</string>

1 0

[translation/orbot] Update translations for orbot
by translation＠torproject.org 20 Jul '12

20 Jul '12

commit 8e96cf6bb3f1a3178b7144557f207c696f3ab876 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Jul 20 21:45:05 2012 +0000 Update translations for orbot --- values-es/strings.xml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/values-es/strings.xml b/values-es/strings.xml index 6d393bb..876434e 100644 --- a/values-es/strings.xml +++ b/values-es/strings.xml @@ -121,7 +121,7 @@ <string name="pref_entrance_node_dialog">Introduzca los nodos de entrada</string>  - <string name="pref_proxy_title">Outbound Network Proxy</string> + <string name="pref_proxy_title">Proxy de Red Saliente</string> <string name="pref_proxy_type_title">Tipo de proxy</string> <string name="pref_proxy_type_summary">Protocolo que se utilizará para la configuracion del servidor proxy: HTTP, HTTPS, SOCKS4, SOCKS5</string> <string name="pref_proxy_type_dialog">Introduzca el Tipo de Proxy</string>

1 0

r25725: {website} scale the image down a bit. (website/trunk/images)
by Andrew Lewman 20 Jul '12

20 Jul '12

Author: phobos Date: 2012-07-20 17:26:56 +0000 (Fri, 20 Jul 2012) New Revision: 25725 Modified: website/trunk/images/InternetDefenseLeague-footer-badge.png Log: scale the image down a bit. Modified: website/trunk/images/InternetDefenseLeague-footer-badge.png =================================================================== (Binary files differ)

1 0

r25724: {website} update the logo to something that makes more sense to strang (website/trunk/images)
by Andrew Lewman 20 Jul '12

20 Jul '12

Author: phobos Date: 2012-07-20 17:22:20 +0000 (Fri, 20 Jul 2012) New Revision: 25724 Modified: website/trunk/images/InternetDefenseLeague-footer-badge.png Log: update the logo to something that makes more sense to strangers. Modified: website/trunk/images/InternetDefenseLeague-footer-badge.png =================================================================== (Binary files differ)

1 0

r25723: {website} add the internet defense league logo in our footer. (in website/trunk: images include)
by Andrew Lewman 20 Jul '12

20 Jul '12

Author: phobos Date: 2012-07-20 17:18:24 +0000 (Fri, 20 Jul 2012) New Revision: 25723 Added: website/trunk/images/InternetDefenseLeague-footer-badge.png Modified: website/trunk/include/foot.wmi Log: add the internet defense league logo in our footer. Added: website/trunk/images/InternetDefenseLeague-footer-badge.png =================================================================== (Binary files differ) Property changes on: website/trunk/images/InternetDefenseLeague-footer-badge.png ___________________________________________________________________ Added: svn:mime-type + image/png Modified: website/trunk/include/foot.wmi =================================================================== --- website/trunk/include/foot.wmi 2012-07-20 17:18:07 UTC (rev 25722) +++ website/trunk/include/foot.wmi 2012-07-20 17:18:24 UTC (rev 25723) @@ -65,6 +65,9 @@ <li><a href="<page docs/faq>">General Tor FAQ</a></li> </ul> </div> + <div class="col"> + <a href="http://internetdefenseleague.org/"><img src="$(IMGROOT)/InternetDefenseLeague-footer-badge.png" alt"Internet Defense League"></a> + </div>  #

1 0

r25722: {website} add new mirror. (in website/trunk: . include)
by Andrew Lewman 20 Jul '12

20 Jul '12

Author: phobos Date: 2012-07-20 17:18:07 +0000 (Fri, 20 Jul 2012) New Revision: 25722 Modified: website/trunk/include/mirrors-table.wmi website/trunk/update-mirrors.pl Log: add new mirror. Modified: website/trunk/include/mirrors-table.wmi =================================================================== --- website/trunk/include/mirrors-table.wmi 2012-07-19 21:12:15 UTC (rev 25721) +++ website/trunk/include/mirrors-table.wmi 2012-07-20 17:18:07 UTC (rev 25722) @@ -1,6 +1,57 @@ <tr> + <td>DE</td> + + <td></td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://torproject.lightning-bolt.net/dist/">http</a></td> + <td><a href="http://torproject.lightning-bolt.net/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + + <td>TN</td> + + <td></td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://torproject.antagonism.org/dist/">http</a></td> + <td><a href="http://torproject.antagonism.org/">http</a></td> + <td><a href="https://torproject.antagonism.org/dist/">https</a></td> + <td><a href="https://torproject.antagonism.org/">https</a></td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + + <td>NL</td> + + <td>Amorphis</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://tor.amorphis.eu/dist/">http</a></td> + <td><a href="http://tor.amorphis.eu/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>PT</td> <td>5ª Coluna</td> @@ -18,6 +69,40 @@ <tr> + <td>US</td> + + <td></td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://ec2-50-112-70-145.us-west-2.compute.amazonaws.com/mirrors/torproject…">http</a></td> + <td><a href="http://ec2-50-112-70-145.us-west-2.compute.amazonaws.com/mirrors/torproject…">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + + <td>NL</td> + + <td>CCC</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://tor.ccc.de/dist/">http</a></td> + <td><a href="http://tor.ccc.de/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>EE</td> <td>CyberSIDE</td> @@ -35,6 +120,23 @@ <tr> + <td>IS</td> + + <td>torproject.is</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://torproject.is/dist/">http</a></td> + <td><a href="http://torproject.is/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>IT</td> <td></td> @@ -103,6 +205,23 @@ <tr> + <td>IS</td> + + <td>myRL.net</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://tor.myrl.net/dist/">http</a></td> + <td><a href="http://tor.myrl.net/">http</a></td> + <td><a href="https://tor.myrl.net/dist/">https</a></td> + <td><a href="https://tor.myrl.net/">https</a></td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>US</td> <td>hessmo</td> @@ -120,19 +239,19 @@ <tr> - <td>DE</td> + <td>LT</td> - <td>crazyhaze.de</td> + <td></td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.crazyhaze.de/dist/">http</a></td> - <td><a href="http://tor.crazyhaze.de/">http</a></td> - <td><a href="https://tor.crazyhaze.de/dist/">https</a></td> - <td><a href="https://tor.crazyhaze.de/">https</a></td> + <td><a href="http://tor.vesta.nu/dist/">http</a></td> + <td><a href="http://tor.vesta.nu/">http</a></td> <td> - </td> <td> - </td> + <td> - </td> + <td> - </td> </tr> <tr> @@ -188,6 +307,23 @@ <tr> + <td>AT</td> + + <td>cyberarmy</td> + + <td>Up to date</td> + + <td> - </td> + <td> - </td> + <td><a href="http://tor.cyberarmy.at/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>US</td> <td>searchprivate</td> @@ -256,6 +392,23 @@ <tr> + <td>US</td> + + <td></td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://tor.onthegetgo.com/dist/">http</a></td> + <td><a href="http://tor.onthegetgo.com/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>HU</td> <td>Unknown</td> @@ -292,6 +445,23 @@ <td>US</td> + <td>NW Linux</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://torproject.nwlinux.us/dist/">http</a></td> + <td><a href="http://torproject.nwlinux.us/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td><a href="rsync://nwlinux.us/tor-web">rsync</a></td> +</tr> + +<tr> + + <td>US</td> + <td>Xpdm</td> <td>Up to date</td> @@ -341,15 +511,15 @@ <tr> - <td>FR</td> + <td>US</td> - <td>LazyTiger</td> + <td>AskApache</td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.taiga-san.net/dist/">http</a></td> - <td><a href="http://tor.taiga-san.net/">http</a></td> + <td><a href="http://tor.askapache.com/dist/">http</a></td> + <td><a href="http://tor.askapache.com/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -409,15 +579,15 @@ <tr> - <td>US</td> + <td>SE</td> - <td></td> + <td>homosu</td> <td>Up to date</td> <td> - </td> - <td><a href="http://torproject.umbrellacorporation.org.uk/dist/">http</a></td> - <td><a href="http://torproject.umbrellacorporation.org.uk/">http</a></td> + <td><a href="http://tor.homosu.net/dist/">http</a></td> + <td><a href="http://tor.homosu.net/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -426,40 +596,6 @@ <tr> - <td>INT</td> - - <td>CoralCDN</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://www.torproject.org.nyud.net/dist/">http</a></td> - <td><a href="http://www.torproject.org.nyud.net/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>TN</td> - - <td></td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://torproject.antagonism.org/dist/">http</a></td> - <td><a href="http://torproject.antagonism.org/">http</a></td> - <td><a href="https://torproject.antagonism.org/dist/">https</a></td> - <td><a href="https://torproject.antagonism.org/">https</a></td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - <td>US</td> <td></td> @@ -467,8 +603,8 @@ <td>Up to date</td> <td> - </td> - <td><a href="http://ec2-50-112-70-145.us-west-2.compute.amazonaws.com/mirrors/torproject…">http</a></td> - <td><a href="http://ec2-50-112-70-145.us-west-2.compute.amazonaws.com/mirrors/torproject…">http</a></td> + <td><a href="http://torproject.umbrellacorporation.org.uk/dist/">http</a></td> + <td><a href="http://torproject.umbrellacorporation.org.uk/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -477,125 +613,6 @@ <tr> - <td>NL</td> - - <td>CCC</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.ccc.de/dist/">http</a></td> - <td><a href="http://tor.ccc.de/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>IS</td> - - <td>myRL.net</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.myrl.net/dist/">http</a></td> - <td><a href="http://tor.myrl.net/">http</a></td> - <td><a href="https://tor.myrl.net/dist/">https</a></td> - <td><a href="https://tor.myrl.net/">https</a></td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>LT</td> - - <td></td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.vesta.nu/dist/">http</a></td> - <td><a href="http://tor.vesta.nu/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>AT</td> - - <td>cyberarmy</td> - - <td>Up to date</td> - - <td> - </td> - <td> - </td> - <td><a href="http://tor.cyberarmy.at/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>US</td> - - <td></td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.onthegetgo.com/dist/">http</a></td> - <td><a href="http://tor.onthegetgo.com/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - - <td>US</td> - - <td>NW Linux</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://torproject.nwlinux.us/dist/">http</a></td> - <td><a href="http://torproject.nwlinux.us/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td><a href="rsync://nwlinux.us/tor-web">rsync</a></td> -</tr> - -<tr> - - <td>US</td> - - <td>AskApache</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.askapache.com/dist/">http</a></td> - <td><a href="http://tor.askapache.com/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - <td>DE</td> <td>[[:bbs:]]</td> @@ -613,15 +630,15 @@ <tr> - <td>NL</td> + <td>INT</td> - <td>Amorphis</td> + <td>CoralCDN</td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.amorphis.eu/dist/">http</a></td> - <td><a href="http://tor.amorphis.eu/">http</a></td> + <td><a href="http://www.torproject.org.nyud.net/dist/">http</a></td> + <td><a href="http://www.torproject.org.nyud.net/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -630,28 +647,11 @@ <tr> - <td>SE</td> - - <td>homosu</td> - - <td>Up to date</td> - - <td> - </td> - <td><a href="http://tor.homosu.net/dist/">http</a></td> - <td><a href="http://tor.homosu.net/">http</a></td> - <td> - </td> - <td> - </td> - <td> - </td> - <td> - </td> -</tr> - -<tr> - <td>DK</td> <td>Zentrum der Gesundheit</td> - <td>Out of date</td> + <td>Up to date</td> <td> - </td> <td><a href="http://tor.idnr.ws/dist/">http</a></td> @@ -664,19 +664,19 @@ <tr> - <td>IS</td> + <td>DE</td> - <td>torproject.is</td> + <td>crazyhaze.de</td> - <td>Out of date</td> + <td>Up to date</td> <td> - </td> - <td><a href="http://torproject.is/dist/">http</a></td> - <td><a href="http://torproject.is/">http</a></td> + <td><a href="http://tor.crazyhaze.de/dist/">http</a></td> + <td><a href="http://tor.crazyhaze.de/">http</a></td> + <td><a href="https://tor.crazyhaze.de/dist/">https</a></td> + <td><a href="https://tor.crazyhaze.de/">https</a></td> <td> - </td> <td> - </td> - <td> - </td> - <td> - </td> </tr> <tr> @@ -783,6 +783,23 @@ <tr> + <td>FR</td> + + <td>LazyTiger</td> + + <td>Unknown</td> + + <td> - </td> + <td><a href="http://tor.taiga-san.net/dist/">http</a></td> + <td><a href="http://tor.taiga-san.net/">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>IS</td> <td>crypto.is</td> Modified: website/trunk/update-mirrors.pl =================================================================== --- website/trunk/update-mirrors.pl 2012-07-19 21:12:15 UTC (rev 25721) +++ website/trunk/update-mirrors.pl 2012-07-20 17:18:07 UTC (rev 25722) @@ -962,6 +962,24 @@ httpsDistMirror => "", rsyncDistMirror => "", hiddenServiceMirror => "", + }, + mirror066 => { + adminContact => "", + orgName => "", + isoCC => "DE", + subRegion => "", + region => "DE", + ipv4 => "True", + ipv6 => "False", + loadBalanced => "No", + httpWebsiteMirror => "http://torproject.lightning-bolt.net/", + httpsWebsiteMirror => "", + rsyncWebsiteMirror => "", + ftpWebsiteMirror => "", + httpDistMirror => "http://torproject.lightning-bolt.net/dist/", + httpsDistMirror => "", + rsyncDistMirror => "", + hiddenServiceMirror => "", } );

1 0

[ooni-probe/master] Implement the testing logic for redirect based censorship detection.
by art＠torproject.org 20 Jul '12

20 Jul '12

commit 777e3fe755b6805293f5ad8e50663fbd61899aae Author: Arturo Filastò <art(a)torproject.org> Date: Fri Jul 20 18:43:41 2012 +0200 Implement the testing logic for redirect based censorship detection. Closes #6437 --- ooni/assets/redirects.yaml | 17 +++++++++++++ ooni/plugins/httpt.py | 54 +++++++++++++++++++++++++++++++++++++++++++- ooni/protocols/http.py | 17 +++++++++++++ 3 files changed, 87 insertions(+), 1 deletions(-) diff --git a/ooni/assets/redirects.yaml b/ooni/assets/redirects.yaml new file mode 100644 index 0000000..43582c4 --- /dev/null +++ b/ooni/assets/redirects.yaml @@ -0,0 +1,17 @@ +o2: + name: O2 Networks + patterns: + - {type: re, value: '(https://bango.net/).*'} + - {type: re, value: 'http://wap.o2.co.uk/(18plusaccess).*'} +orange: + name: Orange Telecom + patterns: {type: eq, value: 'http://orangeworld.co.uk/r/avblocked/'} +three: + name: Three + patterns: {type: re, value: '(http://mobile.three.co.uk/pc/Live/pcreator/live/).*'} +tmobile: + name: T-Mobile + patterns: {type: eq, value: 'http://www.t-mobile.co.uk/common/system_error_pages/outage_wnw.html'} +vodafone: + name: Vodafone UK + patterns: {type: eq, value: 'http://online.vodafone.co.uk/en_GB/assets/static/contentcontrol/unbranded/r…'} diff --git a/ooni/plugins/httpt.py b/ooni/plugins/httpt.py index 4113aef..46f3b17 100644 --- a/ooni/plugins/httpt.py +++ b/ooni/plugins/httpt.py @@ -14,7 +14,8 @@ from ooni.utils import log class httptArgs(usage.Options): optParameters = [['urls', 'f', None, 'Urls file'], ['url', 'u', 'http://torproject.org/', 'Test single site'], - ['resume', 'r', 0, 'Resume at this index']] + ['resume', 'r', 0, 'Resume at this index'], + ['rules', 'y', None, 'Specify the redirect rules file']] class httptTest(http.HTTPTest): implements(IPlugin, ITest) @@ -25,7 +26,58 @@ class httptTest(http.HTTPTest): options = httptArgs blocking = False + + def testPattern(self, value, pattern, type): + if type == 'eq': + return value == pattern + elif type == 're': + import re + if re.match(pattern, value): + return True + else: + return False + else: + return None + + def testPatterns(self, patterns, location): + test_result = False + + if type(patterns) == list: + for pattern in patterns: + test_result |= self.testPattern(location, pattern['value'], pattern['type']) + else: + test_result |= self.testPattern(location, patterns['value'], patterns['type']) + + return test_result + + def testRules(self, rules, location): + result = {} + blocked = False + for rule, value in rules.items(): + current_rule = {} + current_rule['name'] = value['name'] + current_rule['patterns'] = value['patterns'] + current_rule['test'] = self.testPatterns(value['patterns'], location) + blocked |= current_rule['test'] + result[rule] = current_rule + result['blocked'] = blocked + return result + + def processRedirect(self, location): + self.result['redirect'] = None + if self.local_options['rules']: + import yaml + rules = yaml.load(open(self.local_options['rules'])) + log.msg("Testing rules %s" % rules) + redirect = self.testRules(rules, location) + self.result['redirect'] = redirect + else: + log.msg("No rules file. Got a redirect, but nothing to do.") + + def control(self, experiment_result, args): + print self.response + print self.request # What you return here ends up inside of the report. log.msg("Running control") return {} diff --git a/ooni/protocols/http.py b/ooni/protocols/http.py index 100db88..d5573b3 100644 --- a/ooni/protocols/http.py +++ b/ooni/protocols/http.py @@ -71,6 +71,15 @@ class HTTPTest(OONITest): """ pass + def processRedirect(self, location): + """ + Handle a redirection via a 3XX HTTP status code. + + @param location: the url that is being redirected to. + """ + pass + + def experiment(self, args): log.msg("Running experiment") url = self.local_options['url'] if 'url' not in args else args['url'] @@ -85,7 +94,15 @@ class HTTPTest(OONITest): def _cbResponse(self, response): self.response['headers'] = list(response.headers.getAllRawHeaders()) + self.response['code'] = response.code + self.response['length'] = response.length + self.response['version'] = response.length + + if str(self.response['code']).startswith('3'): + self.processRedirect(response.headers.getRawHeaders('Location')[0]) self.processResponseHeaders(self.response['headers']) + self.result['response'] = self.response + finished = defer.Deferred() response.deliverBody(BodyReceiver(finished)) finished.addCallback(self._processResponseBody)

1 0