- anti-censorship-team - lists.torproject.org

OONI stunreachability measurements from Russia
by David Fifield 19 May '22

19 May '22

I spent some time looking for the cause of high rates of anomalies in OONI's torsf test: https://explorer.ooni.org/chart/circumvention?since=2022-04-06&until=2022-0… I looked at the recently available (https://github.com/ooni/backend/issues/573) stunreachability measurements, to test whether the Snowflake connections were failing at the STUN phase. In summary, 3 of the pool of 12 STUN servers are inaccessible in Russia. One of them is blocked by censorship in Russia, and the other two look like geoblocking of Russian clients by the STUN service. I don't think the loss of 3 STUN servers is likely to be a cause of many connection failures; therefore I think the explanation lies elsewhere. Here are the charts: https://explorer.ooni.org/chart/mat?probe_cc=RU&test_name=stunreachability&… stun.voip.blackberry.com and the IP address 178.239.90.248 are on the unified register of blocked sites, the entry dated 2017-04-28: https://reestr.rublacklist.net/record/530002/ Here's a measurement showing the test timing out: https://explorer.ooni.org/measurement/20220224T192519Z_stunreachability_RU_… stun.stunprotocol.org is not on the unified register as far as I can tell, nor are its IP addresses 18.191.223.12 and 2600:1f16:8c5:101::108. This domain resolves to 127.0.0.1 or ::1 for requesters located in Russia. Here's a sample measurement: https://explorer.ooni.org/measurement/20220224T111232Z_stunreachability_RU_… Using ooni-sync, we can see that the domain resolves incorrectly only in Russia and Ukraine: $ ./ooni-sync -directory stunreachability test_name=stunreachability since=2022-05-18 until=2022-05-19 $ jq -r 'select(.test_keys.endpoint=="stun.stunprotocol.org:3478")|[.probe_cc,(.test_keys.queries[].answers[]?|(.ipv4? // .ipv6?)),.probe_network_name,.test_keys.failure]|@tsv' stunreachability/* | sort ... IT 18.191.223.12 2600:1f16:8c5:101::108 Vodafone Italia S.p.A. IT 18.191.223.12 2600:1f16:8c5:101::108 Vodafone Italia S.p.A. KZ 18.191.223.12 Kar-Tel LLC LK 18.191.223.12 2600:1f16:8c5:101::108 Mobitel Pvt Ltd LT 18.191.223.12 2600:1f16:8c5:101::108 Telia Lietuva, AB MX 18.191.223.12 2600:1f16:8c5:101::108 Mega Cable, S.A. de C.V. MX 18.191.223.12 Mega Cable, S.A. de C.V. NL 18.191.223.12 2600:1f16:8c5:101::108 T-Mobile Thuis BV RU 127.0.0.1 ::1 Cloudflare, Inc. generic_timeout_error RU 127.0.0.1 ::1 Iskratelecom CJSC generic_timeout_error RU 127.0.0.1 ::1 JSC "ER-Telecom Holding" generic_timeout_error RU 127.0.0.1 ::1 JSC "ER-Telecom Holding" generic_timeout_error RU 127.0.0.1 ::1 JSC "ER-Telecom Holding" generic_timeout_error RU 127.0.0.1 ::1 Novotelecom Ltd generic_timeout_error RU 18.191.223.12 PJSC "Bashinformsvyaz" SE 18.191.223.12 2600:1f16:8c5:101::108 Telia Company AB SE 18.191.223.12 Vetenskapsradet / SUNET SG 18.191.223.12 2600:1f16:8c5:101::108 Leaseweb Asia Pacific pte. ltd. TW 18.191.223.12 2600:1f16:8c5:101::108 <unknown> TW 18.191.223.12 2600:1f16:8c5:101::108 <unknown> UA 127.0.0.1 ::1 Virtual Systems LLC generic_timeout_error US 18.191.223.12 2600:1f16:8c5:101::108 AT&T Services, Inc. US 18.191.223.12 2600:1f16:8c5:101::108 AT&T Services, Inc. US 18.191.223.12 2600:1f16:8c5:101::108 CABLE ONE, INC. ... stun.altar.com.pl is not on the unified register, nor is its IP address 176.119.42.11. Its failures only start on 2022-03-09. Its domain usually resolves correctly, but in Russia the actual STUN phase usually results in a timeout: $ jq -r 'select(.test_keys.endpoint=="stun.altar.com.pl:3478")|[.probe_cc,(.test_keys.queries[].answers[]?|(.ipv4? // .ipv6?)),.probe_network_name,.test_keys.failure]|@tsv' stunreachability/* | sort | grep RU RU 176.119.42.11 Cloudflare, Inc. RU 176.119.42.11 Iskratelecom CJSC generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 Novotelecom Ltd generic_timeout_error RU 176.119.42.11 PJSC "Bashinformsvyaz" generic_timeout_error But there are also errors for networks in Spain, Hong Kong, Indonesia, and the United States: $ jq -r 'select(.test_keys.endpoint=="stun.altar.com.pl:3478")|[.probe_cc,(.test_keys.queries[].answers[]?|(.ipv4? // .ipv6?)),.probe_network_name,.test_keys.failure]|@tsv' stunreachability/* | sort | grep error ES 176.119.42.11 Orange Espagne SA generic_timeout_error HK Hong Kong University of Science and Technology generic_timeout_error ID Telekomunikasi Indonesia (PT) generic_timeout_error RU 176.119.42.11 Iskratelecom CJSC generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 JSC "ER-Telecom Holding" generic_timeout_error RU 176.119.42.11 Novotelecom Ltd generic_timeout_error RU 176.119.42.11 PJSC "Bashinformsvyaz" generic_timeout_error US 176.119.42.11 2607:7700:0:18::b077:2a0b T-Mobile USA, Inc. generic_timeout_error US 176.119.42.11 2607:7700:0:1f:0:1:b077:2a0b T-Mobile USA, Inc. generic_timeout_error US 176.119.42.11 2607:7700:0:7::b077:2a0b T-Mobile USA, Inc. generic_timeout_error US 176.119.42.11 T-Mobile USA, Inc. generic_timeout_error US 176.119.42.11 T-Mobile USA, Inc. generic_timeout_error To me this looks like selective denial of service to certain client networks. Last weeks discussion on this topic: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-05-12-15.58.log…

1 0

Recommendations to monitoring snowflake-proxy standalone
by Jacobo Nájera 30 Apr '22

30 Apr '22

Hi, I'm thinking about how is the best way to monitor snowflake-proxy standalone with an ansible role. What tools or approaches do you recommend for monitoring, and which indicators do you consider important to keep in mind. Thanks, Jacobo

1 0

Request for Comment on Distributed Snowflake Bridge Design
by Shelikhoo 22 Mar '22

22 Mar '22

Hi anti-censorship team and others, We are in the process in increase the scalability of the snowflake bridge. Here is the design I have drafted for informing proxy about which WebSocket address it should connect to and the security issue associated with that. https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Comments are encouraged so that the design can be improved before moving to the next stage. Shelikhoo

1 0

Rust port of extor-static-cookie
by David Fifield 01 Mar '22

01 Mar '22

As an experiment to wring more performance from the snowflake bridge, I started a rough port of the extor-static-cookie shim to Rust. It's not in what I would call a production-ready state with regard to documentation and tests—I wanted to hack something together quickly in order to decide whether it's worth spending more time on. I am also pretty new to Rust async programming. As a side consideration, I wanted to get an idea of what a pluggable transports library for Rust might look like. https://gitlab.torproject.org/dcf/extor-static-cookie/-/tree/rust Currently commit 559fb5e292b45d82464bdaf978c6e1783c796e3d I have the Rust implementation running now (since 2022-02-27 03:00) under the flakey4 instance. The other three instances are running the Go implementation. It's too early to say whether it reduces CPU usage; an hour after being restarted, the flakey4 instance is still catching up to the traffic of the other instances. But it looks like it's stable and functional, at least.

2 2

bridgedb+rdsys test setup
by meskio 28 Feb '22

28 Feb '22

For the last few months I've being working on integrating rdsys with bridgedb, so bridges are managed by rdsys and bridgedb only does the distribution (https, moat and email)[0]. There will not be any visible changes, but all the insides are different. Thanks to the TPA work now we have a testing setup of the new setup to test it. If you want to test it I'll be happy to hear if you find any issues with it. As this is distributing real bridges only people with accounts in TPO LDAP has access to it. How to access the different distributors: * email. Just write an email to bridges-test(a)torproject.org from your @torproject.org email, and should get a reply with bridges. * https. The web interface is available at https://bridges-test.torproject.org, but only reachable from people.torproject.org. You can make a ssh proxy: $ ssh -D 9999 people.torproject.org and configure your browser to use a SOCKS proxy at localhost with port 9999. * moat. This is reachable the same way that https only from people.torproject.org. Is a bit more complicated to test, but I will stress test it myself :) I wrote I simple script to do the moat captcha request and solution: https://gitlab.torproject.org/-/snippets/127 It's run like: $ MOAT_URL="https://bridges-test.torproject.org/moat" CURL_OPTIONS="--socks5 localhost:9999" ./test-moat Thank you. [0] https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/12 -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

2 5

Surge in Snowflake client polls
by Cecylia Bocovich 28 Feb '22

28 Feb '22

While looking at the snowflake broker metrics for clues about the sudden drops in known proxy NAT types[0], I noticed that the broker metrics show a very large spike in client polls over the last day. I've attached a few graphs that show this, which probably won't have showed up in tor relay metrics yet. One of the graphs is the usage of proxy capacity, which shows a rise in the number of client polls and a large drop in idle proxies. The other graph shows something similar, but for the pool of proxies available to clients with restrictive NATs. Here we see that the overwhelming majority of clients are being denied because all of the proxies are used. To some extent, we expect this and have seen it in the past, particularly for the pool of proxies for clients with restrictive NATs. The pattern follows a natural day-long period, and the number of denied clients never gets so high that a client won't get a proxy after 2-3 tries. However, these attached graphs go back 48 hours and show how much bigger the spike in clients is over the last day. At the peak of the spike, there were 5x as many clients requesting snowflake proxies from the restrictive pool as we had proxies to match. Perhaps this is a good time to push to get more proxies? It seems we could use proxies for both restrictive and unrestrictive NATs at this point. Cecylia [0] https://lists.torproject.org/pipermail/anti-censorship-alerts/2022-February…

1 1

Snowflake bridge outage, 2022-02-18
by David Fifield 24 Feb '22

24 Feb '22

On 2022-02-18 the Snowflake bridge was offline between about 14:02:05 and 15:26:38. I was alerted to the outage by Shelikhoo. The cause of the outage is unknown. There had been some kind of kernel crash (see first attached file for what was visible on the console), which required me to restart the VPS externally. After I restarted it, everything came back as normal. I asked Greenhost support if they knew of anything that might have happened in the data center to cause the problem, and they did not. I asked about using a management API to automatically restart the server if it happens again, but they did not have specific guidance on that. Resource graphs show that the CPU was still active during the outage, but the network usage was quiet.

1 0

Mac app / Safari extension for Snowflake
by David Fifield 22 Feb '22

22 Feb '22

Browsing through some search results, I found: https://apps.apple.com/us/app/torproject-snowflake/id1597501940 (https://web.archive.org/web/20220222083316/https://apps.apple.com/us/app/to…) https://github.com/jayluxferro/snowflake It looks like a repackaging of the Snowflake JavaScript proxy code for Mac and Safari environments. Maybe we could get in touch with the developer and find a way to adopt it into the main repository?

2 1

Snowflake bridge VPS cost estimates
by David Fifield 17 Feb '22

17 Feb '22

I haven't yet got a lead on finding a stronger server for the Snowflake bridge. I did some rough estimates: the cost of a VPS for what we need is about $250/month, and bandwidth will probably be about another $1000/month. The minimum hardware specs that would make moving the server worthwhile is 16 CPUs and 32 GB RAM. To get an idea of the range of pricing: * Linode 32 GB: $240/month https://www.linode.com/pricing/#compute-dedicated https://web.archive.org/web/20220207055603/https://www.linode.com/pricing/#… * Azure F16s v2: $292/month (1 year reserved) https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/ https://web.archive.org/web/20220215193358/https://azure.microsoft.com/en-u… * Hetzner CCX42: €166/month = $189/month https://www.hetzner.com/cloud https://web.archive.org/web/20220214145505/https://www.hetzner.com/cloud The estimate of $1000/month for bandwidth comes from a typical cost of $0.01/GB = $10/TB, and the fact that during the 6 days the staging bridge was active, it averaged about 2.9 TB/day sent (and was rising). https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…

2 2

obfs4proxy-0.0.12 (2021-12-31) fixes the Elligator2 bug
by David Fifield 17 Jan '22

17 Jan '22

The upstream obfs4 repository has a fix to the Elligator2 public key representative leak (https://github.com/agl/ed25519/issues/27) https://gitlab.com/yawning/obfs4/-/commit/393aca86cc3b1a5263018c10f87ece09a… All releases prior to this commit are trivially distinguishable with simple math, so upgrading is strongly recommended. The upgrade is fully backward-compatible with existing implementations, however the non-upgraded side will emit traffic that is trivially distinguishable from random. The file internal/README.md elaborates: All existing versions prior to the migration to the new code (anything that uses agl's code) are fatally broken, and trivial to distinguish via some simple math. For more details see Loup Vaillant's writings on the subject. Any bugs in the implementation are mine, and not his. Representatives created by this implementation will correctly be decoded by existing implementations. Public keys created by this implementation be it via the modified scalar basepoint multiply or via decoding a representative will be somewhat non-standard, but will interoperate with a standard X25519 scalar-multiply. As the obfs4 handshake does not include the decoded representative in any of it's authenticated handshake digest calculations, this change is fully-backward compatible (though the non-upgraded side of the connection will still be trivially distinguishable from random).

2 4