- anti-censorship-team - lists.torproject.org

Fwd: Scheduled Client Panel & API Maintenance on December 3, 2024
by meskio 15 Aug '25

15 Aug '25

I got this email forwarded to me. It looks like there will be downtime in the Client Pannel of CDN77 what we use for domain fronting. > ---------- Forwarded message --------- > From: CDN77 <news(a)cdn77.com> > Date: Tue, Nov 26, 2024 at 9:35 AM > Subject: Scheduled Client Panel & API Maintenance on December 3, 2024 > To: accounting(a)torproject.org <accounting(a)torproject.org> > > > Due to schedule upgrades and maintenance, the CDN77 Client Panel and API > will be temporarily unavailable. > View in browser > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > Docs & API > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > Client > Panel > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > > Dear Client, > > Due to schedule upgrades and maintenance, the CDN77 Client Panel and API > will be temporarily unavailable. Other affected services include purge & > prefetch requests, as well as attempts to modify or change CDN Resource > settings and configurations via API. > > Please schedule any important changes before this time, and plan ahead with > this scheduled maintenance in mind. This maintenance *will not affect your > traffic flow*; all content delivery will continue as normal. > > *When:* December 3, 2024, starting at 10:00 AM UTC > > *Expected Downtime:* Up to 15 minutes > > *Need assistance?* Our 24/7 support team will be available through our > website at www.cdn77.com > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > . > > We apologize for any inconvenience and thank you for your understanding. > > Best Regards, > > *Juraj Kacaba* > > Head of Client Implementations > Email: support(a)cdn77.com > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > > Sent by DataCamp Limited > 9 Coldbath Square, London, United Kingdom > Unsubscribe > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > Contact > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> > Privacy > policy > <https://news.cdn77.com/tb/cl/eyJ2Ijoie1wiYVwiOjUwNjYwMyxcImxcIjoxMzkwNzQzNj…> -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

3 3

Many "??" geolocations on snowflake-01 in July 2025
by David Fifield 09 Aug '25

09 Aug '25

The user count on the snowflake-01 bridge rose greatly in the middle of July 2025, from 20,000 to about 50,000. But the connections with geolocation look normal; the increase was caused by a lot of "??" connections; that is, connections without client IP address geolocation. I've attached a graph that shows June and July 2025. There were also some "??" in June, though fewer. The "??" appear only on the snowflake-01 bridge, not snowflake-02. What could be the cause of geolocation failures? The client geolocation comes from the client IP address, which is supposed to be forwarded by the proxy: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… "??" probably means that the bridge did not receive a forwarded client_ip. I suppose it could also be the result of a geolocation lookup failure, but that seems unlikely. Some ideas about what might be happening: * There is a population of proxies out there that does not properly forward the client IP address. * There are clients out there that are connecting directly to the bridge (i.e., as a WebSocket pluggable transport), without using a proxy. * Maybe it's a researcher doing experiments. * Maybe some kind of attempted attack. "??" connections in June 2025 were previously remarked on here: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… "??" connections in August 2024 were previously remarked on here: https://archive.torproject.org/websites/lists.torproject.org/pipermail/anti… http://lists.torproject.org/pipermail/anti-censorship-team/attachments/2024…

1 0

recent censorship events
by meskio 07 Jul '25

07 Jul '25

Here I try to map the current internet blocks AFAIK, so others can look into them after the break (I'll be AFK next week). # Russia ## situation Main issue: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… FTE (obfs4) is blocked in mobile networks: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… webtunnel is being actively listed: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… ## ideas Does it make sense to start using conjure in Russia? It might solve the problem until we have dynamic webtunnel bridges. Maybe we can start with the community team distributing a bridge line to some users, and not jump all in with the circumvention settings. As we haven't seen many users in conjure yet. Should we explore integrating the domain fronting idea the community has developed in webtunnel and see if we can do something more on this. # Iran ## situation Partial block of snowflake: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… ## ideas Requesting more proxies: https://gitlab.torproject.org/tpo/community/relays/-/issues/116 The community team will need help on this, producing screenshots of grafana or other data for the campaign. We have our vantage points back in the country. I haven't had the time to investigate much. I guess we'll need to figure out if Iran is blocking proxies by fingerprint or by listing them. # China ## situation meek is blocked: https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/162 ## ideas Do this affects also to snowflake and/or moat? Looks like phpmyadmin.net is reachable, but maybe they fingerprinted our meek implementation? if so moat will not work either. Or some other problem with CDN77. We do have a vantage point in China, isn't it? I don't have access to it, maybe something to add to vaultwarden. Maybe someone can investigate this and see how far the block goes? Do we need to move to speed up the migration of moat to netlify? -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

2 1

SQS rendezvous exceeded free tier usage
by Cecylia Bocovich 05 Jun '25

05 Jun '25

I got a notification from AWS that our SQS rendezvous service exceeded the free-tier usage this month with over 1,000,000 SQS API requests. This is in some sense exciting news, because it shows that the rendezvous channel is effective and getting some use. It does, however mean that we will have to start paying for the service. The current billing period falls on month boundaries, from January 1st to January 31st. The budget action fired on January 9th, which is pretty early in the month. Looking at our broker metrics[0,1], there were approximately 38,608 client polls using SQS. That's approximately 2.5 requests per poll, which is about what I'd expect. It's reassuring to know that the budget actions work. I'm going to set them a little higher, to something I can reasonably afford. I don't yet know how the cost will scale with the number of polls, and how that will compare with the cost of domain fronted requests. [0] https://snowflake-broker.torproject.net/metrics [1] https://metrics.torproject.org/collector.html#type-snowflake-stats

4 7

caceled today (May 1st) meeting
by meskio 01 May '25

01 May '25

Today is May 1st, most of the people in the team is AFK and will not be around for our weekly meeting. Let's cancel it. See you next week. -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

1 0

Deprecation of Azure CDN from Edgio (azureedge.net), replaced by Azure Front Door (azurefd.net), affects snowflake-broker.azureedge.net
by David Fifield 22 Apr '25

22 Apr '25

I got an email on 2024-12-05 saying that that Azure CDN from Edgio/Verizon is going to shut down sooner than expected. The shutdown date was supposed to be 2025-11-04, now it is 2024-01-15 (about one month from now). This affects (at least) snowflake-broker.azureedge.net, which was first set up and is still on Azure CDN with Edgio. I think that snowflake-broker.azureedge.net has not been used by this team since 2021 and https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re…. But it is still getting some use from somewhere, as evidenced by the nonzero monthly bills: https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Snowflake-co… Whatever is out there that is still using snowflake-broker.azureedge.net will likely stop working after 2025-01-15. > Date: 5 Dec 2024 17:03:50 +0000 > From: Microsoft Azure <azure-noreply(a)microsoft.com> > Subject: Urgent action required: Migrate workloads on Azure CDN from Edgio (formerly Verizon) before 15 January 2025 > > Migrate to Azure Front Door or other CDN services as soon as possible > to avoid service disruptions. > > Migrate workloads on Azure CDN from Edgio (formerly Verizon) before 15 > January 2025. > > You're receiving this notice because you're currently using Azure CDN > Standard/Premium from Edgio. > > All Azure CDN from Edgio customers must migrate their workloads to > Azure Front Door or other CDN services before 15 January 2025 as Edgio > has advised their platform is currently scheduled to be shut down by > that date. Migrate as soon as possible to avoid an imminent service > shut-down. > > On 31 October 2024, we sent you a notice advising that Azure CDN > Standard/Premium from Edgio (formerly Verizon) will be retired on > 4 November 2025, and that customers of this service must migrate their > workload(s) to a comparable service before this date to avoid service > interruptions. In that email we also advised given that Edgio filed > for Chapter 11 Bankruptcy on 9 September 2024, Microsoft cannot > guarantee that Edgio will continue to support this service through > 4 November 2025 as previously stated. Online references about Azure CDN with Edgio retirement: https://learn.microsoft.com/en-us/azure/cdn/edgio-retirement-faq https://azure.microsoft.com/updates?id=467688 I did try setting up a Front Door CDN profile (which is the recommended replacement for Edgio), following the migration instructions: https://learn.microsoft.com/en-us/azure/frontdoor/migrate-cdn-to-front-door I got the hostname: snowflake-broker-hadmaqbnc4dmcffs.z03.azurefd.net It works as an alias for the broker: curl -i https://snowflake-broker-hadmaqbnc4dmcffs.z03.azurefd.net/debug However, it apparently does not work with domain fronting, so is likely not actually useful. Front Door not working with domain fronting is consistent with prior announcements by Azure: https://github.com/net4people/bbs/issues/67

8 22

set yourself a new password for prometheus / grafana
by Gabriel Filion 17 Apr '25

17 Apr '25

Hello! I'm writing to inform you that TPA's been moving on something that should make it possible to eventually merge both of the prometheus / grafana servers: we're moving towards per-user passwords that need to be set through LDAP and deprecating the shared passwords. I would invite everyone who needs to continue accessing prometheus and grafana (either on prometheus1.tpo or prometheus2.tpo) to head over to https://db.torproject.org/ , login and set yourself a password in the "Web password" field. That field was previously hidden but it was recently added to the form. The password will take some time to get synchronized to the servers, so allow some 1 to 2 hours before you test out your new credentials. When ready, head over to https://grafana2.torproject.org/ and use your ldap username with your new web password to confirm that you're able to login there. The shared passwords that are currently in use are bound to be removed on April 17th. If you're having issues with setting up and/or using your own credentials to access prometheus and grafana, please contact TPA and we'll take a look with you into what's happening. Cheers! additional note for TPA members: we now have a fallback password that's present in our password manager. it should let us access the monitoring sites even if ldap has a disruption. you can try that one as well.

3 6

Fwd: [Docker] Confirmation - Subscription Canceled
by meskio 15 Apr '25

15 Apr '25

I got this email out of the blue. It looks like docker has canceled our sponsored open source subscription without any warning. It looks like all our images are still there, I'm trying to contact them to see what is happening. Forwarded message from Docker (2025-04-15 02:04:16): > [1][IMG] > Hello thetorproject! > We're sad to see you go. This email confirms that your Docker-Sponsored > Open Source subscription for your account has been canceled. > [2]View past invoices > Please keep this email for your records. If you have any questions, reach > out to our [3]support team. > If you would like to regain access, you can [4]manage your subscription. > Thank you, > The Docker Team > > This email was sent to meskio(a)sindominio.net to notify you of an update > that was made to your Docker Account. > [5]© 2025 Docker Inc. > 3790 El Camino Real #1052, Palo Alto, CA 94306 -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

1 1

Snowflake bridges using SQS
by Michael Rogers 26 Mar '25

26 Mar '25

Hi all, After updating Briar's bridge config to use the current settings from Moat, we're seeing two Snowflake bridges consistently failing in our CI tests. They're the two bridges that use SQS. Here's a snippet from the log: ``` INFO: NOTICE Managed proxy "/builds/briar/onionwrapper/onionwrapper-java/test.tmp/35/lyrebird": offer created Feb 04, 2025 1:12:34 PM org.briarproject.onionwrapper.AbstractTorWrapper message INFO: NOTICE Managed proxy "/builds/briar/onionwrapper/onionwrapper-java/test.tmp/35/lyrebird": broker failure operation error SQS: GetQueueUrl, https response error StatusCode: 400, RequestID: 60e91cfa-a2a0-55db-beb0-7ce6b621d324, AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist. ``` Does the queue really not exist, or does this point to some other issue, like the bridges being geoIP restricted or the app needing to pass some extra information to the transport? Thanks, Michael

2 6

What key does the broker rate-limit on?
by David Fifield 28 Feb '25

28 Feb '25

The Snowflake broker's request rate limiting, what does it key on? The source IP address, X-Forwarded-For, or something else? This is where the rate limiting was introduced: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… location ~ ((proxy)|(client)|(answer)|(metrics)|(prometheus)|(amp/client/.*)|(robots.txt)) { limit_req zone=snowflake burst=3; proxy_pass http://127.0.0.1:8080; proxy_http_version 1.1; } limit_req_zone $binary_remote_addr zone=snowflake:10m rate=1r/s; Here's another, more recent snapshot of the configuration: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… location ~ ((proxy)|(client)|(answer)|(metrics)|(prometheus)|(amp/client/.*)|(robots.txt)) { limit_req zone=snowflake burst=3; proxy_pass http://127.0.0.1:8080; proxy_http_version 1.1; proxy_read_timeout 300; proxy_connect_timeout 300; proxy_send_timeout 300; proxy_set_header X-Forwarded-For $proxy_protocol_addr; } limit_req_zone $proxy_protocol_addr zone=snowflake:10m rate=1r/s; If the limited is using the source IP address, then different clients could be causing each other to be rate-limited, because many requests come from the same CDN IP address (or whatever). If the limiter is using X-Forwarded-For or similar, then it's possible to evade the limiter by putting random or incrementing IP addresses in the header.

2 2