Summary: a proposal to limit the retention of GitLab CI data to 1 year
# Background
As more and more Tor projects moved to GitLab and embraced its
continuous integration features, managing the ensuing storage
requirements has been a challenge.
We regularly deal with near filesystem saturation incidents on the
GitLab server, especially involving CI artifact storage, such as
tpo/tpa/team#41402 and recently, tpo/tpa/team#41861
Previously, [TPA-RFC-14][] was implemented to reduce the default
artifact retention period from 30 to 14 days. This, and CI optimization
of individual projects has provided relief, but the long-term issue has
not been definitively addressed since the retention period doesn't apply
to some artifacts such as job logs, which are kept indefinitely by default.
[TPA-RFC-14]:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/tpa-rfc-14-gitlab-artifa…
# Proposal
Implement a daily GitLab maintenance task to delete CI pipelines older
than 1 year in *all* projects hosted on our instance. This will:
* Purge old CI pipeline and job records for the GitLab database
* Delete associated CI job artifacts, even those "kept" either:
* When [manually prevented from expiring][] ("Keep" button on CI job
pages)
* When they're the [latest successful pipeline artifact][]
* Delete old CI job log artifacts
[manually prevented from expiring]:
https://gitlab.torproject.org/help/ci/jobs/job_artifacts#with-an-expiry
[latest successful pipeline artifact]:
https://gitlab.torproject.org/help/ci/jobs/job_artifacts.md#keep-artifacts-…
## Goals
This is expected to significantly reduce the growth rate of CI-related
storage usage, and of the GitLab service in general.
## Affected users
All users of GitLab CI will be impacted by this change.
But more specifically, some projects have "kept" artifacts, which were
manually set not to expire. We'll ensure the concerned users and
projects will be notified of this proposal. GitLab's documentation has
the [instructions to extract this list of non-expiring
artifacts](https://docs.gitlab.com/ee/administration/cicd/job_artifacts_tro….
## Timeline
Barring the need to further discussion, this will be implemented on
Monday, December 9th.
## Costs estimates
### Hardware
This is expected to reduce future requirements in terms of storage hardware.
### Staff
This will reduce the amount of TPA labor needed to deal with filesystem
saturation incidents.
# Alternatives considered
A "CI housekeeping" script is already in place, which scrubs job logs
daily in a hard-coded list of key projects such as c-tor packaging,
which runs an elaborate CI pipeline on a daily basis, and triage-bot,
which runs it CI pipeline on a schedule, every 15 minutes.
Although it has helped up until now, this approach is not able to deal
with the increasing use of personal fork projects which are used for
development.
It's possible to define a different retention policy based on a
project's namespace. For example, projects under the `tpo` namespace
could have a longer retention period, while others (personal projects)
could have a shorter one. This isn't part of the proposal currently as
it could violate the principle of least surprise.
# References
* Discussion ticket: tpo/tpa/team#41874
* [Make It Ephemeral: Software Should Decay and Lose
Data](https://lucumr.pocoo.org/2024/10/30/make-it-ephemeral/)
Hello,
I'm writing to let you know that applications are now open for the
second SEEKCommons Fellowship[1] cohort.
The SEEKCommons Fellowship program is funded by NSF and run by partners
at University of Notre Dame, University of California Davis, University
of Michigan, University of Virginia, and The HDF Group. The goal of the
fellowship is to bring graduate students, post-doctoral researchers, and
professionals from community-based organizations with new perspectives
to socio-environmental research with open technologies.
Application deadline is Dec. 15, 2024.
The fellowship is designed to:
- Encourage new translational and integrative work involving
socio-environmental action research with Open Science practices;
- Provide a space for Fellows and Network members to collaborate on
common research issues, challenges, and solutions.
Fellows may be:
- Graduate students working with open technologies on
socio-environmental issues;
- Post-docs with existing community projects in Science and Technology
Studies, Open Science, and/or Socio-environmental research; and/or
- Community practitioners who are interested in integrating common
technologies into their environmental justice work.
The SEEKCommons website contains all the necessary fellowship
information, including the application link.
https://seekcommons.org/fellowship-application.html
Partnership SEEKCommons + Tor Project
=====================================
This year SEEKCommons is reserving one fellowship to sustainability
studies of community networks in partnership with the Tor Project!
We welcome Fellowship applications on the sustainability of the Tor
network with a focus on energy consumption and relay metadata (such as
ASN, uptime, and platform). The goal of the partnership between
SEEKCommons and Tor is to study the environmental impact of community
networks and promote the use of renewable energy in decentralized
infrastructures.
We would like to support applications that address one (or more) of these questions:
- What Free and Open Source technologies can be used to promote a more
sustainable and distributed Tor infrastructure?
- How can the energy consumption of the Tor network be measured and
optimized to reduce its environmental impact?
- How can the "Tor Snowflake" decentralized proxy model be used to
improve the sustainability of the network?
- How can we use metadata from relays (e.g., ASN / uptime / platform)
to assess the environmental impact of the network?
- What open hardware and renewable energy sources could be used/reused
in the Tor network?
More information: https://seekcommons.org/partnership-tor.html
If you have any questions, please don't hesitate to reach out.
Thank you so much!
Warmly,
Gus
[1] https://seekcommons.org/about.html
--
The Tor Project
Community Team Lead
Hello everyone,
Last month, our user support team worked on an unprecedented number of tickets
mostly due to the evolving Tor censorship in Russia[0] (big shoutout to @nina
and read her report[1] for more details!)
With 4 Tor Browser releases in last month, I worked on a number of Tor Browser
related user support tickets - from general questions about downloading,
installing, updating and troubleshooting to bug reports.
With the year-end campaign into it's second month, I also worked on related support
tickets, answering questions about Tor Browser. Topics included downloading, installing,
using the browser it's various privacy enhancing features and questions about
the Tor network in general.
I wrote and updated user documentation on the Tor Browser User manual[2] and some of the
articles and templates that we use on our various support channels.
Here's a more detailed breakdown of the tickets our user support team worked
on last month:
# Frontdesk (email user support channel)
* 614(↓) RT tickets created
* 558(↓) RT tickets resolved
Tickets by topics and numbers:
1. 226(↑) RT tickets: circumventing censorship in Russian speaking countries.
2. 213(↓) RT tickets: private bridge requests from Chinese speaking users.
3. 14(↓) RT tickets: help with Troubleshooting Tor Browser desktop on Windows, macOS
and Linux.
4. 6(↑) RT tickets: Tor Browser 14 crashing on macOS when visiting some
onionsites. (The bug is resolved with Tor Browser 14.0.2)
5. 5 RT tickets: WebTunnel bridges campaign.[3]
6. 4(↑) RT tickets: circumventing censorship with Tor in Farsi.
7. 4(↑) RT tickets: reports of a website blocking Tor traffic.
8. 3(-) RT tickets: Instructions to download Tor Browser 13.5 legacy for
legacy operating systems.
9. 3 RT tickets: Instructions on how to download and install Tor Browser.
10. 3 RT tickets: help with Troubleshooting Tor Browser on Android.
11. 2(↓) RT tickets: configuring Orbot to use bridges.
12. 2(↑) RT tickets: reports of a fake apps on iOS masquerading as official Tor Browser.
13. 2 RT tickets: include uBlock Origin with Tor Browser.[4]
14. 2 RT tickets: fingerprintability of Smooth Scroll in Tor Browser.[5]
15. Letterboxing is visible even if disabled when using Tor Browser
on Tiled window managers.[6]
16. One report of Tor Browser getting flagged by anti-virus software.
17. Static captcha when fetching bridges from within Tor Browser.[7]
18. What does "onionize" toggle in the search bar on about:tor do?
19. Tor circuit display on Tor Browser for Android.[8]
20. User trying to reach a v2 onion service. v2 onion services have been deprecated.
21. Question about using WebTunnel bridges with Tails. (This is not yet supported) [9]
22. Warning prompt to disable NoScript on resource-heavy websites on Tor Browser Desktop.[10]
# Telegram, WhatsApp and Signal Support channel
* 1548(↑) tickets resolved
Breakdown:
* 1532(↑) tickets on Telegram
* 16(↓) tickets on WhatsApp
* 0(-) ticket on Signal
Tickets by topics and numbers:
1. 1201(↑) tickets: circumventing censorship in Russian speaking
countries.
2. 42(↑) tickets: private bridge requests from Chinese
speaking users.
3. 26(↑) tickets: circumventing censorship with Tor in Farsi.
4. 18(↓) tickets: helping users on iOS, using Onion Browser or
Orbot, to use censorship circumvention methods.
5. 13(↑) tickets: Tor Browser 14 crashing on macOS when visiting some onionsites.
6. 13(↑) tickets: instructions on how to get Tor Browser binaries from GetTor.
7. 9(↑) tickets: help with instructions to use bridges with Tails.
8. 7(↓) tickets: help with troubleshooting Tor Browser on Android.
9. 4(-) tickets: help with using bridges with Orbot.
10. 4(↑) tickets: instructions to download Tor Browser 13.5 legacy for legacy
operating systems.[11]
11. 3(↑) tickets: help with using bridges and snowflake with little-t-tor.
12. 3(↑) tickets: help with instructions to verify Tor Browser signature with GPG.
13. 2(↓) tickets: help with troubleshooting Tor Browser Desktop
on Windows, macOS and Linux.
14. 2(↓) tickets: help with installing Tor Browser on linux.
15. 2(↓) tickets: users seeing a "proxy refused" error when visiting websites on
Tor Browser for Android using Samsung devices.[12]
16. 2(↑) tickets: reports of a fake apps on iOS masquerading as official Tor Browser.
17. 1 ticket: questions about accessing or setting up onion services.
18. 1 ticket: help with setting up Snowflake proxy.
# Discussions from the Tor Forum
1. Tor Expert Bundle on legacy Windows.[13]
2. Tor Browser Desktop and browser fingerprinting.[14]
3. Privacy benefits of running an Onion Service.[15]
Note: (↑), (↓) and (-) are indicating if the number of tickets we
received for these topics have been increasing, decreasing or have been
the same from the previous month respectively.
Thanks everyone!
e.
[0]: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss…
[1]: https://lists.torproject.org/mailman3/hyperkitty/list/tor-project@lists.tor…
[2]: https://gitlab.torproject.org/tpo/web/manual/-/commits/main?author=ebanam
[3]: https://blog.torproject.org/call-for-webtunnel-bridges/
[4]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/17569
[5]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40704
[6]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42670
[7]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42086
[8]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41234
[9]: https://gitlab.tails.boum.org/tails/tails/-/issues/20267
[10]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43286
[11]: https://forum.torproject.org/t/download-tor-browser-13-5-legacy-on-windows-…
[12]: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42714
[13]: https://forum.torproject.org/t/0-4-8-13-tor-expert-bundle-doesnt-work-on-wi…
[14]: https://forum.torproject.org/t/what-information-about-the-computer-on-which…
[15]: https://forum.torproject.org/t/is-onion-site-any-better-if-a-site-also-prov…
Hey everyone!
Here are our meeting logs:
http://meetbot.debian.net/tor-meeting/2024/tor-meeting.2024-12-05-16.01.html
And our meeting pad:
Anti-censorship work meeting pad
--------------------------------
Anti-censorship
--------------------------------
Next meeting: Thursday, December 12 16:00 UTC
Facilitator: shelikhoo
^^^(See Facilitator Queue at tail)
Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC
(channel is logged while meetings are in progress)
This week's Facilitator: onyinyang
== Goal of this meeting ==
Weekly check-in about the status of anti-censorship work at Tor.
Coordinate collaboration between people/teams on anti-censorship at the
Tor Project and Tor community.
== Links to Useful documents ==
* Our anti-censorship roadmap:
*
Roadmap:https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards
* The anti-censorship team's wiki page:
*
https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home
* Past meeting notes can be found at:
* https://lists.torproject.org/pipermail/tor-project/
* Tickets that need reviews: from projects, we are working on:
* All needs review tickets:
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s…
* Project 158 <-- meskio working on it
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues/?label_na…
== Announcements ==
== Discussion ==
* Creating container mirror for anti-cenosorship projects to deal
with docker hub restriction
*
https://gitlab.torproject.org/tpo/tpa/team/-/issues/41914#note_3138851
*
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
* we'll create a repo in gitlab that mirrors the needed images
by it's CI
* WIP MR: Add covert-dtls to proxy and client
*
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
* is a big change, shelikhoo did review it but more eyes are needed
* cohosh will look into it
* Reasonable PollInterval for Orbot?
https://github.com/tladesignz/IPtProxy/pull/58
* standalone proxy does 5seconds poll interval
* orbot was doing the same but being mobile it might be better
to be slower
* 120sec sounds like a good number seeing that there are
already a lot of proxies
== Actions ==
== Interesting links ==
* call out for webtunnel bridges for Russia:
* https://blog.torproject.org/call-for-webtunnel-bridges/
* Deprecation BridgeDB blogpost:
*
https://blog.torproject.org/making-connections-from-bridgedb-to-rdsys/
== Reading group ==
* We will discuss "" on
*
* Questions to ask and goals to have:
* What aspects of the paper are questionable?
* Are there immediate actions we can take based on this work?
* Are there long-term actions we can take based on this work?
* Is there future work that we want to call out in hopes
that others will pick it up?
== Updates ==
Name:
This week:
- What you worked on this week.
Next week:
- What you are planning to work on next week.
Help with:
- Something you need help with.
cecylia (cohosh): 2024-12-05
Last week:
- fixed the logging of the new proxy event (snowflake#40413)
- fixed "bad offer from broker" log spam (snowflake#40408)
- started looking at alerts for censorship events (snowflake#40416)
- updated the RFC 5780 compatible STUN servers in bridge lines
(snowflake#40304)
- deployed new Snowflake bridge lines
(tor-browser-build!1115, rdsys-admin!32)
- worked on analysis of snowflake metrics (snowflake#40394)
- created wiki page for snowflake release procedure
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
This week:
- work on snowflake broker metrics improvements
- write a script to easily test STUN servers in snowflake's torrc
- finish snowflake dependency upgrades that were causing problems
- take a look at snowflake web and webext translations and best
practices
- make changes to Lox encrypted bridge table
-
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/merge_requests/147
Needs help with:
- what was that censorship alerts mailing list?
dcf: 2024-11-21
Last week:
- released goptlib v1.6.0
https://lists.torproject.org/mailman3/hyperkitty/list/anti-censorship-team@…
Next week:
- comment on updates to unreliable snowflake transport
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to have snowflake-client log whenever KCPInErrors
is nonzero
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- parent:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to disable /debug endpoint on snowflake broker
Help with:
meskio: 2024-12-05
Last week:
- rdsys stopped accepting new bridges (rdsys#249)
- deploy onionsproutsbot distributing TorBrowser for win7
(onionsproutsbot#64)
- support webtunnel bridge operators
- prepare splintercon presentations
- more grant writting life
- review BridgeDB deprecation blogpost
Next week:
- AFK at splintercon
Shelikhoo: 2024-12-05
Last Week:
- [Pending] snowflake broker update/reinstall(cont.):
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- [Awaiting Review] Unreliable+unordered WebRTC data channel
transport for Snowflake rev2 (cont.)(
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
) improvements
- Merge request reviews
- Work on finishing snowflake container release(and fix the
comments)
- Incorrectly flattened container image with "pull" command
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
Next Week/TODO:
- Merge request reviews
- Incorrectly flattened container image with "pull" command
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
(cont.)
- Create mirror for frequently used container image
onyinyang: 2024-12-05
Last week(s):
- working on refactor of Lox (library) protocols to improve
issuing efficiency as described in: https://eprint.iacr.org/2024/1552.pdf
- finished aside from check-blockage and trust promotion
protocols which may not be convertable
- Released lox-library and lox-utils 0.2.0 for browser integration
- stablized some dependencies for integration into firefox
with XPCOM
- created bridgeauth feature to separate client/server lox
functionality
- Improved pipeline for lox workspace
- Added lox-context db cleanup
Next week:
- Fix up Troll-patrol MR
- Deploy test distributor
- update lox protocols to return duplicate responses for an
already seen request
- Continue work on implementing issuer efficiency for
check-blockage and trust-promotion protocols
- Work on outstanding milestone issues:
in particular:
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/issues/69
- key rotation automation
Later:
pending decision on abandoning lox wasm in favour of some kind
of FFI?
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43096):
- add pref to handle timing for pubkey checks in Tor browser
- add trusted invitation logic to tor browser integration:
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42974
- improve metrics collection/think about how to show Lox is
working/valuable
- sketch out Lox blog post/usage notes for forum
(long term things were discussed at the meeting!):
- brainstorming grouping strategies for Lox buckets (of
bridges) and gathering context on how types of bridges are
distributed/use in practice
Question: What makes a bridge usable for a given user, and
how can we encode that to best ensure we're getting the most appropriate
resources to people?
1. Are there some obvious grouping strategies that we
can already consider?
e.g., by PT, by bandwidth (lower bandwidth bridges
sacrificed to open-invitation buckets?), by locale (to be matched with a
requesting user's geoip or something?)
2. Does it make sense to group 3 bridges/bucket, so
trusted users have access to 3 bridges (and untrusted users have access
to 1)? More? Less?
theodorsm: 2024-12-05
Last weeks:
- Adjusting to post-student life
- WIP MR: Add covert-dtls to proxy and client
-
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
Next weeks:
- Test Snowflake fork with covert-dtls
- Condensing thesis into paper (on hold)
Help with:
- Test covert-dtls in Snowflake
Facilitator Queue:
onyinyang meskio shelikhoo
1. First available staff in the Facilitator Queue will be the
facilitator for the meeting
--
---
onyinyang
GPG Fingerprint 3CC3 F8CC E9D0 A92F A108 38EF 156A 6435 430C 2036
Hello,
Tor User Support team will be partly unavailable between December 19th,
2024 and January 5th, 2025.
If you reach out to us on our user support channels via email[0], Tor
Forum[1], Telegram[2], WhatsApp[3] or Signal[4], it may take longer than
usual for us to respond. Please rest assured we will get back to your
messages as soon as possible.
Our team will be back at our regular office hours[5] from January 6th,
2025.
Thank you!
Community Team
Tor Project
[0]: frontdesk(a)torproject.org
[1]: https://forum.torproject.org
[2]: https://t.me/TorProjectSupportBot
[3]: https://wa.me/447421000612
[4]: https://signal.me/#p/+17787431312
[5]: https://tb-manual.torproject.org/support/
Hi! Below is my November’24 report!
In November, I resolved 1569 (↑465) tickets:
* On Telegram (@TorProjectSupportBot) - 1351 (↑496);
* On RT (frontdesk@tpo) - 215 (↓33);
* On WhatsApp (+447421000612) - 3 (↑2);
* and on Signal (+17787431312) - 0 (0).
Since August 2024, I've noticed a significant increase in
Russian-speaking user requests on our support channels:
* July: 587 tickets
* August: 928 tickets
* September: 994 tickets
* October: 1104 tickets
* November: 1569 tickets (this report)
My main focus in November was to help Russian-speaking users bypass
censorship and help users to resolve any issues they are facing with Tor
Browser. I also took part in Tor Forum moderation and worked
reviewingGoogle PlayStore users reviews.
In November,we had much more tickets from Russian-speaking users than
usual (+42% comparing with October). It happened due to internet
censorship in Russia is getting more severe, and some pluggable
transports are being blocked or partially blocked[1]. So in November,we
launched a WebTunnelcampaign and asked the Tor community to help users
in Russia by running more bridges [2], which is a pluggable transport
reported to work well in Russia.
Last monthwe got multiple requests from Apple devices users from Russia
regarding what app to use on iOS or how to downloadTor Browser - macOS
does not allow to install apps downloaded from Telegram [3], so users
need to use website mirror.
The process of upgrading CDR.link has started in November[4],which
included upgrade to a newer Zammad version, testing, bug reporting and
consultation with the contractor.
*##**Google Play Reviews for Tor Browser**(TBA)**and Tor Browser
Alpha**for Android*
Tor Browser for Androidhad a Google Play rating of 4.89 (↓) stars in
November 2024, which is lower than in October.
Tor Browser for Android (TBA) got 672 (↓19) reviews out of 59,115 for
the lifetime.
Tor Browser for Android Alpha (TBA-Alpha) app had a rating of 4.241 (↓)
which is lower than in October.
In November, Tor Browser for Android (TBA-Alpha) got 38 (↓2) reviews out
of 8,396 for the lifetime.
Most common issues on the Google Play store reviews:
* Tor Browser doesn’t work: mostly from Russian users, who are
struggling to find a bridge that is not blocked;
* Tor speed is too slow:This issue is currently under investigation,
I'lloffer users some ways to improve the speed and gather their
feedback.
[1]
https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss…
[2] https://blog.torproject.org/call-for-webtunnel-bridges/
[3]
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/4…
[4] https://gitlab.torproject.org/tpo/community/support/-/issues/40165