Hi,
So it's the most wonderful time of the year and everything... This means
that we're actually coming pretty close to the holidays!
TL;DR: everything will be fine, but if you need something from us before
the holidays, ask now!
TPA is, as usual, going to ensure some minimal rotation during the TPI
holidays (December 21st to January 5th, inclusively), so we should be
able to deal with emergencies. We'll have someone checking alerts and
mail on a daily basis, basically. "How to report issues" doesn't change,
see:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/support
If you're curious, you can see who's on rotation in the "TPA rotations"
calendar I have just shared with TPI (an oversight).
That said.
If you think you might need something urgently before the holidays, it's
getting *really* late to tell us now, as we're pretty much trying to
wrap up everything to quiet down our infra before the holidays.
So if you need anything from TPA, the time to ask for this is pretty
much right now, we'll be glad to consider your request! And as the ~3
weeks pass before the holidays, we'll be less and less happy to deal
with urgent "oh I just need this blog post" or "can I get a little VM
cluster setup now?" right before the holidays. :)
(If you need a coordinated release right before the holidays, that's
fine too, but let's plan it no!)
Thank you for your attention, and happy holidays!
a.
--
Antoine Beaupré
torproject.org system administration
Summary: a proposal to limit the retention of GitLab CI data to 1 year
# Background
As more and more Tor projects moved to GitLab and embraced its
continuous integration features, managing the ensuing storage
requirements has been a challenge.
We regularly deal with near filesystem saturation incidents on the
GitLab server, especially involving CI artifact storage, such as
tpo/tpa/team#41402 and recently, tpo/tpa/team#41861
Previously, [TPA-RFC-14][] was implemented to reduce the default
artifact retention period from 30 to 14 days. This, and CI optimization
of individual projects has provided relief, but the long-term issue has
not been definitively addressed since the retention period doesn't apply
to some artifacts such as job logs, which are kept indefinitely by default.
[TPA-RFC-14]:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/tpa-rfc-14-gitlab-artifa…
# Proposal
Implement a daily GitLab maintenance task to delete CI pipelines older
than 1 year in *all* projects hosted on our instance. This will:
* Purge old CI pipeline and job records for the GitLab database
* Delete associated CI job artifacts, even those "kept" either:
* When [manually prevented from expiring][] ("Keep" button on CI job
pages)
* When they're the [latest successful pipeline artifact][]
* Delete old CI job log artifacts
[manually prevented from expiring]:
https://gitlab.torproject.org/help/ci/jobs/job_artifacts#with-an-expiry
[latest successful pipeline artifact]:
https://gitlab.torproject.org/help/ci/jobs/job_artifacts.md#keep-artifacts-…
## Goals
This is expected to significantly reduce the growth rate of CI-related
storage usage, and of the GitLab service in general.
## Affected users
All users of GitLab CI will be impacted by this change.
But more specifically, some projects have "kept" artifacts, which were
manually set not to expire. We'll ensure the concerned users and
projects will be notified of this proposal. GitLab's documentation has
the [instructions to extract this list of non-expiring
artifacts](https://docs.gitlab.com/ee/administration/cicd/job_artifacts_tro….
## Timeline
Barring the need to further discussion, this will be implemented on
Monday, December 9th.
## Costs estimates
### Hardware
This is expected to reduce future requirements in terms of storage hardware.
### Staff
This will reduce the amount of TPA labor needed to deal with filesystem
saturation incidents.
# Alternatives considered
A "CI housekeeping" script is already in place, which scrubs job logs
daily in a hard-coded list of key projects such as c-tor packaging,
which runs an elaborate CI pipeline on a daily basis, and triage-bot,
which runs it CI pipeline on a schedule, every 15 minutes.
Although it has helped up until now, this approach is not able to deal
with the increasing use of personal fork projects which are used for
development.
It's possible to define a different retention policy based on a
project's namespace. For example, projects under the `tpo` namespace
could have a longer retention period, while others (personal projects)
could have a shorter one. This isn't part of the proposal currently as
it could violate the principle of least surprise.
# References
* Discussion ticket: tpo/tpa/team#41874
* [Make It Ephemeral: Software Should Decay and Lose
Data](https://lucumr.pocoo.org/2024/10/30/make-it-ephemeral/)
Hello,
I'm writing to let you know that applications are now open for the
second SEEKCommons Fellowship[1] cohort.
The SEEKCommons Fellowship program is funded by NSF and run by partners
at University of Notre Dame, University of California Davis, University
of Michigan, University of Virginia, and The HDF Group. The goal of the
fellowship is to bring graduate students, post-doctoral researchers, and
professionals from community-based organizations with new perspectives
to socio-environmental research with open technologies.
Application deadline is Dec. 15, 2024.
The fellowship is designed to:
- Encourage new translational and integrative work involving
socio-environmental action research with Open Science practices;
- Provide a space for Fellows and Network members to collaborate on
common research issues, challenges, and solutions.
Fellows may be:
- Graduate students working with open technologies on
socio-environmental issues;
- Post-docs with existing community projects in Science and Technology
Studies, Open Science, and/or Socio-environmental research; and/or
- Community practitioners who are interested in integrating common
technologies into their environmental justice work.
The SEEKCommons website contains all the necessary fellowship
information, including the application link.
https://seekcommons.org/fellowship-application.html
Partnership SEEKCommons + Tor Project
=====================================
This year SEEKCommons is reserving one fellowship to sustainability
studies of community networks in partnership with the Tor Project!
We welcome Fellowship applications on the sustainability of the Tor
network with a focus on energy consumption and relay metadata (such as
ASN, uptime, and platform). The goal of the partnership between
SEEKCommons and Tor is to study the environmental impact of community
networks and promote the use of renewable energy in decentralized
infrastructures.
We would like to support applications that address one (or more) of these questions:
- What Free and Open Source technologies can be used to promote a more
sustainable and distributed Tor infrastructure?
- How can the energy consumption of the Tor network be measured and
optimized to reduce its environmental impact?
- How can the "Tor Snowflake" decentralized proxy model be used to
improve the sustainability of the network?
- How can we use metadata from relays (e.g., ASN / uptime / platform)
to assess the environmental impact of the network?
- What open hardware and renewable energy sources could be used/reused
in the Tor network?
More information: https://seekcommons.org/partnership-tor.html
If you have any questions, please don't hesitate to reach out.
Thank you so much!
Warmly,
Gus
[1] https://seekcommons.org/about.html
--
The Tor Project
Community Team Lead
Hello,
Additionally to the mail server switch on Monday, we're planning to
move/migrate the gitlab server from our Falkenstein-based cluster to the
one in Dallas.
This move will be happening on Thursday november 28th starting at 18:00 UTC.
The disks for the gitlab server are fairly sizeable so the transfer
could be taking a long time.
Our current estimates are that it could take about 18 hours to transfer.
We will try launching an operation to zero out empty parts of the disk
volumes before starting the transfer in the hopes that it could give the
transfer a boost.
So the transfer will most likely take gitlab offline until the next day,
friday 29th, at which time we'll make sure that the service is coming
back online properly.
If there's any issue during the outage, please reach out to us via IRC
or email, but not through gitlab issues since this last one will
obviously be offline at that time.
For status updates, see:
https://status.torproject.org/issues/2024-11-28-gitlab-migration/
Cheers!
Hey everyone!
Here are our meeting logs:
http://meetbot.debian.net/tor-meeting/2024/tor-meeting.2024-11-28-16.00.html
And our meeting pad:
Anti-censorship work meeting pad
--------------------------------
Anti-censorship
--------------------------------
Next meeting: Thursday, December 05 16:00 UTC
Facilitator: onyinyang
^^^(See Facilitator Queue at tail)
Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC
(channel is logged while meetings are in progress)
This week's Facilitator: shelikhoo
== Goal of this meeting ==
Weekly check-in about the status of anti-censorship work at Tor.
Coordinate collaboration between people/teams on anti-censorship at the
Tor Project and Tor community.
== Links to Useful documents ==
* Our anti-censorship roadmap:
*
Roadmap:https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards
* The anti-censorship team's wiki page:
*
https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home
* Past meeting notes can be found at:
* https://lists.torproject.org/pipermail/tor-project/
* Tickets that need reviews: from projects, we are working on:
* All needs review tickets:
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s…
* Project 158 <-- meskio working on it
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues/?label_na…
== Announcements ==
== Discussion ==
(old)
* Snowflake blocking in Russia November 2024
*
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
* No apparent difference in DTLS handshakes between proxies
that continue working and proxies that stop working.
* Reports say that some proxies stop working, but only after
transferring some traffic.
* Hard to test because we can't easily control the proxy. It
would be a good idea to set up a small test deployment to afford us more
experimental control.
* cohosh will work on that.
* Traffic analysis fingerprinting? An old padding patch:
https://github.com/net4people/bbs/issues/255#issuecomment-1566227484\
(New Nov 21)
* Snowflake blocking in Russia November 2024
* Restored copy-paste signaling (manual without broker) to be
able to control the proxy connection.
*
https://gitlab.torproject.org/cohosh/snowflake/-/commit/a4a574a4584332fc282…
* Still blocked with a new proxy: therefore likely some kind of
protocol fingerprinting, not proxy enumeration.
*
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
* Still blocked with client-side padding patch as well.
*
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
* theodorsm's covertDTLS fork
https://gitlab.torproject.org/theodorsm/snowflake/-/tree/covert-dtls-test
* Snowflake broker transition
* Next monday 2024-11-25 there will be a DNS switch from the
current broker to the new broker primary.
* https://gitlab.torproject.org/tpo/tpa/team/-/issues/41878
(new Nov 28)
* Broker Transition Anomaly
* (added by non-Tor member WofWca): Evaluate [the plan for
"Multi-Pool Matching
Support"](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40156#note_3132445)?
* Snowflake blocking in Russia seems to have stopped
== Actions ==
== Interesting links ==
*
== Reading group ==
* We will discuss "" on
*
* Questions to ask and goals to have:
* What aspects of the paper are questionable?
* Are there immediate actions we can take based on this work?
* Are there long-term actions we can take based on this work?
* Is there future work that we want to call out in hopes
that others will pick it up?
== Updates ==
Name:
This week:
- What you worked on this week.
Next week:
- What you are planning to work on next week.
Help with:
- Something you need help with.
cecylia (cohosh): 2024-11-28
Last week:
- followed up on the snowflake blocking in Russia
- helped debug and recover from the snowflake broker upgrade
- caught up a bit on reviews
- fixed the logging of the new proxy event (snowflake#40413)
- started looking at alerts for censorship events (snowflake#40416)
This week:
- alerts for censorship events (snowflake#40416)
- keep looking at snowflake blocking in Russia
- take care of review backlog
- finish snowflake dependency upgrades that were causing problems
- take a look at snowflake web and webext translations and best
practices
- make changes to Lox encrypted bridge table
-
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/merge_requests/147
Needs help with:
- what was that censorship events mailing list?
dcf: 2024-11-21
Last week:
- released goptlib v1.6.0
https://lists.torproject.org/mailman3/hyperkitty/list/anti-censorship-team@…
Next week:
- comment on updates to unreliable snowflake transport
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to have snowflake-client log whenever KCPInErrors
is nonzero
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- parent:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to disable /debug endpoint on snowflake broker
Help with:
meskio: 2024-11-28
Last week:
- investigate why the metrics of the new broker were not
arriving to prometheus (tpa/team#41902)
- try to upgrade snowflake gitlab CI to debian stable and fail
(for now)
- multiple grant writting work (logic model, reviews, ...)
- reports for grants fun
- WIP: update snowflake proxy debian package (snowflake#40414)
Next week:
- update snowflake proxy debian package (snowflake#40414)
Shelikhoo: 2024-11-28
Last Week:
- [Pending] snowflake broker update/reinstall(cont.):
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- [Awaiting Review] Unreliable+unordered WebRTC data channel
transport for Snowflake rev2 (cont.)(
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
) improvements
- Vantage point deployment in Iran
- Snowflake new broker deployment
- Merge request reviews
Next Week/TODO:
- Merge request reviews
- Work on finishing snowflake container release(and fix the
comments)
- Incorrectly flattened container image with "pull" command
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
onyinyang: 2024-11-21
Last week(s):
- working on refactor of Lox (library) protocols to improve
issuing efficiency as described in: https://eprint.iacr.org/2024/1552.pdf
- WIP MR:
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/merge_requests/284
- Adjusted dependencies to line up with non-wasm browser
integration for:
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43096):
-
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/merge_requests/283
- Updating renovate bot to ignore these dependencies (and
maybe others)
- Fixed up wasm changes in
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/merge_requests/271
Next week:
- Fix up Troll-patrol MR
- Deploy test distributor
- Continue work on improving issuer efficiency in Lox protocols
- update lox protocols to return duplicate responses for an
already seen request
- Work on outstanding milestone issues:
in particular:
https://gitlab.torproject.org/tpo/anti-censorship/lox/-/issues/69
- key rotation automation
Later:
pending decision on abandoning lox wasm in favour of some kind
of FFI?
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43096):
- add pref to handle timing for pubkey checks in Tor browser
- add trusted invitation logic to tor browser integration:
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42974
- improve metrics collection/think about how to show Lox is
working/valuable
- sketch out Lox blog post/usage notes for forum
(long term things were discussed at the meeting!):
- brainstorming grouping strategies for Lox buckets (of
bridges) and gathering context on how types of bridges are
distributed/use in practice
Question: What makes a bridge usable for a given user, and
how can we encode that to best ensure we're getting the most appropriate
resources to people?
1. Are there some obvious grouping strategies that we
can already consider?
e.g., by PT, by bandwidth (lower bandwidth bridges
sacrificed to open-invitation buckets?), by locale (to be matched with a
requesting user's geoip or something?)
2. Does it make sense to group 3 bridges/bucket, so
trusted users have access to 3 bridges (and untrusted users have access
to 1)? More? Less?
theodorsm: 2024-10-24
Last weeks:
- Adjusting to post-student life
- Testing out beta releases of pion dtls and webrtc
Next weeks:
- Update Snowflake to use latest pion upstream releases
- Test Snowflake fork with covert-dtls
- Condensing thesis into paper
Help with:
- Feedback on thesis
Facilitator Queue:
onyinyang meskio shelikhoo
1. First available staff in the Facilitator Queue will be the
facilitator for the meeting
2. After facilitating the meeting, the facilitator will be moved to the
tail of the queue
Hello all,
I'm sending a reminder about this since the last communication about it
was more than a month ago.
Hetzner is planning to perform maintenance on their networking equipment
next week, on December 3rd between 3:30 and 5:30 UTC. This is something
that the TPA team does not have control over.
We can expect some network outage for all hosts at Hetzner during that
time (which is just a handful of our servers). None of our hosts will
reboot during that time, they will just momentarily lose network
connectivity.
More details can be seen on status.torproject.org and the maintenance
announcement was already planned as soon as we got the notice from
Hetzner. The announcement lists the services which can be expected to
have an outage:
https://status.torproject.org/issues/2024-12-03-hetzner-router-maintenance/
Thanks all for your understanding,
LeLutin for TPA
Hey everyone,
On November 25st 2024, starting at 11:00 UTC, we will be switching to new
mail servers for receiving and forwarding mail sent to torproject.org. If all
goes well, there will be no downtime and deliverability to external
providers like gmail will improve.
If all does not go well, please reach out on IRC or file an issue on Gitlab,
do not mail TPA since e-mail may obviously not be reliable. For more details on
how to get in touch and how to report e-mail problems, see:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/support
For status updates, see: https://status.torproject.org/issues/2024-11-25-switching-mail-servers/
Fingers crossed :)
x,
groente