Hi! Below is my Setember’23 report!
In September, I resolved 615 tickets:
On Telegram (@TorProjectSupportBot) - 367
On RT (frontdesk@tpo) - 236
On WhatsApp (+447421000612) - 9
and on Signal (+17787431312) - 3.
During that month I -
1. Helped Russian-speaking users to bypass censorship: shared bridges
and assisted with using them and troubleshooting;
2. Collected user feedback;
3. Helped to solve Tor Browser issues like:
- antivirussoftware blocking Tor Browser[1]
- bug"Pluggable Transport process terminated with status code 0"[2],
which is fixed in 0.4.8.x, but Tor Browser stable still use 0.4.7.x Tor
version.
The new attempts of the Russian government to block popular VPN
protocols resulted in many users trying to use little-t-tor to
proxytheir connection and circumvent censorship, so I helped
themtoconfigure it.
In September we also answeredmany tickets related to the domain
frontingissue [3].
I also helped the Tor Localization team reviewing some Russian terms
andtranslations for the Tor Browser and the Tor Project website.
[1]
https://forum.torproject.org/t/torbrowser-12-5-6-no-longer-flagged-by-windo…
[2] https://gitlab.torproject.org/tpo/core/tor/-/issues/33669
[3]
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42120
Hi all :)
This is my monthly status report for September 2023 with the main relevant
activities I have done during the period.
## 0. Research
### Certificates
The work for bringing TLS certificates for Onion Services was focused in the
ACME for Onions proposal (https://acmeforonions.org).
There were a series of relevant updates both on IETF ACME and on the
CA/B Forum's Validation working groups:
* https://lists.cabforum.org/pipermail/validation/2023-September/001927.html
* https://magicalcodewit.ch/cabf-2023-09-07-slides/
* https://mailarchive.ietf.org/arch/msg/acme/LMYC_Ou41E_9RuaVSYPr7SIhCCc/
* https://github.com/AS207960/acme-onion/issues/2
I focused in:
* Helping to figure ways that CAA and .onion descriptors could be handled by
ACME client and servers. I'm still compiling the list of options for an
ACME server to parse and validate an Onion Service descriptor.
* Doing a documentation update about CAA checking:
https://tpo.pages.torproject.net/onion-services/onionplan/appendixes/acme/#…https://gitlab.torproject.org/tpo/onion-services/onionplan/-/commit/0234173…
## Tor Browser Quality Assurance for Onion Services (TBB .onion QA)
I have completed the first three quarters of Tor Browser QA testing (since
2023.Q1).
### Testbed
* Since this QA process started, it's methodology and tooling was bootstrapped
and improved.
* Some basic tests were defined to happen at every Tor Browser release (when
applicable).
* Additional, specific tests were also defined to check for specific and
potential issues.
* The "Faulty Onions" project was prototyped, and is intended to provide test
Onion Services with different errors to check how Tor Browser and other
applications handles them. More details to be expected soon.
* A few alternatives for test automation were researched, to consider
whether some of the regular tests can be automated.
* Public documentation remains yet to be done.
### Versions tested
Eleven Tor Browsers versions were formally tested:
* 12.5.1
* 12.5.2
* 12.5.3
* 12.5.4
* 12.5.5
* 12.5.6
* 13.0a1
* 13.0a2
* 13.0a3
* 13.0a4
* 13.0a5
## 1. Development
### Onionprobe
* Onionprobe 1.1.2 was released:
https://gitlab.torproject.org/tpo/onion-services/onionprobe/-/blob/main/Cha…
## 2. Support
### Documentation Hackweek
As a preparation for the upcoming [Hackweek][], I have submitted four project
proposals:
* Onion MkDocs tryout:
https://gitlab.torproject.org/tpo/community/hackweek/-/issues/13
* Onion TeX Slim enhancements:
https://gitlab.torproject.org/tpo/community/hackweek/-/issues/14
* Onion Reveal coding and documenting:
https://gitlab.torproject.org/tpo/community/hackweek/-/issues/15
* Etherpad management:
https://gitlab.torproject.org/tpo/community/hackweek/-/issues/16
I'm planning to work in just one of these projects, depending in which one is
more popular or gets more attention. I'm also looking for people that wants to
form a team, or even adopt one of these proposals.
Please leave a comment, subscribe yourself or add your user name into the
ticket description if you're interested :)
[Hackeek][]: https://lists.torproject.org/pipermail/tor-project/2023-August/003675.html
### Maintenance
* I also did the ongoing sponsored work with deployment, maintenance and
monitoring of Onion Services.
## 3. Organization
Time spent (from the total available for Tor-related work):
| Category | Percentage
|---------------|------------
| Research | 57
| Development | 1
| Support | 9
| Organization | 33
|---------------|------------
| Total | 100
--
Silvio Rhatto
pronouns he/him
Ahoy!
Well it's just been way too long since we've done this, but we've done
it! Here's our short sysadmin minutes from today's meeting.
# Roll call: who's there and emergencies
onionoo-backend running out of disk space ([tpo/tpa/team#41343][])
[tpo/tpa/team#41343]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41343
# Dashboard cleanup
Normal per-user check-in:
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
General dashboards:
* https://gitlab.torproject.org/tpo/tpa/team/-/boards/117
* https://gitlab.torproject.org/groups/tpo/web/-/boards
* https://gitlab.torproject.org/groups/tpo/tpa/-/boards
Nextcloud roadmap / spreadsheet.
Overall, it seems we are as you would expect when returning from a
rather chaotic vacation. Backlog is large, but things seem to be under
control.
We added SVN back on the roadmap after one too many tickets asking for
setup.
# Metrics of the month
* hosts in Puppet: 89, LDAP: 89, Prometheus exporters: 166
* number of Apache servers monitored: 37, hits per second: 626
* number of self-hosted nameservers: 6, mail servers: 10
* pending upgrades: 1, reboots: 0
* average load: 0.69, memory available: 3.58 TiB/4.98 TiB, running processes: 424
* disk free/total: 53.19 TiB/126.72 TiB
* bytes sent: 403.47 MB/s, received: 269.04 MB/s
* planned bullseye upgrades completion date: 2024-08-02
* [GitLab tickets][]: 196 tickets including...
* open: 0
* icebox: 163
* needs information: 5
* backlog: 13
* next: 9
* doing: 4
* needs review: 2
* (closed: 3301)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
Upgrade prediction graph lives at:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/bookworm/
Now also available as the main Grafana dashboard. Head to
<https://grafana.torproject.org/>, change the time period to 30 days,
and wait a while for results to render.
# Number of the month: 42
34 machines were upgraded from bullseye to bookworm in the two first
days of last week! We calculated this was an average of 20 minutes per
host to upgrade.
The trick, of course, is that things often break *after* the upgrade,
and that "fixing" time is not counted here. That said, last estimate
for this was one hour per machine, and we're doing a whole fleet
upgrade every 2-3 years, which means about ten hours of work saved per
year.
But the number of the month is, of course, 42, as we now have an equal
number of bookworm and bullseye machine, after the upgrade. And that
number is, naturally, [42][].
See also https://xkcd.com/1205/ which, interestingly, we fall out of
scope of.
[42]: https://en.wikipedia.org/wiki/42
--
Antoine Beaupré
torproject.org system administration
Hi everyone!
Here is my status report for September 2023.
This month, I continued working on the refactoring of the Tor
integration in Tor Browser and finally merged the merge requested with
the biggest changes [0]. During the month, I also fixed some bugs it
caused and improved the compatibility with the external Tor daemons.
Then, I worked on some audits for the 102 to 115 transition. In
particular, I reviewed our profiles [1].
Some other tasks included swapping a few branding assets, such as the
About dialog wordmark and the Windows installer icons.
I also fixed the reproducibility bug with generated headers on Windows
[2] and upstreamed the patch to Firefox.
During September, we had two emergency releases. I helped with the
stable channel: I prepared and built both 12.5.4 and 12.5.6. For 13.0a4,
I only rebased it on top of Firefox 115.2.1.
Finally, I worked on enabling system-wide installs for Mullvad Browser.
However, it's a work in progress as it requires some non-trivial changes
to the updater.
Best,
Pier
[0]
https://gitlab.torproject.org/tpo/applications/tor-browser/-/merge_requests…
[1]
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41496
[2]
https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41995
Mirrored from <https://github.com/shadow/shadow/discussions/3187>:
This is part of a series of periodic updates of development in Shadow.
This work is sponsored by the
[NSF](https://shadow.github.io/docs/guide/nsf_sponsorship.html).
Previous update:
[2023-06](https://github.com/shadow/shadow/discussions/3061).
We've merged [74 non-dependabot pull
requests](https://github.com/shadow/shadow/pulls?q=is%3Apr+merged%3A2023-06…
and closed [8
issues](https://github.com/shadow/shadow/issues?q=closed%3A2023-06-30..2023…
since our previous update.
# Latest improvements
## Spawning processes
Shadow now supports running programs that spawn additional processes.
Technically, this means it now supports `fork`, `vfork`, `execve`, and
other related syscalls. Some uses for this:
* The primary way for `tor` to work with [pluggable
transports](https://blog.torproject.org/tor-heart-bridges-and-pluggable-tra…
is for the `tor` process to dynamically spawn the pluggable transport
process. This is currently the *only* way that pluggable transports are
supported in `arti`. This now works in `shadow`.
* Orchestration of multiple processes on a single host in `shadow` can
now be done using e.g. `sh` or `python` scripts, instead of having to
specify every process directly in `shadow`'s configuration file.
* `shadow` can now run other software that spawns worker or helper
processes.
Another exciting application for this in `shadow` *development* is that
we can now more easily run third party test suites, which typically
spawn multiple test processes. For example, running `tor`'s own
self-tests helps validate that `shadow` is correctly emulating the
platform functionality that `tor` uses; currently [more than 99% of the
tests pass](https://github.com/shadow/shadow/issues/3168)!
## New TCP stack
Shadow's experimental new TCP stack is merged, and can be enabled with
the experimental command-line flag `--use-new-tcp`. This new
implementation is in Rust instead of C, and is developed to be very
testable (e.g. in its own crate decoupled from the rest of Shadow, and
with pluggable system dependencies). While some functionality is still
being finished, we expect it will be much easier to maintain and to
validate its correctness than the previous C implementation. Improved
support for pcap file output also makes it easier to review Shadow's
simulated network traffic.
## Shim stability
We've occasionally run into problems due to `shadow`'s preloaded shim
calling into `libc`. This is unsafe because some initialization can run
before `libc` itself is fully initialized, and much of the code runs in
a seccomp signal handler, running afoul of
[async-signal-safety](https://man7.org/linux/man-pages/man7/signal-safety.7.….
Luckily, Rust has a rich ecosystem of code that doesn't depend on libc
(`no_std` code). We've made substantial progress in migrating the shim's
C code to `no_std` Rust code.
## Socket API improvements
Our UDP socket implementation has been migrated from C to Rust, and
along the way we've made many improvements to the socket API for UDP,
TCP, and Unix sockets. We've added support for the following:
* `sendmsg`, `recvmsg`, and `shutdown` syscall support for UDP sockets
* `MSG_TRUNC` support for UDP and Unix sockets
* `MSG_PEEK` support for UDP sockets
* `SO_DOMAIN`, `SO_PROTOCOL`, and `SO_ACCEPTCONN` socket options for TCP
and UDP sockets
* `SIOCGSTAMP` ioctl support for TCP and UDP sockets
We've also made various fixes and improvements to existing socket
functionality so that Shadow more accurately follows the behaviour of Linux.
## C to Rust migration
We continue to progress our migration from C to Rust. In addition to
Rust progress in the shim and in the new TCP stack, other notable
migrations since the last update include the shared memory allocator,
epoll and timerfd descriptors and syscalls, UDP sockets, and several
time-related syscalls. Our current status is 74% Rust code and 21.4% C
code (much of this being C tests) according to Github statistics.
# Release status
We expect to release Shadow 3.1.0 some time in the next few weeks.
# Project status - NSF grant is wrapping up
The last three years of development on Shadow have been sponsored by an
[NSF](https://shadow.github.io/docs/guide/nsf_sponsorship.html) grant.
That grant ends at the end of this month (September 2023). This isn't
the end of development on Shadow — Rob Jansen (@robgjansen) at the U.S.
Naval Research Laboratory continues to head the project. Over the course
of this project, The Tor Project has incorporated Shadow into its own
research and testing workflows, and is likely to continue contributing
as well. We look forward to continue improving and maintaining Shadow,
but at a slower pace of development since we no longer expect
contributions from programmers singularly dedicated to Shadow
development. Jim Newsome (@sporksmith) is continuing to be a Tor Project
employee but shifting focus to other Tor projects; Steven Engler
(@stevenengler) is open to opportunities.
Happy simulating!
The Shadow team
Hey everyone!
Here are our meeting logs:
http://meetbot.debian.net/tor-meeting/2023/tor-meeting.2023-09-28-15.57.html
And our meeting pad:
Anti-censorship work meeting pad
--------------------------------
------------------------------------------------------------------------------------
THIS IS A
PUBLIC PAD
------------------------------------------------------------------------------------
Anti-censorship
--------------------------------
Next meeting: Thursday, Oct 05 16:00 UTC
Facilitator: meskio
Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC
(channel is logged while meetings are in progress)
This week's Facilitator: onyinyang
== Goal of this meeting ==
Weekly check-in about the status of anti-censorship work at Tor.
Coordinate collaboration between people/teams on anti-censorship at the
Tor Project and Tor community.
== Links to Useful documents ==
* Our anti-censorship roadmap:
* Roadmap:
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards
* The anti-censorship team's wiki page:
*
https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home
* Past meeting notes can be found at:
* https://lists.torproject.org/pipermail/tor-project/
* Tickets that need reviews: from sponsors, we are working on:
* All needs review tickets:
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s…
* Sponsor 96 <-- meskio, shell, onyinyang, cohosh
* https://gitlab.torproject.org/groups/tpo/-/milestones/24
* Sponsor 150 <-- meskio working on it
*
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues/?label_na…
== Announcements ==
== Discussion ==
- Update on snowflake domain fronting issues:
- problem with cdn.sstatic.net affected more than just snowflake,
also moat, conjure (no deployment change required)
- fixed in new tor browser update but some users relying on
snowflake/moat bridges will be unable to update
- relying on a the same new front may be detrimental to improve
connection issues for some users, perhaps a pool of randomly selected
domains would improve the situation:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
== Actions ==
== Interesting links ==
== Reading group ==
* We will discuss "" on
*
* Questions to ask and goals to have:
* What aspects of the paper are questionable?
* Are there immediate actions we can take based on this work?
* Are there long-term actions we can take based on this work?
* Is there future work that we want to call out in hopes
that others will pick it up?
== Updates ==
Name:
This week:
- What you worked on this week.
Next week:
- What you are planning to work on next week.
Help with:
- Something you need help with.
cecylia (cohosh): 2023-09-28
Last week:
- rdsys merge request !157
- wrote a feature for snowflake that allows multiple domain fronts
-
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- wrote a forum post for the domain fronting issue with moat
-
https://forum.torproject.org/t/temporary-fix-for-moat-and-connection-assist…
- looked into orbot use of circumvention settings api
- https://github.com/guardianproject/orbot/issues/983
This week:
- finish deploying lox distributor
- follow up on conjure reliability issues
- visualize and write up some snowflake shadow simulation results
Needs help with:
dcf: 2023-09-21
Last week:
- snowflake CDN bookkeeping
https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Snowflake-co…
- noticed a problem with the fastly domain front and
coordinated with cohosh to analyze it
https://lists.torproject.org/pipermail/anti-censorship-team/2023-September/…
Next week:
- revise encapsulation.ReadData redesign to return an error in
the case of a short buffer
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to have snowflake-client log whenever KCPInErrors
is nonzero
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- parent:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- open issue to disable /debug endpoint on snowflake broker
Before EOY 2023:
- move snowflake-02 to new VM
Help with:
meskio: 2023-09-21
Last week:
- make fake descriptors for rdsys (rdsys#171)
- document rdsys development process (rdsys#131)
- use the right content-type in moat (rdsys#175)
- add token authentication to onbasca requests (rdsys#174)
Next week:
- add a DB for bridges to rdsys (rdsys#56)
Shelikhoo: 2023-09-28
Last Week:
- [Merge Request Awaiting] Add SOCKS5 forward proxy support to
snowflake (snowflake!64) (stalled)
- Write Tor Spec for Armored URL
- Remove Go 1.20 CI for Snowflake:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
- Consolidated Snowflake Update
dependencies:https://gitlab.torproject.org/tpo/anti-censorship/pluggable-tr…
- Merge request reviews
Next Week/TODO:
- Write Tor Spec for Armored URL(continue)
- Merge request reviews
onyinyang: 2023-09-28
Last week(s):
- Still need to merge the updated dependencies and make MR to
upstream ZK lib:
- found bug in the zkp crate and will push a change to this
to the upstream branch
- reach out to dalek-cryptography maintainers to give them
a heads up and get a sense of the plan for zkp lib
- we may end up maintaining the zkp lib
- Finished changes to rdsys API at the /resources endpoint
to meet the needs of Lox
- Working on required changes to lox-distributor
This week:
- Hopefully,finish up with the dependencies issue
- Finish up changes to Lox distributor
- continue with metrics
(long term things were discussed at the meeting!):
https://pad.riseup.net/p/tor-ac-community-azaleas-room-keep
- brainstorming grouping strategies for Lox buckets (of
bridges) and gathering context on how types of bridges are
distributed/use in practice
Question: What makes a bridge usable for a given user, and
how can we encode that to best ensure we're getting the most appropriate
resources to people?
1. Are there some obvious grouping strategies that we
can already consider?
e.g., by PT, by bandwidth (lower bandwidth bridges
sacrificed to open-invitation buckets?), by locale (to be matched with a
requesting user's geoip or something?)
2. Does it make sense to group 3 bridges/bucket, so
trusted users have access to 3 bridges (and untrusted users have access
to 1)? More? Less?
--
---
onyinyang
GPG Fingerprint 3CC3 F8CC E9D0 A92F A108 38EF 156A 6435 430C 2036