Hi!
This is a short email to let people know that TPA meetings are suspended
for a while, as we are running under limited staff. I figured I would
still send you those delicious metrics of the month and short updates
like this to keep people informed of the latest.
# Metrics of the month
* hosts in Puppet: 87, LDAP: 90, Prometheus exporters: 141
* number of Apache servers monitored: 28, hits per second: 0
* number of Nginx servers: 2, hits per second: 2, hit ratio: 0.87
* number of self-hosted nameservers: 6, mail servers: 7
* pending upgrades: 0, reboots: 1
* average load: 1.04, memory available: 1.98 TiB/2.74 TiB, running processes: 569
* bytes sent: 269.96 MB/s, received: 162.58 MB/s
* [GitLab tickets][]: 138 tickets including...
* open: 0
* icebox: 106
* backlog: 22
* next: 7
* doing: 4
* (closed: 2225)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
Note that the Apache exporter broke because of a fairly dumb error
introduced in february, so we do not have the right "hits per second"
stats there. Gory details of that bug live in:
https://github.com/voxpupuli/puppet-prometheus/pull/541
# Quote of the week
"Quoting. It's hard."
Okay, I just made that one up, but yeah, that was a silly one.
A.
--
Antoine Beaupré
torproject.org system administration
Hey!
We are looking for somebody to organize next DEMO Day on May 26th.
DEMO days are usually an hour long full of 5 minutes presentations.
Each presentation is of something related to the Tor ecosystem. You can
take a look at the last demo day we had in February in
https://pad.riseup.net/p/tor-demo-day-february-2021-keep
If you are interested in putting this together please send me a message
to gaba at torproject dot org.
thanks,
gaba
--
pronouns she/her/they
GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19
Hi!
Next week we are hacking on small projects all week! There are 10
proposals so far from visualizing metrics data to adding onion services
v3 support to arti.
To kickoff the week we will have an opening session on **Monday March
29th at 15UTC**. Projects will be presented at that session and teams
will be created. The link for the opening session room will be posted on
Monday. We will close the hackweek on Friday with a demo session where
teams will be able to show the work done during the week.
You can take a look at the proposed projects in
https://gitlab.torproject.org/tpo/community/hackweek
1 Torify signal-cli with torsocks
Summary: Work through blocking bugs for getting signal-cli working with
torsocks. Interesting because signal is being blocked in some locales.
Lead by jnewsome and/or dgoulet
2 Prometheus alerts for anti-censorship metrics
Summary: We have BridgeDB exporting prometheus metrics so far, and we
could implement this for Snowflake. It would be great if we could get
alerts when usage changes to notify us of possible censorship events.
Somewhat related, it would also be nice to get alerts when default
bridge usage drops off suddenly or directly connecting Tor users from
different regions.
Lead by cohosh and anarcat
3 Prototype network-namespace-based torsocks
Summary: Use network namespaces (or maybe something else?) to run target
software in an environment where it can't talk to the real network; it
can only talk to the tor socks port and/or some "shim" adapter. Might be
able to remove or lessen dependence on LD_PRELOAD (which isn't available
everywhere, can be "escaped", and can be a bit fragile). If we continued
to use LD_PRELOAD could at least be used to prevent accidental
connections to the real network.
Lead by jnewsome and/or dgoulet
4 Add support for UDP sockets over onion services, possibly with
enabling support for WebRTC in Tor Browser
Lead by sysrqb
5 Visualize Tor metrics's data in ways that it can be useful for community
Summary: The goal is to have a dashboard with Tor usage per country in a
way that is easy to see big changes happening. Right now we need to
select each country to see the Tor usage. It would be good to have a way
to see all the countries and the onews where usage is increasing (via
bridge and direct connection)
Lead by gus and gaba
6 Prototype Rust+Arti-based HTTP frontend cache for directory authorities
Summary: Directory authorities are under a lot of unnecessary load from
excessive download requests. We have other projects in mind to reduce
those requests, but one workaround is to add a frontend cache in front
of one or more of the authorities' HTTP ports. With this project we'll
write a Rust server use Arti's download and validation code to fetch
directory information from an authority, and expose that information via
a set of HTTP requests. With luck, our code will be reuseable in the
future when relays or authorities are rewritten in Rust.
Lead by nickm
7 Onionshare download accelerator
Summary: Journalists are requesting ways to download large files from
onion services faster. This is actually possible already with HTTP,
without any Tor client modifications. By splitting up requests into 250k
chunks using HTTP range requests, and using SOCKS username and password
to allocate these requests properly onto different circuits, we can hit
~6 megabytes/sec per guard (so ~12 megabytes/sec with 2 gaurds), with a
custom HTTP download acelerator. This download accelerator could be
provided as part of onionshare. For more details, see
https://github.com/micahflee/onionshare/issues/1295.
Lead by mike
8 Vanguards doc updates, bugfixes, packages for onionshare and/or Tor
Browser
Summary: The vanguards Tor control port addon provides defense against
Guard discovery attacks, as well as other attacks against onion
services, onion clients, and even Tor Browser exit traffic. Vanguards is
in need of some doc updates, bugfixes, and we could even package it for
one of onionshare or Tor Browser. Packaging it with Onionshare also
helps improve the security properties of the above download accelerator
item. See the bugtracker for specific items:
https://github.com/mikeperry-tor/vanguards/issues
Lead by mike
9 Circuit Padding Simulator
Summary: The Tor circuit padding framework is under active use by
researchers to improve padding defenses against website traffic
fingerprinting, and onion service circuit fingerprinting. The simulator
is in need of update to the latest Tor release, as well as in need of
performance improvements, and a more accurate way to deal with
time-based features. See
https://github.com/pylls/circpad-sim/blob/master/README.md
Lead by mike
10 Onion service v3 support for arti
Summary: arti currently does not do v3 onion services. Let's do some
solid work towards making that possible.
Lead by asn
Hey everyone!
Here are our meeting logs:
http://meetbot.debian.net/tor-meeting/2021/tor-meeting.2021-03-25-15.58.html
and our meeting pad:
Anti-censorship work meeting pad
--------------------------------
Next meeting: Thursday March 25th 16:00 UTC
Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC
(channel is logged while meetings are in progress)
== Goal of this meeting ==
Weekly checkin about the status of anti-censorship work at Tor.
Coordinate collaboration between people/teams on anti-censorship at Tor.
== Links to Useful documents ==
Our anti-censorship roadmap:
Roadmap:
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards
The anti-censorship team's wiki page:
https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home
Past meeting notes can be found at:
https://lists.torproject.org/pipermail/tor-project/
Tickets that need reviews: from sponsors we are working on:
All needs review tickets:
https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s…
Sponsor 30
https://gitlab.torproject.org/groups/tpo/-/milestones/4https://gitlab.torproject.org/groups/tpo/-/milestones/7https://gitlab.torproject.org/groups/tpo/-/milestones/5https://gitlab.torproject.org/groups/tpo/-/milestones/6
Sponsor 28
must-do tickets:
https://gitlab.torproject.org/groups/tpo/-/milestones/10
possible tickets:
https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&…
Public bug-reporting pad:
https://pad.riseup.net/p/tor-anti-censorship-bugs-keep <-- we have
anonymous tickets handling now. Let's see which anti-censorship repos to
include.
== Announcements ==
== Discussion ==
Do we need to upgrade bridges for today's OpenSSL advisories?
https://www.openssl.org/news/secadv/20210325.txt
Next week is hackweek and we're working on anti-censorship alerts
== Actions ==
== Interesting links ==
== Reading group ==
We will discuss "" on
Questions to ask and goals to have:
What aspects of the paper are questionable?
Are there immediate actions we can take based on this work?
Are there long-term actions we can take based on this work?
Is there future work that we want to call out, in hopes that others
will pick it up?
== Updates ==
Name:
This week:
- What you worked on this week.
Next week:
- What you are planning to work on next week.
Help with:
- Something you need help with.
cecylia (cohosh): last updated 2020-03-25
Last week:
- merged snowflake!30
- updated the link to our snowflake documentation wiki on
snowflake.torproject.org
- updated wiki to include a link to jacobo's ansible playbook
- updated pion webrtc version in snowflake (snowflake#40037)
- started a tor browser build for it
- worked on candidate refactor of snowflake (snowflake#40036)
- responded to bridgestrap outage (bridgestrap#16)
- sponsor 28 integration prep
- tried out new private bridges
This week:
- hackweek work on anti-censorship alerts
Ongoing:
- connect bridgedb to rdsys (rdsys#12)
- congestion logging patches for snowflake#40026
- install our snowflake app on android and try it out for a few days
- prepare snowflake for sponsor 28 evaluations
Needs help with:
- review of snowflake!31
arlolra: 2020-10-29
Last week:
-
Next week:
- getting back up to speed
- follow ups to #33365
- start on #31201
Help with:
-
dcf: 2021-03-25
Last week:
Next week:
- snowflake CDN accounting
- look at PT v2 branch
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…
Help with:
Antonela: 2020-08-27
This week:
agix:2021-03-25
Last week:
-Been busy with university stuff for the last few weeks
Next week:
-Work on bridgebox for rdsys
-More research on httpt #4
Help with:
-
hanneloresx: 2021-3-4
Last week:
- Submitted MR for bridgestrap issue #14
Next week:
- Finish bridgestrap #14
- Find new issue to work on
Help with:
-
HashikD: 2021-01-22
This week:
-
Next week:
-
Help with: -
---------- Forwarded message ---------
From: Ole Tange <ota(a)prosa.dk>
Date: Sat, Mar 20, 2021 at 4:36 PM
Subject: Using tor as transport network for LBRY
To: antonela(a)torproject.org <antonela(a)torproject.org>
LBRY is a distributed, open source alternative to YouTube.
They currently use their own protocol for data transfer and has problem
with hole punching (through NAT walls).
I have raised this ticket with LBRY (there are also other privacy concerns
that Tor would solve):
https://github.com/lbryio/lbry-sdk/issues/3235#issuecomment-800547084
Given that you said you are willing to help others using the tor network, I
hope you will ask one of your technicians to post a note, saying you are
here to help them out.
/Ole
--
Antonela Debiasi
UX Team Lead
torproject.org
@antonela
E2330A6D1EB5A0C8
Hello,
Apologies for sharing this late.
Throughout January 2021, the OONI team worked on the following sprints:
* Sprint 30 - Rough Shark (4 - 17 January, 2021)
* Sprint 31 - Rhincodon (18 - 31 January, 2021)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
## Report on internet blocks and outages in Uganda amid 2021 general
election
In collaboration with DefendDefenders, Defenders Protection Initiative,
and CAIDA, we published a new research report documenting the internet
disruptions in Uganda amid its 2021 general election.
Our report is available here:
https://ooni.org/post/2021-uganda-general-election-blocks-and-outage/
We based our investigation on:
* OONI measurements collected from Uganda:
https://explorer.ooni.org/search?until=2021-01-23&since=2020-12-23&probe_cc…
* OONI experiments using the `miniooni` research client (which we plan
to eventually integrate into the OONI Probe apps)
* Public data sources monitoring internet outages (IODA data, Google
traffic data, Oracle Internet Intelligence data, and CloudFlare Radar data)
In the days leading up to Uganda’s 2021 general election, ISPs blocked
access to the Google Play Store (hampering people’s ability to download
apps), as well as to a number of social media apps (including WhatsApp,
Facebook Messenger, and Telegram) and websites (such as facebook.com) –
regardless of OTT tax payment.
Access to certain circumvention tool websites (such as protonvpn.com)
was blocked as well, though both Tor and Psiphon worked throughout the
election period.
Starting from the eve of Uganda’s 2021 general election (in the evening
of 13th January 2021), Uganda was disconnected from the internet entirely.
The country experienced a 4-day internet outage (which included election
day), as shown through several public data sources: Internet Outage
Detection and Analysis (IODA), Oracle’s Internet Intelligence Map,
Cloudflare Radar, and Google traffic data. In our report, we also share
network-level analysis of the internet outage based on IODA data.
The internet outage is further corroborated by the absence of OONI
measurements from Uganda during this time period (since OONI Probe
requires internet connectivity to perform tests), as well as by the
drastic drop in Tor users and Psiphon users during this period.
Even though internet connectivity in Uganda was restored on 18th January
2021, access to social media and circumvention platforms remained blocked.
Notably, Ugandan ISPs only appear to have started blocking access to
YouTube on 18th January 2021, even though the platform is not included
on the OTT list of taxed platforms.
Further details are available through our report.
## Blocking of Signal in Iran
On 25th January 2021, Iran started blocking access to the Signal
messaging app.
Thanks to OONI Probe users in Iran (who contributed relevant
measurements), we were able to detect the blocking of signal.org within
a few hours since the blocking started. We reported this through OONI’s
Twitter account (sharing relevant OONI measurements):
https://twitter.com/OpenObservatory/status/1353637360727109633
OONI measurements on the blocking of signal.org were also shared by
Zoomit, an Iranian technology media outlet:
https://twitter.com/TheZoomit/status/1353636089647558656
Recent OONI measurements show that signal.org remains blocked on several
networks in Iran:
https://explorer.ooni.org/search?until=2021-03-20&since=2021-02-18&domain=s…
## OONI Probe Mobile
In January 2021, we released OONI Probe mobile 2.9.0 and 2.9.1 on:
* Android: https://github.com/ooni/probe-android/releases/tag/v2.9.0 &
https://github.com/ooni/probe-android/releases/tag/v2.9.1
* iOS: https://github.com/ooni/probe-ios/releases/tag/v2.9.0 &
https://github.com/ooni/probe-ios/releases/tag/v2.9.1
With these new releases, we shipped a new OONI Probe test for RiseupVPN
(which was developed by the LEAP collective).
We also:
* Completed UI tests: https://github.com/ooni/probe/issues/1272
* Made various code quality improvements:
https://github.com/ooni/probe-android/pull/402
* Completed the implementation of the check-in API in the OONI Probe
engine and wrote wrappers and integration tests for this implementation:
https://github.com/ooni/probe-engine/pull/1158
* Wrote a script to facilitate mobile app releases:
https://github.com/ooni/probe/issues/1287
* Worked on adding support for dark mode on iOS:
https://github.com/ooni/probe/issues/925
* Improved the crash reporting system:
https://github.com/ooni/probe/issues/1332
## OONI Run
We added localization support to OONI Run (https://run.ooni.io/) to
accommodate translations: https://github.com/ooni/run/pull/50
We prioritized adding localization support to OONI Run in response to
community requests to translate the platform to Chinese (Taiwan).
## OONI Probe Desktop
Throughout January 2021, we worked towards the OONI Probe Desktop 3.2.2
release. This involved development work towards ensuring feature parity
with the OONI Probe mobile app: creating a new Dashboard (with the same
UI as the OONI Probe mobile app), adding support for running all tests
with a single button, and adding the “Choose websites” button for custom
URL testing (similarly to the OONI Probe mobile app).
We also:
Added support for setting the maximum execution time of the Web
Connectivity test: https://github.com/ooni/probe/issues/1280
Added support for storing measurements in the OONI backend (and not on
disk): https://github.com/ooni/probe/issues/1313
Worked towards integrating the new RiseupVPN test into the OONI Probe
desktop app
### Updated OONI Probe macOS homebrew package
We upgraded the OONI Probe homebrew package to run the new ooniprobe 3.x
command line interface. In order to do this, we implemented native
system scheduling features inside of the command line tool (see:
https://github.com/ooni/probe-cli/pull/192).
This allows users to enable automatic testing by running the command:
`ooniprobe autorun start`
More details about this can be found in the following issue:
https://github.com/ooni/probe/issues/1289
The maintenance of the package has since been taken up by the homebrew
community, which is doing an excellent job at keeping it up to date:
https://github.com/Homebrew/homebrew-core/pulls?q=is%3Apr+ooniprobe+is%3Acl…
We updated the installation instructions for the OONI Probe Command Line
Interface (CLI) on our website to enable users to get the latest version
of the OONI Probe command line tool: https://ooni.org/install/cli/macos
## Released Debian package for OONI Probe
We released a Debian package for OONI Probe with support for automatic
testing.
As part of the installation process, this package displays the informed
consent procedure through the use of native Debian packaging features.
Packages are automatically built as part of the continuous integration
workflow. See: https://github.com/ooni/probe/issues/1061
We updated the OONI Probe installation page for Linux to feature the new
Debian package: https://ooni.org/install/cli/ubuntu-debian
## OONI Probe engine
We released v.0.22.0 of the OONI Probe measurement engine:
https://github.com/ooni/probe-engine/releases/tag/v0.22.0
Highlights from this release include:
* Significantly improved QUIC implementation
* Significantly improved DNSCheck implementation
* New `run` experiment that takes in input a list of experiments to run
along with their options as a JSONL
This release includes a series of API changes, improvements, updates,
and bug fixes, as documented here:
https://github.com/ooni/probe-engine/releases/tag/v0.22.0
## Expanding OONI Probe measurement methodologies
We improved upon our QUIC experiment, as documented through the
following pull requests:
https://github.com/ooni/probe-engine/pull/1196https://github.com/ooni/probe-engine/pull/1126https://github.com/ooni/probe-engine/pull/1197https://github.com/ooni/probe-engine/pull/1176
We also improved upon our DNSCheck
(https://github.com/ooni/probe-engine/pull/1166) and urlgetter
(https://github.com/ooni/probe-engine/pull/1182) experiments. We
collaborated with community members who volunteered to work with us on
these experiments.
### New OONI Probe RiseupVPN test
In January 2021, we shipped a brand new OONI Probe test for RiseupVPN!
You can now check whether RiseupVPN works on your network. Learn all
about this test here: https://ooni.org/nettest/riseupvpn/
The RiseupVPN test (developed by the LEAP collective) was first made
available through the 2.9.1 release of the OONI Probe mobile app
(https://ooni.org/install/mobile). The test is now also available in the
OONI Probe desktop app.
All RiseupVPN test results from around the world are openly published in
near real-time on OONI Explorer:
https://explorer.ooni.org/search?since=2020-12-30&test_name=riseupvpn
## OONI Explorer
We released OONI Explorer 2.4.0
(https://github.com/ooni/explorer/releases/tag/v2.4.0) which includes
improvements to the metadata of the measurement pages for the purpose of
search engine optimization. We also integrated percy.io for the visual
testing of OONI Explorer (https://percy.io/407b7763/ooni-explorer/), and
fixed some issues in measurements
(https://github.com/ooni/explorer/pull/543).
## OONI backend
Throughout January 2021, we worked on the following backend activities:
* Deployed an event detector on test and updated it for the new ams-pg
codebase
* Monitored the dashboard for the reactive test-list
* Investigated the counters bug and created a new
counters_hourly_software table
* Investigated the cause of the measurement drops
* Handled measurement keys in the fastpath pipeline
* Fixed the counters table update bug
* Fixed the pagination in the API
* Changed the DB query timeout in the API to fix test-url timeout bugs
in probes
* Fixed the API enumeration/paging bug
* Worked on implementing the probe check-in API
* Generated report IDs
* Fixed the category code filtering
## New partnership
We formed a new partnership with civil society members in Sudan.
We previously had the opportunity to collaborate with this partner as
part of Internews’ 6-week internet shutdown measurement training program
for human rights defenders, which we helped lead. This training program
(which also included OONI modules) helped lay the ground for this
partnership.
## Test list updates
In January 2021, we made a few minor updates to the Citizen Lab test
lists, as documented through the following pull requests:
* Hong Kong test list: https://github.com/citizenlab/test-lists/pull/711
* Global test list: https://github.com/citizenlab/test-lists/pull/713
## Community activities
### OTF Virtual Summit 2021
Between 11th-15th January 2021, we attended the OTF Virtual Summit 2021.
On 13th January 2021, we participated as a panelist in the online
session “Collaboration between organizations in the internet freedom
field”, during which we shared some of the challenges, opportunities,
and best practices we have identified in forming partnerships and
working with digital rights organizations around the world.
## Userbase
In January 2021, 7,188,644 OONI Probe measurements were collected from
5,270 networks in 203 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
--
Maria Xynou
Research & Partnerships Director
Open Observatory of Network Interference (OONI)
https://ooni.org/
PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E
Hello!
Here's a (late, again) report from our monthly sysadmin meeting.
# Roll call: who's there and emergencies
* anarcat
* hiro
* gaba
No emergencies.
# Roadmap review
Review and prioritize [the board][].
[the board]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
* CiviCRM setup discussion. gaba will look at plan
* anarcat sent a formal proposal to the tor-internal mailing list
with [jenkins retirement plan][] ([issue 40167][]), will be
proposed to tor-internal soon
* SMTP out only server is resuming in ~Next
* Discourse situation: wait for a few months until hiro can take it
back ([issue 40183][])
[issue 40183]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40183
[issue 40167]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40167
[jenkins retirement plan]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-10-jenkin…
# Documentation and communication
Are the current processes to document our work okay? Do we have
communication problems? Let's clarify expectations on how to manage
work and tickets.
## What is working
* anarcat's docs work great, but could use a TL;DR (:+1:)
* monthly meetings in voice calls
* jump on a call when there is an issue or misunderstanding (:+1:)
## What can be improved
* irc can be frustrating when communicating, jump on a voice call
when necessary!
* wiki is good for documentation, but not great to get feedback,
because we don't want to delete other people's stuff and things get
lost. better to use issues with comments for proposals.
* hard time understanding what is going on some tickets, because of
the lack of updates. We can write more comments in the tickets.
* when triaging: if you assign to someone, then that person needs to
know. when assigning an active queue (~Next or ~Doing), make sure
the ticket is assigned.
# Triage
Is our current triage system working? How can others (AKA gaba)
prioritize our work?
Note that ahf is also working on triage, automation, more
specifically, through the [triage ops][] project.
[triage ops]: https://gitlab.torproject.org/ahf/triage-ops/
We might want to include the [broader TPA dashboard][] eventually, but
this requires serious triage work first.
[broader TPA dashboard]: https://gitlab.torproject.org/groups/tpo/tpa/-/boards
Discussion postponed.
# On call
Which services/issues we can call TPA about when nobody is working?
Review and discuss the [current support policy][], which is basically
"none, things may be down until we return"...
Discussion postponed.
[current support policy]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-2-support…
# Other discussions
## Anonymous ticket system
Postponed.
# Next meeting
April 6th, 15:00UTC, equivalent to: 08:00 US/Pacific, 12:00
America/Montevideo, 11:00 US/Eastern, 17:00 Europe/Paris.
# Metrics of the month
* hosts in Puppet: 85, LDAP: 88, Prometheus exporters: 139
* number of Apache servers monitored: 28, hits per second: 50
* number of Nginx servers: 2, hits per second: 2, hit ratio: 0.87
* number of self-hosted nameservers: 6, mail servers: 7
* pending upgrades: 4, reboots: 0
* average load: 0.93, memory available: 1.98 TiB/2.73 TiB, running
processes: 627
* bytes sent: 267.74 MB/s, received: 160.59 MB/s
* [GitLab tickets][]: ? tickets including...
* open: 0
* icebox: 107
* backlog: 15
* next: 9
* doing: 7
* (closed: 2213)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
# Grafana dashboards of the month
The [Postfix dashboard][] was entirely rebuilt and now has accurate
"acceptance ratios" per host. It was used to manage the latest
newsletter mailings. We still don't have great ratios, but at least
now we know.
[Postfix dashboard]: https://grafana.torproject.org/d/Ds5BxBYGk/postfix-mtail
The [GitLab dashboard][] now has a "CI jobs" panel which shows the
number of queued and running jobs, which should help you figure out
when your precious CI job will get through!
[GitLab dashboard]: https://grafana.torproject.org/d/QrDJktiMz/gitlab-omnibus?orgId=1
--
Antoine Beaupré
torproject.org system administration