Hello everyone,
We have another new job opening to share:
Metrics Data Architect (https://www.torproject.org/about/jobs/metrics-data-architect/). The job description is attached as a PDF and pasted below in plain text.
We are also hoping to fill this position as soon as possible, so please help us spread the word! :)
Thank you!
Best,
Erin Wyatt
Director of People Operations
ewyatt(a)torproject.org
PGP: 35E7 2A9F 6655 45F9 2CB6 6624 BA0C 9400 F80F 91CE
https://www.torproject.org | http://expyuzz4wqqyqhjn.onion/
——————————————>8
Internet Freedom Nonprofit Seeks Metrics Data Architect
February 4, 2021
The Tor Project, Inc., a 501(c)(3) nonprofit organization advancing human rights and freedoms by creating and deploying free and open source anonymity and privacy technologies, is seeking an experienced Data Architect to take our metrics work to the next level.
Tor is for everyone, and we are actively working to build a team that represents people from all over the world — people from diverse ethnic, national, and cultural backgrounds; people from all walks of life. Racial minorities, non-gender-binary people, women, and people from any group that is generally underrepresented in tech are encouraged to apply.
This is a full-time remote position.
Our Metrics Team has been collecting data since 2004 to help improve the tools we build and learn more about the Tor network. For example, we monitor the number of relays and clients in the network, their respective capabilities, the number of clients connecting via bridges, fluctuations in network speed, etc. Gathering this data results in huge data archives, so we are also working to develop tools to process this data and make it available to everyone.
How we achieve our goals:
• Robustness (We want to avoid bugs and/or bad design decisions that cause us to miss data)
• Timeliness (users need up-to-date network status information)
• Scalability (as the network grows, so does our data)
• Transparency (our community rightly wants to know what data we're collecting)
The most interesting challenge for metrics collection at Tor is how to gather data on an anonymity system without de-anonymizing users.
We are integrating the metrics team with a new network health team that will be also focusing on taking care of the health of the Tor network. Additionally, Tor has just begin a large network performance improvement project that will vastly increase network utilization and client performance. There will be close interaction between this role and our performance effort, in the form of ensuring data is collected properly during deployment of new performance features and performance tuning. This will also include producing custom visualizations, doing analysis, and comparisons between historical and current data.
The person in this position will work directly with helping us maintain existing systems, and design new systems for gathering and analyzing data. They will help the rest of the teams understand the data available to improve our tools as well as the Tor network's health.
The bulk of our code is written in Java, but smaller portions are written in R, Python, PostgreSQL, and JavaScript. Part of this job will be to analyze and fix bugs in our current code and review patches.
You will be working with the existing teams in Tor with support and advice on data collection and measurements for their needs in their respective projects.
Our main five codebases:
• Collector
• metrics-lib
• Onionoo
• Exonerator
• metrics-web
Requirements:
Technical abilities/experience:
• Have experience finding your way into existing Java, R, and PostgreSQL-based code bases and the ability to review patches and make changes to fix bugs/smaller enhancements.
• Have experience with web application development in the Python programming language. Experience with known frameworks is a plus.
• Able to identify shortcomings in our data pipeline and suggest improvements to reduce complexity and future maintenance efforts.
• Experience working with Git and Gitlab or similar issue tracking tools.
• Ability to learn quickly and can adapt to our current processes; being able to improve future processes for releasing software and operating services.
• Understanding of the inherent privacy implications of gathering data in an anonymity system, the security implications of gathering metrics data from semi-trusted relays in the Tor network, and the challenges of processing large amounts of data per day (specifically performance and scalability challenges).
• Data analysis: Ability to make sense of data sets and use data analysis tools to find and visualize interesting patterns.
• Mathematics: Knowledge of basic statistics.
Collaborative requirements:
• Ability to work remotely 100% of the time, as synchronization happens via email and/or IRC.
• Participation in weekly IRC meetings and monthly team video chats.
• Language: write and speak fluent English.
• Comfortable posting to a public mailing list or speaking up in a public IRC channel to ask questions, even when you think the question might be obvious or silly.
Bonus skills:
• Open source experience: You have contributed to an open source project before, and you're accustomed to a pattern of early and frequent releases without attempting to finalize things on your own.
• Scientific writing: Experience writing technical reports about data findings.
• Networking background: Experience working with networks and measurements in the past.
• You support Internet Freedom!
To apply, submit a cover letter, your CV/resume (including three professional references), and a link to a code sample or some non-trivial software project you have significantly contributed to. In your cover letter, please include the reason you want to work at the Tor Project.
IMPORTANT: Please email application materials in plain text or PDF format to job-metrics at torproject dot org with “Metrics Data Architect” in the subject line.
The Tor Project's workforce is smart, committed, and hard working. We currently have a paid and contract staff of around 24 developers and operational support people, plus many thousands of volunteers who contribute to our work. The Tor Project is funded in part by government research and development grants, and in part by individual, foundation, and corporate donations.
Salary for this position depends on experience and there is voluntary opt-in salary transparency for employees and contractors.
The Tor Project has a competitive benefits package, including a generous PTO policy, 16 paid holidays per year (including the week between Christmas and New Year's, when the office is closed), and flexible work schedule. Insurance benefits vary by employment status and country of residence.
The Tor Project, Inc., is an equal opportunity, affirmative action employer.
Hi everyone,
Here's a (somewhat late) report from our monthly sysadmin meeting.
# Roll call: who's there and emergencies
anarcat, gaba, hiro
* hiro will be doing security reboots for [DSA-483][]
[DSA-483]: https://www.debian.org/security/2021/dsa-4843
# Dashboard review
We reviewed the [dashboard][] to prioritise the work in February.
[dashboard]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
anarcat is doing triage for the next two weeks, as now indicated in
the IRC channel topic.
# Communications discussion
We wanted to touch base on how we organise and communicate, but didn't
have time to do so. Postponed to next meeting.
Reminder:
* Documentation about documentation:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/service/documentation
* Policies: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy
# Next meeting
March 2nd, 2021, same time
# Metrics of the month
* hosts in Puppet: 83, LDAP: 86, Prometheus exporters: 135
* number of Apache servers monitored: 27, hits per second: 182
* number of Nginx servers: 2, hits per second: 3, hit ratio: 0.83
* number of self-hosted nameservers: 6, mail servers: 12
* pending upgrades: 11, reboots: 71
* average load: 0.41, memory available: 1.94 TiB/2.67 TiB, running
processes: 520
* bytes sent: 281.62 MB/s, received: 163.47 MB/s
* [GitLab tickets][]: 130 tickets including...
* open: 0
* icebox: 96
* backlog: 18
* next: 10
* doing: 7
* (closed: 2182)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
I've been collecting those dashboard metrics for a while, and while I
don't have pretty graphs to show you yet, I do have this fancy table:
| date | open | icebox | backlog | next | doing | closed |
|------------|------|--------|---------|------|-------|--------|
| 2020-07-01 | 125 | 0 | 26 | 13 | 7 | 2075 |
| 2020-11-18 | 1 | 84 | 32 | 5 | 4 | 2119 |
| 2020-12-02 | 0 | 92 | 20 | 9 | 8 | 2130 |
| 2021-01-19 | 0 | 91 | 20 | 12 | 10 | 2165 |
| 2021-02-02 | 0 | 96 | 18 | 10 | 7 | 2182 |
Some observations:
* the "Icebox" keeps piling up
* we are closing tens and tens of tickets (about 20-30 a month)
* we are getting better at keeping Backlog/Next/Doing small
* triage is working: the "Open" queue is generally empty after the
meeting
As usual, some of those stats are available in the main Grafana
dashboard. Head to <https://grafana.torproject.org/>, change the time
period to 30 days, and wait a while for results to render.
--
Antoine Beaupré
torproject.org system administration
TL;DR: the 2021 roadmap was adopted, see the details here:
<https://gitlab.torproject.org/tpo/tpa/team/-/wikis/roadmap/2021>
Followup of the [last meeting][] to complete the work on the 2021
roadmp.
[last meeting]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/meeting/2021-01-19
# Roll call: who's there and emergencies
anarcat, gaba, hiro
[Problem with gmail][], not a rush but priority.
[problem with gmail]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40149
# Roadmap review
We looked at the [draft 2021 roadmap proposal][] anarcat sent last
week.
[draft 2021 roadmap proposal]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/roadmap/2021
## Need to have / nice to have / non-goals
* need to prioritise fixing the blog (formatting, moderation), but
those fixes will probably not come before Q3, because of capacity
* we decided to *not* retire schleuder: hiro fixed a bunch of stuff
yesterday, it should work better now. no need to retire it as we
will still want encrypted mailing lists in the future
* service admins; let's not reopen that discussion
* added the bullseye upgrade to "nice to have", but not a hard
priority for 2021 (and will be, along with the python3 upgrade, for
2022)
* search.tpo (#33106) and "web metrics" (#32996) are postponed to
2022
* people suggested retiring "testnet" in the survey, but we don't
quite know what that is, so we presumably need to talk with the
network team about this
* we agreed to cover for some metrics: we updated [ticket 40125][]
with the remaining services to reallocate. covering for a service
means that TPA will reboot services and allocate disk/ram as
needed, but we are not in a position to make major reengineering
changes
[ticket 40125]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40125
## Quarterly prioritization
* there's a lot in Q1, but a lot of it is actually already done
* sponsor 9 requires work from hiro, so we might have capacity
problems
We added a few of the "needs to have" in the quarterly allocation to
make sure those are covered. We agreed we'd review the global roadmap
every quarter, and continue doing the monthly "kanban board" review
for the more daily progress.
# Next meeting
Going back to our regular programming, i have set a recurring meeting
on tuesdays, 1500UTC on the first tuesday of the month, for TPA.
# Metrics of the month
Skipped because last meeting was a week ago. ;)
--
Antoine Beaupré
torproject.org system administration
Hello everyone,
We have an update and a new job opening to share!
1. Cecylia Bocovich (cohosh), presently our developer on the Anti-Censorship Team, will stepping up to lead the team as Philipp moves on. Thank you, Cecylia!
2. With the upcoming vacancy on the Anti-Censorship Team, we have a new job opening! You can find the job description pasted below, attached as a PDF, and on our website athttps://www.torproject.org/about/jobs/software-developer-anticensorship/.
We are hoping to fill this position as soon as possible, so please help us spread the word! :)
Thank you!
Best,
Erin Wyatt
Director of People Operations
ewyatt(a)torproject.org
PGP: 35E7 2A9F 6655 45F9 2CB6 6624 BA0C 9400 F80F 91CE
https://www.torproject.org | http://expyuzz4wqqyqhjn.onion/
----------------------------->8
Internet Freedom Nonprofit Seeks Software Developer for Anti-Censorship Team
The Tor Project, Inc., a 501(c)(3) nonprofit organization advancing human rights and freedoms by creating and deploying free and open source anonymity and privacy technologies, is seeking an experienced Software Developer to take our anti-censorship work to the next level.
This developer will be tasked with improving the user experience and process of finding alternate routes to the Tor network when global censorship events block access to the Tor network. A personal commitment to free and open source software and the application of advanced programming skills for the greater good is essential.
This is a full-time remote position.
This developer position will be an integral part of our Anti-Censorship Team. Main responsibilities for this position are:
- Continue development of our bridge distribution system, rdsys.
- Maintain and improve the pipeline that distributes bridges to censored users.
- Evaluate and incorporate new pluggable transports.
- Collaborate with other teams in the Tor Project, external software projects (e.g. OONI), and research groups to move forward anti-censorship technology.
Required qualifications:
- Experience with designing, implementing, testing, and reviewing complex codebases (ideally in Go and Python).
- Ability to work in a self-directed and independent way.
- Comfortable with working remotely, across time zones.
- Strong familiarity with computer science fundamentals.
- Familiarity with Internet security and censorship circumvention technologies.
Preferred qualifications:
- Familiarity with reading research papers and turning research ideas into prototypes.
- Understanding of the Tor pluggable transport ecosystem and the censorship circumvention research space.
- Past contributions to and familiarity with the practices of free software projects.
- Passion and dedication to Internet freedom is an added plus.
To apply, submit a cover letter, your CV/resume (including three professional references), and a link to a code sample or some non-trivial software project you have significantly contributed to. In your cover letter, please include the reason you want to work at the Tor Project.
IMPORTANT: Please email application materials in plain text or PDF format, if possible, to job-anticensor at torproject dot org with “Anti-Censorship Developer” in the subject line.
Tor is for everyone, and we are actively working to build a team that represents people from all over the world — people from diverse ethnic, national, and cultural backgrounds – people from all walks of life. We encourage applications from BIPOC, non-cis people, women, and from people in any group that is underrepresented in tech.
The Tor Project's workforce is smart, committed, and hard working. We currently have a paid and contract staff of around 24 developers and operational support people, plus many thousands of volunteers who contribute to our work. The Tor Project is funded in part by government research and development grants, and in part by individual, foundation, and corporate donations.
Salary for this position depends on experience and there is voluntary opt-in salary transparency for employees and contractors.
The Tor Project has a competitive benefits package, including a generous PTO policy, 16 paid holidays per year (including the week between Christmas and New Year's, when the office is closed), and flexible work schedule. Insurance benefits vary by employment status and country of residence.
The Tor Project, Inc., is an equal opportunity, affirmative action employer.
Hi!
It feels so strange to say that this year around, but... happy new year
everyone! Let's hope we can do better this time around. ;)
Here's your first sysadmin report for 2021, hopefully we'll keep you
informed of our progress steadily in the coming year. Right now we're
working on the roadmap and, even though we asked you for feedback in the
user survey, it's still time to steer us in the good direction. We have
a meeting coming up where we're likely to set that more in stone, so now
is a good time if you forgot to respond to the survey...
Now onto the minutes.
Agenda:
- Roll call: who's there and emergencies
- Dashboard review
- Roadmap 2021 proposal
- 2020 retrospective
- Services survey
- Goals for 2021
- Other discussions
- Next meeting
- Metrics of the month
# Roll call: who's there and emergencies
present: hiro, gaba, anarcat
[GitLab backups are broken][]: it might need more disk space than we
need. just bump disk space in the short term, consider changing the
backups system, in the long term.
[GitLab backups are broken]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40143
# Dashboard review
We [reviewed the dashboard][], too much stuff in January, but we'll
review in February.
[reviewed the dashboard]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
# Roadmap 2021 proposal
We discussed the [roadmap project][] anarcat worked on. We reviewed
the 2020 retrospective, talked about the services survey, and
discussed goals for 2021.
[roadmap project]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/roadmap/2021
## 2020 retrospective
We reviewed and discussed the [2020 roadmap evaluation][] that anarcat
prepared:
[2020 roadmap evaluation]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/roadmap/2021#2020-roadma…
* **what worked?** we did the "need to have" even through the
apocalypse, staff reduction and all the craziness of 2020! success!
* **what was a challenge?**
* monthly tracking was not practical, and hard to do in
Trac. things are a lot easier with GitLab's dashboard.
* it was hard to work through the pandemic.
* **what can we change?**
* do quarterly-based planning
* estimates were off because so many things happened that we did
not expect. reserve time for the unexpected, reduce expectations.
* ticket triage is rotated now.
## Services survey
We discussed the [survey results analysis][] briefly, and how it is
used as a basis for the roadmap brainstorm. The two major services
people use are GitLab and email, and those will be the focus of the
roadmap for the coming year.
[survey results analysis]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/roadmap/2021#survey-resu…
## Goals for 2021
* email services stabilisation ("submission server", "my email end up
in spam", CiviCRM bounce handling, etc) - consider [outsourcing
email services][]
* gitlab migration continues (Jenkins, gitolite)
* simplify / improve puppet code base
* stabilise services (e.g. gitlab, schleuder)
[outsourcing email services]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/submission#cost
Next steps for the roadmap:
* try to make estimates
* add need to have, nice to have
* anarcat will work on a draft based on the brainstorm
* we meet again in one week to discuss it
# Other discussions
Postponed: metrics services to maintain until we hire new person
# Next meeting
Same time, next week.
# Metrics of the month
Fun fact: we crossed the 2TiB total available memory back in November
2020, almost double from the previous report (in July), even with the
number of hosts in Puppet remained mostly constant (78 vs 72). This is
due (among other things) to the new Cymru Ganeti cluster that added a
whopping 1.2TiB of memory to our infrastructure!
* hosts in Puppet: 82, LDAP: 85, Prometheus exporters: 134
* number of Apache servers monitored: 27, hits per second: 198
* number of Nginx servers: 2, hits per second: 3, hit ratio: 0.86
* number of self-hosted nameservers: 6, mail servers: 12
* pending upgrades: 3, reboots: 0
* average load: 0.29, memory available: 2.00 TiB/2.61 TiB, running
processes: 512
* bytes sent: 265.07 MB/s, received: 155.20 MB/s
* [GitLab tickets][]: 113 tickets including...
* open: 0
* icebox: 91
* backlog: 20
* next: 12
* doing: 10
* (closed: 2165)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
Now also available as the main Grafana dashboard. Head to
<https://grafana.torproject.org/>, change the time period to 30 days,
and wait a while for results to render.
--
Antoine Beaupré
torproject.org system administration