This is your monthly dose of sysadmin meeting minutes! More coming next week, for people really into that stuff... # Roll call: who's there and emergencies All hands present. Dragon died, but situation stable, not requiring us to abort the meeting. # Express check-in We tried a new format for the check-in for our monthly meeting, to speed things up to leave more room for the actual discussions. How are you doing, and are there any blockers? Then pass the mic to the next person. # 2026 Roadmap review This is a copy of the notes from the [TPA meetup][]. Review and amend to get a final version. [TPA meetup]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/meeting/2025-10 Things to add already: - add hardware replacement plan to yearly roadmap, to solve the [manage the lifecycle of systems issue][] - OpenVox packaging [manage the lifecycle of systems issue]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/29304 We split the 2026 roadmap in "must have", "nice to have" and "won't do": ## Must have Recurring: - YEC (@lavamind) - regular upgrades and reboots, and other chores (stars) - no hardware replacements than the ones already planned with tails (dragon etc) Non-recurring: - tails moving to Prometheus, requires TPA prometheus server merge (because we need the space, mostly, @zen) - shift merge, which requires tails moving to prometheus (stars) - email mailboxes (TPA-RFC-45, @groente) - authentication merge phase 1 (after mailboxes, @groente) - completed trixie upgrades (stars) - SVN retirement or migration (@anarcat) - mailman merge (maybe delegate to tails team? @groente can followup) - MinIO migration / conversion to Garage? (@lelutin) - marble on community, blog, and www.tpo websites (@lavamind) - donate-neo CAPTCHA fixes (@anarcat / @lavamind) - TPA-RFC-38 wikis, perhaps just for TPA's wiki for starters? (@anarcat) - OpenVox packaging (@lavamind) ## Nice to have - RFC reform (maybe already done in 2025, @anarcat) - firewall merge, requires TPA and Tails to migrate to nftables (@zen) - Tails websites merge - Tails mirror coordination (postpone to 2027?) - Tails DNS merge - Tails TLS merge - (TPA?) in-person meeting (@anarcat) - reform deb.tpo, further idea for a roadmap to fix the tor debian package (@lelutin / @lavamind, filed as [tpo/tpa/team#42374][]) [tpo/tpa/team#42374]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/42374 Let's move that deb.tpo item list to an epic or issue. ## Won't do - backups merge (postponed to 2027) ## Observations - lots of stuff, hard to tell whether we'll be able to pull it off - we assigned names, but that's flexible - we don't know exactly when those things will be done, will be allocated in quarterly reviews - this is our wishlist, we need to get feedback from other teams, web team and perhaps team leads / ops meeting coming up about that # holidays vacation planning - zen AFK Jan 5 - 23 (3 weeks) - zen takes the two weeks holidays for tails - lelutin and lavamind share them for TPA - vacation calendar currently lost, but TPO closing weeks expected to be from dec 22nd to jan 2nd - announce your AFK times and add them to the calendar! # skill-share proposals We talked about doing skill-shares/trainings/presentations at our meetup. We still don't know when: during office hours, after check-ins? - Offer (zen): Tails Translation Platform setup (i.e. weblate + staging website + integration scripts) "What's new in TPA" kind of billboard. Presenter decides if it's mandatory, if it is, make it part of the regular meeting schedule. # RFC to ADR conversion Short presentation of the [ADR-95 proposal][]. [ADR-95 proposal]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/0095-adr postponed # long term (2030) roadmap - review the tails merge roadmap - what's next for tpa? postponed # Next meeting Next week, to tackle the other two conversations we skipped above. # Metrics of the month * host count: 99 * number of Apache servers monitored: 33, hits per second: 705 * number of self-hosted nameservers: 6, mail servers: 12 * pending upgrades: 0, reboots: 0 * average load: 1.98, memory available: 4.4 TB/7.2 TB, running processes: 294 * disk free/total: 122.4 TB/228.4 TB * bytes sent: 545.6 MB/s, received: 354.9 MB/s * [GitLab tickets][]: 249 tickets including... * open: 0 * ~Roadmap::Icebox: 128 * ~Roadmap::Future: 42 * ~Needs Information: 3 * ~Roadmap::Backlog: 41 * ~Roadmap::Next: 20 * ~Roadmap::Doing: 12 * ~Needs Review: 4 * (closed: 4277) * [~Technical Debt][]: 12 open, 39 closed [Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards [~Technical Debt]: https://gitlab.torproject.org/groups/tpo/tpa/-/issues/?state=opened&label_name[]=Technical%20Debt Upgrade prediction graph lives at https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/trixie/ Now also available as the main Grafana dashboard. Head to <https://grafana.torproject.org/>, change the time period to 30 days, and wait a while for results to render. -- Antoine Beaupré torproject.org system administration
On 2025-11-10 15:56:01, Antoine Beaupré via tor-project wrote:
This is your monthly dose of sysadmin meeting minutes! More coming next week, for people really into that stuff...
And here's the extra! # Roll call: who's there and emergencies all hands present # Express check-in How are you doing, and are there any blockers? Then pass the mic to the next person. # Server decision - decisions - 3 supermicro servers instead of 2 lenovos (amd, newer arch, but lower single threaded performance) - converging over those specifications: - memory: 128GB DDR5 4800 ECC - CPU: EPYC 4484P - disks: - 2xM2 2TB - **2x2.5" 8TB** (this is larger than the current specs) - frame/board: supermicro AS-1015A-MT - which colo? - graber's personal colo? - next steps - questions for graber - space for 3U? - can we go when he's on holiday? - get numbers from elco: - ETA - price - ask for 2 different brands or batches of disks? - make sure to double the size of sata disks (see above) - get approval from accounting using elco and HIVE numbers - decide on which colo - order from elco, shipping to colo - draw the rest of the fucking owl # RFC to ADR conversion Short presentation of the [ADR-100 proposal](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/0100-adr). Feedback: - good change - good to separate things in multiple documents - should they be mutable? - anarcat worried about losing history in the object-storage RFC, but lelutin doesn't feel that's an issue - lavamind would prefer to keep proposals immutable, because it can be hard to dig back in history, could be overlooked if kept only in git, feels strange to modify RFCs, worried about internal consistency - ADR process includes a "superseded" state next steps: - keep ADRs immutable, apart from small changes - two more ADRs for deliberations and comms - file all of those together? # long term (2030) roadmap - review the tails merge roadmap - what's next for tpa? postponed to December # Next meeting In two weeks, December 1st. -- Antoine Beaupré torproject.org system administration
participants (1)
-
Antoine Beaupré