Hello!
I'm writing to inform you that TPA's been moving on something that should make it possible to eventually merge both of the prometheus / grafana servers: we're moving towards per-user passwords that need to be set through LDAP and deprecating the shared passwords.
I would invite everyone who needs to continue accessing prometheus and grafana (either on prometheus1.tpo or prometheus2.tpo) to head over to https://db.torproject.org/ , login and set yourself a password in the "Web password" field. That field was previously hidden but it was recently added to the form.
The password will take some time to get synchronized to the servers, so allow some 1 to 2 hours before you test out your new credentials. When ready, head over to https://grafana2.torproject.org/ and use your ldap username with your new web password to confirm that you're able to login there.
The shared passwords that are currently in use are bound to be removed on April 17th.
If you're having issues with setting up and/or using your own credentials to access prometheus and grafana, please contact TPA and we'll take a look with you into what's happening.
Cheers!
additional note for TPA members: we now have a fallback password that's present in our password manager. it should let us access the monitoring sites even if ldap has a disruption. you can try that one as well.
On 2025-04-01 16:41:55, Gabriel Filion wrote:
[...]
Some precisions here, if I may. :)
The delay is normally between 5 to 15 minutes:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#know-when-will...
The shared passwords that are currently in use are bound to be removed on April 17th.
Specifically, here we're talking about the "metrics" Grafana (and Prometheus!) user you folks have been sharing around with each other. I don't know exactly how far it was shared, but you need to, now, let those people know that this password will stop working soon.
Also be aware, as announced in TPA-RFC-33, that this effectively gives access to all users with an LDAP account to the Prometheus server. When we last discussed this, it was okayed by people here, but it's still time to review that policy. It would be more complicated for us because we'd need to grant access on a username basis (like "hiro can login") but it's possible.
[...]
For TPA folks, that's in services/prometheus.torproject.org. You *should* use your "web password" to operate prometheus on a daily basis however: for now, you can write that in your .netrc file (like we do for the KGB bot password), but I have plans to hook that up to pass(1) instead.
Let me know if you think that should be expedited.
a.
On 2025-04-01 17:15:58, Antoine Beaupré wrote:
Another thing that came up is that we didn't realize this, but the all-mighty "metrics" user has admin privileges! Your new user in Grafana will *not* have such privileges. We have not thought about that aspect *at all*, so I guess, for now, we'll just transitively grant you admin access on the org?
So far, i've given geko access, if others need access, let us know and we'll promote your user. We can't proactively do so, you need to login with your user first.
A.
Quoting Antoine Beaupré via anti-censorship-team (2025-04-02 18:35:56)
The people in the anti-censorship team needs to be able to edit the anti-censorship dashboards in grafana. I guess it can be done with some non-admin rights.
Thank you.
On 2025-04-02 19:42:16, meskio wrote:
Right, that's an issue!
I've hacked something together with "Teams", see this comment in GitLab for details:
https://gitlab.torproject.org/tpo/tpa/team/-/issues/41636#note_3184282
a.
anti-censorship-team@lists.torproject.org
-
Antoine Beaupré -
Gabriel Filion -
meskio