[tor-bugs] #29387 [Internal Services/Tor Sysadmin Team]: Publish our puppet repository

[Tor Bug Tracker & Wiki]

#29387: Publish our puppet repository
-------------------------------------------------+-------------------------
 Reporter:  ln5                                  |          Owner:  anarcat
     Type:  task                                 |         Status:
                                                 |  assigned
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by anarcat):

 so concretely, the '''TL;DR:''' of what I am proposing is this:

  1. convert everything to hiera (#30020) - this requires creating `roles`
 for each machine (more or less)
  2. move current `modules/` into `profiles/` and audit for private data
  3. move any private data into `hiera/`
  4. move `3rdparty` modules into `modules/`
  5. publish everything but `hiera/` as a new repository

 '''Final picture'''

 Once this is done, the final picture will look like this in `/etc/puppet`:

  * `hiera/` - private data. `machine -> role` assignements, secret stuff
 like the alias file, machine location, price and other similar metadata
 and details (see also #29816)
  * `modules/` - equivalent of the current `3rdparty/` directory: fully
 public, reusable code that's aimed at collaboration. mostly code from the
 Puppet forge or our own repository if no equivalent there
  * `profiles/` - magic sauce on top of 3rd party `modules/`, already
 created a few `modules/profiles/` for grafana and prometheus, the profiles
 configure official 3rd party classes with our site-specific criteria
  * `roles/` - abstract classes that regroup a few profiles. for example
 `roles::monitoring` could currently include `profiles::nagiosmaster`,
 `profiles::prometheus::server` and `profiles::grafana` as an
 implementation

 This could all be done in the current repository, without creating a new
 clean history one, but it would prepare us for that final step. And that
 step would simply be to move `modules/`, `profiles/`, and `roles/` into a
 public repository, while keeping `hiera/` private in its own repository.

 '''Alternative proposal'''

 The alternative approach is simply to create an entirely new repository
 that is identical to the current one, minus the `virtual` aliases file.
 But then I don't know where I would put the alias file, and I think it
 would be a missed opportunity to follow the industry's best practices I
 documented earlier in this ticket.

 '''Further discussion'''

 I would love to get feedback on this before I foray any further into this
 maze. For now I think it's safe to keep going on the Hiera conversion, as
 I discussed this with weasel and it seems to be consensual. But it seems
 the other ideas here (namely to use this opportunity to reshuffle the
 repository structure) seem to be less consensual.

 Also note that I kept trocla out of the picture for now. We could keep
 using the current `hkdf` in this system, but it would be the last function
 left in the `puppetmaster` module, from what I can tell, which is another
 reason why I'm tempted to replace it as well.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29387#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online