[tor-bugs] #29387 [Internal Services/Tor Sysadmin Team]: Publish our puppet repository

[Tor Bug Tracker & Wiki]

#29387: Publish our puppet repository
-------------------------------------------------+-------------------------
 Reporter:  ln5                                  |          Owner:  tpa
     Type:  task                                 |         Status:
                                                 |  assigned
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by anarcat):

 From what i understand, there *are* (very few but still) sensitive parts
 to the repo, so we'll have to create a new history. If we're going to do
 that, I'd like to take that opportunity to rearchitecture the repository a
 little bit. There are standards on how to organize puppet code and,
 because of its old history, the current repository is slightly out of sync
 with those.

 For example, we store "3rd party modules" in `3rdparty`. Those usually go
 in `modules` instead. And what we have in `modules` often look more like
 `profiles` than `modules`. A more complete introduction and documentation
 for those ideas is better described here:

 https://puppet.com/docs/pe/2017.2/r_n_p_intro.html

 Similarly, we use a "monorepo" approach for now which makes things much
 simpler, but really complicates collaboration with external, public
 modules. I've been bitten by this trying to patch the Prometheus module,
 for example - it's been quite difficult to have local changes to the 3rd
 party module as I had to commit the changes in our repo, then ''copy''
 those changes to an external checkout of the repository. I ended up with a
 hybrid approach of having the 3rd party module checked out under
 `3rdparty/modules/prometheus/.git` while ''at the same time'' added to the
 parent `.git` repo, which means I need to commit changes ''twice'' for
 changes to propagate, which isn't much better than copying stuff around.

 Most people seem to be using either r10k, librarian and/or code manager to
 handle that problem. There's upstream documentation on how to configure
 this here:

 https://puppet.com/docs/pe/2017.2/cmgmt_managing_code.html

 Improving this setup would also allow us to bootstrap a new puppetmaster
 more easily which, in turn, might allow us to do continuous integration
 (CI) on the Puppet setup, which would reduce a lot of "YOLO" commits we
 often have to do on the puppet repo because we can't test changes locally.

 I know this opens a broader range of things to do, but I figured it was a
 good opportunity to bring it up. Besides, I doubt the repository in its
 current form will encourage much collaboration if it's non-standard. If we
 adopt community practices, we will be able to collaborate much more than
 with the current approach which is, after all, the objective of sharing
 that code...

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29387#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online