[tor-bugs] #29796 [Internal Services/Tor Sysadmin Team]: synchronize puppet and LDAP hosts

[Tor Bug Tracker & Wiki]

#29796: synchronize puppet and LDAP hosts
-----------------------------------------------------+-----------------
     Reporter:  anarcat                              |      Owner:  tpa
         Type:  defect                               |     Status:  new
     Priority:  Medium                               |  Milestone:
    Component:  Internal Services/Tor Sysadmin Team  |    Version:
     Severity:  Normal                               |   Keywords:
Actual Points:                                       |  Parent ID:
       Points:                                       |   Reviewer:
      Sponsor:                                       |
-----------------------------------------------------+-----------------
 We have hosts that are in Puppet and not in LDAP and vice versa. Every
 host in LDAP should be in Puppet and vice versa.

 This is the current diff:

 {{{
 $ diff puppet ldap
 29a30,31
 > geyeri.torproject.org
 > gillii.torproject.org
 36d37
 < hyalinum.torproject.org
 74a76,78
 > weissii.torproject.org
 > winklerianum.torproject.org
 > woronowii.torproject.org
 }}}

 That is, right now, we have the following hosts in LDAP but not in Puppet:

  * geyeri.torproject.org
  * gillii.torproject.org
  * weissii.torproject.org
  * winklerianum.torproject.org
  * woronowii.torproject.org

 The following is in Puppet, but not LDAP:

  * hyalinum.torproject.org

 The two lists (`puppet` and `ldap`) were obtain using the following
 commands:

 {{{
 ssh -t pauli.torproject.org 'sudo -u postgres psql puppetdb -P pager=off
 -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS NULL"'
 | tee puppet
 tail -n +2 puppet | sort | sponge puppet
 ssh alberti.torproject.org 'ldapsearch -h db.torproject.org -x -ZZ -b
 dc=torproject,dc=org -LLL "hostname=*.torproject.org" hostname | awk "\$1
 == \"hostname:\" {print \$2}" | sort' > ldap
 }}}

 ... as detailed in the [https://help.torproject.org/tsa/howto/puppet/ new
 Puppet docs].

 I'm not exactly sure how to resolve this. When weasel saw a previous
 version of this list, he said:

 {{{
 12:30:00 <weasel> from a quick glance, all but the arm hosts can go.
 12:30:06 <weasel> best to double-check with ldap.
 12:30:19 <weasel> if they are not in ldap, and they haven't done a puppet
 run in a while, they should be removed from puppet also.
 12:30:45 <weasel> gillii and geyeri are the old CRM hosts.  I think linus
 wants to kill them soon but maybe keep them around (and offline) for now.
 }}}

 According to nagios, hyalinum has not checked into Puppet since
 2018-02-12T08:53:13.339Z, over a month ago. So presumably that should be
 removed from puppet, and we should double-check the retirement procedure
 to see if it was completed correctly.

 The hosts in LDAP and not in Puppet should probably be added to puppet,
 carefully (--noop is your friend) to see if it breaks anything.

 In the future, we might want to add a Nagios check on the Puppet server to
 make sure this is synchronized.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29796>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online