[tor-bugs] #29796 [Internal Services/Tor Sysadmin Team]: synchronize puppet and LDAP hosts

Tor Bug Tracker & Wiki blackhole at torproject.org
Tue Mar 19 14:31:27 UTC 2019


#29796: synchronize puppet and LDAP hosts
-------------------------------------------------+---------------------
 Reporter:  anarcat                              |          Owner:  tpa
     Type:  defect                               |         Status:  new
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+---------------------
Description changed by anarcat:

Old description:

> We have hosts that are in Puppet and not in LDAP and vice versa. Every
> host in LDAP should be in Puppet and vice versa.
>
> This is the current diff:
>
> {{{
> $ diff puppet ldap
> 29a30,31
> > geyeri.torproject.org
> > gillii.torproject.org
> 36d37
> < hyalinum.torproject.org
> 74a76,78
> > weissii.torproject.org
> > winklerianum.torproject.org
> > woronowii.torproject.org
> }}}
>
> That is, right now, we have the following hosts in LDAP but not in
> Puppet:
>
>  * geyeri.torproject.org
>  * gillii.torproject.org
>  * weissii.torproject.org
>  * winklerianum.torproject.org
>  * woronowii.torproject.org
>
> The following is in Puppet, but not LDAP:
>
>  * hyalinum.torproject.org
>
> The two lists (`puppet` and `ldap`) were obtain using the following
> commands:
>
> {{{
> ssh -t pauli.torproject.org 'sudo -u postgres psql puppetdb -P pager=off
> -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS
> NULL"' | tee puppet
> tail -n +2 puppet | sort | sponge puppet
> ssh alberti.torproject.org 'ldapsearch -h db.torproject.org -x -ZZ -b
> dc=torproject,dc=org -LLL "hostname=*.torproject.org" hostname | awk "\$1
> == \"hostname:\" {print \$2}" | sort' > ldap
> }}}
>
> ... as detailed in the [https://help.torproject.org/tsa/howto/puppet/ new
> Puppet docs].
>
> I'm not exactly sure how to resolve this. When weasel saw a previous
> version of this list, he said:
>
> {{{
> 12:30:00 <weasel> from a quick glance, all but the arm hosts can go.
> 12:30:06 <weasel> best to double-check with ldap.
> 12:30:19 <weasel> if they are not in ldap, and they haven't done a puppet
> run in a while, they should be removed from puppet also.
> 12:30:45 <weasel> gillii and geyeri are the old CRM hosts.  I think linus
> wants to kill them soon but maybe keep them around (and offline) for now.
> }}}
>
> According to nagios, hyalinum has not checked into Puppet since
> 2018-02-12T08:53:13.339Z, over a month ago. So presumably that should be
> removed from puppet, and we should double-check the retirement procedure
> to see if it was completed correctly.
>
> The hosts in LDAP and not in Puppet should probably be added to puppet,
> carefully (--noop is your friend) to see if it breaks anything.
>
> In the future, we might want to add a Nagios check on the Puppet server
> to make sure this is synchronized.

New description:

 We have hosts that are in Puppet and not in LDAP and vice versa. Every
 host in LDAP should be in Puppet and vice versa.

 We have 78 hosts in LDAP and 74 in Puppet, with 73 hosts in common. This
 is the current diff:

 {{{
 $ diff puppet ldap
 29a30,31
 > geyeri.torproject.org
 > gillii.torproject.org
 36d37
 < hyalinum.torproject.org
 74a76,78
 > weissii.torproject.org
 > winklerianum.torproject.org
 > woronowii.torproject.org
 }}}

 That is, right now, we have the following hosts in LDAP but not in Puppet:

  * geyeri.torproject.org
  * gillii.torproject.org
  * weissii.torproject.org
  * winklerianum.torproject.org
  * woronowii.torproject.org

 The following is in Puppet, but not LDAP:

  * hyalinum.torproject.org

 The two lists (`puppet` and `ldap`) were obtain using the following
 commands:

 {{{
 ssh -t pauli.torproject.org 'sudo -u postgres psql puppetdb -P pager=off
 -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS NULL"'
 | tee puppet
 tail -n +2 puppet | sort | sponge puppet
 ssh alberti.torproject.org 'ldapsearch -h db.torproject.org -x -ZZ -b
 dc=torproject,dc=org -LLL "hostname=*.torproject.org" hostname | awk "\$1
 == \"hostname:\" {print \$2}" | sort' > ldap
 }}}

 ... as detailed in the [https://help.torproject.org/tsa/howto/puppet/ new
 Puppet docs].

 I'm not exactly sure how to resolve this. When weasel saw a previous
 version of this list, he said:

 {{{
 12:30:00 <weasel> from a quick glance, all but the arm hosts can go.
 12:30:06 <weasel> best to double-check with ldap.
 12:30:19 <weasel> if they are not in ldap, and they haven't done a puppet
 run in a while, they should be removed from puppet also.
 12:30:45 <weasel> gillii and geyeri are the old CRM hosts.  I think linus
 wants to kill them soon but maybe keep them around (and offline) for now.
 }}}

 According to nagios, hyalinum has not checked into Puppet since
 2018-02-12T08:53:13.339Z, over a month ago. So presumably that should be
 removed from puppet, and we should double-check the retirement procedure
 to see if it was completed correctly.

 The hosts in LDAP and not in Puppet should probably be added to puppet,
 carefully (--noop is your friend) to see if it breaks anything.

 In the future, we might want to add a Nagios check on the Puppet server to
 make sure this is synchronized.

--

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29796#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list