[tor-bugs] #34185 [Internal Services/Tor Sysadmin Team]: ganeti clusters don't like automatic upgrades

[Tor Bug Tracker & Wiki]

#34185: ganeti clusters don't like automatic upgrades
-------------------------------------------------+-------------------------
 Reporter:  anarcat                              |          Owner:  hiro
     Type:  defect                               |         Status:
                                                 |  assigned
 Priority:  High                                 |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Major                                |     Resolution:
 Keywords:  tpa-roadmap-may                      |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by hiro):

 Openvswitch was updated together with the following group of packages:

 {{{
 2020-05-10 06:12:53,754 INFO Packages that will be upgraded: base-files
 distro-info-data iputils-arping iputils-ping iputils-tracepath
 libbrlapi0.6 libfuse2 l
 ibpam-systemd libsystemd0 libudev1 linux-compiler-gcc-8-x86 linux-headers-
 amd64 linux-image-amd64 linux-kbuild-4.19 openvswitch-common openvswitch-
 switch post
 fix postfix-cdb rake rubygems-integration systemd systemd-sysv tzdata udev
 }}}

 Checking openvswitch status it has not been restarted since the 10th of
 may:

 {{{
 Loaded: loaded (/lib/systemd/system/openvswitch-switch.service; enabled;
 vendor preset: enabled)
    Active: active (exited) since Sun 2020-05-10 14:05:11 UTC; 2 weeks 3
 days ago

 }}}

 And from the log on that day I actually see it died twice:

 2020-05-10T06:13:16.534Z|00003|vlog(monitor)|INFO|opened log file
 /var/log/openvswitch/ovs-vswitchd.log
 2020-05-10T06:13:16.534Z|00004|daemon_unix(monitor)|INFO|pid 3211 died,
 exit status 0, exiting
 2020-05-10T06:13:16.787Z|00001|vlog|INFO|opened log file
 /var/log/openvswitch/ovs-vswitchd.log
 2020-05-10T06:13:16.788Z|00002|ovs_numa|INFO|Discovered 12 CPU cores on
 NUMA node 0
 2020-05-10T06:13:16.788Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and
 12 CPU cores
 2020-05-10T06:13:16.788Z|00004|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connecting...
 2020-05-10T06:13:16.788Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connected
 2020-05-10T06:13:16.791Z|00006|bridge|INFO|ovs-vswitchd (Open vSwitch)
 2.10.1
 2020-05-10T06:13:17.332Z|00002|daemon_unix(monitor)|INFO|pid 29781 died,
 exit status 0, exiting
 2020-05-10T06:13:17.621Z|00001|vlog|INFO|opened log file
 /var/log/openvswitch/ovs-vswitchd.log
 2020-05-10T06:13:17.623Z|00002|ovs_numa|INFO|Discovered 12 CPU cores on
 NUMA node 0
 2020-05-10T06:13:17.623Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and
 12 CPU cores
 2020-05-10T06:13:17.623Z|00004|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connecting...
 2020-05-10T06:13:17.623Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connected
 2020-05-10T06:13:17.630Z|00006|bridge|INFO|ovs-vswitchd (Open vSwitch)
 2.10.1

 ...

 2020-05-10T14:02:23.078Z|00036|bridge|INFO|bridge br0: using datapath ID
 00007eb83553f345
 2020-05-10T14:02:23.398Z|00037|bridge|INFO|bridge br0: deleted interface
 br0 on port 65534
 2020-05-10T14:02:23.578Z|00002|daemon_unix(monitor)|INFO|pid 29951 died,
 exit status 0, exiting
 2020-05-10T14:05:05.241Z|00001|vlog|INFO|opened log file
 /var/log/openvswitch/ovs-vswitchd.log
 2020-05-10T14:05:05.247Z|00002|ovs_numa|INFO|Discovered 12 CPU cores on
 NUMA node 0
 2020-05-10T14:05:05.247Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and
 12 CPU cores
 2020-05-10T14:05:05.247Z|00004|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connecting...
 2020-05-10T14:05:05.247Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
 connected
 2020-05-10T14:05:05.250Z|00006|bridge|INFO|ovs-vswitchd (Open vSwitch)
 2.10.1
 2020-05-10T14:05:09.384Z|00007|ofproto_dpif|INFO|system at ovs-system:
 Datapath supports recirculation
 2020-05-10T14:05:09.384Z|00008|ofproto_dpif|INFO|system at ovs-system: VLAN
 header stack length probed as 2
 2020-05-10T14:05:09.384Z|00009|ofproto_dpif|INFO|system at ovs-system: MPLS
 label stack length probed as 1
 2020-05-10T14:05:09.384Z|00010|ofproto_dpif|INFO|system at ovs-system:
 Datapath supports truncate action
 2020-05-10T14:05:09.384Z|00011|ofproto_dpif|INFO|system at ovs-system:
 Datapath supports unique flow ids


 {{{

 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/34185#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online