[tor-bugs] #32937 [Internal Services/Tor Sysadmin Team]: install a new node in the gnt-fsn cluster (fsn-node-03)

Tor Bug Tracker & Wiki blackhole at torproject.org
Wed Jan 22 15:44:54 UTC 2020


#32937: install a new node in the gnt-fsn cluster (fsn-node-03)
-------------------------------------------------+-------------------------
 Reporter:  anarcat                              |          Owner:  anarcat
     Type:  project                              |         Status:
                                                 |  accepted
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by anarcat):

 worked on the setup-storage config, which currently looks like this:

 {{{
 # open questions
 # --align=optimal?
 # leave keys in /tmp/fai or specify passphrase?
 # use sameas: to set all disk names earlier?
 # bios_grub flag?

 disk_config nvme0n1 disklabel:gpt bootable:2
 # bios grub second stage
 primary -       8MiB    -       -
 # /boot
 primary -       512MiB  -       -
 # rest is RAID+LUKS+LVM
 primary -       0-      -       -

 disk_config nvme1n1 disklabel:gpt bootable:2
 # same as above
 primary -       8MiB    -       -
 primary -       512MiB  -       -
 primary -       0-      -       -

 disk_config sda disklabel:gpt
 primary -       0-      -       -

 disk_config sdb disklabel:gpt
 primary -       0-      -       -

 disk_config raid fstabkey:uuid
 raid1   /boot   nvme0n1p2,nvme1n1p2     ext4    rw,noatime,errors=remount-
 ro
 raid1   -       nvme0n1p3,nvme1n1p3     -       -
 raid1   -       sda1,sdb1       -       -

 # FAI defaults to -c aes-xts-plain64 -s 256
 disk_config cryptsetup
 luks    -       /dev/md1        -       -
 luks    -       /dev/md2        -       -

 disk_config lvm fstabkey:uuid
 # previous convention was "vg_$hostname"
 vg      vg_nvme crypt_format_md1
 vg_nvme-root    /       30G     ext4    rw
 vg_nvme-swap    swap    1G      swap    sw
 vg      vg_hdd  crypt_format_md2

 # HDD disks config intentionally left blank

 }}}

 and fails like this:

 {{{
 root at rescue ~ # setup-storage -f setup-storage-fsn-node-3 -d -X
 disklist: nvme0n1
 nvme1n1
 sda
 sdb
 Starting setup-storage 2.2
 Using config file: setup-storage-fsn-node-3
 Input was:
 # open questions
 # --align=optimal?
 # leave keys in /tmp/fai or specify passphrase?
 # use sameas: to set all disk names earlier?
 # bios_grub flag?

 disk_config nvme0n1 disklabel:gpt bootable:2
 # bios grub second stage
 primary -       8MiB    -       -
 # /boot
 primary -       512MiB  -       -
 # rest is RAID+LUKS+LVM
 primary -       0-      -       -

 disk_config nvme1n1 disklabel:gpt bootable:2
 # same as above
 primary -       8MiB    -       -
 primary -       512MiB  -       -
 primary -       0-      -       -

 disk_config sda disklabel:gpt
 primary -       0-      -       -

 disk_config sdb disklabel:gpt
 primary -       0-      -       -

 disk_config raid fstabkey:uuid
 raid1   /boot   nvme0n1p2,nvme1n1p2     ext4    rw,noatime,errors=remount-
 ro
 raid1   -       nvme0n1p3,nvme1n1p3     -       -
 raid1   -       sda1,sdb1       -       -

 # FAI defaults to -c aes-xts-plain64 -s 256
 disk_config cryptsetup
 luks    -       /dev/md1        -       -
 luks    -       /dev/md2        -       -

 disk_config lvm fstabkey:uuid
 # previous convention was "vg_$hostname"
 vg      vg_nvme crypt_format_md1
 vg_nvme-root    /       30G     ext4    rw
 vg_nvme-swap    swap    1G      swap    sw
 vg      vg_hdd  crypt_format_md2

 # HDD disks config intentionally left blank
 (CMD) parted -s /dev/nvme0n1 unit TiB print 1> /tmp/vg8ajSauDw 2>
 /tmp/vMMazFsqO7
 Executing: parted -s /dev/nvme0n1 unit TiB print
 (STDERR) Error: /dev/nvme0n1: unrecognised disk label
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme0n1: 0.87TiB
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: unknown
 (STDOUT) Disk Flags:
 Parted could not read a disk label (new disk?)
 (CMD) parted -s /dev/nvme0n1 mklabel gpt 1> /tmp/MK287MFjgl 2>
 /tmp/55bOjMK73A
 Executing: parted -s /dev/nvme0n1 mklabel gpt
 (CMD) parted -s /dev/nvme0n1 unit TiB print 1> /tmp/NuP83_L6et 2>
 /tmp/kubLOQr9Wo
 Executing: parted -s /dev/nvme0n1 unit TiB print
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme0n1: 0.87TiB
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start  End  Size  File system  Name  Flags
 (STDOUT)
 (CMD) parted -s /dev/nvme0n1 unit B print free 1> /tmp/FN2KfViTkm 2>
 /tmp/y2Pytod_rd
 Executing: parted -s /dev/nvme0n1 unit B print free
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme0n1: 960197124096B
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End            Size           File system  Name
 Flags
 (STDOUT)         17408B  960197107199B  960197089792B  Free Space
 (STDOUT)
 (CMD) parted -s /dev/nvme0n1 unit chs print free 1> /tmp/1IjsTMlqdY 2>
 /tmp/lU63MMaJBC
 Executing: parted -s /dev/nvme0n1 unit chs print free
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme0n1: 116737,80,62
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) BIOS cylinder,head,sector geometry: 116737,255,63.  Each cylinder
 is 8225kB.
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End           File system  Name  Flags
 (STDOUT)         0,0,34  116737,80,29  Free Space
 (STDOUT)
 (CMD) parted -s /dev/nvme1n1 unit TiB print 1> /tmp/QjjsC5zar2 2>
 /tmp/7UcTC3hxOf
 Executing: parted -s /dev/nvme1n1 unit TiB print
 (STDERR) Error: /dev/nvme1n1: unrecognised disk label
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme1n1: 0.87TiB
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: unknown
 (STDOUT) Disk Flags:
 Parted could not read a disk label (new disk?)
 (CMD) parted -s /dev/nvme1n1 mklabel gpt 1> /tmp/NcxnmAx4H4 2>
 /tmp/GHflvDI9cH
 Executing: parted -s /dev/nvme1n1 mklabel gpt
 (CMD) parted -s /dev/nvme1n1 unit TiB print 1> /tmp/F_ioSuepVF 2>
 /tmp/RGHrXNq3ax
 Executing: parted -s /dev/nvme1n1 unit TiB print
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme1n1: 0.87TiB
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start  End  Size  File system  Name  Flags
 (STDOUT)
 (CMD) parted -s /dev/nvme1n1 unit B print free 1> /tmp/JMHhWwaRE4 2>
 /tmp/3yByvQ8pC0
 Executing: parted -s /dev/nvme1n1 unit B print free
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme1n1: 960197124096B
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End            Size           File system  Name
 Flags
 (STDOUT)         17408B  960197107199B  960197089792B  Free Space
 (STDOUT)
 (CMD) parted -s /dev/nvme1n1 unit chs print free 1> /tmp/DvwVu2WqiU 2>
 /tmp/79TMfEZhI3
 Executing: parted -s /dev/nvme1n1 unit chs print free
 (STDOUT) Model: SAMSUNG MZQLB960HAJR-00007 (nvme)
 (STDOUT) Disk /dev/nvme1n1: 116737,80,62
 (STDOUT) Sector size (logical/physical): 512B/512B
 (STDOUT) BIOS cylinder,head,sector geometry: 116737,255,63.  Each cylinder
 is 8225kB.
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End           File system  Name  Flags
 (STDOUT)         0,0,34  116737,80,29  Free Space
 (STDOUT)
 (CMD) parted -s /dev/sda unit TiB print 1> /tmp/JIkqtOkUuP 2>
 /tmp/3EiOFTm7w9
 Executing: parted -s /dev/sda unit TiB print
 (STDERR) Error: /dev/sda: unrecognised disk label
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sda: 9.10TiB
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: unknown
 (STDOUT) Disk Flags:
 Parted could not read a disk label (new disk?)
 (CMD) parted -s /dev/sda mklabel gpt 1> /tmp/P3Ze3gJlhk 2> /tmp/MWTeWjTArm
 Executing: parted -s /dev/sda mklabel gpt
 (CMD) parted -s /dev/sda unit TiB print 1> /tmp/YUdeL80vX3 2>
 /tmp/TW2EWrJtu0
 Executing: parted -s /dev/sda unit TiB print
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sda: 9.10TiB
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start  End  Size  File system  Name  Flags
 (STDOUT)
 (CMD) parted -s /dev/sda unit B print free 1> /tmp/SHw1gM3OgI 2>
 /tmp/YyrsyIdxOz
 Executing: parted -s /dev/sda unit B print free
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sda: 10000831348736B
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End              Size             File system
 Name  Flags
 (STDOUT)         17408B  10000831331839B  10000831314432B  Free Space
 (STDOUT)
 (CMD) parted -s /dev/sda unit chs print free 1> /tmp/7jB9P7heVI 2>
 /tmp/YA8IORj9eZ
 Executing: parted -s /dev/sda unit chs print free
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sda: 1215865,39,45
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) BIOS cylinder,head,sector geometry: 1215865,255,63.  Each
 cylinder is 8225kB.
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End            File system  Name  Flags
 (STDOUT)         0,0,34  1215865,39,12  Free Space
 (STDOUT)
 (CMD) parted -s /dev/sdb unit TiB print 1> /tmp/xSDW4y246S 2>
 /tmp/4uJcTQmrF5
 Executing: parted -s /dev/sdb unit TiB print
 (STDERR) Error: /dev/sdb: unrecognised disk label
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sdb: 9.10TiB
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: unknown
 (STDOUT) Disk Flags:
 Parted could not read a disk label (new disk?)
 (CMD) parted -s /dev/sdb mklabel gpt 1> /tmp/sokuCJH_uw 2> /tmp/YpCDmAqg0m
 Executing: parted -s /dev/sdb mklabel gpt
 (CMD) parted -s /dev/sdb unit TiB print 1> /tmp/chgQkdwPwZ 2>
 /tmp/L70AY4kmhr
 Executing: parted -s /dev/sdb unit TiB print
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sdb: 9.10TiB
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start  End  Size  File system  Name  Flags
 (STDOUT)
 (CMD) parted -s /dev/sdb unit B print free 1> /tmp/H6cCqPfSjv 2>
 /tmp/B6Q2Il5Tgz
 Executing: parted -s /dev/sdb unit B print free
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sdb: 10000831348736B
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End              Size             File system
 Name  Flags
 (STDOUT)         17408B  10000831331839B  10000831314432B  Free Space
 (STDOUT)
 (CMD) parted -s /dev/sdb unit chs print free 1> /tmp/v5U6BwCVbR 2>
 /tmp/G3R5gZBg0D
 Executing: parted -s /dev/sdb unit chs print free
 (STDOUT) Model: ATA TOSHIBA MG06ACA1 (scsi)
 (STDOUT) Disk /dev/sdb: 1215865,39,45
 (STDOUT) Sector size (logical/physical): 512B/4096B
 (STDOUT) BIOS cylinder,head,sector geometry: 1215865,255,63.  Each
 cylinder is 8225kB.
 (STDOUT) Partition Table: gpt
 (STDOUT) Disk Flags:
 (STDOUT)
 (STDOUT) Number  Start   End            File system  Name  Flags
 (STDOUT)         0,0,34  1215865,39,12  Free Space
 (STDOUT)
     No volume groups found.
 (CMD) mdadm --examine --scan --verbose -c partitions 1> /tmp/hgUrKMw3a3 2>
 /tmp/HxEzhckhyV
 Executing: mdadm --examine --scan --verbose -c partitions
 Current disk layout
 $VAR1 = {
           '/dev/sdb' => {
                           'disklabel' => 'gpt',
                           'bios_cylinders' => '1215865',
                           'begin_byte' => 0,
                           'bios_sectors_per_track' => '63',
                           'partitions' => {},
                           'bios_heads' => '255',
                           'sector_size' => '512',
                           'end_byte' => '10000831348735',
                           'size' => '10000831348736'
                         },
           '/dev/nvme0n1' => {
                               'end_byte' => '960197124095',
                               'size' => '960197124096',
                               'bios_cylinders' => '116737',
                               'begin_byte' => 0,
                               'disklabel' => 'gpt',
                               'sector_size' => '512',
                               'partitions' => {},
                               'bios_sectors_per_track' => '63',
                               'bios_heads' => '255'
                             },
           '/dev/nvme1n1' => {
                               'end_byte' => '960197124095',
                               'size' => '960197124096',
                               'bios_cylinders' => '116737',
                               'begin_byte' => 0,
                               'disklabel' => 'gpt',
                               'bios_heads' => '255',
                               'bios_sectors_per_track' => '63',
                               'partitions' => {},
                               'sector_size' => '512'
                             },
           '/dev/sda' => {
                           'end_byte' => '10000831348735',
                           'size' => '10000831348736',
                           'disklabel' => 'gpt',
                           'bios_cylinders' => '1215865',
                           'begin_byte' => 0,
                           'bios_sectors_per_track' => '63',
                           'partitions' => {},
                           'bios_heads' => '255',
                           'sector_size' => '512'
                         }
         };
 Current LVM layout
 $VAR1 = {};
 Current RAID layout
 $VAR1 = {};
 Current device tree
 $VAR1 = {
           '/dev/nvme0n1' => undef,
           '/dev/nvme1n1' => undef,
           '/dev/sda' => undef,
           '/dev/sdb' => undef
         };
 Cannot determine size of /dev/crypt_format_md1 - scheme unknown
 }}}

 ping'd people in #fai for help - it sure looks like it's not partitionning
 the disk at all...

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32937#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list