Leider hat sich ergeben, dass sich eine Festplatte schön langsam aufgelöst hat.
Status des Software-RAIDs zeigt den Ausfall der Festplatte /dev/sdb:
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sda4[0] sdb4[1](F)
1839090112 blocks super 1.2 [2/1] [U_]
bitmap: 9/14 pages [36KB], 65536KB chunk
md1 : active raid1 sda2[0] sdb2[1](F)
523712 blocks super 1.2 [2/1] [U_]
md0 : active raid1 sda1[0] sdb1[1](F)
16760832 blocks super 1.2 [2/1] [U_]
md2 : active raid1 sda3[0] sdb3[1](F)
1073610752 blocks super 1.2 [2/1] [U_]
bitmap: 6/8 pages [24KB], 65536KB chunk
unused devices: <none>
Alle Device zeigen Fehler:
root@h1 ~ # mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Tue May 7 20:42:25 2019
Raid Level : raid1
Array Size : 16760832 (15.98 GiB 17.16 GB)
Used Dev Size : 16760832 (15.98 GiB 17.16 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Mar 4 11:00:59 2022
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Consistency Policy : resync
Name : rescue:0
UUID : 54e5acea:e5928e65:f6d6a669:cf1fb9d2
Events : 531
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
- 0 0 1 removed
1 8 17 - faulty /dev/sdb1
# mdadm --detail /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Tue May 7 20:42:25 2019
Raid Level : raid1
Array Size : 523712 (511.44 MiB 536.28 MB)
Used Dev Size : 523712 (511.44 MiB 536.28 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Mar 4 06:37:29 2022
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Consistency Policy : resync
Name : rescue:1
UUID : 8eb2e7c6:f87a8a3f:8f82d581:1f346131
Events : 323
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
- 0 0 1 removed
1 8 18 - faulty /dev/sdb2
# mdadm --detail /dev/md2
/dev/md2:
Version : 1.2
Creation Time : Tue May 7 20:42:26 2019
Raid Level : raid1
Array Size : 1073610752 (1023.88 GiB 1099.38 GB)
Used Dev Size : 1073610752 (1023.88 GiB 1099.38 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Fri Mar 4 16:12:02 2022
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Consistency Policy : bitmap
Name : rescue:2
UUID : 3af3850c:507630d7:28a96c3a:84ec1f66
Events : 1849768
Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
- 0 0 1 removed
1 8 19 - faulty /dev/sdb3
# mdadm --detail /dev/md3
/dev/md3:
Version : 1.2
Creation Time : Tue May 7 20:42:28 2019
Raid Level : raid1
Array Size : 1839090112 (1753.89 GiB 1883.23 GB)
Used Dev Size : 1839090112 (1753.89 GiB 1883.23 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Fri Mar 4 16:11:14 2022
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Consistency Policy : bitmap
Name : rescue:3
UUID : 145fb3fe:45b60dcb:2d4d904a:5df4003a
Events : 674039
Number Major Minor RaidDevice State
0 8 4 0 active sync /dev/sda4
- 0 0 1 removed
1 8 20 - faulty /dev/sdb4
Jetzt die Festplatte aus dem RAID-Array nehmen:
# mdadm /dev/md0 -r /dev/sdb1 mdadm: hot removed /dev/sdb1 from /dev/md0 # mdadm /dev/md1 -r /dev/sdb2 mdadm: hot removed /dev/sdb2 from /dev/md1 # mdadm /dev/md2 -r /dev/sdb3 mdadm: hot removed /dev/sdb3 from /dev/md2 # mdadm /dev/md3 -r /dev/sdb4 mdadm: hot removed /dev/sdb4 from /dev/md3
Mal kurz feststellen, welcher Partitionstyp in Verwendung ist:
root@h1 ~ # parted -l Model: ATA ST33000651AS (scsi) Disk /dev/sda: 3001GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 5 1049kB 2097kB 1049kB bios_grub 1 2097kB 17.2GB 17.2GB raid 2 17.2GB 17.7GB 537MB raid 3 17.7GB 1117GB 1100GB raid 4 1117GB 3001GB 1883GB raid Model: Linux Software RAID Array (md) Disk /dev/md2: 1099GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0.00B 1099GB 1099GB ext4 Model: Linux Software RAID Array (md) Disk /dev/md0: 17.2GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0.00B 17.2GB 17.2GB linux-swap(v1) Model: Linux Software RAID Array (md) Disk /dev/md3: 1883GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0.00B 1883GB 1883GB ext4 Model: Linux Software RAID Array (md) Disk /dev/md1: 536MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0.00B 536MB 536MB ext3
Herausfinden der Seriennummer der defekten Platte:
chris@h1:~$ /sbin/udevadm info --query=property --name=sdb | grep ID_SERIAL ID_SERIAL=WDC_WD3000FYYZ-01UL1B2_WD-WMC1F0E4CNYD ID_SERIAL_SHORT=WD-WMC1F0E4CNYD
Über hdparm funktioniert es nicht mehr:
# hdparm -i /dev/sdb | grep SerialNo HDIO_DRIVE_CMD(identify) failed: Input/output error HDIO_GET_IDENTITY failed: No message of desired type
Via smartctl war nix mehr auszulesen …
Jetzt der Auftrag an Hetzner …
Huch! Der Tausch der Festplatte ist innerhalb von 30 Minuten erledigt! 😳
Partitionstabelle auf die neue Festplatte übertragen:
# sgdisk --backup=sda_parttable_gpt.bak /dev/sda The operation has completed successfully. # sgdisk --load-backup=sda_parttable_gpt.bak /dev/sdb Creating new GPT entries. Warning! Current disk size doesn't match that of the backup! Adjusting sizes to match, but subsequent problems are possible! The operation has completed successfully.
und einen neue UUID für die neue Festplatte erzeugen:
# sgdisk -G /dev/sdb Warning: The kernel is still using the old partition table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) The operation has completed successfully.
Einmal eine Partition in das Software-RAID hängen:
# mdadm /dev/md3 -a /dev/sdb4 mdadm: added /dev/sdb4
und schauen, wie es synchronisiert:
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[2] sda4[0]
1839090112 blocks super 1.2 [2/1] [U_]
[>....................] recovery = 1.3% (23962624/1839090112) finish=466.6min speed=64820K/sec
bitmap: 9/14 pages [36KB], 65536KB chunk
md2 : active raid1 sda3[0]
1073610752 blocks super 1.2 [2/1] [U_]
bitmap: 7/8 pages [28KB], 65536KB chunk
md1 : active raid1 sda2[0]
523712 blocks super 1.2 [2/1] [U_]
md0 : active (auto-read-only) raid1 sda1[0]
16760832 blocks super 1.2 [2/1] [U_]
unused devices: <none>
Wird ein wenig dauern …
Auch die anderen Partitionen eingehängt:
root@h1 ~ # mdadm /dev/md2 -a /dev/sdb3
mdadm: added /dev/sdb3
root@h1 ~ # mdadm /dev/md1 -a /dev/sdb2
mdadm: added /dev/sdb2
root@h1 ~ # mdadm /dev/md0 -a /dev/sdb1
mdadm: added /dev/sdb1
root@h1 ~ # cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[2] sda4[0]
1839090112 blocks super 1.2 [2/1] [U_]
[>....................] recovery = 2.1% (39605888/1839090112) finish=427.5min speed=70147K/sec
bitmap: 9/14 pages [36KB], 65536KB chunk
md2 : active raid1 sdb3[2] sda3[0]
1073610752 blocks super 1.2 [2/1] [U_]
resync=DELAYED
bitmap: 7/8 pages [28KB], 65536KB chunk
md1 : active raid1 sdb2[2] sda2[0]
523712 blocks super 1.2 [2/1] [U_]
resync=DELAYED
md0 : active raid1 sdb1[2] sda1[0]
16760832 blocks super 1.2 [2/1] [U_]
resync=DELAYED
unused devices: <none>
Am nächsten Morgen war die Festplatte synchronisiert:
$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[2] sda4[0]
1839090112 blocks super 1.2 [2/2] [UU]
bitmap: 1/14 pages [4KB], 65536KB chunk
md2 : active raid1 sdb3[2] sda3[0]
1073610752 blocks super 1.2 [2/2] [UU]
bitmap: 3/8 pages [12KB], 65536KB chunk
md1 : active raid1 sdb2[2] sda2[0]
523712 blocks super 1.2 [2/2] [UU]
md0 : active raid1 sdb1[2] sda1[0]
16760832 blocks super 1.2 [2/2] [UU]
unused devices: <none>
Quellen:
Hetzner: Festplattenaustausch im Software-RAID
Blog Dominic Pratt: Software-RAID-Reparatur bei Hetzner
Schreibe eine Antwort