Am I doing a soft raid restore correctly on CentOS 7.1?

Question

linux

Good afternoon!
I have questions about soft raid on CentOS:
Am I doing a system restore correctly?
Sometimes, after removing the disk and restoring, the swap or other partition disappears. What can be wrong. Below is an example when swap is dropped.
Starting configuration:
Virtual environment, two disks - sda, sdb 10Gb each. When installing CentOS 7.1 raised RAID-1 on MD.
[[email protected] ~]# fdisk -l
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000953af
Device Boot Start End Blocks Id System
/dev/sdb1 2048 4200447 2099200 fd Linux raid autodetect
/dev/sdb2 * 4200448 6299647 1049600 fd Linux raid autodetect
/dev/sdb3 6299648 20971519 7335936 fd Linux / raid
1 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier : 0x000e028d
Device Boot Start End Blocks Id System
/dev/sda1 2048 4200447 2099200 fd Linux raid autodetect
/dev/sda2 * 4200448 6299647 1049600 fd Linux raid autodetect /dev/sda3 6299648 20971519
7335936 fd Linux raid autodetect
Disk /dev/md127: 2147 MB, 2147483648 bytes, by 4194304 sectors
1 * Units 5 = 2 sectors of
logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/md126: 7507 MB, 7507804160 bytes, 14663680 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/md125: 1074 MB, 1074724864 bytes, 2099072 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[[email protected] ~]#
[[email protected] ~]# cat /proc/mdstat
Personalities : [ raid1]
md125 : active raid1 sdb2[1] sda2[0]
1049536 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md126 : active raid1 sdb3[1] sda3[0]
7331840 blocks super 1.2 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
md127 : active raid1 sdb1[1] sda1[0]
2097152 blocks super 1.2 [2/2] [UU]
unused devices :
[[email protected] ~]# mount | grep md1
/dev/md126 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/md125 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
[[email protected] ~]#
/dev/md126 - swap.

We disable the sdb disk in the hypervisor, do find . / , swapoff -a , swapon to display disk swap and see:

[[email protected] var]# cat /proc/mdstat
Personalities : [raid1]
md125 : active raid1 sdb2[1](F) sda2[0]
1049536 blocks super 1.0 [2/1] [U_]
bitmap: 1/1 pages [4KB], 65536KB chunk
md126 : active raid1 sdb3[1](F) sda3[0]
7331840 blocks super 1.2 [2/1] [U_]
bitmap: 1 /1 pages [4KB], 65536KB chunk
md127 : active raid1 sdb1[1](F) sda1[0]
2097152 blocks super 1.2 [2/1] [U_]
unused devices:
[[email protected] var]#
[[email protected] var]#

Everything seems to be fine, sdb disks disappeared everywhere, except for swap.
We shutdown the machine, add sdb again and turn it on.

[[email protected] ~]# fdisk -l | Grep SD
Disk / Dev / SDB: 10.7 GB, 10737418240 Bytes, 20971520 Sectors
Disk / Dev / SDA: 10.7 GB, 10737418240 Bytes, 20971520 Sectors
/ Dev / SDA1 2048 4200447 2099200 FD Linux Raid Autodetect
/ Dev / SDA2 * 4200448 6299647 1049600 FD Linux raid autodetect
/dev/sda3 6299648 20971519 7335936 fd Linux raid autodetect
[[email protected] ~]# You

can see that there is an unallocated disk /dev/sdb
Copy the file system:

[[email protected] ~]# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdb: 1305 cylinders, 255 heads, 63 sectors/track
sfdisk: /dev/sdb: unrecognized partition table type
Old situation:
sfdisk: No partitions found
New situation:
Units: sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdb1 2048 4200447 4198400 fd Linux raid autodetect
/dev/sdb2 * 4200448 6299647 2099200 fd Linux raid autodetect
/ dev/sdb3 6299648 20971519 14671872 fd Linux raid autodetect
/dev/sdb4 0 - 0 0 Empty
Warning: partition 1 does not end at a cylinder boundary
Warning: partition 2 does not start at a cylinder boundary
Warning: partition 2 does not end at a cylinder boundary
Warning: partition 3 does not start at a cylinder boundary
Warning: partition 3 does not end at a cylinder boundary
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
[[email protected] ~]#
Add spare partitions to array:
[[email protected] ~]# mdadm /dev/ md125 -a /dev/sdb3
mdadm: added /dev/sdb3
[[email protected] ~]# mdadm /dev/md126 -a /dev/sdb2
mdadm: added /dev/sdb2
[[email protected] ~]# mdadm /dev/md127 -a /dev/sdb1
mdadm: added /dev/sdb1
[[email protected] ~]#
Starting sync:
[[email protected] ~]# cat /proc/mdstat
Personalities : [raid1]
md125 : active raid1 sdb3[2] sda3[0]
7331840 blocks super 1.2 [2/1] [U_ ]
[=====>...............] recovery = 25.3% (1858432/7331840) finish=3.3min speed=27190K/sec
bitmap: 1/1 pages [4KB ], 65536KB chunk
md126 : active raid1 sdb2[2] sda2[0]
1049536 blocks super 1.0 [2/1] [U_]
resync=DELAYED
bitmap: 1/1 pages [4KB], 65536KB chunk
md127 : active raid1 sdb1[2] sda1[0]
2097152 blocks super 1.2 [2/1] [U_]
resync=DELAYED
unused devices:
[[email protected] ~]#
Successful:
[[email protected] ~]# cat / proc/mdstat
Personalities : [raid1]
md125 : active raid1 sdb3[2] sda3[0]
7331840 blocks super 1.2 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md126 : active raid1 sdb2[ 2] sda2[0]
1049536 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md127 : active raid1 sdb1[2] sda1[0]
2097152 blocks super 1.2 [2/2 ] [UU]
unused devices:
[[email protected] ~]#
Update bootloaders
[[email protected] ~]# grub2-install /dev/sda
Installing for i386-pc platform.
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
Installation finished. No error reported.
[[email protected] ~]# grub2-install /dev/sdb
Installing for i386-pc platform.
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
Installation finished. No error reported.
[[email protected]~]#

We stop the system, turn it off, remove the first disk and try to boot. It loads, but not 5-7 seconds, but more than 1 minute, the strip at the bottom crawls for a long time.
Loading dracut-033.. etc. (Initramfs)
After loading, reboot and get:
[[email protected] ~]# cat /proc/mdstat
Personalities : [raid1]
md125 : active raid1 sda3[2]
7331840 blocks super 1.2 [2/1] [_U]
bitmap: 1 /1 pages [4KB], 65536KB chunk
md126 : active raid1 sda2[2]
1049536 blocks super 1.0 [2/1] [_U]
bitmap: 1/1 pages [4KB], 65536KB chunk
md127 : inactive sda1[2](S )
2097152 blocks super 1.2
unused devices:
[[email protected] ~]#
[[email protected] ~]#

Those. swap moved off... Let me remind you, I deleted the first disk... And the disk that we now see as sda was previously sdb.

Answer 1

It's not very clear what you're doing here, but I have a feeling that you forgot
1) mark the drive being disconnected as non-working
mdadm --manage /dev/md125 --fail /dev/sdb2
2) remove it from the array
mdadm --manage /dev /md125 --remove /dev/sdb2
before attaching the "new" drive.
Although it is possible that your problem is not at all, but that when a disk is disconnected in the hypervisor, the partition table collapses on it (which should not happen if the real disk is physically disconnected).

Answer 2

And here, what do grub2-mkconfig and grub2-install swear at? How critical is this?
In this case, sda hangs on SCSI 0:0, and sdb on 0:1 - loading is fast and normal. FS in place.
If you stop the car and change the disks in places, then everything is fine too.
In this example, another machine is / - 2G , swap - 1G.
[[email protected] ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..
Found linux image:
/boot/vmlinuz-3.10.0-327.13.1.el7.x86_64 .img
/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..
/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..
/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'.
Found linux image: /boot/vmlinuz-3.10.0-327.el7.x86_64 Found
initrd image: /boot/initramfs-3.10.0-327.el7.x86_64.img
Found linux image
: /boot/initramfs-0-rescue-fc6a64c41e2e4536b262a8c9cf31abd0.img
/usr/sbin/grub2-probe: warning: volume'n't find physical .
done
[[email protected] ~]# grub2-install /dev/sda
Installing for i386-pc platform.
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
Installation finished. No error reported.
[[email protected] ~]# grub2-install /dev/sdb
Installing for i386-pc platform.
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
grub2-install: warning: Couldn't find physical volume '(null)'. Some modules may be missing from core image..
Installation finished. No error reported.
[[email protected]~]#
[[email protected]tos7 ~]#
[[email protected] ~]#
[[email protected] ~]# fdisk -l
Disk /dev/sdb: 3221 MB, 3221225472 bytes, 6291456 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes /
512 bytes
Disk label type: dos
Disk identifier:
0x00000000
fd Linux raid autodetect
/dev/sdb2 4200448 6291455 1045504 fd Linux raid autodetect
Disk /dev/sda: 3221 MB, 3221225472 bytes, 6291456 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 4200447 2099200 fd Linux raid autodetect
/dev/sda2 4200448 6291455 1045504 fd Linux raid autodetect
Disk /dev/md127: 1070 MB, 1070006272 bytes, 2089856 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size 5 (logical/ bytes Sector size 5) bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/md126: 2149 MB, 2149515264 bytes, 4198272 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[[email protected] ~]#
[[email protected] ~]# cat /proc/mdstat
Personalities : [ raid1]
md126 : active raid1 sdb1[2] sda1[3]
2099136 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md127 : active raid1 sdb2[2] sda2[3]
1044928 blocks super 1.2 [2/2] [UU]
unused devices:
[[email protected] ~]#
[[email protected] ~]# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,selabel,size=932904k,nr_inodes=233226,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev /shm type tmpfs (rw,nosuid,nodev,selabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,selabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,selabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,selabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec ,relatime)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec, relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec, relatime, blkio)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/md126 on / type xfs (rw ,relatime,seclabel,attr2,inode64,noquota)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp =1,timeout=300,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
debugfs on / sys/kernel/debug type debugfs (rw,relatime)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,selabel,size=188440k,mode=700)
fusectl on /sys/fs/fuse/ connections type fusectl (rw,relatime)
[[email protected]~]#