Forum Discussion
NimbostratusUnable to add disk to array
Hello everyone,
We have an F5 7000S that we have purchased new in 2013 and kept it running pretty much 24/7 all this time.
After a datacenter outage one of the drives in the device died. We have replaced it with another disk, but addig it back into the array using GUI gives "NO access" error.
Attempting to follow https://my.f5.com/manage/s/article/K12756 does not help.
When executing from command line the following error is given:
tm_install::process::Proces_array_member_add --sdb ownership is none
The F5 is still running 11.4.1.
The replacement disk is of bigger capacity (2TB instead of 1TB).
Does anyone know of this specific error - searching this forum and internet returns nothing. I wonder if this is fixed in later OS images or maybe F5 refuses to work with different disk sizes (most normal raids can accept bigger disk as replacement)
Any help is apprecaited!
8 Replies
CA_ValliMVP
I'm guessing the replacement disk you installed was not F5-supplied, correct?
First thing I'd check is if it's being seen by the unit, do the LEDs blink? What's the output of tmsh show sys raid disk ?Also, what procedure did you use to unmount the disk? Did you remove it from the array before switching it as detailed in https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/platform-b7000/4.html#c_reuse_fru_hd_about ?
- nycdude
That is correct.
The replacement disk is a Samsung EVO 870 SSD.
I tried replacing it with the same identical vendor (not F5 sourced though) WD-Velociraptor 1TB, but the results are the same.
here is the output of the command. The disk is recognized, just wont add to raid.
---------------------------------------------------------------------
Sys::Raid::Disk
Name Serial Number Array Array Status Model
Member
---------------------------------------------------------------------
HD1 WD-WXR1E63HMUYD yes ok ATA WDC WD1000CHTZ-0
HD2 S6PNNJ0W412826X no undefined ATA Samsung SSD 870
Michael09The error "tm_install::process::Process_array_member_add --sdb ownership is none" suggests that there may be a configuration or ownership issue with the replacement disk on your F5 7000S running OS version 11.4.1. Since the replacement disk is of a bigger capacity, it's possible that the RAID configuration is not recognizing the new disk properly. Consider updating the F5 OS to the latest available version, as newer versions may have addressed compatibility issues with different disk sizes.
- CA_Valli
Thanks for the outputs.
So, let's go back to the output log in your first query. SD-A and SD-B should be the the lables of the disk in the array. In your log, it refers no ownership to SD-B so there might still be issues with "accepting" the new disk even if output shows it's present.Try running these commands:
dmesg |grep --color sd[a-b]
sfdisk -l /dev/sdb # I think new disk should be seen as sdb while existing should be sda
sfdisk -l /dev/sdaIf they return disk as failed, or if they return no output, the problem is the new disk, likely.
One more question, just fr myself to understand this better -- what was the status of the old disk that was removed?I'd also like to check if the linux subsystem sees the same status of the array as well .. Can you run array command too? You'll want to confront its output with the following tmsh commands (again - just to confirm again disk status)
tmsh list /sys disk all-properties # should list both disks
tmsh show sys raid # it's like the one I gave u last time, just has more outputLast thing would be checking if the mdadm process is running and in sync -- but I'm not sure if it already exists in version v11, and anyways having a disk in that status will likely result in bad output for the mdadm.
grep -e "_U" -e "U_" -e "__" /proc/mdstat | wc -l
- nycdude
Thank you CA_Valli, i really appreciate your responses.
About original hard drive - when we had DC outage (complete power failure) the device restarted and basically showed No disk in bay 2 (Despite the disk being there). When i did the commands to check array status with the old disk still inserted it confirmed that the device was not seeing it in either TMOS or BSH. (I have had disks fail like that in PC's where it was spinning and powered but not visible to OS).
Here are the outputs:
[:Active:Standalone] ~ # dmesg |grep --color sd[a-b]
sd 1:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
sd 1:0:0:0: [sda] 4096-byte physical blocks
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda: sda1 sda2
sd 1:0:0:0: [sda] Attached SCSI disk
md: bind<sda1>
md: could not bd_claim sda1.
sd 0:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 0:0:0:0: [sdb] Write Protect is off
sd 0:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sdb: unknown partition table
sd 0:0:0:0: [sdb] Attached SCSI disk
sdb: unknown partition table
sdb:
[:Active:Standalone] ~ # sfdisk -l /dev/sdbDisk /dev/sdb: 243201 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0Device Boot Start End #cyls #blocks Id System
/dev/sdb1 0 - 0 0 0 Empty
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
[:Active:Standalone] ~ #
(cfg-sync Standalone)(Active)(/Common)(tmos)# list /sys disk all-properties
sys disk application-volume afmdata {
logical-disk MD1
owner afm
preservability precious
resizeable false
size 3900
volume-set-visibility-restraint none
}
sys disk application-volume avrdata {
logical-disk MD1
owner avr
preservability precious
resizeable false
size 3900
volume-set-visibility-restraint none
}
sys disk application-volume mysqldb_MD1.1 {
logical-disk MD1
owner mysql
preservability precious
resizeable false
size 12288
volume-set-visibility-restraint MD1.1
}
sys disk logical-disk HD1 {
mode control
size 953869
vg-free 867148
vg-in-use 85784
vg-reserved 40
}
sys disk logical-disk HD2 {
mode none
size 1907729
vg-free 0
vg-in-use 0
vg-reserved 30720
}
sys disk logical-disk MD1 {
mode mixed
size 953869
vg-free 863192
vg-in-use 89740
vg-reserved 30720
}
(cfg-sync Standalone)(Active)(/Common)(tmos)#
(cfg-sync Standalone)(Active)(/Common)(tmos)# show sys raid---------------------
Sys::Raid::Array: MD1
---------------------
Size (MB) 931.5K---------------------------------------------------------
Sys::Raid::ArrayMembers
Bay ID Serial Number Name Array Member Array Status
---------------------------------------------------------
1 WD-WXR1E63HMUYD HD1 yes ok-------------------------------------------------------------
Sys::Raid::Bay
Bay Shelf Name Serial Number Array Member Array Status
-------------------------------------------------------------
1 1 HD1 WD-WXR1E63HMUYD yes ok
2 1 HD2 S6PNNJ0W412826X no undefined---------------------------------------------------------------------
Sys::Raid::Disk
Name Serial Number Array Array Status Model
Member
---------------------------------------------------------------------
HD1 WD-WXR1E63HMUYD yes ok ATA WDC WD1000CHTZ-0
HD2 S6PNNJ0W412826X no undefined ATA Samsung SSD 870(cfg-sync Standalone)(Active)(/Common)(tmos)#
[Active:Standalone] ~ # grep -e "_U" -e "U_" -e "__" /proc/mdstat | wc -l
19Thanks again and very much appreciated!!!
- CA_Valli
I've reviewed this further.
According to https://my.f5.com/manage/s/article/K15525 (old article, but should be relevant for your version) we should perform a logical rebuild of the RAID array.This brings us back, again, to that ownership log.
If you run the following command: ls /dev/ -l | grep sd
do you see different permissions assinged to the sda* / sdb* partitions?If that's the case (my suspect would be SDB1 to SDB4 do not have the root:disk ownership) , I think that you'll need to change owner and/or permissions of the mount.
However!Before attemtping to change the permissions, I would try to perform a physical rebuild of the array.According to https://my.f5.com/manage/s/article/K12756#logical%20rebuild
steps would be:
0- backup configuration in an ucs archive and copy it to another system
1- shut down the device
2- remove "bad" drive
3- start device with single disk and wait for RAID master to be assigned to remaining drive
4- shutdown device again
5- reinsert drive
6- start appliance
7- (my addition) potentially check for this https://my.f5.com/manage/s/article/K12380
If this still fails, you might want to try with changing permissions + logical rebuild of the array ..<< Disclaimer >> I would normally suggest you to contact support for this type of issue but I recognize software has been out of support for quite a long time. Use at your own risk.
This would be done via:
chown root:disk < /path/to/the/mount > ## check group name (disk) via ls -l /dev/ as said before
chmod g+rw < /path/to/the/mount >
tmsh modify sys raid array <array name> remove <disk name> ## might not be required -- disk name should be HD2, I guess?
tmsh modify sys raid array <array name> add <disk name>
- LiefZimmerman
Admin
- nycdude
Unfortunately I have not had a chance to try anything that requires a reboot as this is our active ingress/outgress device.
We have made a choice to replace it with Netgate PFS+ since we no longer have the requirement to aggregate over 10G of througput (multiwan).
Once it is done i will try the steps on the device when it is no longer in service. I appreciate all the help here!
