Forum Discussion

Sylvain_85827

Cirrus

Jun 23, 2015

VCMP space disk allocation explanations

Hello,

Can please someone help me to understand the disk allocation of this VCMP system?

1/ Total physical disk space is : 400GB according to fdisk:

[root@XXX:Active:Standalone] config  fdisk -l

Disk /dev/sda: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1       48641   390707808   8e  Linux LVM

2/ Guests vdisks use 100G of lvm space but... (so you can guess 500GB used... :/)

[root@XXX:Active:Standalone] config  ls -lh /shared/vmdisks/
total 23G
drwx------ 2 root root  16K Apr  8 16:45 lost+found
-rw-r--r-- 1 root root 100G Jun 23 11:45 GUEST1.img
-rw-r--r-- 1 root root   23 Jun 22 15:25 GUEST1.info
-rw-r--r-- 1 root root 100G Jun 23 11:45 GUEST2.img
-rw-r--r-- 1 root root   23 Jun 22 15:25 GUEST2.info
-rw-r--r-- 1 root root 100G Jun 23 11:45 GUEST3.img
-rw-r--r-- 1 root root   23 Jun 22 15:25 GUEST3.info
-rw-r--r-- 1 root root 100G Jun 23 11:45 GUEST4.img
-rw-r--r-- 1 root root   23 Jun 22 15:28 GUEST4.info
-rw-r--r-- 1 root root 100G Jun 23 11:45 GUEST5.img
-rw-r--r-- 1 root root   23 Jun 22 15:25 GUEST5.info

3/ ... but the physical space usage is as below:

[root@XXX:Active:Standalone] config  tmsh  show vcmp virtual-disk detail

--------------------------------------------------------
Sys::Vcmp::VirtualDisk
Name          Operating System  Status  Disk Use (bytes)
--------------------------------------------------------
GUEST1.img              tmos  in-use              4.6G
GUEST2.img              tmos  in-use              4.4G
GUEST3.img              tmos  in-use              4.5G
GUEST4.img              tmos  in-use              4.3G
GUEST5.img              tmos  in-use              4.5G

4/ and indeed pysically this matches (4.6 + 4.4 + 4.5 + 4.3 + 4.5 = 23 GB) and confirmed with command du:

[root@XXX:Active:Standalone] config  du -hs /shared/vmdisks/
23G     /shared/vmdisks/

5/ But when I run command df I can't summ the disk space to reach 400GB:

[root@XXX:Active:Standalone] config  df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg--db--sda-set.3.root
                      380M  212M  149M  59% /
/dev/mapper/vg--db--sda-set.3._config
                      3.0G   72M  2.8G   3% /config
/dev/mapper/vg--db--sda-set.3._usr
                      2.5G  1.9G  465M  81% /usr
/dev/mapper/vg--db--sda-set.3._var
                      3.0G  431M  2.4G  16% /var
/dev/mapper/vg--db--sda-dat.share.1
                       30G  6.0G   23G  22% /shared
/dev/mapper/vg--db--sda-dat.log.1
                      6.9G  163M  6.4G   3% /var/log
none                   24G  916K   24G   1% /dev/shm
none                   24G   41M   24G   1% /shared/rrd.1.2
none                   24G  8.2M   24G   1% /var/tmstat
none                   24G  1.6M   24G   1% /var/run
prompt                4.0M   28K  4.0M   1% /var/prompt
none                   24G     0   24G   0% /var/loipc
/dev/mapper/vg--db--sda-app.vcmp.dat.vmdisks
                      277G   23G  240G   9% /shared/vmdisks

How should I interpret all thoses disk usage values to figure out the real explanation?

Thanks for your help. Kind regards.

Sylvain

Jim_43841
Jun 25, 2015
Yes, those set.N items are related to the install locations HD1.N. If you want to destroy one, you can issue "tmsh delete sys software volume HD1.N", either in the guest or the host, wherever you're trying to eke out more free disk space.

Jim_43841
Historic F5 Account
Jun 23, 2015
The "missing" space is likely hiding in a different install volume. Try running lvscan to see those.

Sylvain_85827

Cirrus

Jun 25, 2015

Thanks Jim, for your help.

Lvscan command returns this:

[root@XXX:Active:Standalone] config  lvscan -av
    Finding all logical volumes
  ACTIVE            '/dev/vg-db-sda/dat.boot' [372.00 MB] inherit
  ACTIVE            '/dev/vg-db-sda/dat.share.1' [30.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/dat.log.1' [7.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/dat.swapvol.1' [5.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.1.root' [392.00 MB] inherit
  ACTIVE            '/dev/vg-db-sda/set.1._usr' [2.20 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.1._config' [3.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.1._var' [3.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.2.root' [392.00 MB] inherit
  ACTIVE            '/dev/vg-db-sda/set.2._usr' [2.48 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.2._config' [3.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.2._var' [3.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/app.vcmp.dat.vmdisks' [276.72 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.3.root' [392.00 MB] inherit
  ACTIVE            '/dev/vg-db-sda/set.3._usr' [2.48 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.3._config' [3.00 GB] inherit
  ACTIVE            '/dev/vg-db-sda/set.3._var' [3.00 GB] inherit

Do you know all these "set.*_*" volumes are all necessary? Can the "missing" space coming from there:

'/dev/vg-db-sda/set.1._usr' 
'/dev/vg-db-sda/set.1.root' 
'/dev/vg-db-sda/set.1._config' 
'/dev/vg-db-sda/set.1._var' 
'/dev/vg-db-sda/set.2.root' 
'/dev/vg-db-sda/set.2._usr' 
'/dev/vg-db-sda/set.2._config' 
'/dev/vg-db-sda/set.2._var' 
'/dev/vg-db-sda/set.3.root' 
'/dev/vg-db-sda/set.3._usr' 
'/dev/vg-db-sda/set.3._config' 
'/dev/vg-db-sda/set.3._var'

What do you suggest please?

Thanks again for your help.

Sylvain

Sylvain_85827
Cirrus
Jun 25, 2015
Maybe I just found the explanation thanks to you Jim. You're right there are 3 installations installed respectively on HD1.1, HD1.2 and HD1.3 as boot locations. Can you confirm if this corresponds to /dev/vg-db-sda/set.1* , /dev/vg-db-sda/set.2* and /dev/vg-db-sda/set.3* ? If you do, I think we found the culprit ;-) Cheers. Sylvain

amolari
Cirrostratus
Jun 25, 2015
those are for the host only. vCMP guests are in the volume

ACTIVE '/dev/vg-db-sda/app.vcmp.dat.vmdisks' [276.72 GB] inherit

With df, you will see 100G per guest, which looks like to be "reserved" space.
tmsh show vcmp virtual-disk detail
shows the effective disk usage per guest

Diskspace on vCmp system is scarce. I would recommend to use no more than 2 volumes per guest (and on the host)
Jim_43841
Historic F5 Account
Jun 25, 2015
Yes, those set.N items are related to the install locations HD1.N. If you want to destroy one, you can issue "tmsh delete sys software volume HD1.N", either in the guest or the host, wherever you're trying to eke out more free disk space.
Sylvain_85827
Cirrus
Jun 26, 2015
Thanks for your help Jim and amolari. Very useful :) Take care.