Forum Discussion

David_M's avatar
David_M
Icon for Cirrostratus rankCirrostratus
Jul 02, 2019

/var/lib/mysql is 100% full

Hi I am seeing this right now, and I am aware of AVR and APM KB articles regarding this but here its impacting ASM mostly, I cant export policies nor view request logs. Any ideas on what we c...
application delivery
ASM Advanced WAF
  • David_M's avatar
    David_M
    Aug 25, 2019

    Hi Andrew,

     

    This was a bug: we were previously using the system with VCMP and then it was deprovisioned from that to a normal LTM with ASM provisioned.

     

    In this change the VMCP leaves behind a /var/mysqldb of size 2GB which is enough for VCMP to function but not for ASM.

     

    TAC asked us to reinstall the TMOS into a new boot location and then we got a new /var/mysqldb of size 24GB.

     

    Cheers!

     

David_M's avatar
David_M
Icon for Cirrostratus rankCirrostratus

This is on v13.1.x.x

Yes this is a definite bug, we have multiple cases with these.

Do we have a fix yet in a future version?

We freed some space by deleting AVR files and got the usage to 89% but in 4 days its back to 94%

The TAC used these commands to fix the table.

mysql -t -u root -p$(perl -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw) -e 'check table PRX.REQUEST_LOG'
 
mysql -t -u root -p$(perl -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw) -e 'repair table PRX.REQUEST_LOG'
Andrew-F5's avatar
Andrew-F5
Icon for Employee rankEmployee
Jul 08, 2019

DavidMas,

 

I would still be curious to know what the output of

# lsof -ws | grep -i 'size\|deleted'

looks like since daemons like AVR can hold onto files even after deletion. The only way to remove files listed with lsof is to restart AVRD or reboot.

 

Best,

Andrew