Forum Discussion

Sriram_Kannan_1's avatar
Sriram_Kannan_1
Icon for Nimbostratus rankNimbostratus
Jan 21, 2015

BIP-IP c2400 VCMP Archive Bug

We are currently facing issue to run archive process on all VCMP on c2400 chassis. f5 identified this to be an bug and provided an hotfix but we are unable to apply the fix due to maintenance window. We are running behind on backup for months now.

 

Is there any other manual process that we can run a archive process so we can restore the box in case of any crash scenario ?

 

application delivery
archive
LTM
management
performance
vcmp
  • shaggy's avatar
    shaggy
    Icon for Nimbostratus rankNimbostratus
    Jan 21, 2015
    you'll have to describe the issue you are currently facing - does the issue prevent the creation of a UCS archive on a vCMP guest? Is the bug isolated to creating the UCS in the UI or UCS creation in general? also, what version of BIGIP is the guest running?
  • Sriram_Kannan_1's avatar
    Sriram_Kannan_1
    Icon for Nimbostratus rankNimbostratus
    Jan 21, 2015
    Bug prevents VCMP guest to create an UCS archive file as it spike CPU Utilization. UCS Archive on the chassis itself is fine. The problem is isolated only to VCMP. Running an archive process triggers the issue caused by "syscalld" daemon which occupies the CPU resources, forcing the process in a deadlock situation. Manual Kill of the daemon process is required to recover the situation. VCMP are running BIG-IP 11.6.0 Build 1.0.403 Hotfix HF1
  • Brad_Parker's avatar
    Brad_Parker
    Icon for Cirrus rankCirrus
    Jan 23, 2015

    As a last resort backup, you can grab a copy of your /config directory via SCP or SFTP using something like WinSCP until you can apply the hotfix. This will allow you to rebuild in case of a disaster by grabing the bigip_base.conf and bigip.conf. If the guests are in an HA pair you just need to rebuild the guest and copy bigip_base.conf and re-sync if the unforeseen happens.

     

  • RussT's avatar
    RussT
    Icon for Nimbostratus rankNimbostratus
    Mar 26, 2015
    I had a similar issue on 11.6.0 upto hotfix 3 with Viprion C2400, all VCMP guest's cold not archive config, it was due to rest javad. a restart from cli with bigstart restart restjavad would resolved the issue but re-occur within days and require a new restart of service to archive.
  • Sriram_Kannan_1's avatar
    Sriram_Kannan_1
    Icon for Nimbostratus rankNimbostratus
    Mar 26, 2015

    bigip.conf contains all guest info and its management and vlans associated. bigip_base contains all necessary chassis information. Should the host hold config file for each guest or would it only be available in the respective guest. Is there a way to recover all the configuration of individual guest through a file located on the host ?

     

    • boneyard's avatar
      boneyard
      Icon for MVP rankMVP
      Apr 05, 2015
      the host wont have any inside guest info. you need the bigip / _base / .... files from there,
  • alex100's avatar
    alex100
    Icon for Cirrostratus rankCirrostratus
    Jun 24, 2015

    I have an identical problem and is currently working with Support. I am glad I was able to find this post with corresponding bug ID. In meantime, I was able to come up with workaround.

    So, it looks like the Ucs_save process does not complete correctly at some point, staying active on the system. That in terms prevents any concurrent ucs_save jobs from running. As the pile of ucs jobs grows bigger syscalld seems to go out of control excessively consuming resources.

    So, if you run 

    ps aux |grep ucs
    you should be able identify running ucs_joubs... you might see a bunch of them. Identify the oldest one. That should be the one that is hang up. Kill it by executing 
    kill -9

    Restart "syscalld" by executin:

    bigstart restart syscalld