Forum Discussion

Stefan_Klotz's avatar
Stefan_Klotz
Icon for Cumulonimbus rankCumulonimbus
Jan 09, 2023

BIG-IQ statistics in dashboard missing/interrupted

Dear all, any idea, why statistics data is missing/interrupted for a few days like in the following screenshot?   Or how I can verify any issues? In which logs can I found any hints? The mi...
application delivery
BIQ-IQ
statistics
  • Stefan_Klotz's avatar
    Stefan_Klotz
    May 11, 2023

    We could solve the issue in the meanwhile together with official F5-support.

    It's related to ID1044337:

    This can occur intermittently during statistics collection from BIG-IP to BIG-IQ and can happen when the retention processor causes a loss of some retention records (book-keeper-start entries are lost while book-keeper-end entries are retained).”

    From the BIG-IQ command line:

    1) Make a backup of the book-keeper-end index:

    curl -sk http://localhost:9200/_reindex?pretty -H "content-type: application/json" -d '{"source":{"index":"book-keeper-end"},"dest":{"index":"bu-book-keeper-end"}}'

    2) Delete the book-keeper-end index:

    curl -skX DELETE http://localhost:9200/book-keeper-end

    Note:  The book-keeper-end index will be rebuilt automatically and will cause no loss of statistic data.

     

    It took some time (I don't know how long exactly), but on the next day I could see statistics data back to one year again.

    Regards Stefan 🙂

Nikoolayy1's avatar
Nikoolayy1
Icon for MVP rankMVP
Jan 09, 2023

Maybe check the /var/log/avr and /var/log/restjavad on the BIG-IP as the Analytics AVR is the way to stream logs to DCD and the restjavad is the HTTP API  that BIg-IQ uses to control BIG-IP and also check the BIG-IQ device and system logs on the BIG-IQ for the BIG-IP devices as BIG-IQ also manages the DCD.

 

Just in case see in GUI if the BIG-IP has log statistics for this period or upload qkview to ihealth and check again the Statistics of the BIG-IP as to exclude BIG-IP issue and check in health for any bugs under diagnostics.

  • Leslie_Hubertus's avatar
    Leslie_Hubertus
    Ret. Employee
    Jan 18, 2023

    Stefan_Klotz  - did Nikoolayy1 's suggestion work for you? Inquiring minds want to know! 🙂

    (also, if yes, don't forget to hit the Accept as Solution button if it did work - and if it didn't, let us know so we can see if there's another solution for you. 

  • Stefan_Klotz's avatar
    Stefan_Klotz
    Icon for Cumulonimbus rankCumulonimbus
    Feb 24, 2023

    Sorry for late reply.

    I don't found any indications in any of the mentioned logs. Since end of December logging for all of our BIG-IPs seems to run stable again as you can see in the following screenshot:

    To be more precise and the reason why I'm asking is, that I wonder if and how recording these statistics data depends on the free disc space of the BIG-IQ itself (not the DCDs). Background: until end of last year the "/shared" partition of the BIG-IQ was full, as there were too much ucs-files from all of our BIG-IPs. After christmas, one BIG-IP cluster was removed, which released some GB (because all corresponding ucs-files where deleted as well). Additionally I increased this partition to 100GB beginning of January. And as you can see in the above screenshot, statistics data is available again since around that date!

    Are there any dependencies or how can I verify, why there are no statistics data before christmas 2022?

    I don't expect any issue with the BIG-IPs or the network. It seems for me like a local issue with the BIG-IQ.

    Are there any additional information/documentation about the availability of the statistics data?

    Thank you!

     

    Regards Stefan 🙂

    • Stefan_Klotz's avatar
      Stefan_Klotz
      Icon for Cumulonimbus rankCumulonimbus
      Mar 06, 2023

      It seems only around the last 30 days are stored? I changed nothing and now also the statistics from January are gone 😞

      I found the following settings, which I assume are default:

      But the "max storage Limit" is not reached:


      Any other ideas, why statistics data are not stored up to one year? Or how can I further investigate this issue?
      Thank you!

      Regards Stefan 🙂