BIG-IQ statistics in dashboard missing/interrupted
Dear all,
any idea, why statistics data is missing/interrupted for a few days like in the following screenshot?
Or how I can verify any issues? In which logs can I found any hints?
The missing data is visible for device statistics as well as for Local Traffic. Am I right, that these statistics are the ones, which are stored on the DCDs? And why is there no data before 27.11.2022? Is there any limit configured that just the last 30 days (or so) will be stored? Does this indicate to an connectivity issue between the DCDs and the BIG-IPs?
As of now I could only verify that the ucs backups were successfully executed during these days, so connectivity between the BIG-IQ and the BIG-IPs was fine?
Thanks for any useful ideas or hints.
Regards Stefan 🙂
We could solve the issue in the meanwhile together with official F5-support.
It's related to ID1044337:
This can occur intermittently during statistics collection from BIG-IP to BIG-IQ and can happen when the retention processor causes a loss of some retention records (book-keeper-start entries are lost while book-keeper-end entries are retained).”
From the BIG-IQ command line:
1) Make a backup of the book-keeper-end index:
curl -sk http://localhost:9200/_reindex?pretty -H "content-type: application/json" -d '{"source":{"index":"book-keeper-end"},"dest":{"index":"bu-book-keeper-end"}}'
2) Delete the book-keeper-end index:
curl -skX DELETE http://localhost:9200/book-keeper-end
Note: The book-keeper-end index will be rebuilt automatically and will cause no loss of statistic data.
It took some time (I don't know how long exactly), but on the next day I could see statistics data back to one year again.
Regards Stefan 🙂