Datagroup access during a datagroup update

Hi,

We're testing updates to a relatively large datagroup while accessing the datagroup from an iRule. With an internal datagroup on 10.2.0 containing ~5000 entries, we run apachebench against the VS and log a debug line if the number of class entries changes:


when HTTP_REQUEST {
   if {[class size my_class]] != 5000}{
     log local0. "size: [class size my_class]"
   }
}

When clicking update on the GUI's datagroup page for this test class, we see ~3000 instances of varying counts in the course of sending 100 concurrent requests for a total of 100,000. It looks like the class length count starts to drop and then increments back up to the correct total. The reason we tried testing this count was we were seeing class lookups fail for entries which should have existed in the class.

I thought a datagroup update was an atomic operation where the class would continue to exist in it's past configuration state for an existing TCP connection and wouldn't be updated until a new TCP connection was established after the datagroup update was complete.

Can anyone clarify whether this variation in class entries is expected behavior? If so, has F5 done any performance testing that would give a rough idea of when you might start to see a service affecting issue while updating a datagroup on the various platforms? As it is, it's looking like we'd need to suggest to customers that they only update datagroups during low traffic periods or maintenance windows.

Also, would it have less impact on live traffic to do a diff between the old/new class contents and then use 'b class CLASS_NAME add|delete' to modify the datagroup?

Thanks, Aaron

17 Replies

hoolio
Cirrostratus
Jun 29, 2010
Any initial thoughts on the intended behavior for this scenario or should I open a case?

Thanks, Aaron
Michael_Yates
Nimbostratus
Jun 29, 2010
I wish I could help you with your issue. Unfortunately, I haven’t had to create a datagroup that is that large yet.

I would be interested in hearing if you get a different behavior / result by doing your "b class CLASS_NAME add|delete" or a Merge File to see if you get the same behavior.
Hamish
Cirrocumulus
Jun 29, 2010
Hi Hoolio.

What happens if you use a single iControl call to update the datagroup rather than going via the GUI? Just a theory, but it may be that the GUI removes all the entries and then recreates them all one-by-one... In v9.4 that was the only way I ever managed to get a reliable update.

Now assuming this is a string class, and you're doing key/value lookups, then it's easier to just update the values rather than remove & re-add. (I haven't seen any problems with doing this, but then I've only got a few entries in the DG's I'm currently playing with).

H
hoolio
Cirrostratus
Jun 29, 2010
Thanks for the suggestions guys.

Initially we were testing with iControl and saw HTTP request failures during the update. This might have been due to resource starvation during the config change. I believe there were also issues with lookups returning no results for entries that should have been in the class. To simplify the testing, we tried clicking save on the datagroup in the GUI and not using iControl. I didn't retest the iControl update to see if the class size changed. I'll give that a try tomorrow.

For the customer's project, we'll be using name value pairs in a string datagroup. For the basic load testing, we were just using a name only string datagroup. I'm not sure what the customer will do in terms of updates to the source data. I assume only a small portion of the datagroup name/values will change. So it would be ideal to add/modify/remove only new/changed/stale entries. Our developer said iControl supports these changes in 10.2. So hopefully the class size changing issue will be mitigated. It will be interesting to see whether the load during the changes decreases. I'll post an update after we test more tomorrow.

I'd still like to get an idea of the expected results for updating a small and/or large datagroup via the GUI. A majority of iRules I've written for customers reference datagroups and it would be a significant change in administration for customers if they needed to limit datagroup updates to maintenance windows. Should the datagroup contents be removed incrementally and then be restored incrementally? If so, is this considered a defect? Has the behavior changed over recent versions?

Thanks, Aaron
Robert_Reznicek
Nimbostratus
Sep 08, 2010
Hi Aaron,

we were experiencing exactly the same problem on v9.4.7. When updating large datagroup from the CLI,the entries are flushed first and than reinitialized. There is even bug, that b class show command shows the correct number of entries, even if they are not available.

Did you get any feedback from F5? Or did you find any workaround?

Thank you,

Robert
hoolio
Cirrostratus
Sep 20, 2010
Hi Robert,

I didn't end up opening a case on this as the project was delayed. If you end up finding out more on this, could you reply back. Else, if I do, I'll let you know.

Thanks, Aaron
hoolio
Cirrostratus
Sep 22, 2010
Hi Robert,

Thanks for the update. What was the bug ID they quoted?

Aaron
Robert_Reznicek
Nimbostratus
Sep 23, 2010
Hi Aaron,

The bug id is 222648.

They proposed to use bp class ... add/delete instead of loading the whole datagroup. I'm going to test this....

Robert
hoolio
Cirrostratus
Mar 16, 2011
Hi Robert,

Did you ever try testing this? I'll probably need to revisit this issue for an upcoming iRule and figured I'd check with you on it.

ID222648 doesn't look to have been acted on for 11.0. So if anyone sees value in getting this looked at again, it would help to open cases with F5 Support asking for them to raise the priority on the request.

Thanks, Aaron
Austin_Geraci
MVP
Mar 16, 2011
Ahh i've noticed the same behavior..I'll open a case this week to support the cause.