Forum Discussion
AltostratusDynamic import of data groups
Hello.
We use data groups for various kind of black lists, such as undesirable user agents, for instance. That's really efficient, but requires a BigIP administrator intervention for any update. We'd like to switch authoritative origin for those lists to an external location, such as an internal git repository, in order to allow trusted people without access to the administration interface to update those lists in auditable manner, as we do for instance with our firewalls using "dynamic list" feature.
There seems to be no such native fonctionality in BigIPs, as even "external" dynamic lists actually relies on files hosted on local filesystem, not to arbitrary URLs. We could probably use a cron task to implement a pull-based update mechanism, or use the API to periodically push changes, but I'm not sure of the reliability of such ad-hoc mechanism, and the potential consequences in case of failure.
Is there any alternative for such kind of configuration delegation ?
Regards,
Guillaume
2 Replies
BIG-IP doesn’t natively support URL-based dynamic data groups, so to delegate updates safely you need automation around it;
the most robust approach is to treat a Git repository as the source of truth and use a CI/CD pipeline to validate and push updates to BIG-IP via the iControl REST API (fully auditable and controlled),
while a simpler alternative is using external data groups with a pull mechanism (cron or iCall) to fetch files from a central repo, though this requires careful handling for failures and validation; in practice, most enterprises prefer the Git + pipeline + API (push model) because it provides better reliability, access control, and rollback compared to ad-hoc pull-based solutions.
You can try following options of your choice for automating this kind of request:
1. iControl REST API + Git (Push Model): Use a CI/CD pipeline to validate and push data group updates from Git to BIG-IP via API — most reliable and auditable.
2. External Data Group + File Sync (Pull Model): BIG-IP periodically pulls a file from a central repo and reloads config — simple but needs careful error handling.
3. iCall Automation: Native BIG-IP scripting to fetch and update data groups on a schedule — avoids external schedulers but has limited robustness.
4. AS3 (Application Services 3): Manage data groups declaratively in JSON and deploy via API — ideal for modern GitOps-style environments.
5. Hybrid Approach: Combine Git, CI/CD, and optional fallback pull mechanisms — best balance of reliability, control, and resilience in enterprise setups.
Let me know if you want to discuss more details.
HTH
F5 Design engineer
- Juergen_Mang
MVP
I would create a simple pipeline script that updates the Datagroup via REST-API on repository change. I personally use GitLab and GitLab Runners for such things in my daily job and it works great.
If you want to create simple script to manage F5 objects via REST API I can suggest Restsh, but that's not an unbiased statement, since I wrote it.
