Update a DataGroup using REST and Powershell
Problem this snippet solves:
We have load balancers in two geographically disperse datacenters that are also globally load balanced. It started becoming an inconvenience to add allowed IP addresses to our SMTP relay data group in both datacenters as we spun up additional printers at one of our many sites, so I created this script to use the REST API and update both load balancers with the same configuration simultaneous for this one group. This script is single purpose for me but I figure all anyone really needs to do is edit it for their own use and potentially save themselves some time. DataGroups don't allow a REST delete so I am basically recreating the JSON and overwriting the existing configuration.
How to use this snippet:
When you run it you will be presented with four options.
- Add - This will add an individual IP address. It also makes a backup of the current DataGroup configuration in case something goes wrong.
- Delete - This will delete an individual IP address. It also makes a backup of the current DataGroup configuration in case something goes wrong.
- Backup - This will backup up the existing configuration on both load balancers. These get saved as ArchiveDG-Date and will not be removed unless done manually(I like being able to do a point in time.)
- Restore - This will restore a backed up configuration.
After you select the option you wish to use you will be prompted for the credentials to your F5, the username and password has to be the same on both load balancers. I will most likely save an encrypted credential long term, but this works for my needs at the moment. The oldest backups will be cleared after 10 saves have been completed.
Code :
## Create a policy to trust all Certs add-type @" using System.Net; using System.Security.Cryptography.X509Certificates; public class TrustAllCertsPolicy : ICertificatePolicy { public bool CheckValidationResult( ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) { return true; } } "@ [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy ##Variables $LoadBalancer1 = "1.1.1.1" $LoadBalancer2 = "2.2.2.2" $DG = "SMTP_Allowed" $DGURL1 = "https://$LoadBalancer1/mgmt/tm/ltm/data-group/internal/~Common~$DG" $DGURL2 = "https://$LoadBalancer2/mgmt/tm/ltm/data-group/internal/~Common~$DG" $Partition = "Common" ##Functions function BackupDG { Param( $FileName = "BackupDG", $RESTURL ) $Date = Get-Date -Format MMddyy-hh_mm_ss Get-ChildItem BackupDG* | where{-not $_.PsIsContainer} | sort CreationTime -desc| select -Skip 20 | Remove-Item -Force $DGJSONBody = @{name=$DG;partition=$Partition;records=@()} $Records = @() ## Lookup Virtual Servers $DataGroupLookup = Invoke-RestMethod $RESTURL -Credential $Credential foreach ($DataGroup in $DataGroupLookup) { foreach ($AllowedIPAddr in $DataGroup.Records) { $AllowedIPAddrName = $AllowedIPAddr.Name $Records += @{name=$AllowedIPAddrName} } } $DGJSONBody.Records = $Records $DGJSONBody = $DGJSONBody | ConvertTo-Json $DGJSONBody | Out-File "./$FileName-$Date.json" $RecordCount = ($DataGroup.Records).Count Write-Host "Successfully Backed Up $RecordCount Records" } ##Run Script Write-Host "1. Add" Write-Host "2. Remove" Write-Host "3. Backup" Write-Host "4. Restore" $DGMaint = Read-Host "Selection" if ($DGMaint -eq "1"){ $Credential = Get-Credential $AllowedPrinter = Read-Host "Printer IP Address" BackupDG -RESTURL $DGURL1 -FileName "BackupDGLB1" #BackupDG -RESTURL $DGURL2 -FileName "BackupDGLB2" $DGJSONBody = @{name=$DG;partition=$Partition;records=@()} $Records = @() $Records += @{name="$AllowedPrinter/32"} ## Lookup Virtual Servers $DataGroupLookup = Invoke-RestMethod $DGURL1 -Credential $Credential foreach ($DataGroup in $DataGroupLookup) { foreach ($AllowedIPAddr in $DataGroup.Records) { $AllowedIPAddrName = $AllowedIPAddr.Name $Records += @{name=$AllowedIPAddrName} } } $DGJSONBody.Records = $Records $DGJSONBody = $DGJSONBody | ConvertTo-Json #$DGJSONBody Write-Host "Adding Printer IP: $AllowedPrinter to the SMTP allow list." $InvokeAdd1 = Invoke-RestMethod $DGURL1 -Credential $Credential -Method Put -Body $DGJSONBody $InvokeAdd2 = Invoke-RestMethod $DGURL2 -Credential $Credential -Method Put -Body $DGJSONBody } elseif ($DGMaint -eq "2"){ $Credential = Get-Credential $RemovePrinter = Read-Host "Printer IP Address" BackupDG -RESTURL $DGURL1 -FileName "BackupDGLB1" #BackupDG -RESTURL $DGURL2 -FileName "BackupDGLB2" $DGJSONBody = @{name=$DG;partition=$Partition;records=@()} $Records = @() ## Lookup Virtual Servers $DataGroupLookup = Invoke-RestMethod $DGURL1 -Credential $Credential foreach ($DataGroup in $DataGroupLookup) { foreach ($AllowedIPAddr in $DataGroup.Records) { $AllowedIPAddrName = $AllowedIPAddr.Name if ($AllowedIPAddrName -eq "$RemovePrinter/32"){} else{ $Records += @{name=$AllowedIPAddrName} } } } $DGJSONBody.Records = $Records $DGJSONBody = $DGJSONBody | ConvertTo-Json #$DGJSONBody Write-Host "Removing Printer IP: $RemovePrinter from the SMTP allow list." $InvokeRemove1 = Invoke-RestMethod $DGURL1 -Credential $Credential -Method Put -Body $DGJSONBody $InvokeRemove2 = Invoke-RestMethod $DGURL2 -Credential $Credential -Method Put -Body $DGJSONBody } elseif ($DGMaint -eq "3"){ $Credential = Get-Credential BackupDG -RESTURL $DGURL1 -FileName "ArchiveDGLB1" BackupDG -RESTURL $DGURL2 -FileName "ArchiveDGLB2" } elseif ($DGMaint -eq "4"){ $Credential = Get-Credential Write-Host "Configuration Backup Files" $ConfigFiles = Get-ChildItem *.json |Select Name foreach ($ConfigFile in $ConfigFiles){ Write-Host $ConfigFile.Name } $RestoreFile = Read-Host "File to Restore?" $DGJSONBody = Get-Content "./$RestoreFile" $InvokeRestore1 = Invoke-RestMethod $DGURL1 -Credential $Credential -Method Put -Body $DGJSONBody $InvokeRestore2 = Invoke-RestMethod $DGURL2 -Credential $Credential -Method Put -Body $DGJSONBody } else {}
- ScottishEngineeNimbostratus
I have a scenario where I need the "data" too, not just the "name". This script does what I want but I need to add/update the "data", which is the "Value:" textbox in the F5 admin portal for iRules Data Groups. When I run the following I get the "code":400,"message":"one or more configuration identifiers must be provided" error:
$Records += @{name=$AllowedIPAddrName}
$Records += @{data=$AllowedIPAddrData}
The first line above is your entry and the second one is mine after I created the variable to hold the value for "data".
The data is correct as the DGJSONBody contains the records with the correct "data", it gives me the error when it is put in the Invoke-RestMethod.
Here is my code for the Add new entry only:
## Create a policy to trust all Certs add-type @" using System.Net; using System.Security.Cryptography.X509Certificates; public class TrustAllCertsPolicy : ICertificatePolicy { public bool CheckValidationResult( ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) { return true; } } "@ [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy ##Variables $LoadBalancer1 = "xx.xx.xxx.xxx" $DG = "Scott_Test" $DGURL1 = "https://$LoadBalancer1/mgmt/tm/ltm/data-group/internal/~Common~$DG" $Partition = "Common" ##Functions function BackupDG { Param( $FileName = "Scott_Test_BackupDG", $RESTURL ) $Date = Get-Date -Format MMddyy-hh_mm_ss Get-ChildItem BackupDG* | where{-not $_.PsIsContainer} | sort CreationTime -desc| select -Skip 20 | Remove-Item -Force $DGJSONBody = @{name=$DG;partition=$Partition;records=@()} $Records = @() ## Lookup Virtual Server and get Data Group $DataGroupLookup = Invoke-RestMethod $RESTURL -Credential $Credential foreach ($DataGroup in $DataGroupLookup) { foreach ($AllowedIPAddr in $DataGroup.Records) { $AllowedIPAddrName = $AllowedIPAddr.name $AllowedIPAddrData = $AllowedIPAddr.data $Records += @{name=$AllowedIPAddrName} $Records += @{data=$AllowedIPAddrData} } } $DGJSONBody.Records = $Records $DGJSONBody = $DGJSONBody | ConvertTo-Json $DGJSONBody | Out-File "C:/F5/$FileName-$Date.json" $RecordCount = ($DataGroup.Records).Count Write-Host "Successfully Backed Up $RecordCount Records" } ##Add New IP address and append $Records variable Write-Host "1. Add" #Write-Host "2. Remove" #Write-Host "3. Backup" #Write-Host "4. Restore" $DGMaint = Read-Host "Selection" if ($DGMaint -eq "1"){ #$Credential = Get-Credential $AllowedCustomerIP = Read-Host "Customer IP Address" $AllowedCustomerDataNew = Read-Host "Description & JSD Number - NO spaces, Alphanumeric and dashes/underscores only" BackupDG -RESTURL $DGURL1 -FileName "BackupDGLB1" $DGJSONBody = @{name=$DG;partition=$Partition;records=@()} $Records = @() $Records += @{name="$AllowedCustomerIP/32"} $Records += @{data=$AllowedCustomerDataNew} ## Lookup Virtual Servers $DataGroupLookup = Invoke-RestMethod $DGURL1 -Credential $Credential foreach ($DataGroup in $DataGroupLookup) { foreach ($AllowedIPAddr in $DataGroup.Records) { $AllowedIPAddrName = $AllowedIPAddr.name $AllowedCustomerData = $AllowedIPAddr.data $Records += @{name=$AllowedIPAddrName} $Records += @{data=$AllowedCustomerData} } } $DGJSONBody.Records = $Records $DGJSONBody = $DGJSONBody | ConvertTo-Json #$DGJSONBody Write-Host "Adding Customer IP: $AllowedCustomerIP to the SMTP allow list." $InvokeAdd1 = Invoke-RestMethod $DGURL1 -Credential $Credential -Method Put -Body $DGJSONBody }
You'll see I hacked it up pretty good! No other error occurs prior to sending at line 85. The error is so vague, no pointers to anything, and a Google search is no joy either, maybe a string/integer issue with how the "data" field is populated in the Record?
Any information would be greatly appreciated, thanks!