BIG-IP API - WAF suggestions - issue in selecting properties

[jeromeb]

Hello there,

I'm currently working on a Python application (web + API) retrieving policies and suggestions from F5 via F5's REST API.

I have a usecase in which I want to know, for one given suggestion, if it has at least one blocked request. I'm getting suggestion data using following API request :

https://f5-ip/mgmt/tm/asm/policies/policy-id/suggestions/suggestion-id

I'm selecting various properties with '$select' query parameter, and to get the info I want on blocked requests, I'm expanding 'requestReferences' property, and then select what I'm interested in, which leads to (simplified to reproduce the issue):

?$expand=requestReferences&
$select=requests/enforcementState/isBlocked,requests/enforcementState/isUnblocked

The problem I'm encountering is the F5 API is only retrieving 'isUnblocked' property, and not 'isBlocked' property. Doing more tests, it seems it's only retrieving the last property of the '$select' parameter. This only happens with "level 2" properties, I have no issue selecting other properties of the suggestion.

Doing some research on here, I found a F5 SDK exists for Python apps. I tested it and encounter the same issue (only last level 2 property is retrieved).

As a workaround to this problem, I discovered using wildcard * allows to retrieve all properties of 'enforcementState' object. But in my usecase, this results in overloading the result with data I won't be using.

Is this a limitation of the F5 API? Or am I just missing something in how I'm using the API?

Thanks a lot for your help!

F5 API version : 13.1.0

F5 SDK version tested : 3.0.21

[Ivan_Chernenkii]

Hello jeromed,

If I understood you usecase correctly, you want to know, for one given suggestion, if it has at least one blocked request or not.

May be it will be better to use next $filter in such case - ?$expand=requests&$filter=requests/requestStatus eq 'blocked'

Thanks, Ivan

[jeromeb]

Hello Ivan,

And thank you for your quick answer!

Your understanding of my usecase is correct. I tried to test what you suggested, but it seems the property 'requestStatus' does not exist in requests properties. Close match could be 'acceptStatus' but I'm not sure it would have the same meaning.

As you suggest to filter on this property, do you agree that I should be able to select (using $select) it also?

Thanks,

Jérôme

[Ivan_Chernenkii]

Hello Jerome,

it is very strange that you don’t see requestStatus for request. It should always exist.

Could you post your Rest request and response here?

As I understand $select is not needed and you can just use $filter.

Thanks, Ivan

[jeromeb]

Hello Ivan,

This is an example request, with only expanding requestReferences:

https://f5-ip/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q/suggestions/CtRldjHPKNuGDAQJULFuOg?$expand=requestReferences

And the response I'm getting (replacing sensitive data and HTTP requests):

{
    "parentEntityId": "",
    "isAutomaticallyLearnable": false,
    "isRead": true,
    "occurrences": 3433,
    "status": "pending",
    "lastOccurrenceDatetime": "2020-04-08T09:59:41Z",
    "kind": "tm:asm:policies:suggestions:suggestionstate",
    "selfLink": "https://localhost/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q/suggestions/CtRldjHPKNuGDAQJULFuOg?$expand=requestReferences&ver=13.1.0",
    "entityId": "",
    "entityName": "Policy General Settings",
    "trustedSourcesCount": 0,
    "id": "CtRldjHPKNuGDAQJULFuOg",
    "averageViolationRating": 4.8,
    "violationRatingCounts": [
        {
            "violationRating": "0",
            "occurrences": 0
        },
        {
            "violationRating": "1",
            "occurrences": 136
        },
        {
            "violationRating": "2",
            "occurrences": 0
        },
        {
            "violationRating": "3",
            "occurrences": 0
        },
        {
            "violationRating": "4",
            "occurrences": 10
        },
        {
            "violationRating": "5",
            "occurrences": 3287
        }
    ],
    "requests": [
        {
            "deviceId": "",
            "tagReferences": [],
            "clientIpIntelligence": [],
            "serverPort": 443,
            "schema": "https",
            "selfLink": "https://localhost/mgmt/tm/asm/events/requests/6844098169184131093?ver=13.1.0",
            "sessionId": "adf7ba8367f7ed34",
            "url": "/app/Appli.git/info/refs",
            "id": "6844098169184131093",
            "responseCode": 500,
            "loginResult": "N/A",
            "username":"xxxx",
            "serverIp": "1.2.3.4",
            "acceptStatus": "none",
            "suggestionReferences": [
                {
                    "link": "https://localhost/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q/suggestions/CtRldjHPKNuGDAQJULFuOg?ver=13.1.0"
                }
            ],
            "rawResponse": {
                "isTruncated": false,
                "exclusionReason": "disabled",
                "wasCompressed": false,
                "isBase64Encoded": false
            },
            "blockingExceptionReason": "none",
            "rawRequest": {
                "isTruncated": false,
                "actualSize": 397,
                "httpRequestUnescaped": "GET /...",
                "httpRequest": "GET /...",
                "isBase64Encoded": false
            },
            "mobileAppVersion": "",
            "isRead": false,
            "maxRequestHeaderLength": 57,
            "enforcementState": {
                "isBlocked": false,
                "hasViolations": true,
                "isUnblocked": false,
                "hasRequestViolations": false,
                "rating": 1,
                "hasStagingViolations": false,
                "isAlarmed": true,
                "hasResponseViolations": true,
                "attackTypeReferences": [
                    {
                        "link": "https://localhost/mgmt/tm/asm/attack-types/1DsOkn6MTcm3RAj6BJWfJg?ver=13.1.0"
                    }
                ],
                "severity": "informational"
            },
            "mobileAppName": "",
            "requestPolicyReference": {
                "link": "https://localhost/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q?ver=13.1.0"
            },
            "matchedLoggingProfile": true,
            "requestDatetime": "2020-04-08T09:59:41Z",
            "responseContentType": "text/html; charset=utf-8",
            "violations": [
                {
                    "httpResponseCode": 500,
                    "entityType": "response",
                    "location": "response",
                    "violationReference": {
                        "link": "https://localhost/mgmt/tm/asm/violations/cSJYowU55AQWD18mPfODOg?ver=13.1.0"
                    },
                    "enforcementState": {
                        "isBlocked": false,
                        "isLearned": true,
                        "isAlarmed": true,
                        "isInStaging": false
                    },
                    "severity": "informational"
                }
            ],
            "geolocationCountryCode": "ES",
            "method": "GET",
            "slot": 0,
            "isVisible": true,
            "responseDatetime": "2020-04-08T09:59:41Z",
            "clientType": "uncategorized",
            "maxRequestCookieLength": 0,
            "host": "abc.fr",
            "clientIp": "1.2.3.4",
            "clientPort": 36098,
            "comment": "",
            "virtualServerName": "/PART_UO0/vs-name",
            "captchaResult": "captcha-unknown"
        }
	],
	"entityKind": "tm:asm:policies:general:generalstate",
    "entityReference": {
        "link": "https://localhost/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q/general?ver=13.1.0"
    },
    "reason": "violation-mitigation",
    "score": 100,
    "sectionReference": {
        "link": "https://localhost/mgmt/tm/asm/policies/9cBBD8iON_18jWFIBpEa_Q/sections/xMpCOKC5I4INzFCab3WEmw?ver=13.1.0"
    },
    "firstOccurrenceDatetime": "2020-02-04T14:57:39Z",
    "description": "Add 500 to Allowed Response Codes.",
    "isGloballyAcceptable": false,
    "entityChanges": {
        "allowedResponseCodes": [
            500
        ]
    },
    "untrustedSourcesCount": 20,
    "comment": "",
    "violationReference": {
        "link": "https://localhost/mgmt/tm/asm/violations/cSJYowU55AQWD18mPfODOg?ver=13.1.0"
    },
    "action": "update-append",
    "isTighteningSuggestion": false
}

There are many more requests in the original response I'm getting from F5, but not any of them have the requestStatus property.

I'm focusing on using $select more than $filter because my app is retrieving suggestions from F5, whether they have blocked requests or not (and I'm also selecting various other properties at the same time). But I think what you're proposing here should work in both cases anyway 🙂

Thanks again for your help!

Jérôme

[Ivan_Chernenkii · Accepted solution]

Hello Jerome,

Yes, it looks you don't have requestStatus.

Most probably we don't have it in version 13.1.0, while we have it in further versions.

In such case you can use next request - /mgmt/tm/asm/policies/6224t7jz2UltQZsOfifTog/suggestions/cMuuspF_0p5Me-nAYZyWRg?$expand=requestReferences&$filter=requests/enforcementState/isBlocked eq true and requests/enforcementState/isUnblocked eq false. This request will give you 404 response in case of no blocked requests for suggestion and 200 response in case of blocked requests exist.

As I understand, you don't need to use $select and you need to use $filter.

Thanks, Ivan

[jeromeb]

Hello Ivan,

Thanks for your feedback.

As I explained earlier (I guess it was not clear), I'd prefer to use $select because I'm getting a bunch of suggestions from F5 (that can have blocked requests or not) to list them in my application. Knowing if at least one request was blocked is just like getting any other property for me, but I don't want to explicitly filter on this criteria.

In all cases I'll try your suggestion and see if I can just do an extra request and see the performance impact in my application (i.e. do not change my current process and add one more API request to know if a request was blocked).

Two more questions if I may :

• are you able to reproduce the original problem? If so, would you consider it as a bug?
• you said requestStatus property is available in a newer version? Would you know by any chance in which version? Maybe we could consider an update on our side.

Thanks a lot for your help,

Jérôme

[Ivan_Chernenkii]

1) Yes, I think this is a bug. I will open it.

2) requestStatus property was introduced in version 15.0.0

Thanks, Ivan

[Ivan_Chernenkii]

Hello Jerome,

Does any solution help you?

Thanks, Ivan

[jeromeb]

Hello Ivan,

I've been able to get the information on blocked requests using a mix of all suggestions in our discussion (using $select with requests/enforcementState/*). Result is acceptable for now and not impacting too much performance.

Thanks again for your help !

Jérôme