How to add Syslog headers to Bot Defense logs over HSL? (Missing formatting options)
Hi DevCentral Community, I am running into issue with logging Bot Defense events to our SEIM (AIsaac) and could use some advice on best practices. We have logging profile configured to send both Application Security (ASM) and Bot Defense logs to a Remote Publisher. The Publisher is currently tied to a Remote HSL(High-Speed-Logging) destination. The Problem: For standard ASM WAF logs, we can easily format the log string directly in the GUI under the Applications Security logging tab. However, under the Bot Defense logging tab, there is no option to customize the log format. Because it is sending directly to a raw HSL destination, the Bot Defense logs are arriving at out SEIM completely stripped of standard Syslog headers. Without these headers, the SEIM cannot parse the logs correctly. My Questions: Is inserting a Syslog formatted destination before the HSL destination the official way to inject standard headers into Bot Defense logs? Is there any hidden tmsh command or iRule method to actually customize the Bot Defense log payload format, or is the payload structure strictly fixed by the system?F5 AWAF/ASM learning only from Trusted traffic?
I found this nice option "Only from Trusted Traffic" for the Policy Builder but this is seems to relevant only after the learning period has passed. I did increase the thresholds to the max possible value 1000000000 under "Loosen Policy" for "Untrusted Traffic "as to never learn from not trusted IP addresses in the initial learning period that is 7 days. I think that is the correct way ? I would have been nice to have a global option or option under "Loosen Policy" to learn from "Only from Trusted Traffic" like in "Track Site ".F5 AWAF/ASM ASM_RESPONSE_VIOLATION event seem to not trigger on 17.1.x
Hey Everyone, The F5 AWAF/ASM ASM_RESPONSE_VIOLATION event seem to not trigger on 17.1.x. I have enabled irules support the waf policy and I tested in Normal and Compatibility mode but no luck. The other events trigger without an issue. I created 2 custom signatures for response and request match and request match one has no issues so it seems a bug to me. This can be easily tested with the below irule that logs to /var/log/asm when ASM_REQUEST_DONE { log local3. "test request" } when ASM_RESPONSE_VIOLATION { log local3. "test response" } The custom response signature is in the policy to just trigger alarm. I tried string or regex match " (?i)failed " PCRE-style as F5 15.x and up are using this regex style.
Is it possible to select ASM BoT profile from irule?
Hi. . Is it possible to select BoT profile from irule? . Concept is we have different set of IP which need to allow "some" BoT type. That why we can't use whitelist IP in BoT profile because it will allow all BoT type. So We want to use iRule to check if it IP A > use BoT profile which have some exception, but if all other IP > use normally BoT profile. . when HTTP_REQUEST { # Check IP and select BoT profile from that if { [IP::client_addr] eq "A" } { ASM::enable allow_some_bot_profile } else { ASM::enable normally_bot_profile } } ps. I didn't see any document about how to select BoT profile. So I'm not sure if ASM::enable can do that.Trigger js challenge/Captcha for ip reputation/ip intelligence categories
Problem solved by this Code Snippet Because some ISP or cloud providers do not monitor their users a lot of times client ip addresses are marked as "spam sources" or "windows exploits" and as the ip addresses are dynamic and after time a legitimate user can use this ip addresses the categories are often stopped in the IP intelligence profile or under the ASM/AWAF policy. This usually happens in Public Clouds that do not monitor what their users do and the IP gets marked as bad then another good user after a day or two has this ip address and this causes the issue. For many of my clients I had to stop the ip reputation/ip intelligence category "spam sources" and in some cases "windows exploits" so having a javascript/captcha checks seems a nice compromise 😎 To still make use of this categories the users coming from those ip addresses can be forced to solve captcha checks or at least to be checked for javascript support! How to use this Code Snippet Have AWAF/ASM and ip intelligence licensed Add AWAF/ASM policy with irule support option (by default not enabled under the policy) or/and Bot profile under the Virtual server Optionally add IP intelligence profile or enable the Ip intelligence under the WAF policy without the categories that cause a lot of false positives, Add the irule and if needed modify the categories for which it triggers Do not forget to first create the data group, used in the code or delete that part of the code and to uncomment the Bot part of the code, if you plan to do js check and not captcha and maybe comment the captcha part ! Code Snippet Meta Information Version: 17.1.3 Coding Language: TCL Code You can find the code and further documentation in my GitHub repository: reputation-javascript-captcha-challlenge/ at main · Nikoolayy1/reputation-javascript-captcha-challlenge when HTTP_REQUEST { # Take the ip address for ip reputation/intelligence check from the XFF header if it comes from the whitelisted source ip addresses in data group "client_ip_class" if { [HTTP::header exists "X-Forwarded-For"] && [class match [IP::client_addr] equals "/Common/client_ip_class"] } { set trueIP [HTTP::header "X-Forwarded-For"] } else { set trueIP [IP::client_addr] } # Check if IP reputation is triggered and it is containing "Spam Sources" if { ([llength [IP::reputation $trueIP]] != 0) && ([IP::reputation $trueIP] contains "Spam Sources") }{ log local0. "The category is [IP::reputation $trueIP] from [IP::client_addr]" # Set the variable 1 or bulean true as to trigger ASM captcha or bot defense javascript set js_ch 1 } else { set js_ch 0 } # Custom response page just for testing if there is no real backend origin server for testing if {!$js_ch} { HTTP::respond 200 content { <html> <head> <title>Apology Page</title> </head> <body> We are sorry, but the site you are looking for is temporarily out of service<br> If you feel you have reached this page in error, please try again. </body> </html> } } } # when BOTDEFENSE_ACTION { # Trigger bot defense action javascript check for Spam Sources # if {$js_ch && (not ([BOTDEFENSE::reason] starts_with "passed browser challenge")) && ([BOTDEFENSE::action] eq "allow") }{ # BOTDEFENSE::action browser_challenge # } # } when ASM_REQUEST_DONE { # Trigger ASM captcha check only for users comming from Spam sources that have not already passed the captcha check (don't have the captcha cookie) if {$js_ch && [ASM::captcha_status] ne "correct"} { set res [ASM::captcha] if {$res ne "ok"} { log local0. "Cannot send captcha_challenge: \"$res\"" } } } Extra References: BOTDEFENSE::action ASM::captcha ASM::captcha_statusF5 ASM/AWAF Preventing unauthorized users accessing admin path using iRule script
The below code uses the new BIG-IP variables " [ASM::is_authenticated] " and " [ASM::username] " and the code is simple enough as if you are authenticated but not admin then you will not get access to the url path " /about.php " and this is logged in the /var/log/asm logs because " log local3. ". At the end of the article I have shown how with APM you can accomplish AD group limit for specific urls but then the Authentication is moved on the APM while the AWAF iRule example the authentication is on the origin web server and the AWAF just handles the URL Authorization. when ASM_REQUEST_DONE { if { [ASM::is_authenticated] && [HTTP::path] equals "/about.php" } { log local3. "This request was sent by user [ASM::username]." if {[ASM::username] equals "admin"} { log local3. "The admin has logged!" return } else { drop } } } Github link: Nikoolayy1/F5_AWAF-ASM-ADMIN-Access: F5 BIG-IP iRule code for limiting users by to access urls! The harder part is that you need to do several prerequisites that I will explain here: Enable iRule support in the ASM policy. Configure a login page and optionally login enforcement (if " /about.php " is not blocked by the origin server to not be accessible before login this is a needed step!) Enable session tracking by login page Attach the irule Test and see Example logs: cat /var/log/asm ......... Jun 25 03:59:33 bigip1.com info tmm2[11400]: Rule /Common/f5-asm-allow-admin <ASM_REQUEST_DONE>: This request was sent by user admin. Jun 25 03:59:33 bigip1.com info tmm2[11400]: Rule /Common/f5-asm-allow-admin <ASM_REQUEST_DONE>: The admin has logged! [root@bigip1:Active:Standalone] config # The DVWA app was used for this demo that is old but gold and there are many F5 demos how to configure login enforcement for it! Here is a youtube video for assistance: BIG-IP AWAF Demo 32 - Use Login Page Enforcement with F5 BIG-IP Adv WAF (formerly ASM) Extra links (there is also a new event "ASM_RESPONSE_LOGIN"): ASM::username ASM::is_authenticated https://clouddocs.f5.com/api/irules/ASM.html AD group url enforcement: If you want to control access to URLs based on AD groups I suggest seeing the F5 APM/Acess module that will take of the authentication and with Layer 7 ACL each AD group could be limited what it has access to. APM and AWAF can work together as with layered virtual server AWAF can be before the APM as by default is after it and then to get the username you need to use the login page feature and not "Use APM username and Session ID" feature in the AWAF policy. Configuring Access Control Lists https://my.f5.com/manage/s/article/K00363504 https://my.f5.com/manage/s/article/K03113285 https://my.f5.com/manage/s/article/K54217479 Example APM profile of type LTM+APM and the APM policy for anyone interested where the APM uses AD to authenticate the users and query for group data and the members for of the guest group have an ACL assigned that limits their access 😜 Summary: This probably will be seen as well in F5 NEXT with many more cool features !
