Weblogic JSessionID Persistence
Problem this snippet solves: Contributed by: unRuleY, Summarized by: deb Note: The previous version of this iRule contained escaped newlines following the session command, which in versions 10.0 - 10.2.0 causes TMM to core as documented in CR135937 / SOL11427. This was fixed in 10.2.1. See this related Codeshare example for details on how to take advantage of session replication on the WebLogic servers with targeted node failover in an iRule. Provides persistence on the jsessionid value found in either the URI or a cookie. When a request is received, the iRule first looks for a "jsessionid" cookie, and if not found, for a "jsessionid" parameter in the requested URI. If either is found, a persistence record is created if it doesn't already exist, or followed if it does. If neither is found, the request is load balanced according to the load balancing method applied to the virtual server and persisted based on the client's IP address. In order to ensure the second and subsequent requests follow the first, LTM must create a persistence record indicating the pool member to which the first request was load balanced. If the server is setting the jsessionid in a cookie, the persistence key value may be extracted from the server response to create the persistence record. If the server is setting the jsessionid in the URLs, source address persistence with a short timeout is recommended to track the original destination until the jsessionid is sent. How to use this snippet: To ensure a new persistence record is followed when a request is re-load balanced in a client-side Keep-Alive connection, apply a OneConnect profile to the virtual server. The iRule assumes the jsessionid is in upper case when used as a cookie name. If this isn't the case, please update the example. To persist on jsessionid, create the iRule below and create a custom Universal persistence profile, with Match Across Services enabled, that uses the iRule. Then use this custom Universal persistence profile as the Default Persistence profile on your Virtual Server. Applying a Fallback Persistence profile of type Source Address Affinity with a host mask and a short timeout (the default source_addr persistence profile will do the trick) to your Virtual Server is also recommended. Attention, if you are running firmware 11.0 - 11.2.1 and enabled "Match Across Services"! There is a bug inside. SOL14061 This iRule requires LTM v10. or higher. Code : when HTTP_REQUEST { # Log details for the request set log_prefix "[IP::client_addr]:[TCP::client_port]" log local0. "$log_prefix: Request to [HTTP::uri] with cookie: [HTTP::cookie value JSESSIONID]" # Check if there is a JSESSIONID cookie if { [HTTP::cookie "JSESSIONID"] ne "" }{ # Persist off of the cookie value with a timeout of 1 hour (3600 seconds) persist uie [string tolower [HTTP::cookie "JSESSIONID"]] 3600 # Log that we're using the cookie value for persistence and the persistence key if it exists. log local0. "$log_prefix: Used persistence record from cookie. Existing key? [persist lookup uie [string tolower [HTTP::cookie "JSESSIONID"]]]" } else { # Parse the jsessionid from the path. The jsessionid, when included in the URI, is in the path, # not the query string: /path/to/file.ext;jsessionid=1234?param=value set jsess [findstr [string tolower [HTTP::path]] "jsessionid=" 11] # Use the jsessionid from the path for persisting with a timeout of 1 hour (3600 seconds) if { $jsess != "" } { persist uie $jsess 3600 # Log that we're using the path jessionid for persistence and the persistence key if it exists. log local0. "$log_prefix: Used persistence record from path: [persist lookup uie $jsess]" } } } when HTTP_RESPONSE { # Check if there is a jsessionid cookie in the response if { [HTTP::cookie "JSESSIONID"] ne "" }{ # Persist off of the cookie value with a timeout of 1 hour (3600 seconds) persist add uie [string tolower [HTTP::cookie "JSESSIONID"]] 3600 log local0. "$log_prefix: Added persistence record from cookie: [persist lookup uie [string tolower [HTTP::cookie "JSESSIONID"]]]" } }Encrypted UCS Backup with REST-API
Because it seems this nowhere documented: Create a encrypted F5 backup with REST-API including private keys. This script should creates the task, starts it and get's it status. #!/usr/bin/env bash CURL_OPTS=("--fail-with-body" "--show-error" "-s" "-k" "-u" "user:pass" "-H" "Content-Type: application/json" "-H" "Accept: application/json, */*") # Create task and get id TASK_ID=$(jq -n --arg name /var/local/ucs/test.ucs \ --arg passphrase "testpw" \ '{ "command": "save", "name": $name, "options": [ { "passphrase": $passphrase } ] }' \ | curl "${CURL_OPTS[@]}" -X POST -d @- https://f5-lab/mgmt/tm/task/sys/ucs \ | jq -r "._taskId") # Start task jq -n '{ "_taskState": "VALIDATING" }' | curl "${CURL_OPTS[@]}" -X PUT -d @- "https://f5-lab/mgmt/tm/task/sys/ucs/$TASK_ID" # Get task status curl "${CURL_OPTS[@]}" --retry 5 --retry-all-errors --retry-delay 10 "https://f5-lab/mgmt/tm/task/sys/ucs/$TASK_ID" \ | jq -r "._taskState" Reference was https://my.f5.com/manage/s/article/K000138875 and the passphrase options was found by trial and error.Logstash pipeline tester
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. Short Description A tool that makes developing logstash pipelines much much easier. Problem solved by this Code Snippet Oh. The problem... Have you ever tried to write a logstash pipeline? Did you suffer hair loss and splitting migraines? So did I. Presenting, logstash pipeline tester which gives you a web interface where you can paste raw logs, send them to the included logstash instance and see the result directly in the interface. The included logstash instance is also configured to automatically reload once it detects a config change. How to use this Code Snippet TLDR; Don't do this, read the manual or checkout the video below Still here? Ok then! 🙂 Install docker Clone the repo Run these commands in the repo root folder:sudo docker-compose build # Skip sudo if running Windows sudo docker compose up # Skip sudo if running WindowsGo to http://localhost:8080 on your PC/Mac Pick a pipeline and send data Edit the pipeline Send data Rince, repeat Version info v1.0.27: Dependency updates, jest test retries and more since 1.0.0 https://github.com/epacke/logstash-pipeline-tester/releases/tag/v1.0.29 Video on how to get started: https://youtu.be/Q3IQeXWoqLQ Please note that I accidentally started the interface on port 3000 in the video while the shipped version uses port 8080. It took me roughly 5 hours and more retakes than I can count to make this video, so that mistake will be preserved for the internet to laugh at. 🙂 The manual: https://loadbalancing.se/2020/03/11/logstash-testing-tool/ Code Snippet Meta Information Version: Check GitHub Coding Language: NodeJS, Typescript + React Full Code Snippet https://github.com/epacke/logstash-pipeline-testerF5 DNS/GTM External Monitor(EAV) with SNI support and response code check
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. The example DNS/GTM health monitor is for versions before 16.1 as BIG-IP supports SNI for default DNS/GTM HTTPS monitor in the latest version but if you have still not upgraded then this is for you! I have used this monitor for XC Distributed Cloud as the HTTP LB share by default the same tenant IP address and SNI support is needed. You can order dedicated public IP addresses for each HTTP LB and enable "Default Load Balancer" ( https://my.f5.com/manage/s/article/K000152902 ) option but it will cost you extra 😉 The script is a modified version of External https health monitor for SNI-enabled pool as to handle response codes and to set the SNI globally for the entire pool and it's members. If you are uploading from Windows machine see External monitor fails to run as you could hit the bug. This could be needed for F5 DNS/GTM below 16.1 that do not support SNI in HTTPS monitors. The only mandatory variable is "SNI" that should be set in the external monitor config that references this uploaded bash script. The "URI" variable by default is set to "/" and "$2" variable by default is empty or 443, the default expected response code 200. #!/bin/sh # External monitoring script for checking HTTP status code # $1 = IP (::ffff:nnn.nnn.nnn.nnn notation or hostname) # $2 = port (optional; defaults to 443 if not provided) # Default SNI to IP if not explicitly provided node_ip=$(echo "$1" | sed 's/::ffff://') # Remove IPv6 compatibility prefix SNI=${SNI:-"$node_ip"} # Assign sanitized IP to SNI # Default variables MON_NAME=${MON_NAME:-"MyExtMon$$"} pidfile="/var/run/$MON_NAME.$1..$2.pid" # PID file path DEBUG=${DEBUG:-0} # Enable debugging if set to 1 EXPECTED_STATUS=${EXPECTED_STATUS:-200} # Default HTTP status code to 200 URI=${URI:-"/"} # Default URI DEFAULT_PORT=443 # Default port (used if $2 is unset) # Set port to default if $2 is not provided if [ -z "${2}" ]; then PORT=${DEFAULT_PORT} else PORT=${2} fi # Kill old process if pidfile exists if [ -f "$pidfile" ]; then kill -9 -$(cat "$pidfile") > /dev/null 2>&1 fi echo "$$" > "$pidfile" # Perform the HTTP(S) request via single curl (fetch status code only) status_code=$(curl -s -o /dev/null -w '%{http_code}' --connect-timeout 5 --resolve "${SNI}:${PORT}:${node_ip}" "https://${SNI}:${PORT}${URI}") # Cleanup rm -f "$pidfile" > /dev/null 2>&1 # Output server status based on HTTP status code match if [ "$status_code" -eq "$EXPECTED_STATUS" ]; then echo "up" else echo "down" fi # Debugging if [ "$DEBUG" -eq 1 ]; then echo "Debugging on..." echo "SNI=${SNI}" echo "URI=${URI}" echo "IP=${node_ip}" echo "PORT=${PORT}" echo "MON_NAME=${MON_NAME}" echo "STATUS_CODE=${status_code}" echo "EXPECTED_STATUS=${EXPECTED_STATUS}" echo "curl -s -o /dev/null -w '%{http_code}' --connect-timeout 5 --resolve ${SNI}:${PORT}:${node_ip} https://${SNI}:${PORT}${URI}" fiLess than 60 seconds lab setup
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. Today I'll share with you my less than 60 seconds lab setup which I use for testing basic stuff. It's an AS3 declaration that will setup two virtuals, the first virtual that accepts any http traffic on port 80 and forwards it to a second virtual that will respond 200 OK to any HTTP request. The lab can easily be extended to add a https virtual. Purpose of this setup I use this configuration for many scenarios. With this setup I can test different profiles, TLS configurations (requires small adjustments), AWAF rules and iRules attached to the first virtual server without the requirement to setup any backend application. Deploying this AS3 declaration takes less than 20 seconds and I have a basic lab environment ready. Prerequisites In order to use this config, you must have AS3 installed on your BIG-IP. If you have not worked with AS3 yet and you are new to automation, I recommend you to start with Visual Studio Code and install The F5 Extension. From The F5 Extension you can connect your BIG-IP and install the AS3 extension and deploy the declaration. Furthermore: if you have not with AS3 yet - you're damn late to the party! My AS3 declaration The full declaration is available on GitHub, let's just look at the iRules. The iRules are the important part of this lab config. Don't get confused that you won't see the iRule code in the AS3 declaration. It's there, but it's base64 encoded. Forwarding iRule The iRule attached to the first virtual just forwards to the second virtual. Don't get confused by the path /simple_testing/responder_service/. AS3 works with Partitions, so called tenants. Therefore I must reference the second virtual with the name of its partition and application. when HTTP_REQUEST { virtual /simple_testing/responder_service/service_http_200 } HTTP Responder iRule The second iRule is attached to the second virtual server. It will just return a HTML page that says 200 OK to any request. when HTTP_REQUEST { HTTP::respond 200 content { <html> <head> <title>BIG-IP</title> </head> <body> 200 OK </body> </html> } } Deployment As said above, for starting with this you don't need anything but a BIG-IP and Visual Studio Code. After installing the F5 Extension you can connect (using the + symbol) to your BIG-IP from VS Code. After connecting you can install the AS3 extension on your BIG-IP. And then you are ready to deploy the AS3 declaration linked above. The deployment will take less than 60 seconds. Once the deployment is done, you will have a Partition called on your BIG-IP. There you will find the two virtual servers. The website is nothing special... What's next? In the next couple of days, I will share with you a simple website I made with the help of ChatGPT. It can run on any webserver, NGINX, Apache, IIS... The website has 4 flavors (red, blue, green and yellow) and I use it for testing LTM use-cases like persistence, priority groups, http profiles, SNAT, etc. This will be my less than 600 seconds lab.Less than 600 seconds lab
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. In my previous post I shared with you, how you can deploy a lab environment in less than 60 seconds with AS3. This time let's take a look at another lab, that you can set up in less than 10 minutes. Purpose of this lab This lab requires a web server. And some minimal knowledge of Linux (Debian) and git. In my example, I use NGINX. The web application consists of four pages in four colours (red, blue, yellow and green) that are designed to demonstrate the load balancing functionality of the F5 Local Traffic Manager (LTM). You can use the app to familiarise yourself with load balancing functionalities such as: different load balancing methods and priority groups different types of persistence caching HTTP, SSL and other profiles SNAT The web application has a couple of nice features real-time server information display like Server hostname Request timestamp (ISO 8601 format) Request URI Source IP address X-Forwarded-For (XFF) header User-Agent informatio modern, responsive UI picture gallery Prerequisites First you need to set up and configure the web server. Add multiple IPs to the web server (Debian 11+). Edit /etc/network/interfaces: sudo nano /etc/network/interfaces Add the following: allow-hotplug eth0 iface eth0 inet static address 192.168.1.10/24 gateway 192.168.1.1 auto eth0:1 allow-hotplug eth0:1 iface eth0:1 inet static address 192.168.1.11/24 auto eth0:2 allow-hotplug eth0:2 iface eth0:2 inet static address 192.168.1.12/24 auto eth0:3 allow-hotplug eth0:3 iface eth0:3 inet static address 192.168.1.13/24 Restart networking: sudo systemctl restart networking Note: Replace eth0 with your actual interface name. Generate SSL Certificate Create a self-signed SSL certificate with RSA 2048-bit key (no password): openssl req -x509 -nodes -days 365 -newkey rsa:2048 \ -keyout nginx-selfsigned.key -out nginx-selfsigned.crt \ -subj "/C=US/ST=State/L=City/O=Organization/CN=example.com" Installing the web application Example for NGINX 1. Clone the repository git clone https://github.com/webserverdude/ltm-demo-html.git cd webpages 2. Deploy to your web server sudo cp -r * /var/www/ltm-demo-html 3. Configure your web server see below NGINX Configuration The configuration includes HTTP as well as HTTPS listeners. Add this configuration to your NGINX server block: server { listen 192.168.1.10:8000 default_server; root /var/www/ltm-demo-html; index index_red.html; server_name _; add_header X-Backend-Server 1; add_header Set-Cookie "X-Backend-Server=1; Max-Age=10"; location / { try_files $uri $uri/ =404; } # Enable the substitution filter sub_filter_once off; # Allow multiple substitutions # Replace template variables with actual NGINX variables sub_filter '{{server_name}}' '$hostname'; sub_filter '{{time_iso8601}}' '$time_iso8601'; sub_filter '{{request_uri}}' '$request_uri'; sub_filter '{{remote_addr}}' '$remote_addr'; sub_filter '{{http_x_forwarded_for}}' '$http_x_forwarded_for'; sub_filter '{{http_user_agent}}' '$http_user_agent'; } server { listen 10.0.2.71:443 ssl default_server; ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; # SSL configuration ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384; ssl_prefer_server_ciphers off; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; root /var/www/ltm-demo-html; index index_red.html; server_name _; add_header X-Backend-Server 1; add_header Set-Cookie "X-Backend-Server=$request_id; Max-Age=10; Secure; SameSite=Strict"; location / { try_files $uri $uri/ =404; } # Enable the substitution filter sub_filter_once off; # Allow multiple substitutions # Replace template variables with actual NGINX variables sub_filter '{{server_name}}' '$hostname'; sub_filter '{{time_iso8601}}' '$time_iso8601'; sub_filter '{{request_uri}}' '$request_uri'; sub_filter '{{remote_addr}}' '$remote_addr'; sub_filter '{{http_x_forwarded_for}}' '$http_x_forwarded_for'; sub_filter '{{http_user_agent}}' '$http_user_agent'; } Note: This is just a snippet for one HTTP and one HTTPS virtual. The full config for all four pages is available at my Git repository in the nginx_config folder. Once this is done, check the web pages from your browser. Make sure they work as expected. Configure your BIG-IP After the web server is running and serving all 4 pages with HTTP and HTTPS, you can configure your BIG-IP. My AS3 declaration includes an HTTP and an HTTPS virtual server, two pools and some http and persistence profiles. Here is a snippet: { "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/main/schema/latest/as3-schema.json", "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.0.0", "LTM_Demo": { "class": "Tenant", "LTM_Demo": { "class": "Application", "vs_http": { "class": "Service_HTTP", "virtualAddresses": [ "192.168.3.80" ], "persistenceMethods": [], "profileHTTP": { "use": "pr_http_xff" }, "pool": "pl_ltm-demo_http", "snat": { "use": "pl_SNAT_addresses" } }, ... The complete AS3 configuration can be found in my Git repository. The repository also contains an additional AS3 declaration with further configuration options. Note: You should not deploy the second declaration with the optional configurations; instead, merge the snippets you want to use into ltm_demo.json. Deployment The deployment of the AS3 declaration works similar like I described in my previous post. What's next? You can try differnet load balancing algorithms, persistence methods, caching, SSL configurations. Once you set up the web app and the LTM config, play around - the sky is the limit. Have fun!F5 MCP(Model Context Protocol) Server
This project is a MCP( Model Context Protocol ) server designed to interact with F5 devices using the iControl REST API. It provides a set of tools to manage F5 objects such as virtual servers (VIPs), pools, iRules, and profiles. The server is implemented using the FastMCP framework and exposes functionalities for creating, updating, listing, and deleting F5 objects.
Credit Card Scrubber
Problem this snippet solves: This iRule illustrates how to scrub out Credit Card Numbers from HTTP traffic. Let's say you want to specify a policy to not allow any credit card numbers outside of your network. How would you go about scrubbing out Credit Card Numbers? This isn't as simple as searching for a string pattern. CCNs vary in length depending on the issuer of the card. But one thing is common: they all must pass the Luhn Formula. Info on the Luhn Formula or MOD 10 can be found here. An excellent reference on credit card number makeup (beyond the 5 types checked in this iRule) is available here. How to use this snippet: This rule will match Diners (13 digit), Amex (15 digit), Visa (13 and 16 digit) Mastercard (16 Digit) and Discover (16 Digit). This example will look matching patterns looking like credit cards and return their indexes into the payload. Then the number is run through the Luhn formula (with optimizations by unRuleY). If it is indeed a valid credit card number, it is masked with X's. Further modifications added support for CCNs with - or a blank between the numbers. i.e. xxxx-xxxx-xxxx-xxxx, xxxx xxxx xxxx xxxx, xxxxxxxxxxxxxxxx, would match. To mask all but the last N digits with X's do the following. At the bottom of the script, you see this line: HTTP::payload replace $card_start $card_len [string repeat "X" $card_len] Add a small line before it like this: Here N=4. set card_len [expr {$card_len - 4}] I used the number 4 to replace all but the last 4 digits. Change this to the number of digits you want to leave untouched. so you end up with: set card_len [expr {$card_len - 4}] HTTP::payload replace $card_start $card_len [string repeat "X" $card_len] Note for an alternate method of implementing this iRule using the stream profile, check the Codeshare example. Code : when HTTP_REQUEST { # Prevent the server from sending a compressed response # remove the compression offerings from the client HTTP::header remove "Accept-Encoding" # Don't allow response data to be chunked if { [HTTP::version] eq "1.1" } { # Force downgrade to HTTP 1.0, but still allow keep-alive connections. # Since HTTP 1.1 is keep-alive by default, and 1.0 isn't, # we need make sure the headers reflect the keep-alive status. # Check if this is a keep alive connection if { [HTTP::header is_keepalive] } { # Replace the connection header value with "Keep-Alive" HTTP::header replace "Connection" "Keep-Alive" } # Set server side request version to 1.0 # This forces the server to respond without chunking HTTP::version "1.0" } } when HTTP_RESPONSE { # Only check responses that are a text content type (text/html, text/xml, text/plain, etc). if { [HTTP::header "Content-Type"] starts_with "text/" } { # Get the content length so we can collect the data (to be processed in the HTTP_RESPONSE_DATA event) # Limit collection to 1Mb (1048576 minus a little to spare) - See SOL6578 for details if { [HTTP::header exists "Content-Length"] } { if { [HTTP::header "Content-Length"] > 1048000 }{ # Content-Length over 1Mb so collect 1Mb set content_length 1048000 } else { # Content-Length under 1Mb so collect actual length set content_length [HTTP::header "Content-Length"] } } else { # Response did not have Content-Length header, so use default of 1Mb set content_length 1048000 } # Don't collect content if Content-Length header value was 0 if { $content_length > 0 } { HTTP::collect $content_length } } } when HTTP_RESPONSE_DATA { # Find ALL the possible credit card numbers in one pass set card_indices [regexp -all -inline -indices\ {(?:3[4|7]\d{2})(?:[ ,-]?(?:\d{5}(?:\d{1})?)){2}|(?:4\d{3})(?:[ ,-]?(?:\d{4})){3}|(?:5[1-5]\d{2})(?:[ ,-]?(?:\d{4})){3}|(?:6011)(?:[ ,-]?(?:\d{4})){3}}\ [HTTP::payload]] foreach card_idx $card_indices { set card_start [lindex $card_idx 0] set card_end [lindex $card_idx 1] set card_len [expr {$card_end - $card_start + 1}] set card_number [string range [HTTP::payload] $card_start $card_end] # Remove dash or space if they exist and count the occurrences in variable cutouts. set cutouts [regsub -all {[- ]} $card_number "" card_number] # Adjsut card_len variable but keep it for later use. set new_card_len [expr {$card_len - $cutouts}] set double [expr {$new_card_len & 1}] set chksum 0 set isCard invalid # Calculate MOD10 for { set i 0 } { $i < $new_card_len } { incr i } { set c [string index $card_number $i] if {($i & 1) == $double} { if {[incr c $c] >= 10} {incr c -9} } incr chksum $c } # Determine Card Type switch [string index $card_number 0] { 3 { set type AmericanExpress } 4 { set type Visa } 5 { set type MasterCard } 6 { set type Discover } default { set type Unknown } } # If valid card number, then mask out numbers with X's if { ($chksum % 10) == 0 } { set isCard valid HTTP::payload replace $card_start $card_len [string repeat "X" $card_len] } # Log Results log local0. "Found $isCard $type CC# $card_number" } }F5 Velos/rSeries/F5OS code for automating config backup with the new RESTCONF API and Ansible
On the new F5OS devices a new RESTCONF based API interface is used that allows everything to be done via that API. Now you can even send API command to make F5 to export the configuration file in outbound connection with HTTPS/SCP and this is an extra security for me. F5 has even released Ansible collections for Velos but some things are still not possible with the collection but with Ansible the URI module can used to do the things I am doing with Postman as even the HTTP headers can be added in the URI module. Some may use python but personally I like Ansible more (look at the end of this article for the Ansible Example) 🙂 https://clouddocs.f5.com/products/orchestration/ansible/devel/velos/velos.html https://clouddocs.f5.com/products/orchestration/ansible/devel/f5os/f5os.html https://docs.ansible.com/ansible/latest/collections/ansible/builtin/uri_module.html This code allows the automation of the configuration backups for the F5 Velos/rSeries using the new API. To get started with the F5OS API I recommend going through the Devcentral article https://community.f5.com/t5/technical-articles/exploring-f5os-automation-features-on-velos/ta-p/295318 The Velos Postman collections are at https://clouddocs.f5.com/api/velos-api/velos-api-workflows.html The Velos API documentation can be found at F5OS/F5OS-C OpenAPI Documentation . The F5OS API supports Basic and Bearer token authentication but it is much better to use the BASIC auth just to retrive the Token as shown in the examples below. Generate Bearer Token in Postman. This is from the F5 Postman collection. Endpoint: https://{{Chassis1_System_Controller_IP}}:8888/restconf/data/openconfig-system:system/aaa 'No body' 2. Create a config backup (now it is not called UCS but database configuration backup in F5OS). Endpoint: 8888/restconf/data/openconfig-system:system/f5-database:database/f5-database:config-backup Body: { "f5-database:name": "api-backup", "f5-database:overwrite": "true" } Note! For rSeries "f5-database:overwrite": "true" may need to be removed as 1.3.1 does not support to select to overwrite an existing backup or not. 3a. Download the config backup with ‘root’ with SCP from ‘/var/confd/configs/’, for example Back up and restore the F5OS-C configuration on a VELOS system 3b. Make F5 to send the backup with HTTPS to the backup server with the new file transfer utility that can be triggered with API commands for the F5 to start the file transfer. Endpoint: :8888/restconf/data/f5-utils-file-transfer:file/export Body: { "f5-utils-file-transfer:username": "test", "f5-utils-file-transfer:password": "test", "f5-utils-file-transfer:local-file": "configs/api-backup", "f5-utils-file-transfer:remote-url": "https://1.1.1.1/file" } In some versions the variable "insecure" : "true" can't be set, so maybe the web server will need a valid and not self-signed SSL cert. 3c. Export the backup with SCP/AFTP initiated from the F5 device with an API command. This is something that will be possible in the future as it seems as of now it is still not possible as I tried to follow the API documentation but sometimes, I get errors about missing element ‘’known-hosts’’ but this file should be created with the below API call as maybe the workaround is to go to the Linux with a root account and create this file but I still have not found where to create it. Another error is unknown element ‘remote-host’ but this should exist, so it is a bug or the documentation has some mistakes but as this is a new feature it will work eventually. As a note you need to add the fingerprints for the Velos or rSeries to start the SCP connection as an extra security step and this is really nice 😀 Endpoint: /restconf/data/f5-utils-file-transfer:file/known-hosts Body: { "f5-utils-file-transfer:known-host": [ { "remote-host": "string", "config": { "remote-host": "string", "key-type": "rsa", "fingerprint": "string" }, "state": { "remote-host": "string", "key-type": "rsa", "fingerprint": "string" } } ] } Now with F5OS when accessing the GUI, you can use Fiddler or F12 (the devtools) just to see the RESTCONF commands that are used and the use them in Postman/Ansible/Python etc. EDIT: 4. Using Ansible URI Module with F5OS for Basic Auth, Token generation and Config Backup Here is an example to do the same tasks but with using the Ansible URI module. The Ansible URI module allows us to make our own API requests when there is no build-in module and it even supports basic and form based authentication and after that the token can be saved and used a varible in the next requests that generate the backup and then the backup can be transfered with SCP triggered with cron job or another URI module task can be written that uses the file transfer utility. Ansible Playbook using jinja2 template as json body: root@niki1:/home/niki/ansible# cat f5os_backup.yml --- - name: F5OS_BACKUP hosts: lb connection: local gather_facts: false vars: Chassis_IP : X.X.X.X backup_name : api3_backup tasks: - name: Create a Basic request ansible.builtin.uri: url: https://{{ Chassis_IP }}:8888/restconf/data/openconfig-system:system/aaa user: xxx password: xxx method: GET force_basic_auth: yes status_code: 200 body_format: json validate_certs: false headers: Content-Type: application/yang-data+json X-Auth-Token: rctoken return_content: yes register: result - name: Save the token to a fact variable set_fact: metatoken: "{{ result.x_auth_token }}" - name: Create Backup ansible.builtin.uri: url: https://{{ Chassis_IP }}:8888/restconf/data/openconfig-system:system/f5-database:database/f5-database:config-backup method: POST status_code: 200 body_format: json validate_certs: false body: "{{ lookup('ansible.builtin.template','f5os.json') }}" headers: Content-Type: application/yang-data+json X-Auth-Token: "{{ metatoken }}" f5os.json Template: { "f5-database:name": "{{ backup_name }}", "f5-database:overwrite": "true" } Edit: Now there is an F5 Ansible collection for this 🙂 https://clouddocs.f5.com/products/orchestration/ansible/devel/f5os/modules_3_0/f5os_config_backup_module.html As of F5OS 1.8 now ":8888/restconf" can be replaced with ":443/api".Knowledge sharing: Velos and rSeries (F5OS) basic troubleshooting, logs and commands
This another part of my Knowledge sharing articles, where I will take a deeper look into Velos and rSeries investigation of issues, logs and command. 1. Velos HA controller and blade issues. As the Velos system is the one with two controllers in active/standby mode only with Velos it could be needed to check if there is an issue with the controller's HA. As the controller's HA order can be different for the system and the different partitions to check the HA for the system use the /var/log_controller/cc-confd file or for a partition HA issue look at the partition velos log at /var/F5/partition<ID>/log/velos.log . Also you can enable HA debug for the controllers with " system dbvars config debug confd ha-state-machine true ". Overview of HA: https://support.f5.com/csp/article/K19204400 Controller HA: https://support.f5.com/csp/article/K21130014 Partition HA: https://support.f5.com/csp/article/K58515297 List of Velos/rSeries services: Overview of F5 VELOS chassis controller services Overview of F5 VELOS partition services Overview of F5 rSeries system services 2. Entering into F5OS objects. The rSeries and Velos tenants are like vCMP quests with VIPRION and sometimes if there are access issues with them it could be needed to open their console. For this the "virtctl" command can be used and as an example " /usr/share/omd/kubevirt/virtctl console <tenant_name>-<tenant_instance_ID> ". Also as velos uses blades and partitions it could be needed to ssh to a blade with " ssh slot<number> " or to enter a partition with " docker exec -it partition<ID>_cli su admin " as sometimes for example to see the GUI logs entering the GUI container for the partition could be needed but F5 support will for this in most cases and maybe this will be the way to enter the BIG-IP NEXT CLI. Overview of VELOS system architecture: https://support.f5.com/csp/article/K73364432 Overview of rSeries system architecture: https://support.f5.com/csp/article/K49918625 rSeries tanant access: https://support.f5.com/csp/article/K33373310 Velos blade and tenant access: https://support.f5.com/csp/article/K65442484 Velos partition access: https://support.f5.com/csp/article/K11206563 3. Usefull commands and logs. For Velos/rSeries as this is a system with a cluster the "show cluster" command is usefull to see any issues (look fo "cluster is NOT ready."). Also the velos.log for the controller and partitions is a great place to start and debug level can be enabled for it under " SYSTEM SETTINGS Log Settings " as this is also the place for rSeries logging to be set to debug. Also the /var/log/openshift.log is good be checked with velos if there are cluster issues or or ks3.log in rSeries. Also the confd logs are like mcpd logs, so they are really usefull for Velos or rSeries. Other nice commands are docker ps, oc get pod --all-namespaces -o wide, kubectl get pod --all-namespaces -o wide but the support will ask for them in most cases. Velos cluster status: https://support.f5.com/csp/article/K27427444 Velos debug: https://support.f5.com/csp/article/K51486849 Velos openshift example issue: https://support.f5.com/csp/article/K01030619 Monitoring Velos: https://clouddocs.f5.com/training/community/velos-training/html/monitoring_velos.html Monitoring rSeries: https://clouddocs.f5.com/training/community/rseries-training/html/monitoring_rseries.html 4. Velos and rSeries tcpdumps packet captures, file utility and qkview files. For Velos qkviews ca be created for controller or partition as they are seperate qkviews. Tcpdumps for client traffic are done a tcpdump utility from the F5OS (su - admin) and a tcpdump in the Linux kernel is just for the managment ip addresses of the appliance , controller (floating or local) , partition or tenant. The file utility allows for file transfers to remote servers or even downloading any log from the Velos/rSeries to your computer as this was not possible before with iSeries or Viprion. Also the file utility starts outbound session to the remote servers so this an extra security as no inbound sessions need to be allowed on the firewall/web proxy and it can be even triggered by API call and I may make a codeshare article for this. Velos tcpdump utility: https://support.f5.com/csp/article/K12313135 rSeries tcpdump utility: https://support.f5.com/csp/article/K80685750 Qkview Velos: https://support.f5.com/csp/article/K02521182 Qkview Velos CLI location: https://support.f5.com/csp/article/K79603072 Qkview rSeries: https://support.f5.com/csp/article/K04756153 SCP: https://support.f5.com/csp/article/K34776373 For rSeries 2000/4000 tcpdump is different as SR-IOV not FPGA (rSeries Networking (f5.com)) is used to attach interfaces directly to the tenant VM: Article Detail (f5.com) 5. A final fast check could be to use ''kubectl get pods -o wide--all-namespaces'' (with Velos also ''oc get pods -o wide --all-namespaces'' should also work) to see that all pods are ok and running. Also ''docker ps'' or '' docker ps --format 'table {{.Names}}\t{{.RunningFor}}\t{{.Status}}' '' are usefull to see a container that could be going down and up and this can be correlated with issues seen with "show cluster" command. 6. The new F5OS has much better hardware diagnostics than the old devices, so no more the need to do EUD tests as all system hardware components and their health can be viewed from the GUI or CLI and also this is shown in F5 ihealth! https://techdocs.f5.com/en-us/velos-1-5-0/velos-systems-administration-configuration/title-system-settings.html 7. For Velos and rSeries always keep the software up to date as for example I will give with the Velos 1.5.1 the cluster rebuild because of the openshift ssl cert being 1 year is much simpler or the F5 rSeries and the Cisco Nexus issues or the corrupt Qkview generation when the GUI not the CLI is used (the velos cluster rebuild with touch /var/omd/CLUSTER_REINSTALL can solve many issues but it will cause some timeout): http://cdn.f5.com/product/bugtracker/ID1135853.html https://my.f5.com/manage/s/article/K000092905 https://support.f5.com/csp/article/K79603072 In the future ''docker'' commands could be not available but then just use "crictl" as this replaces the docker init system for kubernetes. 8. F5OS 1.8 has added several cool features that I will discuss. system rollback initiate proceed - F5OS config option to rollback to the previous config. system diagnostics os-utils docker restart node platform service xxx - config option to restart a docker service from the F5OS without the use or root bash. Also can be scheduled through API! f5sh - to enter F5OS from root bash like tmsh command. system diagnostics core-files list - to see the core files and which process made them as to know where to focus. system diagnostics net-utils xxx - to run ping, dig, traceroute from the F5os without bash access.
