Working with JSON data in iRules - Part 1
When TMOS version 21 dropped a few months ago, I released a three part article series focused on managing MCP in iRules. MCP is JSON-RPC2.0 based, so this was a great use case for the new JSON commands. But it's not the only use case. JSON has been the default data format for the web transport for well over a decade. And until v21, doing anything with JSON in iRules was not for the faint of heart as the Tcl version iRules uses has no native parsing capability. In this article, i'll do a JSON overview, introduce the test scripts to pass simple JSON payloads back and forth, and get the BIG-IP configured to manage this traffic. In part two, we'll dig into the iRules. JSON Structure & Terminology Let's start with some example JSON, then we'll break it down. { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [1, 2, 3], "my_object": { "nested_string": "I'm nested", "nested_array": ["a", "b", "c"] } } JSON is pretty simple. The example shown there is a JSON object. Object delimeters are the curly brackets you see on lines 1 and 11, but also in the nested object in lines 7 and 10. Every key in JSON must be a string enclosed in double quotes. The keys are the left side of the colon on lines 2-9. The colon is the separator between the key and its value The comma is the separator between key/value pairs There are 6 data types in JSON String - should be enclosed with double quotes like keys Number - can be integer, floating point, or exponential format Boolean - can only be true or false, without quotes, no capitals Null - this is an intentional omission of a value Array - this is called a list in python and Tcl Object - this is called a dictionary in python and Tcl Objects can be nested. (If you've ever pulled stats from iControl REST, you know this to be true!) Creating a JSON test harness Since iControl REST is JSON based, I could easily pass payloads from my desktop through a virtual server and onward to an internal host for the iControl REST endpoints, but I wanted something I could simplify with a pre-defined client and server payload. So I vibe coded a python script to do just that if you want to use it. I have a ubuntu desktop connected to both the client and server networks of the v21 BIG-IP in my lab. First I tested on localhost, then got my BIG-IP set up to handle the traffic as well. Local test Clientside jrahm@udesktop:~/scripts$ ./cspayload.py client --host 10.0.3.95 --port 8088 [Client] Connecting to http://10.0.3.95:8088/ [Client] Sending JSON payload (POST): { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [ 1, 2, 3 ], "my_object": { "nested_string": "I'm nested", "nested_array": [ "a", "b", "c" ] } } [Client] Received response (Status: 200): { "message": "Hello from server", "type": "response", "status": "success", "data": { "processed": true, "timestamp": "2026-01-29" } } Serverside jrahm@udesktop:~/scripts$ ./cspayload.py server --host 0.0.0.0 --port 8088 [Server] Starting HTTP server on 0.0.0.0:8088 [Server] Press Ctrl+C to stop [Server] Received JSON payload: { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [ 1, 2, 3 ], "my_object": { "nested_string": "I'm nested", "nested_array": [ "a", "b", "c" ] } } [Server] Sent JSON response: { "message": "Hello from server", "type": "response", "status": "success", "data": { "processed": true, "timestamp": "2026-01-29" } } Great, my JSON payload is properly flowing from client to server on localhost. Now let's get the BIG-IP setup to manage this traffic. BIG-IP config This is a pretty basic setup, just need a JSON profile on top of the standard HTTP virtual server setup. My server is listening on 10.0.3.95:8088, so i'll add that as a pool member and then create the virtual in my clientside network at 10.0.2.50:80. Config is below. ltm virtual virtual.jsontest { creation-time 2026-01-29:15:10:10 destination 10.0.2.50:http ip-protocol tcp last-modified-time 2026-01-29:16:21:58 mask 255.255.255.255 pool pool.jsontest profiles { http { } profile.jsontest { } tcp { } } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled vlans { ext } vlans-enabled vs-index 2 } ltm pool pool.jsontest { members { 10.0.3.95:radan-http { address 10.0.3.95 session monitor-enabled state up } } monitor http } ltm profile json profile.jsontest { app-service none maximum-bytes 3000 maximum-entries 1000 maximum-non-json-bytes 2000 } BIG-IP test, just traffic, no iRules yet Ok, let's repeat the same client/server test to make sure we're flowing properly through the BIG-IP. I'll just show the clientside this time as the serverside would be the same as before. Note the updated IP and port in the client request should match the virtual server you create. jrahm@udesktop:~/scripts$ ./cspayload.py client --host 10.0.2.50 --port 80 [Client] Connecting to http://10.0.2.50:80/ [Client] Sending JSON payload (POST): { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [ 1, 2, 3 ], "my_object": { "nested_string": "I'm nested", "nested_array": [ "a", "b", "c" ] } } [Client] Received response (Status: 200): { "message": "Hello from server", "type": "response", "status": "success", "data": { "processed": true, "timestamp": "2026-01-29" } } Ok. Now we're cooking and BIG-IP is managing the traffic. Part two will drop as soon as I can share some crazy good news about a little thing happening at AppWorld you don't want to miss!Working with JSON data in iRules - Part 2
In part one, we covered JSON at a high level, got scripts working to pass JSON payload back and forth between client and server, and got the BIG-IP configured to manage this traffic. In this article, we'll start with an overview of the new JSON events, walk through an existing Tcl procedure that will print out the payload in log statements and explain the JSON:: iRules commands in play, and then we'll create a proc or two of our own to find keys in a JSON payload and log their values. But before that, we're going to have a little iRules contest at this year's AppWorld 2026 in Vegas. Are you coming? REGISTER in the AppWorld mobile app for the contest (to be released soon)...seats are limited! when CONTEST_SUBMISSION { set name [string toupper [string replace Jason 1 1 ""]] log local0. "Hey there...$name here." log local0. "You might want to speak my language: structured, nested, and curly-braced." } Some details are being withheld until we gather at AppWorld for the contest, but there just might be a hint in that psuedo-iRule code above. Crawl, Walk, Run! Crawling Let's start by crawling. With the new JSON profile, there are several new events: JSON_REQUEST JSON_REQUEST_MISSING JSON_REQUEST_ERROR JSON_RESPONSE JSON_RESPONSE_MISSING JSON_RESPONSE_ERROR From there let's craft a basic iRule to see what triggers the events. Simple log statements in each. when HTTP_REQUEST { log local0. "HTTP request received: URI [HTTP::uri] from [IP::client_addr]" } when JSON_REQUEST { log local0. "JSON Request detected successfully." } when JSON_REQUEST_MISSING { log local0. "JSON Request missing." } when JSON_REQUEST_ERROR { log local0. "Error processing JSON request. Rejecting request." } when JSON_RESPONSE { log local0. "JSON response detected successfully." } when JSON_RESPONSE_MISSING { log local0. "JSON Response missing." } when JSON_RESPONSE_ERROR { log local0. "Error processing JSON response." } Now we need some client and server payload. Thankfully we have that covered with the script I shared in part one. We just need to unleash it! I have my Visual Studio Code IDE fired up with the F5 Extension and iRules editor marketplace extensions connected to my v21 BIG-IP, I have the iRule above loaded up in the center pane, and then I have the terminal on the right pane split three ways so I can a) generate traffic in the top terminal, b) view the server request/response in the middle terminal, and c) watch the logs from BIG-IP in the bottom terminal. Handy to have all that in one view in the IDE while working. For the first pass, I'll send a request expected to work through the BIG-IP and get a response back from my test server. That command is: ./cspayload.py client --host 10.0.2.50 --port 80 And the result can be seen in the picture below (shown here to show the VS Code setup, I'll just show text going forward.) You can see that the request triggered HTTP_REQUEST, JSON_REQUEST, and JSON_RESPONSE as expected. Now, I'll send an empty payload to verify that JSON_REQUEST_MISSING will fire. The command for that is: ./cspayload1.py client --host 10.0.2.50 --port 80 --no-json We get the event triggered as expected, but interestingly, the request is still processed and sent to the backend and the response is sent back just fine. (timestamps removed) Rule /Common/irule.jsontest <HTTP_REQUEST>: HTTP request received: URI / from 10.0.2.95 Rule /Common/irule.jsontest <JSON_REQUEST_MISSING>: JSON Request missing. Rule /Common/irule.jsontest <JSON_RESPONSE>: JSON response detected successfully. My test script serverside code doesn't balk at an empty payload, but most services likely will, so you'll likely want to manage a reject or response as appropriate in this event. Now let's trigger an error by sending some invalid JSON. The command I sent is: ./cspayload1.py client --host 10.0.2.50 --port 80 --malformed-custom '{invalid: "no quotes on key"}' And that resulted in a successfully triggered JSON_REQUEST_ERROR and no payload was sent back to the backend server. Rule /Common/irule.jsontest <HTTP_REQUEST>: HTTP request received: URI / from 10.0.2.95 Rule /Common/irule.jsontest <JSON_REQUEST_ERROR>: Error processing JSON request. Rejecting request. Walking After validating our events are triggering, let's take a look at the example iRule below that will use a procedure to print out the JSON payload. when JSON_REQUEST { set json_data [JSON::root] call print $json_data } proc print { e } { set t [JSON::type $e] set v [JSON::get $e] set p0 [string repeat " " [expr {2 * ([info level] - 1)}]] set p [string repeat " " [expr {2 * [info level]}]] switch $t { array { log local0. "$p0\[" set size [JSON::array size $v] for {set i 0} {$i < $size} {incr i} { set e2 [JSON::array get $v $i] call print $e2 } log local0. "$p0\]" } object { log local0. "$p0{" set keys [JSON::object keys $v] foreach k $keys { set e2 [JSON::object get $v $k] log local0. "$p${k}:" call print $e2 } log local0. "$p0}" } string - literal { set v2 [JSON::get $e $t] log local0. "$p\"$v2\"" } default { set v2 [JSON::get $e $t] if { $v2 eq "" && $t eq "null" } { log local0. "${p}null" } elseif { $v2 == 1 && $t eq "boolean" } { log local0. "${p}true" } elseif { $v2 == 0 && $t eq "boolean" } { log local0. "${p}false" } else { log local0. "$p$v2" } } } } If you build a lot of JSON utilities, I'd recommend creating an iRule that is just a library of procedures you can call from the iRule where your application-specific logic is. In this case, it's instructional so I'll keep the proc local to the iRule. Let's take this line by line. Lines 1-4 are the logic of the iRule. Upon the JSON_REQUEST event trigger, use the JSON::root command to load the JSON payload into the json_data variable, then pass that data to the print proc to, well, print it (via log statements.) Lines 5-47 detail the print procedure. It takes the variable e (for element) and acts on that throughout the proc. Lines 6-7 set the type and value of the element to the t and v variables respectively Lines 8-9 are calculating whitespace requirements for each element's value that will be printed Lines 10-38 are conditional logic controlled by the switch statement based on the element's type set by the JSON::type command, with lines 11-19 handling an array, lines 20-29 handling an object, lines 30-33 a string or literal, and lines 34-27 the default catchall. Lines 11 - 19 cover the JSON array, which in Tcl is a list. The JSON::array size command gets the list size and iterates through each list item in the for loop. The JSON::array get command then sets the value at that index in the loop to a second element variable (e2) and recursively calls the proc to start afresh on the e2 element. Lines 20-29 cover the JSON object, which in Tcl is a key/value dictionary. The JSON::object keys command gets the keys of the element and iterates through each key. The rest of this action is identical to the JSON array with the exception here of using the JSON::object get command. Lines 30-33 cover the string and literal types. Simple action here, uses the JSON::get command with the element and type and then logs it. For lines 34-43, this is the catch all for other types. Tcl represents a null type as an empty string, and the boolean values of true and false as 1 and 0 respectively. But since we're printing out the JSON values sent, it's nice to make sure they match, so I modified the function to print a literal null as a string for that type, and a literal true/false string for their 1/0 Tcl counterparts. Otherwise, it will print as is. Ok, let's run the test and see what we see. Clientside view: ./cspayload2.py client --host 10.0.2.50 --port 80 [Client] Connecting to http://10.0.2.50:80/ [Client] Sending JSON payload (POST): { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [ 1, 2, 3 ], "my_object": { "nested_string": "I'm nested", "nested_array": [ "a", "b", "c" ] } } [Client] Received response (Status: 200): { "message": "Hello from server", "type": "response", "status": "success", "data": { "processed": true, "timestamp": "2026-01-29" } } Serverside view: jrahm@udesktop:~/scripts$ ./cspayload2.py server --host 0.0.0.0 --port 8088 [Server] Starting HTTP server on 0.0.0.0:8088 [Server] Mode: Normal JSON responses [Server] Press Ctrl+C to stop [Server] Received JSON payload: { "my_string": "Hello World", "my_number": 42, "my_boolean": true, "my_null": null, "my_array": [ 1, 2, 3 ], "my_object": { "nested_string": "I'm nested", "nested_array": [ "a", "b", "c" ] } } [Server] Sent JSON response: { "message": "Hello from server", "type": "response", "status": "success", "data": { "processed": true, "timestamp": "2026-01-29" } } Resulting log statements on BIG-IP (with timestamp through rule name removed for visibility): <JSON_REQUEST>: { <JSON_REQUEST>: my_string: <JSON_REQUEST>: "Hello World" <JSON_REQUEST>: my_number: <JSON_REQUEST>: 42 <JSON_REQUEST>: my_boolean: <JSON_REQUEST>: true <JSON_REQUEST>: my_null: <JSON_REQUEST>: null <JSON_REQUEST>: my_array: <JSON_REQUEST>: [ <JSON_REQUEST>: 1 <JSON_REQUEST>: 2 <JSON_REQUEST>: 3 <JSON_REQUEST>: ] <JSON_REQUEST>: my_object: <JSON_REQUEST>: { <JSON_REQUEST>: nested_string: <JSON_REQUEST>: "I'm nested" <JSON_REQUEST>: nested_array: <JSON_REQUEST>: [ <JSON_REQUEST>: "a" <JSON_REQUEST>: "b" <JSON_REQUEST>: "c" <JSON_REQUEST>: ] <JSON_REQUEST>: } <JSON_REQUEST>: } The print procedure is shown here to include the whitespace necessary to prettify the output. Neat! Running Now that we've worked our way through the print function, let's do something useful! You might have a need to evaluate the value of a key somewhere in the JSON object and act on that. For this example, we're going to look for the nested_array key, retrieve it's value, and if an item value of b is found, reject the request by building a new JSON object to return status to the client. First, we need to build a proc we'll name find_key that is similar to the print one above to recursively search the JSON payload. While learning my way through this, I also discovered I needed to create an additional proc we'll name stringify to, well, "stringify" the values of objects because they are still encoded. stringify proc proc stringify { json_element } { set element_type [JSON::type $json_element] set element_value [JSON::get $json_element] set output "" switch -- $element_type { array { append output "\[" set array_size [JSON::array size $element_value] for {set index 0} {$index < $array_size} {incr index} { set array_item [JSON::array get $element_value $index] append output [call stringify $array_item] if {$index < $array_size - 1} { append output "," } } append output "\]" } object { append output "{" set object_keys [JSON::object keys $element_value] set key_count [llength $object_keys] set current_index 0 foreach current_key $object_keys { set nested_element [JSON::object get $element_value $current_key] append output "\"${current_key}\":" append output [call stringify $nested_element] if {$current_index < $key_count - 1} { append output "," } incr current_index } append output "}" } string - literal { set actual_value [JSON::get $json_element $element_type] append output "\"$actual_value\"" } default { set actual_value [JSON::get $json_element $element_type] append output "$actual_value" } } return $output } There really isn't any new magic in this proc, though I did expand variable names to make it a little more clear than our original example. It's basically a redo of the print function, but instead of printing it's just creating the string version of objects so I can execute a conditional against that string. Nothing new to learn, but necessary in making the find_key proc work. find_key proc proc find_key { json_element search_key } { set element_type [JSON::type $json_element] set element_value [JSON::get $json_element] switch -- $element_type { array { set array_size [JSON::array size $element_value] for {set index 0} {$index < $array_size} {incr index} { set array_item [JSON::array get $element_value $index] set result [call find_key $array_item $search_key] if {$result ne ""} { return $result } } } object { set object_keys [JSON::object keys $element_value] foreach current_key $object_keys { if {$current_key eq $search_key} { set found_element [JSON::object get $element_value $current_key] set found_type [JSON::type $found_element] if {$found_type eq "object" || $found_type eq "array"} { set found_value [call stringify $found_element] } else { set found_value [JSON::get $found_element $found_type] } return $found_value } set nested_element [JSON::object get $element_value $current_key] set result [call find_key $nested_element $search_key] if {$result ne ""} { return $result } } } } return "" } In the find_key proc, the magic happens in line 10 for a JSON array (Tcl list) and in lines 18-32 for a JSON object (Tcl dictionary.) Nothing new in the use of the JSON commands, but rather than printing all the keys and values found, we're looking for a specific key so we can return its value. For the array we are iterating through list items that will have a single value, but that value might be an object that needs to be stringified. For the object, we need to iterate through all the keys and their values, also which might be objects or nested objects to be stringified. Recursion for the win! Hopefull you're starting to get the hang of using all the interrogating JSON commands we've covered, because now wer'e going to create something with some new commands! iRule logic Once we have the procs defined to handle their specific jobs, the iRule to find the key and then return the rejected status message becomes much cleaner: when JSON_REQUEST priority 500 { set json_data [JSON::root] if {[call find_key $json_data "nested_array"] contains "b" } { set cache [JSON::create] set rootval [JSON::root $cache] JSON::set $rootval object set obj [JSON::get $rootval object] JSON::object add $obj "[IP::client_addr] status" string "rejected" set rendered [JSON::render $cache] log local0. "$rendered" HTTP::respond 200 content $rendered "Content-Type" "application/json" } } Let's walk through this one line by line. Lines 1 and 13 wrap the JSON_REQUEST payload. Line 2 retrieves the current JSON::root, which is our payload, and stores it in the json_data variable. Lines 3 and 12 wrap the if conditional, which is using our find_key proc to look for the nested_array key, and if that stringified value includes b, reject the response. (in real life looking for "b" would be a terrible contains pattern to look for, but go with me here.) Line 4 creates a JSON context for the system. Think of this as a container we're going to do JSON stuff in. Line 5 gets the root element of our JSON container. At this point it's empty, we're just getting a handle to whatever will be at the top level. Line 6 now actually adds an object to the JSON container. At this point, it's just "{ }". Line 7 gets the handle of that object we just created so we can do something with it. Line 8 adds the key value pair of "status" and our reject message. Line 9 now takes the entire JSON context we just created and renders it to a JSON string we can log and respond with. Line 10 logs to /var/log/ltm Line 11 responds with the reject message in JSON format. Note I'm using a 200 error code instead of a 403. That's just because the cilent test script won't show the status message with a 403 and I wanted to see it. Normally you'd use the appropriate error code. Now, I offer you a couple challenges. lines 4-9 in the JSON_REQUEST example above should really be split off to become another proc, so that the logic of the JSON_REQUEST is laser-focused. How would YOU write that proc, and how would you call it from the JSON_REQUEST event? The find_key proc works, but there's a Tcl-native way to get at that information with just the JSON::object subcommands that is far less complex and more performant. Come at me! Conclusion When I started this JSON article series, I knew A LOT less about the underlying basics of JSON than I thought I knew. It's funny how working with things on the wire requires a little more insight into protocols and data formats than you think you need. Happy iRuling out there, and I hope to see you at AppWorld next month!
5 Years Later: OpenAJAX Who?
Five years ago the OpenAjax Alliance was founded with the intention of providing interoperability between what was quickly becoming a morass of AJAX-based libraries and APIs. Where is it today, and why has it failed to achieve more prominence? I stumbled recently over a nearly five year old article I wrote in 2006 for Network Computing on the OpenAjax initiative. Remember, AJAX and Web 2.0 were just coming of age then, and mentions of Web 2.0 or AJAX were much like that of “cloud” today. You couldn’t turn around without hearing someone promoting their solution by associating with Web 2.0 or AJAX. After reading the opening paragraph I remembered clearly writing the article and being skeptical, even then, of what impact such an alliance would have on the industry. Being a developer by trade I’m well aware of how impactful “standards” and “specifications” really are in the real world, but the problem – interoperability across a growing field of JavaScript libraries – seemed at the time real and imminent, so there was a need for someone to address it before it completely got out of hand. With the OpenAjax Alliance comes the possibility for a unified language, as well as a set of APIs, on which developers could easily implement dynamic Web applications. A unified toolkit would offer consistency in a market that has myriad Ajax-based technologies in play, providing the enterprise with a broader pool of developers able to offer long term support for applications and a stable base on which to build applications. As is the case with many fledgling technologies, one toolkit will become the standard—whether through a standards body or by de facto adoption—and Dojo is one of the favored entrants in the race to become that standard. -- AJAX-based Dojo Toolkit , Network Computing, Oct 2006 The goal was simple: interoperability. The way in which the alliance went about achieving that goal, however, may have something to do with its lackluster performance lo these past five years and its descent into obscurity. 5 YEAR ACCOMPLISHMENTS of the OPENAJAX ALLIANCE The OpenAjax Alliance members have not been idle. They have published several very complete and well-defined specifications including one “industry standard”: OpenAjax Metadata. OpenAjax Hub The OpenAjax Hub is a set of standard JavaScript functionality defined by the OpenAjax Alliance that addresses key interoperability and security issues that arise when multiple Ajax libraries and/or components are used within the same web page. (OpenAjax Hub 2.0 Specification) OpenAjax Metadata OpenAjax Metadata represents a set of industry-standard metadata defined by the OpenAjax Alliance that enhances interoperability across Ajax toolkits and Ajax products (OpenAjax Metadata 1.0 Specification) OpenAjax Metadata defines Ajax industry standards for an XML format that describes the JavaScript APIs and widgets found within Ajax toolkits. (OpenAjax Alliance Recent News) It is interesting to see the calling out of XML as the format of choice on the OpenAjax Metadata (OAM) specification given the recent rise to ascendancy of JSON as the preferred format for developers for APIs. Granted, when the alliance was formed XML was all the rage and it was believed it would be the dominant format for quite some time given the popularity of similar technological models such as SOA, but still – the reliance on XML while the plurality of developers race to JSON may provide some insight on why OpenAjax has received very little notice since its inception. Ignoring the XML factor (which undoubtedly is a fairly impactful one) there is still the matter of how the alliance chose to address run-time interoperability with OpenAjax Hub (OAH) – a hub. A publish-subscribe hub, to be more precise, in which OAH mediates for various toolkits on the same page. Don summed it up nicely during a discussion on the topic: it’s page-level integration. This is a very different approach to the problem than it first appeared the alliance would take. The article on the alliance and its intended purpose five years ago clearly indicate where I thought this was going – and where it should go: an industry standard model and/or set of APIs to which other toolkit developers would design and write such that the interface (the method calls) would be unified across all toolkits while the implementation would remain whatever the toolkit designers desired. I was clearly under the influence of SOA and its decouple everything premise. Come to think of it, I still am, because interoperability assumes such a model – always has, likely always will. Even in the network, at the IP layer, we have standardized interfaces with vendor implementation being decoupled and completely different at the code base. An Ethernet header is always in a specified format, and it is that standardized interface that makes the Net go over, under, around and through the various routers and switches and components that make up the Internets with alacrity. Routing problems today are caused by human error in configuration or failure – never incompatibility in form or function. Neither specification has really taken that direction. OAM – as previously noted – standardizes on XML and is primarily used to describe APIs and components - it isn’t an API or model itself. The Alliance wiki describes the specification: “The primary target consumers of OpenAjax Metadata 1.0 are software products, particularly Web page developer tools targeting Ajax developers.” Very few software products have implemented support for OAM. IBM, a key player in the Alliance, leverages the OpenAjax Hub for secure mashup development and also implements OAM in several of its products, including Rational Application Developer (RAD) and IBM Mashup Center. Eclipse also includes support for OAM, as does Adobe Dreamweaver CS4. The IDE working group has developed an open source set of tools based on OAM, but what appears to be missing is adoption of OAM by producers of favored toolkits such as jQuery, Prototype and MooTools. Doing so would certainly make development of AJAX-based applications within development environments much simpler and more consistent, but it does not appear to gaining widespread support or mindshare despite IBM’s efforts. The focus of the OpenAjax interoperability efforts appears to be on a hub / integration method of interoperability, one that is certainly not in line with reality. While certainly developers may at times combine JavaScript libraries to build the rich, interactive interfaces demanded by consumers of a Web 2.0 application, this is the exception and not the rule and the pub/sub basis of OpenAjax which implements a secondary event-driven framework seems overkill. Conflicts between libraries, performance issues with load-times dragged down by the inclusion of multiple files and simplicity tend to drive developers to a single library when possible (which is most of the time). It appears, simply, that the OpenAJAX Alliance – driven perhaps by active members for whom solutions providing integration and hub-based interoperability is typical (IBM, BEA (now Oracle), Microsoft and other enterprise heavyweights – has chosen a target in another field; one on which developers today are just not playing. It appears OpenAjax tried to bring an enterprise application integration (EAI) solution to a problem that didn’t – and likely won’t ever – exist. So it’s no surprise to discover that references to and activity from OpenAjax are nearly zero since 2009. Given the statistics showing the rise of JQuery – both as a percentage of site usage and developer usage – to the top of the JavaScript library heap, it appears that at least the prediction that “one toolkit will become the standard—whether through a standards body or by de facto adoption” was accurate. Of course, since that’s always the way it works in technology, it was kind of a sure bet, wasn’t it? WHY INFRASTRUCTURE SERVICE PROVIDERS and VENDORS CARE ABOUT DEVELOPER STANDARDS You might notice in the list of members of the OpenAJAX alliance several infrastructure vendors. Folks who produce application delivery controllers, switches and routers and security-focused solutions. This is not uncommon nor should it seem odd to the casual observer. All data flows, ultimately, through the network and thus, every component that might need to act in some way upon that data needs to be aware of and knowledgeable regarding the methods used by developers to perform such data exchanges. In the age of hyper-scalability and über security, it behooves infrastructure vendors – and increasingly cloud computing providers that offer infrastructure services – to be very aware of the methods and toolkits being used by developers to build applications. Applying security policies to JSON-encoded data, for example, requires very different techniques and skills than would be the case for XML-formatted data. AJAX-based applications, a.k.a. Web 2.0, requires different scalability patterns to achieve maximum performance and utilization of resources than is the case for traditional form-based, HTML applications. The type of content as well as the usage patterns for applications can dramatically impact the application delivery policies necessary to achieve operational and business objectives for that application. As developers standardize through selection and implementation of toolkits, vendors and providers can then begin to focus solutions specifically for those choices. Templates and policies geared toward optimizing and accelerating JQuery, for example, is possible and probable. Being able to provide pre-developed and tested security profiles specifically for JQuery, for example, reduces the time to deploy such applications in a production environment by eliminating the test and tweak cycle that occurs when applications are tossed over the wall to operations by developers. For example, the jQuery.ajax() documentation states: By default, Ajax requests are sent using the GET HTTP method. If the POST method is required, the method can be specified by setting a value for the type option. This option affects how the contents of the data option are sent to the server. POST data will always be transmitted to the server using UTF-8 charset, per the W3C XMLHTTPRequest standard. The data option can contain either a query string of the form key1=value1&key2=value2 , or a map of the form {key1: 'value1', key2: 'value2'} . If the latter form is used, the data is converted into a query string using jQuery.param() before it is sent. This processing can be circumvented by setting processData to false . The processing might be undesirable if you wish to send an XML object to the server; in this case, change the contentType option from application/x-www-form-urlencoded to a more appropriate MIME type. Web application firewalls that may be configured to detect exploitation of such data – attempts at SQL injection, for example – must be able to parse this data in order to make a determination regarding the legitimacy of the input. Similarly, application delivery controllers and load balancing services configured to perform application layer switching based on data values or submission URI will also need to be able to parse and act upon that data. That requires an understanding of how jQuery formats its data and what to expect, such that it can be parsed, interpreted and processed. By understanding jQuery – and other developer toolkits and standards used to exchange data – infrastructure service providers and vendors can more readily provide security and delivery policies tailored to those formats natively, which greatly reduces the impact of intermediate processing on performance while ensuring the secure, healthy delivery of applications.
F5 SSLO Unified Configuration API Quick Introduction
Introduction Prior to the introduction of BIG-IQ 8.0, you had to use the BIG-IQ graphical user interface (GUI) to configure F5 SSL Orchestrator (SSLO) Topologies and their dependencies. Starting with BIG-IQ 8.0, a new REST unified, supported and documented REST API endpoint was created to simplify SSLO configuration workflows. The aim is to simplify the configuration of F5 SSLO using standardized API calls. You are now able to store the configuration in your versioning tool (Git, SVN, etc.), and easily integrate the configuration of F5 SSLO in your automation and pipeline tools. For more information about F5 SSLO, please refer to this introductory video. An overview of F5 SSL Orchestrator is provided in K1174564. As a reminder the BIG-IQ API reference documentation can be found here. Documentation for the Access Simplified Workflow can be found here. The figure below shows a possible use for the SSLO Unified API. A few shortcuts are taken in the figure above as it is meant to illustrate the advantage of the simplified workflow. Example Configuration For the configuration the administrator needs to: -Create a JSON blurb or payload that will be sent to the BIG-IQ API -Authenticate to the BIG-IQ API -Send the payload to the BIG-IQ -Ensure that the workflow completes successfully The following aims to provide a step-by-step configuration of SSLO leveraging the API. In practice, the steps may be automated and may be included in the pipeline used to deploy the application leveraging the enterprise tooling and processes in place. 1.- Authenticate to the API API interactions with the BIG-IQ API requires the use of a token. The initial REST call should look like the following: REST Endpoint : /mgmt/shared/authn/login HTTP Method: POST Headers: -content-type: application/json Content: { "username": "", "password": "", "loginProviderName": "" } Example: POST https://10.0.0.1/mgmt/shared/authn/login HTTP/1.1 Headers: content-type: application/json Content: { "username": "username", "password": "complicatedPassword!", "loginProviderName": "RadiusServer" } The call above will authenticate the user “bob” to the API. The result of a successful authentication is the response from the BIG-IQ API with a token. 2.- Push the configuration to BIG-IQ The headers and HTTP request should look like the following: URI: mgmt/cm/sslo/api/topology HTTP Method: POST Headers: -content-type: application/json -X-F5-Auth-Token: [token obtained from the authentication process above] To send the configuration to the BIG-IQ you will need to send the following payload - the blurb is cut up in smaller pieces for readability. The JSON blurb is divided in multiple parts - the full concatenated text is available in the file in attachment. Start by defining an new topology with the following characteristics: Name: "sslo_NewTopology" Listening on the "/Common/VLAN_TRAP" VLAN The topology is of type "topology_l3_outbound" The SSL settings defined below named: "ssloT_NewSsl_Dec" The policy is called: "ssloP_NewPolicy_Dec" The JSON payload starts with the following: { "template": { "TOPOLOGY": { "name": "sslo_NewTopology ", "ingressNetwork": { "vlans": [ { "name": "/Common/VLAN_TAP" } ] }, "type": "topology_l3_outbound", "sslSetting": "ssloT_NewSsl_Dec", "securityPolicy": "ssloP_NewPolicy_Dec" }, The SSL settings used above are defined in the following JSON that creates a new profile with default values: "SSL_SETTINGS": { "name": "ssloT_NewSsl_Dec" }, The security policy is configured as follows: name: ssloP_NewPolicy_Dec function: introduces a pinning policy doing a policy lookup - matching requests are bypassed (no ssl decryp) with the associated service chain "ssloSC_NewServiceChain_Dec" that is defined further down below. "SECURITY_POLICY": { "name": "ssloP_NewPolicy_Dec", "rules": [ { "mode": "edit", "name": "Pinners_Rule", "action": "allow", "operation": "AND", "conditions": [ { "type": "SNI Category Lookup", "options": { "category": [ "Pinners" ] } }, { "type": "SSL Check", "options": { "ssl": true } } ], "actionOptions": { "ssl": "bypass", "serviceChain": "ssloSC_NewServiceChain_Dec" } }, { "mode": "edit", "name": "All Traffic", "action": "allow", "isDefault": true, "operation": "AND", "actionOptions": { "ssl": "intercept" } } ] }, The service chain configuration is defined below to forward the traffic to the "ssloS_ICAP_Dec" service. this is done with the following JSON: "SERVICE_CHAIN": { "ssloSC_NewServiceChain_Declarative": { "name": "ssloSC_NewServiceChain_Dec", "orderedServiceList": [ { "name":"ssloS_ICAP_Dec" } ] } }, The "ssloS_ICAP_Dec" service is defined with the JSON below with IP 3.3.3.3 on port 1344 "SERVICE": { "ssloS_ICAP_Declarative": { "name": "ssloS_ICAP_Dec", "customService": { "name": "ssloS_ICAP_Dec", "serviceType": "icap", "loadBalancing": { "devices": [ { "ip": "3.3.3.3", "port": "1344" } ] } } } } }, The configuration will be deployed to the target defined below: "targetList": [ { "type": "DEVICE", "name": "my.bigip.internal" } ] } After the HTTP POST, the BIG-IQ will respond with a transaction id. A sample of what looks like is given below: { […] "id":"edc17b06-8d97-47e1-9a78-3d47d2db70a6", "status":"STARTED", […] } You can check on the status of the deployment task by submitting a request as follows: -HTTP GET Method -Authenticated with the use of the custom authentication header X-F5-Auth-Token -Sent to the BIG-IQ to URI GET mgmt/cm/sslo/tasks/api/{{status_id}} HTTP/1.1 -With Content-Type header set to: Application/JSON Once the status of the task changes to FINISHED. The configuration is successfully completed. You can now check the F5 SSLO interface to make sure the new topology has been created. The BIG-IQ interface will show the new topology as depicted in the example below: The new topology has been deployed to the BIG-IP automatically. You can connect to the BIG-IP to verify, the interface should like the one depicted below: Congratulations, you now have successfully deployed a fully functional topology that your users can start using. Note that, you can also use the BIG-IQ REST API to delete the items that were just created. This is done by sending HTTP DELETE to the different API endpoints for the topology, service, security profile etc. For example, for the example above, you would be sending HTTP DELETE requests to the following URI’s: - For the topology: /mgmt/cm/sslo/api/topology/sslo_NewTopology_Dec - For the service chain: /mgmt/cm/sslo/api/service-chain/ssloSC_NewServiceChain_Dec - For the ICAP service: /mgmt/cm/sslo/api/ssl/ssloT_NewSsl_Dec All the requests listed above need to be sent to the BIG-IQ system to its management IP address with the following 2 headers: - content-type: application/json - X-F5-Auth-Token: [value of the authentication token obtained during authentication] Conclusion BIG-IQ makes it easier to manage SSLO Topologies thanks to its REST API. You can now make supported, standardized API calls to the BIG-IQ to create and modify topologies and deploy the changes directly to BIG-IP.
Parsing complex BIG-IP json structures made easy with Ansible filters like json_query
JMESPath and json_query JMESPath (JSON Matching Expression paths) is a query language for searching JSON documents. It allows you to declaratively extract elements from a JSON document. Have a look at this tutorial to learn more. The json_query filter lets you query a complex JSON structure and iterate over it using a loop structure.This filter is built upon jmespath, and you can use the same syntax as jmespath. Click here to learn more about the json_query filter and how it is used in Ansible. In this article we are going to use the bigip_device_info module to get various facts from the BIG-IP and then use the json_query filter to parse the output to extract relevant information. Ansible bigip_device_info module Playbook to query the BIG-IP and gather system based information. - name: "Get BIG-IP Facts" hosts: bigip gather_facts: false connection: local tasks: - name: Query BIG-IP facts bigip_device_info: provider: validate_certs: False server: "xxx.xxx.xxx.xxx" user: "*****" password: "*****" gather_subset: - system-info register: bigip_facts - set_fact: facts: '{{bigip_facts.system_info}}' - name: debug debug: msg="{{facts}}" To view the output on a different subset, below are a few examples to change the gather_subset and set_fact values in the above playbook from gather_subset: system-info, facts: bigip_facts.system_info to any of the below: gather_subset: vlans , facts: bigip_facts.vlans gather_subset: self-ips, facts: bigip_facts.self_ips gather_subset: nodes. facts: bigip_facts.nodes gather_subset: software-volumes, facts: bigip_facts.software_volumes gather_subset: virtual-servers, facts: bigip_facts.virtual_servers gather_subset: system-info, facts: bigip_facts.system_info gather_subset: ltm-pools, facts: bigip_facts.ltm_pools Click here to view all the information that can be obtained from the BIG-IP using this module. Parse the JSON output Once we have the output lets take a look at how to parse the output. As mentioned above the jmespath syntax can be used by the json_query filter. Step 1: We will get the jmespath syntax for the information we want to extract Step 2: We will see how the jmespath syntax and then be used with json_query in an Ansible playbook The website used in this article to try out the below syntax: https://jmespath.org/ Some BIG-IP sample outputs are attached to this article as well (Check the attachments section after the References). The attachment file is a combined output of a few configuration subsets.Copy paste the relevant information from the attachment to test the below examples if you do not have a BIG-IP. System information The output for this section is obtained with above playbook using parameters: gather_subset: system-info, facts: bigip_facts.system_info Once the above playbook is run against your BIG-IP or if you are using the sample configuration attached, copy the output and paste it in the relevant text box. Try different queries by placing them in the text box next to the magnifying glass as shown in image below # Get MAC address, serial number, version information msg.[base_mac_address,chassis_serial,platform,product_version] # Get MAC address, serial number, version information and hardware information msg.[base_mac_address,chassis_serial,platform,product_version,hardware_information[*].[name,type]] Software volumes The output for this section is obtained with above playbook using parameters: gather_subset: software-volumes facts: bigip_facts.software_volumes # Get the name and version of the software volumes installed and its status msg[*].[name,active,version] # Get the name and version only for the software volume that is active msg[?active=='yes'].[name,version] VLANs and Self-Ips The output for this section is obtained with above playbook using parameters gather_subset: vlans and self-ips facts: bigip_facts Look at the following example to define more than one subset in the playbook # Get all the self-ips addresses and vlans assigned to the self-ip # Also get all the vlans and the interfaces assigned to the vlan [msg.self_ips[*].[address,vlan], msg.vlans[*].[full_path,interfaces[*]]] Nodes The output for this section is obtained with above playbook using parameters gather_subset: nodes, facts: bigip_facts.nodes # Get the address and availability status of all the nodes msg[*].[address,availability_status] # Get availability status and reason for a particular node msg[?address=='192.0.1.101'].[full_path,availability_status,status_reason] Pools The output for this section is obtained with above playbook using parameters gather_subset: ltm-pools facts: bigip_facts.ltm_pools # Get the name of all pools msg[*].name # Get the name of all pools and their associated members msg[*].[name,members[*]] # Get the name of all pools and only address of their associated members msg[*].[name,members[*].address] # Get the name of all pools along with address and status of their associated members msg[*].[name,members[*].address,availability_status] # Get status of pool members of a particular pool msg[?name=='/Common/pool'].[members[*].address,availability_status] # Get status of pool # Get address, partition, state of pool members msg[*].[name,members[*][address,partition,state],availability_status] # Get status of a particular pool and particular member (multiple entries on a member) msg[?full_name=='/Common/pool'].[members[?address=='192.0.1.101'].[address,partition],availability_status] Virtual Servers The output for this section is obtained with above playbook using parameters gather_subset: virtual-servers facts: bigip_facts.virtual_servers # Get destination IP address of all virtual servers msg[*].destination # Get destination IP and default pool of all virtual servers msg[*].[destination,default_pool] # Get me all destination IP of all virtual servers that a particular pool as their default pool msg[?default_pool=='/Common/pool'].destination # Get me all profiles assigned to all virtual servers msg[*].[destination,profiles[*].name] Loop and display using Ansible We have seen how to use the jmespath syntax and extract information, now lets see how to use it within an Ansible playbook - name: Parse the output hosts: localhost connection: local gather_facts: false tasks: - name: Setup provider set_fact: provider: server: "xxx.xxx.xxx.xxx" user: "*****" password: "*****" server_port: "443" validate_certs: "no" - name: Query BIG-IP facts bigip_device_info: provider: "{{provider}}" gather_subset: - system_info register: bigip_facts - debug: msg="{{bigip_facts.system_info}}" # Use json query filter. The query_string will be the jmespath syntax # From the jmespath query remove the 'msg' expression and use it as it is - name: "Show relevant information" set_fact: result: "{{bigip_facts.system_info | json_query(query_string)}}" vars: query_string: "[base_mac_address,chassis_serial,platform,product_version,hardware_information[*].[name,type]]" - debug: "msg={{result}}" Another example of what would change if you use a different query (only highlighting the changes that need to made below from the entire playbook) - name: Query BIG-IP facts bigip_device_info: provider: "{{provider}}" gather_subset: - ltm-pools register: bigip_facts - debug: msg="{{bigip_facts.ltm_pools}}" - name: "Show relevant information" set_fact: result: "{{bigip_facts.ltm_pools | json_query(query_string)}}" vars: query_string: "[*].[name,members[*][address,partition,state],availability_status]" The key is to get the jmespath syntax for the information you are looking for and then its a simple step to incorporate it within your Ansible playbook References Try the queries - https://jmespath.org/ Learn more jmespath syntax and example - https://jmespath.org/tutorial.html Ansible lab that can be used as a sandbox - https://clouddocs.f5.com/training/automation-sandbox/Working with iControl REST Data on the Command Line
I'm still pretty entrenched in "old school" iControl with the soap interface, but with Colin's new series on the iControl REST interface underway, I thought I'd start taking more than a glance. While Colin's sed-fu in his latest article is impressive, I want to see the json data returned from iControl REST calls parsed properly (no offense, Colin!) This can be done with any number of bash, perl, python, and more tools, but I ran across one on twitter last night that is somewhat magical. jq is a single binary you can download and it just works. So why bother? Well, as Colin pointed out, the returned json data (by default) is none too fabulous in format: But with jq, you get nice pretty text with field callouts (jq .): But wait! There's more! You can also narrow the selection to the fields you want (jq '{ name, apiAnonymous }'): If you want to pull all the rules together, but still only return the name and code, just index by item (jq '.items[] | { name, partition, apiAnonymous }'):The Next Evolution of Application Architecture is Upon Us
And what it means to #devops One of the realities of application development is that there are a lot of factors that go into its underlying architecture. I came of age during the Epoch of Client-Server Architecture (also known as the 1990s and later the .com era) and we were taught, very firmly, that despite the implication of "client-server" that there were but two, distinct "tiers" comprising an application, there were actually three. The presentation layer, the GUI, resided on the client. All business logic resided on the server, and a third, data access tier completed the trifecta. This worked well because developers tailored each application to a specific business purpose. Thus, implementation of business logic occurred along with application logic. There was no real reason to separate it out. As applications grew in complexity and use, SOA came of age. SOA introduced the principles of reuse we still adhere to today, and the idea that business logic should be consistent and shared across all applications that might need it. I know, revolutionary, wasn't it? Before SOA could complete its goal of world domination, however, the Internet and deployment models changed. Dramatically. [Interestingly enough, a post two years ago on this topic was fairly accurate on this migration ] THE NEW APP WORLD ORDER Today, an "application" is a no longer defined necessarily by business function, but by a unique combination of client and business function. It's not just the business logic, it's the delivery and presentation mechanism that make it unique and of course, more challenging for operations. Business logic has moved into a converged business logic-data tier more commonly known as The API. The client is still, of course, responsible for the presentation. But application logic - the domain of state and access - is in flux for some. As illustrated by the ill-advised decision to place application-logic in the presentation layer, some developers haven't quite adopted the practice of deploying an application (or domain) logic tier to intermediate and maintain consistent application behavior. But there always exist unique processing that must occur based on context. In some cases, some data might be marked cacheable as a means to achieve better performance for mobile clients when communicating over a mobile network while in others, it might not be because the client is a web-based application running on a PC over a LAN. A native mobile application deals with the state of the user "are they logged in" differently than a web application relying on cookies. The business logic should not be impacted by this. Ultimately the business logic for retrieving order X for customer Y does not inherently change based on client characteristics. It cannot, in fact, be shared - reused - if it contains application specific details regarding the validation of state, unless such an implementation uses a lot of conditional statements (that must be modified every time a new method is introduced, by the way) to determine whether a user is logged in or not. Thus, we move the application-specific, the domain, logic to its own tier, usually implemented by or on a proxy that intermediates between the API and the client. Domain Logic Tier Business Logic Tier • Maintains an internally consistent model representation on both sides of the app (client and server) • Is ontological • Often involves application state and access requirements such as "the user must be logged-in and an admin" or "this object is cacheable" • Consists of elements and functions that are specific to this application • Concerned with coordinating valid interactions between presentation and data (client and data) • Is teleological • Often involves direct access to data or application of business requirements such as “if order is > 1000 apply discount X” • Consists of elements that are common to all applications, i.e. does not rely on a given UI or interface What's most fascinating about this change is that a "proxy" tier traditionally proxies for the server-side application, but in this new model it is proxying for the client. That's not odd for developers, because if you break down the traditional model the "server" piece of the three-tiered architecture really was just a big application-specific proxy to the data. But it is odd for operations, because the new model takes advantage of a converged application-network proxy that is capable of performing tasks like load balancing and authentication and caching as well as mediating for an API, which may include transformative and translative functions. IMPACT ON DEVOPS What this ultimately means for devops is an increasing role in application architecture, from inception to production. It means devops will need to go beyond concerns of web performance or application deployment lifecycle management and into the realm of domain logic implementation and deployment. That may mean a new breed of developer; one that is still focused on development but does so in a primarily operational environment, in the network. It means enterprise architects will need to extend their view into operations, into the network, and codify for developers the lines of demarcation between domain and business logic. It means a very interesting new application model that basically adopts the premise of application delivery but adds domain logic to its catalog of services. It means devops is going to get even more interesting as more applications adopt this new, three-tiered architecture.Programmable Cache-Control: One Size Does Not Fit All
#webperf For addressing challenges related to performance of #mobile devices and networks, caching is making a comeback. It's interesting - and almost amusing - to watch the circle of technology run around best practices with respect to performance over time. Back in the day caching was the ultimate means by which web application performance was improved and there was no lack of solutions and techniques that manipulated caching capabilities to achieve optimal performance. Then it was suddenly in vogue to address the performance issues associated with Javascript on the client. As Web 2.0 ascended and AJAX-based architectures ruled the day, Javascript was Enemy #1 of performance (and security, for that matter). Solutions and best practices began to arise to address when Javascript loaded, from where, and whether or not it was even active. And now, once again, we're back at the beginning with caching. In the interim years, it turns out developers have not become better about how they mark content for caching and with the proliferation of access from mobile devices over sometimes constrained networks, it's once again come to the attention of operations (who are ultimately responsible for some reason for performance of web applications) that caching can dramatically improve the performance of web applications. [ Excuse me while I take a breather - that was one long thought to type. ] Steve Souders, web performance engineer extraordinaire, gave a great presentation at HTML5DevCon that was picked up by High Scalability: Cache is King!. The aforementioned articles notes: Use HTTP cache control mechanisms: max-age, etag, last-modified, if-modified-since, if-none-match, no-cache, must-revalidate, no-store. Want to prevent HTTP sending conditional GET requests, especially over high latency mobile networks. Use a long max-age and change resource names any time the content changes so that it won't be cached improperly. -- Better Browser Caching Is More Important Than No Javascript Or Fast Networks For HTTP Performance The problem is, of course, that developers aren't putting all these nifty-neato-keen tags and meta-data in their content and the cost to modify existing applications to do so may result in a prioritization somewhere right below having an optional, unnecessary root canal. In other cases the way in which web applications are built today - we're still using AJAX-based, real-time updates of chunks of content rather than whole pages - means simply adding tags and meta-data to the HTML isn't necessarily going to help because it refers to the page and not the data/content being retrieved and updated for that "I'm a live, real-time application" feel that everyone has to have today. Too, caching tags and meta-data in HTML doesn't address every type of data. JSON, for example, commonly returned as the response to an API call (used as the building blocks for web applications more and more frequently these days) aren't going to be impacted by the HTML caching directives. That has to be addressed in a different way, either on the server side (think Apache mod_expire) or on the client (HTML5 contains new capabilities specifically for this purpose and there are usually cache directives hidden in AJAX frameworks like jQuery). The Programmable Network to the Rescue What you need is the ability to insert the appropriate tags, on the appropriate content, in such a way as to make sure whatever you're about to do (a) doesn't break the application and (b) is actually going to improve the performance of the end-user experience for that specific request. Note that (b) is pretty important, actually, because there are things you do to content being delivered to end users on mobile devices over mobile networks that might make things worse if you do it to the same content being delivered to the same end user on the same device over the wireless LAN. Network capabilities matter, so it's important to remember that. To avoid rewriting applications (and perhaps changing the entire server-side architecture by adding on modules) you could just take advantage of programmability in the network. When enabled as part of a full-proxy, network intermediary the ability to programmatically modify content in-flight becomes invaluable as a mechanism for improving performance, particularly with respect to adding (or modifying) cache headers, tags, and meta-data. By allowing the intermediary to cache the cacheable content while simultaneously inserting the appropriate cache control headers to manage the client-side cache, performance is improved. By leveraging programmability, you can start to apply device or network or application (or any combination thereof) logic to manipulate the cache as necessary while also using additional performance-enhancing techniques like compression (when appropriate) or image optimization (for mobile devices). The thing is that a generic "all on" or "all off" for caching isn't going to always result in the best performance. There's logic to it that says you need the capability to say "if X and Y then ON else if Z then OFF". That's the power of a programmable network, of the ability to write the kind of logic that takes into consideration the context of a request and takes the appropriate actions in real-time. Because one size (setting) simply does not fit all.Infrastructure Architecture: Whitelisting with JSON and API Keys
Application delivery infrastructure can be a valuable partner in architecting solutions …. AJAX and JSON have changed the way in which we architect applications, especially with respect to their ascendancy to rule the realm of integration, i.e. the API. Policies are generally focused on the URI, which has effectively become the exposed interface to any given application function. It’s REST-ful, it’s service-oriented, and it works well. Because we’ve taken to leveraging the URI as a basic building block, as the entry-point into an application, it affords the opportunity to optimize architectures and make more efficient the use of compute power available for processing. This is an increasingly important point, as capacity has become a focal point around which cost and efficiency is measured. By offloading functions to other systems when possible, we are able to increase the useful processing capacity of an given application instance and ensure a higher ratio of valuable processing to resources is achieved. The ability of application delivery infrastructure to intercept, inspect, and manipulate the exchange of data between client and server should not be underestimated. A full-proxy based infrastructure component can provide valuable services to the application architect that can enhance the performance and reliability of applications while abstracting functionality in a way that alleviates the need to modify applications to support new initiatives. AN EXAMPLE Consider, for example, a business requirement specifying that only certain authorized partners (in the integration sense) are allowed to retrieve certain dynamic content via an exposed application API. There are myriad ways in which such a requirement could be implemented, including requiring authentication and subsequent tokens to authorize access – likely the most common means of providing such access management in conjunction with an API. Most of these options require several steps, however, and interaction directly with the application to examine credentials and determine authorization to requested resources. This consumes valuable compute that could otherwise be used to serve requests. An alternative approach would be to provide authorized consumers with a more standards-based method of access that includes, in the request, the very means by which authorization can be determined. Taking a lesson from the credit card industry, for example, an algorithm can be used to determine the validity of a particular customer ID or authorization token. An API key, if you will, that is not stored in a database (and thus requires a lookup) but rather is algorithmic and therefore able to be verified as valid without needing a specific lookup at run-time. Assuming such a token or API key were embedded in the URI, the application delivery service can then extract the key, verify its authenticity using an algorithm, and subsequently allow or deny access based on the result. This architecture is based on the premise that the application delivery service is capable of responding with the appropriate JSON in the event that the API key is determined to be invalid. Such a service must therefore be network-side scripting capable. Assuming such a platform exists, one can easily implement this architecture and enjoy the improved capacity and resulting performance boost from the offload of authorization and access management functions to the infrastructure. 1. A request is received by the application delivery service. 2. The application delivery service extracts the API key from the URI and determines validity. 3. If the API key is not legitimate, a JSON-encoded response is returned. 4. If the API key is valid, the request is passed on to the appropriate web/application server for processing. Such an approach can also be used to enable or disable functionality within an application, including live-streams. Assume a site that serves up streaming content, but only to authorized (registered) users. When requests for that content arrive, the application delivery service can dynamically determine, using an embedded key or some portion of the URI, whether to serve up the content or not. If it deems the request invalid, it can return a JSON response that effectively “turns off” the streaming content, thereby eliminating the ability of non-registered (or non-paying) customers to access live content. Such an approach could also be useful in the event of a service failure; if content is not available, the application delivery service can easily turn off and/or respond to the request, providing feedback to the user that is valuable in reducing their frustration with AJAX-enabled sites that too often simply “stop working” without any kind of feedback or message to the end user. The application delivery service could, of course, perform other actions based on the in/validity of the request, such as directing the request be fulfilled by a service generating older or non-dynamic streaming content, using its ability to perform application level routing. The possibilities are quite extensive and implementation depends entirely on goals and requirements to be met. Such features become more appealing when they are, through their capabilities, able to intelligently make use of resources in various locations. Cloud-hosted services may be more or less desirable for use in an application, and thus leveraging application delivery services to either enable or reduce the traffic sent to such services may be financially and operationally beneficial. ARCHITECTURE is KEY The core principle to remember here is that ultimately infrastructure architecture plays (or can and should play) a vital role in designing and deploying applications today. With the increasing interest and use of cloud computing and APIs, it is rapidly becoming necessary to leverage resources and services external to the application as a means to rapidly deploy new functionality and support for new features. The abstraction offered by application delivery services provides an effective, cross-site and cross-application means of enabling what were once application-only services within the infrastructure. This abstraction and service-oriented approach reduces the burden on the application as well as its developers. The application delivery service is almost always the first service in the oft-times lengthy chain of services required to respond to a client’s request. Leveraging its capabilities to inspect and manipulate as well as route and respond to those requests allows architects to formulate new strategies and ways to provide their own services, as well as leveraging existing and integrated resources for maximum efficiency, with minimal effort. Related blogs & articles: HTML5 Going Like Gangbusters But Will Anyone Notice? Web 2.0 Killed the Middleware Star The Inevitable Eventual Consistency of Cloud Computing Let’s Face It: PaaS is Just SOA for Platforms Without the Baggage Cloud-Tiered Architectural Models are Bad Except When They Aren’t The Database Tier is Not Elastic The New Distribution of The 3-Tiered Architecture Changes Everything Sessions, Sessions EverywhereEt Tu, Browser?
Friends, foes, Internet-denizens … lend me your browser. Were you involved in any of the DDoS attacks that occurred over the past twelve months? Was your mom? Sister? Brother? Grandfather? Can you even answer that question with any degree of certainty? Reality is that the reason for attack on the web is subtly shifting to theft not necessarily of data, but of resources. While the goal may still be to obtain personal credentials for monetary gain, it is far more profitable to rip hundreds or thousands of credentials from a single source than merely getting one at a time. From a miscreant’s point of view, the return on investment is simply much higher targeting a site than it is targeting you, directly. But that doesn’t mean you’re off the hook. In fact, quite the opposite. For there are other just as nefarious purposes to which your resources can be directed, including inadvertently participating in a grand-scale DDoS attack for what is now-a-days called “hactivism.” In both cases, you are still a victim, but you may not be aware of it as the goal is to stealth-install the means by which your compute resources can be harnessed to perpetrate an attack and it may not be caught by the security you have in place (you do have some in place, right?). You can’t necessarily count on immunity from infection because you only visit “safe sites”. That’s because one of the ways in which attackers leverage your compute resources is not through installation of adware or other malware, but directly through JavaScript loaded via infected sites. At issue is the possible collision between web application and browser security. attackers are recommending to develop a system by which people are lured to some other content, such as SPAM SPAM SPAM !!!graphy, but by visiting the website would invisibly launch the DDOS JavaScript tool. -- Researchers say: DDoS "Low Orbit Ion Cannon" attackers could be easily traced Now consider the number of serious vulnerabilities reported by WhiteHat Security during the Fall of 2010. Consider the rate across Social Networking sites. Assume an attacker managed to exploit one of those vulnerabilities and plant the DDoS JavaScript tool such that unsuspecting visitors end up playing a role in a DDoS attack. It gets worse, as far as the potential impact goes. The recent revelation of a new SSL/TLS vulnerability (BEAST) includes a pre-condition that JavaScript be injected into the browser. CSRF (Cross-Site Request Forgery) is a fairly common method of managing such a trick, and is listed by WhiteHat in the aforementioned report as having increased to 24% of all vulnerabilities. So, too, is XSS (Cross-Site Scripting) which ranks even higher in WhiteHat’s list, tying “information leakage” for the number one spot at 64%. In order to execute their attack, Rizzo and Duong use BEAST (Browser Exploit Against SSL/TLS) against a victim who is on a network on which they have a man-in-the-middle position. Once a victim visits a high-value site, such as PayPal, that uses TLS 1.0, and logs in and receives a cookie, they inject the client-side BEAST code into the victim's browser. This can be done through the use of an iframe ad or just loading the BEAST JavaScript into the victim's browser. -- New Attack Breaks Confidentiality Model of SSL, Allows Theft of Encrypted Cookies Such an attack is designed to steal high-value data such as might be stored in an encrypted cookie used to conduct transactions with Paypal or an online banking service. Depending on the level of protection at the web application layer, the delivery of such JavaScript may go completely unnoticed. Most web application security focuses on verifying user input, not application responses, are free from infection. And too many consumers believe running anti-virus scanning solutions are enough to detect and prevent infection in general, not realizing that a dynamically injected JavaScript (something many sites do all the time for monitoring performance and to enable real-time interaction) may, in fact, be “malicious” or at the very least an attempt at resource theft. How do you stop a browser that essentially stabs you in the back by accepting, without question, questionable content? Without layering additional security on the browser that parses through each and every piece of content delivered, there isn’t a whole lot you can do – other than turn off the ability to execute JavaScript which, today, essentially renders the Internet useless. GO to the SOURCE If we look at the source of browser infections we invariably find the only viable, reasonable, effective answer is to eliminate the source. When you have a pandemic you figure out what’s causing it and you go to the source. Yes, you treat the symptoms of the victims if possible, but what you want to really do is locate and whack the source so it stops spreading. In “Perceptions about Network Security” (Ponemon Institute, June 2011) surveys show that the top three sources of a breach were insider abuse (52%), malicious software download (48%), and malware from a website (43%). Interestingly, 29% indicated the breach resulted from malicious content coming from a “social networking site”, which would – when added to the malware from a website (of which social networking sites are certainly a type) that source tops the chart with 72% of all causes of breaches being a direct result of the failure of a website to secure itself, and essentially allow itself to become a carrier of an outbreak. Certainly if you have control over the desktops, laptops, and mobile devices from which a client will interact with your web site or network and you have the capability to deploy policies on those clients that can aid in securing and protecting that client, you should. But that capability is rapidly dwindling with the introduction of a vast host of clients with wildly different OS footprints and the incompatibility of client-side, OS specific agents and apps capable of supporting a holistic client-side security strategy. Enforcing policies regarding interaction with corporate resources is really your best and most complete option. Like a DDoS attack, you are unlikely to be able to stop the infection of a client. You can, however, stop the spread and possible infection of your corporate resources as a carrier. The more organizations that attend to their own house’s security and protection, the better off end-users will likely be. Reducing the sources of the pandemic of client-side infections will reduce the risk not only to your own organization and users, but to others. And if we can all reduce the potential sources down to sites relying on user’s specifically visiting an infected site, the client-side mechanisms in place to protect users against known malware distribution sites will get us further to a safer and more enjoyable Internet. When the Data Center is Under Siege Don’t Forget to Watch Under the Floor The Many Faces of DDoS: Variations on a Theme or Two Spanish police website hit by Anonymous hackers (June 2011) What We Learned from Anonymous: DDoS is now 3DoS Custom Code for Targeted Attacks Defense in Depth in Context The Big Attacks are Back…Not That They Ever Stopped (IP) Identity Theft in Cloud Computing Environments If Security in the Cloud Were Handled Like Car Accidents
