BER and DER: Why Encoding and Decoding Matter
When dealing with software development in nearly any capacity you will be met with, often times rather rapidly, the concept of abstraction. That is, being able to build around a given piece of a program or system without worrying about how it is specifically implemented or represented. This requires a means of leaving things open ended in the sense that you don’t have specifics from a package or system, but know that one will be there, and trust that you will be able to communicate with it.
To achieve this there are many options, but undoubtedly the most common is the OSI model. Yes, that too is an abstraction system. It’s a series of interdependent layers that, as you travel up the stack, allows you to have more and more abstraction built into the system operating at that layer. At layer 7, you can trust that ones and zeroes will always be passable by lower layers, for instance. These strings of ones and zeroes will always pass on the lower layers, regardless of what the layer 7 system and logic looks like. This ensures that any application can pass data in this format without having to worry about whether any other application can accept it.
It is likely for this reason, among others, that the OSI’s method of specifying abstraction (Abstract Syntax Notation One, or ASN.1) has a set of rules for presenting data objects in precisely this format. BER (Basic Encoding Rules) is a simple, flexible system for notation that has the capacity to define multiple data types (integers, bit strings, sets, sequences and more), while remaining quite simple itself, only requiring a string of eight bit octets. What does all that detail mean? It means BER is an extremely simple, very widely accepted means of taking what might be complex data and encoding into a simple to pass, understand and parse format. This allows applications everywhere to effectively speak the same language when passing information.
Why do you care? If applications everywhere are passing data in a given language, your network should probably understand it, right? The importance of this premise increases in direct proportion with the amount of intelligence, business logic and automation you want your network to provide. If you’re looking to extend your application logic into the network space, then the network had better understand what your apps are doing. For quite some time this has meant doing by-hand parsing of ASN.1 (BER) encoded data, which is … not exactly a joy. Times have changed, though, and now there are built in BER (and DER, which is a subset of BER) encoding and decoding capabilities built directly into iRules. This means that the pain and suffering of those iRulers that have gone before you need not be your own. You can avoid this suffering by making use of these new, powerful commands. Let’s take a look at them, shall we:
The first in the command line-up, the element command is pretty much what you’d expect. If you’re going to inspect/encode/decode elements, you need to first be able to select and define them. This command allows you to perform many different specific functions that will return data about ASN1 encoded elements. Whether you’re looking for the encoding type, the size, the tag, or an offset portion of the data within an element, this command can get you there, and beyond. The first and most important thing you’ll be doing with this command is the “init” function. Running the command:
ASN1::element init <BER|DER>
Or in the real world:
set ele [ASN1::element init BER]
will return an object handle that can be used by subsequent commands to allow them to function appropriately with a specified encoding format.
This is your bread and butter when looking to unravel the ASN1 formatted data that has been encoded and passed by other applications up stream. If you have encoded data and you’re looking to unpack it so that you can read or manipulate it, the decode command is what you’re looking for.
Here you specify an element, which is the data that’s going to be decoded, and then a “formatString” that is made up of a set of characters that define how the data is encoded. The results of that decoding are then stored in a variable, which is why this command requires an output variable name. That variable may either exist or not, this command will create as necessary. A simple example of this command would be:
ASN1::decode element formatString ?varName varName
Or in a real world example:
ASN1::decode $ele "?a?aa?b" ruleId type matchValue dnAttrs
Keep in mind that this command’s usage will vary widely depending on the formatting of the data at hand. Check the DevCentral page for details on all of the formatString options and more details on usage, as well as an example.
Last but certainly not least is the encode command. Just like the decode command is the piece that allows you to take the encoded data and unpack it for reading/modification/action, the encoding command is the counterpart that puts the pieces back together. Whether you’re looking to re-encode data that you’ve just decoded and processed or create your own BER or DER encoded message to send off box, ASN1::encode has your covered.
The syntax is shockingly similar to the decode command if all you’re trying to do is encode an entire string. For that you end up with something like:
ASN1::encode encodingType formatString ?value value
Or for a real world peek:
ASN1::encode "?a?aa?b" ruleId type matchValue dnAttrs
which is pretty much the same as decoding, syntax wise.
The fun doesn’t stop there, however. There are more options when encoding. At times you won’t want to simply encode an entire string. You may want to insert data into a certain point in a string, or replace data in a string. Those too are options with the ASN1::insert and ASN1::replace commands, that are effectively subsets of the ASN1::encode command. For specifics on all three of these commands I recommend checking out the DevCentral docs on ASN1::encode.
The best way I can illustrate, I believe, is to give you a before and after, akin to a makeover reveal on TV. For our makeover we’re going to be using a beloved iRule in my world. This iRule allows LDAP to function with F5’s OneConnect. LDAP clients generally send explicit unbind commands to the server, which then cause the server to terminate the server side connection, thereby making OneConnect or any other connection pooling technology moot. To combat that we need to watch for those unbind commands, stop them, and only them, and then force the client to unbind, thinking the server’s side is happily terminated.
This has been doable for years in iRules, but due to the decoding and binary scanning necessary, it was not exactly for the faint of heart. Here is how this used to look:
1: when CLIENT_ACCEPTED {
2: TCP::collect
3: }
4: 5: when CLIENT_DATA {
6: binary scan [TCP::payload] xc ber_len
7: if { $ber_len < 0 } {
8: set ber_index [expr 2 + 128 + $ber_len]
9: } else {
10: set ber_index 2
11: }
12: 13: # message id</pre>
14: binary scan [TCP::payload] @${ber_index}xcI ber_len ber_len_ext
15: if { $ber_len < 0 } {
16: set ext_len [expr 128 + $ber_len]
17: set ber_len [expr ($ber_len_ext & 0xffffffff) >>(4-$ext_len)*8)]
18: } else {
19: set ext_len 0
20: } 21: incr ber_index [expr 2 + $ext_len + $ber_len]
22: 23: # ldap message
24: binary scan [TCP::payload] @${ber_index}c ber_type
25: if { [expr $ber_type & 0x1f] == 2 } {
26: log local0. "unbind => detach"
27: TCP::payload replace 0 [TCP::payload length] ""
28: LB::detach
29: } 30: TCP::release
31: TCP::collect
32: }Not exactly the most easy to read rule in existence, right? And, I’d wager, unless you’re quite comfortable with BER encoding and what is required, likely not something you’d be putting together yourself any time soon. I know I wasn’t.
So, what does this look like now with the built in ASN1 commands to handle the BER heavy lifting? Trust me when I say the new commands make your life easier:
1: when CLIENT_ACCEPTED {
2: TCP::collect
3: } 4: when CLIENT_DATA {
5: set ele [ASN1::element init BER]
6: # Skip ahead two elements and check for an unbind tag (0x62)
7: if { [ASN1::element tag [ASN1::element next $ele 2]] == 0x62 } {
8: log local0. "unbind => detach"
9: TCP::payload replace 0 [TCP::payload length] ""
10: LB::detach
11: } 12: TCP::release
13: TCP::collect
14: }
If that isn’t obviously shorter, simpler, easier to write, read and maintain…I don’t know what is. These new commands to parse ASN1 defined BER and DER encoding represent a huge step towards application fluency, app focused programmability, and app integration to the network layer. Our programmability story is far from over, however, and there is a lot more in the works to continue giving our code weaving brothers and sisters the tools they need to make their network’s programmable nature a valuable arrow in their app design, implementation and maintenance quiver.
