Fun with x509: extract RSA Public from SSL cert for use with iRule CRYPTO commands

Problem this snippet solves:

iRule CRYPTO commands do not allow the use of SSL / TLS certificates for encryption and decription operations.

This snippet provides a simple way to extract the RSA Public key from a x509 certificate by parsing its ASN.1 structure. The output key can then be given as [-key] input to the CRYPTO::[encrypt|decrypt] commands in combination with the use of the rsa-pub and rsa-priv algorithms [-alg].

The example only encrypts a test string with the Client's SSL certificate to demonstrate the functionality but you have unlimited options here.

For instance, you are not limited to using your client's certificate (e.g. [SSL::cert 0]): any PEM (base64) or DER representation of a public x509 cert will also work. When using the PEM format just don't forget to perform a b64decode before feeding it to the getRSAPubFromX509 proc.

Credits to Kevin Stewart for the original idea and code for performing Extended X509 Certificate Parsing

How to use this snippet:

1. Create Virtual Server

2. Select "http" under "HTTP Profile"

3. Create SSL Profile (Client)
   3.1 Configure the "Certificate Key Chain" with a proper or self-signed ssl key pair
   3.2 Select "require" under "Client Certificate"

4. Assign the SSL Profile to the Virtual Server

5. Create an iRule out of the snippet below and assign it to the Virtual Server

6. Generate traffic by use of an appropriate client SSL key pair

7. JSON Parse the response.

Test decryption with openssl:

Copy the "encryptedMsg" JSON value within the response payload (e.g. using JSON.Parse()) in a file called enc.txt, then decode it from base64 into the encrypted binary string and use the client's private key to decrypt.

base64 -d enc.txt > encbin.txt
openssl rsautl -inkey key.pem -decrypt -in encbin.txt

Code :

when HTTP_REQUEST {

  set cert [SSL::cert 0]

  set rsaFromX509 [call getRSAPubFromX509 $cert]

  log local0. "PEM encoded RSA Public: $rsaFromX509"

  set testString "test string to encrypt"

  set encBytes [CRYPTO::encrypt -alg rsa-pub -key $rsaFromX509 $testString]
  set encBytesPrintable [b64encode $encBytes]

  log local0. "Encrypted b64encoded string: $encBytesPrintable"
  
  set jsonResponse "{ \"encryptedMsg\": \"$encBytesPrintable\" }"

  HTTP::respond 200 content $jsonResponse
}

proc getRSAPubFromX509 { bincert } {
  
  if { [catch {
    # rsaEncryption OID=1.2.840.113549.1.1.1 - HEX=\x06\x09\x2A\x86\x48\x86\xF7\x0D\x01\x01\x01
    set offset [string first \x06\x09\x2A\x86\x48\x86\xF7\x0D\x01\x01\x01 $bincert]
    # Shift the offset before the ASN.1 sequence length
    set offset [expr { $offset - 4 }]
    binary scan "[string index $bincert $offset][string index $bincert [expr $offset + 1]]" S seqlen
    # Convert the ASN.1 length to unsigned integer
    set seqlen [expr {$seqlen & 0xffff}]
    # Set a pointer to the beginning of the sequence
    set startSeq [expr $offset - 2]
    # Set a pointer to the end of the sequence
    set endSeq [expr $startSeq + 3 + $seqlen]
    set rsaEncryption [string range $bincert $startSeq $endSeq]
  } error] } {
    set rsaEncryption 0
  }

  if { $rsaEncryption == 0 } {
    return 0
  }
  
  binary scan $rsaEncryption a* rsaEncryption

  set encodedKey [b64encode $rsaEncryption]

  set formattedKey "-----BEGIN PUBLIC KEY-----\x0A"

  for {set i 0} {$i < [string length $encodedKey]} {incr i 64} {
    set formattedKey "${formattedKey}\x0A[string range $encodedKey $i [expr {$i + 63}]]"
  }

  set formattedKey "${formattedKey}\x0A-----END PUBLIC KEY-----\x0A"

  return $formattedKey

}

Tested this on version: