Extract content of Certificate key file with REST or Ansible

Stefan_Engel · ‎01-Jun-2021

Hi Community,

I'm working on an automation for renewing Certificates on multiple BIG-IP's using Ansible.

As not all available Ansible F5 modules provide what is required, I'm currently using a mix of modules and REST calls (which is call from Ansible).

F5 Module Index

What works so far is:

Create new CSR/Key on BIG-IP
Get new "CA based" Cert and upload to the BIG-IP
Upload the same Cert to other BIG-IP's
Update SSL profiles on multiple BIG-IP's
and some others tasks, like irules..etc

Anyhow, what doesnt work so far is to get the content of the key which was created on the first device together with the CSR. Basically I dont have the key which needs to be uploaded to the other BIG-IP's as well.

From the CLI, the following gives me what I need:

cat /config/filestore/files_d/Common_d/certificate_key_d/*name.key*

The problem with this is, I cant integrate it in Ansible using the bigip_command – Run TMSH and BASH commands on F5 devices module. Looks like only tmsh commands are supported even though it states BASH as well. Plus I try to avoid using this module whenever possible in a first place.

Through the GUI, simple export and import on an other device - done, but obviously not automated.

I have tried all possible Ansible modules as well as REST calls, but dont get the content out of the .key file.

I thought that this would/should be a simple tasks. If anyone's done this using any approach please share.

I could create a new key and get a cert for each device, but first try to find out if there's another way.

Thanks in advance,

Stefan

Daniel_Wolf · ‎02-Jun-2021

Hi Stefan,

I guess using SOAP is not an option: https://clouddocs.f5.com/api/icontrol-soap/Management__KeyCertificate.html ?

Another option, but that is also sort of working around the problem, you can download files like described here: https://support.f5.com/csp/article/K41763344#download_generic

Means the private key file must exist in https://<ip address>/mgmt/shared/file-transfer/uploads/<filename>, maybe as a copy.

KR

Daniel

Stefan_Engel · ‎03-Jun-2021

Thanks Daniel.

https://support.f5.com/csp/article/K41763344#download_generic might work, but not supported in v15.x.x or later anymore.

I solved it like this..may not the prettiest, but works:

- name: Extract Certificate Private Key
    raw: cat /config/filestore/files_d/Common_d/certificate_key_d/\:Common:vanity_{{request_id}}.key*
    register: vanity_key
    become: yes
    delegate_to: "{{ provider.server }}"
    vars:
      ansible_ssh_user: "{{ provider.user }}"
      ansible_ssh_pass: "{{ provider.password }}"
 
- name: Import Cert Key
    bigip_ssl_key:
      name: "vanity_{{ request_id }}"
      state: present
      content: "{{ vanity_key.stdout }}"
      provider: "{{ provider }}"

StephanManthey · ‎08-Jul-2021

Exporting key material is not supported as far as I know. Using the CLI is the only option I am aware of.

Only a CSR can be exported.

Creating a CSR:

- name: create string of subject alternative names when defined in inventory
  set_fact:
    san_list: "{{ san_list | default([]) + ['DNS:%s' | format(item.name)] }}"
  with_items: "{{ hostvars[inventory_hostname].certificates.server[0].subject_alternative_names | selectattr('name', 'defined') | list }}"
  when: hostvars[inventory_hostname].certificates.server[0].subject_alternative_names is defined
 
- name: add signing request name and properties to data structure
  set_fact:
    csr_data: "{{ {
        'csr_consumer': 'ltm',
        'csr_prefix': 'cert',
        'csr_suffix': device_info[inventory_hostname].ansible_host_time,
        'csr_name': hostvars[inventory_hostname].certificates.server[0].name,
        'csr_cn': hostvars[inventory_hostname].certificates.server[0].subject.common_name,
        'csr_san': san_list | default(['DNS:%s' | format(hostvars[inventory_hostname].certificates.server[0].name)]) | join(','),
        'csr_country': hostvars[inventory_hostname].certificates.server[0].subject.location_country | default('US'),
        'csr_state': hostvars[inventory_hostname].certificates.server[0].subject.location_state | default(''),
        'csr_city': hostvars[inventory_hostname].certificates.server[0].subject.location_city | default(''),
        'csr_organization': hostvars[inventory_hostname].certificates.server[0].subject.organization_name | default(''),
      } }}"
    
- name: create certificate signing request on load balancer
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/sys/crypto/csr
    method: POST
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      'name': "{{ csr_data.csr_prefix + '_' + csr_data.csr_name + '_' + csr_data.csr_suffix }}"
      'key': "{{ csr_data.csr_prefix + '_' + csr_data.csr_name + '_' + csr_data.csr_suffix }}"
      'common-name': "{{ csr_data.csr_cn }}"
      'subject-alternative-name': "{{ csr_data.csr_san }}"
      'country':  "{{ csr_data.csr_country }}"
      'state': "{{ csr_data.csr_state }}"
      'city': "{{ csr_data.csr_city }}"
      'organization': "{{ csr_data.csr_organization }}"
      'consumer': "{{ csr_data.csr_consumer }}"
  register: result
  until: result.status == 200
  retries: 40
  delay: 5
  when: csr_data is defined

Exporting a CSR:

- name: add certificate signing request name to data structure
  set_fact:
    csr_data: "{{ {
        'csr_prefix': 'cert',
        'csr_suffix': device_info[inventory_hostname].ansible_host_time,
        'csr_name': hostvars[inventory_hostname].certificates.server[0].name,
        'csr_file_extension': 'csr'
      } }}"
      
- name: debug data
  debug: 
    msg: "{{ csr_data }}"
    
- name: export certificate signing request
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/util/bash
    method: POST
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      command: run
      utilCmdArgs: "-c 'tmsh list sys crypto csr {{ csr_data.csr_prefix + '_' + csr_data.csr_name + '_' + csr_data.csr_suffix }} | sed -re \'/-----/,/-----/!d\''"
  register: csr_result
  
- name: debug data
  debug:
    msg: "{{ csr_result.json.commandResult }}"
    
- name: save certificate signing request to local file and replace encoded new line characters
  copy:
    content: "{{ csr_result.json.commandResult | regex_replace('\\n', '\n') }}"
    dest: "{{ '~/ssl/server/csr/%s.%s' | format(csr_data.csr_prefix + '_' + csr_data.csr_name + '_' + csr_data.csr_suffix, csr_data.csr_file_extension) }}"
    force: yes

Uploading all key material:

- name: register cert file properties
  stat:
    path: "{{ crt_file_path }}"
  register: crt_properties
  when: crt_file_path is defined
  
- name: set certificate file size information
  set_fact:
    crt_file_size: "{{ crt_properties.stat.size }}"
  when: crt_properties is defined
  
- name: register key file properties
  stat:
    path: "{{ key_file_path }}"
  register: key_properties
  when: key_file_path is defined
  
- name: set key file size information
  set_fact:
    key_file_size: "{{ key_properties.stat.size }}"
  when: key_properties is defined
  
- name: register chain file properties
  stat:
    path: "{{ chain_file_path }}"
  register: chain_properties
  when: chain_file_path is defined
  
- name: set chain file size information
  set_fact:
    chain_file_size: "{{ chain_properties.stat.size }}"
  when: chain_properties is defined
  
- name: copy certificate to temp directory
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/shared/file-transfer/uploads/{{ crt_file_name }}
    method: POST
    headers:
      Content-Range: "0-{{ crt_file_size | int - 2 }}/{{ crt_file_size }}"
      Content-Type: "application/octet-stream"
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body: "{{ lookup('file', crt_file_path) }}"
    
- name: copy key to temp directory
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/shared/file-transfer/uploads/{{ key_file_name }}
    method: POST
    headers:
      Content-Range: "0-{{ key_file_size | int - 2 }}/{{ key_file_size }}"
      Content-Type: "application/octet-stream"
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body: "{{ lookup('file', key_file_path) }}"
    
- name: copy chain to temp directory
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/shared/file-transfer/uploads/{{ chain_file_name }}
    method: POST
    headers:
      Content-Range: "0-{{ chain_file_size | int - 2 }}/{{ chain_file_size }}"
      Content-Type: "application/octet-stream"
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body: "{{ lookup('file', chain_file_path) }}"
    
- name: copy certificate to TMOS filestore (TMOS v14+)
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/sys/crypto/cert
    method: POST
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      command: install
      name: "{{ '%s_%s_%s' | format(crt_prefix,crt_name,crt_suffix) }}"
      from-local-file: "/var/config/rest/downloads/tmp/{{ '%s_%s_%s.%s' | format(crt_prefix,crt_name,crt_suffix,crt_file_extension) }}"
      
- name: copy key to TMOS filestore (TMOS v14+)
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/sys/crypto/key
    method: POST
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      command: install
      name: "{{ '%s_%s_%s' | format(crt_prefix,crt_name,crt_suffix) }}"
      from-local-file: "/var/config/rest/downloads/tmp/{{ '%s_%s_%s.%s' | format(crt_prefix,crt_name,crt_suffix,key_file_extension) }}"
      
- name: copy chain to TMOS filestore (TMOS v14+)
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/sys/crypto/cert
    method: POST
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      command: install
      name: "{{ '%s_%s_%s' | format(chain_prefix,crt_name,crt_suffix) }}"
      from-local-file: "/var/config/rest/downloads/tmp/{{ '%s_%s_%s.%s' | format(chain_prefix,crt_name,crt_suffix,crt_file_extension) }}"
      
- name: specifiy name of profile (TMOS v14+ does not use the file extension)
  set_fact:
    profile: "{{ '%s_%s_%s' | format(profile_prefix,crt_name,profile_suffix) }}"
    
- name: lookup current cert, key and chain assignments in client-ssl profile
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/ltm/profile/client-ssl/{{ profile }}?$select=cert,key,chain
    method: GET
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
  register: lookup_result
      
- name: modify existing client-ssl persistence
  uri:
    validate_certs: no
    url: https://{{ inventory_hostname }}/mgmt/tm/ltm/profile/client-ssl/{{ profile }}
    method: PATCH
    headers:
      X-F5-Auth-Token: "{{ device_info[inventory_hostname].token }}"
    body_format: json
    body:
      cert: "{{ '%s_%s_%s' | format(crt_prefix,crt_name,crt_suffix) }}"
      key: "{{ '%s_%s_%s' | format(crt_prefix,crt_name,crt_suffix) }}"
      chain: "{{ '%s_%s_%s' | format(chain_prefix,crt_name,crt_suffix) }}"

If you are exporting the key material anyway, why not creating the key pair and CSR outside the F5? (see next reply, please)

StephanManthey · ‎08-Jul-2021

As mentioned before, I prefer creating the private/public key pair outside the BIG-IP and finally upload the signed certificate and chain to the BIG-IP device(s).

Here is how I create the key material and CSR (and sign it locally for testing purposes):

- hosts: localhost
  gather_facts: yes
  
  tasks:
  
  - name: retrieve system time information (YYYYMMDD_HHMMSS)
    set_fact:
      ansible_clock_information: "{{ '%s%s%s_%s%s%s' | format(ansible_date_time.year, ansible_date_time.month, ansible_date_time.day, ansible_date_time.hour, ansible_date_time.minute, ansible_date_time.second) }}"
    delegate_to: localhost
    
  - name: debug ansible_clock_information
    debug:
      msg: "{{ ansible_clock_information }}"
      
  - name: include variables file (required for local CA access)
    no_log: true
    include_vars: sample_credentials.yml
    
  - name: include certificate variables file and suppress logging
    include_vars: 
      file: "{{ certificate_data }}"
      name: certificate_config
      
  - name: debug variables
    debug:
      msg: "{{ certificate_config.server_certificate }}"
      
  - name: create string of subject alternative names when defined in inventory
    set_fact:
      san_list: "{{ san_list | default([]) + ['DNS:%s' | format(item.name)] }}"
    with_items: "{{ certificate_config.server_certificate.subject_alternative_names | selectattr('name', 'defined') | list }}"
    when: certificate_config.server_certificate.subject_alternative_names is defined
    
  - name: debug string of subject alternative names
    debug:
      msg: "{{ san_list }}"
      
  - name: add signing request name and properties to data structure
    set_fact:
      csr_data: "{{ {
          'csr_type': certificate_config.server_certificate.key_type,
          'csr_length': certificate_config.server_certificate.key_length,
          'csr_prefix': 'cert',
          'csr_suffix': ansible_clock_information,
          'csr_name': certificate_config.server_certificate.name,
          'csr_cn': certificate_config.server_certificate.subject.common_name,
          'csr_san': san_list | default(['DNS:%s' | format(certificate_config.server_certificate.name)]) | join(','),
          'csr_country': certificate_config.server_certificate.subject.location_country | default('US'),
          'csr_state': server_certificate.subject.location_state | default(''),
          'csr_city': certificate_config.server_certificate.subject.location_city | default(''),
          'csr_organization': certificate_config.server_certificate.subject.organization_name | default(''),
        } }}"
        
  - name: debug data
    debug: 
      msg: "{{ csr_data }}"
      
  - name: assemble key material file name
    set_fact:
      key_file_name: "{{ csr_data.csr_prefix + '_' + csr_data.csr_name + '_' + csr_data.csr_suffix }}"
      
  - name: assemble certificate subject
    set_fact:
      certificate_subject: "{{ '/C=' + csr_data.csr_country + '/ST=' + csr_data.csr_state + '/L=' + csr_data.csr_city + '/O=' +csr_data.csr_organization + '/CN=' + csr_data.csr_cn }}"
      
  - name: generate a new private/public key pair
    command: openssl gen{{ csr_data.csr_type | lower }} -out ~/ssl/server/material/{{ key_file_name }}.key  {{ csr_data.csr_length }}
    
  - name: generate temporary config files with subject alternative names
    copy:
      dest: "~/ssl/server/tmp/{{ key_file_name }}.cnf"
      content: |
        [SAN]
        subjectAltName={{ csr_data.csr_san }}
        [ req ]
        distinguished_name  = req_distinguished_name
        [ req_distinguished_name ]
        
  - name: generate CSRs
    command: openssl req -new -sha256 -reqexts SAN -key ~/ssl/server/material/{{ key_file_name }}.key -out ~/ssl/server/material/{{ key_file_name }}.csr  -subj '{{ certificate_subject }}' -config ~/ssl/server/tmp/{{ key_file_name }}.cnf
    
  - name: generate key/csr information
    set_fact:
      key_csr_info:
        reference: "{{ csr_data.csr_name }}"
        timestamp: "{{ csr_data.csr_suffix }}"
        new_key: "{{ key_file_name + '.key' }}"
        new_csr: "{{ key_file_name + '.csr' }}"
        
  - name: save timestamp information to local file
    copy:
      content: "{{ key_csr_info | to_nice_yaml }}"
      dest: "{{ '~/ssl/server/material/cert_%s.info' | format(csr_data.csr_name) }}"
      force: yes
      
  - name: print file locations
    debug:
      msg: 
      - "{{ 'new private key created: ~/ssl/server/material/' + key_file_name + '.key' }}"
      - "{{ 'new signing req created: ~/ssl/server/material/' + key_file_name + '.csr' }}"
      - "{{ 'new key/csr information: ~/ssl/server/material/cert_%s.info' | format(csr_data.csr_name) }}"
      
  - name: sign certificate with local CA
    command: openssl ca -notext -md sha256 -days 730 -batch -passin pass:{{ intermediate_ca_credentials.secret }} -out ~/ssl/server/material/{{ key_file_name }}.crt -config ~/ssl/intermediate/openssl.cnf -extensions server_cert -in ~/ssl/server/material/{{ key_file_name }}.csr
    
  - name: save intermediate CA chain (src is referencing the locally saved chain of intermediate CAs)
    copy:
      src: "~/ssl/intermediate/crt/intermediate-ca.crt"
      dest: "{{ '~/ssl/server/material/chain_' + csr_data.csr_name + '_' + csr_data.csr_suffix + '.crt' }}"
      
  - name: print file locations
    debug:
      msg: 
      - "{{ 'new signed cert created: ~/ssl/server/material/' + key_file_name + '.csr' }}"
      - "{{ 'new intermediate saved: ~/ssl/server/material/chain_' + csr_data.csr_name + '_' + csr_data.csr_suffix + '.crt' }}"

Cheers, Stephan

DevCentral

Extract content of Certificate key file with REST or Ansible