Forum Discussion

Patrik_Jonsson's avatar
Aug 02, 2022

Unable to update device cert

Hi!

Working on a hobby project to manage F5 certificates using Kubernetes and cert-manager (Ref: https://community.f5.com/t5/technical-forum/kubernetes-cert-manager-letsencrypt-f5/td-p/299218). However, I have been running into problems when configuring httpd to use the new certificate+key.

Pretty much trying to follow https://support.f5.com/csp/article/K12522815

  1. Copied new cert
  2. Copied the new key
  3. Set the certificate + key via a REST call to /mgmt/tm/sys/httpd (like the article suggests)
  4.  Next step is to restart httpd using a rest call to /mgmt/tm/sys/service but by then httpd is broken

Try to restart httpd via ssh:

# bigstart restart httpd
Stopping httpd:                                            [  OK  ]

Broadcast message from systemd-journald@bigip.xip.se (Tue 2022-08-02 13:13:51 PDT):

httpd[4453]: [ssl:emerg] [pid 4453] (13)Permission denied: AH02201: Init: Can't open server certificate file /config/httpd/conf/ssl.crt/management.crt

Starting httpd:                                            [FAILED]

The certificate is there:

ls -la /config/httpd/conf/ssl.crt
total 28
drwx------. 2 root root   4096 Aug  2 13:12 .
drwxr-xr-x. 7 root root   4096 Aug  1 14:52 ..
-rw-r-----. 1 root apache 5582 Aug  2 13:12 management.crt
-rw-------. 1 root root   5582 Aug  2 12:28 server.crt
-rw-------. 1 root root   1521 Aug  2 12:27 server.crt.old

So is the key:

ls -la /config/httpd/conf/ssl.key
total 20
drw-------. 2 root root 4096 Aug  2 12:58 .
drwxr-xr-x. 7 root root 4096 Aug  1 14:52 ..
-rw-------. 1 root root 1674 Aug  2 12:58 management.key
-rw-------. 1 root root 1674 Aug  2 12:27 server.key
-rw-------. 1 root root 1675 Aug  2 12:27 server.key.old

They certificate and key matches:

# openssl x509 -noout -modulus -in /config/httpd/conf/ssl.crt/management.crt 
Modulus=B0E9B8F6A9084A134E5575E6374E4BCF30D56D7A2F728921AA4A3A433E0A349EED6455C9AFF2F5D151642D74073DB20BD8250E9DDCE06CA2CA9A3A4ADC55923396AE4F3EA612E4EBC66C2F82DEBEC27F87ED63E051A40FE3B7E33754EA65598467E7B0E81F13E5B508966F3A10B16E6E62A7EB181104C66AA52F013D3BD879A545D3A4C8A473D03B380625823E36D4EA3EAD9039A4BC7F01E9E3C6DF005B593AB5EAD62D59A9CCF7B2AFDC8F744D8BFACDB9B92DB471E52709186540E34893B9525B8DF1A4C4837A98032A648FFCB9D70F2BEDEF388504BAB59191BA893389CC8BD3B0E446143086D1CDAE9D51CB13704C2CF7A61C679293939AAB6FC63F0B87

# openssl rsa -noout -modulus -in /config/httpd/conf/ssl.key/management.key
Modulus=B0E9B8F6A9084A134E5575E6374E4BCF30D56D7A2F728921AA4A3A433E0A349EED6455C9AFF2F5D151642D74073DB20BD8250E9DDCE06CA2CA9A3A4ADC55923396AE4F3EA612E4EBC66C2F82DEBEC27F87ED63E051A40FE3B7E33754EA65598467E7B0E81F13E5B508966F3A10B16E6E62A7EB181104C66AA52F013D3BD879A545D3A4C8A473D03B380625823E36D4EA3EAD9039A4BC7F01E9E3C6DF005B593AB5EAD62D59A9CCF7B2AFDC8F744D8BFACDB9B92DB471E52709186540E34893B9525B8DF1A4C4837A98032A648FFCB9D70F2BEDEF388504BAB59191BA893389CC8BD3B0E446143086D1CDAE9D51CB13704C2CF7A61C679293939AAB6FC63F0B87

 If I run a "bigstart restart" AND "bigstart restart httpd" it suddenly works fine:

# bigstart restart
# bigstart status httpd
httpd (pid 20978) is running...
# bigstart restart httpd
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]

 I am guessing additional services needs to be restarted before httpd? What am I missing?

Kind regards,
Patrik

  • Patrik_Jonsson's avatar
    Patrik_Jonsson
    Aug 07, 2022

    Final solution:

     

     

    def set_management_cert(self, cert_name, key_name):
    
        self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.crt/{cert_name}')
        self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.key/{key_name}')
    
        self.session.put(
            f'https://{self.device}/mgmt/tm/sys/httpd',
            json={
                'sslCertfile': '/config/httpd/conf/ssl.crt/management.crt',
                'sslCertkeyfile': '/config/httpd/conf/ssl.key/management.key'}
        )
        try:
            logger.info('Restarting httpd')
            self.run_bash_command('bigstart restart httpd; killall -9 httpd;bigstart restart httpd;')
        except:
            logger.info('Waiting for management interface to restart')
            time.sleep(3)
            httpd_config = self.get_httpd_config()
    
            if os.path.basename(httpd_config['sslCertfile']) == cert_name \
                    and os.path.basename(httpd_config['sslCertkeyfile']) == key_name:
                print('Certificate has been updated and the httpd interface is responding')
            else:
                raise Exception('Failed to update the certificate')

     

     

    Thank you for the suggestions and tips Dario_Garrido . Definitely helped me find the solution!

    • Patrik_Jonsson's avatar
      Patrik_Jonsson
      Icon for MVP rankMVP

      No article planned, but that was a good idea. Might write one if I can get it to work. 👍

  • Good ideas, I forgot about tomcat!

    I'm afraid neither of the articles worked. Same issue.

    Kind regards,
    Patrik

    • Patrik_Jonsson's avatar
      Patrik_Jonsson
      Icon for MVP rankMVP

      Same thing happens if I update the cert via REST, pause the script and run:

      tmsh modify sys httpd ssl-certfile /config/httpd/conf/ssl.crt/management.crt ssl-certkeyfile /config/httpd/conf/ssl.key/management.key

      httpd[4571]: [ssl:emerg] [pid 4571] (13)Permission denied: AH02201: Init: Can't open server certificate file /config/httpd/conf/ssl.crt/management.crt

      • Patrik_Jonsson's avatar
        Patrik_Jonsson
        Icon for MVP rankMVP

        Came a bit further just now. Looks like SELinux might be making my life harder:

        restorecon -RvF  /var/config/rest/downloads/management.crt /config/httpd/conf/ssl.crt/
        restorecon -RvF /var/config/rest/downloads/management.key /config/httpd/conf/ssl.key/

         Then the permission error goes away but restarting the service does not work. Then the article Dario_Garrido gave came in handy:

         

        bigstart restart httpd
        killall -9 httpd
        bigstart restart httpd;

         

        Will see if I can solve this further tomorrow. Good input with https://support.f5.com/csp/article/K13292945 Dario_Garrido . Saved me quite some time!

        Kind regards,
        Patrik

    • PeteWhite's avatar
      PeteWhite
      Icon for Employee rankEmployee

      good job - i'd have suggested restorecon. Always a good idea to try it first with files, permissions etc