02-Aug-2022 13:35
Hi!
Working on a hobby project to manage F5 certificates using Kubernetes and cert-manager (Ref: https://community.f5.com/t5/technical-forum/kubernetes-cert-manager-letsencrypt-f5/td-p/299218). However, I have been running into problems when configuring httpd to use the new certificate+key.
Pretty much trying to follow https://support.f5.com/csp/article/K12522815
Try to restart httpd via ssh:
# bigstart restart httpd
Stopping httpd: [ OK ]
Broadcast message from systemd-journald@bigip.xip.se (Tue 2022-08-02 13:13:51 PDT):
httpd[4453]: [ssl:emerg] [pid 4453] (13)Permission denied: AH02201: Init: Can't open server certificate file /config/httpd/conf/ssl.crt/management.crt
Starting httpd: [FAILED]
The certificate is there:
ls -la /config/httpd/conf/ssl.crt
total 28
drwx------. 2 root root 4096 Aug 2 13:12 .
drwxr-xr-x. 7 root root 4096 Aug 1 14:52 ..
-rw-r-----. 1 root apache 5582 Aug 2 13:12 management.crt
-rw-------. 1 root root 5582 Aug 2 12:28 server.crt
-rw-------. 1 root root 1521 Aug 2 12:27 server.crt.old
So is the key:
ls -la /config/httpd/conf/ssl.key
total 20
drw-------. 2 root root 4096 Aug 2 12:58 .
drwxr-xr-x. 7 root root 4096 Aug 1 14:52 ..
-rw-------. 1 root root 1674 Aug 2 12:58 management.key
-rw-------. 1 root root 1674 Aug 2 12:27 server.key
-rw-------. 1 root root 1675 Aug 2 12:27 server.key.old
They certificate and key matches:
# openssl x509 -noout -modulus -in /config/httpd/conf/ssl.crt/management.crt
Modulus=B0E9B8F6A9084A134E5575E6374E4BCF30D56D7A2F728921AA4A3A433E0A349EED6455C9AFF2F5D151642D74073DB20BD8250E9DDCE06CA2CA9A3A4ADC55923396AE4F3EA612E4EBC66C2F82DEBEC27F87ED63E051A40FE3B7E33754EA65598467E7B0E81F13E5B508966F3A10B16E6E62A7EB181104C66AA52F013D3BD879A545D3A4C8A473D03B380625823E36D4EA3EAD9039A4BC7F01E9E3C6DF005B593AB5EAD62D59A9CCF7B2AFDC8F744D8BFACDB9B92DB471E52709186540E34893B9525B8DF1A4C4837A98032A648FFCB9D70F2BEDEF388504BAB59191BA893389CC8BD3B0E446143086D1CDAE9D51CB13704C2CF7A61C679293939AAB6FC63F0B87
# openssl rsa -noout -modulus -in /config/httpd/conf/ssl.key/management.key
Modulus=B0E9B8F6A9084A134E5575E6374E4BCF30D56D7A2F728921AA4A3A433E0A349EED6455C9AFF2F5D151642D74073DB20BD8250E9DDCE06CA2CA9A3A4ADC55923396AE4F3EA612E4EBC66C2F82DEBEC27F87ED63E051A40FE3B7E33754EA65598467E7B0E81F13E5B508966F3A10B16E6E62A7EB181104C66AA52F013D3BD879A545D3A4C8A473D03B380625823E36D4EA3EAD9039A4BC7F01E9E3C6DF005B593AB5EAD62D59A9CCF7B2AFDC8F744D8BFACDB9B92DB471E52709186540E34893B9525B8DF1A4C4837A98032A648FFCB9D70F2BEDEF388504BAB59191BA893389CC8BD3B0E446143086D1CDAE9D51CB13704C2CF7A61C679293939AAB6FC63F0B87
If I run a "bigstart restart" AND "bigstart restart httpd" it suddenly works fine:
# bigstart restart
# bigstart status httpd
httpd (pid 20978) is running...
# bigstart restart httpd
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
I am guessing additional services needs to be restarted before httpd? What am I missing?
Kind regards,
Patrik
Solved! Go to Solution.
07-Aug-2022 01:55 - edited 08-Aug-2022 13:05
Final solution:
def set_management_cert(self, cert_name, key_name):
self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.crt/{cert_name}')
self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.key/{key_name}')
self.session.put(
f'https://{self.device}/mgmt/tm/sys/httpd',
json={
'sslCertfile': '/config/httpd/conf/ssl.crt/management.crt',
'sslCertkeyfile': '/config/httpd/conf/ssl.key/management.key'}
)
try:
logger.info('Restarting httpd')
self.run_bash_command('bigstart restart httpd; killall -9 httpd;bigstart restart httpd;')
except:
logger.info('Waiting for management interface to restart')
time.sleep(3)
httpd_config = self.get_httpd_config()
if os.path.basename(httpd_config['sslCertfile']) == cert_name \
and os.path.basename(httpd_config['sslCertkeyfile']) == key_name:
print('Certificate has been updated and the httpd interface is responding')
else:
raise Exception('Failed to update the certificate')
Thank you for the suggestions and tips @Dario_Garrido . Definitely helped me find the solution!
04-Aug-2022 12:02
Hello mate!
Looking forward to know more about your project. Are you planning a future article for this? 😊
Regarding your issue, never faced before, but I was wondering if tomcat daemon could be involved.
https://support.f5.com/csp/article/K03531511
04-Aug-2022 13:09
No article planned, but that was a good idea. Might write one if I can get it to work. 👍
04-Aug-2022 12:06
I've also found this. Funny article (don't use REST 😂)
https://support.f5.com/csp/article/K13292945
04-Aug-2022 13:04
Good ideas, I forgot about tomcat!
I'm afraid neither of the articles worked. Same issue.
Kind regards,
Patrik
04-Aug-2022 14:29
Same thing happens if I update the cert via REST, pause the script and run:
tmsh modify sys httpd ssl-certfile /config/httpd/conf/ssl.crt/management.crt ssl-certkeyfile /config/httpd/conf/ssl.key/management.key
httpd[4571]: [ssl:emerg] [pid 4571] (13)Permission denied: AH02201: Init: Can't open server certificate file /config/httpd/conf/ssl.crt/management.crt
04-Aug-2022 15:03
Came a bit further just now. Looks like SELinux might be making my life harder:
restorecon -RvF /var/config/rest/downloads/management.crt /config/httpd/conf/ssl.crt/
restorecon -RvF /var/config/rest/downloads/management.key /config/httpd/conf/ssl.key/
Then the permission error goes away but restarting the service does not work. Then the article @Dario_Garrido gave came in handy:
bigstart restart httpd
killall -9 httpd
bigstart restart httpd;
Will see if I can solve this further tomorrow. Good input with https://support.f5.com/csp/article/K13292945 @Dario_Garrido . Saved me quite some time!
Kind regards,
Patrik
07-Aug-2022 01:55 - edited 08-Aug-2022 13:05
Final solution:
def set_management_cert(self, cert_name, key_name):
self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.crt/{cert_name}')
self.run_bash_command(f'restorecon -RvF /config/httpd/conf/ssl.key/{key_name}')
self.session.put(
f'https://{self.device}/mgmt/tm/sys/httpd',
json={
'sslCertfile': '/config/httpd/conf/ssl.crt/management.crt',
'sslCertkeyfile': '/config/httpd/conf/ssl.key/management.key'}
)
try:
logger.info('Restarting httpd')
self.run_bash_command('bigstart restart httpd; killall -9 httpd;bigstart restart httpd;')
except:
logger.info('Waiting for management interface to restart')
time.sleep(3)
httpd_config = self.get_httpd_config()
if os.path.basename(httpd_config['sslCertfile']) == cert_name \
and os.path.basename(httpd_config['sslCertkeyfile']) == key_name:
print('Certificate has been updated and the httpd interface is responding')
else:
raise Exception('Failed to update the certificate')
Thank you for the suggestions and tips @Dario_Garrido . Definitely helped me find the solution!
07-Aug-2022 02:21
Great solution! Thanks mate 👏
08-Aug-2022 10:13
Nice @Patrik_Jonsson!! Curious what your helper functions look like (get_management_cert, run_bash_command)...
04-Aug-2022 13:10
@PeteWhite Do you have an ace up your sleeve?
07-Aug-2022 13:05
good job - i'd have suggested restorecon. Always a good idea to try it first with files, permissions etc