Licensing Wizard Error - FIPS

The problem appears to be with the way the cryptographic services are set up on in your environment. And the issue could be local to the SCOM server or could be controlled by a domain wide security policy. The error that you get is basically stating that the encryption algorithm attempted on the client side for communicating with the F5 Licensing Server is not standards based (Federal Information Processing Standards / FIPS compliant).

 

 

The F5 Management Pack Licensing Wizard (which is the client in the licensing scenario), attempts to create an MD5 hash of a local dossier file, created during the licensing/setup process, which is then sent out to the F5 Licensing Server for storing it with the license key. When the MD5 hash is being attempted by the Licensing Wizard, you get the FIPS compliance error, reported by the underlying .NET Framework libraries (on your local system). This problem occurs because the MD5 algorithm is not FIPS compliant.

 

 

We can try to troubleshoot this over a remote session (using GoToMeeting), or if you provide us more information about the error and your system settings, we can try to pinpoint the possible cause and suggest a fix / workaround. Most probably the workaround would be to to DISABLE (temporarily) the FIPS compliant encryption algorithm requirement on your system.

 

 

Please check / provide the following information (you can send this information directly to to managementpack(at)f5(dot)com:

 

 

- send us the setup.log file, in the %Program Files%\F5 Networks\Management Pack\log folder.

 

- run the SystemInformation.ps1 diagnostic script and send us the output (see this article: http://devcentral.f5.com/wiki/default.aspx/MgmtPack/GeneralTroubleshooting.html)

 

- check the following registry keys (if present and their values):

 

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\FipsAlgorithmPolicy\Enabled (DWORD value): what is the value?

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\fipsalgorithmpolicy (DWORD value): what is the value?

 

 

(You should have either one or the other of these keys (depending on the Windows Server OS version).

 

 

- check the if the FIPS compliance policy in your local security policy: gpedit > Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options > System Cryptography: Use FIPS compliant algorithms for encryption, hashing and signing.

 

 

My personal suggestion would be to disable (temproarily the FIPS compliance policy, while you attempt to license the F5 product.

 

 

Let me know your thoughts.

 

Julian

 

 

 

I am available for remote troubleshooting if needed.

 

 

Info requested:

 

 

- check the following registry keys (if present and their values):

 

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\fipsalgorithmpolicy (DWORD value): what is the value? Value is 1

 

 

 

- check the if the FIPS compliance policy in your local security policy: gpedit > Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options > System Cryptography: Use FIPS compliant algorithms for encryption, hashing and signing.

 

This is set to Enabled

 

 

- send us the setup.log file, in the %Program Files%\F5 Networks\Management Pack\log folder.

 

This is available for review

 

 

- run the SystemInformation.ps1 diagnostic script and send us the output (see this article: This is available for review

Thanks for the quick update!

 

 

I would suggest disabling the FIPS compliance in your local security policy settings, if this is a valid option for you. At least temporarily, until you're done with the licensing. After you do this, make sure the related registry key value turns to 0. (I'm not sure if you'll need to restart the system, with that change. It would prompt you anyway, if you'll have to). Then open up a command prompt and run the licensing wizard with the following command (in the %Program Files%\F5 Networks\Management Pack folder):

 

 

f5mpgui /UL

 

 

This will run the F5 MP Licensing Wizard for updating the license.

 

Let me know how it goes, or if disabling the FIPS-compliance is not an option for you.

 

 

Julian

 

 

 

 

 

I tried the work around you listed to disable FIPS setting in the registry by turning it to 0. This worked and allowed the license process to complete successfully.

 

 

I then turned the registry key back to 1 to enable FIPS.

 

 

Checked the F5 Monitoring Service windows service and it was disabled and stopped. Attempted to start and go an error 1 Incorrect Function. Checked the Windows Application log and saw the same FIPS error listed during service startup. I then turned the registry entry back to 0 for FIPS and the service started.

 

 

It appears that in order for the management pack installation to function I have to have the FIPS setting disabled but this will not be feasible and is set by GPO.

I agree, the FIPS compliance check should stay OFF even after the licensing wizard is done, because with every start / restart of the F5 Monitoring Service, there is a license validity check performed by the F5 Monitoring Service, which uses the same security API calls (into MD5CryptoServiceProvider) that are causing the failure.

 

 

I haven’t thought about this in the first instance, when I was suggesting the temporary solution. I’m sorry.

 

 

So, if disabling FIPS in not an option for you, we need to find another way to get around this problem. Our possibilities are vey limited, as the F5 Licensing Server accepts only MD5 hashes for the license dossier files. And the MD5 encryption algorithm is not FIPS compliant.

 

 

The F5 Licensing Server accepts only MD5 hashes for dossier files. Such a license dossier file is crafted on your system, through calls into the .NET Framework, by calling into the local MD5CryptoServiceProvider supplier of encrypted hashes. When your local security policy detects this call, it will deny access to the hashing request.

 

 

Everything’s happening locally on your system, while the error occurs.

 

 

The workaround if any, would be to make your system less demanding on allowing security hashes to be created through the MD5 cryptographic provider. Which again, would be the least resistant way of solving this issue, opposed to re-designing the F5 Licensing Server to accept FIPS-compliant hashes, which I agree should be eventually addressed. Currently the F5 Licensing Server handles licensing requests across the entire F5 product line, not only the F5 Management Pack, and I’m sure this is a known issue (or limitation) of the service, and if needed could be escalated to a different group.

 

 

Coming back to our problem, I’ll try to find a less intrusive workaround compared to disabling FIPS. Hopefully one that would be supported by Microsoft and would be in harmony with your options. I’ll work on this and will keep you updated.

 

 

And thanks for your patience. I’ve been through similar frustrations and pains you’re having. I’ll do my best to unblock you.

 

 

julian

 

Ok Rob.

Here’s something that we can try. I’ve tested this in my environment and apparently works. I’m going to try to do a more involved testing but the change is not intrusive and you can give it a try as well. We’re going to do just an ISOLATED change, by forcing the runtime configuration of the F5 Monitoring Service, EXCLUSIVELY, to ignore the FIPS policy. All of the other system processes would still follow the FIPS GPO.

Here’re the steps (and I assume you’ve already completed the licensing wizard successfully, by temporarily disabling FIPS):

1. Make sure FIPS is enabled / ON (according to your GPO settings).

2. Go to Services Control Manager and stop the F5 Monitoring Service;

3. Go to %Program Files%\F5 Networks\Management Pack and edit the f5mpsvc.exe.config file. Scroll down to the bottom of the file, and BEFORE the closing tag, insert the following XML snippet:

4. Save the f5mpsvc.exe.config file

5. Restart the F5 Monitoring Service.

6. Check for errors.

Let me know how it goes.

Thank you.

julian