Upgrading from 14.1.0.3 to 15.0.1 Virtual

Hi,

​

Anyone got any ideas on this one?

​

Thanks..​

Try to forceload the mcpd by executing command below.

touch /service/mcpd/forceload

reboot

What else do you on the ltm log file?

Hi Markie,

Maybe a "tmsh load sys config verify" help the troubleshoot ?

Hi..

Thanks for the suggestions but both have been tried but nothing I can see that is really telling me what the issue is..

Here are some logs and the result of load config:

[root@F5BIGIP02:INOPERATIVE:] config # tmsh load sys config verify

Validating system configuration...

 /defaults/asm_base.conf

 /defaults/config_base.conf

 /defaults/ipfix_ie_base.conf

 /defaults/ipfix_ie_f5base.conf

 /defaults/low_profile_base.conf

 /defaults/low_security_base.conf

 /defaults/policy_base.conf

 /defaults/wam_base.conf

 /defaults/analytics_base.conf

 /defaults/apm_base.conf

 /defaults/apm_oauth_base.conf

 /defaults/apm_saml_base.conf

 /defaults/app_template_base.conf

 /defaults/classification_base.conf

 /var/libdata/dpi/conf/classification_update.conf

 /defaults/ips_base.conf

 /var/libdata/ips/ips_update.conf

 /defaults/daemon.conf

 /defaults/pem_base.conf

 /defaults/profile_base.conf

 /defaults/sandbox_base.conf

 /defaults/security_base.conf

 /defaults/urldb_base.conf

 /usr/share/monitors/base_monitors.conf

 /defaults/cipher.conf

 /defaults/ilx_base.conf

Validating configuration...

Loading schema version: 14.1.0.3

 /config/bigip_base.conf

 /config/bigip_user.conf

 /config/bigip.conf

 /config/bigip_script.conf

Loading schema version: 15.0.1

---------------------------------------------------

ep 29 14:38:20 F5BIGIP02.home.local warning tmsh[6183]: 01420013:4: [api-status-warning] pem/protocol/profile/gx is deprecated

Sep 29 14:38:20 F5BIGIP02.home.local warning tmsh[6183]: 01420013:4: Per-invocation log rate exceeded; throttling.

Sep 29 14:38:21 F5BIGIP02.home.local notice tmsh[6183]: 01420012:5: Loading schema version: 14.1.0.3

Sep 29 14:38:22 F5BIGIP02.home.local info tmsh[6183]: cli schema (14.1.0.3) has been loaded from schema data files.

Sep 29 14:38:23 F5BIGIP02.home.local notice tmsh[6183]: 01420012:5: Loading schema version: 15.0.1

Sep 29 14:38:23 F5BIGIP02.home.local notice mcpd[6366]: 01071038:5: Loading keys from the file.

Sep 29 14:38:23 F5BIGIP02.home.local notice mcpd[6366]: 01071038:5: Unit key hash from key header: 6f:f8:31:26:25:ea:4e:3a:1f:d3:1d:57:36:d5:13:f8:64:0c:00:b1

Sep 29 14:38:23 F5BIGIP02.home.local notice mcpd[6366]: 01071038:5: Unit key hash computed from read key:6f:f8:31:26:25:ea:4e:3a:1f:d3:1d:57:36:d5:13:f8:64:0c:00:b1

Sep 29 14:38:25 F5BIGIP02.home.local notice sod[6837]: 010c0063:5: Waiting for Mcpd without a response. Try again...

Sep 29 14:38:25 F5BIGIP02.home.local notice mcpd[6366]: 01390002:5: The size of the configuration DB has been extended by 5159040 bytes, now using a total of 26130560 bytes

Sep 29 14:38:26 F5BIGIP02.home.local notice mcpd[6366]: 01390002:5: The size of the configuration DB has been extended by 5159040 bytes, now using a total of 31289600 bytes

Sep 29 14:38:26 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: scheduling datasync_renew_conf because device was added: device_name /Common/F5BIGIP01.home.local is_self_device 0

Sep 29 14:38:26 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: scheduling datasync_renew_conf because device was added: device_name /Common/F5BIGIP02.home.local is_self_device 1

Sep 29 14:38:33 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: created datasync device-stats objects for tables [ CRYPT_ASM_DOSL7 CRYPT_FPM CS_ASM_DOSL7 CS_FPM STATIC_FPM RSA CAPTCHA CS_PERM_FPM ]

Sep 29 14:38:33 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: scheduling datasync_renew_conf because global device-group /Common/datasync-global-dg was updated during config load

Sep 29 14:38:33 F5BIGIP02.home.local notice mcpd[6366]: 010719e7:5: Virtual Address /Common/1.1.1.1 general status changed from BLUE to RED.

Sep 29 14:38:35 F5BIGIP02.home.local notice mcpd[6366]: 01390002:5: The size of the configuration DB has been extended by 5159040 bytes, now using a total of 36448640 bytes

Sep 29 14:38:36 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Resuming log processing at this invocation; held 31 messages.

Sep 29 14:38:36 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:38:36 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-457.0.iso_50620_1) dst (/shared/apm/images/epsec-1.0.0-457.0.iso)

Sep 29 14:38:38 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-489.0.iso_52663_1) dst (/shared/apm/images/epsec-1.0.0-489.0.iso)

Sep 29 14:38:41 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-718.0.iso_65092_1) dst (/shared/apm/images/epsec-1.0.0-718.0.iso)

Sep 29 14:38:55 F5BIGIP02.home.local notice sod[6837]: 010c0063:5: Waiting for Mcpd without a response. Try again...

Sep 29 14:38:56 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-753.0.iso_65225_1) dst (/shared/apm/images/epsec-1.0.0-753.0.iso)

Sep 29 14:39:11 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-787.0.iso_69633_1) dst (/shared/apm/images/epsec-1.0.0-787.0.iso)

Sep 29 14:39:25 F5BIGIP02.home.local notice sod[6837]: 010c0063:5: Waiting for Mcpd without a response. Try again...

Sep 29 14:39:25 F5BIGIP02.home.local notice sod[6837]: 010c0064:5: Waiting for Mcpd without a response. Happened 5 times. Reconnect.

Sep 29 14:39:29 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-804.0.iso_74218_1) dst (/shared/apm/images/epsec-1.0.0-804.0.iso)

Sep 29 14:39:45 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: copy_file: EPSEC in EspecFileObjMgr::copy_file - src (/config/filestore/files_d/Common_d/epsec_package_d/:Common:epsec-1.0.0-810.0.iso_74147_1) dst (/shared/apm/images/epsec-1.0.0-810.0.iso)

Sep 29 14:40:05 F5BIGIP02.home.local notice mcpd[6366]: 0107165d:5: init_system_epsec_software_status: EPSEC:: loading of Epsec software status passed

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 01070406:5: Removed publication with publisher id ha_table_publish

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: renewing datasync configuration

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: Initializing datasync configuration for provisioned modules [ none ]

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: created initial datasync global profiles for tables [ CRYPT_ASM_DOSL7 CRYPT_FPM CS_ASM_DOSL7 CS_FPM STATIC_FPM RSA CAPTCHA CS_PERM_FPM ] with activation_epoch 1569764406

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: created datasync device-stats objects for tables [ CRYPT_ASM_DOSL7 CRYPT_FPM CS_ASM_DOSL7 CS_FPM STATIC_FPM RSA CAPTCHA CS_PERM_FPM ]

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 010718ed:5: DATASYNC: Done initializing datasync configuration for provisioned modules [ none ]

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Configuration error: MCPProcessor::check_initialization: The input string contains invalid characters

Sep 29 14:40:06 F5BIGIP02.home.local err mcpd[6366]: 01070734:3: Per-invocation log rate exceeded; throttling.

Sep 29 14:40:06 F5BIGIP02.home.local notice mcpd[6366]: 01070404:5: Add a new Publication for publisherID ha_table_publish and filterType 4194304

Sep 29 14:40:36 F5BIGIP02.home.local notice sod[6837]: 010c0063:5: Waiting for Mcpd without a response. Try again...

Sep 29 14:41:01 F5BIGIP02.home.local warning diskmonitor[6228]: 011d0002:4: Cannot access the database because mcpd is not running.

Sep 29 14:41:06 F5BIGIP02.home.local notice sod[6837]: 010c0063:5: Waiting for Mcpd without a response. Try again...

Hi, have you try to start or retstart manually mcpd daemon ?

Yeah. Still does nothing..

​

"Waiting for Mcpd without a response. Try again..​"

May be too late to answer  , but I came across a very similar if not the same problem upgrading from 13.x to 15.0.1, hopefully this helps someone down the line.

Traced it to an original get_dossier error not long after bootup that looks like this:

err get_dossier[8484]: 01170005:3: /config/ssl/ssl.csr/f5_api_com.csr stat fails: No such file or directory.

This error precedes the symptomatic looping errors of:

• error mcpd Configuration Error: MCPProcessor::check_inilitization: The input string contains invalid characters
• Waiting for mcpd without a response

which misguides the search.

At the tail end of the solution for the original get_dossier error, it suggests verifying the license.

https://support.f5.com/csp/article/K70420769

Usually, after an upgrade, if a unit requires re-licensing, it would pop you back to the re-licensing screen on the gui after the initial boot in the new version, so I don't think it dawned upon me to do a license check.

sure enough, checking the license shows it to be inoperational

# tmsh show /sys license

Can't load license, may not be operational

apparently, this invalid license problem messes things so bad in this version, that even the gui can't load

so, now you that can't get your warm fuzzy gui to relicense, you have to follow :

https://support.f5.com/csp/article/K2595

to manually relicense.

Followed the steps, and waited for the system to sort itself out and it actually came back online.

 good luck!