F5MonitoringService Keeps Crashing

We are having this issue as well.

 

 

Windows 2008 x64 sp2

 

SCOM 2007 R2 RTM

 

F5 Management Pack is 1.2.0.579-x64

 

 

Configured to watch 6 BIG-IP LTMs

 

 

Log Name: F5 Monitoring Log

 

Source: F5 Events

 

Date: 7/28/2009 9:03:00 AM

 

Event ID: 201

 

Task Category: None

 

Level: Error

 

Keywords: Classic

 

User: N/A

 

Computer: crisscom01.us.aegon.com

 

Description:

 

Unhandled exception in DefaultDomain

 

 

Detailed Exception Information:

 

F5Networks.F5Exception: Unhandled exception in DefaultDomain ---> System.InvalidOperationException: Cannot modify key property 'TargetServicePort', that would change the object's identity.

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.SdkConnector.UpdateObject(MonitoringObject monitoringObj, EntityData entityChanges, ConfigData configData)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.SdkConnector.UpdateObject(Guid monitoringId, EntityData entityChanges, ConfigData configData)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter._UpdateDeviceConfig_Modify(SdkConnector connector, IPAddress deviceIP, ConfigChange configChange, DeviceConfigStats configStats, List`1 entitiesNeedingStateUpdates)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter.<>c__DisplayClass22.b__1e()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.<>c__DisplayClass6.b__5()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.CoreDoOperation[ReturnResult](GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrConnectionPool`1.CoreDoOperationWithExceptionMapping[ReturnResult](ManagedOpsMgrConnection`1 connection, GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.<>c__DisplayClass9`1.b__8()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ManagedConnectionBase`1.DoOperationWithRecovery[ReturnResult](GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.UseConnectionForOperation[ReturnResult](ManagedConnectionType& connection, Object connectionTag, GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.UseConnectionForOperation(ManagedConnectionType& connection, Object connectionTag, VoidVoidDelegate operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.LockedConnectionPoolReference`2.UseConnection(VoidVoidDelegate operation)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter.UpdateDeviceConfig(ConfigChange configChange, IPAddress deviceIP, Guid deviceId, ObjectsModified objectsModified, List`1 entitiesNeedingStateUpdates, List`1 entitiesAdded)

 

at F5Networks.ManagementPack.DataAccess.Configuration.ConfigManager`8.UpdateDeviceConfig(Device device, Queue`1 configurationPackets, Boolean saveRuleUpdates, ObjectsModified objectsModified, List`1 entitiesNeedingStateUpdates, List`1 entitiesAdded)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._UpdateDeviceDataConfiguration(Device device, DataReceivedEventArgs dataArgs)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._ConnectionDataReceived(ListeningDeviceConnectionCache connectionCache, DataReceivedEventArgs dataReceivedArgs)

 

at F5Networks.ManagementPack.ConnectionCache.ListeningDeviceConnectionCache.DoDataOperation(DeviceConnection deviceConnection, DataReceivedEventArgs args)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._ConnectionDataReceived(ListeningDeviceConnectionCache connectionCache, DataReceivedEventArgs dataReceivedArgs)

 

at F5Networks.ManagementPack.Services.DeviceMonitor.ParsedResponseProcessor.ProcessData(Queue`1 responses)

 

at F5Networks.ManagementPack.Services.DeviceMonitor.BatchedParsedResponseProcessor._FlushQueue(Object synchronousCall)

 

--- End of inner exception stack trace ---

 

Event Xml:

 

 

 

 

201

 

2

 

0

 

0x80000000000000

 

 

74

 

F5 Monitoring Log

 

crisscom01.us.aegon.com

 

 

 

 

Unhandled exception in DefaultDomain

 

 

Detailed Exception Information:

 

F5Networks.F5Exception: Unhandled exception in DefaultDomain ---> System.InvalidOperationException: Cannot modify key property 'TargetServicePort', that would change the object's identity.

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.SdkConnector.UpdateObject(MonitoringObject monitoringObj, EntityData entityChanges, ConfigData configData)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.SdkConnector.UpdateObject(Guid monitoringId, EntityData entityChanges, ConfigData configData)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter._UpdateDeviceConfig_Modify(SdkConnector connector, IPAddress deviceIP, ConfigChange configChange, DeviceConfigStats configStats, List`1 entitiesNeedingStateUpdates)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter.<>c__DisplayClass22.b__1e()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.<>c__DisplayClass6.b__5()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.CoreDoOperation[ReturnResult](GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrConnectionPool`1.CoreDoOperationWithExceptionMapping[ReturnResult](ManagedOpsMgrConnection`1 connection, GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.<>c__DisplayClass9`1.b__8()

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ManagedConnectionBase`1.DoOperationWithRecovery[ReturnResult](GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.UseConnectionForOperation[ReturnResult](ManagedConnectionType& connection, Object connectionTag, GenericVoidHandler`1 operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.ConnectionPoolBase`2.UseConnectionForOperation(ManagedConnectionType& connection, Object connectionTag, VoidVoidDelegate operation)

 

at F5Networks.ManagementPack.DataAccess.ManagedConnections.LockedConnectionPoolReference`2.UseConnection(VoidVoidDelegate operation)

 

at F5Networks.ManagementPack.DataAccess.OperationsManager.OpsMgrMonitoringAdapter.UpdateDeviceConfig(ConfigChange configChange, IPAddress deviceIP, Guid deviceId, ObjectsModified objectsModified, List`1 entitiesNeedingStateUpdates, List`1 entitiesAdded)

 

at F5Networks.ManagementPack.DataAccess.Configuration.ConfigManager`8.UpdateDeviceConfig(Device device, Queue`1 configurationPackets, Boolean saveRuleUpdates, ObjectsModified objectsModified, List`1 entitiesNeedingStateUpdates, List`1 entitiesAdded)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._UpdateDeviceDataConfiguration(Device device, DataReceivedEventArgs dataArgs)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._ConnectionDataReceived(ListeningDeviceConnectionCache connectionCache, DataReceivedEventArgs dataReceivedArgs)

 

at F5Networks.ManagementPack.ConnectionCache.ListeningDeviceConnectionCache.DoDataOperation(DeviceConnection deviceConnection, DataReceivedEventArgs args)

 

at F5Networks.ManagementPack.Services.DeviceMonitor._ConnectionDataReceived(ListeningDeviceConnectionCache connectionCache, DataReceivedEventArgs dataReceivedArgs)

 

at F5Networks.ManagementPack.Services.DeviceMonitor.ParsedResponseProcessor.ProcessData(Queue`1 responses)

 

at F5Networks.ManagementPack.Services.DeviceMonitor.BatchedParsedResponseProcessor._FlushQueue(Object synchronousCall)

 

--- End of inner exception stack trace ---

 

 

 

Hi, The only condition that could cause this error as far as I can tell would be an LTM pool member changing it's port. Since it is not possible to change a pool member's port via the configuration utility (you have to delete the pool member and recreate), I'm not sure how this could be happening.

 

 

It would really help if I could take a look at your F5 monitoring log. There are instructions here for turning on verbose logging:

 

 

http://devcentral.f5.com/wiki/default.aspx/MgmtPack/GeneralTroubleshooting.html

 

 

Please turn on verbose logging, then recreate the condition, then send the log to ManagementPack(at)f5.com or post it here.

 

 

Thanks, Joel Hendrickson

 

 

Those logs were sent in yesterday via email.

We have found the cause of this problem and put in a fix that will be in the next release, scheduled for the week of august 10th (2 weeks from now).

 

 

If a pool member with port 4444 is discovered, any state changes (such as enable/disable/up/down) to that pool member will cause the service to crash. This has to do with the way that Big-IP translates some ports when it notifies us of state updates (in this case, 4444="KRB524").

 

 

There is no workaround at this point besides restarting the service.

 

 

My apologies for the problems you are experiencing -- I have been told that it will be up in 1-2 hours so please check back soon. Thanks for your patience,

 

 

Joel

The release is now available (v 1.3.0.715)

 

 

Joel

Hi Joel,

 

Thanks for posting this new release, there appears to be an issue with the x86 version, I can not open the archive.

 

Regards,

 

Dean.

It looks like there was an issue with the zip files being corrupted for some people. We've rezipped and posted the files. Please let me know if they work for you.

 

 

Thanks!

 

 

Joel

That's fine now Joel, thanks for your help.

 

Regards,

 

Dean.