Does the application fail immediately upon adding the monitor to the pool, or does it take a while? If it's after a while, what does the netstat -a output on the server show? Can you capture a tcpdump of the traffic to see if there is something out of the ordinary happening in the communication?
A standard TCP monitor should include a three-way TCP handshake and a four way close. Here's an example of a monitor checking an HTTP pool member:
192.168.101.10.54405 > 192.168.101.224.http: S 1414792763:1414792763(0) win 5840 (DF)
192.168.101.224.http > 192.168.101.10.54405: S 1733432186:1733432186(0) ack 1414792764 win 5840 (DF)
192.168.101.10.54405 > 192.168.101.224.http: . ack 1 win 5840 (DF)
192.168.101.10.54405 > 192.168.101.224.http: F 1:1(0) ack 1 win 5840 (DF)
192.168.101.224.http > 192.168.101.10.54405: F 1:1(0) ack 2 win 92 (DF)
192.168.101.10.54405 > 192.168.101.224.http: . ack 2 win 5840 (DF)
Aaron