csyncd Command failed: cmd=rsync rc=10?

We are at 11.4.1 build 625.0

 

/var/log/ltm is getting about 100 of these log entries every minute:

 

Sep 19 18:11:29 slot2/CH_F5_AA01 warning csyncd[5765]: 013b0005:4: Command failed: cmd=rsync -qaI --timeout=15 rsync://127.3.0.1/csync/config/e: /config/e:, rc=10

 

Obviously it will eventually consume /var, but my immediate concern is that some of my pool members are acting erratically. Most notably the monitors say everything is fine, but getting http sessions established to the pools stopped for a server that we rebooted.

 

Does the return code of 10 give me any additional information?

 

top - 18:15:17 up 17 days, 1:42, 1 user, load average: 0.10, 0.07, 0.07 Tasks: 322 total, 2 running, 320 sleeping, 0 stopped, 0 zombie Cpu(s): 13.8%us, 1.6%sy, 0.3%ni, 84.2%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 32952224k total, 32756428k used, 195796k free, 245828k buffers Swap: 5242872k total, 0k used, 5242872k free, 1537472k cached

 

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

 

9286 root RT 0 29.1g 177m 128m S 101.5 0.6 10713:21 tmm

 

6832 root 20 0 239m 61m 4444 S 3.0 0.2 47:55.76 java

 

7263 root 25 5 19632 7540 2592 S 2.7 0.0 597:52.35 merged

 

6985 root 20 0 40976 31m 18m S 2.3 0.1 916:36.36 bcm56xxd

 

6409 root 20 0 29016 15m 9756 S 2.0 0.0 414:38.34 csyncd

 

6991 root 20 0 141m 75m 14m S 0.7 0.2 301:52.87 mcpd

 

6093 root 20 0 15616 3848 3136 S 0.3 0.0 0:54.26 stpd

 

6828 root 20 0 24776 7440 3332 S 0.3 0.0 2:44.68 alertd

 

6986 root 20 0 31704 17m 3252 S 0.3 0.1 105:39.68 statsd

 

1 root 20 0 2172 664 568 S 0.0 0.0 0:01.20 init

 

I did see atleast one support ticket that may be related but did not indicate impact to traffic: https://support.f5.com/kb/en-us/solutions/public/15000/900/sol15957.html

 

Thanks! Jay