Load balancing terminal servers w/o Session Directory

We are having trouble with load balancing 2 terminal servers without Session Directory enabled. User will disconnect. When they log on again the big ip will not always send them to the same server so ...