Load Balancing not working with least session

We are having least session load balancing setup with SSL persistence.SSL is offloaded at server. It works fine for a while and then all the requests are directed to only one server in a pool of two ...