I normally see this happen when an iRule has some faulty logic in it that creates an endless loop.
I would suggest using a tool like HTTPWatch (http://www.httpwatch.com/), to see exactly what it is doing to know where to look in the iRule. You could also use logging statements in the iRule to see where the logic is looping.
You could also post any iRules that are applied to the Virtual Server in the forum and we could take a look.
If am incorrect, please add more detail on the Virtual Server that you are having problems with.