Windows 2012 Direct Access and F5

I have Windows 2012 with Direct Access installed on a stand alone server using a single NIC with static IPv6 and IPv4 addresses. The F5 is passing https to the IPv4 address and all is well, clients can connect externally.

However, when I enable load balancing with another Direct Access server and use the option "external load balancer" clients can no longer connect. During the NLB setup its asks for a DIP (dedicated IPv4) which I enter and it uses the original IPv4 IP as the new VIP, so no changes are needed on the VIP, the NLB cluster is created successfully but clients are in a constant "connecting" state to the DA server and the VIP in the pool for Direct Access is now down (not passing hhts 443). Any ideas? Thanks.