I'd group the IP:ports by application. So if each of the six ports on one server are running a different application, I'd create one pool per app. Then add all of the corresponding ports serving the same application on the second server to the same pools as the apps on the first server. In other words:
server 1 (1.1.1.1)
port 80(app1)
port 81(app2)
port 82(app3)
server 1 (1.1.1.1)
port 80(app3)
port 81(app2)
port 82(app1)
app1_pool
1.1.1.1:80
1.1.1.2:82
app2_pool
1.1.1.1:81
1.1.1.2:81
app3_pool
1.1.1.1:82
1.1.1.2:80
The default LTM behavior for load balancing between pool members is to select a different pool member if a new connection comes in and a previously used server is down. The 'action on service down' setting is only used for new connections. For a TCP application, it's generally best to set it to reject so the client immediately gets a TCP reset and knows to retry the connection.
Aaron