TCP Connection Reset between VIP and Client

[hmian_178112]

Topology:

Pulse Authentication Servers <--> F5 <--> FORTIGATE <--> JUNOS RTR <--> Internet <--> Client/users.

Background:

Clients on the internet attempting to reach a VPN app VIP (load-balances 3 Pulse VPN servers). Nodes + Pool + Vips are UP. I manage/configure all the devices you see. No SNAT/NAT: due to client requirement to see all IP's on Fortigate logs. Client rejected solution to use F5 logging services.

I successfully assisted another colleague in building this exact setup at a different location. For some odd reason, not working at the 2nd location I'm building it on. Compared config scripts. NO differences. Very puzzled. Concerned about FW rules on Fortigates so I am in the middle of comparing the Fortigate FW rule configurations at both locations, but don't let that persuade you.

Results:

Client can't reach VIP using pulse VPN client on client machine. Client also failed to telnet to VIP on port 443, traffic is reaching F5 --> leads to connection resets. I can successfully telnet to pool members on port 443 from F5 route domain 1.

Configuration:

ltm monitor https MONITOR_HTTPS_EXAMPLE {
    adaptive disabled
    cipherlist DEFAULT:+SHA:+3DES:+kEDH
    compatibility enabled
    defaults-from https
    destination *:*
    interval 10
    ip-dscp 0
    recv "200 OK"
    recv-disable 500
    send "GET /dana-na/healthcheck/healthcheck.cgi HTTP/1.1\r\nHost: newark.dxc.technology\r\nConnection: Close\r\n\r\n"
    time-until-up 0
    timeout 31
}
ltm node NODE1 {
    address 30.1.1.138%1
}
ltm node NODE2 {
    address 30.1.1.139%1
}
ltm node NODE3 {
    address 30.1.1.140%1
}
ltm persistence source-addr source-addr-2700 {
    app-service none
    match-across-services enabled
    mirror enabled
    timeout 2700
}
ltm pool POOL_EXAMPLE {
    allow-nat no
    allow-snat no
    load-balancing-mode least-connections-member
    members {
        NODE1:https {
            address 30.1.1.138%1
            session monitor-enabled
            state up
        }
        NODE2:https {
            address 30.1.1.139%1
            session monitor-enabled
            state up
        }
        NODE3:https {
            address 30.1.1.140%1
            session monitor-enabled
            state up
        }
    }
    monitor MONITOR_HTTPS_EXAMPLE
}
ltm virtual VIP_EXAMPLE {
    destination 40.1.1.30%1:https
    ip-protocol tcp
    mask 255.255.255.255
    persist {
        source-addr-2700 {
            default yes
        }
    }
    pool POOL_EXAMPLE
    profiles {
        fastL4 { }
    }
    source 0.0.0.0%1/0
    translate-address enabled
    translate-port enabled
    vs-index 4
}
net route-domain SE_EXT_DMZ {
    description SE_EXT_DMZ
    id 1
    vlans {
        SERVER-VLAN-EXAMPLE
        VIP-VLAN-EXAMPLE
    }
}
net self HA_IP {
    address 10.10.0.1/29
    traffic-group traffic-group-local-only
    vlan HA_VLAN
}
net self VIP-VLAN-EXAMPLE_NON_FLOATER {
    address 40.1.1.5%1/24
    allow-service {
        default
    }
    traffic-group traffic-group-local-only
    vlan VIP-VLAN-EXAMPLE
}
net self SERVER-VLAN-EXAMPLE_NON {
    address 30.1.1.133%1/25
    allow-service {
        default
    }
    traffic-group traffic-group-local-only
    vlan SERVER-VLAN-EXAMPLE
}
net self SERVER-VLAN-EXAMPLE {
    address 30.1.1.132%1/25
    allow-service {
        default
    }
    floating enabled
    traffic-group traffic-group-1
    unit 1
    vlan SERVER-VLAN-EXAMPLE
}
net self VIP-VLAN-EXAMPLE {
    address 40.1.1.4%1/24
    allow-service {
        default
    }
    floating enabled
    traffic-group traffic-group-1
    unit 1
    vlan VIP-VLAN-EXAMPLE
}
net trunk F5_VLAN_TRUNK {
    bandwidth 20000
    cfg-mbr-count 2
    id 0
    interfaces {
        5.0
        6.0
    }
    lacp enabled
    mac-address f4:15:63:ee:7c:09
    media 10000
    working-mbr-count 2
}
net vlan VIP-VLAN-EXAMPLE {
    if-index 848
    interfaces {
        F5_VLAN_TRUNK {
            tagged
        }
    }
    tag 461
}
net vlan SERVER-VLAN-EXAMPLE {
    if-index 864
    interfaces {
        F5_VLAN_TRUNK {
            tagged
        }
    }
    tag 457
}

TCPDUMP

See next post

[AceDawg1]

What are the Pulse/VPN servers using as their default gateway? They should be using the F5 if SNAT is not in use to avoid asymmetric routing.

[AceDawg1]

I would do the following then test:

1. Change the VIP to use SNAT. Test.
2. If it works, reverse the VIP configuration in step 1 (e.g. no SNAT)
3. Disable all pool members in POOL_EXAMPLE except for 30.1.1.138
4. Change the gateway for 30.1.1.138 to 30.1.1.132. Test.

[AceDawg1]

In addition, do you have a VIP configured for port 4500? Noticed in the traffic capture that there is traffic going to TCP port 4500:

18:01:03.427463 IP (tos 0x0, ttl 64, id 5134, offset 0, flags [DF], proto TCP (6), length 60)
30.1.1.133.51704 > 30.1.1.139.4500: Flags [S], cksum 0x609e (incorrect -> 0x57ee), seq 2071882144, win 14600, options [mss 1460,sackOK,TS val 4213873347 ecr 0,nop,wscale 7], length 0 out slot1/tmm0 lis=

[hmian_178112]

THank you AceDawg, your first answer was on point and resolved the issue.

Comment made 4 hours ago by AceDawg 202What are the Pulse/VPN servers using as their default gateway? They should be using the F5 if SNAT is not in use to avoid asymmetric routing.

[AceDawg1]

Excellent! I'll post said response as an answer to your question.

Could you "Accept" for me please?

[AceDawg1]

What are the Pulse/VPN servers using as their default gateway? They should be using the F5 if SNAT is not in use to avoid asymmetric routing.

I would do the following then test:

1. Change the VIP to use SNAT. Test.
2. If it works, reverse the VIP configuration in step 1 (e.g. no SNAT)
3. Disable all pool members in POOL_EXAMPLE except for 30.1.1.138
4. Change the gateway for 30.1.1.138 to 30.1.1.132. Test.

[hmian_178112]

This was it, I had to change the Gateway for the POOL MEMBERS to the F5 SELF IP rather than the Fortigate Firewall upstream because we are not using SNAT.

[AceDawg_204810 · Accepted solution]

What are the Pulse/VPN servers using as their default gateway? They should be using the F5 if SNAT is not in use to avoid asymmetric routing.

I would do the following then test:

1. Change the VIP to use SNAT. Test.
2. If it works, reverse the VIP configuration in step 1 (e.g. no SNAT)
3. Disable all pool members in POOL_EXAMPLE except for 30.1.1.138
4. Change the gateway for 30.1.1.138 to 30.1.1.132. Test.

[hmian_178112]

This was it, I had to change the Gateway for the POOL MEMBERS to the F5 SELF IP rather than the Fortigate Firewall upstream because we are not using SNAT.

[hmian_178112]

It was the first response. Not the one you posted -->

I'll accept once you post the first response you sent (below)

Comment made 5 hours ago by AceDawg 204 What are the Pulse/VPN servers using as their default gateway? They should be using the F5 if SNAT is not in use to avoid asymmetric routing.

[AceDawg1]

Mea culpa. I added both answers/responses as the second provides a quick procedure on how things should be configured.

Many thanks.