Forum Discussion
jesusangel
Altostratus
AltostratusDNS: reply from unexpected source
Hi,
Firstly, I must say that I am a complete newbie when it comes to BIG-IP products. On the other hand, the load balancer was configured by experts, so I am pretty confident that they made a reasonably good job.
I have two DNS servers (DNS1 and DNS2) and a BIG-IP F5 14.1.4.6 load balancer. Both DNS servers' default gateway is the F5. When a DNS client asks the DNS service in the F5, the load balancer sends the request to one of the DNS servers keeping the client's IP. Then, on receiving the reply from the DNS server, the F5 sends the reply to the client using its own IP (with SNAT). This way, the DNS client only talks to the F5. On the other hand, when a DNS client ask one of the DNS servers directly, the DNS server sends the reply to the default gateway (the F5) and the packet is routed to its destination without any change.
Nevertheless, every now and then I am facing replies from unexpected sources. For instance, sometimes the client asks DNS1 but it gets the reply from the F5. Thus, I get messages like this:
;; reply from unexpected source: bigip#53, expected dns1#53
It looks like that, on receiving the reply from DNS1, the F5 replaces the packet's source IP (SNAT) with its own ip.
Example:
Right behaviour:
PC -> DNS query to F5 service -> F5 -> sends a query to DNS1 or DNS2 keeping the PC's source IP -> DNS1 replies to the F5 (its default gateway) -> F5 replaces source IP (SNAT) and sends the reply to the PC -> the PC receives the reply from the service it asked to.
PC -> sends a DNS query to DNS1 -> DNS1 replies to the F5 (its default gateway) -> the F5 somehow knows that the reply was sent directly to DNS1 and forwards the reply to the PC keeping DNS1's IP address -> the PC receives the reply from the server it asked to (DNS1).
Wrong behaviour
PC -> sends a DNS query to DNS1 -> DNS1 replies to the F5 (its default gateway) -> the F5 replaces the packet's source IP (DNS1) with its own IP and forwards the reply to the PC -> the PC receives the reply from an unexpected server (F5).
Can you give me a hand to solve this? Maybe you can just give me a hint to start looking for the solution.
Thanks in advance.
Regards,
- Kevin_Stewart
Employee
Maybe the first thing to clear up is that the BIG-IP does not SNAT to the client. SNAT is a server side function where the BIG-IP changes the packet source address to its own to aid in return routing. Let's use an example.
- Client is 1.1.1.1
- BIG-IP DNS VIP is 2.2.2.2
- BIG-IP server side self-IP is 3.3.3.3
- DNS server is 3.3.3.4
- DNS server is configured to GW back through 3.3.3.3
- There's no SNAT on the BIG-IP VIP
So then packet flow through the BIG-IP VIP looks like this:
- Client 1.1.1.1 reaches the BIG-IP VIP 2.2.2.2
- BIG-IP VIP load balances to 3.3.3.4
- The DNS server sees traffic from 1.1.1.1 (no SNAT) and routes its response back through 3.3.3.3 (self-IP)
- The client sees a DNS response from 2.2.2.2 (BIG-IP VIP)
If the client goes directly to the DNS server:
- Client 1.1.1.1 reaches the DNS server (3.3.3.4) directly
- The DNS server sees traffic from 1.1.1.1 and routes its response back through 3.3.3.3 (self-IP)
- The client sees a DNS response from 2.2.2.2 (BIG-IP VIP)
So it's not that the BIG-IP is SNAT'ing the traffic to the client, but that the client is intentionally talking to the BIG-IP to get a DNS response in one flow, and incorrectly getting a response from the BIG-IP VIP (instead of the DNS server directly) in the other flow. The easiest option here would just be to enable SNAT on the BIG-IP (to the DNS servers), and remove the BIG-IP gateway route. That way DNS through the BIG-IP will look like it's coming from the BIG-IP and will return that way. DNS directly from the client will go around the BIG-IP.
jesusangelDear Kevin,
I do appreciate your answer. However, I would prefer not to enable SNAT on the BIG-IP to the DNS servers as I need to log in the DNS servers the client's IP. Should I enable SNAT con the BIG-IP (to the DNS servers), then all DNS queries comming from the BIG-IP system would have the BIG-IP server side self-IP (3.3.3.3).
Moreover, the thing is that when a client goes directly to the DNS server, the vast majority of the responses arrive to the client with the DNS server IP (3.3.3.4). There are just a handful of them that arrive with the BIG-IP VIP (2.2.2.2). That is what I can not wrap my head around to. I thought that BIG-IP somehow tracks queries that come throught it and does SNAT to the client when it is appropiate.
Regards,
Jesús Ángel.
- Kevin_Stewart
So to be clear, some clients will get DNS by making a request to a BIG-IP VIP (2.2.2.2), and other clients will go directly to the DNS server (3.3.3.4), by going around the BIG-IP? And as the DNS server uses a BIG-IP self-IP as its gateway, you would expect all return traffic to return through the BIG-IP?
Is there more than one DNS server, and is there any chance that one of those servers has a static route applied for some client IP subnet that doesn't pass back through the BIG-IP?
