Residential and Mobile Proxy Networks – The good and the not-so-good!
Keeping your privacy and identity under control in today's online world is critical, whether you're up to good or not-so-good things. That’s where residential and mobile proxies networks come in. These networks help hide your real IP address by making it look like your internet traffic is coming from regular people’s devices instead of data centers or well-known VPNs and proxies. These networks may resemble the TOR network, conceived initially to anonymize the internet using a decentralized network model to route traffic through volunteer-operated servers. Still, despite their similar nature, they have different architecture and drivers and are operated by private companies. Residential and Proxy Networks: A residential proxy network routes internet traffic through IP addresses assigned by the Internet Service Providers (ISPs) to homeowners. This makes traffic appear to be coming from a regular residential user when it hits its target. You can easily google the search term “residential proxy” and find that many companies are offering these services, allowing users to access geographically restricted content, perform web scraping without getting blocked by IP reputation systems, conduct competitive analysis without revealing their identity, and perform all sorts of cyberattacks, ranging from the reconnaissance phase up to the data exfiltration phase when the breach already has taken place. These services can also be leveraged for legitimate purposes like ad verification, market research, and SEO monitoring. Mobile Proxy Networks: Mobile proxy networks use IP addresses assigned to mobile devices by mobile carriers. These proxies provide an even higher level of legitimacy because mobile IPs rotate frequently and are associated with actual mobile devices or sometimes with regional CGNAT pools. This makes them particularly useful for tasks that require high anonymity and dynamic IP changes. Typically, the same companies that offer residential proxy services also have an offer for mobile proxy services. These are often used to test mobile apps and websites, manage social media accounts, bypass geographical restrictions on mobile content, and, let’s not forget, perform cyberattacks. How these networks operate Affiliation and Recruitment Programs Residential and mobile proxy companies often offer affiliate programs to incentivize developers to integrate their SDK into mobile apps, TV apps, browser extensions, VPN apps, etc. These programs allow developers to earn commissions or other benefits by integrating the SDK and becoming a network node to proxy traffic when remotely instructed by their “command-and-control” network. Figure 1 These affiliation programs can be a vital source of revenue for developers who still struggle to generate enough revenue from their applications. Figure 2 Some companies are stricter; others are not so much, but ultimately, it all depends on one’s ability to monitor what is being proxied by these SDKs to be able to prevent becoming part of a malicious Botnet, and this is a hard task to expect from a regular end-user. Below is a fragment from one of the SDK developer’s End-User Agreements. Figure 3 Traffic Flow Figure 4 Utilization for malicious activities: While residential and mobile proxies have legitimate uses, they are also increasingly used for cyberattacks. Here are some of the ways these networks are utilized for malicious activities: Web Scraping and Data Theft: Illegitimate Scraping: Today, with the AI hype, more than ever, DATA is GOLD and not only cybercriminals use residential mobile proxies to perform large-scale web scraping, extracting sensitive or proprietary information from websites without being detected or blocked. Credential Stuffing and Account Takeover: By blending their traffic using a mix of clean residential and mobile IPs and masking their identity, attackers can use stolen credentials to gain unauthorized access to user accounts across multiple platforms. Most importantly, they can validate the large dataset of credentials to ensure that when they sell it, a warranty is provided for the buyers. At the end of the day, Cybercriminals also need to keep their reputation, right? Carding: A very similar mechanism to Credential Stuffing applies to Carding, but here, cybercriminals can stealthily validate credit card numbers to make sure each one has not been flagged as compromised and is active for selling and being utilized by fraudsters. Gift Card Abuse: Fraudsters love Gift Cards because of their untraceable nature. Imagine combining that with the ability to brute-force numbers, validate and balance-check compromised ones. Distributed Denial of Service (DDoS) Attacks: Traffic Diversion: Residential and mobile proxies help in distributing attack traffic across numerous IP addresses, making it challenging for defenders to mitigate DDoS attacks effectively. Ad Fraud: Click Fraud: Attackers use these proxies to simulate legitimate clicks on ads, defrauding advertisers by generating fake traffic. Impression Fraud: By repeatedly loading advertisements through residential proxies, fraudsters can inflate the number of ad impressions, misleading advertisers about the reach and effectiveness of their ads. Bypassing Geo-blocks and Anti-fraud Mechanisms: Content Manipulation: Malicious actors use proxies to bypass geographical restrictions and access region-specific content or services. Avoiding Detection: Proxies help in evading anti-fraud systems designed to detect and block suspicious activities, thereby facilitating various forms of online fraud. Residential and mobile proxy networks provide a legitimate service for enhancing online privacy and enabling activities like market research and ad verification. However, their misuse for cyberattacks poses significant challenges for cybersecurity professionals. Understanding the dual-use nature of these technologies is essential for developing effective countermeasures and ensuring the internet remains a safe and secure environment. F5’s Bot and Fraud prevention solutions can distinguish between human-originated requests and software-originated requests by leveraging the ability to collect untamperable client-side signals. This unique capability is layered with our surveillance network, which tracks residential and mobile proxies using proprietary mechanisms. This offers our customers complete visibility and protection against malicious traffic originating from different sources, regardless of whether the attacker is blending their attacks using clean residential or mobile IP addresses.
Three things your proxy can’t do unless it’s a full-proxy
Proxies are one of the more interesting (in my no-doubt biased opinion) “devices” in the network. They’re the basis for caching, load balancing, app security, and even app acceleration services. They’re also a bridge between dev and ops and the network, being commonplace to all three groups and environments in most data center architectures. But not all proxies are built on the same architectural principles, which means not all proxies are created equal. A large number of proxies are half-proxies while others are full-proxies, and the differences between them are what mean the difference between what you can and cannot do with them. In fact, there are three very important things you can do with a full-proxy that you can’t do with a regular old proxy. Before we jump into those three things, let’s review the differences between them, shall we? Half-Proxy Half-proxy is a description of the way in which a proxy, reverse or forward, handles connections. Basically it’s describing the notion that the proxy only mediates connections on the client side. So it only proxies half the communication between the client and the app. The most important thing to recognize about a half-proxy is that it has only one network stack that it shares across both client and server. Full-Proxy By contrast, a full-proxy maintains two distinct network stacks – one on the client side, one of the app side – and fully proxies both sides, hence the name. While a full-proxy can be configured to act like a half-proxy, its value is in its typical configuration, which is to maintain discrete connections to both the client and the server. It is this dual-stack approach that enables a full-proxy to provide capabilities that a half-proxy with its single network stack simply cannot. The Three Things A full-proxy completely understands the protocols for which it proxies and is itself both an endpoint and an originator for those protocols and connections. This also means the full-proxy can have its own TCP connection behavior for each network stack such as buffering, retransmits, and TCP options. With a full-proxy each connection is unique; each can have its own TCP connection behavior. This means that a client connecting to the full-proxy device would likely have different connection behavior than the full-proxy might use for communicating with servers. Full-proxies can look at incoming requests and outbound responses and can manipulate both if the solution allows it. #1 Optimize client side and server side Because it can maintain separate network stacks and characteristics, a full-proxy can optimize each side for its unique needs. The TCP options needed to optimize for performance on the client side’s lower-speed, higher-latency network connection – particularly when mobile devices are being served – are almost certainly very different than those needed to optimize for performance on the server side’s high-speed, low latency data center network connection. A full-proxy can optimize both at the same time and thus provide the best performance possible in all situations. A half-proxy, with its single network stack, is forced to optimize for the average of its connections, which certainly means one side or the other is left with less than optimal performance. #2 Act as a protocol gateway Protocol gateways are an important tool in the architect’s toolbox particularly when transitioning from one version of an application protocol to another, like HTTP/1 to HTTP/2 or SPDY. Because a full proxy maintains those two unique connections, it can accept HTTP/2 on the client side, for example, but speak HTTP/1 to the server (app). That’s because a full-proxy terminates the client connection (the proxy is the server) and initiatives a different connection to the server (the proxy is the client). The protocol used on the client side doesn’t restrict the choice of protocols on the server side. Realistically, any protocol transition that makes sense (and even those that don’t) can be managed with a full-proxy. A programmable full-proxy ensures that even if its an uncommon (and thus not universally supported) that you can code up a gateway yourself without expending effort on reinventing the proxy-wheel. #3 Terminate SSL/TLS Technically this is a specialized case of a protocol gateway but the ascendancy of HTTP/S (and the urgency with which we are encouraged to deploy SSL Everywhere and Encrypt All The Things) makes me treat this as its own case. Basically terminating SSL/TLS is a critical capability in modern and emerging architectures because of the need to inspect and direct HTTP-based traffic (like REST API calls) based on information within the HTTP protocol that would otherwise be invisible thanks to encryption. The ability to terminate SSL/TLS means the proxy becomes the secure endpoint to which clients connect (and ultimately trust). Termination means the proxy is responsible for decrypting requests and encrypting responses and is thus able to “see” into the messages and use the data therein to make routing and load balancing decisions. So the next time you’re looking at a proxy, don’t forget to find out whether it’s a full proxy or not. Because without a full-proxy, you’re limiting your ability to really take advantage of its capabilities and reaping the benefits it can offer modern and emerging application architectures.
