Node.js at 100K+ requests per second
Folks that work with Node.js are familiar with the limitations it has. Single threaded nature make Node highly dependent on single CPU performance, which seems to be the main limitation for throughput. Achieving 10K HTTP requests per second is not unheard of, and a barely noticeable breeze in the internet traffic world. Today’s sites are demanding high throughput and spinning new server instances can be wasteful. LineRate offers several ways to optimize your WebApp and resources for high throughput. LineRate is a high performance Application Proxy with full programmable access to HTTP data path in form of a Scripting API that utilizes Node.js. LineRate has all enterprise grade Load Balancer features such as multiple load balancing algorithms to choose from, connection and session persistence, and SSL offloading. LineRate optimizes available resources to achieve high throughput and 100K+ requests per second. The scripting API enables developers to reuse their Node.js code and offload simple processing (such as inspecting or rewriting theheaders or body of HTTP requests)onto a single LineRate instance. For each HTTP request received, Node.js script code can inspect and modify the request before forwarding it to the web server. Compute intensive tasks such as authentication or virus scanning could be offloaded to external services in a non-blocking fashion. To get started visit linerate.f5.com.
Microservices and HTTP/2
It's all about that architecture. There's a lot of things we do to improve the performance of web and mobile applications. We use caching. We use compression. We offload security (SSL and TLS) to a proxy with greater compute capacity. We apply image optimization and minification to content. We do all that because performance is king. Failure to perform can be, for many businesses, equivalent to an outage with increased abandonment rates and angry customers taking to the Internet to express their extreme displeasure. The recently official HTTP/2 specification takes performance very seriously, and introduced a variety of key components designed specifically to address the need for speed. One of these was to base the newest version of the Internet's lingua franca on SPDY. One of the impacts of this decision is that connections between the client (whether tethered or mobile) and the app (whether in the cloud or on big-iron) are limited to just one. One TCP connection per app. That's a huge divergence from HTTP/1 where it was typical to open 2, 4 or 6 TCP connections per site in order to take advantage of broadband. And it worked for the most part because, well, broadband. So it wouldn't be a surprise if someone interprets that ONE connection per app limitation to be a negative in terms of app performance. There are, of course, a number of changes in the way HTTP/2 communicates over that single connection that ultimately should counteract any potential negative impact on performance from the reduction in TCP connections. The elimination of the overhead of multiple DNS lookups (not insignificant, by the way) as well as TCP-related impacts from slow start and session setup as well as a more forgiving exchange of frames under the covers is certainly a boon in terms of application performance. The ability to just push multiple responses to the client without having to play the HTTP acknowledgement game is significant in that it eliminates one of the biggest performance inhibitors of the web: latency arising from too many round trips. We've (as in the corporate We) seen gains of 2-3 times the performance of HTTP/1 with HTTP/2 during testing. And we aren't alone; there's plenty of performance testing going on out there, on the Internets, that are showing similar improvements. Which is why it's important (very important) that we not undo all the gains of HTTP/2 with an architecture that mimics the behavior (and performance) of HTTP/1. Domain Sharding and Microservices Before we jump into microservices, we should review domain sharding because the concept is important when we look at how microservices are actually consumed and delivered from an HTTP point of view. Scalability patterns (i.e. architectures) include the notion of Y-axis scale which is a sharding-based pattern. That is, it creates individual scalability domains (or clusters, if you prefer) based on some identifiable characteristic in the request. User identification (often extricated from an HTTP cookie) and URL are commonly used information upon which to shard requests and distribute them to achieve greater scalability. An incarnation of the Y-axis scaling pattern is domain sharding. Domain sharding, for the uninitiated, is the practice of distributing content to a variety of different host names within a domain. This technique was (and probably still is) very common to overcome connection limitations imposed by HTTP/1 and its supporting browsers. You can see evidence of domain sharding when a web site uses images.example.com and scripts.example.com and static.example.com to optimize page or application load time. Connection limitations were by host (origin server), not domain, so this technique was invaluable in achieving greater parallelization of data transfers that made it appear, at least, that pages were loading more quickly. Which made everyone happy. Until mobile came along. Then we suddenly began to realize the detrimental impact of introducing all that extra latency (every connection requires a DNS lookup, a TCP handshake, and suffers the performance impacts of TCP slow start) on a device with much more limited processing (and network) capability. I'm not going to detail the impact; if you want to read about it in more detail I recommend reading some material from Steve Souder and Tom Daly or Mobify on the subject. Suffice to say, domain sharding has an impact on mobile performance, and it is rarely a positive one. You might think, well, HTTP/2 is coming and all that's behind us now. Except it isn't. Microservice architectures in theory, if not in practice, are ultimately a sharding-based application architecture that, if we're not careful, can translate into a domain sharding-based network architecture that ultimately negates any of the performance gains realized by adopting HTTP/2. That means the architectural approach you (that's you, ops) adopt to delivering microservices can have a profound impact on the performance of applications composed from those services. The danger is not that each service will be its on (isolated and localized) "domain", because that's the whole point of microservices in the first place. The danger is that those isolated domains will be presented to the outside world as individual, isolated domains, each requiring their own personal, private connection by clients. Even if we assume there are load balancing services in front of each service (a good assumption at this point) that still means direct connections between the client and each of the services used by the client application because the load balancing service acts as a virtual service, but does not eliminate the isolation. Each one is still its own "domain" in the sense that it requires a separate, dedicated TCP connection. This is essentially the same thing as domain sharding as each host requires its own IP address to which the client can connect, and its behavior is counterproductive to HTTP/2*. What we need to do to continue the benefits of a single, optimized TCP connection while being able to shard the back end is to architect a different solution in the "big black box" that is the network. To be precise, we need to take advantage of the advanced capabilities of a proxy-based load balancing service rather than a simple load balancer. An HTTP/2 Enabling Network Architecture for Microservices That means we need to enable a single connection between the client and the server and then utilize capabilities like Y-axis sharding (content switching, L7 load balancing, etc...) in "the network" to maintain the performance benefits of HTTP/2 to the client while enabling all the operational and development benefits of a microservices architecture. What we can do is insert a layer 7 load balancer between the client and the local microservice load balancers. The connection on the client side maintains a single connection in the manner specified (and preferred) by HTTP/2 and requires only a single DNS lookup, one TCP session start up, and incurs the penalties from TCP slow start only once. On the service side, the layer 7 load balancer also maintains persistent connections to the local, domain load balancing services which also reduces the impact of session management on performance. Each of the local, domain load balancing services can be optimized to best distribute requests for each service. Each maintains its own algorithm and monitoring configurations which are unique to the service to ensure optimal performance. This architecture is only minimally different from the default, but the insertion of a layer 7 load balancer capable of routing application requests based on a variety of HTTP variables (such as the cookies used for persistence or to extract user IDs or the unique verb or noun associated with a service from the URL of a RESTful API call) results in a network architecture that closely maintains the intention of HTTP/2 without requiring significant changes to a microservice based application architecture. Essentially, we're combining X- and Y-axis scalability patterns to architect a collaborative operational architecture capable of scaling and supporting microservices without compromising on the technical aspects of HTTP/2 that were introduced to improve performance, particularly for mobile applications. Technically speaking we're still doing sharding, but we're doing it inside the network and without breaking the one TCP connection per app specified by HTTP/2. Which means you get the best of both worlds - performance and efficiency. Why DevOps Matters The impact of new architectures - like microservices - on the network and the resources (infrastructure) that deliver those services is not always evident to developers or even ops. That's one of the reasons DevOps as a cultural force within IT is critical; because it engenders a breaking down of the isolated silos between ops groups that exist (all four of them) and enables greater collaboration that leads to more efficient deployment, yes, but also more efficient implementations. Implementations that don't necessarily cause performance problems that require disruptive modification to applications or services. Collaboration in the design and architectural phases will go along way towards improving not only the efficacy of the deployment pipeline but the performance and efficiency of applications across the entire operational spectrum. * It's not good for HTTP/1, either, as in this scenario there is essentially no difference** between HTTP/1 and HTTP/2. ** In terms of network impact. HTTP/2 still receives benefits from its native header compression and other performance benefits.
Node.js HTTP Message Body: Transfer-Encoding and Content-Length in LineRate lrs/virtualServer Module
Summary: (refer to AskF5 solution article 15884) In the LineRate scripting environment we consult the Content-Length header of HTTP messages entering the system (see the LineRate Developer Scripting Guide under the topic "Writing Past the Content-Length"). Therefore when performing transformations to an HTTP message body, be it a request or response, we must ensure that if a Content-Length header is present in the original message we send accurate information reflecting our changes in the message we transmit to the end host. Full Story: When I first began investigating what node.js was I stumbled upon a video presentation by Ryan Dahl (node's creator) from 2011. If you fast-forward to about 14:00 in the video, Ryan is talking about the node HTTP module. One of the features Ryan mentions is that the HTTP module's “server” object provides HTTP/1.1 responses. Along with the Keep-Alive header, the node HTTP server automatically "chunks" the response; that is, the server will automatically apply the "Transfer-Encoding: chunked" header to a response. Ryan explains that this enables node to automatically handle variable length responses. This also enables you to call the write() method multiple times on an active HTTP response object without prior knowledge of the total length of the response you want to send. This is a great feature when you know all of your clients will be fully HTTP/1.1 compliant. The LineRate application proxy’s scripting API provides for manipulating the HTTP data path, and when acting as part of a full-proxy architecture the data path requires the management of both HTTP server functionality as well as an HTTP client functionality. This enables users of LineRate to make transformations to HTTP responses that originate from other servers ("real-servers" in LROS configuration terms). Since LineRate stands between the origin servers and the HTTP clients we must take care to preserve application behavior, regardless of the version of HTTP the server provides. Herein lies one of the differences in the design of the LineRate lrs/virtualServer module and the node HTTP module's server objects. Bringing the previous paragraphs together: when you apply content transformations to HTTP message bodies using the lrs/virtualServer module, you must be conscious of the original message's framing method; that is, does the message use "Transfer-Encoding: chunked", or does it use a "Content-Length:" header value to delineate boundaries of HTTP message bodies? The LineRate Scripting Developer Guide addresses this issue, and it is briefly outlined in a recent AskF5 solution article: Overview of Node.js auto-chunking behavior on LineRate systems. In order to better understand this behavior I built a small node module that appends some chosen content to the end of an HTTP response: This script makes use of the Node.js 'Modules' API, where it adds the object name “ResponseAppender” to the "exports" namespace. In order to use this script as a module on your LineRate system simply copy this code as a file to the directory /home/linerate/data/scripting/, and then 'require' the module from within an in-line script where we call 'new' on the call to the exported function, passing three parameters: the virtual server we want to use the module with as the first parameter, an arbitrary string as the second parameter, and a boolean as the third parameter ('true' means we correct the value of the Content-Length to the final response if present, 'false' means we leave the headers exactly as the real-server provided them). Here is an example of a partial LROS configuration with a virtual server and inline script that makes use of this module (assuming the above module file is named 'response_appender.js'): With the above configuration objects (and assuming "test-vip" and "some-server" refer to appropriate corresponding configuration objects) we can easily toggle the "FIX_HEADER" value from the LineRate Manager web GUI, or by using the CLI from the configuration context with the command: LROS(config-script:appender)# source edit vim ## or LROS(config-script:appender)# source edit emacs Once this scenario is configured we should be able to observe the following: CASE #1: FIX_HEADER == false, and the real-server provides a Content-Length header Here we would expect to see no change in a browser when we receive the response from the virtual server. Because the Content-Length header is preserved form the original response the browser stops short of reading the appended content. However, if we use a tool like tcpdump or wireshark to capture the network stream, we can actually see the appended string being sent over the wire. CASE #2: FIX_HEADER == true, and the real-server provides a Content-Length header This time because FIX-HEADER is true we actually calculate a new Content-Length value that includes the byte count of the appended string. Now if we make a request to the test-vip with a browser we expect to the the pop-up text "SURPRISE!" because now the browser knows to read the additional bytes of the appended string. CASE #3: the real-server provides the response with Transfer-Encoding: chunked or TE: chunked: In this case nothing changes. We can observe that LineRate behaves exactly like the node HTTP server objects, where any content written to the response stream is properly framed as an HTTP/1.1 chunk. We expect the response to result in a browser displaying the "SURPRISE!" pop-up message. Now that you know how it works, download your own copy of LROS, get a free tier license and try it yourself!
Enforcing HSTS (HTTP Strict Transport Security) in LineRate
HTTP Strict Transport Securityis a policy between your customer's browsers and your servers to increase security. It forces the browser to always use HTTPS when connecting to your site. The server or proxy needs to set the Strict-Transport-Security header. If the client connects sometime in the future and isn't offered a valid SSL cert, it should show an error to the user. Also, if the client is somehow directed to a plaintext URL at your site, for instance from the address bar, it should turn it into an HTTPS URL before connecting. This policy can prevent simple attacks where an attacker is positioned in the network temporarily between a client and your site. If they can connect to the client plaintext, and the user isn't carefully checking for the green browser lock symbol, they can act as a man in the middle and read all the data flowing between clients and servers. SeeMoxie Marlinspike's presentation. HSTS also saves you if you're worried about some piece of infrastructure accidentally emitting a non-secure link. Modern sites are a hybrid of client-side and server-side code; browser facing content and APIs; core applications and peripheral systems like analytics, support, or advertising. What are the odds that one of these systems will someday use a URL that's not HTTPS? With HSTS, you can prevent attacks that take advantage of this accident. To make HSTS effective in this case, you should place it in a proxyoutsideof all of these systems. First I'll show a simple script to enable HSTS on LineRate; next I'll show an enhancement to detect the plaintext URLs that are leaking on clients that don't obey HSTS. Simple: Add HSTS Header to Responses To enable HSTS for your site, simply catch responses and add the "Strict-Transport-Security" header: var vsm = require('lrs/virtualServerModule'); // Set this to the amount of time a browser should obey HSTS for your site var maxAge = 365*24*3600; // One year, in seconds function setHsts(servReq, servRes, cliReq) { cliReq.on('response', function (cliRes) { cliRes.bindHeaders(servRes); servRes.setHeader('Strict-Transport-Security', 'max-age=' + maxAge); cliRes.fastPipe(servRes); }); cliReq(); } vsm.on('exist', 'yourVirtualServerName', function (vs) { vs.on('request', setHsts); }); For any requests that come through a virtual server named yourVirtualServerName , LineRate will add the header to the response. In this example, the maxAge variable means that the browser should enforce HTTPS for your site for the next year from when it saw this response header. As long as users visit your site at least once a year, their browser will enforce that all URLs are HTTPS. Advanced: Detect plaintext leaks and HSTS issues In a more advanced script, you can also detect requests to URLs that aren't HTTPS. Note that HSTS requires the browser to enforce the policy; some browsers don't support it (Internet Explorer does not as of this writing; Safari didn't until Mavericks). For those users, you'll still need to detect any plaintext "leaks". Or, maybe you're a belt-and-suspenders kind of person, and you want to tell your servers to add HSTS, but also detect a failure to do so in your proxy. In these cases, the script below will detect the problem, collect information, record it, and workaround by redirecting to the HTTPS URL. var vsm = require('lrs/virtualServerModule'); var util = require('util'); // Set this to the domain name to redirect to var yourDomain = 'www.yoursite.com'; // Set this to the amount of time a browser should obey HSTS for your site var maxAge = 365*24*3600; // One year, in seconds var stsValParser = /max-age=([0-9]+);?/; function detectAndFixHsts(servReq, servRes, cliReq) { cliReq.on('response', function (cliRes) { cliRes.bindHeaders(servRes); var stsVal = cliRes.headers['strict-transport-security']; var stsMatch = stsVal ? stsValParser.match(stsVal) : []; if (stsMatch.length !== 1) { // Strict-Transport-Security header not valid. console.log('[WARNING] Strict-Transport-Security header not set ' + 'properly for URL %s. Value: %s. Request Headers: %s' + ', response headers: %s', servReq.url, stsVal, util.inspect(servReq.headers), util.inspect(cliRes.headers)); servRes.setHeader('Strict-Transport-Security', 'max-age=' + maxAge); } cliRes.fastPipe(servRes); }); cliReq(); } function redirectToHttps(servReq, servRes, cliReq) { // This is attached to the non-SSL VIP. var referer = servReq.headers['referer']; if (referer === undefined || (referer.lastIndexOf('http://' + yourDomain, 0) == -1)) { // Referred from another site on the net; not a leak in your site. } else { // Leaked a plaintext URL or user is using a deprecated client console.log('[WARNING] Client requested non-HTTPS URL %s. ' + 'User-Agent: %s, headers: %s', servReq.url, servReq.headers['user-agent'], util.inspect(servReq.headers)); } var httpsUrl = 'https://' + yourDomain + servReq.url; var redirectBody = '<html><head><title> ' + httpsUrl + ' Moved</title>' + '</head><body><p>This page has moved to <a href="' + httpsUrl + '">' + httpsUrl + '</a></p></body></html>'; servRes.writeHead(302, { 'Location': httpsUrl, 'Content-Type' : 'text/html', 'Content-Length' : redirectBody.length }); servRes.end(redirectBody); } vsm.on('exist', 'yourHttpsVirtualServer', function (vs) { vs.on('request', detectAndFixHsts); }); vsm.on('exist', 'yourPlainHttpVirtualServer', function (vs) { vs.on('request', redirectToHttps); }); Note that logging every non-HTTPS request can limit performance and fill up disk. Alternatives include throttled logging (try googling "npm log throttling"), or recording URLs to a database, or keeping a cache of URLs that we've already reported. If you're interested, let me know in the comments and I can cover some of these topics in future blog posts.Unable to grant license key: Error 51092
Hi, after pasting the dossier to https://activate.f5.com/license/license.do I get this reply: Unable to grant license key: Error 51092, This license has already been activated on a different unit. Please contact technical support for assistance I had to reinstall the loadbalancer, since I got the underlying linux in such a state that it would not boot. I had naturally tried the license registration, but the phone home function was not working at that point. how can we fix this licensing issue? ilariCloud bursting, the hybrid cloud, and why cloud-agnostic load balancers matter
Cloud Bursting and the Hybrid Cloud When researching cloud bursting, there are many directions Google may take you. Perhaps you come across services for airplanes that attempt to turn cloudy wedding days into memorable events. Perhaps you'd rather opt for a service that helps your IT organization avoid rainy days. Enter cloud bursting ... yes, the one involving computers and networks instead of airplanes. Cloud bursting is a term that has been around in the tech realm for quite a few years. It, in essence, is the ability to allocate resources across various public and private clouds as an organization's needs change. These needs could be economic drivers such as Cloud 2 having lower cost than Cloud 1, or perhaps capacity drivers where additional resources are needed during business hours to handle traffic. For intelligent applications, other interesting things are possible with cloud bursting where, for example, demand in a geographical region suddenly needs capacity that is not local to the primary, private cloud. Here, one can spin up resources to locally serve the demand and provide a better user experience.Nathan Pearcesummarizes some of the aspects of cloud bursting inthis minute long video, which is a great resource to remind oneself of some of the nuances of this architecture. While Cloud Bursting is a term that is generally accepted by the industry as an "on-demand capacity burst,"Lori MacVittiepoints out that this architectural solution eventually leads to aHybrid Cloudwhere multiple compute centers are employed to serve demand among both private-based resources are and public-based resources, or clouds, all the time. The primary driver for this: practically speaking,there are limitations around how fast data that is critical to one's application (think databases, for example) can be replicated across the internet to different data centers.Thus, the promises of "on-demand" cloud bursting scenarios may be short lived, eventually leaning in favor of multiple "always-on compute capacity centers"as loads increase for a given application.In any case, it is important to understand thatthat multiple locations, across multiple clouds will ultimately be serving application content in the not-too-distant future. An example hybrid cloud architecture where services are deployed across multiple clouds. The "application stack" remains the same, using LineRate in each cloud to balance the local application, while a BIG-IP Local Traffic Manager balances application requests across all of clouds. Advantages of cloud-agnostic Load Balancing As one might conclude from the Cloud Bursting and Hybrid Cloud discussion above, having multiple clouds running an application creates a need for user requests to be distributed among the resources and for automated systems to be able to control application access and flow. In order to provide the best control over how one's application behaves, it is optimal to use a load balancer to serve requests. No DNS or network routing changes need to be made and clients continue using the application as they always did as resources come online or go offline; many times, too, these load balancers offer advanced functionality alongside the load balancing service that provide additional value to the application. Having a load balancer that operates the same way no matter where it is deployed becomes important when resources are distributed among many locations. Understanding expectations around configuration, management, reporting, and behavior of a system limits issues for application deployments and discrepancies between how one platform behaves versus another. With a load balancer like F5's LineRate product line, anyone can programmatically manage the servers providing an application to users. Leveraging this programatic control, application providers have an easy way spin up and down capacity in any arbitrary cloud, retain a familiar yet powerful feature-set for their load balancer, ultimately redistribute resources for an application, and provide a seamless experience back to the user. No matter where the load balancer deployment is, LineRate can work hand-in-hand with any web service provider, whether considered a cloud or not. Your data, and perhaps more importantly cost-centers, are no longer locked down to one vendor or one location. With the right application logic paired with LineRate Precision's scripting engine, an application can dynamically react to take advantage of market pricing or general capacity needs. Consider the following scenarios where cloud-agnostic load balancer have advantages over vendor-specific ones: Economic Drivers Time-dependent instance pricing Spot instances with much lower cost becoming available at night Example: my startup's billing system can take advantage in better pricing per unit of work in the public cloud at night versus the private datacenter Multiple vendor instance pricing Cloud 2 just dropped their high-memory instance pricing lower than Cloud 1's Example: Useful for your workload during normal business hours; My application's primary workload is migrated to Cloud 2 with a simple config change Competition Having multiple cloud deployments simultaneously increases competition, and thusyour organization's negotiated pricing contracts become more attractiveover time Computational Drivers Traffic Spikes Someone in marketing just tweeted about our new product. All of a sudden, the web servers that traditionally handled all the loads thrown at them just fine are gettingslashdottedby people all around North America placing orders. Instead of having humans react to the load and spin up new instances to handle the load - or even worse: doing nothing - your LineRate system and application worked hand-in-hand to spin up a few instances in Microsoft Azure's Texas location and a few more in Amazon's Virginia region. This helps you distribute requests from geographically diverse locations: your existing datacenter in Oregon, the central US Microsoft Cloud, and the east-coast based Amazon Cloud. Orders continue to pour in without any system downtime, or worse: lost customers. Compute Orchestration A mission-critical application in your organization's private cloud unexpectedly needs extra computer power, but needs to stay internal for compliance reasons. Fortunately, your application can spin up public cloud instances and migrate traffic out of the private datacenter without affecting any users or data integrity. Your LineRate instance reaches out to Amazon to boot instances and migrate important data. More importantly, application developers and system administrators don't even realize the application has migrated since everything behaves exactly the same in the cloud location. Once the cloud systems boot, alerts are made to F5's LTM and LineRate instances that migrate traffic to the new servers, allowing the mission-critical app to compute away. You just saved the day! The benefit to having a cloud-agnostic load balancing solution for connecting users with an organization's applications not only provides a unified user experience, but provides powerful, unified way of controlling the application for its administrators as well. If all of a sudden an application needs to be moved from, say, aprivate datacenter with a 100 Mbps connection to a public cloud with a GigE connection, this can easily be done without having to relearn a new load balancing solution. F5's LineRate product is available for bare-metal deployments on x86 hardware, virtual machine deployments, and has recently deployed anAmazon Machine Image (AMI). All of these deployment types leverage the same familiar, powerful tools that LineRate offers:lightweight and scalable load balancing, modern management through its intuitive GUI or the industry-standard CLI, and automated control via itscomprehensive REST API.LineRate Point Load Balancerprovides hardened, enterprise-grade load balancing and availability services whereasLineRate Precision Load Balanceradds powerful Node.js programmability, enabling developers and DevOps teams to leveragethousands of Node.js modulesto easily create custom controlsfor application network traffic. Learn about some of LineRate'sadvanced scripting and functionalityhere, ortry it out for freeto see if LineRate is the right cloud-agnostic load balancing solution for your organization.Authentication behind F5 LineRate issue.
Hello everyone, We are testing F5 Linerate as a LB solution for 2 of our core applications and found when we authenticate on the application directly the AD users are able to login without problems. However, when we try to authenticate to the application server using the URL with the hostname in F5 I got our application asking for the Active Directory credentials (same as we use login directly) but it keeps asking for credentials and fails after three times with an IIS 401 error which does not occur when authenticate directly to the application server without F5. Any idea what are we missing to be able to authenticate to the application behind the F5 LineRate? Thanks in advance for any help, Elio Diaz
