The Disadvantages of DSR (Direct Server Return)
I read a very nice blog post yesterday discussing some of the traditional pros and cons of load-balancing configurations. The author comes to the conclusion that if you can use direct server return, you should. I agree with the author's list of pros and cons; DSR is the least intrusive method of deploying a load-balancer in terms of network configuration. But there are quite a few disadvantages missing from the author's list. Author's List of Disadvantages of DSR The disadvantages of Direct Routing are: Backend server must respond to both its own IP (for health checks) and the virtual IP (for load balanced traffic) Port translation or cookie insertion cannot be implemented. The backend server must not reply to ARP requests for the VIP (otherwise it will steal all the traffic from the load balancer) Prior to Windows Server 2008 some odd routing behavior could occur in In some situations either the application or the operating system cannot be modified to utilse Direct Routing. Some additional disadvantages: Protocol sanitization can't be performed. This means vulnerabilities introduced due to manipulation of lax enforcement of RFCs and protocol specifications can't be addressed. Application acceleration can't be applied. Even the simplest of acceleration techniques, e.g. compression, can't be applied because the traffic is bypassing the load-balancer (a.k.a. application delivery controller). Implementing caching solutions become more complex. With a DSR configuration the routing that makes it so easy to implement requires that caching solutions be deployed elsewhere, such as via WCCP on the router. This requires additional configuration and changes to the routing infrastructure, and introduces another point of failure as well as an additional hop, increasing latency. Error/Exception/SOAP fault handling can't be implemented. In order to address failures in applications such as missing files (404) and SOAP Faults (500) it is necessary for the load-balancer to inspect outbound messages. Using a DSR configuration this ability is lost, which means errors are passed directly back to the user without the ability to retry a request, write an entry in the log, or notify an administrator. Data Leak Prevention can't be accomplished. Without the ability to inspect outbound messages, you can't prevent sensitive data (SSN, credit card numbers) from leaving the building. Connection Optimization functionality is lost. TCP multiplexing can't be accomplished in a DSR configuration because it relies on separating client connections from server connections. This reduces the efficiency of your servers and minimizes the value added to your network by a load balancer. There are more disadvantages than you're likely willing to read, so I'll stop there. Suffice to say that the problem with the suggestion to use DSR whenever possible is that if you're an application-aware network administrator you know that most of the time, DSR isn't the right solution because it restricts the ability of the load-balancer (application delivery controller) to perform additional functions that improve the security, performance, and availability of the applications it is delivering. DSR is well-suited, and always has been, to UDP-based streaming applications such as audio and video delivered via RTSP. However, in the increasingly sensitive environment that is application infrastructure, it is necessary to do more than just "load balancing" to improve the performance and reliability of applications. Additional application delivery techniques are an integral component to a well-performing, efficient application infrastructure. DSR may be easier to implement and, in some cases, may be the right solution. But in most cases, it's going to leave you simply serving applications, instead of delivering them. Just because you can, doesn't mean you should.
Like Cars on a Highway.
Every once in a while, as the number of people following me grows (thank you, each and every one), I like to revisit something that is fundamental to the high-tech industry but is often overlooked or not given the attention it deserves. This is one of those times, and the many-faceted nature of any application infrastructure is the topic. While much has changed since I last touched on this topic, much has not, leaving us in an odd inflection point. When referring to movies that involve a lot of CGI, my oldest son called it “the valley of expectations”, that point where you know what you’d like to see and you’re so very close to it, but the current offerings fall flat. He specifically said that the Final Fantasy movie was just such a production. The movie came so close to realism that it was disappointing because you could still tell the characters were all animations. I thought it was insightful, but still enjoyed the movie. It is common to use the “weakest link in the chain” analogy whenever we discuss hardware, because you have parts sold by several vendors that include parts manufactured by several more vendors, making the entire infrastructure start to sound like the “weakest link” problem. Whether you’re discussing individual servers and their performance bottlenecks (which vary from year to year, depending upon what was most recently improved upon), or network infrastructures, which vary with a wide variety of factors including that server and its bottlenecks. I think a better analogy is a busy freeway. My reasoning is simple, you have to worry about the manufacture and operation of each vehicle (device) on the road, the road (wire) itself, interchanges, road conditions, and toll booths. There is a lot going on in your infrastructure, and “weakest link in the chain” is not a detailed enough comparison. In fact, if you’re of a mathematical bent, then the performance of your overall architecture could be summarized by the following equation: Where n is the number of infrastructure elements required for the application to function correctly and deliver information to the end user. From databases to Internet connections to client bandwidth, it’s all jumbled up in there. Even this equation isn’t perfect, simply because some performance degradation is so bad that it drags down the entire system, and other issues are not obvious until the worst offender is fixed. This is the case in the iterative improvement of servers… Today the memory is the bottleneck, once it is fixed, then the next bottleneck is disk, once it is improved, the next bottleneck is network I/O… on and on it goes, and with each iteration we get faster overall servers. And interestingly enough, security is very much the same equation, with the caveat that a subset of infrastructure elements is likely to be looked at for security, just because not everything is exposed to the outside world – for example, the database only need be considered if you allow users to enter data into forms that will power a DB query directly. So what is my point? well simply put, when you are budgeting, items that impact more than one element – from a security or performance perspective – or more than one application, should be prioritized over things that are specific to one element or one application. The goal of improving the overall architecture should trump the needs of individual pieces or applications, because IT – indeed, the business – is built upon the overall application delivery architecture, not just a single application. Even though one application may indeed be more relevant to the business (I can’t imagine that eBay has any application more important than their web presence, for example, since it is their revenue generation tool), overall improvements will help that application and your other applications. Of course you should fix those terribly glaring issues with either of these topics that are slowing the entire system down or compromising overall security, but you should also consider solutions that will net you more than a single-item fix. Yes, I think an advanced ADC product like F5’s BIG-IP is one of these multi-solution products, but it goes well beyond F5 into areas like SSDs for database caches and such. So keep it in mind. Sometimes the solution to making application X faster or more secure is to make the entire infrastructure faster or more secure. And if you look at it right, availability fits into this space too. Pretty easily in fact.Layer 4 vs Layer 7 DoS Attack
Not all DoS (Denial of Service) attacks are the same. While the end result is to consume as much - hopefully all - of a server or site's resources such that legitimate users are denied service (hence the name) there is a subtle difference in how these attacks are perpetrated that makes one easier to stop than the other. SYN Flood A Layer 4 DoS attack is often referred to as a SYN flood. It works at the transport protocol (TCP) layer. A TCP connection is established in what is known as a 3-way handshake. The client sends a SYN packet, the server responds with a SYN ACK, and the client responds to that with an ACK. After the "three-way handshake" is complete, the TCP connection is considered established. It is as this point that applications begin sending data using a Layer 7 or application layer protocol, such as HTTP. A SYN flood uses the inherent patience of the TCP stack to overwhelm a server by sending a flood of SYN packets and then ignoring the SYN ACKs returned by the server. This causes the server to use up resources waiting a configured amount of time for the anticipated ACK that should come from a legitimate client. Because web and application servers are limited in the number of concurrent TCP connections they can have open, if an attacker sends enough SYN packets to a server it can easily chew through the allowed number of TCP connections, thus preventing legitimate requests from being answered by the server. SYN floods are fairly easy for proxy-based application delivery and security products to detect. Because they proxy connections for the servers, and are generally hardware-based with a much higher TCP connection limit, the proxy-based solution can handle the high volume of connections without becoming overwhelmed. Because the proxy-based solution is usually terminating the TCP connection (i.e. it is the "endpoint" of the connection) it will not pass the connection to the server until it has completed the 3-way handshake. Thus, a SYN flood is stopped at the proxy and legitimate connections are passed on to the server with alacrity. The attackers are generally stopped from flooding the network through the use of SYN cookies. SYN cookies utilize cryptographic hashing and are therefore computationally expensive, making it desirable to allow a proxy/delivery solution with hardware accelerated cryptographic capabilities handle this type of security measure. Servers can implement SYN cookies, but the additional burden placed on the server alleviates much of the gains achieved by preventing SYN floods and often results in available, but unacceptably slow performing servers and sites. HTTP GET DoS A Layer 7 DoS attack is a different beast and it's more difficult to detect. A Layer 7 DoS attack is often perpetrated through the use of HTTP GET. This means that the 3-way TCP handshake has been completed, thus fooling devices and solutions which are only examining layer 4 and TCP communications. The attacker looks like a legitimate connection, and is therefore passed on to the web or application server. At that point the attacker begins requesting large numbers of files/objects using HTTP GET. They are generally legitimate requests, there are just a lot of them. So many, in fact, that the server quickly becomes focused on responding to those requests and has a hard time responding to new, legitimate requests. When rate-limiting was used to stop this type of attack, the bad guys moved to using a distributed system of bots (zombies) to ensure that the requests (attack) was coming from myriad IP addresses and was therefore not only more difficult to detect, but more difficult to stop. The attacker uses malware and trojans to deposit a bot on servers and clients, and then remotely includes them in his attack by instructing the bots to request a list of objects from a specific site or server. The attacker might not use bots, but instead might gather enough evil friends to launch an attack against a site that has annoyed them for some reason. Layer 7 DoS attacks are more difficult to detect because the TCP connection is valid and so are the requests. The trick is to realize when there are multiple clients requesting large numbers of objects at the same time and to recognize that it is, in fact, an attack. This is tricky because there may very well be legitimate requests mixed in with the attack, which means a "deny all" philosophy will result in the very situation the attackers are trying to force: a denial of service. Defending against Layer 7 DoS attacks usually involves some sort of rate-shaping algorithm that watches clients and ensures that they request no more than a configurable number of objects per time period, usually measured in seconds or minutes. If the client requests more than the configurable number, the client's IP address is blacklisted for a specified time period and subsequent requests are denied until the address has been freed from the blacklist. Because this can still affect legitimate users, layer 7 firewall (application firewall) vendors are working on ways to get smarter about stopping layer 7 DoS attacks without affecting legitimate clients. It is a subtle dance and requires a bit more understanding of the application and its flow, but if implemented correctly it can improve the ability of such devices to detect and prevent layer 7 DoS attacks from reaching web and application servers and taking a site down. The goal of deploying an application firewall or proxy-based application delivery solution is to ensure the fast and secure delivery of an application. By preventing both layer 4 and layer 7 DoS attacks, such solutions allow servers to continue serving up applications without a degradation in performance caused by dealing with layer 4 or layer 7 attacks.They’re Called Black Boxes Not Invisible Boxes
Infrastructure can be a black box only if its knobs and buttons are accessible I spent hours at Interop yesterday listening to folks talk about “infrastructure.” It’s a hot topic, to be sure, especially as it relates to cloud computing. After all, it’s a keyword in “Infrastructure as a Service.” The problem is that when most of people say “infrastructure” it appears what they really mean is “server” and that just isn’t accurate. If you haven’t been a data center lately there is a whole lot of other “stuff” that falls under the infrastructure moniker in a data center that isn’t a server. You might also have a firewall, anti-virus scanning solutions, a web application firewall, a Load balancer, WAN optimization solutions, identity management stores, routers, switches, storage arrays, a storage network, an application delivery network, and other networky-type devices. Oh there’s more than that but I can’t very well just list every possible solution that falls under the “infrastructure” umbrella or we’d never get to the point. In information technology and on the Internet, infrastructure is the physical hardware used to interconnect computers and users. Infrastructure includes the transmission media, including telephone lines, cable television lines, and satellites and antennas, and also the routers, aggregators, repeaters, and other devices that control transmission paths. Infrastructure also includes the software used to send, receive, and manage the signals that are transmitted. In some usages, infrastructure refers to interconnecting hardware and software and not to computers and other devices that are interconnected. However, to some information technology users, infrastructure is viewed as everything that supports the flow and processing of information. -- TechTarget definition of “infrastructure” The reason this is important to remember is that people continue to put forth the notion that cloud should be a “black box” with regards to infrastructure. Now in a general sense I agree with that sentiment but if – and only if – there is a mechanism to manage the resources and services provided by that “black boxed” infrastructure. For example, “servers” are infrastructure and today are very “black box” but every IaaS (Infrastructure as a Service) provider offers the means by which those resources can be managed and controlled by the customer. The hardware is the black box, not the software. The hardware becomes little more than a service. That needs to – nay, must extend to – the rest of the infrastructure. You know, the network infrastructure that is ultimately responsibly for delivering the applications that are being deployed on that black-box server infrastructure. The devices and services that interconnect users and applications. It simply isn’t enough to wave a hand at the network infrastructure and say “it doesn’t matter” because as a matter of fact it certainly does matter.Service Virtualization Helps Localize Impact of Elastic Scalability
Service virtualization is the opposite of – and complementary implementation to – server virtualization. One of the biggest challenges with any implementation of elastic scalability as it relates to virtualization and cloud computing is managing that scalability at run-time and at design (configuration) time. The goal is to transparently scale out some service – network or application – in such a way as to eliminate the operational disruption often associated with scaling up (and down) efforts. Service virtualization allows virtually any service to be transparently scaled out with no negative impact to the service and, perhaps more importantly, to the applications and other services which rely upon that service.I Can Has UR .htaccess File
Notice that isn’t a question, it’s a statement of fact Twitter is having a bad month. After it was blamed, albeit incorrectly, for a breach leading to the disclosure of both personal and corporate information via Google’s GMail and Apps, its apparent willingness to allow anyone and everyone access to a .htaccess file ostensibly protecting search.twitter.com made the rounds via, ironically, Twitter. This vulnerability at first glance appears fairly innocuous, until you realize just how much information can be placed in an .htaccess file that could have been exposed by this technical configuration faux pas. Included in the .htaccess file is a number of URI rewrites, which give an interesting view of the underlying file system hierarchy Twitter is using, as well as a (rather) lengthy list of IP addresses denied access. All in all, not that exciting, because many of the juicy bits that could be configured via .htaccess for any given website are not done so in this easily accessible .htaccess file. Some things you can do with .htaccess, in case you aren’t familiar: Create default error document Enable SSI via htaccess Deny users by IP Change your default directory page Redirects Prevent hotlinking of your images Prevent directory listing .htaccess is a very versatile little file, capable of handling all sorts of security and application delivery tasks. Now what’s interesting is that the .htaccess file is in the root directory and should not be accessible. Apache configuration files are fairly straight forward, and there are plethora examples of how to prevent .htaccess – and its wealth of information – from being viewed by clients. Obfuscation, of course, is one possibility, as Apache’s httpd.conf allows you to specify the name of the access file with a simple directive: AccessFileName .htaccess It is a simple enough thing to change the name of the file, thus making it more difficult for automated scans to discover vulnerable access files and retrieve them. A little addition to the httpd.conf regarding the accessibility of such files, too, will prevent curious folks from poking at .htaccess and retrieving them with ease. After all, there is no reason for an access file to be viewed by a client; it’s a server-side security configuration mechanism, meant only for the web server, and should not be exposed given the potential for leaking a lot of information that could lead to a more serious breach in security. ~ "^\.ht"> Order allow,deny Deny from all Satisfy All Another option, if you have an intermediary enabled with network-side scripting, is to prevent access to any .htaccess file across your entire infrastructure. Changes to httpd.conf must be done on every server, so if you have a lot of servers to manage and protect it’s quite possible you’d miss one due to the sheer volume of servers to slog through. Using a network-side scripting solution eliminates that possibility because it’s one change that can immediately affect all servers. Here’s an example using an iRule, but you should also be able to use mod_rewrite to accomplish the same thing if you’re using an Apache-based proxy: when HTTP_REQUEST { # Check the requested URI switch -glob [string tolower [HTTP::path]] { "/.ht*" { reject } default { pool bigwebpool } } } However you choose to protect that .htaccess file, just do it. This isn’t rocket science, it’s a straight-up simple configuration error that could potentially lead to more serious breaches in security – especially if your .htaccess file contains more sensitive (and informative) information. An Unhackable Server is Still Vulnerable Twittergate Reveals E-Mail is Bigger Security Risk than Twitter Automatically Removing Cookies Clickjacking Protection Using X-FRAME-OPTIONS Available for Firefox Stop brute force listing of HTTP OPTIONS with network-side scripting Jedi Mind Tricks: HTTP Request Smuggling I am in your HTTP headers, attacking your application Understanding network-side scriptingIt’s 2am: Do You Know What Algorithm Your Load Balancer is Using?
The wrong load balancing algorithm can be detrimental to the performance and scalability of your web applications. When you’re mixing and matching virtual or physical servers you need to take care with how you configure your Load balancer – and that includes cloud-based load balancing services. Load balancers do not at this time, unsurprisingly, magically choose the right algorithm for distributing requests for a given environment. One of the nice things about a load balancing solution that comes replete with application-specific templates is that all the work required to determine the optimal configuration for the load balancer and its associated functionality (web application security, acceleration, optimization) has already been done – including the choice of the right algorithm for that application. But for most applications there are no such templates, no guidance, nothing. Making things more difficult are heterogeneous environments in which the compute resources available vary from instance to instance. These variations make some load balancing algorithms unsuited to such environments. There is some general guidance you can use when trying to determine which algorithm is best suited to meeting the performance and scalability needs of your applications based on an understanding of how the algorithms are designed to make decisions, but if you want optimal performance and scalability you’ll ultimately have to do some testing. Heterogeneous environments can pose a challenge to scale if careful consideration of load balancing algorithms is not taken. Whether the limitations on compute resources are imposed by a virtualization solution or the hardware itself, limitations that vary from application instance to application instance are an important factor to consider when configuring your load balancing solution. Let’s say you’ve got a pool of application instances and you know the capacity of each in terms of connections (X). Two of the servers can handle 500 concurrent connections, and one can handle 1000 concurrent connections. Now assume that your load balancer is configured to perform standardround robin load balancingbetween the three instances. Even though the total capacity of these three servers appears to be 2000 concurrent connections, by the time you hit 1501, the first of the three servers will be over capacity because it will have to try to handle 501 connections. If you tweak the configuration just a bit to indicate the maximum connection capacity for each node (instance) you can probably avoid this situation, but there are no guarantees. Now let’s make a small change in the algorithm – instead of standard round robin we’ll useweightedround robin (often called “ratio”), and give the largest capacity server a higher weight based on its capacity ratio to the other servers, say 2. This means the “bigger” server will receive twice as many requests as the other two servers, which brings the total capacity closer to what is expected. You might be thinking that aleast connection algorithmwould be more appropriate in a heterogeneous environment, but that’s not the case. Least connection algorithms base distribution upon the number of connections currently open on any given server instance; it does not necessarily take into consideration the maximum connection capacity for that particular node. Fastest response time combined with per node connection limits would be a better option, but afastest response time algorithmtends to result in a very unequal distribution as load increases in a heterogeneous environment. This does not, however, say anything about the performance of the application when using any of the aforementioned algorithms. We do know that as application instances near capacity performance tends to degrade. Thus we could extrapolate that the performance for the two “smaller” servers will degrade faster than the performance for the bigger server because they will certainly reach capacity under high load before the larger server instance – when using some algorithms, at least. Algorithms like fastest response time and least connections tend to favor higher performing servers which means in the face of a sudden spike of traffic performance may degrade usingthatalgorithm as well. How about more “dynamic” algorithms that take into consideration multiple factors? Dynamic load balancing methods are designed to work with servers that differ in processing speed and memory. The resulting load balancing decisions may be uneven in terms of distribution but generally provides a more consistent user experience in terms of performance. For example, theobserved dynamic load balancing algorithmdistributes connections across applications based on a ratio calculated every second, andpredictive dynamic load balancinguses the same ratio but also takes into consideration the change between previous connection counts and current connection counts and adjusts the ratio based on the delta. Predictive mode is more aggressive in adjusting ratio values for individual application instances based on connection changes in real-time and in a heterogeneous environment is likely better able to handle the differences between server capabilities. What is TCP multiplexing? TCP multiplexing is a technique used primarily by load balancers and application delivery controllers (but also by some stand-alone web application acceleration solutions) that enables the device to "reuse" existing TCP connections. This is similar to the way in which persistent HTTP 1.1 connections work in that a single HTTP connection can be used to retrieve multiple objects, thus reducing the impact of TCP overhead on application performance. TCP multiplexing allows the same thing to happen for TCP-based applications (usually HTTP / web) except that instead of the reuse being limited to only 1 client, the connections can be reused over many clients, resulting in much greater efficiency of web servers and faster performing applications. Interestingly enough, chatting withDan Bartow(now CloudTest Evangelist and Vice President atSOASTA) about his experiences as Senior Manager of Performance Engineering atIntuit, revealed that testing different algorithms under heavy load generated externally finally led them to the discovery that a simple round robin algorithm combined with the application ofTCP multiplexing optionsyielded a huge boost in both capacity and performance. But that was only after testing under conditions which were similar to those the applications would experience during peaks in usage and normalization of the server environment. This illustrates well that performance and availability isn’t simply a matter of dumping a load balancing solution into the mix – it’s important to test, to tweak configurations, and test again to find the overall infrastructure configuration that’s going to provide the best application performance (and thus end-user experience) while maximizing resource utilization. Theoretical mathematically accurate models of load balancing are all well and good, but in the real world the complexity of the variables and interaction between infrastructure solutions and applications and servers is much higher, rendering the “theory” just that – theory. Invariably which load balancing algorithm is right for your application is going to depend heavily on what metrics are most important to you. A balance of server efficiency, response time, and availability is likely involved, but which one of these key metrics is most important depends on what business stakeholders have deemed most important to them. The only way to really determine which load balancing algorithm will achieve the results you are looking for is totest them, under load, and observe the distribution and performance of the application. FIRE and FORGET NOT a GOOD STRATEGY The worst thing you can do is “fire and forget” about your load balancer. The algorithm that might be right for one application might not be right for another, depending on the style of application, its usage patterns, the servers used to serve it, and even the time of year. Unfortunately we’re not quite at the point where the load balancer can automatically determine the right load balancing algorithm for you, butthere are ways to adjust – dynamically – the algorithm based on not just the application but also the capabilities of the servers(physical and/or virtual) being load balanced so one day it is quite possible that through the magic of Infrastructure 2.0, load balancing algorithms will be modified on-demand based on the type of servers that make up the pool of resources. In order for the level of sophistication we’d (all) like to see, however, it’s necessary to first understand the impact of the load balancing algorithm on applications and determine which one is best able to meet the service level agreements in various environments based on a variety of parameters. This will become more important as public and private cloud computing environments are leveraged in new ways and introduce more heterogeneous environments. Seasonal demand might, for example, be met by leveraging different “sizes” of unused capacity across multiple servers in the data center. These “servers” would likely be of different CPU and RAM capabilities and thus would certainly be impacted by the choice of load balancing algorithm. Being able todynamically modify the load balancing algorithmbased on the capacities of application instances is an invaluable tool when attempting to maximize the efficiency of resources while minimizing associated costs. There is, of course, a lack of control over algorithms in cloud computing environments, as well, that make the situation more difficult. With a limited set of choices available from providers the algorithm that’s best for your application and server resource composition may not be available. Providers need to make it easier for customers to take advantage of modern, application and resource-aware algorithms that have evolved through trial-and-error over the past decade. Again, Infrastructure 2.0 enables this level of choice but must be leveraged by the provider to extend that choice and control to its customers. For now, it’s going to have to be enough to (1) thoroughly test the application and its supporting infrastructure under load and (2) adjust the load balancing algorithm to meet your specific performance criteria based on what is available. You might be surprised to find how much better your response time and capacity can be when you’re using the “right” load balancing algorithm for your application – or at least one that’s more right than it is wrong if you’re in a cloud computing environment.Is Your Cloud Opaque or Transparent?
Cloud computing promises customers the ability to deliver scalable applications on-demand without the overhead of a massive data center. The visibility - and flexibility as well as control - you have into and over the cloud computing environment depends on whether the provider you select offers an opaque or a transparent cloud computing environment. OPAQUE CLOUD COMPUTING MODEL In an opaque cloud computing model all details are hidden from the organization. The hardware and software infrastructure details are not necessarily known or controlled by the organization but are completely managed by the cloud computing provider. This allows for a completely on-demand environment in which all resources necessary to deliver applications according to service level agreements are automatically provisioned, de-provisioned, and managed by the cloud computing provider. The organization need only develop and deploy the application to the cloud, the rest of the details are opaque and handled for the organization by the cloud computing provider. Most opaque cloud computing providers currently allow the organization to determine how many "instances" of their application is running at any given time, with the details of provisioning the appropriate resources to support those instances hidden from the customer view. In many ways, SaaS (Software as a Service) providers such as Salesforce.com have been using an opaque cloud computing model for many years, as the implementation details in terms of hardware and software infrastructure are completely hidden from and unavailable to the customer (the organization). The difference between a SaaS offering and an opaque cloud computing model is in the application development and deployment processes. A SaaS offering such as Salesforce.com or Google Apps requires development of applications on a specific platform, almost always proprietary to the SaaS provider and non-negotiable. An opaque cloud computing provider allows the organization to determine the platform upon which applications will be deployed, more akin to some of Amazon's cloud offerings, such as EC2, though the underlying operating system and hardware may not be known due to the extensive use of operating system virtualization by the cloud computing provider. This is why virtualization is inexorably tied to cloud computing - it is the most efficient way to deploy an application to a remote, virtual data center without the overhead of configuration and management of the incredibly high number of combinations possible for application platforms. Opaque cloud computing providers like Joyent have an infrastructure already constructed to scale. Customers may be aware of what that infrastructure comprises, but cannot necessarily specify the choice of switches, routers, or application delivery infrastructure. TRANSPARENT CLOUD COMPUTING MODEL In a transparent cloud computing model the organization is left with determining its specific needs. The organization decides how much computing power it requires, what hardware and software solutions it will require, and it manages its provisioned resources in the cloud. The transparent cloud computing model is more akin to an outsourced data-center, a virtual data center if you will, than it is to an on-demand opaque cloud computing model. The acquisition and provisioning of resources becomes much difficult in a transparent cloud computing model. The prospect of automated on-demand computing in a transparent cloud computing model is good and in some cases already available for some functions. The same mechanisms used to manage the opaque cloud computing environment could become customer-facing ones. Some management and configuration of cloud computing resources are currently being offered to customers by providers like RightScale, though what infrastructure functions can be delegated to the organization vary greatly from provider to provider. Rackspace is a good example of a transparent cloud computing model, as are many of the traditional hosting providers. The transparent cloud computing environment is still evolving, currently comprising several different models for architecting your infrastructure. Some, like Areti, offer flexibility in infrastructure choices by taking advantage of virtual appliances. This allows the customer to choose from a number of application and network infrastructure solutions while keeping the cost of acquisition and management down for both the provider and the customer. Other providers continue to focus on physical infrastructure deployment in a fully or collaboratively managed environment, understanding that some organizations will require dedicated, proven solutions if they move into the cloud. THE HYBRID MODEL CIO.com recently offered a list of 11 cloud computing vendors to watch, culled from a Forrester Research report. The list comprises a mix of opaque and transparent providers, with a tendency to lean toward the opaque. A few of the providers on the list are leaning toward a hybrid cloud computing model, with the ability to specify a choice of infrastructure devices from those supported by the provider while providing fully managed services. A hybrid model is likely where providers will eventually converge, as it promises to provide the greatest flexibility for customers without sacrificing some of the control necessary on the part of the provider. After all, while allowing customers to manage and configure some components of the application delivery network, others (routers, switches) are likely to not require such hands-on management by customers. In fact, such mucking around at layer 2 and 3 could very well do more harm than good, as most value added features and functionality for application delivery comes at layer 4 and above and is best handled by a solution separate from core network infrastructure. And while many customers will be comfortable testing the waters with virtual appliances or open-source solutions for application delivery infrastructure, eventually more proven solutions will be required as customers begin to demand more flexibility and functionality in the cloud. Cloud computing providers who evolve quickly and take advantage of componentized application delivery infrastructure will be able to better differentiate their offerings by offering additional features such as acceleration, security, rate shaping, and advanced authentication. These advanced features are one of the primary reasons that "Option #3" will be increasingly important for application delivery and networking infrastructure providers. Cloud computing providers appear willing to support such features, but require that the solutions be able to be integrated and remotely managed, on-demand, before they will do so.Architecting for Speed
I'm going to give you an engine low to the ground. An extra-big oil pan that'll cut the wind underneath you. That'll give you more horsepower. I'll give you a fuel line that'll hold an extra gallon of gas. I'll shave half an inch off you and shape you like a bullet. When I get you primed, painted and weighed... ...you're going to be ready to go out on that racetrack. You're going to be perfect. (From the movie: Days of Thunder) In the monologue above, Harry Hogge, crew chief, is talking to the framework of a car; explaining how it is that he's going to architect her for speed. What I love about this monologue is that Harry isn't focusing on any one aspect of the car, he's looking at the big picture - inside and out. This is the way we should architect web application infrastructures for speed: holistically and completely, taking the entire application delivery infrastructure into consideration, because each component in that infrastructure can have an effect - positive or negative - on the performance of web applications. Analyst firm Forrester recently hosted a teleconference (download available soon) on this very subject entitled "Web Performance Architecture Best Practices." In one single slide analysts Mike Gualtieri and James Staten captured the essence of Harry's monologue by promoting a holistic view of web application performance that includes the inside and outside of an application. "Performance depends upon a holistic view of your architecture" SOURCE: "Teleconference: Web Performance Architecture Best Practices", Forrester Research, July 2008. The discussion goes on to describe how to ensure speedy delivery of applications, and includes the conclusion that cutting Web-tier response time by half delivers an overall 40% improvement in the performance of applications. Cutting response time is the primary focus of web application acceleration solutions. Combining intelligent caching and compression with technologies that make the browser more efficient improve the overall responsiveness of the web tier of your web applications. And what's best is that you don't have to do anything to the web applications to get that improvement. While improving performance in the application and data tiers of an application architecture can require changes to the application including a lot of coding, the edge and application infrastructure can often provide a significant boost in performance simply by transparently adding the ability to optimize web application protocols as well as their underlying transport protocols (TCP, HTTP). Steve Souders, author of "High Performance Web Sites" (O'Reilly Media, Inc., 2007) further encourages an architecture that includes compressing everything as well as maximizing the use of the browser's cache. But my absolute favorite line from the teleconference? "Modern load balancers do far more than just spread the load." Amen, brothers! Can I get a hallelujah? If you weren't able to attend, I highly recommend downloading the teleconference when it's available and giving it a listen. It includes a great case study, as well, on how to build a high performing, scalable web application that helps wrap some reality around the concepts discussed. Perhaps one day we'll be talking to our applications like Harry Hogge does to the car he's about to build... I'm going to give you code with tightly written loops. An extra-fast infrastructure that'll offload functionality for you. That'll give you more horsepower. I'll give you a network that'll hold an extra megabit of bandwidth. I'll compress and shape your data like a bullet. When I get you optimized, secured and deployed... ...you're going to be ready to go out on the Internet. You're going to be perfect.Performance Testing Web 2.0: Don't forget the APIs
Keynote, well known for its application performance testing and monitoring services, just announced a new version of its KITE (Keynote Internet Testing Environment) that is now capable of testing Web 2.0 sites that make use of AJAX, Flash, and other "hidden" methods of obtaining content. Announcement of KITE 2 Performance testing dynamic and HTML websites is now a fairly straightforward process, however the rise of Web 2.0 sites that don’t rely on clicking to reveal another new page have been almost impossible to test. However Keynote has now developed a scripted system that allows you to test Web 2.0 sites as easily as Web 1.0 sites. KITE 2.0 (Keynote Internet Testing Environment), is the latest version of Keynote’s product for testing and analysing the performance of Web applications. KITE 2.0 gives users the flexibility of testing instantly from their desktop or from geographic locations across Keynote’s on-demand global test and measurement network. According to Keynote KITE 2.0 enables Web developers, QA professionals, performance analysts and others, to execute rapid performance analysis and validation by measuring the end user experience of next Web 2.0 applications that include AJAX and asynchronously downloaded content. Ensuring that the additional burden placed on servers by "hidden" requests is imperative when attempting to understand both capacity and end-user performance. But when you're laying out that testing plan, don't forget about any APIs you might be providing. Like Twitter, Google, Facebook, and Amazon, if you're using an API to allow integration with other Web 2.0 sites or applications make certain you performance test that, as well. And then test them at the same time you're load-testing your application. As we've learned from Twitter's most public stability issues, the APIs are going to put additional stress on your network, servers, and databases that must be considered when determining capacity and performance levels. Important, too, is to take into consideration RSS feeds coming from your site. Include those in your performance test, as the retrieval of those files adds an additional burden on servers and can impact the performance of the entire site. Most feed-readers and aggregators poll for RSS feeds on a specified interval so set up a script to grab that RSS from multiple clients every 10 minutes or so during the testing to ensure that you're simulating maximum load as closely as possible. Performance testing sites today is getting more complex because we're adding so many entry points into our sites and applications. Don't forget about those extra entry points when you perform your load-testing or you might find out in a most public way that your application just can't handle the load.
