Infrastructure Architecture: Whitelisting with JSON and API Keys
Application delivery infrastructure can be a valuable partner in architecting solutions …. AJAX and JSON have changed the way in which we architect applications, especially with respect to their ascendancy to rule the realm of integration, i.e. the API. Policies are generally focused on the URI, which has effectively become the exposed interface to any given application function. It’s REST-ful, it’s service-oriented, and it works well. Because we’ve taken to leveraging the URI as a basic building block, as the entry-point into an application, it affords the opportunity to optimize architectures and make more efficient the use of compute power available for processing. This is an increasingly important point, as capacity has become a focal point around which cost and efficiency is measured. By offloading functions to other systems when possible, we are able to increase the useful processing capacity of an given application instance and ensure a higher ratio of valuable processing to resources is achieved. The ability of application delivery infrastructure to intercept, inspect, and manipulate the exchange of data between client and server should not be underestimated. A full-proxy based infrastructure component can provide valuable services to the application architect that can enhance the performance and reliability of applications while abstracting functionality in a way that alleviates the need to modify applications to support new initiatives. AN EXAMPLE Consider, for example, a business requirement specifying that only certain authorized partners (in the integration sense) are allowed to retrieve certain dynamic content via an exposed application API. There are myriad ways in which such a requirement could be implemented, including requiring authentication and subsequent tokens to authorize access – likely the most common means of providing such access management in conjunction with an API. Most of these options require several steps, however, and interaction directly with the application to examine credentials and determine authorization to requested resources. This consumes valuable compute that could otherwise be used to serve requests. An alternative approach would be to provide authorized consumers with a more standards-based method of access that includes, in the request, the very means by which authorization can be determined. Taking a lesson from the credit card industry, for example, an algorithm can be used to determine the validity of a particular customer ID or authorization token. An API key, if you will, that is not stored in a database (and thus requires a lookup) but rather is algorithmic and therefore able to be verified as valid without needing a specific lookup at run-time. Assuming such a token or API key were embedded in the URI, the application delivery service can then extract the key, verify its authenticity using an algorithm, and subsequently allow or deny access based on the result. This architecture is based on the premise that the application delivery service is capable of responding with the appropriate JSON in the event that the API key is determined to be invalid. Such a service must therefore be network-side scripting capable. Assuming such a platform exists, one can easily implement this architecture and enjoy the improved capacity and resulting performance boost from the offload of authorization and access management functions to the infrastructure. 1. A request is received by the application delivery service. 2. The application delivery service extracts the API key from the URI and determines validity. 3. If the API key is not legitimate, a JSON-encoded response is returned. 4. If the API key is valid, the request is passed on to the appropriate web/application server for processing. Such an approach can also be used to enable or disable functionality within an application, including live-streams. Assume a site that serves up streaming content, but only to authorized (registered) users. When requests for that content arrive, the application delivery service can dynamically determine, using an embedded key or some portion of the URI, whether to serve up the content or not. If it deems the request invalid, it can return a JSON response that effectively “turns off” the streaming content, thereby eliminating the ability of non-registered (or non-paying) customers to access live content. Such an approach could also be useful in the event of a service failure; if content is not available, the application delivery service can easily turn off and/or respond to the request, providing feedback to the user that is valuable in reducing their frustration with AJAX-enabled sites that too often simply “stop working” without any kind of feedback or message to the end user. The application delivery service could, of course, perform other actions based on the in/validity of the request, such as directing the request be fulfilled by a service generating older or non-dynamic streaming content, using its ability to perform application level routing. The possibilities are quite extensive and implementation depends entirely on goals and requirements to be met. Such features become more appealing when they are, through their capabilities, able to intelligently make use of resources in various locations. Cloud-hosted services may be more or less desirable for use in an application, and thus leveraging application delivery services to either enable or reduce the traffic sent to such services may be financially and operationally beneficial. ARCHITECTURE is KEY The core principle to remember here is that ultimately infrastructure architecture plays (or can and should play) a vital role in designing and deploying applications today. With the increasing interest and use of cloud computing and APIs, it is rapidly becoming necessary to leverage resources and services external to the application as a means to rapidly deploy new functionality and support for new features. The abstraction offered by application delivery services provides an effective, cross-site and cross-application means of enabling what were once application-only services within the infrastructure. This abstraction and service-oriented approach reduces the burden on the application as well as its developers. The application delivery service is almost always the first service in the oft-times lengthy chain of services required to respond to a client’s request. Leveraging its capabilities to inspect and manipulate as well as route and respond to those requests allows architects to formulate new strategies and ways to provide their own services, as well as leveraging existing and integrated resources for maximum efficiency, with minimal effort. Related blogs & articles: HTML5 Going Like Gangbusters But Will Anyone Notice? Web 2.0 Killed the Middleware Star The Inevitable Eventual Consistency of Cloud Computing Let’s Face It: PaaS is Just SOA for Platforms Without the Baggage Cloud-Tiered Architectural Models are Bad Except When They Aren’t The Database Tier is Not Elastic The New Distribution of The 3-Tiered Architecture Changes Everything Sessions, Sessions EverywhereWhat is server offload and why do I need it?
One of the tasks of an enterprise architect is to design a framework atop which developers can implement and deploy applications consistently and easily. The consistency is important for internal business continuity and reuse; common objects, operations, and processes can be reused across applications to make development and integration with other applications and systems easier. Architects also often decide where functionality resides and design the base application infrastructure framework. Application server, identity management, messaging, and integration are all often a part of such architecture designs. Rarely does the architect concern him/herself with the network infrastructure, as that is the purview of “that group”; the “you know who I’m talking about” group. And for the most part there’s no need for architects to concern themselves with network-oriented architecture. Applications should not need to know on which VLAN they will be deployed or what their default gateway might be. But what architects might need to know – and probably should know – is whether the network infrastructure supports “server offload” of some application functions or not, and how that can benefit their enterprise architecture and the applications which will be deployed atop it. WHAT IT IS Server offload is a generic term used by the networking industry to indicate some functionality designed to improve the performance or security of applications. We use the term “offload” because the functionality is “offloaded” from the server and moved to an application network infrastructure device instead. Server offload works because the application network infrastructure is almost always these days deployed in front of the web/application servers and is in fact acting as a broker (proxy) between the client and the server. Server offload is generally offered by load balancers and application delivery controllers. You can think of server offload like a relay race. The application network infrastructure device runs the first leg and then hands off the baton (the request) to the server. When the server is finished, the application network infrastructure device gets to run another leg, and then the race is done as the response is sent back to the client. There are basically two kinds of server offload functionality: Protocol processing offload Protocol processing offload includes functions like SSL termination and TCP optimizations. Rather than enable SSL communication on the web/application server, it can be “offloaded” to an application network infrastructure device and shared across all applications requiring secured communications. Offloading SSL to an application network infrastructure device improves application performance because the device is generally optimized to handle the complex calculations involved in encryption and decryption of secured data and web/application servers are not. TCP optimization is a little different. We say TCP session management is “offloaded” to the server but that’s really not what happens as obviously TCP connections are still opened, closed, and managed on the server as well. Offloading TCP session management means that the application network infrastructure is managing the connections between itself and the server in such a way as to reduce the total number of connections needed without impacting the capacity of the application. This is more commonly referred to as TCP multiplexing and it “offloads” the overhead of TCP connection management from the web/application server to the application network infrastructure device by effectively giving up control over those connections. By allowing an application network infrastructure device to decide how many connections to maintain and which ones to use to communicate with the server, it can manage thousands of client-side connections using merely hundreds of server-side connections. Reducing the overhead associated with opening and closing TCP sockets on the web/application server improves application performance and actually increases the user capacity of servers. TCP offload is beneficial to all TCP-based applications, but is particularly beneficial for Web 2.0 applications making use of AJAX and other near real-time technologies that maintain one or more connections to the server for its functionality. Protocol processing offload does not require any modifications to the applications. Application-oriented offload Application-oriented offload includes the ability to implement shared services on an application network infrastructure device. This is often accomplished via a network-side scripting capability, but some functionality has become so commonplace that it is now built into the core features available on application network infrastructure solutions. Application-oriented offload can include functions like cookie encryption/decryption, compression, caching, URI rewriting, HTTP redirection, DLP (Data Leak Prevention), selective data encryption, application security functionality, and data transformation. When network-side scripting is available, virtually any kind of pre or post-processing can be offloaded to the application network infrastructure and thereafter shared with all applications. Application-oriented offload works because the application network infrastructure solution is mediating between the client and the server and it has the ability to inspect and manipulate the application data. The benefits of application-oriented offload are that the services implemented can be shared across multiple applications and in many cases the functionality removes the need for the web/application server to handle a specific request. For example, HTTP redirection can be fully accomplished on the application network infrastructure device. HTTP redirection is often used as a means to handle application upgrades, commonly mistyped URIs, or as part of the application logic when certain conditions are met. Application security offload usually falls into this category because it is application – or at least application data – specific. Application security offload can include scanning URIs and data for malicious content, validating the existence of specific cookies/data required for the application, etc… This kind of offload improves server efficiency and performance but a bigger benefit is consistent, shared security across all applications for which the service is enabled. Some application-oriented offload can require modification to the application, so it is important to design such features into the application architecture before development and deployment. While it is certainly possible to add such functionality into the architecture after deployment, it is always easier to do so at the beginning. WHY YOU NEED IT Server offload is a way to increase the efficiency of servers and improve application performance and security. Server offload increases efficiency of servers by alleviating the need for the web/application server to consume resources performing tasks that can be performed more efficiently on an application network infrastructure solution. The two best examples of this are SSL encryption/decryption and compression. Both are CPU intense operations that can consume 20-40% of a web/application server’s resources. By offloading these functions to an application network infrastructure solution, servers “reclaim” those resources and can use them instead to execute application logic, serve more users, handle more requests, and do so faster. Server offload improves application performance by allowing the web/application server to concentrate on what it is designed to do: serve applications and putting the onus for performing ancillary functions on a platform that is more optimized to handle those functions. Server offload provides these benefits whether you have a traditional client-server architecture or have moved (or are moving) toward a virtualized infrastructure. Applications deployed on virtual servers still use TCP connections and SSL and run applications and therefore will benefit the same as those deployed on traditional servers. I am wondering why not all websites enabling this great feature GZIP? 3 Really good reasons you should use TCP multiplexing SOA & Web 2.0: The Connection Management Challenge Understanding network-side scripting I am in your HTTP headers, attacking your application Infrastructure 2.0: As a matter of fact that isn't what it meansClickjacking Protection Using X-FRAME-OPTIONS Available for Firefox
But browser support is only half the solution, don’t forget to implement the server-side, too. Clickjacking, unlike more well-known (and understood) web application vulnerabilities, has been given scant amount of attention despite its risks and its usage. Earlier this year, for example, it was used as an attack on Twitter, but never really discussed as being a clickjacking attack. Maybe because aside from rewriting applications to prevent CSRF (adding nonces and validation of the same to every page) or adding framekillers there just haven’t been many other options to prevent the attack technique from being utilized against users. Too, it is one of the more convoluted attack methods out there so it would be silly to expect non-technical media to understand it let alone explain how it works to their readers. There is, however, a solution on the horizon. IE8 has introduced an opt-in measure that allows developers – or whomever might be in charge of network-side scripting implementations – to prevent clickjacking on vulnerable pages using a custom HTTP header to prevent them from being “framed” inappropriately: X-FRAME-OPTIONS. The behavior is described in the aforementioned article as: If the X-FRAME-OPTIONS value contains the token DENY, IE8 will prevent the page from rendering if it will be contained within a frame. If the value contains the token SAMEORIGIN, IE will block rendering only if the origin of the top level-browsing-context is different than the origin of the content containing the X-FRAME-OPTIONS directive. For instance, if http://shop.example.com/confirm.asp contains a DENY directive, that page will not render in a subframe, no matter where the parent frame is located. In contrast, if the X-FRAME-OPTIONS directive contains the SAMEORIGIN token, the page may be framed by any page from the exact http://shop.example.com origin. But that’s only IE8, right? Well, natively, yes. But a development version of NoScript has been released that supports the X-FRAME-OPTIONS header and will provide the same protections as are natively achieved in IE8. The problem is that this is only half the equation: the X-FRAME-OPTIONS header needs to exist before the browser can act on it and the preventive measure for clickjacking completed. As noted in the Register, “some critics have contended the protection will be ineffective because it will require millions of websites to update their pages with proprietary code.” That’s not entirely true as there is another option that will provide support for X-FRAME-OPTIONS without updating pages/applications/sites with proprietary code: network-side scripting. The “proprietary” nature of custom HTTP headers is also debatable, as support for Firefox was provided quickly via NoScript and if the technique is successful will likely be adopted by other browser creators. HOW-TO ADD X-FRAME-OPTIONS TO YOUR APPLICATION – WITH or WITHOUT CODE CHANGES Step 1: Add the custom HTTP header “X-FRAME-OPTIONS” with a value of “DENY” or “SAMEORIGIN” before returning a response to the client Really, that’s it. The browser takes care of the rest for you. OWASP has a great article on how to implement a ClickjackFilter for JavaEE and there are sure to be many more blogs and articles popping up describing how one can implement such functionality in their language-of-choice. Even without such direct “how-to” articles and code samples, it is merely a matter of adding a new custom HTTP header – examples of which ought to be easy enough to find. Similarly a solution can be implemented using network-side scripting that requires no modification to applications. In fact, this can be accomplished via iRules in just one line of code: when HTTP_RESPONSE { HTTP::header insert "X-FRAME-OPTIONS" “(DENY || SAMEORIGIN)”} I believe the mod_rewrite network-side script would be as simple, but as I am not an expert in mod_rewrite I will hope someone who is will leave an appropriate example as a comment or write up a blog/article and leave a pointer to it. A good reason to utilize the agility of network-side scripting solutions in this case is that it is not necessary to modify each application requiring protection, which takes time to implement, test, and deploy. An even better reason is that a single network-side script can protect all applications, regardless of language and deployment platform, without a lengthy development and deployment cycle. Regardless of how you add the header, it would be a wise idea to add it as a standard part of your secure-code deployment requirements (you do have those, don’t you?) because it doesn’t hurt anything for the custom HTTP header to exist and visitors using X-FRAME-OPTIONS enabled browsers/solutions will be a lot safer than without it. Stop brute force listing of HTTP OPTIONS with network-side scripting Jedi Mind Tricks: HTTP Request Smuggling I am in your HTTP headers, attacking your application Understanding network-side scripting 9 ways to use network-side scripting to architect faster, scalable, more secure applicationsWARNING: Security Device Enclosed
If you aren’t using all the security tools at your disposal you’re doing it wrong. How many times have you seen an employee wave on by a customer when the “security device enclosed” in some item – be it DVD, CD, or clothing – sets off the alarm at the doors? Just a few weeks ago I heard one young lady explain the alarm away with “it must have be the CD I bought at the last place I was at…” This apparently satisfied the young man at the doors who nodded and turned back to whatever he’d been doing. All the data the security guy needed to make a determination was there; he had all the context necessary in which to analyze the situation and make a determination based upon that information. But he ignored it all. He failed to leverage all the tools at his disposal and potentially allowed dollars to walk out the door. In doing so he also set a precedent and unintentionally sent a message to anyone who really wanted to commit a theft: I ignore warning signs, go ahead.I Can Has UR .htaccess File
Notice that isn’t a question, it’s a statement of fact Twitter is having a bad month. After it was blamed, albeit incorrectly, for a breach leading to the disclosure of both personal and corporate information via Google’s GMail and Apps, its apparent willingness to allow anyone and everyone access to a .htaccess file ostensibly protecting search.twitter.com made the rounds via, ironically, Twitter. This vulnerability at first glance appears fairly innocuous, until you realize just how much information can be placed in an .htaccess file that could have been exposed by this technical configuration faux pas. Included in the .htaccess file is a number of URI rewrites, which give an interesting view of the underlying file system hierarchy Twitter is using, as well as a (rather) lengthy list of IP addresses denied access. All in all, not that exciting, because many of the juicy bits that could be configured via .htaccess for any given website are not done so in this easily accessible .htaccess file. Some things you can do with .htaccess, in case you aren’t familiar: Create default error document Enable SSI via htaccess Deny users by IP Change your default directory page Redirects Prevent hotlinking of your images Prevent directory listing .htaccess is a very versatile little file, capable of handling all sorts of security and application delivery tasks. Now what’s interesting is that the .htaccess file is in the root directory and should not be accessible. Apache configuration files are fairly straight forward, and there are plethora examples of how to prevent .htaccess – and its wealth of information – from being viewed by clients. Obfuscation, of course, is one possibility, as Apache’s httpd.conf allows you to specify the name of the access file with a simple directive: AccessFileName .htaccess It is a simple enough thing to change the name of the file, thus making it more difficult for automated scans to discover vulnerable access files and retrieve them. A little addition to the httpd.conf regarding the accessibility of such files, too, will prevent curious folks from poking at .htaccess and retrieving them with ease. After all, there is no reason for an access file to be viewed by a client; it’s a server-side security configuration mechanism, meant only for the web server, and should not be exposed given the potential for leaking a lot of information that could lead to a more serious breach in security. ~ "^\.ht"> Order allow,deny Deny from all Satisfy All Another option, if you have an intermediary enabled with network-side scripting, is to prevent access to any .htaccess file across your entire infrastructure. Changes to httpd.conf must be done on every server, so if you have a lot of servers to manage and protect it’s quite possible you’d miss one due to the sheer volume of servers to slog through. Using a network-side scripting solution eliminates that possibility because it’s one change that can immediately affect all servers. Here’s an example using an iRule, but you should also be able to use mod_rewrite to accomplish the same thing if you’re using an Apache-based proxy: when HTTP_REQUEST { # Check the requested URI switch -glob [string tolower [HTTP::path]] { "/.ht*" { reject } default { pool bigwebpool } } } However you choose to protect that .htaccess file, just do it. This isn’t rocket science, it’s a straight-up simple configuration error that could potentially lead to more serious breaches in security – especially if your .htaccess file contains more sensitive (and informative) information. An Unhackable Server is Still Vulnerable Twittergate Reveals E-Mail is Bigger Security Risk than Twitter Automatically Removing Cookies Clickjacking Protection Using X-FRAME-OPTIONS Available for Firefox Stop brute force listing of HTTP OPTIONS with network-side scripting Jedi Mind Tricks: HTTP Request Smuggling I am in your HTTP headers, attacking your application Understanding network-side scriptingI am in your HTTP headers, attacking your application
Zero-day IE exploits and general mass SQL injection attacks often overshadow potentially more dangerous exploits targeting lesser known applications and attack vectors. These exploits are potentially more dangerous because once proven through a successful attack on these lesser known applications they can rapidly be adapted to exploit more common web applications, and no one is specifically concentrating on preventing them because they're, well, not so obvious. Recently, SANS Internet Storm Center featured a write up on attempts to exploit Roundcube Webmail via the HTTP Accept header. Such an attack is generally focused on exploitation of operating system, language, or environmental vulnerabilities, as the data contained in HTTP headers (aside from cookies) is rarely used by the application as user-input. An example provided by SANS of an attack targeting Roundcube via the HTTP Accept header: POST /roundcube/bin/html2text.php HTTP/1.1 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 Host: xx.xx.xx.xx Accept: ZWNobyAoMzMzMjEyKzQzMjQ1NjY2KS4iICI7O3Bhc3N0aHJ1KCJ1bmFtZSAtYTtpZCIpOw== Content-Length: 54 What the attackers in this example were attempting to do is trick the application into evaluating system commands encoded in the Accept header in order to retrieve some data they should not have had access to. The purpose of the attack, however, could easily have been for some other nefarious deed such as potentially writing a file to the system that could be used as a cross-site scripting attack, or deleting files, or just generally wreaking havoc with the system. This is the problem security professionals and developers face every day: what devious thing could some miscreant attempt to do? What must I protect against. This is part of what makes secure coding so difficult - developers aren't always sure what they should be protecting against, and neither are the security pros because the bad guys are always coming up with a new way to exploit some aspect of an application or transport layer protocols. Think HTTP headers aren't generally used by applications? Consider the use of the custom HTTP header "SOAP Action" for SOAP web services, and cookies, and E-tags, and ... well, the list goes on. HTTP headers carry data used by applications and therefore should be considered a viable transport mechanism for malicious code. So while the exploitation of HTTP headers is not nearly as common or rampant as mass SQL injection today, the use of it to target specific applications means it is a possible attack vector for the future against which applications should be protected now, before it becomes critical to do so. No, it may never happen. Attackers may never find a way to truly exploit HTTP headers. But then again, they might and apparently have been trying. Better safe than sorry, I say. Regardless of the technology you use to, the process is the same: you need to determine what is allowed in HTTP headers and verify them just as you would any other user-generated input or you need to invest in a solution that provides this type of security for you. RFC 2616 (HTTP), specifically section 14, provide a great deal of guidance and detail on what is acceptable in an HTTP header field. Never blindly evaluate or execute upon data contained in an HTTP header field. Treat any input, even input that is not traditionally user-generated, as suspect. That's a good rule of thumb for protecting against malicious payloads anyway, but especially a good rule when dealing with what is likely considered a non-traditional attack vector (until it is used, and overused to the point it's considered typical, of course). Possible ways to prevent the potential exploitation of HTTP headers: Use network-side scripting or mod_rewrite to intercept, examine, and either sanitize or outright reject requests containing suspicious data in HTTP headers. Invest in a security solution capable of sanitizing transport (TCP) and application layer (HTTP) protocols and use it to do so. Investigate whether an existing solution - either security or application delivery focused - is capable of providing the means through which you can enforce protocol compliance. Use secure coding techniques to examine - not evaluate - the data in any HTTP headers you are using and ensure they are legitimate values before using them in any way. A little proactive security can go along way toward not being the person who inadvertently discovers a new attack methodology. Related articles by Zemanta Gmail Is Vulnerable to Hackers The Concise Guide to Proxies 3 reasons you need a WAF even though your code is (you think) secure Stop brute forcing listing of HTTP OPTIONS with network-side scripting What's the difference between a web application and a blog?Cloud Computing: Vertical Scalability is Still Your Problem
Horizontal scalability achieved through the implementation of a load balancing solution is easy. It's vertical scalability that's always been and remains difficult to achieve, and it's even more important in a cloud computing or virtualized environment because now it can hurt you where it counts: the bottom line. Horizontal scalability is the ability of an application to be scaled up to meet demand through replication and the distribution of requests across a pool or farm of servers. It's the traditional load balanced model, and it's an integral component of cloud computing environments. Vertical scalability is the ability of an application to scale under load; to maintain performance levels as the number of concurrent requests increases. While load balancing solutions can certainly assist in optimizing the environment in which an application needs to scale by reducing overhead that can negatively impact performance (such as TCP session management, SSL operations, and compression/caching functionality) it can't solve core problems that prevent vertical scalability. The problem is that a single database table or SQL query that is poorly constructed can destroy vertical scalability and actually increase the cost of deploying in the cloud. Because you generally pay on a resource basis, if the application isn’t scaling up well it will require more resources to maintain performance levels and thus cost a lot more. Cloud computing isn’t going to magically optimize code or database queries or design database tables with performance in mind, that’s still squarely in the hands of the developers regardless of whether or not cloud computing is used as the deployment model. The issue of vertical scalability is very important when considering the use of cloud computing because you’re often charged based on compute resources used, much like the old mainframe model. If an application doesn’t vertically scale well, it’s going to increase the costs to run in the cloud. Cloud computing providers can't, and probably wouldn't if they could (it makes them money, after all), address vertical scalability issues because they are peculiar to the application. No external solution can optimize code such that the application will magically scale up vertically. External solutions can improve overall performance, certainly, by optimizing protocols, reducing protocol and application overhead, and reducing bandwidth requirements, but it can't dig into the application code and rearrange the order in which joins are performed inside an SQL query, rewrite a particularly poorly written loop, or refactor code to use a more efficient data structure. Vertical scalability, whether the application is deployed inside the local data center or out there in the cloud, is still the domain of the application developer. While developers can certainly take advantage of technologies like network-side scripting and inherent features in application delivery solutions to assist with efforts to increase vertical scalability, there is a limit to what solutions can do to address the root cause of an application's failure to vertically scale. Improving the vertical scalability of applications is important in achieving the benefits of a reduction in costs associated with cloud computing and virtualization. Applications that fail to vertically scale well may end up costing more when deployed in the cloud because of the additional demand on compute resources required as demand increases. Things you can do to improve vertical scalability Optimize SQL / database queries Take advantage of offload capabilities of application delivery solutions available Be aware of the impact on performance and scalability of decomposing applications into too finely grained services Remember that API usage will impact vertical scalability Understand the bottlenecks associated with the programming language(s) used and address them Cloud computing and virtualization can certainly address vertical scalability limitations by using horizontal scaling techniques to ensure capacity meets demand and performance level agreements are met. But doing so may cost you dearly and eliminate many of the financial incentives that led you to adopt cloud computing or virtualization in the first place. Related articles by Zemanta Infrastructure 2.0: The Diseconomy of Scale Virus Why you should not use clustering to scale an application Top 10 Concepts That Every Software Engineer Should Know Can Today's Hardware Handle the Cloud? Vendors air the cloud's pros and cons Twitter and the Architectural Challenges of Life Streaming ApplicationsWho Took the Cookie from the Cookie Jar … and Did They Have Proper Consent?
Cookies as a service enabled via infrastructure services provide an opportunity to improve your operational posture. Fellow DevCentral blogger Robert Haynes posted a great look at a UK law regarding cookies. Back in May a new law went info effect regarding “how cookies and other “cookie-like” objects are stored on users’ devices.” If you haven’t heard about it, don’t panic – there’s a one-year grace period before enforcement begins and those £500 000 fines are being handed out. The clock is ticking, however. What do the new regulations say? Well essentially whereas cookies could be stored with what I, as a non-lawyer, would term implied consent, i.e. the cookies you set are listed along with their purpose and how to opt out in some interminable privacy policy on your site, you are now going to have to obtain a more active and informed consent to store cookies on a user’s device. -- The UK Cookie Law – Robert goes on to explain that the solution to this problem requires (1) capturing cookies and (2) implementing some mechanism to allow users to grant consent. Mind you, this is not a trivial task. There are logic considerations – not all cookies are set at the same time – as well as logistical issues – how do you present a request for consent? Once consent is granted, where do you store that? In a cookie? That you must gain consent to store? Infinite loops are never good. And of course, what do you if consent is not granted, but the application depends on that cookie existing? To top it all off, the process of gathering consent requires modification to application behavior, which means new code, testing and eventually deployment. Infrastructure services may present an alternative approach that is less disruptive technologically, but does not necessarily address the business or logic ramifications resulting from such a change. COOKIES as a SERVICE Cookies as a Service, a.k.a. cookie gateways, wherein cookie authorization and management is handled by an intermediate proxy, is likely best able to mitigate the expense and time associated with modifying applications to meet the new UK regulation. As Robert describes, he’s currently working on a network-side scripting solution to meet the UK requirements that leverages a full-proxy architecture’s ability to mediate for applications and communicate directly with clients before passing requests on to the application. Not only is this a valid approach to managing privacy regulations, it’s also a good means of providing additional security against attacks that leverage cookies either directly or indirectly. Cross-site scripting, browser vulnerabilities and other attacks that bypass the same origin policy of modern web browsers – sometimes by design to circumvent restrictions on integration methods – as well as piggy-backing on existing cookies as a means to gain unauthorized access to applications are all potential dangerous of cookies. By leveraging encryption of cookies in conjunction with transport layer security, i.e. SSL, organizations can better protect both users and applications from unintended security consequences. Implementing a cookie gateway should make complying with regulations like the new UK policy a less odious task. By centralizing cookie management on an intermediate device, they can be easily collected and displayed along with the appropriate opt-out / consent policies without consuming application resources or requiring every application to be modified to include the functionality to do so. AN INFRASTRUCTURE SERVICE This is one of the (many) ways in which an infrastructure service hosted in “the network” can provide value to both application developers and business stakeholders. Such a reusable infrastructure-hosted service can be leveraged to provide services to all applications and users simultaneously, dramatically reducing the time and effort required to support such a new initiative. Reusing an infrastructure service also reduces the possibility of human error during the application modification, which can drag out the deployment lifecycle and delay time to market. In the case of meeting the requirements of the new UK regulations, that delay could become costly. According to a poll of CIOs regarding their budgets for 2010, The Society for Information Management found that “Approximately 69% of IT spending in 2010 will be allocated to existing systems, while about 31% will be spent on building and buying new systems. This ratio remains largely the same compared to this year's numbers.” If we are to be more responsive to new business initiatives and flip that ratio such that we are spending less on maintaining existing systems and more on developing new systems and methods of management (i.e. cloud computing ) we need to leverage strategic points of control within the network to provide services that minimize resources, time and money on existing systems. Infrastructure services such as cookie gateways provide the opportunity to enhance security, comply with regulations and eliminate costs in application development that can be reallocated to new initiatives and projects. We need to start treating the network and its unique capabilities as assets to be leveraged and services to be enabled instead of a fat, dumb pipe. Understanding network-side scripting This is Why We Can’t Have Nice Things When the Data Center is Under Siege Don’t Forget to Watch Under the Floor IT as a Service: A Stateless Infrastructure Architecture Model You Can’t Have IT as a Service Until IT Has Infrastructure as a Service F5 Friday: Eliminating the Blind Spot in Your Data Center Security Strategy The UK Cookie Law –The URL Shortener-SEO Cage Match
Making the case for a Hungarian Notation variation for URL hierarchies One of the top discussions out in the ether these days revolves around URL shortening. One of the reasons folks flock to URL shortening services like bit.ly and TinyURL is because web sites and applications use exceedingly long URLs. Many times this is because of exposed file system hierarchies (a potential security risk, by the way) and a desire to take advantage of descriptive file names for SEO and informational reasons. Recently Delicious founder Joshua Schachter expressed his opinion that URL Shorteners are bad for the web, while Ben Parr at Mashable refutes with a few reasons he believes the services are useful. The use of URL Shorteners are somewhat out of the control of developers. They can’t know whether they’ll be used or not, and it’s really out of scope of their responsibilities. But the question then becomes, why don’t developers simply generate shorter URLs in the first place? Why don’t they tighten up the file system hierarchy and present something more reasonable to the user? Wouldn’t this, then, eliminate the need for URL shorteners (and the debate)? After all, very long file names and deeply nested hierarchies waste bandwidth and money. It turns out that there’s a lot more to consider before shortening URLs or condensing file system hierarchies. Last week’s discussion on the potential waste of bandwidth (and money) from excessively long URLs elicited several responses that were valid and insightful. LONG URLs AREN’T THE ONLY BANDWIDTH HOG The first reminded me that cookies are far more piggish in their consumption of bandwidth, with 4-5KB cookies stuffed in HTTP headers like so much stuffing at Thanksgiving being not unusual these days. That reminder is a good one, because it’s a trickier problem to solve than simply shortening a URL. In other words, that’s a discussion for another day – but we will have it eventually. Suffice to say there is a technical solution (a cookie jar) to the problem and we’ll dive into later because, well, it’s not really germane to this discussion. The second response was more directly related to the negative impact of automatically shortening URLs on business related functions and goals. Having targeted keywords in your url can provide a (slight) SEO boost to a pages relevance if it matches the query. http://www.seomoz.org/blog/11-best-practices-for-urls http://www.seomoz.org/article/search-ranking-factors Using a meaningful url structure can help with usability as it can provide context (or what Jakob Nielson calls information scent) for the site’s architecture. http://www.useit.com/alertbox/20030630.html This raises an important point to consider before deciding to reduce the length of file names. If the importance of “search” to your business is such that the loss of search traffic due to the removal of keywords from URLs and subsequent SED (Search Engine De-optimization) would be significant then perhaps URL shortening is not a good idea. If you’d notice it, in page ranking or traffic flow or however you’re measuring the effectiveness of SEO in your organization, then you’ll definitely want to consider this against the potential costs of long URLs. TRIM THE FAT ELSEWHERE If it’s the case that the business – and they really must be the final arbiter of such a decision because, well, business is the reason you’re serving up web sites and applications in the first place – is willing to absorb the cost of lengthy URLs in the name of SEO then you may need to look elsewhere for a reduction in bandwidth costs. As mentioned earlier, cookies are a common bandwidth hog. Unfortunately the implementation of a cookie-jar functionality is not as straightforward as it seems, though it is likely one of the places you can immediately trim fat that will have a significant impact on bandwidth use. If you can trim them down in the application, do so from the start – it’s always easier to develop an application with leaner cookies in mind than to rework the application later or implement a cookie jar on an intermediary. For example, perhaps it’s the case that your URLs need to contain certain keywords that are going to necessarily make them long. But do the supporting assets need to contain lengthy, descriptive file names? Do you really need to use /assets/monthly/images/this/is/ridiculous/myimage.jpg when something much shorter, and simpler, would suffice? There are plenty of supporting object types that make up a web page that are not as likely to affect SEO efforts one way or another. Consider the file system hierarchy you have in place for: Scripts CSS Images Commonly reused content elements (headers, footers, menus, etc…) And examine whether they are using unnecessarily lengthy URLs. The impact of reducing the length of supporting objects’ file names while maintaining a lengthy page URL in the name of SEO is likely more significant, because the ratio of objects to page URL is, on average, 50:1. A reduction even in the hierarchy naming structure can have a significant impact on bandwidth. These types of reductions are the reasons folks turn to compression. The trade off between performance and compression is lessened with the use of an intermediary capable of intelligent compression, but just turning on compression in general can certainly buy you quite a bit of bandwidth, which means you can leave the long URLs and file system hierarchies alone. TRUST ME, YOU’LL LOVE THIS SITE As is the case with URL shortening services, there are trust issues with shortened URLs. Rick-rolls are the least of your concerns when presented with a shortened URL for which the domain and page URL are obfuscated. Using an intermediary to shorten the URL alleviates some of the concern because the domain is clearly visible, but many folks want more information in the URL from which they can extrapolate information and make a decision whether to follow the link or not. The difference between shortening a URL yourself using an intermediary or network-side scripting (or just doing it as a matter of design during development) is that you aren’t obfuscating the URL, you’re just shortening it. The concerns with URL shortening services is partially due to the fact that they are misnamed. Oh, they shorten the URL, but in order to do so they completely obliterate any reference to the original and essentially present the user with gibberish. There is a big difference between obfuscation and shortening. While the latter can lead to the former, it doesn’t have to. Shortening schemes can be developed that adhere – at least in spirit – to usability guidelines such as those presented by Jakob Nielson and that don’t erode the user’s trust. It takes time to design such a hierarchy or scheme, but it can be done. SHORTEN, BUT DON’T OBFUSCATE Consider that programmers have long used such techniques that allow other developers to identify variable names “at a glance”. Hungarian notation is one such technique. By prefixing variable names using a codified system of identifiers, developers can easily distinguish a short integer (sNum) from a long integer (lNum) and so on. So perhaps what we need is to develop a “Hungarian Notation” for web sites; something that’s simple, easy to learn, and can be used as identifying information in a URL so that we can simultaneously shorten URLs (at least by reducing the hierarchy) while maintaining the business’ need for SEO.Using Resource Obfuscation to Reduce Risk of Mass SQL Injection
One of the ways miscreants locate targets for mass SQL injection attacks that can leave your applications and data tainted with malware and malicious scripts is to simply seek out sites based on file extensions. Attackers know that .ASP and .PHP files are more often than not vulnerable to SQL injection attacks, and thus use Google and other search engines to seek out these target-rich environments by extension. Using a non-standard extension will not eliminate the risk of being targeted by a mass SQL injection attack, but it can significantly reduce the possibility because your site will automatically turn up in cursory searches seeking vulnerable sites. As Jeremiah Grossman often points out, while cross-site scripting may be the most common vulnerability discovered in most sites, SQL injection is generally the most exploited vulnerability, probably due to the ease with which it can be discovered, so anything you can do to reduce that possibility is a step in the right direction. You could, of course, embark on a tedious and time-consuming mission to rename all files such that they do not show up in a generic search. However, this requires much more than simply replacing file extensions as every reference to the files must also necessarily be adjusted lest you completely break your application. You may also be able to automatically handle the substitution and required mapping in the application server itself by modifying its configuration. Alternatively there is another option: resource obfuscation. Using a network-side scripting technology like iRules or mod_rewrite, you have a great option at your disposal to thwart the automated discovery of potentially vulnerable applications. HIDE FILE EXTENSIONS You can implement network-side script functionality that simply presents to the outside world a different extension for all PHP and ASP files. While internally you are still serving up application.php the user – whether search engine, spider, or legitimate user – sees application.zzz. The network-side script must be capable of replacing all instances of “.php” with “.zzz” in responses while interpreting all requests for “.zzz” as “.php” in order to ensure that the application continues to act properly. The following iRule shows an example of both the substitution in the response and the replacement in the request to enable this functionality: when HTTP_REQUEST { # This replaces “.zzz” with ".php” in the URI HTTP::uri [string map {".zzz" ".php"} [HTTP::uri]] } when HTTP_RESPONSE { STREAM::disable If {[HTTP::header value "Content-Type"] contains "text" } { STREAM::expression "@.php@.zzz@" STREAM::enable } } One of the benefits of using a network-side script like this one to implement resource obfuscation is that in the event that the bad guys figure out what you’re doing, you can always change the mapping in a centralized location and it will immediately propagate across all your applications – without needing to change a thing on your servers or in your application. HIDE YOUR SERVER INFORMATION A second use of resource obfuscation is to hide the server information. Rather than let the world know you’re running on IIS or Apache version whatever with X and Y module extensions, consider changing the configuration to provide minimal – if any – information about the actual application infrastructure environment. For Apache you can change this in httpd.conf: ServerSignature Off ServerTokens Prod These settings prevent Apache from adding the “signature” at the bottom of pages that contains the server name and version information and changes the HTTP Server header to simply read “Apache”. In IIS you can disable the Server header completely by setting the following registry key to “1”. HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\DisableServerHeader If you’d rather change the IIS Server header instead of removing it, this KnowledgeBase Note describes how to use URLScan to achieve your goals. If you’d like to change the HTTP Server header in a centralized location you can use mod_security or network-side scripting to manipulate the Server header. As with masking file extensions, a centralized location for managing the HTTP Server header can be beneficial in many ways, especially if there are a large number of servers on which you need to make configuration changes. Using iRules, just replace the header with something else: when HTTP_RESPONSE { HTTP::header replace Server new_value } Using mod_security you can set the SecServerSignature directive: SecServerSignature "My Custom Server Name" These techniques will not prevent your applications from being exploited nor do they provide any real security against an attack, but they can reduce the risk of being discovered and subsequently targeted by making it more difficult for miscreants to recognize your environment as one that may be vulnerable to attack.
