An Aristotlean Approach to Devops and Infrastructure Integration

Aristotle’s famous four questions can be applied to infrastructure integration as a means to determine whether an API or SDK is the right tool for the job.

While bouncing back and forth last week with Patrick Debois on the role of devops , vendors and infrastructure integration he left a comment on the blog post that started the discussion that included the following assertion:

On a side note: vendors should treat their API's as first class citizens. Too often (and i personally feel iControl too) API's expose a thinking model based upon the internal implementation of the product and they are not focused on using it from a business perspective. Simplicity to understand Load balancer -> create_network, ... vs. understanding all the objects. There is real work to be done there!
Object Oriented languages are great, but sometimes a scripted language goes around easier.

Which was distilled down to: APIs need to be more than a service-enabled SDK. Nothing new there, I’ve made that assertion before (and so have many, many, many other pundits, experts, and architects). What Patrick is saying, I think, is that today it is often the case that an infrastructure developer needs not only understand the concept and relationship between a load balancer, the network, and the resources it is managing, but each individual object that comprises those entities within the SDK. In order to create a “load balancer”, for example, you have to understand not only what a “load balancer” is, but the difference between a pool and a member, monitoring objects, virtual servers, and a lengthy list of options that can be used to configure each of those objects. What’s needed is an operationally-focused API in addition to a component and object-focused SDK.

One of the failings of SOA was that it too often failed to move beyond service-enablement into true architecture. It failed to adequately represent business objects and too often simply wrapped up programmatic design components with cross-platform protocols like SOAP and HTTP. It made it easier to integrate, in some ways, and in others did very little to encourage the efficiency through re-use necessary for SOA to make the impact it was predicted to make.

Okay, enough of the lamentation for SOA. The point of an API – even in the infrastructure world – should be to abstract and ultimately encapsulate the business or operational tasks that comprise a process. Which is a fairly wordy way to say “an API call should do everything necessary to achieve a single operational task.” What we often have today in the infrastructure world is still a service-enabled SDK; every function you could ever want to perform is available. But they are not aggregated or collected into discrete, reusable task-oriented API calls. The former are methods, the latter are process integration and invocation points. Where SOA encapsulated business functions, APIs for infrastructure encapsulate operational tasks.

That said, the more I thought about it the more I realized we really do need both. Basically I think what we have here is a “right tool for the job” issue. The question is which tool is right for which job?

LET’S ASK ARISTOTLE

Illustration: Toothpaste for Dinner

Aristotle (384 – 322 BC) is in part known for his teleological philosophy. He more or less invented the rules of logic and was most certainly one of the most influential polymaths of his era (and likely beyond). In other words, he was really, really smart.

One of his most famous examples is his four causes, in which four questions are asked about a “thing” as a means to identify and understand it. These causes were directly related to his biological research and contributed greatly to our understanding for many eons about the nature of life and animals.

MATERIAL CAUSE: What is it made of?
FORMAL CAUSE: What sort of thing is it?
EFFICIENT CAUSE: What brought it into being?
FINAL CAUSE: What is it for?

These may, for a moment, seem more applicable to determining the nature of a table; a question most commonly debated by students of philosophy late at night in coffee shops and not something that broods on the mind of those who are more concerned with meeting deadlines, taking out the garbage or how to make it to the kids’ basketball game if that meeting runs late. But they, are in fact, more applicable to IT and in particular the emerging devops discipline than it might first appear; especially when we start discussing the methods by which infrastructure and systems are integrated and managed by such a discipline.

There’s a place, I think, for both interface mechanisms – API or service-enabled SDK – but in order to determine which one is best in any given situation, you’ll need to get Aristotlean and ask a few questions. Not about the integration method (API, SDK) but about the integration itself, i.e. what you’re trying to do and how that fits with the integration and invocation points provided by the infrastructure.

The reason such questions are necessary is because the SDK provides a very granular set of entry points into the infrastructure. The API is then (often) layered atop the SDK, aggregating and codifying the specific methods/functions needed to implement a specific operational task, which is what an infrastructure API should encapsulate. That means it’s abstracted and generalized by the implementers to represent a set of common operational tasks. The API should be more general than the SDK. So if your specific operational process has unique needs it may be necessary to leverage the SDK instead to achieve such a process integration. The reason this is important is that the SDK often comes first because inter-vendor and even intra-vendor infrastructure integration is often accomplished using the same SDK that is offered to devops. The granularity of an SDK is necessary to accomplish specific inter-vendor integration because it is highly specific to the vendors, the products, and the integration being implemented. So the SDK is necessary to promote the integration of infrastructure components as a means to collaborate and share context across data center architectures.

Similarly, the use-case of the integration needs to be considered. Run-time (dynamic policy enforcement) is different beast than configuration-time (provisioning) methods and may require the granular control offered by an SDK. Consider that dynamic policy enforcement may involve the tweaking of a specific “application” to for one response but not another or in response to the application of a downstream policy. An application or other infrastructure solution may deem a user/client/request to be malicious, for example, and need the means by which it can instruct the upstream infrastructure to deny the request, block the user, or redirect the client. Such “one time” actions are generally implemented through specific SDK calls because they are highly customized and unique to the implementation and or solutions’ integration.

CONCLUSION: Standardized (i.e. commoditized) operational process: API. Unique operational process: SDK.

FLEXIBILITY REQUIRES OPTIONS

Because the very nature of codifying processes and integrating infrastructure implies myriad use-cases, scenarios, and requirements, there is a need for flexibility. That means options for integration and remote management of infrastructure components.

We need both SDKs and APIs to ensure that the drive for simplicity does not eliminate the opportunity and the need for granularity in creating unique integrations supporting operational and business interests.

Many infrastructure solutions today are lacking an SDK (one of the reasons cloud, specifically IaaS, makes it difficult to replicate an established data center architecture), and those with an SDK are often lacking an API. Do we need service-enabled SDKs? Yes. Do we need operational APIs? Yes. An API is absolutely necessary for enterprise devops to fully realize its goals of operational efficiency and codification of common provisioning and deployment processes. They’re necessary to create repeatable deployments and architectures that reduce errors and time to deploy. Simply implementing an API as a RESTful or scripting-friendly version of the SDK, i.e. highly granular function calls encapsulated using ubiquitous protocols, is not enough.

What’s necessary is to recognize that there is a difference between an operational API and a service-enabled SDK. The API can then be used to integrate into “recipes” or what-have-you to enable devops tools such as Puppet and Chef that can be distributed and, ultimately, improved upon or modified to fit the specific needs of a given organization. But we do need both, because without the ability to get granular we may lose the flexibility and ultimately the control over the infrastructure necessary to continue to migrate from the traditional, static data centers of yesterday toward the dynamic and agile data centers of tomorrow. Without operationally commoditized APIs it is less likely that data centers will be able to leverage Infrastructure 2.0 as one of the means to bridge the growing gap between the cost of managing infrastructure components and the static budgets and resources that ultimately constrain data center innovation.