Data Diversity: a leading UK tech writer posts…

To change things up a little TechView has invited tech luminary Adrian Bridgwater to share a few thoughts on Big Data and related management issues.  More about Adrian can be found at the end of this post.  He can be found in many places online, including  Twitter and at ComputerWeekly.com.  Thanks for contributing Adrian!



Data Diversity: protecting the species inside the “infinite variety”

Big data, complex event processing, multi-core systems and real time events are spirally upwards and outwards into a combined vortex of increased interconnectivity with the resultant data now traveling across a multiplicity of network protocols -- so are we headed for a fall, or do we have the application and services management layers in place to be able to survive in this new ecosystem?

Let’s approach this question organically and ecologically.

I first came across the term “infinite variety” while watching the naturalist David Attenborough present the BBC series Life On Earth, but I believe its actually a turn of phrase attributable to Shakespeare’s Antony and Cleopatra. Either way, the rise of terms like so-called “big data” have come to the fore as we now see data streams and data sets evolving into an (almost) infinite number of new forms.

Data is interconnecting and (in some cases) self-populating through automation controls. The question now is whether we have taken the trouble to prepare and architect for this new data-enriched landscape that we find ourselves in.

After all, huge swathes of data without management and analysis functions… are just huge swathes of data. The “value” fulcrum tips in our favour when and only when we can evidence some degree of control and power over the data before us.

But it’s not just about data management is it? The problem here (somewhat predictably perhaps) comes back to data security in the first instance.

You may have already read about (Chris) Hoff’s Law on F5 DevCentral which states that: “If your security practices suck in the physical realm, you’ll be delighted by the surprising lack of change when you move to cloud.”

We can further postulate that the corollary or upshot of Hoff’s Law is that if your data security practices DO NOT suck in the physical realm, you’ll most likely be concerned by the inability to continue that practice when you move to cloud and the world of big data.

So tying our two themes together here: we have the “infinite variety” of human data usage and its constantly evolving new data streams across new form factors and devices as they emerge -- and we also have security and the need to protect our species and the lifeblood and food-source (in this case data) that it thrives upon.

In the real world we arm ourselves with defences and build a stronghold inside which we can live and function and go about our normal functions. In the virtual world, we should perhaps treat encryption techniques and anti-malware controls as body armour, but recognise that we still need deeper controls such as software appliances with functionality like multi-tenant support to really batten down the hatches.

From this point, we can explore new ecosystems and planets and populate our world further without fear of disease or plague -- or (if that’s one analogy too far for you) at least, we can keep our data locked down at get on with life.

 

Adrian’s bio:

Adrian Bridgwater is a freelance journalist specialising in cross platform software application development as well as all related aspects of software engineering and project management.

Adrian is a regular writer and blogger with Computer Weekly, Dr Dobbs Journal and others covering the application development landscape to detail the movers, shakers and start-ups that make the industry the vibrant place that it is.

His journalistic creed is to bring forward-thinking, impartial, technology editorial to a professional (and hobbyist) software audience around the world. His mission is to objectively inform, educate and challenge - and through this champion better coding capabilities and ultimately better software engineering.