Real Time Isn’t Just for Real Time

Not surprisingly, in our work with beta customers and in numerous conversations with enterprise IT leaders, the topic of real time data is a constant theme. After all, real time data is what we do. Our initial conversations usually revolve around very specific real time use cases. And then it becomes clear how much more we can help with. Because real time data management isn’t just for “real time” anymore.

That might sound counterintuitive, or even plain silly. But consider this – as high volume, variegated data environments become the rule, more than just data stores and analytic tools have to change. The core data infrastructure for processing, moving and managing data has to change. Data “grunt work” like ETL, batch processing, quality filtering, data enrichment, legacy integration, and data operations overall has to change. Manual or high touch operations projects aren’t just expensive and painful – they can’t keep pace with data. And if every new data initiative needs custom programming before it gets off the ground, then aren’t you just boxing yourself into a new set of data problems and becoming beholden to a new set of specialized, hard-to-hire skills?

That’s why real time data management matters.  When real time processing is the requirement, the opportunities to simplify and gain flexibility are rich. Treating all data as data in motion results in flexible information streams, ready to be combined, correlated, filtered and routed in a responsive manner. Focus becomes on “feeds and speeds” rather than schema and stored processes. You make decisions on what to do with data instantaneously, and continually evolve your practices. The painful drudgery of data management goes away. And if you do it right, this can all be achieved through an agile configuration framework, without the need for deploying and maintaining custom code.

Granted, most organizations are going to move toward real time data to suit specific real time use cases. We’re happy about that and look forward to helping them. But we’ll continue to spread the word that real time isn’t just for real time anymore.

Thoughts on “Big Data’s Second Act”

A recent article by Holly Moore Haynes of Silver Lake Sumeru on CNNMoney is an interesting read for a few reasons. First and foremost, she points out something too many miss – that “Big Data” isn’t about any one emerging set of tools displacing everyone else, rather, “we are witnessing the evolution of enterprise intelligence.” She calls it “Big Data’s Second Act.”  We agree with that sentiment wholeheartedly, and dig the Second Act concept. Not only is data “bigger” (3 V’s and all that), but the expectation that companies need to have responsive, reactive data driven practices at the core of their operations is becoming a must-have, not an esoteric goal toward which to strive.

That said, we think she left a few things unsaid that are really important. While we agree with Ms. Haynes that intelligence applications for business users will be really key, you can’t overlook the importance of core infrastructure technology changes that will be needed to accommodate this Second Act.

In one of the sources Ms. Haynes references, IDC makes some important observations, some of which are laid out in this press release, and some of which you’ll need to get your own copy of the full report:

  • While software and services will constitute a larger share of overall revenue, growth rates for infrastructure technologies is projected to be higher.
  • “Big Data in Motion” and the high speed messaging technologies needed to support it are one of the key aspects of Big Data growth.
  • There’s a Big Data talent gap on both the IT and analytics expertise sides, representing a real opportunity for appliance offerings that make it easy to deploy and utilize next generation concepts in the data infrastructure space.

The bottom line is that there’s more than just some new data stores and cool BI tools out there on the horizon. There are fundamental changes to the core data management, data integration and enterprise information access processes on the way. Innovators who are poised to offer new, easy to deploy platforms to enable Big Data’s Second Act have a bright future, and we’d like to think we’re one of them.