Big Data Gets a Real-Time Face-Lift

big-data[Cross posted from my JackBe blog]

Read through any technology publication, website or blog and I dare you to tell me you didn’t come across a Big Data story. Big Data is hyped. And that’s not the first time I’ve said that. In fact, the last time I addressed this topic I declared that one of the biggest problem with Big Data is the disconnect between how much it’s talked about and how much it’s understood. Cindi Howson of BI Scorecard recently hit the nail on the head by drawing an integral connection between Big Data and BI. She dared to say—against advice from a fellow Strata Conference attendee that it might be considered blasphemy—that Big Data is more than Hadoop. She couldn’t be more right.

And Gartner’s Hype Cycle for Emerging Technologies 2012 prominently featured Big Data and linked it to various other emerging technologies, with Big Data just entering the “peak of inflated expectations” and predicts it will take 3-5 years to reach the “plateau of productivity.” But don’t just walk away. Big Data as much as it is hyped, is also real and it is here to stay and is making new inroads every day.

My colleague Dan Malks recently shared some of his 2013 BI predictions and now I would like to follow it up with a few of my own. Yes, I’m focusing on Big Data. But it also turns out—in a great minds think alike kind of way—that real-time intelligence will be a critical element to navigate the way in which we consume, analyze and leverage data of all types and from all places, for instant decision-making. Here goes…

  • Big Data will go more real-time, for real. Forget about batch processing. Data that is stashed away without the ability to understand it on a continuous real-time basis is just another old data warehouse, albeit a Big Data warehouse.
  • Business users will want direct access to insights derived from Big Data, again in real-time, and anywhere. Here I am talking about the kind of tooling and processes needed to make it easy for the business decision makers to have insights delivered to them in real-time from analyzing Big Data warehouses— especially on smartphones and tablets.
  • Big Data will get further segmented into Big Data, Big Small Data, Fast Big Data, and Fast Small Data, along the dimensions of speed/velocity, size and volume. The capabilities (technical and non-technical) needed to handle these are different for different segments.

That’s my two cents. What do you think will happen next year? Share your predictions with us in the comments section or on Twitter using the hashtag #BIin2013 for consideration in an upcoming post.

Published by


Cofounder & CTO @