Getting Value from Big Data

By Ellie Fields 27 Mayo, 2011

Shawn Rogers, industry analyst at Enterprise Management Associates, just released a whitepaper on big data.

He is doing a webinar on June 1, so register if you're interested in hearing more from him.

Shawn is not just a critic. He's been a player: he has more than 19 years of hands-on IT experience, with a focus on Internet-enabled technology. He also co-founded the BeyeNETWORK. He's been watching as business intelligence has adapted to deal with ever-more complex environments, caused in large part by the explosion in the amount of data that companies now use.

Shawn's main advice for working with Big Data:

  • Use self-service solutions. When you're dealing with huge volumes of data, not every report can be anticipated and built months in advance. Users must be able to ask and answer thier own questions.
  • Don't forget about collaboration when working with big data. It's more effective than having lone analysts crunching numbers, and lets the data have more of an impact.
  • In-memory capabilities are critical to big data environments because they let people speed up and work with millions of rows of data on their PCs.
  • But the ability to connect directly to data is necessary too. Customers need to leverage fast databases if they have them.
  • Blending data from different sources solves some key big data scenarios.


Submitted by richardwesley (no verificado) on

Hi Ellie -

Noce white paper. I did want to make one comment about the emphasis on "in-memory" databases, though. The Tableau Data Engine is actually based on a newer idea called "memory-biased" (it's so new, a Google search will not turn up anything useful - I've only heard it at academic conferences). The idea behind memory biased databases is that they try to adaptively load data into RAM, so they can also work with larger data sets by paging intelligently. The TDE is a column store and has all the I/O benefits of that technology (reduced reads, compressed storage and operations on compressed data) but because it is memory-biased, you don't have the problem of trying to cram all the data into a footprint that will run on everyone's desktop machine. This is especially important in the server case because most publishers will not know the exact hardware description of their server. This simplifies the lives of both producers and consumers and makes Tableau easier to use. It also blurs the line between "fast databases" and "in-memory technology" since the TDE is using the same basic technology as these fast databases that also try to stay in-memory as much as possible for maximum speed.

non-humans click here