Showing posts with label Data Warehousing. Show all posts
Showing posts with label Data Warehousing. Show all posts

Saturday, February 23, 2013

Big Data, Big Confusion?

Big Data
Big Data (Photo credit: Kevin Krejci)
The Problem with Our Data Obsession
however objective data may be, interpretation is subjective, and so is our choice about which data to record in the first place. While it might seem obvious that data, no matter how “big,” cannot perfectly represent life in all its complexity, information technology produces so much information that it is easy to forget just how much is missing..... life is messy, and not everything can be abstracted into data for computers to act upon
There are obvious limitations to Big Data, but overall it is a force for good. The solution to Big Data blind spots seems to be even more Big Data. No?
Enhanced by Zemanta

Tuesday, January 29, 2013

Long Data Is Still Big Data

Image representing Hadoop as depicted in Crunc...
Image via CrunchBase
You add the time dimension to Big Data and you get Long Data. Long Data is still Big Data.

Stop Hyping Big Data and Start Paying Attention to ‘Long Data’

crunching big numbers can help us learn a lot about ourselves. ..... But no matter how big that data is or what insights we glean from it, it is still just a snapshot: a moment in time. ..... as beautiful as a snapshot is, how much richer is a moving picture, one that allows us to see how processes and interactions unfold over time? ..... many of the thi

Structure of Evolutionary Biology - Blue
Structure of Evolutionary Biology - Blue (Photo credit: Wikipedia)
ngs that affect us today and will affect us tomorrow have changed slowly over time ...... Datasets of long timescales not only help us understand how the world is changing, but how we, as humans, are changing it — without this awareness, we fall victim to shifting baseline syndrome. This is the tendency to shift our “baseline,” or what is considered “normal” — blinding us to shifts that occur across generations (since the generation we are born into is taken to be the norm). ..... Shifting baselines have been cited, for example, as the reason why cod vanished off the coast of the Newfoundland: overfishing fishermen failed to see the slow, multi-generational loss of cod since the population decrease was too slow to notice in isolation. ..... Fields such as geology and astronomy or evolutionary biology — where data spans millions of years — rely on long timescales to explain the world today. History itself is being given the long data treatment, with scientists attempting to use a quantitative framework to understand social processes through cliodynamics, as part of digital history. Examples range from understanding the lifespans of empires (does the U.S. as an “empire” have a time limit that policy makers should be aware of?) to mathematical equations of how religions spread (it’s not that different from how non-religious ideas spread today). ...... building a clock that can last 10,000 years .... the 26,000-year cycle for the precession of equinoxes ...... Just as big data scientists require skills and tools like Hadoop, long data scientists will need special skillsets. Statistics are essential, but so are subtle, even seemingly arbitrary pieces of knowledge such as how our calendar has changed over time
Enhanced by Zemanta

Tuesday, July 31, 2012

Big Money In Big Data

Big Data, The Moving Parts: Fast Data, Big Ana...
Big Data, The Moving Parts: Fast Data, Big Analytics, and Deep Insight (Photo credit: Dion Hinchcliffe)
I do think there is big money in Big Data. A lot of people do. But here is a disagreeing thought.

Is There Big Money in Big Data?
Peter Fader says a flood of consumer data collected from mobile devices may not help marketers as much as they think. ..... Few ideas hold more sway among entrepreneurs and investors these days than "Big Data." The idea is that we are now collecting so much information about people from their online behavior and, especially, through their mobile phones that we can make increasingly specific predictions about how they will behave and what they will buy. ..... what was going on 15 years ago with CRM (customer relationship management) .... ask anyone today what comes to mind when you say "CRM," and you'll hear "frustration," "disaster," "expensive," and "out of control." It turned out to be a great big IT wild-goose chase. And I'm afraid we're heading down the same road with Big Data ..... many "big data" people don't know what they don't know. ..... the still-powerful rubric of RFM: recency, frequency, monetary value. .... Ask anyone in direct marketing about RFM, and they'll say, "Tell me something I don't know." But ask anyone in e-commerce, and they probably won't know what you're talking about. ...... Chartists are looking at the data without developing fundamental explanations for why those movements are taking place ..... Among financial academics, chartists tend to be regarded as quacks. But a lot of the Big Data people are exactly like them. They say, "We are just going to stare at the data and look for patterns, and then act on them when we find them." In short, there is very little real science in what we call "data science," and that's a big problem. .... the more data we have, the more false confidence we will have
If his point is that collecting Big Data is not enough, you also have to make sense of it. I agree. But in my definition the whole idea behind Big Data is that of course you are going to make sense of it.

One part where I agree is that Big Data enthusiasm will have plenty of accompanying froth.

What he is saying is making sense of data is going to be more important than collecting data. I agree. But that is what I thought Big Data was all about. To me it never was simply collecting.
Enhanced by Zemanta