What big data can’t do: the limits of computers and analytics

Big data is all about the software — but software has it’s limits. In a recent column, “What Data Can’t Do“, New York Times Op-Ed columnist David Brooks breaks down the limitations of computers. “Data needs direction” he says. That is to say, without the guidance of someone providing real context or value, it’s impossible for data to distinguish between the importance of viral memes or a valuable masterpiece for example.

Some of these limitations are those that are considered mechanistic, as opposed to the inherently humanistic values such as intuition, context and creativity. For example a computer’s inability to “capture your devotion to the childhood friends you see twice a year” or its inability to comprehend social context. The power of storytelling. We humans are able to weave together “multiple causes and multiple contexts” which makes it easier to gain perspective and direction.

More limitations involve physical obstacles such as intellectual property or simply the act of physically mining and storing vast amounts of data could be tedious. In a recent article, “Elusive big data: The thing, and not the thing” by The Economist, the question of defining big data as well as its asset values is discussed. As it points out, “copyright law means these sort of ‘meta mining’ studies require researchers to buy access to each article, just as if it were the 19th century and a pair of human eyes were to read it.” So the legality of obtaining data is another big obstacle.

Another clear example where the boundaries were pushed, was when Google was accused of stealing private Wi-Fi data using its Street View cars that were supposedly only taking panoramic photos of their surroundings. The company faced legal repercussions in especially Europe, and was fined by France. Although Google managed to hold its own in terms of legal whitewash and reputation, there are still the ethicalities to be considered.

By having too little data, certain values are excluded and therefore would end up being inconclusive or misleading. On the other hand, having too much data would become confusing and vague. As David Brooks says in his article, “the haystack gets bigger, but the needle we are looking for is still buried deep inside.” So basically, although big data (and all sorts of data for that matter) can bring about outstanding predictions, there will always be room for scepticism. As needs be, the creativity, ethics, perspective and context are what will distinguish us from the computers.