The Big Data revolution: 5 startups mining the trend

Big Data is, well, big.

In the information age, companies are literally drowning in vast oceans of data. Petabytes of data are being gathered everywhere in the digital age at a furious rate, and by a variety of devices.

Websites and devices that are collecting data on a large scale include ubiquitous information-sensing mobile devices, cameras, microphones, RFID readers, and wireless sensor networks. Most large corporates are sitting on mines of data by virtue of their everyday operations and vast networks. Social networks have also become information goldmines, generating massive amounts of user data.

Harvard Business Review refers to the “Social Data Revolution“, a result of the vast amounts of real-time and archive information emanating from people using and interacting on social networks. Sociologists and researchers are using this social data to “nowcast” and forecast trends such as unemployment, flu outbreaks, travel spending and political opinions in a way that is faster, more accurate and cheaper than government reports or polls.

Research House Gartner recently featured Big Data on its 2011 Hype Cycle Special Report, calling it the next big thing.

Big Data is important to companies because this information can be analysed and filtered and used to solve and understand problems from spotting business trends, to preventing diseases, and combating crime. It’s happening on a grand scale, because much of it is collected automatically via the devices we use.

Large businesses have sought to mine their data to better understand and target their customers for some time. More information, after all, should mean that you have a more accurate means of tracking what your customers like about your product and the ways in which you deliver it.

Companies have generally been weak, however, at using and interpreting their vast amounts of data, because they lack the tools or knowledge on how to mine and sift through all this information. It’s no wonder then that there has been startup activity in this area around companies offering solutions.

The thing about all this data is useless unless it is mined and interpreted with the right tools. Memeburn takes a look at five Silicon Valley startups and how they’re looking to use Big Data:

1.Rocket Fuel,a digital media targeting company, is looking at the digital advertising business and how data mining is helping to improve results for advertisers. The company applies a wide variety of behavioural, contextual and other measures to improve advertising results.

The company uses various techniques to serve the right ad to the right person and to make those decisions in milliseconds — a tough job especially when you have to query multiple, very massive databases.

2. VirtuOZ provides a virtual agent that is used to provide support to a client’s customers by leveraging advanced computer linguistics. The company claims that the virtual agents are so good that people sometimes ask them out on a date even though it’s clearly labelled that they are conversing with a software program.

This might be a new form of Turing Test: Instead of trying to emulate a human, computers are able to emulate a dateable human! Maybe Match.com should offer a virtual partner to its customers to practice real life encounters.

3. Panasas provides storage for high performance data intensive applications. File sizes run into the terabytes just for one file. Customers are dealing with massive amounts of data, such as those in oil, and life sciences.

4. Coupa aims to help companies save money on many of their business operating expenses, such as buying shelving, or sourcing local goods. It’s a competitor to Ariba. The company compares similar businesses and develops a set of best sourcing practices that can add up to substantial savings.

5. Ticketfly is trying to take on online ticketing giant Ticketmaster. The company claims to offer event promoters and venues next-generation ticketing and powerful website and social marketing tools, saving them time and money.

One of Ticketfly’s most potent tools in attracting promoters and venue owners is a comprehensive artist database, where clients share artist promotional materials and info such as photos, bios, website URLs, YouTube videos, and Facebook and Twitter links, so you don’t have to enter that stuff each time.

With additional reporting by Tom Foremski

Image: Center for Image in Science and Art