The more data you have, the better your business: Data Without Limits [TNW]

Data is the watchword. If you do not embrace data you may soon be obsolete.

“Data has changed a lot of areas. Science has been fundamentally changed by the collections of large amounts of data,” says Werner Vogels, CTO of Amazon.

Speaking at the The Next Web Conference in New York City, Vogels argues that today (big) data drives scientific research, startups, sports, consumer businesses, television, and industrial manufacturing.

He says that scientific data gathering began with observations then theory and models, and now we have facts.

Using the example of the Human Genome Project: the collaborative project aimed at sequencing every single aspect of the human genetic code that took 13 years and billions of dollars to complete — all driven by data.

“Huge datasets are being spat out, terabyte scales of datasets taking up significant computational power. Now the 1 000 Genome Project is being made available through the cloud so that all research can have access to all that data and work on it,” says Vogels.

The technologist reckons that in a consumer business — which is Amazon’s core focus — there is a lot of uncertainty around how consumers will behave.

As a consumer business, you have to pre-empt costumer behaviour, he says. You have to figure out how people will purchase and what they like, and where they consume your product. To answer these questions you need to have data. Data helps you personalise. It helps you recommend products for your consumers. It also allows you to perform better A/B testing.

Introducing a certain number of your customers to a new product or feature allows you to collect data around what customers might like.

Bigger is better, especially in real time

“The more data you collect the better service you will provide,” says Vogels. “Real time is the next step of harnessing big data. There is a tremendous drive towards real time, this is something we are seeing at Amazon.”

Vogels uses Netflix (one of the most high profile Amazon Web Services clients) as an example of a hotbed for real time data tracking.

The video streaming platform currently sees more than 3-million searches a day. Netflix needs to know what customers watching now and what their friends are watching. They need to track real time data. “This requires you to build a very different infrastructure so that you can process this data and not lose it,” says Vogels.

Industrial data is the future

There is a big move toward industrial big data, using big data applications in the gas and oil industry and healthcare.

“By tagging patiences with difference diseases in different rooms [and] through data collections, hospitals can handle more patiences and give better care. Data takes the place of intuition and guess work,” says Vogels.

The CTO says there is more to come with how data is being used but the key element is a move away from thinking that data can only be used from a consumer driven mindset. He argues that the best use of data is around efficiency and collaboration.