big data exabyte terrabyte or gigabyte in very large data set cloud computing storage

What is big data? Do I have it? And, what do I do with it?

Big data defined

Big data is a blanket term used to describe large and complex data sets. It’s a popular phrase used to describe the constant growth and availability of data, both structured and unstructured.

Gartner analyst Doug Lancy in his 2001 research report entitled “3D Management: Controlling Data Volume, Velocity, and Variety” defined data growth challenges and opportunities as being three-dimensional:

1) Volume (the increasing amount of data)

2) Velocity (the speed of data in and out)

3) Variety (the range of data types and sources)

The industry still uses this “3Vs” model for describing big data.

Do I have big data? And if so, what do I do with it?

Most likely, you do. Organizations everywhere have big data of all shapes and sizes. Specifically, big data relates to any data creation, storage, retrieval, and analysis of data sets so large and complex that they are impractical to manage with traditional software tools. Sound familiar?

A few examples of businesses in the private sector that gather big data:

  • eBay – has two data warehouses at 7.5 petabytes and 40 petabytes as well as a 40 petabyte Hadoop cluster for search, consumer recommendations, and merchandising.
  • Amazon.com – handles millions of back-end operations every day, as well as queries from more than half a million 3rd-party vendors.
  • Facebook – handles over 50 billion photos from its users.

In fact, the sheer volume of business data worldwide, across all companies, doubles every 1.2 years, according to an October 2013 article entitled “Leading Priorities for Big Data for Business and IT”.

Why should big data matter to you?

The real issue is not that you have acquired large amounts of data; it’s what you do with the data that counts. Organizations should be able to take data from any source, harness the relevant data, and analyze it to find areas where they can make smart business decisions. By looking at big data, you can see areas where your company might be able to reduce costs, grow revenue, reduce time to market, customize product offerings, and realize new sources of competitive advantage.

What big data does your organization collect, and how will you plan to make that data work for you in the future?