News and opinion | Georgina | 20 August 2014

You can’t move for articles about big data at the moment – and this has been the case for some time now. What began as a hyped-up concept has actually turned into reality for many organisations. The million-pound question, though, is ‘what exactly IS this reality?’.

Defining big data

The very definition of big data might make a lot of people automatically assume it doesn’t apply to them. Gartner says in its usual succinct way that big data is “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”.

IBM makes this a little easier to understand by telling us that it’s “being generated by everything around us at all times” and requires “optimal processing power, analytics capabilities and skills”. The upshot of this is that if you find any part of your business generating such large quantities of data that you need someone or something to help you process it all, you’re effectively dealing with big data.

Big data in the real world

Here’s an example of a business scenario relating to big data. An online retailer of accessories receives large quantities of data relating to customers’ locations due to the information that is generated whenever orders are placed.

The retailer can store this on a database to access it when orders are being packed and posted, but the company might also invest in some form of analytics to get more use from the information.

For example, address details could be broken down by country, region and city and compared with local sales data for umbrellas and weather information to find out which parts of the world buy the most umbrellas, when sales tend to occur and even how this compares with the typical climate in those places.

The resulting data could then be used to predict future sales of umbrellas and tailor marketing campaigns depending on the locale and time of year, as well as to ensure stocks are kept in the right months.

A note on unstructured data

The sales data mentioned above is known as structured data, but the benefits of harnessing big data can also extend to unstructured data, such as posts on social media sites. This type of information isn’t all stored in one place for easy access, but there are tools that can still pull this data from various locations for your analysis-related needs.

You might not think this is entirely relevant to your business, but estimates from Gartner suggest that as much as 80 per cent of organisations’ information assets are made up of unstructured data.

“Blogs, tweets, comments and ratings are a reflection of the current state of public sentiment at any given point in time,” says Darin Stewart, research director at Gartner. “More traditional web content such as news articles, product information and simple corporate informational web pages become an extension of internal research when tamed.”

Can small businesses really do anything with big data?

Yes, they can, according to Andrew Jennings of data analytics firm FICO. He told BBC News earlier this year that while one-man bands are probably not going to be able to directly apply big data to their operations, they can instead “access services that rely on big data analytics, such as maps and weather forecasts”.

Matt Asay of MongoDB added: “These days, all businesses, whether small or large, have to keep track of their customer data and metadata so they can respond quickly to what their customers are doing and push messages to them.

“The need to deal with modern, messy data exists whether you’re building an app for 20 people or 20,000 people.”

The flipside of big data

Of course, it’s not all sales booms and increased revenues when it comes to big data. Handle things in the wrong way and your organisation could end up bearing the brunt of a significant costs. This might happen if you lack the manpower to handle the analytics tools processing your big data, meaning all the money spent on this software goes to waste because no one has the time to use it, or if you try to do too much too soon.

As former IBM data scientist Marco Visibelli recently concluded in an article for Wired: “Don’t commit your time and resources without first establishing whether your project will really be profitable. Only fools rush in.”