Published on the 08/09/2020 | Written by Heather Wright

It’s there, it’s proliferating, so what are you going to do about it?…

The data age might be here, but most companies are drowning in rapidly proliferating dark data, rather than benefiting from an abundance of new insights.

A new survey of more than 2,250 business and IT leaders worldwide, including Australia, shows 66 percent of those surveyed reported that half or more of their organisation’s data is dark – up 10 percent on last year’s report – and that a similar number expect their data quantity to grow 5x by 2025, with a near proportionate increase in dark data expected.

For Australia, the projected growth is even higher at 5.9x – second only to the Netherland’s, at 7x, across the eight markets surveyed in The Data Age is here. Are you ready? report, from Splunk.

“Whether the technology creates data directly or not, each will have a significant impact on data volume and the data age.”

Indeed, the World Economic Forum says it’s estimated that by 2025, 463 exabytes of data will be created globally each day. That’s the equivalent of 212,765,957 DVDs per day.

For business leaders, that increase in data is threatening to overwhelm them with fifty-seven percent of those surveyed admitting the volume of data is growing faster than their organisation’s ability to keep up. That data volume is the primary challenge in tapping into dark data for four in five respondents.

And the feeling of overwhelm is likely exacerbated by the clear understanding most companies have of the importance of the data, with 81 percent saying it is very or extremely valuable to their organisation’s success. Its impact on innovation (75 percent) and cybersecurity (78 percent) are also similarly high rankers.

A recent Forrester report for Collibra, The Business Impact of Data Intelligent Management, found data centric businesses were 58 percent more likely to exceed their revenue goals.

But for companies flooded with data, harnessing the value in it is proving difficult.

Take, for example, retail. The report says while there’s a stronger than average recognition that data carries significant value for organisational success, 72 percent of retail organisations say half or more of their data is dark. In healthcare 62 percent say half or more of their data is dark.

Across the board, just 14 percent of companies said they were prepared for the fast-rising volumes of data.

As to the primary drivers of the data age, the report calls out six emerging technologies powered by, and contributing to, the exponential data growth: 5G, IoT, AI and machine learning, augmented and virtual reality, blockchain and edge computing.

“Whether the technology creates data directly or not, each will have a significant impact on data volume and the data age, allowing it to be put to work in ever more productive ways,” The Data Age is Here report says.

But it warns that too few companies appreciate what it means for dark data – namely that if companies don’t prepare well, the adoption of new technologies will worsen their dark data challenges and make it more difficult for them to capture the value and opportunities offered.

The report says there may be a very high penalty for companies burying their head’s in the sand, and tremendous rewards for those that meet the data age head-on.

In order to do so, it’s recommendations for companies include:

Put all your data to work and inventory all the data you hold, the sources of that data and the processes by which it is ingested, managed and analysed.

Build a data strategy from your business strategy. What data will drive, or derive from, your business growth? How will improvements to products or services incorporate new data and new technologies? “Recognising the changes that will result from your IT roadmap and your five-year business plan, incorporate a strategy for how you will be prepared to manage all that data and leverage it appropriately across your entire organisation.”

Outgrow the piecemeal approach and recognise that data must be managed holistically. “Your CFO doesn’t oversee some of your spending and similarly, management of your assets should span your entire organisation. Standardise tools processes and skill sets, so every person in your organisation has the right access to the right data.”