A handful of major enterprises around the world are now entering the “age of discovery” with their big data projects according to Professor Maria Garcia de la Banda, deputy dean of the IT Faculty at Monash University, which last week held a sold-out big data forum attracting speakers from big data pioneers such as Walmart, IBM and Telstra.

Prof Garcia de la Banda said that there were three stages in big data deployments – first when organisations started collecting data and issuing alerts, second when they started performing analysis, and finally the “age of discovery” where organisations used big data to support evidence-based decision making, asset management or testing.

Progress toward that third phase is, however, patchy. Technology analyst Gartner recently released the results of a survey of 302 of its international Research Circle members to find that while 73 percent have invested or plan to invest in big data initiatives by June 2016, up from 64 percent a year ago, few have progressed much beyond the pilot stage.

Issues still holding back big data progress include poor data quality, data hoarding by individual departments which fear having their performance judged and a lack of standardisation.

The standards issue has been highlighted in a study by the Centre for International Finance and Regulation which, for example, found that one corporate entity had been described in 140 different ways in a single database making comprehensive analysis of all the data associated with that entity almost impossible.

Access to skills has also been identified as a big data barrier. Professor Garcia de la Banda however said that Australian universities had a wealth of data analytics capabilities that could be tapped by the private sector and that from next year Monash would be offering a Masters in data science. Deakin University also offers big data related courses as does the University of South Australia, Macquarie and UTS among others.

Professor Garcia de la Banda said that; “Monash has the biggest group in data science in Asia Pacific. It’s silly to have such a strong team, but nobody knows about it.”

She said Monash was now collaborating with a growing number of enterprises in data science projects, but that some organisations still battled a data hoarding mentality. “There is a reluctance from some departments to share data, they are worried that they will be evaluated.”

Those “very very few” enterprises which had overcome these issues and were now entering the “age of discovery” with big data projects were able to migrate from using data as a trigger for individual alerts, toward developing “real insight to help with decision making, evidence-based asset management and evidence-based testing,” of new strategies, products and services.

“But our impression is that there are very, very few in this age of discovery,” said Professor Garcia de la Banda.