Published on the 10/09/2013 | Written by Newsdesk
More than 14,000 Australian organisations currently use Amazon Web Services’ Sydney infrastructure – a growing cohort of them for big data processing…
Organisations with static inelastic data centres could find themselves limited in terms of their ability to perform big data analysis as their data collections continue to grow. It’s one reason why organisations such as Suncorp, BigCommerce and Fairfax are turning to the cloud according to Dr Matt Wood, global general manger for data science for Amazon.
Wood who was in Sydney last week for meetings with clients said that, “If you have a static inelastic data centre, then you tend to frame questions based on the resources available. In a utility environment the constraints fall away and you are able to ask the questions that add value for the organisation.”
Wood said that a common theme among enterprises conducting big data analysis was to develop a better understanding of their customers through what he described as “passive collaboration”. He offered the example of a video game designer which could analyse how people used the game to determine which elements were particularly popular and then use that insight to steer direction of future games.
Most current corporate infrastructure, he argued, had been designed to cope with data collections from a time when collecting information was expensive and the rate-limiting element in growing data reservoirs. Now he said that the high throughput of data coming from mobile devices and social networks was placing enormous pressure on enterprise infrastructure as organisations’ data collections grew from gigabytes to terabytes to petabytes.
This he said was leading organisations to explore cloud-based data analysis.
Although Gartner has forecast that the new rate limiting factor for big data exploitation is the lack of data analytics skills – it puts the global shortfall as high as 4.4 million by 2015 – Wood disputed this, saying that, “data scientists are just computer scientists whose role is to interpret the data”.
It’s a convenient notion given that many computer scientists are finding their enterprise roles under threat from the rise of cloud computing and software-as-a-service. However the skills required for data analysis, interpretation and exploitation or application do remain subtly different.
