The global pandemic has driven home the fact that data is vital to the success of every organisation. Companies across Australia and New Zealand (ANZ) are realising the importance of scaling and growing their analytics capabilities, something that has become even more critical in the ‘covid’ era.

In fact, 67 percent of businesses across A/NZ now view BI and analytics programs as more or much more important to business operations now than before the pandemic.

But what does it take as a data engineer to keep up with the innovation needs of today’s fast-paced businesses? And are we expecting too much from today’s analytics talent?

‘The solvers of all business problems’
Despite widespread recognition of the value of data in business, budgetary constraints, skills challenges, education around data, and the best use of employees’ time, remain key challenges. There is also the risk of data scientists being seen as ‘the solvers of all business problems’ – and they become overloaded with irrelevant questions.

“A number of facets are key to successful data and analytics, and they’re all very much related to people.”

In fact, what we’re seeing more than ever is the data analyst, and the data analytics teams, suddenly becoming the ‘perceived source of all business decisions’, while the CEO, the CTO, the CFO are just ‘hammering them’ with questions.

Yes, there are certainly a few questions that are really important and require a good analyst or data scientist to help solve. But there’s also a bunch of questions that can be solved by the questioner going back into the BI tool and pulling up their own chart.

In reality, this skill a data engineer has is a limited resource, so companies have to start thinking about how to prioritise and send the right questions, the most impactful and the hardest question, to the data team.

It’s important to match the right contributor to the right type of question and not just ‘throw’ data scientists into a lot of basic reporting. Companies should also not use data analysts to try to solve massive predictive problems or use either one of them to solve low-level tasks like routine group and count.

In this regard, understanding the role of data for business success is key while making the most of the talent and skills a data analyst has within a business. The right BI and analytics tool can make a good difference, however business must also consider adequate resourcing and skills to best utilise these tools.

Avoid the pressure to ‘attack data with one tool’
There’s pressure on data teams to try and get all data into the same format, and to ‘attack it with one tool’.

There are two challenges in this. Firstly, the technical challenge – not every question is answered with one particular tool, there’s no one analytics that will always work. Businesses need a whole tool chain that changes based on the question you’re asking.

Secondly, there’s the over standardisation of the data culture and the people. When you teach a single methodology for analysing data, for approaching questions and getting access to the answers, and what kind of answers you’re looking for, you end up with single results.

The more you over-standardise, the more you force every answer to look the same. It’s when you have unusual approaches, or two or three approaches to the same question, which allows you to really have confidence in the data and you don’t get that with over standardisation.

Interestingly, Dr Alex Antic, voted as one of the top 10 analytics leaders in Australia by IAPA, also aptly points out, in order to not over-standardise it’s important to have a suite of tools, an open-source framework, as well as any vendor-specific tools,

“This also gives analysts scope to be able to use the tools they prefer, depending on the particular problem at hand,” he says.

Critical skills of a modern data engineer
According to Dr Antic, there are a number of key elements that support great data talent within organisations:

“There are a number of facets which are key to successful data and analytics, and they’re all very much related to people,” he explains. “Having the right people in the organisation in the right roles and frequencies to be analysts, data scientists, and software engineers to enable capabilities from a technical viewpoint is vital.”

A strong technical leader, who understands the solutions and can guide the team is also key, he adds. This leader needs to be adept at finding the right problem to solve and able to work closely with people as an evangelist and trusted advisor within the organisation.

Understanding the power of democratisation of data within the organisation is also vital, Dr Antic adds. This process supports education (such as improving data literacy) and helps create and support the data strategy for the organisation so that the C-suite has a clear understanding of how to use data.

“This all contributes to a culture of innovation, experimentation, and exploration,” he explains. “It’s vital to collaborate, share data, and knowledge throughout the organisation.”

Having a solid grasp of ethical use of data/privacy is also a must in today’s compliance-driven and highly regulatory environment.

“It’s becoming much more important for organisations to be able to understand ‘responsible use of AI’, how to deal with bias, and the ethical concerns around fairness in data, as well as how it will be used by the end-user,” Dr Antic adds.

Finally, a great data engineer needs to have great communication skills, as it’s communication that brings all these elements together.

“Communicating the benefits of a strong analytics program empowers you to build up a winning coalition, so people, analysts, and business leaders can understand and embrace the benefits of analytics internally,” he says.

The evolution of the modern data team
The future looks exciting. Companies that are moving beyond traditional BI are those that are disrupting markets. They’ve invested in modern data teams to optimise insights across the organisation, driving growth and ROI in the process.

Where data teams of the past may have had three traditional participants: database administrators responsible for data warehousing capabilities and capacity, data analysts focused on data modeling, and BI architects in charge of building dashboards and related self-service reporting capabilities, today the team is so much more.

Typically a team that partners with the business but has its own specialised skills, this group does everything from outlining business rules to modeling data to building single sources of truth.

Most importantly, these individuals provide ad hoc analysis, the kind of assessment that you don’t really get from traditional BI. Ask business questions like “what should my revenue model be?” and the group will test its hypotheses to surface unexpected insights. They may apply machine learning or some statistical and predictive technologies to determine patterns or gaps where your organisation simply does not have ideal data.

On top of this, can your data analysts blend model data with raw data from multiple sources? This is a nuanced but worthwhile question. Ideally, your data team can take raw data coming in from a new system and join it with existing model data, such as account, opportunity, or service data, to answer questions that may not have been anticipated in a current data model.

If data is blended from multiple sources into a single warehouse, does that process occur before or after it arrives at the warehouse? For example, a more modern approach would blend the data post warehousing.

The key takeaway is that, unlike a traditional BI team that provides only a data model, today’s data team operates as a core function of the business and recommends a strategy built on data. The group has a seat at the table, offering informed opinions – based on data – about what the company should be doing.

ABOUT SCOTT CASTLE//
Scott is the VP & GM for Cloud Data Teams at Sisense. He brings over 25 years of experience in software development and product management at leading technology companies including Adobe, Electric Cloud, and FileNet. Scott is a prolific writer and speaker on all things data, appearing at events like the Gartner Enterprise Data Conference, Data Champions, and Strata Data NYC.

Data is a precious asset. But the key to unlocking its potential is through its consistent, strategic and wider application in everyday situations. In this article, we unpack why making better data-driven decisions needs a mindset shift, and what steps organisations can take to make data more accessible across their wider teams.

A large majority of respondents in a recent Harvard Business Review (HBR) Analytic Services survey indicated that analysed data is essential to their company’s innovation strategy. Although data analytics has had a positive impact on customer experience and operational efficiency efforts, survey respondents believe data insights are not being sufficiently leveraged to make improvements or discover fresh business opportunities.

“Data analytics can only deliver results if people are allowed to use it.”

Closer to home, in a recent Sisense report, 67 percent of respondents said BI and analytics programs are more important or much more important to their daily operations today.

By enabling access to data across teams, departments, business units, leaders, and ‘every place in between’, your organisation can instill a data-centric culture that ‘permeates’ through your business. This ‘infused’ approach is the key to unlocking better data-driven decisions at every level.

Removing data hurdles
Training a diverse employee workforce to use highly complex, technical and specialised BI technology is unnecessary and unachievable. You’re essentially asking them to play data scientist, statistician, and business strategist. Still, organisations attempt this fruitless exercise in order to achieve a data-driven mindset.

Our relationship with technology has made us want information immediately. We now expect that same level of instantaneous insight in a work setting. The effective application of data can inform us of developing situations and deliver insights where decisions are made.

Infusing automated data intelligence into processes, workflows, and applications can provide a frictionless user experience with no complex tools or technical instruction required. Employees don’t even need to leave their workflows. With ‘analytics infusion’, actionable intelligence is presented when and where it is needed.

These automated processes will continue to be infused in organisations with a Sisense report revealing 78 percent of businesses will either maintain or increase spend on BI/data analytics initiatives, including software, tools, time, and team members, in the near future.

Risk of sharing data
There is a perceived risk in sharing data broadly. In a data-centric organisation, leaders need to blend governance with implementing useful data throughout the workforce. They understand that with careful planning, they can offset the risks to achieve the business successes that are inherent in a data-centric organisation.

Technology itself is an enabler but can also hamper deployment if team members can’t access the data when they need it.

The employee viewpoint is critical here because the technology has an impact, not just on their own team or department, but throughout the company. Instead of requiring employees to deviate from their workflow rhythm and comfort level for analysed data, why not embed analytics in their commonly used applications? By making information accessible at the point of need, organisations enable real-time decisions from within a familiar application.

C-level leads the mindset shift
Saying goodbye to data gatekeepers takes executive-level buy-in. Other than the chief data officer, leadership tends to think the application of data and analytics is reserved for specialists. A deficiency in skills and training is not only limited to the workforce at large; it can also be an impediment to executives.

It’s time to think differently and gain a rudimentary understanding of what can be gained from data to ‘better lead the troops’. Setting the example, by gaining a familiarity of the methods and approaches involved in creating insights from data, enables leaders to embrace data-informed strategies and nurture a data-driven culture.

An understanding of the insights required, an appreciation for the value of clean data, and the capacity to recognise gaps in the data ecosystem – these are the factors that allow leaders to re-orchestrate the way operational decisions are made, internally and even externally. This belief in data’s unique ability to fuel business value forms the basis for creating a data analytics culture that is scalable.

Who is getting it right?
There are thousands of great organisations currently reaping the rewards of infusing analytics into their workflows. One organisation getting it right is Australian Fintech Profectus, which whittled a 20 minute search query to only 20 seconds by infusing a robust data analytics solution.

Meanwhile Air Canada, serving more than 50 million passengers every year, leverage data analytics to build powerful proofs of concept, such as pushing actionable data to their end users (both frontline employees and executives). This enabled them to look at highly important data that deals with their day-to-day duties to ultimately improve and add insights into all aspects of safety across Air Canada.

More recently, Bakers Delight announced they are leveraging analytics to benefit internal stakeholders and franchisees by reducing the likelihood of bakery items selling out, as well as providing more personalized shopping experiences through customer loyalty initiatives to ultimately benefit the customer.

Blending people and processes
Democratising the data is only one obstacle. Data analytics can only deliver results if people are allowed to use it. When everyone within an organisation has access integrated via their existing workflows, they have data analytics at their fingertips to perform their jobs better. Moving from where the data is to where the work is completed is a thing of the past. Instead, the infusion of analytics reveals insights and promotes decision making there and then.

To derive full benefit from accumulated data, organisations must consistently and proactively promote and inspire a data-driven culture, starting with C-level commitment and granting each employee data access infused in their workflows.

ABOUT SCOTT CASTLE//
Scott is the VP & GM for Cloud Data Teams at Sisense. He brings over 25 years of experience in software development and product management at leading technology companies including Adobe, Electric Cloud, and FileNet. Scott is a prolific writer and speaker on all things data, appearing at events like the Gartner Enterprise Data Conference, Data Champions, and Strata Data NYC.