Published on the 27/07/2023 | Written by Heather Wright

And the three steps to take now to ensure that ROI… 

Local organisations can expect to start seeing returns on their generative AI investments within the next two to three years, but it might not be in the areas first expected and there is some work to be done first. 

Sam Higgins, a Sydney-based principle analyst with Forrester, says Australian and New Zealand organisations who already had infrastructure in place for natural language processing or conversational AI, and who understand their risk appetite, are ‘really excited’ and moving quickly to take advantage of generative AI.

“The vast majority though are looking at this as a wait and see – they’re not rushing in with their eyes closed, because there seems to be recognition that we have to assess our risk appetite first,” Higgins told iStart.

“A lot of enterprises would love to have a data lake but instead look forlornly over their data swamp.”

Nonetheless, Forrester’s Top 10 Emerging Technologies in 2023 report says generative AI will begin to deliver significant return on investment for most enterprises in the next two to four years.

It lists generative AI, alongside autonomous workplace assistants (replacing intelligent agents) and conversational AI (replacing natural language processing) as technologies with emerging but proven capabilities that should deliver expected ROI for most firms within the next two years. ‘Many use cases will exist with substantial potential for benefits’ across financial services, healthcare, insurance, public sector, retail and smart manufacturing, the report notes.

“Mainstream firms should begin to invest or continue investing in them with reasonable expectations for measurable benefits quickly,” Forrester says.

Higgins defines generative AI, at its most fundamental, as a combination of a large language model (LLM), natural language processor (NLP) and a natural language generator – the input processor, the output generator and some sort of machine learning under the covers.

While it’s easy to think of generative AI as an overnight success, he points out the boom has been a long time in the making, with companies harnessing conversational AI and machine learning models for some time.

Forrester’s Priorities Survey for 2023 showed 53 percent of A/NZ organisations were planning to invest and adopt more conversational AI during 2023, with 52 percent planning to adopt more natural language processing.

“So over half were already wanting to do more in this space,” Higgins says.

“The big advance has been a whole stack of emerging technologies in that broader AI family finally coming together in something that we would consider to be truly general purpose and that didn’t require an end-user enterprise assembly process,” he says.

“OpenAI really cracked the code on the general purpose nature of these large language models and made them accessible for us as individuals all the way through to large enterprises.

That ties in with work Higgins was doing in 2018, pre-Forrester, with an Oracle-run think tank on the autonomous enterprise which was discussing algorithms, robotic process automation and other forms of AI. He says almost all CIOs and CTOs involved said they felt it would be very rare for them to take a DIY approach to AI.

“Rather than DIY, they wanted what I call the MODIFY approach – make others do it for you. And that’s coming through very strongly with generative AI too.”

“Not everyone needs to make the sauce from scratch. A lot of people just want to go to the store and buy it pre-bottled in a particular context for a particular thing and that is why we are seeing vendors trying to imbue their offerings with some sort of AI add-on.”

But while generative AI holds plenty of promise, there are also caveats and hurdles to be climbed for many companies.

“Let’s face it, a lot of enterprises would love to have a data lake but instead look sadly and forlornly over their data swamp,” Higgins notes.

Letting generative AI lose to make decisions on that swamp is clearly not the best idea, with Higgins urging companies to also start thinking about what their policies for these tools are going to be.

Those organisations who have already built some of their AI infrastructure around NLP and conversational AI have already bitten off some of the data management and data cleansing challenges, positioning them well to take advantage of the uplift on their existing solution that can be gained via generative AI.

Higgins cautions, however, that those taking a wait and see approach also need to be aware that the technology, with its general purpose nature and ready consumer adoption, comes with a BYO AI risk.

“If you already have concerns around information security literacy of your employees, partners and customers, this sadly is just another front in that ongoing battle to make people understand the implications of how they treat data,” he says.

With the potential productivity dividend offered by the technology, it’s easy to see the lure.

Several recent reports have highlighted the gains to be had, including a 59 percent per hour productivity boost for business writing, and a massive 126 percent a week boost for developers.

Interestingly, a third study found customer service productivity doesn’t increase as much – just 14 percent.

Higgins notes an anecdote form a US colleague who says an executive noted they thought they were going to save a lot of money in customer service, but found they needed to spend twice as much on data scientists to get the data into a position where it worked for a highly structured, highly nuanced customer service environment.

“So there is a bit of a false ROI in some people’s mind, and there does need to be a recognition that general purpose technologies have a productivity dividend in places you might not notice and sometimes the thing we might naturally go to is not necessarily the place we find the most benefit.”

The right use cases, however come with massive benefits: Higgins says research indicates just two percent of use cases for the technology will generate 20x productivity lift.

“That means when you find the right use case the payoffs are massive, but there are actually really very few of them beyond the more general purpose things like software development and business writing, which as we know does not require much integration. Microsoft and Google that are going to bring those to us as part of our existing tools.” 

So what can local organisations do now to ensure they see ROI on any generative AI implementations?

Higgins says start by educating users about the technology now.

“You need to get rid of the fear, uncertainty and doubt. In five years’ time generative AI tools will just be a general purpose productivity feature of our working landscape, like using email or other productivity tools. But there is still a skills gap today between what we have and what this will look like tomorrow, so start educating people now and do it in grass roots fashion.”

He notes a bank operating across Australia and New Zealand which had one individual, not in the tech team, who was passionate about AI. He’s not trained thousands of colleagues and created a community of practice around the technology because of his passion. (The tech team did buddy up with him to ensure what he was saying was accurate.)

From a commercial aspect, organisations need to consider if they have the pre-requisites in place to take advantage of the tools vendors are rushing out.

“If you are in an arrangement with Google, Microsoft, Salesforce or SAP are you on a version of the product that is compatible with these new solutions when they roll out? What does the roadmap look like from these large platform vendors in your industry for your particular enterprise?”

Companies using on-premise Office products for example, won’t be able to harness the new offerings.

“You are going to be left behind. That is the stark reality.

“And that can be as simple as not having the right license in the case of Microsoft,” Higgins says. Microsoft Copilot currently in preview and being trialled by a number of Australian and Kiwi businesses, will only be available to Microsoft 365 E3, E5, Business standard and business premium customers.

His final tip is to start thinking about your organisation’s risk and data appetite and where your existing data concerns lie.

“I do hear organisations lamenting their ability to really motivate business leaders or their board in investing in dealing with that data swamp. If nothing else, if you’re not going to drain it, you certainly need to ringfence the aspects of it that are really quite terrible and try to remediate what is there, so there is a need to reinvigorate your information and data management.”