Data sharing at scale

Published on the 19/06/2025 | Written by Heather Wright


Data sharing at scale

Critical success factors…

“Common understanding of what good data sharing is, is very, very absent.”

Kayelle Drinkwater is a big advocate for finding or growing data savvy lawyers if you’re embarking on a ‘data at scale’ project. It is, she says, a critical success factor.

“Find – or grow – data savvy lawyers.”

Drinkwater is the Department of Social Services data access branch manager and heads up an ambitious program pulling together the $68.3 million National Disability Data Asset  (NDDA) providing de-identified, and previously siloed, data from across Australian jurisdictions and domains, such as housing, education, health and justice, on people with disabilities.

It’s expected to provide a more complete picture of the programs and services those with disabilities use, whether they’re working and where improvements can be made, and will bring together 208 datasets covering 28 million individual records at any one given time.

The first release, late last year, included 18 data sets with three more releases planned for 2025, increasing the data available. Accredited researchers at Australian universities are able to apply to access the data.

“It’s the first time we’ve have linked data like this at scale,” Drinkwater says.

It’s also the first use of the Data Availability and Transparency Act 2022. (the DAT Act is under review as to whether it will continue beyond its sunset date of April 2027. All of the data sharing agreements registered under the Data Scheme relate to the NDDA.)

“The data was super siloed, each system used different terms for person [a patient in healthcare, a tenant in housing, a defendant in justice].

“Putting the puzzle together becomes about the person, not what domain they live in. It’s linked, not segmented and it’s focused on an outcome so we can see all the aspects [of their journey],” Drinkwater told this week’s Gartner Data and Analytics Conference.

Data sharing by the Department of Social Security isn’t anything new. But the scale and the types of data shared through the National Disability Data Asset is. It wasn’t however, the data work – cleaning, harmonising, integrating, doing probabilistic matching and so forth – that would be the challenge. In fact, Drinkwater says, ‘all the data things turned out to be the easy parts of the project’.

The hardest part, instead, was stakeholder management including managing expectations of data custodians.

“Just having them all understand the separation principle [to protect the identities of individuals and organisations in datasets] was really kind of profound.

“We do it every day, but for people who don’t integrate data like we do, they just don’t have it in their mind, so explaining that was pretty important.”

Sharing data across jurisdictions – hospital data for example being shared not only with the Commonwealth but with all other states and jurisdictions – adding to the complexity.

The data sharing agreement which in the past for sharing with organisations like the Bureau of Statisttics or Institute of Health and Welfare were only 12-15 pages, blew out to 150 pages.

“Because we were sharing this under the DAT Act for the first time, the idea was to have the same data sharing agreement for pretty much 208 datasets,” Drinkwater says.

“It wasn’t just sharing with us, and us sharing with them. It was actually sharing with everyone. And that got super complicated.”

Using hospitals as an example, she noted health lawyers from respective jurisdictions had to be involved, along with data sharing lawyers – Drinkwater notes there’s not too many of those as it happens – and the data domain experts were required.

“When you multiply all that by the 208, it is well over 208 lawyers.”

The Australian National Data Integration Infrastructure has been stood up. It includes a national spine and linkage model, data governance and streamlined data sharing arrangements. It also includes a cloud-based solution ANDII ICT to improve capacity for linking datasets, analysis and is scalable for future data assets. It required all states and territories to sign off on it as well as their cyber assessments.

Drinkwater admits there have been plenty of ‘ugh’ moments.

“There have been great moments of ‘we’ve got that!’ and then there have been ‘ugh, back again’ moments.

“Whenever we felt meh about it, and we did, it was really good to come back and go, you know what we are trying to do here is improve lives of people with a disability. It’s not actually about the data. That’s just the mechanics. Anytime we felt low, being able to reach back to that mission was an important part of success.”

Other critical success factors?

Alongside stakeholder management and engagement, there’s that aspect of finding or growing data-savvy lawyers. One lesson learned: Don’t put a data sharing agreement in front of lawyers until you’ve had a ‘data 101’ talk, including talking about the data lifecycle, with them first and answered their questions. Then go back with the data sharing agreement – preferably one that isn’t 150 pages.

Good program governance and having the right people, at the right time and at scale was also critical.

“When we were starting out we probably didn’t have enough people with the right skills to hit the ground running. It took us a while to build that up.

“The data teams were ok, but it was really around the stakeholder engagement and management, I’m going to say across mostly jurisdictions and data custodians, not especially with the disability community – that had already been a long-established process.”

And when changes occur, as they will do, reprioritise and look at the bigger picture – always keeping an eye on whether you’re achieving the strategic intent.

Post a comment or question...

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

MORE NEWS:

Processing...
Thank you! Your subscription has been confirmed. You'll hear from us soon.
Follow iStart to keep up to date with the latest news and views...
ErrorHere