Published on the 17/04/2019 | Written by Pat Pilcher

NZ Statistician fesses up on census issues. After #censusfail in Aus questions need answering…

One in seven Kiwis failed to complete New Zealand’s first online census – a record low for census completion – as New Zealand follows Australia in stumbling in attempts at a ‘digital first’ national survey.

The New Zealand figures were revealed by the government’s chief statistician, Liz MacPherson, after she was threatened with being held in contempt of Parliament.

Statistics NZ opted for the online survey in 2018. As well as a record low in Kiwi’s filling in the form, there was a doubling in the partial response rate according to National’s State Services spokesperson Nick Smith.

“It is critical that trust and confidence in future censuses are maintained and enhanced.”

According to economic consultants, Berl “…around 480,000 people did not participate in Census 2018. Given the estimated population at the time was 4.88 million, this equates to 10 per cent of the population. To provide some further context, the 480,000 people is around the same size as the entire Waikato Region.”

The scale of the abnormally low 2018 census turnout led much hair pulling and shouting in parliament. The government agreed to appoint a team to conduct an independent review of what went wrong. According to Stats NZ, the review will be undertaken by management consultant, Murray Jack, and a Canadian census expert, Connie Graziadei. Findings from the review are expected to be released in July.

According to the terms and references of the review “the Government statistician is seeking robust independent advice on the 2018 Census, with a focus on understanding what factors contributed to the lower than expected participation rate, so that future censuses and other household surveys can be as effective as possible. Stats NZ is a learning organisation. It holds itself and the way it delivers official statistics to a high standard. Given the importance of the census and the role it plays in informing decisions that affect the lives of New Zealanders, it is critical that trust and confidence in future censuses are maintained and enhanced.”

New Zealand’s census predicament is not unique. In Australia, the 2016 Online Census was marred by the Australian Bureau of Statistics taking the Census website down after what it said was a series of denial of service attacks followed by a hardware failure that led to a misidentification of a larger denial of service attempt.

The political fallout was predictably swift. The then Australian prime minister, Malcolm Turnbull, lashed out at IBM (who had won the contract to host the census in 2014) and the Australian Bureau of Statistics, saying “There has clearly been a failure in the work that was done” and that “My prediction is that there will be some very serious consequences to this”. Shortly afterwards, the Australian government appointed Alastair MacGibbon, the Australian Prime Minister’s Special Advisor on Cyber Security, to conduct a review of the failed census.

The MacGibbon team included senior members from what read like an A-Z of Australian Government Departments. These included the Prime Minister’s office, the Australian Signals Directorate (ASD), the Australian Information Commissioner, the Australian Department of Finance, the Treasury, the Australian Digital Transformation Office and the Attorney-General’s Department plus a liaison officer from the Australian Bureau of Statistics.

The Australian Signals Directorate later confirmed that no data was compromised.

According to the findings of the review “the ABS severely underutilised social media as a communications tool to keep the public up to date and informed of the incident. The ABS’s lack of timely and transparent communications lost it trust because it opened the door to speculation. The continued slow updates and virtual absence from the media meant that ABS struggled to win back the trust of the public in the following days. Ministers must also be supported with clear and accurate advice, and senior executives must be equipped to understand and talk about cyber security as a matter of business risk.”

The rationale for a government to switch to a digital census appears compelling. Not only is it a considerably less costly to undertake than printing and mailing census forms to New Zealand households, but online digital forms can also be simpler to complete in that they can warn of missed or incorrectly answered questions and provide more predefined answers than a printed form. An all-digital census should, in theory, also be a significantly faster operation for statisticians.

As is so often the case, reality proves far more complicated than theory. In a blog commenting on lower than expected turnouts for the 2018 census, New Zealand’s 20/20 Trust, which promotes digital inclusivity, identified several issues that hindered response rates. These included the census not being properly accessible (with difficulties in obtaining a paper census form for those unable to get online) and a lack of briefing to helpdesk staff, who 20/20 also says were inexperienced, compounded issues. The 20/20 Trust also note that the online census FAQ was “fairly superficial”.

Perhaps most critically of all, there was too much emphasis on ‘Digital First’ – it was difficult to get paper forms, and the different user experience of paper forms versus their online counterparts was also not fully appreciated.

The pre-set answers alienated some groups, and many had concerns about privacy. Last but by no means least, testing was fragmented, not ‘whole process’ and didn’t include all the right groups.

The biggest test of an online census is about to happen. In 2020, the US census gets underway and goes digital. Armies of volunteers armed with iPhones will descend on US households to gather census data. There are good reasons for the move to a digital census. Costing US$12 billion, the 2010 census was the most expensive in US history.

Given the mixed track record of online census taking to date, experts are understandably worried that the US Bureau of Statistics could leave itself exposed to numerous new risks, including technology failures and cybersecurity issues.

Meanwhile, New Zealand awaits the findings from the census review team. Watch this space.