Digital Council for Aotearoa New Zealand. (2020). Towards trustworthy and trusted automated decision-making in Aotearoa.

Abstract

We spoke to over 180 people throughout Aotearoa about different situations where ADM has specific impacts on the lives of individuals, whānau and communities. We heard loud and clear that ADM and other decision-making systems should be built for — and with — the people who are impacted. This is essential for ensuring trusted and trustworthy systems.

When workshop participants talked about ADM, they focused on more than the technology itself. Instead, they talked about algorithms as being part of a much wider system that also included the way data is collected and used, the people and organisations that develop the systems, and the interventions resulting from decisions. Participants thought that ADM, with its ability to process data fast and at scale, is well suited to some situations. However, they were clear that it can be harmful in other situations and can intensify pre-existing bias and discrimination — especially when the decision has major impacts on the lives of individuals and their whānau.

Participants provided clear and concise suggestions of what would make them feel more comfortable in situations where ADM is used. They want systems that are built to meet the needs and reflect the values of the communities impacted. To achieve this, it is important to participants that people who have similar lived experience to them are involved in the development of decision-making systems and the interventions that result from them. Participants told us they would be more comfortable if there was transparency and clear communication about how the government uses ADM and how it is used to make decisions.

We took these clear and urgent suggestions and used them as a basis to develop a set of recommendations to the government. We looked at work already underway and the barriers preventing systemic change, and gathered input from experts to inform our thinking