As education leads at Te Whare Hauora, Dougal and Carolyn Stott have used their background in technology to support kaimahi and whānau using their services. Alongside kaimahi, they co-designed and built an online platform, Te Pātaka Mātauraka, which offers self-directed learning modules for clients and can be used by staff for professional development, induction and onboarding.
Generative AI (GenAI) tools are used within the platform, but their inclusion was deeply researched and considered. This process started at the governance level, with an agreement that the technology had to reflect organisational values of whakapapa, manaakitanga, tikanga, and honour Māori data as a taonga.
The first question to address was whether AI tools should be used at all. Dougal admits to constantly balancing the ethical considerations underpinning AI, particularly the environmental impacts, against its potential to be used for social good: “It challenges my tikanga and it is a constant balancing act. But AI is here and it can do good, if managed right,” he says.
He emphasises that if providers use GenAI tools, they need to understand and mitigate against the risks, particularly their tendencies to make mistakes and their inherent bias against Indigenous peoples. “If we’re not careful, they can replicate the same systems that haven’t worked for our people. That’s why kaupapa has to lead its use,” he warns.
Being careful involves developing clear policies around how GenAI tools are used within organisations. For Te Whare Hauora, this means using it as a support, rather than a decision-maker. Dougal explains: “It helps to organise knowledge, reduce duplication and give kaimahi a starting point, but it doesn’t replace relationships or judgement.” He is clear that kaimahi should not use GenAI transcription tools with clients or to summarise case notes. He also emphasises that no client names or identifying information should go into open systems like ChatGPT or Claude.
He advises providers using GenAI tools to get expert assistance in understanding the technology, specifically what happens to data once it goes into a GenAI tool. This must also be communicated to clients so they can give genuinely informed consent when receiving services that use GenAI.
The Stotts have four questions that they recommend all governance boards ask when considering GenAI tools:
- Where is the data going?
- Does the tool use New Zealand-based servers or servers in other jurisdictions? (this determines what law applies to how the data can be used)
- Who has access to the stored data and under what authority?
- Does this tool uphold Māori data principles?
Overall, Dougal sees GenAI tools as having the potential to assist the sector, but clear governance direction and policies are needed, along with a good understanding of how they work. He says: “AI is here and it will keep moving. We have an opportunity to shape how these tools are used in ways that that uphold values, protect our people and strengthen our practice.”
