Keeping humans in the loop

Rather than treating AI as an isolated ethical issue, IESBA’s guidance examines characteristics of emerging technologies – including opacity, dependence on data, adaptivity, autonomy, scalability and speed – that can create or amplify threats to the profession’s fundamental ethical principles. These circumstances range from overreliance on the technology, automation bias, limited explainability, privacy and security risk, reduced human oversight, accountability gaps and the amplification of errors across systems and processes.

Is it appropriate?

‘Start with the job the AI system is being asked to do and whether it is appropriate for that task,’ says Jenn Kosar, AI and assurance leader at PwC US. ‘Accountants should understand what good performance looks like, what could go wrong, the quality and source of the information being used, and what level of testing and human review the risk of the use case requires.’

Alex Koh, head of audit at KPMG, Singapore, says one practical test is whether the professional can ‘clearly articulate’ why the conclusion is reasonable and what evidence supports it. AI-enabled risk assessment tools can incorporate specific checkpoints where human teams will need to apply their knowledge of the business, to guide AI to reach a conclusion.

‘This creates a human-in-the-loop journey, where humans are working alongside machines and clearly guiding them to arrive at a certain outcome,’ he says.

That evidence might include the provenance and quality of the underlying data, system testing and validation, documented assumptions, access controls, review procedures and a record of where human judgment was applied.

‘For AI, that trail may need to be recorded in terms of the inputs, the prompts, the parameters, how the sources were checked, what outputs were considered, what exceptions were applied,’ says Lane, who suggests a simple test: what was the system asked to do, what information did it use, what limitations and uncertainties applied, what checks were carried out, and what evidence supports the conclusion?

This is not necessarily a wholly new standard of professional care, says James Polson, audit partner at Deloitte UK. ‘The underlying principle is the same: the professional needs sufficient appropriate evidence for the degree of reliance being placed on the information,’ he says. ‘A generative output may require checks similar to those applied to work produced by a person, including corroborating facts, assumptions and conclusions.’

Build boundaries

Some AI systems give different answers to the same question even when the information provided is identical or similar, while others change how they work as they learn from new data, user interactions or changing conditions. Koh notes that, as AI systems become more autonomous and adaptive, accountants should focus less on whether an output is produced the same way each time, and more on whether the system stays within the boundaries set for it.

‘Those boundaries need to be designed from the start: a clear purpose, defined access to data and systems, and limits on what the AI can do without human sign-off,’ he says. ‘It is more reliable when tasks are specific and well defined, because its outputs are then easier to test, monitor and trace.’

For Ian Gregory, chief technology officer at Advancetrack, the nature of the technology itself is an important starting point: a deterministic tool ‘is going to say the same thing every time, while a large language model tends to be more inconsistent. That is a big difference from an accounting point of view.’

Harvey Dhillon, CEO of Zmartly, says missing invoices are a classic case where a system will happily produce a clean return from incomplete books. ‘The more routine the work becomes, the fewer people look at it, and errors now arrive in bulk, one wrong rule applied to a thousand lines rather than one slip on one line. The person who does look has to look harder,’ he says.

There remains a gap between organisations saying they have responsible AI and being able to demonstrate outputs can be trusted. ‘Many organisations understand the principles of responsible AI, but the harder work is translating those principles into testing, monitoring, accountability and controls that operate consistently and efficiently as AI scales,’ Kosar says.

This means that organisations must establish governance, approved-use policies, training, controls and escalation routes. ‘Responsible AI becomes meaningful when an organisation can demonstrate, for a particular system and use case, why its outputs can be relied upon, what its limitations are, how performance is monitored and what happens when it goes wrong,’ Polson notes.

Train to challenge

IESBA says accountants must have, or develop, a ‘sufficient understanding’ to judge whether a technology is appropriate and its outputs reliable. Organisations therefore need to train people not just in how to use AI, but how to challenge it.

‘That means preserving the fundamentals of professional scepticism, including asking what evidence supports an answer, what might be missing, what assumptions have been made and what alternative explanation could lead to a different conclusion,’ Polson says.

For accountants and auditors, there is an advantage because professional scepticism is already part of their professional DNA, according to Koh. However, ‘There is a risk that professionals become used to accepting AI-generated outcomes at face value, especially as the usage of AI grows more prevalent both at work and in their daily lives.’

Organisations should therefore reinforce the expectation that AI outputs are there to inform decisions, not replace judgment. ‘The ability to challenge conclusions, apply judgement and remain accountable for outcomes will be a defining professional capability in the intelligent age,’ he says.