Evaluating Monitoring, Internal Audit and Management Review in an AIMS

Why Clause 9 Is Where AIMS Audits Get Serious

An AI Management System built on ISO 42001 can have a polished risk register, a well-written AI policy, and clearly documented objectives. But if the organisation cannot demonstrate that it is actually monitoring what matters, running credible internal audits, and feeding real information into management review, the system is performing on paper only.

Clause 9 of ISO 42001 covers performance evaluation. It is the point in the audit where you move from asking whether the system was designed correctly to asking whether it is working. That shift in focus catches a lot of organisations off guard, particularly those that are newer to AI governance or that have copied their approach from a QMS or ISMS without thinking carefully about what is different in an AI context.

This article walks through the three core elements of Clause 9 in an AIMS audit: monitoring and measurement under Clause 9.1, internal audit requirements under Clause 9.2, and management review under Clause 9.3. It is written for auditors conducting internal or external audits, and for quality, compliance, and AI governance professionals preparing their systems for scrutiny.

Clause 9.1: Monitoring and Measurement in an AI Management System

Clause 9.1 of ISO 42001 requires the organisation to determine what needs to be monitored and measured, how it will be done, when results will be analysed, and who is responsible. This is structurally similar to the same clause in ISO 9001 and ISO 27001, but the content of what gets monitored is quite different in an AIMS.

What Should an Organisation Actually Be Monitoring?

When you sit down with the AI governance team and ask what they monitor, the answers often start with operational metrics: system uptime, processing speed, error rates. Those things matter, but they are not what Clause 9.1 is primarily concerned with. The standard is asking about the performance of the management system and the effectiveness of the controls applied to AI systems.

In practical terms, that means the organisation should be tracking things like:

• Whether AI objectives set under Clause 6.2 are being achieved
• The status of AI risk treatment actions from the risk register
• The results of AI impact assessments conducted under Clause 8.4
• Compliance with the AI policy and any applicable regulatory requirements
• Incidents involving AI systems, including near misses and unintended outputs
• Feedback from users and affected parties about AI system behaviour

The key question is not just whether these things are being measured, but whether the measurements are actually informing decisions. If the organisation collects data on AI incidents but nobody reviews it until the annual management review, that is a monitoring process in name only.

What Auditors Look For in Clause 9.1

When auditing Clause 9.1, the most important thing to establish is whether the monitoring approach is documented, assigned, and actually happening. Ask to see the documented procedure or process for monitoring. Then ask to see evidence that it has been followed. Those two requests will tell you a great deal very quickly.

Common gaps include:

• No defined frequency for reviewing monitoring results
• Monitoring activities assigned to a team but with no named individual responsible
• Metrics that measure AI system performance but not AIMS effectiveness
• Results that are collected but not analysed or acted upon
• No connection between monitoring outputs and the inputs to management review

One scenario worth highlighting: an organisation running a customer-facing AI recommendation system had documented a monthly review of recommendation accuracy. When the auditor asked to see the last three months of review records, the team produced a spreadsheet of accuracy scores. When asked what decisions had been made based on those scores, there was a long pause. The data existed, but it was not driving any action. That is a conformity issue, not just an observation.

For a deeper look at how monitoring and measurement works in practice across management systems, the article on Monitoring and Measuring AIMS Performance Under Clause 9.1 covers the specific requirements in detail.

Evaluating Compliance Obligations

ISO 42001 also requires the organisation to evaluate its compliance with applicable legal, regulatory, and other requirements related to AI. In Australia, this increasingly includes obligations under the Privacy Act, emerging AI-specific guidance from regulators, and sector-specific requirements in areas like financial services and healthcare.

Auditors should ask whether the organisation has identified its compliance obligations related to AI, and whether there is a process for evaluating whether those obligations are being met. A legal register that covers general privacy and data requirements but has not been updated to reflect AI-specific obligations is a gap worth noting.

Clause 9.2: Internal Audit Requirements in an AIMS

Clause 9.2 requires the organisation to conduct internal audits at planned intervals to determine whether the AIMS conforms to the organisation's own requirements and to the requirements of ISO 42001, and whether it is effectively implemented and maintained.

This is standard internal audit territory, but there are some specific considerations that apply in an AI governance context.

The Audit Programme: Is It Risk-Based and Objective-Driven?

The internal audit programme for an AIMS should be based on risk. Higher-risk AI systems, areas with previous nonconformities, and processes that have changed significantly should attract more audit attention than stable, lower-risk areas.

When reviewing the audit programme, ask:

• Has the organisation defined objectives for the audit programme?
• Is the frequency and scope of audits linked to risk?
• Does the programme cover all relevant clauses of ISO 42001 over a defined cycle?
• Are AI impact assessment processes included in the scope?
• Are operational controls for individual AI systems sampled as part of the programme?

A common problem in newer AIMS implementations is that the internal audit programme mirrors the structure used for the QMS or ISMS, without accounting for the specific requirements of ISO 42001. The organisation audits documentation, leadership, and general planning, but never gets into the actual AI systems, their impact assessments, or the operational controls in Annex A. That leaves significant gaps in coverage.

Auditor Competence and Independence

Clause 9.2 requires that auditors are selected to ensure objectivity and impartiality, and that they are competent to conduct the audit. In an AIMS context, competence has an additional dimension. The auditor needs to understand not only management system auditing, but also enough about AI systems to ask meaningful questions about how they work, what risks they carry, and whether the controls applied are appropriate.

This does not mean the internal auditor needs to be a data scientist. But they do need to understand concepts like training data, model outputs, bias, and human oversight well enough to evaluate whether the organisation's controls are actually addressing those risks. If the internal audit team has no one with that background, it is worth noting as a gap in auditor competence.

Independence is also worth examining carefully. In many organisations implementing ISO 42001, the AI governance function and the internal audit function overlap significantly. The person responsible for designing the AI risk assessment process should not be auditing it. Ask who conducted the last internal audit and what their role is in the AIMS on a day-to-day basis.

What the Internal Audit Records Should Show

Documented information is required as evidence of the audit programme and the audit results. When reviewing internal audit records, look for:

• An audit plan with defined scope, criteria, and objectives
• Evidence that the audit was conducted as planned
• Findings documented with reference to specific requirements
• Nonconformities raised where appropriate, not just observations
• A clear record of who conducted the audit and when
• Follow-up on previous findings

One thing that often comes up in AIMS audits is that internal audit reports are very high-level. They confirm that documents exist and that people are aware of the AI policy, but they do not go into the substance of whether AI impact assessments are being conducted correctly, whether risk treatment actions are effective, or whether the outputs of monitoring are being used. That level of depth is what separates a meaningful internal audit from a box-ticking exercise.

For a broader perspective on internal audit requirements under ISO 42001, see the article on Internal Audit Requirements in ISO 42001: Clause 9.2 Explained.

Clause 9.3: Management Review in an AIMS

Management review is the mechanism through which top management takes stock of the AIMS, evaluates its continuing suitability and effectiveness, and makes decisions about what needs to change. Clause 9.3 of ISO 42001 specifies what inputs must be considered and what outputs the review must produce.

Reviewing the Management Review Inputs

The standard requires that management review considers a defined set of inputs. In an AIMS, these include the status of actions from previous reviews, changes in external and internal issues relevant to the AIMS, information on AI system performance and the effectiveness of the management system, resource adequacy, and the results of risk assessments and impact assessments.

When auditing management review, the first thing to check is whether all required inputs were actually present. Ask to see the management review agenda and the minutes or records. Then work through the required inputs one by one and confirm each one was addressed.

What you often find is that some inputs are covered thoroughly, usually the easy ones like previous action status and resource discussions, while others are glossed over or absent entirely. Information about AI impact assessment results, feedback from affected parties about AI system behaviour, and the results of compliance evaluations are frequently missing or treated superficially.

Are the Outputs Decisions, Not Just Discussions?

Clause 9.3 requires that the outputs of management review include decisions and actions related to continual improvement opportunities and any need for changes to the AIMS. This is where a lot of management reviews fall short.

It is common to see management review minutes that record a discussion of each agenda item, but produce no concrete decisions. The review notes that AI incident rates have increased but does not decide what will be done about it. It acknowledges that an AI impact assessment is overdue but does not assign responsibility or a deadline for completing it.

When auditing management review outputs, ask:

• What specific decisions were made at the last review?
• What actions were assigned, to whom, and with what deadline?
• How are those actions tracked between reviews?
• Were any changes to the AIMS decided as a result of the review?

If the answers are vague or the minutes show a lot of discussion with no clear decisions, that is a finding. Management review that does not produce actionable outputs is not fulfilling its purpose under the standard.

Frequency and Top Management Participation

ISO 42001 requires management review to be conducted at planned intervals. It does not specify a minimum frequency, but the review must happen often enough to be meaningful. For most organisations, annual reviews are the minimum. Organisations with rapidly evolving AI systems, significant regulatory scrutiny, or a history of AI-related incidents should be reviewing more frequently.

Top management participation is not optional. The review must be conducted by top management, not delegated entirely to the AI governance team or a middle management committee. When auditing this, ask who attended the review and what their role is. If the records show that senior leaders were present but the AI governance manager did all the talking and made all the decisions, that is worth probing further.

For a detailed look at what Clause 9.3 requires in an AIMS, the article on Management Review of an AIMS: What Clause 9.3 Covers provides a thorough walkthrough.

Common Nonconformities in AIMS Performance Evaluation

Across the three elements of Clause 9, certain nonconformities come up repeatedly in AIMS audits. Being aware of them helps auditors know where to probe and helps organisations address weaknesses before a certification audit.

• Monitoring without analysis: Data is collected but not reviewed in a structured way or used to drive decisions.
• Internal audits that avoid the hard questions: Audits confirm documents exist but do not test whether AI impact assessments are adequate or whether operational controls are working.
• Audit programmes with no defined objectives: The programme lists what will be audited but does not state what the audits are trying to achieve.
• Management review inputs that are incomplete: Required inputs such as compliance evaluation results or AI impact assessment outcomes are missing from the review.
• Management review outputs that are not decisions: Minutes record discussions but do not produce assigned actions with owners and deadlines.
• Top management absent from management review: The review is conducted by the AI governance team without meaningful participation from senior leadership.
• No connection between Clause 9 activities: Monitoring results do not feed into the internal audit programme, and internal audit findings do not appear as inputs to management review. The three elements should form a connected loop, not operate in isolation.

Practical Audit Approach for Clause 9

When planning an audit of Clause 9 in an AIMS, a process-based approach works well. Rather than working through the clauses sequentially, trace the flow of information from monitoring through to management review.

Start by asking to see the monitoring and measurement process. Review what is being tracked and how often. Then ask to see recent monitoring results and evidence that they have been reviewed and acted upon.

Move to the internal audit programme. Review the programme document, then sample the records from the most recent internal audit. Look at the scope, the findings, and the follow-up. Ask who conducted the audit and verify their independence from the areas they audited.

Finally, review the most recent management review. Check that required inputs were present, that top management participated, and that the outputs include concrete decisions and assigned actions. Then check whether those actions have been followed up.

This end-to-end trace is more revealing than a document-by-document review, because it tests whether the three elements of Clause 9 are actually connected and driving improvement, or whether they are operating as separate administrative exercises.

Building Auditor Competence for AIMS Audits

Auditing an AI management system requires a specific combination of skills. You need solid management system auditing technique, an understanding of ISO 42001 and its unique requirements, and enough familiarity with AI concepts to ask meaningful questions about how AI systems are governed.

That combination does not come from reading the standard once. It comes from structured training that covers the standard in depth, combined with practical audit experience. If you are preparing to conduct AIMS audits or to sit for an ISO 42001 lead auditor credential, investing in quality training makes a significant difference to the depth and confidence you bring to the audit.

Audit Workshop offers ISO 42001 auditor training at internal auditor and lead auditor levels, delivered by an experienced lead auditor who has conducted hundreds of external certification audits. The training is practical, grounded in real audit scenarios, and designed for practitioners who want to audit AI management systems with genuine competence, not just theoretical familiarity. If you are building your capability in this area, it is worth exploring the courses available at auditworkshop.com.