Consolidating and aligning guidance for audits of Privatisation with ISSAI 100 – GUID on privatisation

Leave a comment

Please read the exposure drafts and the explanatory memorandum before sending your comments.

Comments are closed.

Performance Audit Subcommittee (PAS)

Some of the comments provided by the Performance Audit Subcommittee (PAS) in the following are similar to comments provided by the committee in July 2020 (earlier draft). These are important considerations that specifically relate to the application of performance audit (PA) in a subject matter specific GUID. Depending on an initial problem analysis, it seems an audit of privatisation in many cases might be a compliance audit. Generally, the draft is not sufficiently convincing in its application of PA, and does not demonstrate how or why an audit of privatisation would in fact “always” be a PA. This should be included and described in length in the introduction, or the idea of focusing the GUID specifically on performance audit may be re-examined. If the current angle is kept, we suggest that the GUID focus narrowly on the subject matter while explicitly referring to the relevant standards and application GUIDs vis-à-vis the relevant type of audit. However, we return to the relevance of referring exclusively to performance audit below, as we have identified some challenges in this regard. The critical point is that performance audit standards are to be applied in the same way regardless of subject matter, and any content on performance audit in a subject matter specific GUID has to be presented in a context that does not introduce additional or optional requirements that are not valid for performance audit. The GUID should instead focus on specific considerations when auditing this particular subject matter, which we believe would largely be the same for the different audit types. The application of methodology for conducting a performance audit requires references to the 3000 series, including the GUIDs. PAS made this comment in our feedback to the previous draft, in July 2020. We fail to understand why the GUID refers exclusively to ISSAI 300, as we believe it is not the right level to provide or refer how-to guidance in the case of PA. As a minimum, there should be a disclaimer in the introduction explaining why ISSAI 300 is the main source of reference. The draft frequently refers to performance audit specific considerations as a starting point for an audit of privatisation, but PAS does not consider that appropriate or relevant in a subject matter specific GUID. Regardless of subject matter, identifying and deciding on which type of audit to conduct will depend on the outcome of an initial assessment of risk and other considerations, and in each individual case. The GUID frequently displays examples of audit objectives that in fact identify as compliance, and not performance audit. An alternative approach in this GUID could be to pay less/no attention to performance audit specific considerations in an audit of privatisation, and focus more on specific considerations when auditing the privatisation process. It may not be relevant to identify - in this case - one type of audit when it is the assessment of problems and risks in a privatisation process that determine what type of audit should be conducted. There are also methodological considerations at play. Performance audits are, by nature, usually ex post. It means that a certain period needs to have passed since the initial privatisation process started, in order to be able to apply the performance audit approach and for identifiable outcomes or impact to have taken effect. If this GUID focuses only on the initial stages of privatisation, a performance audit focus may not be very relevant. The starting point of this GUID should rather be the specificities of auditing privatisation, and relevant type of audit should be applied based on an assessment of auditability, problem identification and risk in each case. It is critical that the GUID include some elaboration on the fact that a PA is usually ex-post by tying this to stages in the privatisation process, but this problem may also be avoided by not going too much into PA-specific content, or by abandoning the angle on PA as the default mode in audit of privatisation. The drafting team already lists a number of specific elements to be considered in auditing privatisation - these are elements that may not occur frequently when auditing other subject matters (such as specific types of data and access to data; organisational issues; stakeholders and informants, including access to these; a description of the privatisation process/stages, etc.). We believe these elements and their elaboration should be the focus of this GUID. The GUID requires a more informative introduction. Through the introduction, the reader will need to learn: why would an audit of privatisation “automatically” be a performance audit? At what point in a privatisation process is it relevant to conduct a performance audit? The structure of the current version follows the stages of a performance audit, rather than the stages of the privatisation process. We believe the purpose of this GUID should be to contribute to understanding the privatisation process rather than repeating considerations and procedures in the different phases of a performance audit. In the following are some comments to specific (selected) paragraphs: Para 1: the reader needs to understand why this GUID exclusively covers performance audit (PA). Which audit type to apply depends on a number of factors, as we mention elsewhere in our comments. PA is ex post; it requires examining output/outcome or impact over time. Para 2: we suggest calling it (subject matter specific) additional guidance. The PA GUIDs are to be consulted for how-to guidance on conducting a PA, regardless of subject matter. Para 4 (objectives) states: “The objective of this GUID is to provide guidance to the auditor on how to conduct performance audits related to the specific subject matter of privatisation…” etc. The objective of the GUID cannot be to provide practical guidance on how to conduct such a performance audit, especially as long as it refers exclusively to the principles in ISSAI 300. The reader should consult ISSAI 3000 and corresponding GUIDs for guidance on how to conduct a performance audit, regardless of subject matter. We therefore suggest including in an introductory part: “The auditor shall be guided by the principles of ISSAI 300 and the requirements of ISSAI 3000 when performing this audit. For practical guidance on how to conduct a performance audit, the ISSAIs should be read in conjunction with the corresponding performance audit implementation GUIDs 3910 and 3920”. Para 5: We question why, throughout, references are exclusively to ISSAI 300 and not 3000. In the 2020 version reviewed by PAS, we suggested to consider at what (ISSAI) level how-to guidance should be presented. One alternative is to discontinue the how-to approach, another is to isolate it from the application of one or more types of audit as these are already covered in the audit specific documents. Para 11: “selecting the individual topic for a PA of privatisation”? What determines if it will be a PA? In many instances, it will and perhaps should be a compliance audit, especially if the logic model is not being applied to examine performance over time. If the main objective is to check if the government has achieved its goals, it will be, in most cases, a compliance audit. However, if the main objective is to promote improvements in the effectiveness, efficiency and economy, thus investigating the output or impact of the privatisation in terms of profit or value for money, or increased quality of services, it will be a performance audit. Para 13: need to include more on mandate and in what instances it would be relevant to conduct an audit of privatisation. Additionally: are we referring to auditing the privatisation process, or the actual performance after finalising the privatisation process (over time - ex post)? In many cases, merely auditing a process or parts of a process will be a compliance audit, unless the appropriate methodology for performance audit is applied. Para 27 audit objectives and para 28 audit scope: The GUID should refer to the 3000 series for guidance on how to assess and identify the problem/risk, how to establish the objective of the audit (which will again determine whether to apply performance or compliance audit), and how to develop the audit questions. The objectives listed in para 27 are for the most part compliance objectives (to establish the correct administration of public resources and compliance with current regulations) and do not relate to the objectives of a performance audit, which is to promote improvements in the effectiveness, efficiency and economy of operations. A performance audit is usually ex post, which must allow some time to have passed before the output/outcome or impact can be measured. Para 48: example of how the GUID does not provide how-to guidance, when it mentions how “appraising the pre-marketing exercise and analysing the valuation procedure would have different procedures”. To reveal what these procedures are would be providing how-to guidance on audit of privatisation, since these activities are specific to the privatisation process. This para also provides an opportunity to add more content on privatisation process-specific content. We recommend that the focus should be the privatisation process or outcome, not the audit process, thus it may be relevant to focus on procedures such as those mentioned above instead of referring to established audit procedures that are sourced and elaborated on elsewhere. Para 49: how will the “stage at which the auditor makes the intervention (is that the audit?)… influence the conduct of the audit”? What is the intervention? Para 49: the systems approach is not immediately recognisable from the usual PA-perspective. For guidance on audit approach, it is vital to refer to the PA standards and applicable GUIDs for elaboration. Para 51: The standards establish that PA conclusions are usually persuasive, and not conclusive. Additionally: the GUID indicates, “Analysis and interpretation will have to be a more sophisticated exercise”, but again, it does not go into the practicalities of how-to (what are these more sophisticated exercises?), despite indicating in para four that this GUID includes practical how-to guidance. Cont. 51: elements under “review” have very loose ties to the 3Es, most are about verifying, assessing, comparing, reviewing, examining, enquiring, and these could be either performance or compliance specific, which makes the distinction here a bit problematic because they do not indicate how these relate to the 3Es. Further, to “gather appropriate evidence to the effect”, and “devise a methodology to place reliance on the contents of such reports to be adopted as evidence” indicate methods that may not be established in performance audit or, as a minimum, would require more explanation in order to be useful for the reader. On reporting and follow-up: concerning coverage of PA related procedures, references to the PA standards (and especially the requirements and GUIDs) would suffice. In the current draft, the level of detail varies from one paragraph/chapter to another. We believe the most important content is the subject matter specific considerations listed here, and to provide guidance on at what time or stage in the privatisation process it is relevant to conduct an audit or do the follow-up (when applicable). Additionally: If a SAI has the mandate to audit privatisation, would they not in some cases consider follow-up? If the audit objective is to check compliance with what was originally intended or assumed at the time privatisation was decided, that would be a compliance audit and it may not require follow-up. The challenge is that it may not be the case that an audit of privatisation would more often be a performance audit. From the PAS comments to the previous draft, quote: “…the decision to limit the scope [of this GUID] to PA has certain consequences. […] …it can be challenging to maintain focus on one type of audit in a subject matter specific guidance document, because the subject matter itself usually becomes the centre of attention. This is also the case with this draft”. At this point, we would recommend that the subject matter should be the focus, as it not necessary or even practically possible to single out one type of audit in such a GUID.