3072 Documentation of Work Performed Using Technology Solutions as Part of an Audit

Jun-2020

Work performed using technology solutions is documented and retained in accordance with the requirements in OAG Audit 1100. Examples of technology solutions include

• Computer Assisted Audit Techniques (CAATs);

• Data auditing techniques, automations and visualizations using technologies such as Excel or IDEA;

• Solutions built using IDEA, Python, R, SAS or other technologies, including solutions used to test IT system configurations or security settings.

The documentation requirements apply regardless of whether the solution was developed and/or the work was performed by an engagement team member or by specialists such as members of the Data Analytics and Research Methods Team, or other specialists in accounting or auditing.

When performing procedures using technology solutions, the following is documented in the audit file:

• The scope and purpose of the work to be performed, including the audit risks to be addressed. If an auditor’s expert or specialist in accounting or auditing is performing the work, the engagement team members need to discuss the scope and purpose with the expert or specialist for purposes of understanding and documenting or reviewing documentation of the work. Documentation includes

  • The scope of testing (e.g., consolidation, specific set of accounting records, entities or company codes);

  • The system(s) or data source(s) included in the audit procedure. Examples include

    • Level at which the testing is performed (e.g., consolidation, general ledger, subledger) when using a technology solution;

    • The specific functionalities/attributes of controls to be tested (system configurations, application security settings, etc.) when testing the entity’s internal controls (application controls and/or ITGCs).

  • The specific procedures to be performed and coverage dates.

• A description of the work completed, including the following:

  • The procedures performed to determine the completeness and accuracy of the information used in the procedure(s);

  • The procedures performed, including the nature, timing, extent and results (including exceptions) of tests performed, who performed and reviewed the work and the date such work was performed and reviewed;

  • Procedures performed and/or logic applied to generate the results, as applicable;

  • Description of original entity file(s) used and the source(s) from which the data was obtained (e.g., person, system and date received);

  • A summary of analysis and findings;

  • Any follow-up actions required by the audit team (if the work was performed by a specialist in accounting or auditing or auditor’s expert); and

  • Changes or additions to the original scope, if applicable.

Documenting whether a technology was used and the procedures performed and/or logic applied by the technology

Specific to documenting the procedures performed and/or logic applied to generate results from a technology solution, there are various options available, such as

• Including the output from the technology that includes the formulae or other criteria (e.g. sorting, filtering) used in executing the procedure (e.g., include the Excel worksheet).

• Attaching the actual script/programming (e.g., SAS, SQL, IDEA Script, IDEA Project Overview (Table and Graph) or IDEA History) or an export of the analytical software’s workflow, the visualization output, or other system logic in the audit file.

• When doing so, if the formulae, criteria, or logic are complex, consider how the reviewer or “an experienced auditor having no previous connection with the audit” would be able to understand what procedures were performed. In some cases, “plain English” explanations may need to be incorporated along with the workflow, visualization, or script logic to facilitate review. For example, this might be accomplished by using explanation boxes above each function executed within the workflow to explain in simple terms what the function is doing and including the workflow in the workpapers.

• Including “plain English” documentation of the procedures performed by the technology. For example, when using script to perform certain procedures, we may provide the following documentation: “IDEA was programmed to extract [for example, a list of loan terms] from the underlying contract, populate the spreadsheet with each term and corresponding amount in a different column and calculate the difference between the balance per management’s system in column C with the amount populated by IDEA in column D.”

• Including tickmarks in the workpapers explaining where and how the amounts were obtained/calculated.

• When using a visualization to perform an audit procedure, there are two documentation options: (1) export screenshots of the visualizations and insert them into audit file or (2) retain the visualization file itself (e.g., IDEA .idash file). Consider the complexity of the visualization when choosing an option. The more filters, formulae or other criteria are used to reconfigure the underlying data sets, the more likely it is that retaining the visualization file will be necessary to meet the documentation requirements.

If the procedures performed using the technology are complex (e.g., the procedures may involve multiple inputs, multiple relationships between the inputs or multiple iterations of a calculation), we typically run test transactions through the workflow or bot to test both positive and negative conditions, as well as various permutations and conditions, as necessary, to verify it functions as intended. However, such testing to verify the technology functions as intended is typically not necessary if the solution is an approved tool. Document the testing performed and/or retain the documentation of the test transactions and test results in the audit file.

The nature and extent of our procedures and related documentation needs to be commensurate with the assessed risk of material misstatement and how important that audit procedure is to our audit strategy and audit plan. The higher the assessed risk and/or the complexity and the more important the audit procedure, the greater the extent of documentation.

These documentation principles apply equally when a spreadsheet, such as Microsoft Excel, is used as a technology solution in an audit. For example, when using Microsoft Excel to examine, sort, filter or analyze data, we consider using the comment functionality to include “plain English” documentation of the nature of the Excel functions or formulae being used. Likewise, if Visual Basic coding is used, we consider using comments in the Visual Basic Editor (“VBE”) to prepare “plain English” documentation of the nature of the Visual Basic program behavior and indicating in the Excel worksheet that this documentation has been included in VBE.

In addition to documentation considerations, supervision/review requirements are interrelated and are considered together when determining whether the technology used is appropriate in the circumstances and what and how much to document in the working papers. Refer to OAG Audit 3073 for supervision and review considerations when using technology solutions.

Documentation considerations when using an approved tool

In cases when we are using a technology solution approved for use by the Office we document how we used the tool and its related outputs, but do not need to document the tool’s logic. For example, when using IDEA Script to analyze general ledgers, where this tool is an approved tool for use on engagements, documentation in the engagement database regarding the software code and logic is not necessary. The reliability of approved tools is the responsibility of the Office because documentation is expected to be maintained centrally and does not need to be duplicated by the engagement team.

Retention of client data used with technology solutions

Generally, data obtained from the entity that is not required to be retained pursuant to CAS 230 and OAG Audit Manual (refer to OAG Audit 1110 and OAG Audit 1121) would not be kept after completion of the engagement. However, there may be reasons for using data previously provided by the entity such as to provide benefit in a subsequent year audit, such as:

• Performance of trend analysis covering multiple periods of data
• Facilitating the entity's assembly of subsequent year data
• Training new team members on the engagement to understand data nuances

As per current practice, it is important to ensure audit documentation is properly assembled during the final file assembly period and deleted from temporary storage devices within the 60 days period. All documentation that supports the audit conclusion must be retained in the audit working paper software. A best practice is to advise the client that data received in the past will be deleted as per our policy and that we may ask for it again in the future. When data obtained from the entity will not be retained, our documentation of the procedure performed using the client data includes sufficient detail of the data used (e.g., the name, date and any other relevant parameters of the client report or query used to generate the data used in the procedure). Whether or not the data is retained, we document the procedures performed over the reliability of the data in line with OAG Audit 6025.

Refer also to OAG Audit 3101 for retention considerations when work is performed by specialists in accounting or auditing.

Practical examples of documentation when using technology solutions

The following examples focus on some relevant considerations when making judgments about:

• How to document the procedures performed/logic applied by technology
• The documentation needed to support supervision and review

These examples do not cover other documentation typically included in the workpapers, such as documentation in support of the completeness and accuracy of the source documents.