Application Auditing Scope, Approach,

1
Application AuditingScope, Approach, Execution

• January 2009
• Michael Kirk, CIA, CISA

2
Introduction

• Mike Kirk, CIA, CISA
• Application experience includes Oracle,
  PeopleSoft, JD Edwards, and industry-specific and
  customized applications for healthcare,
  insurance, energy/utility, manufacturing,
  construction, and environmental industries.
• Board member of the Central Ohio Chapter of ISACA

3
Introduction
JD Edwards
4
Overview

• Risk Considerations for Application Auditing
• Defining Your Scope
• Developing Your Approach
• Execution
• Q A

5
Where do you start?

• Audit Methodology

• Understand the environment
• Understand business technical processes
• Assess risks controls
• Develop improvements

6
Defining Your Scope

• Understand the business environment the
  application supports
• Develop a complete understanding of the business
  process flow (inputs, processing, outputs),
• and, the data flow, including interfaces

7
Defining Your Scope

• Understand the applications technical
  environment
• IT control environment
• Off-the-shelf or customized
• Legacy or web-based
• Housed internally or external provider
• Developed in-house or 3PP
• User population
• Dispersion of users

8
Defining Your Scope

• Understand the business process and technical
  environment
• Assess business information risks

9
Defining Your Scope

• Business Information Risks
• Regulatory compliance
• F/S integrity
• PCI-related
• Integration of an acquisition
• Data privacy
• Integrity of the application
• Process control validation

10
Lessons Learned on Scope

• Critical dependencies personnel, technical,
  external providers BCP
• Canned still seems to get modified
• Dont forget spreadsheets report-writers!

• Leverage existing flow diagrams
• Invest the time to understand the process flow...
• Adds value to your internal client
• You may find that you are now the expert!

11
Developing Your Approach

• Top-down Approach
• Auditing around the application
• Information Technology General
• Controls (ITGCs)
• Bottom-up Approach
• Auditing the insides
• Application Controls Testing

12
Top-down Approach

• Relationship between ITGCs and application
  controls
• Development
• Change Management
• Security Administration
• Operations

13
Top-down Approach
Source COBIT 4.1 Framework
14
Top-down Approach

• Development
• AI2 Acquire and Maintain Application Software
• AI2.7 Development of Application Software
• Ensure that automated functionality is developed
  in accordance with design specifications,
  development and documentation standards, QA
  requirements, and approval standards.

Source COBIT 4.1 Framework
15
Top-down Approach

• Change Management
• AI6 Manage Changes
• AI6.1 Change Standards and Procedures
• Set up formal change management procedures to
  handle all requests (including maintenance and
  patches) for changes to applications, procedures,
  processes, system and service parameters, and the
  underlying platforms in a standardised manner.

Source COBIT 4.1 Framework
16
Top-down Approach

• Security Administration
• AI2 Acquire and Maintain Application Software
• AI2.4 Application Security and Availability
• Address application security and availability
  requirements in response to identified risks and
  in line with the organisations data
  classification, information architecture,
  information security architecture and risk
  tolerance.

Source COBIT 4.1 Framework
17
Top-down Approach
Application Controls
18
Lessons Learned on Top-down

• Security roles and functionality
• Software Development Life Cycle
• Test plans supporting documentation
• Testing to break vs. testing to pass

19
Lessons Learned on Top-down

• Dispersion and diversity of the business,
  processes, and technology compounds the effort
• Dont underestimate the value of a thorough
  general controls review!

20
Bottom-up Approach

• Auditing the functionality and control
  effectiveness of the application can only be
  determined based on the maturity level and
  effectiveness of the ITGCs
• Auditing the insides Application Controls Testing

21
Bottom-up Approach
Input
Processing
Output
22
Bottom-up Approach

• Understand the business process and technical
  environment
• Assess business information risks

23
Bottom-up Approach

• Understand the transaction process related to the
  application identify key transactions
• Assess Application Controls

24
Bottom-up Approach

• Transaction and Data Flow Detail

25
Bottom-up Approach
Application Walk Through Key Screens
26
Bottom-up Approach
Source COBIT 4.1 Framework
27
Bottom-up Approach

• Application Controls
• AC1 Source Data Preparation and Authorisation
• Ensure that source documents are prepared by
  authorised and qualified personnel following
  established procedures
• AC2 Source Data Collection and Entry
• Establish that data input is performed timely
• AC3 Accuracy, Completeness and Authenticity
  Checks
• Ensure that transactions are accurate, complete
  and valid. Validate data that were input, and
  edit or send back for correction as close to the
  point of origination as possible.

Source COBIT 4.1 Framework
28
Bottom-up Approach

• Application Controls
• AC4 Processing Integrity and Validity
• Maintain the integrity and validity of data
  throughout the processing cycle.
• AC5 Output Review, Reconciliation and Error
  Handling
• Establish procedures and associated
  responsibilities to ensure that verification,
  detection and correction of the accuracy of
  output occurs.
• AC6 Transaction Authentication and Integrity
• When sharing data between internal applications
  and business/operational functions maintain
  integrity.

Source COBIT 4.1 Framework
29
Bottom-up Approach

• Testing Documentation
• Application Function (screen mapping)
• Function Description
• Testing Procedures
• Control Observations (testing results)
• Recommendations
• Key Risks Impact
• Lots of screen shots!

30
Bottom-up Approach
sample
31
Bottom-up Approach

• Testing Procedures
• Good code vs. not-so-good code

• Sequence checks
• Limit checks
• Range checks
• Validity checks
• Reasonableness checks

• Table lookups
• Existence checks
• Completeness checks
• Duplicate checks
• Logical relationships

32
System-Based Data Entry Integrity Controls
33
Bottom-up Approach

• Testing Procedures contd.
• Field formats are defined locked
• Grayed fields better yet, linked to user and
  displayed only if applicable to functionality
• On screen, visual feedback
• Not-so-good better have monitoring reports as
  mitigating controls!

34
Lessons Learned on Bottom-up

• Applications pre-dating the current climate of
  control
• Baselining
• Value of an implementation project audit
• Application functionality best if conducted
  throughout testing stage
• End user training desktop procedures

35
Lessons Learned on Bottom-up

• Monitoring controls issues typically arise from
  the one 1,000,000 transaction, not from the
  million 1 transactions
• Please remember to conduct audit work in a Test
  environment
• Dont underestimate the time commitment!
• Emerging technology trends web-based, PDAs, now
  smartphones

36
Summary

• Risk considerations for application auditing
• Risk-based approach drives increased efficiency
  decreased costs organizational value

37
Questions Discussion
Mike Kirk t 614.403.7700 e m_kirk01_at_yahoo.com