Fasilkom UI

INTRODUCTION TO INFORMATION
SYSTEMS AUDIT

Introduction to
Cobit Framework – Week 3
 Agenda

Cobit Context
Cobit Principle
 Bussiness Focused
 Process Control Oriented
 IT Resources
 Performance Measurement
Overall Cobit Interrelationship

2
 Introduction - Challenges
 Hardware and software technology
constantly changes Senior managers of
 Manage relationship between the IS function
information systems and other functions
 Role of information systems in
competitive strategy
 Auditors can evaluate top management
by how well they perform their four major
functions: Planning, organizing, leading
and controlling
 Evaluating the Planning Function

Top management is responsible for preparing a

master plan for long and short term IS
 Recognizing opportunities and problems
 Identifying the resources required
 Formulating strategies and tactics to
acquire the resources.
Auditors evaluate whether senior
management has formulated a high
quality information system plan
 Poor IS planning can lead to controls
deteriorating and loss of competitiveness
 Types of Plans - Strategic

Current information assessment

 existing IS systems, platform, personnel , technology,
strengths , weaknesses and opportunities
Strategic directions
 future information services
Development strategy
 Vision statement for IT application and
databases, platform, finances, implementation
Operational plan covers one to three years
 Progress report, Initiatives, Implementation schedule
 First Step in Developing an
IT Strategic Plan

 Understand the business objectives whether

stated or implied.
 Guides management in evaluating investments,
assessing risk, or implementing controls
Example: Business strategy for an online
bookstore
 “The business should have desired outcomes-
market share gains, higher customer satisfaction
levels, and shortened cycle times.”
 Question: please figure out where IT factor into
that.

6
 Linking Business and IT
BUSINESS/IT ALIGNMENT STRATEGIC IT PLANNING CYCLE PRIORITIZATION & FUNDING

PROJECTS & PROGRAMS

BUSINESS STRATEGY

Corporate &
Project And Application Global Project-specific
Driven Requests Architecture Architectures
Direction

Technology

AAL
L
EAI / BPI / BPM

Other

TTUU
Operational Data

EPP
CRM

CE
NC
Agent Pr oduct

ON
Core P&C / Life Applications Supporting Claims

Request

CCO
Information Definitons

Steering
Business
Applications
Browser Rating
Rating Underwriting
Underwriting Imaging
Financials
Financials Home UW Rules

Technology
Engine Engine Data
Engine Engine (SAP)
(SAP)

Internal &
Windows Needs
Needs Human
Human
Producer
Producer Resources Specialty Auto Rating Tables
Analysis
Analysis Resources
Manager
Manager
Engine
Engine
Commissions
Commissions
Telephone
Rating Claims Financial
Rating Claims Commer cial Financials
B2B
B2BElectronic
Electronic Institutions
Workbench Manager

Evaluation
Workbench Manager Interfaces
Interfaces

Committee
Pager

External
Licensing
Licensing//
Product
Product Underwriting
Underwriting Per sonal Profiles Billing
Appointment
Appointment
Workbench Workbench

Requests
Workbench Workbench Manager
Manager
Terminal
Contract
Contract // Customer Human
Proposal Data Registration Life
Proposal Data Capture
Capture Registration Info Resources
Manager
Manager Manager
Manager Manager
Manager
Wireless
Forms Data Warehouse
Forms
Policy Scheduled Fulfillment
Fulfillment
Policy Scheduled

Requests
PDA Issuance
Issuance Activities
Activities
Engine
Engine Engine
Engine Billing
Billing

Data Marts
Forms
Forms && Correspondence
Correspondence Engine
Engine Reporting
Reporting

Long Term Blueprint

PROGRAM RESULTS
PLAN UPDATES

Operations And Infrastructure Industry

Driven Requests Analyst,
Vendor,
Expert
Input

MONITORING & CONTROL PLAN REVIEW AND FEEDBACK REVIEW PROCESS

7
 IT Architecture vs.
Infrastructure
 IT infrastructure
 Physical facilities, services, and management
that support all computing resources in an
organization.

 IT architecture
 A high-level map or plan that explain & guide
how IT elements work together
• Business activities and processes
• Data sets and information flows
• Applications, software, technology

8
 Contingency
Approach to Planning(1)

 Harvard - McFarlan
 Support – small planning
 Factory – short run resource needs
 Turnaround – long
run application needs
 Strategic - both
 Sullivan
 Traditional
 Federation
 Backbone
 Complex
 Contingency Approach to
Planning (2)
 Harvard - McFarlan
 Support – small Importance of Proposed
planning Systems
Low High
 Factory – short run
resource needs
 Turnaround – long Support Turnaround
Importance of Low
run application needs
Current
 Strategic - both
systems Factory Strategic
 Sullivan High
 Traditional
 Federation Systems Infusion and Integration
 Backbone Low High
 Complex Systems Low Traditional Backbone
Diffusion
and dispersion High Federation Complex
 Strategic IT Planning

Provides a roadmap for operating plans

Provides a framework for evaluating technology
investments

“The truth is that those IT leaders who don't

master the art of strategic planning won't last
long.” (CIO.com, 2008)

Challenges:
 Many companies lack well-defined strategies

11
 Types of Plans - Operational

 Progress report
 Current plan initiatives achieved or missed
 Platform changes
 Initiatives to be undertaken
 Systems, platform, personnel, financial
resources
 Implementation schedule
 start / finish dates, milestones,
control procedures
 Role of the Steering Committee

Take ultimate responsibility

Functions and makeup depending upon
how critical IS is to the organization
 Strategic Organizations - chaired by CEO
 Support - Middle management
 More Diffusion - broader membership
 More Infusion - steering committee
much more important
Role of the Steering Committee
 Evaluating the
Organizing Function
Segregation Of Duties
Resourcing
Staffing
Centralization Versus
decentralization of the information
systems function
Internal organization
Location
 Segregation of Duties

Ensures that single individuals do not possess

excess privileges that could result in
unauthorized activities such as fraud or the
manipulation or exposure of sensitive data.
The choices for mitigating a SOD issue include:
 Reduce access privileges
 Introduce a new mitigating control
Segregation of Duties
 Resourcing the IS Function

 Acquire resources needed

 Hardware software,
Problem Related
personnel, finances, and
Resourcing
facilities the IS Function
 Detailed requirements
Projects late?
 Requests for proposals
Projects cancelled?
 Submissions evaluated Moral in IS?
 Contracts Day-today operations OK?
IS role understood by
 Testing and modification top management?
 Staffing the IS Function

 Personnel acquisition
 Top management evaluates the integrity and capabilities
of applicants
 Background check, screening mental and physical health,
bonding, explaining organizational protocols,
indoctrination
 Personnel development
 promotional and personal growth opportunities
 Education, reviews, identifying opportunities for
personal growth, training and continuing education
 Personnel termination
 Notification, security review
 replacement training, exit interview
Centralization Vs Decentralization
of The IS Function
Advantages
 Centralization
• better control and economies of scale
 Decentralization
• more flexible and less communication cost
Dimensions
 control - responsibility for decision making about
IS
 location of facilities
 functions - development, operations, maintenance
 Internal Organization of IS
 Workstation Specialist
 Systems Analyst
 End/User Support
 Application Programmer
 Quality Assurance
 Systems Programmer
 Executive IS
 Data Administrator
 Expert Systems
 Database Administrator
 Operations
 Security Administrator
 Operator
 Network Administrator
 Librarian
 Data Entry
 Administrative Support
More Recent Organization
Job Title Position Description

Systems Elicits information requirements for new and existing

analyst applications; designs information systems
architectures to meet these requirements; facilitates
implementation of information systems; writes
procedures and user documentation.

Application Designs programs to meet information requirements;

programmer codes, tests, and debugs programs; documents
programs; modifies programs to remove errors,
better meet user requirements, and improve
efficiency

Systems Maintains and enhances operating systems software,

programmer network software, library software, and utility
software; provides assistance when unusual systems
failures occur.
Data Elicits the data requirements of the users of information
administrator systems services; formulates data policies; plans the
evolution of the corporate databases; maintains data
documentation.
Database Responsible for the operational efficiency of corporate
administrator databases; maintains access control over the database;
assists users to use databases better.

Security Implements and maintains physical and logical security

administrator over the information systems function; monitors the
status of security over the information systems
function; investigates security breaches; assists users to
design controls; maintains access control mechanisms.

Network Responsible for planning, implementing, and

administrator maintaining data and voice networks.
Workstation Advises on the selection, implementation, operation,
specialist and maintenance of different types of workstations,
e.g., data entry workstations, end-user workstations,
computer aided design workstations.

End-user/client Advises end users on analysis, design, and

support specialist implementation of systems; determines needs for end-
user tools; supports use of end-user tools, e.g., high-
level languages.
Quality assurance Establishes quality control standards for the
specialist information systems function; ensures all new and
modified systems conform with quality assurance
requirements before they are released into production.

Executive Elicits requirements and designs and builds executive

information information systems and decision support systems;
systems/decision undertakes corporate modeling; determines needs for
support systems new executive support and decision support tools.
specialist
Expert Elicits requirements; designs, builds, and maintains
systems expert systems; documents expert systems; determines
specialist needs for new expert-systems tools.

Operations Plans and controls day-to-day operations; monitors and

specialist improves operational efficiency; assists with capacity
planning.

Operator Operates and maintains computer equipment.

Librarian
Data entry Maintains library of magnetic media and documentation.
operator Prepares and enters data at workstations or terminals.
Administrativ Maintains and operates transfer pricing system; acquires
e support clerk consumables needed by the information systems
function; registers and follows up on user complaints;
maintains and operates information systems function
accounting systems; handles user inquiries; collates and
distributes reports.
 Location of IS

Depend upon McFarlan’s Strategic grid

 Separate department?
 Under top management or controller?
 Dispersed to user groups?
 Leading the IS Function

Motivating IS Matching leadership

personnel styles with IS personnel
 Auditors should and their jobs
examine variable
which may indicate  Authoritarian to democratic
motivation levels Effectively communicating
• turnover, failure to with IS personnel
meet budgets,
absenteeism  Examine evidence of
communication
 Interviews
 Controlling the IS Function

Overall control
Technology diffusion and control
Control of IS activities
Control over users of IS services
 Overall Control of IS

How much? Value for Money?

Industry averages
Benchmarking
Look at spending as a capital investment
rather than an expense.
Post implementation - Benefits Versus Costs
Sustaining competitive advantage / cost
savings / obsolescence
 Technology Diffusion and
Control of IS
Nolan S curve – Stage Growth of IS
Expenditure :
 Initiation
• New installation, little control, loose budget, FIFO
 Contagion
• promotion of use, high status, lax budget, few
standards
 Control
• control oriented management, many controls,
budgets
 Integration
• Resource oriented planning and control,
refinement, master plan
 Control of IS Activities

Establishment and Type Of Standard

enforcement of:  Methods Standards
 Policies - broad  Performance Standards
general guidelines  Documentation Standards
 Standards -  Project-Control Standards
specific guidelines
 Post Audit Standards
for behavior
 depends upon
type of structure
 Control over User of IS Services

Zero Based Budgeting Type of charge

 Highlight applications  Allocated cost
which have outlived  Standard Cost
their usefulness  Dual Price
Options for transfer  Negotiated Prices
pricing and charge-out  Market Price
 Cost center Purpose and other
 Profit Center factors
 Investment Center  stimulate innovation
 Hybrid Center  responsibility level
 maturity level
 Symptom Problem In
Management Control/IT
Governance
Discontentment
 Burned-out or overworked IT staff, low IT morale, high
turnover, and malaise among end users (about IT-
supported systems)
 IT department that lacks maturity and is falling behind
on its methodology or is applying Band-Aid fixes to
systems.
Poor system performance :
 Excessive incidents of unscheduled downtime, a large
backlog of support tasks, and long wait times indicate a
lack of attention to the quality of applications.
 Symptom Problem In Management
Control/IT Governance
 Nonstandard hardware or software
 A mix of hardware or software technologies among applications or
end-user systems may indicate a lack of technology standards, or
the failure to enforce standards that are already in place.
 Project dysfunction
 IT department suffering from late projects, aborted projects, and
budget-busting projects indicates a lack of program and project
management discipline.
 Highly critical personnel
 A disproportionate over-reliance on a few IT personnel indicates
that responsibilities are not fairly apportioned over the entire IT
staff. This may be a result of a lack of training, unqualified
personnel, or high turnover.
 Summary
 Evaluation of Top Management :
 Require a sound knowledge the principle of good management
 Ensure to evaluate planning, organizing, leading & controlling
function
 Determine critical area for each function in control perspective.
Fasilkom UI

INTRODUCTION TO INFORMATION
SYSTEMS AUDIT

Systems Development (SD) and

Management Controls
 Introduction

SD management has responsibility for those

functions concerned with IS :
 analyzing / designing,
 building / implementing
 and maintaining
Art rather than science
Auditors can conduct three types
of reviews of the SD process
Major approaches to SD
Tasks in SD and controls over tasks
 Approaches to Auditing SD

Concurrent Audit - Participate as a team member

 Early correction of errors versus independence
 What should happen? What has happened?
Corrections?
Ex Post audit - Post-implementation review
 What went right / wrong during the process?
 Likely strengths and weaknesses of the system?
 System scrapped, continued , modified?
General audit - Evaluate SD
process in general
 Can we reduce the extent
of substantive testing?
 Normative Models of the
Model for Systems
SD Process
Actual Practice
Development

Discrepancies and inefficient

or ineffective controls

Systems development life cycle approach

Socio-technical design approach
Political approach
Soft-systems approach
Prototyping approach
Contingency approach
Systems Development Life Cycle
Approach
Application of Project management techniques
Each phase should:
 be planned and controlled Plans
Plans
 comply with development standards
 be adequately documented
Schedules/
 be staffed by competent employees Milestones
 have project checkpoints and signoffs
Phases can occur concurrently Documentation

Phases can be iterated

Standards
Systems Development Life Cycle
Approach
Feasibility Study

Information Analysis

Systems Design

Program Development

Procedures and Forms Development

Acceptance Testing

Conversion

Operation and Maintenance

 Socio-technical Design
Approach
Avoid Resistance / sabotage with SDLC
Impact on organization structure design
Steps Interactions
Social Technical
•Diagnosis and Entry
system system
•Management of the
Change Process
•System Design
High quality High Task
(Technical and social) of life Accomplishment
•Adjust Coordinating
Mechanisms
•Implementation
Joint
Optimization
 Start
Political Approach
Historical analysis to determine
ST - involve users power structure

Users undermine
progress Will
No proposed Yes
IS changes power system change
IS - influence others the power
structure?
IS - symbolic power
Replace involvement
with negotiation User Face to face negotiation
participation and compromise
Confront users
Powerful ‘fixer’ Continue
 Soft-systems approach
Recognize the problem situation
 problem solver
 problem owner
 decision taker (power)
Express the problem situation
 roles, norms and values
 rich pictures
Produce root definitions of relevant systems
 customers, actors and transformations,
Weltanschauung, owner, and environment
(CATWOE)
 Soft-systems approach
Develop conceptual models of relevant systems
 ‘systems thinking’
 ideal model
Compare conceptual models
with perceived problem situation
 exploration
 diagnosis
 design
Identify desirable and feasible changes
Take action to improve the problem situation
 Soft systems approach

Recognize the Take action

problem
situation Identify desirable
and feasible
Express the changes
problem
Compare
situation
conceptual model
and problem Real World

Systems Thinking

Produce root Develop conceptual

definitions models of relevant
systems
 Prototyping approach
Elicit user
requirements

Powerful low-cost microcomputers

End-user development
Do end users have
Design
prototype
Powerful high-level, end-user
sufficient
programming languages
knowledge to Implement
 users develop their own systems
design and prototype
 rapid development of prototypes
implement high-
Users have a central role
quality in systems
information
development Use prototype
systems
 strategic and decision support systems
Rapid iteration between alternate
designs Build production
system
Inefficiency overcome by faster
hardware
 Contingency approach
Social systems impact
Tasks systems impact
Systems size
Commonality
Requirements uncertainty
Technology uncertainty
Identify the factors with the most impact
on the development process

Decide on the best development

Evaluating the Major Phases
of The SD Process
What basis can auditors use to evaluate the
process?
How can they obtain assurance about the
quality of the information systems when the
development process is widely dispersed?
How can they rely on controls when in some
cases developers and users take for granted
that they do not fully understand the nature of
the systems they are trying to develop?
Assume a common list of phases - agenda of
issues for stakeholders (designers, users,
 Evaluating the Major Phases of The
SD Process (1)
Problem / opportunity definition
Management of the change process
Entry and feasibility assessment
Analysis of the existing system
Studying the existing organizational history,
structure, and culture.
Formulating strategic requirements
Organizational and job design
Evaluating the Major Phases of The
SD Process (2)
Information processing systems design
Elicitation of detailed requirements
Design of the data/information flow
Design of the database
Design of the user interface
Physical design
Design of the hardware /system software platform
 Evaluating the Major Phases of The
SD Process (3)
Application software acquisition and development
Hardware/system software acquisition
Procedures development
Acceptance testing
Conversion
Operations and maintenance
 Auditor’s Consideration

How does the conduct of each phase vary

depending upon:
 the system’s task and social impact?
 the size of the system
 the commonality of the system?
 the requirements and technological uncertainty?
How will controls differ depending upon the
levels of these contingent factors?
 Problem / opportunity definition
Stakeholder concerns about -> Nature of the
problem
 Structured? Scope? Definition clear? Impact of structures
and jobs? New technology?
Auditors concerns about the activities carried out:
 Large systems ?-> Are there formal terms of reference
approved by steering/project committee?
 Large impact on tasks or social systems? - Acceptance
levels high among stakeholders? -> Need for negotiation
and consultation?
 High levels of requirements or technological
uncertainty? -> Strategies to alleviate
uncertainty?
 Do stakeholders agree on problem definition?
-> Approaches to reach consensus?
 Management of the change
process
Change facilitating activities parallel project
management activities
 Unfreezing the organization - education,
feedback, participatory decision making
 Moving the organization
 Refreezing the organization - positive feedback
 Negotiation and compromise
Auditors evaluate the quality of decisions made
about project management and change
facilitation - contingency perspective
 Entry and feasibility
assessment
Technical feasibility:Technology acquired or
developed?
Operational feasibility: Input data available? Output
useable?
Economic feasibility: Benefits exceed costs?
Technical Impact on quality of user’s life
Behavioral feasibility:
Operational

Economic

Behavioral

Systems development process

Stop Go
 Analysis of the existing system
.
Studying the existing organizational history,
structure, and culture
Studying the existing product and information flows

Auditors:
Oldabout
System Culture
Evaluate designers decisions what needed to be studied and to
what extent
Nature and extent
Historyof examination Structure
High-quality methodology New System
Computer aided software engineering tools CASE
Product Flows
Information Flows
 Formulating strategic
requirements
Vague or specific?
Auditors evaluate:
Early
 Doorsystems
late? designers recognize the
importance of articulating strategic
requirements for the quality of subsequent
Strategic requirements are
design work?
identified based on
 If there are substantial behavioral impacts,
perceived deficiencies in
are there procedures in place to reach
the existing system or
agreement on strategic requirements?
perceived opportunities for
If substantial uncertainty surrounds the
enhanced task
proposed system, they should examine and
accomplishment and
evaluate the procedures to help clarify
quality of working life.
strategic requirements.
 Organizational and job design

Redesign of jobs and structures

Match with strategic requirements
Behavioral problems of change
Auditors evaluate:
 advice sought from experts?
 Redesign included stakeholder representatives
 Consensus reached and conflict avoided?
 Impact on control risk?
 Information processing systems
design
Data and Information
Flow design

User Interface
Design
Database
Software design

Platform Design
Requirements
Elicitation
 Elicitation of detailed
requirements
Ask the stakeholders what they require
Discover the requirements through analysis and
experimentation
Have designers chosen
appropriate
requirements-elicitation
strategies, and
methodologies.
Evidence of satisfactory
consensus and
documentation?
 Design of the data/information
flow
The flow of data and information and the
transformation points
The frequency and timing of the data and
information flows
The extent to which data and information will be
formalized

Evaluate the activities

carried out – does the
design meet
requirements.
 Design of the database
Conceptual modeling – entities/objects, attributes,
relationships, static and dynamic constraints,
triggers and business rules
Data modeling – relational or other data model to
permit manipulation by SQL and program
languages.
Storage
Auditors should
structure – records,
evaluate
design : tuples,
Database
relationship scope and pointers
structures
Waslayout
Physical the structure
design –designed using locations,
storage media,
well known
client/server, database
internet, design systems
enterprise
methodologies?
Case tools used during the design?
 Design of the user interface

Source documents to capture raw data

Hard copy output reports
Screen layouts for dedicated source document
input
Inquiry screens
Command languages
Auditors should evaluate:
Interrogation languages
Critical activity – major source of
Graphical and color
control displays
as users interact with the
Voice input
system.
or output
Light penValidation,
or mouse error control
Icons Good design practices followed?
 Physical design

Hardware
Batch / on-line / real time
Cycle
Design of the hardware /system software
Auditors should evaluate:
platform
Efficiency and effectiveness
Good design practices followed?
Adequate testing?
Modularity and generality– ease of
upgrade and change
Quality of connections and communication
 Application software acquisition
Auditors should evaluate
and: development
Acquired software
Software acquired or developed
 Quality of specifications to vendors
Generalized packages configured
 Quality of procedures to evaluateand perhaps
software:
modified and accuracy,
functionality, adapted. completeness,
Prototyping
documentation, vendor stability and support,
SDLC
natureand program development from scratch –n
of contracts
Quality
see next of software and maintenance
chapter.
Developed software
 Procedures during developme4n see next
chapter
Control risks during development
Testing and implementation
 Insertion of audit routines and modules
 Hardware/system software acquisition

Purchased hardware
Request for proposal
Vendor submission evaluated
Selection process

Auditors should evaluate:

Evidence of above process
Review documentation
Quality of hardware acquired
 Procedures development

Design of procedures
Testing of procedures
Implementation of procedures
Documentation of procedures

Auditors should evaluate:

Quality of procedures design
Compliance obtained – stakeholders
represented.
Approach to testing of procedures
 Acceptance testing
Identify errors and
deficiencies in the system
Auditors should evaluate:Program testing
Need a plan
How was the testing processSystem planned?testing
Howbewere
Can’t complete User and
the test data designed Testing
 developed?
Number of execution paths Quality assurance testin
 What
Need experience with the
test data were used?
systemtest results were obtained?
What
Many conditions
 What actions werecan’t taken
be as result of errors or
simulated
deficiencies identified?
 Difficulty of determining
What subsequent modifications to test data
correctness
were made in the light of testing experience?
 Cost
How was control exercised over test data and
 Conversion

Steps to place the new system in operation

Abrupt changeover
Phased changeover
Parallel changeover

Personnel training
Installation of new hardware and software
Conversion of files and programs
Schedule of operations and test running
 Operations and maintenance

Repair maintenance
Adaptive maintenance
COBIT Guidleines
Planning & Organization
Define a Strategic Information Technology Plan
PO1 Maturity
Model