Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Naresh K. Kaushik
Delft University of technology
PREDICTION OF
PROJECT
PERFORMANCE
Naresh Kaushik
Student Number 4141555
PROJECT DETAILS
Author: Naresh K. Kaushik
Student Number: 4141555
Email: naresh_kasuhik@hotmail.com
This report is for thesis graduation project for:
Graduation Date:
Graduation committee:
PREFACE
This thesis report on development of performance prediction model is the result of my
graduation thesis for master program System engineering policy analysis and management
at Delft University of technology. I performed this thesis research as graduation intern at
Fluor Corporation at their Haarlem Office.
The past 7 months of my master thesis has been a great learning experience academically,
professionally and personally. The research topic turned to be quite complex and resulted
into lot a large scope for research. However, I enjoyed every bit of this research.
At the conclusion of my research, I convey my warm thanks to my supervisors at TU
delft: Mamadou Seck, Wijnand veeneman and Alexander Verbraeck for their continuous
support and encouragement. My special thanks to Mamadou seck for extra support in form
of frequent meetings and discussion that help my research.
I would like to thank Robert V. Velzen for providing me this opportunity to conduct this
research at Fluor and his invaluable role as my supervisor. In addition, I would like to
thank Erik for his continuous guidance and feedback on my research. Furthermore, I
would like to thank everybody at Fluor Haarlem that contributed to my research in form of
semi-structured interviews and informal discussions.
Finally, I would like to thank Kees Berends, Professor Hans Bakker from shell and Ted
Ong from Exxon for providing their useful insights during interviews.
I hope you all enjoy reading the results
Naresh Kaushik
Delft, March 2013
EXECUTIVE SUMMARY
The projects in oil and chemical (O&C) industry often experience problems during their
execution, because of those problems, some of the project ends with large cost and
schedule overruns. The poor performance of projects not only affects the strategic objective
of projects owner but also poses a dual threat to engineering and construction (E&C)
companies. They negatively affect their profit margins and their business objectives. Given
the strict budget constraints imposed by the present global economic situation, owners and
stakeholders expect their projects to be delivered cost effectively and efficiently. Therefore,
it is important for E&C companies to strive for improvement in their project management
practices.
The current thesis research is a step in direction to introduce a new concept for
improvement in performance management practices. For that purpose, the research
introduces early detection of project problems as the main instrument and uses the
quantitative information from past project to develop a body of knowledge and first
conceptual model to predict the future performance of projects at their early stages.
The research is conducted in five phases, the first phase of the research explores O&C
project and their performance management practices. Based on the gathered knowledge via
literature study and available information, the main research question is formulated as
How can future problems and performance of a current O&C project be predicted at early
stages using knowledge and experience from past projects in an EPC environment?
Thereafter, a series of sub questions were formulated aimed to answer the above-mentioned
research question. The later part of the first phase developed a structured research approach
and research methods.
In the second part of the research, efforts were directed to find the so-called early
warnings of problems. To identify the early warnings, two main sources were explored,
literature and experts from O&C project industry. Each investigation into respective
sources resulted into number of early warnings. Each identified early warning was
evaluated on selection criteria with three selection parameters. After the careful evaluation,
the following ten early warnings were selected.
ID
LES
PTE
COC
NCO
CCO
FED
PH
PS
CE
The selected early warnings were carried to the third phase of the research, in which four
detailed case studies were performed to have observatory evidence. The case studies in this
phase consisted of four project with different performance levels. The difference in
performance levels of case projects set the contrast in which the predictive capability of
early warnings could be observed. The case study investigation found that there is a
relationship between early warnings, project problems and project performance.
After obtaining the observatory evidence, the fourth phase of the research adopted a purely
quantitative approach and studied the behavior of early warnings in a relatively larger set of
past projects. Subsequently correlation analysis was performed to find correlations between
early warnings and final project outcomes (which collectively asses the project
performance). The quantitative analysis did present interesting and encouraging results.
The main results are mentioned as follows:
I. Early warnings do behave differently in case of poor and good performance
projects, few in terms of their absolute value and few in their incremental changes.
II. Correlations do exist between EWI and project outcomes, however not all the EWI
found to be correlated with all project outcomes.
III. The EWI indicators does show a dynamic quantitative relationship with project
outcomes over engineering duration of the project
Using the results from quantitative analysis, an attempt is made in the last phase of this
research for the development of prediction model, which can predict the future
performance of projects. The results of pilot prediction model were analyzed and compared
with forecasts made via traditional forecasting methods. The comparison of forecasts found
that prediction model does make prediction that is more accurate. However, there are errors
with-in prediction models. In addition, the external validation of model suggested limited
reliability and accuracy of pilot model.
The dataset used for quantitative analysis and building of prediction model is relatively
small and limit the generalization of findings. Therefore, to have a more accurate prediction
in good projects, a dataset is required which contains a balance of Successful and less than
successful performance projects. Despite the smaller dataset, the findings and approaches
presented in this research can be used to build a useful model and subsequently applied in
O&C project industry. A set of insights and recommendations (short term and long term)
has been made for Fluor to implement the findings of this research to develop an
operational performance prediction system.
The research possibly has following main contributions to scientific and industry.
Contribution to scientific community
I. A shift from reactive project management to proactive project management
TABLE OF CONTENTS
1
Introduction .....................................................................................................................12
Literature study................................................................................................................34
4.1
Summary ................................................................................................................34
4.2
Author affiliations..................................................................................................34
4.3
Concept of project success.....................................................................................35
4.4
Concept of early warnings.....................................................................................40
4.5
Conclusions and discussions .................................................................................45
6
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
7
Case Studies.....................................................................................................................63
Summary ................................................................................................................63
Case study design ..................................................................................................63
Case study selection...............................................................................................64
Case 1 (Less than successful project) ....................................................................65
Case 2 (successful project) ....................................................................................68
Case 3 (successful project) ....................................................................................71
Case 4 (Less than successful project) ....................................................................73
Cross case analysis................................................................................................77
Discussion and conclusion ....................................................................................79
Quantitative analysis .......................................................................................................81
Summary 81
7.1
Analysis approach .................................................................................................81
7.2
Exploratory data analysis......................................................................................82
7.3
Quantitative analysis .............................................................................................85
7.4
Correlations over engineering duration ................................................................89
7.5
Discussion and conclusions...................................................................................93
8
10
11
References......................................................................................................................135
LIST OF FIGURES
Figure 1 Success and failure of O&C projects........................................................................................... 13
Figure 2 Phases of OG&C projects............................................................................................................ 17
Figure 3: The generic control cycle............................................................................................................ 19
Figure 4: Conceptual procedure for controlling of projects ..................................................................... 21
Figure 5: Cost of reactive approach........................................................................................................... 23
Figure 6: Existing knowledge gaps............................................................................................................. 24
Figure 7: The wheel of science (Wallace, 1971) ........................................................................................ 29
Figure 8: Fundamental research approach................................................................................................ 31
Figure 9: Iron triangle of projects.............................................................................................................. 36
Figure 10: Project success criteria............................................................................................................. 37
Figure 11: 95 % engineering completion milestone .................................................................................. 39
Figure 12 Potential benefits of EWI ........................................................................................................... 45
Figure 13: Early warning selection criteria............................................................................................... 48
Figure 14 Classification of early warnings from literature by source ...................................................... 51
Figure 15 Early warnings from literature by sub-category ....................................................................... 52
Figure 16: Early warning from experts by sub-category........................................................................... 56
Figure 17 Early warning mentioned by numbers of experts ...................................................................... 57
Figure 18 Framework for mapping the relationship between early warnings, project problems, and
project outcomes.......................................................................................................................................... 64
19-27 Confidential
Figure 28: Quantitative analysis approach................................................................................................ 82
28-37 Confidential
Figure 38 Significant correlations between EWI and project outcomes at each prediction moment...... 91
Figure 39 Significant correlations of project outcomes with EWI over engineering duration................. 92
Figure 40 : Step approach for development of prediction model ............................................................ 101
Figure 41 Predictive capability comparison of traditional method and developed prediction tool....... 105
Figure 42 : Errors in prediction of final TIC ........................................................................................... 108
Figure 43 Errors in prediction of ESI....................................................................................................... 108
Figure 44 Errors in prediction of MHI..................................................................................................... 109
Figure 45 Errors in prediction of MCI ..................................................................................................... 110
Figure 46: Model validation: prediction of TIC....................................................................................... 111
Figure 47 Model validation: prediction of MCI....................................................................................... 112
Figure 48: Model validation- prediction of ESI ....................................................................................... 113
Figure 49: Usability of prediction Model................................................................................................. 121
Figure 50 future project problems and project outcomes associated with NCO .................................... 128
Figure 51: synthesis of answer to main research question ...................................................................... 129
Figure 52: Data collection moments ........................................................................................................ 153
10
KEY ABBREVIATIONS
O&C
EPC
E&C
E&P
BOD
Basis of Design
BDP
CII
FEED
IPA
PEP
EVM
EWI
ESI
TIC
MCI
COP
Cost of Problems
11
1 INTRODUCTION
The oil and chemical owner companies rely heavily on engineering and construction (E&C)
companies to meet their strategic objectives such as building of a new assets, expansion
and performance improvement of existing assets. Moreover, given the strict budget
constraints imposed by the present global economic situation, owners and stakeholders
expect their projects to be delivered cost effectively and efficiently. E&C companies are
working hard to match these expectations by changing their project management methods,
tools and the way they execute projects.
However, there are sufficient examples of projects, where E&C companies face problems
in meeting their as sold cost estimates, agreed upon schedules and desired quality
requirements. The number of project, that fail to meet their stated objectives vary
significantly per industry, mainly due to the difference in complexity, the industrys market
dynamics, the type of stakeholders and their influence levels. Many researchers
investigated the reasons for poor performance of projects (Flyvbjerg & Bruzelius, 2003;
Morris & Hough, 1987; Turner, 1999; Thamhain & Wilemon, 1986).
For example, handbook of project-based management by Turner mentions several reasons
for projects poor performance such as poor project establishment in terms of priorities, bad
initial planning, inefficient control procedures and many more (Turner, 1999). Flyvbjer and
Bruzelius (2003) suggested that in projects decision-making, planning and management are
typically multi-actor processes with conflicting interests and therefore, projects are often
faced with mistrust, violation of good project governance practices, ambiguity and poor
collective decision-making (Flyvbjerg & Bruzelius, 2003). The above-mentioned behaviors
of stakeholders penetrate through the permeable boundaries of project plans and can lead a
project to high cost and schedule overruns.
In this respect, projects in the oil and chemical (O&C) industry are no exception. Although
the performance of O&C projects seems to be better than that of civil or mining projects,
there are still ample examples of poor performing projects. Mckenna, Wilczynski and
Vandersee (2006) estimated that about 30-40 % of capital project in O&C industry suffer
from a budget and/or schedule overrun larger than 10%.
Figure 1 shows the result of a study conducted by Independent Project Analysis (IPA). The
study includes 318 projects across the O&C industry. Out of those projects, only 50% can
be categorized as successful. The other 50% incurred either 33% cost overrun and/or
schedule overrun of more than 30% (Merrow, 2012). Two third of the projects even failed
to meet the production schedule or targets, thus affecting the profitability of its investors.
The above results definitely are of serious concern for both the E&C and the owner
companies.
12
From the above discussion, it can be concluded that O&C projects had experienced
problems in the past and might encounter problems and challenges in future. There have
been multiple attempts by the academic as well as industry experts to explore potential
areas of improvement such as risk management, stakeholder management, benchmarking
practices, and project control practices to improve the situation. Accordingly, there have
been achievements such as development of value improvement practices (VIP), industrys
best practices, and front end loading (FEL) to mention a few. However, the majority of the
research has been focused in the frond end phase of the projects.
Prominently, the importance of the FEED phase for improving project performance is
suggested over the years (Artto, Lehtonen, & Saranen, 2000; Thamhain & Wilemon, 1986)
and little focus has been given to the execution phase of the project, where the problems
actually surface and affect the project performance. The control mechanism of project
execution phase (EPC) has seen little advancement and is still relying on the principles
defined in 50-60s such as principle of deviation management (Vanhoucke, 2011;
Nikander, 2002).
Why the deviation based traditional control mechanisms would not be suitable for
successful control of project execution? There are two major problems with traditional
deviation based control methods. First, the deviations are reported on aggregated level
therefore, the poor performance in one part of the project is masked by good performance
in other part of the project (Vanhoucke, 2011; Nikander, 2002). Secondly, even if the
localized deviations are observed, they are seen in limited manner. The cascading effect of
localized deviations on other activities is neither reported nor anticipated by these methods.
Therefore, the accuracy of future forecasts of project performance based on the deviations
is somewhat debatable.
As a consequence to above mentioned fallacies in deviation management principles, often
problems in a project are not visible until they are already manifested and degraded the
project performance. The corrective strategy to manifested problems can be termed as
13
reactive approach, as the mangers act to correct what has already gone wrong. This reactive
approach brings additional schedule requirements and incurs substantial cost, thus add into
the cost and schedule overrun of the projects.
In addition, forecasting of future project performance based on traditional methods is
vulnerable to optimism bias. The mangers are often seen as very optimistic against the
localized deviations and do not consider them as potential risk to future activities.
Interestingly, forecasters never mention optimism bias as a main cause of inaccurate
forecasts.
Then the question arises, where to look then for the improvement in the project control
management? A guided investigation of poor and good performance of past O&C project
could provide us an answer to this question. If problems could not be eliminated from
projects, can we predict problems, allowing longer correction time at lesser cost? This
capability will allow for their pro-active management with considerably less cost and
schedule impact.
The aim of this thesis is to take a first step in creating a scientific understanding of
prediction of problems via early warnings. Using this understanding, an attempt is made
within this research to build a quantitative model to predict the future performance of
project based on selected identified early warnings.
Looking at the different chapters that build this thesis, chapter 2 provides the background
for conducting this research by defining important concepts and delineating the main
research problem. The problem delineation guides the formulation of research questions.
Furthermore, research goals are introduced, the relevance of these goals is explained and
the main deliverables are defined.
In chapter 3, the design of this research is presented. The fundamental approach is
described with logical sequence of research phases. Subsequently, the employed research
methods and tools are explained and coupled with the goals set in chapter 2.
A literature study regarding project performance of O&C projects is provided in chapter 4.
The adopted measures of O&C project performance are presented. In this literature study,
the concept of early warnings is explored and relevant literature is reviewed. The chapter
also highlights the potential benefits of operationalizing early warnings in projects. Chapter
5 describes the identification of the early warnings from literature and from experts from
O&C project industry. Furthermore, the selection criteria for selecting key early warnings
are formulated, by focusing on the main objective of research. Each early warning is
evaluated on selection criteria and few were selected for further analysis.
In chapter 6, in-depth case studies are performed with an objective to have the preliminary
evidence of relation between early warning, problems and their relation with project
performance. In addition, Individual case conclusion and cross case analysis is performed
and presented.
In chapter 7, quantitative analysis is performed to 1) analyze the dynamic behavior of EWI
over engineering duration of the project to map the behavior with successful or less than
14
successful projects. 2) Correlation analysis of EWI with specific project outcomes and 3)
Longitudinal correlation analysis to find suitable EWIs for development of prediction
model at each prediction moment
In chapter 8, based on the past project data, early warnings are assigned quantitative
indicators and an effort is been made to build a performance prediction model. The results
of developed pilot model are analyzed and external validation is performed.
Insights and general recommendations are provided in chapter 9. In addition, a short term
and long term implementation strategy is presented in chapter 9.
Finally, chapter 10 concludes the research by revisiting the research questions and their
answers and evaluating the contribution to scientific and O&C project industry.
Reflections have been made towards research approach, adopted methods, and results of
the research.
15
2 RESEARCH DESCRIPTION
2.1 SUMMARY
The main objective of the current chapter is to understand the research problem, its context
and the research questions, which needs to be answered in order to find a solution to the
main research problem. In addition, the scientific and social relevance of research had been
provided.
This objective has been achieved sequentially by understanding the 1) execution of O&C
projects 2) their controlling mechanisms.
With the understanding of context, a critical review of current practices enabled the
delineation of research problem and existed knowledge gaps. The problem delineation
helped in forming the main research question. Furthermore, the main research question has
been broken into sub questions that need to be answered to obtain the solution to main
research problem.
The section 2.3.1 provides the overview of oil and chemical projects. Section 2.2.2
provides information on the subject of controlling mechanism of projects. Section 2.2.3,
integrates the above two sections and shift the attention specifically on current project
controlling mechanism employed in O&C projects.
Section 2.3 provides a critical overview of the current controlling mechanism and describes
the research problem. Based on the defined research problem, research questions are
formulated in section 2.4. Section 2.5 describes the research goals and main research
deliverables followed by relevance of research in scientific, social and business domains
(section 2.6).
O&C plants are also addressed as process plants, mainly due the fact that they have
chemical processes at their heart. The chemical process convert the input (Crude oil,
chemicals) into other chemicals with higher economic value (Fuels, industrial chemicals)
by means of mechanical equipments, auxiliary facilities and the infrastructure to support
the whole plant.
The process plants are strategic assets of major petrochemical companies and are
fundamental to their business. Furthermore, the O&C chemical projects should not be seen
just as economical assets, they do contribute significantly in meeting the rising demands for
energy of society as a whole. Although an increase in production of renewable energy is
expected, experts still believe that the O&C industry will play an important role as energy
producer, at least in near future.
16
O&C projects are capital intensive and do require systematic economic and project
planning to deliver their intended results. Therefore, almost every (O&C) project is
executed in systematic phases and its project life cycle encompasses the total time between
identification of the project need to its completion.
The different phases in project life cycle are (sub) projects in themselves and are separated
by gates or decision points. The gated project lifecycle means that at certain points in the
life cycle of project, the evolving design or plant concept and associated parameters (e.g.
cost, schedule, and environmental impact) must pass through certain decision/review gates.
The gated process allows for the evaluation of options based on the intended objectives of
its stakeholders and consequently the selection of optimal option. In this sense, the gated
process for a project allows for the structured way of decision-making. In addition, due to
the comprehensive reviews, the project stakeholders are more informed about the
deficiencies and/or risks in the project at a certain gate.
The figure 2 shows the stage gated project life cycle of a typical O&C project from scope
definition phase to its completion. Harpum, in his article in book titled: The Wiley guide
to managing projects defined the basic rules for a project to pass through these gates
(Harpum, 2004). However, the specific rules and passing requirements differ according to
the individual companys procedures and criticality of a project.
17
Furthermore, in practice, the strictness of these gates also depends upon many other
factors such as capital expenditure, urgency of project, contracting philosophy of owner
to name a few. The following paragraphs describe each phase of an O&C project in brief
manner.
At scope definition level, the requirements of owner are identified and what has to be
done is defined on a broader level. At the conceptual phase, basic functional
characteristics of a project are described as a system in terms of input(s), throughput(s),
outputs and major equipments required to achieve the desired production. In addition, the
major interconnections between subsystems of a project are determined based on the
process philosophy of the project (CII, 2004).
Subsequently, the preliminary design is performed to provide basic design information i.e.
process flow sheets, general design specifications, preliminary equipment specifications
and their arrangements, preliminary plot plans, preliminary estimates and preliminary
project execution strategy. In oil and chemical industry, conceptual and preliminary
engineering phase together are called front-end engineering design (FEED) and a key
deliverable at the end of FEED phase is the basic design package (BDP) (CII, 2004).
In the detailed engineering design phase, the BDP is detailed further as engineering
disciplines initiates detailed engineering in their respective domains. The main deliverables
of this phase are technical, procurement and construction documents. Table 1 shows the
main engineering disciplines typically involved in typical O&C project and their associated
main deliverables.
Table 1 Main engineering deliverables of detailed engineering phase
Engineering disciplines
Process engineering
Mechanical
Piping engineering
Civil,
structural
Architectural
and
In procurement phase, the buying process is initiated based on the design specifications of
equipment, instruments and materials. Later the contracts for civil works and installation of
mechanical, electrical, instruments and piping materials are awarded.
18
During the construction phase, the facility is constructed according to the drawings and
specifications prepared during detailed design phase using material and equipment obtained
via procurement. In the start-up phase equipment are subjected to testing and inspection,
both individually and in combination to validate the proper functioning of the facility. The
phases detailed engineering, procurement and construction phase together are commonly
known as EPC phase of project (CII, 2004).
2.2.2
Controlling of projects
Controlling is the measurement and correction of performance in order to make sure that enterprise
objectives and the plans devised to attain them are accomplished.
- Harold Koontz (1909-1984)
In the control cycle, What to measure varies with the type of project and the perspective
of the organization managing the project. The same is true for how to measure.
19
Corrective actions are management prerogatives that are available to project manager based
on the type of organization and authorities of the project manager. Taking action to
improve the performance refers to the corrective action necessary to bring deviation to a
minimum level. Various examples of corrective action employed in project are fast
tracking, adding additional resources, scope reduction, trade-offs, increasing risks and
disciplinary actions and so on. Moreover, a specific corrective action is depending on the
type of problem causing the deviation.
2.2.3
Having defined the control mechanisms, the project execution control of O&C projects
could be seen in similar manner except the variables to be measured and tools could vary in
accordance with O&C projects.
The section 2.2.2 implies that for controlling, the first requirement is to establish a baseline
against which we could measure the deviation and actual performance of project. To
establish a project baseline for an O&C project, the following project information should be
in available.
I. Overall cost estimates (-10%/+20% variation)
II. Work scope (refers to activities need to be accomplished to achieve the project
objectives)
III. Cost breakdown structure (Cost associated with activities i.e. services, equipments,
overheads, contingency)
IV. Project approved schedule (Milestones dates, activity durations)
V. Comprehensive risk analysis along with accepted risks, planned mitigation
strategies and actions
VI. Commercial baseline: As sold pricing, time bound revenue and margins.
The above documents act as basis for baseline developments. The final baselines for scope,
schedule and cost are established along with control strategies and parameters to identify
the deviations from the baseline.
20
The Figure 4 above shows the applied concept of control cycle, specific to O&C projects.
As soon as the project proceeds into detailed engineering execution, progress and
performance are measured and monitored. In addition, the risks are monitored and dealt
with during the course of execution.
The progress in engineering, procurement and construction is monitored through earned
value1 (EVM) concept with visualization via progress curves (cost progress and schedule
progress). The primary instruments of project control are deviations between planned value
of work (PV), earned value of the work performed (EV), actual cost (AC).
Performance ratios are calculated at project level, phase level and discipline level,
signifying the performance at respective levels. Based on the deviations and performance
ratios, the required resources and cost for the balanced scope of work is forecasted along
with incorporation of any strategy to recover the deviations (Vanhoucke, 2011). Along with
cost and schedule performance ratios, multiple key performance indicators (KPI) such as
safety performance, quality performance are monitored.
Earned value management is a concept, in which progress is measured via integration of scope, cost
and schedule. (For more information on EVM, please refer Christensen, 1998; Lipke et.al, 2009.)
21
2.3.1
During project execution, projects are evaluated periodically using above described
parameters such as earned value, performance ratios, KPIs and variances from baseline.
Such conventional methods are based on the principle of deviation management. At a
certain moment in time, aggregated deviations reflect two aspects of project execution 1.)
How much project is deviating from its baseline 2) given the deviations, how the project is
performing i.e. performance of project?
When aggregated deviations in the project are visible and are regarded as significant, it
implies that there is/are problem(s) that has already manifested and degrading the project
performance: the problem can no longer be avoided. After identifying deviations and nondesirable performance ratios, backward analysis is performed to search for the problems
and strategies to manage the impacts of the problem(s).
It should be noted that the deviations in project are mostly seen on aggregated level and the
impacts of deviations within an area are often seen as limited that area, as their impact on
total project performance is not clear. These localized problems become more critical if
they have significant effect on downstream parts of the project. However, in current
practices, these localized problems are not seen as problems but overlooked by aggregated
performance of project might be still in acceptable limits. Furthermore, when localized
problems develop into project problems, their delayed identification leads to additional
cost.
Figure 5 below explains this current problem more explicitly. The additional cost due to
reactive approach called as cost of reactive approach which could be significant based on
the nature of the problem and the timing of problem detection.
Generally, this cost of reactive approach contributes significantly to cost overrun on
projects. In addition, if the reporting of deviations is delayed due to any reason the cost to
fix those problems will increase significantly, driving the project cost and schedule way off
the baseline. The key to manage a project with predictability and certainty is to manage the
problems before they affect the project outcomes. In other words, acting proactively based
on the symptoms of problems (termed as early warnings) rather than reactively to the
problems.
22
If these localized problems appear in the early phase of a project, given the interdependent
nature of project activities in O&C projects it is almost certain that will have negative
effect of downstream activities. However, the cascading effect of these problems at
aggregated level performance reporting is likely delayed. Therefore, the future forecasts
and project performance based on aggregated current performance is inaccurate.
The above paragraphs clearly indicate that the current controlling and performance
management practices lack the capability to detect the problems early enough and are
always somewhat late. In addition, the forecast based on these traditional methods might
not capture the change in dynamics of project due to localized problems.
Thus, rather than minimizing the cost and schedule overrun in projects they add to it by
providing inaccurate picture of project performance. However, if the localized problems
can be measured as early warnings in projects and proactive management of these early
warnings could minimize their impact and could significantly reduce the cost and schedule
overrun in projects.
In addition, having a more focused proactive approach can predict the future performance
of project with more certainty, But how can E&C companies can achieve that is still to be
discovered. Despite the vast body of literature covering the topic of project control and
project performance, there is still no clear knowledge regarding early detection of problems
and performance prediction based on the early warnings of problems (Vanhoucke, 2011;
(Nikander & Eloranta, 2001). Most of the literature either focuses on quantifying
deviations, diagnosis of deviation cause or corrective action decision making signifying a
clear knowledge gap.
23
2.3.2
Although in the domain of project management, each project is considered unique, certain
types of projects are characterized by large amount of similarities, especially in types of
deliverables, work sequence and procedures.
For example, building of a O&C facility follows the same sequence of engineering design.
Engineering deliverables flow through a logical path across these disciplines thus having
standardized engineering work processes (CII, 2004; Fluor, 2012). Similarly, dependencies
exist between engineering, procurement and construction. The above dependencies are not
completely one directional rather a cycle of reviews and revisions occur before a design is
finalized. As a result, the dependencies multiply the negative effects of a local problem and
significantly influence the project performance.
A detailed analysis of past projects can provide us with valuable quantitative and
qualitative information regarding relationships between early warnings of problems and
their impact on project performance.
In the EPC phase of a project, the engineering cost is significantly lower compared to
procurement and construction costs. However, actions and decisions during engineering
influence the cost of a project much more than during procurement and construction,
therefore detection of potential problems in the engineering phase can avoid large amount
of rework/problems that ultimately cost/schedule overruns.
However, at an O&C project industry, this knowledge is not captured in practice and only
limited information is extracted from past projects. The current practices can be seen as
under-utilization of past project information. Almost none of the available literature focus
on capturing the actual quantitative relationship between early warnings and project
performance. Rather they stress on capturing more qualitative information as lesson learned
24
or limited quantitative data for improving their benchmarking database (Barber, 2004;
Williams, 2004).
This fallacy in past project analysis is another identified knowledge gap, which this
proposed research intends to fill. The fulfillment of above two identified knowledge gaps
can be seen as complement to each other towards the development of performance
prediction model.
How can future problems and performance of a current O&C project be predicted at
early stages using knowledge and experience from past projects in an EPC
environment?
In order to find the answer to this main research question, it is necessary to proceed
systematically through a series of sub questions. The first set of sub-questions will
investigate performance assessment criteria employed in O&C projects and the concept
early warnings of potential future problems.
RQ.1 What constitutes project success and what are performance assessment criteria of
O&C projects?
RQ.2 What do we understand by early warnings of project problems?
The second set of sub questions will focus on identifying the early warnings in project
execution in general followed by identifying early warnings that are specific to O&C
projects. After having a set of early warnings the efforts will be directed to search the early
warnings that can be used in an accurate performance prediction model.
RQ.3 What early warnings can be identified in project execution?
RQ.4 Which early warnings can be operationalized to build a performance prediction
model.
The third set will use the identified early warnings in RQ.4 and investigates their detection,
problem prediction capability and their relation with project performance
RQ.5 What are the dynamics between early warnings and project performance?
25
The final set of sub questions investigates the development of prediction model for
predicting the probable future performance of O&C projects.
RQ.6 What early warnings are indicative of deviation in project performance?
RQ.7 How performance prediction model could predict the future performance of O&C
projects?
2.6 RELEVANCE
The relevance of the research results presented in this thesis is both scientific and social.
2.6.1
Scientific relevance
This thesis will contribute to scientific knowledge on project management, with a specific
focus on project execution of O&C projects, by
I. Exploring and gathering industry specific knowledge regarding early detection of
future project problems
26
II. Providing a methodology to utilize past project information by pointing out the
early warnings and their relations to project performance
III. Exploring usefulness of performance prediction modeling techniques in project
management
The points mentioned above can act as a starting point for future research in project
management, marking a shift from traditional methods of project control to more enhanced
performance prediction. In addition, the content of this thesis will highlight the usefulness
of past project data, beyond their current use as estimation and planning benchmarks.
2.6.2
Social relevance
The insight gained from this research can be used to improve controlling practices in O&C
projects. The systematic process of early problem detection and development of prediction
model will be most important contribution, which can be applied to other industries. The
concept can be extended to other industry such as offshore facility development or civil
infrastructure.
More realistic predictions could lead to more proactive and informed decision making and
ultimately to better project performance. In a world of projects, where the capital
investments are high and efficient capital utilization is a prerequisite for development of
new projects, an improved project performance can provide strategic certainty in capital
planning.
27
3 RESEARCH DESIGN
3.1 SUMMARY
As indicated in chapter 2, the main objective of this research project is to develop a model
to predict the future performance of projects at early stages by capitalizing on the
knowledge from past projects. The main instrument argued in chapter 2 for development of
such capability is the early warnings of problems in projects.
To direct the research efforts towards the achievement of the objective, a clear research
approach has been designed and is presented in this chapter. The section 3.2 illustrates the
scope of the research together with the argumentation for its selection. Subsequently, the
chapter provides a blue print of the researchs fundamental approach (section 3.3).
The section 3.4 aims to provide an overview of the research methods, tools and data
collection methodology. The chapter concludes by discussing the possible limitations of the
adopted research approach.
28
The fundamental approach has been illustrated in figure 8 with a detailed description in
following paragraphs along with the research processes. The research methods are
described in more detail in section 3.4.
In the first phase of the research, the concept of project performance is explored and criteria
for measurement of project performance are defined. The second part of this phase includes
exploration of early warnings concept, its application in project management and its
potential benefits during control of project execution.
In the second phase of the research, relevant literature is explored and experts are
interviewed to find out to what early warnings can be detected during execution of O&C
projects. Semi-structured interviews are held with experts in the O&C industry. The
majority of expert interviews are conducted within Fluor Corporation, along with some
experts from owner companies (to get the perspective of project owners). The second phase
is concluded by consolidating the early warnings from both literature and interviews,
29
followed by the selection of those early warnings that are used in further investigation. The
employed selection criteria are based on main objective of the research project.
Third phase of the research is focused on the in-depth explanatory case studies. The cases
are selected from projects executed by Fluor in the past. Choosing different projects within
one company reduces the variations in execution procedures of projects, as all the projects
were executed with more or less same standard of project execution processes. The selected
projects include both Successful and less than successful performance projects to set the
contrast in which the differences can be visible.
In observatory sense, this phase is used as a reality check of our hypothesis and at the same
explained the relationship between early warnings, project problems and project outcomes
(performed in subsequent sections). As a result, this phase has a more explanatory
character.
The fourth phase of the research is purely quantitative in nature and investigates the
quantitative data from past projects with an objective to establish the predictive relationship
between early warnings and project performance. The quantitative data from eight O&C
projects is collected via available project documentation such as close out reports, project
status reports and detailed monthly progress reports.
The final phase of the research explores the methodology for building the prediction model
and presents the model itself. In this phase, several quantitative prediction methods are
presented and discussed, followed by selection of stepwise multi-regression as adopted
method. The developed model has been evaluated with a new past project (different from
projects those used to develop the model).
The research is concluded at two levels,
I. Presenting a set of recommendations and implementation strategy for Fluor
Corporation to adopt the model in their project control processes
II. Discussing the results of each phase and drawing conclusions from them and
subsequently integrating the parts of research to provide answer the main research
question.
30
3.3.1 Limitations
Having provided the detailed overview of research problem and adopted fundamental research
approach. It is necessary to realize the limitations of research approach and methods.
Quantitative analysis is highly depended on availability of data corresponding to early
warnings. The early warnings, for which the past data is not available, will be excluded from
quantitative analysis. This in turn, will affect the quality of research and subsequently,
development of prediction model.
Another identified and more critical limitation is that the past project data is very limited
therefore could limit the accuracy and reliability of prediction model. Furthermore, the data
is specific to Fluor Corporation. Thus, the data will likely be product of the standard and
practices of Fluor rather than O&C industry as whole.
The proposed research project consists of an evaluation of the existing knowledge on the
concepts of project management and primarily on early detection of problems in projects.
Relevant literature from scientific and professional domains was studied.
The main aim of this part is to understand the tools and procedures applied in management of
O&C projects. Project performance and success are defined based on the academic,
professional literature study and Fluors measurement standards. The concept of early
warnings was defined by the study of available literature by academicians, professional
organizations such as CII, IPA, and PMI along with expert interviews.
3.4.2
Expert interviews
Identifying early warnings relevant to O&C projects is an important task of the proposed
research. For that, the concept of early warning is defined upfront. Experts from O&C
industry were asked to provide potential early warnings based on their experience. The expert
interview is selected as suitable method because there is little or no literature is available
regarding early warnings, especially during the execution of O&C projects. Past project could
be seen as potential source of selecting early warnings. Nevertheless, the time required for
analysis of vast project data does not fit into the available timeframe, yet the past projects
played as role for observatory evidence and provider of past data to build the prediction
model.
Interview base include experienced project directors, project managers and project control
managers within Fluor Corporation and from some external owner companies. Interviewees
were asked to provide potential early warning along with their possible measurement criteria.
Furthermore, the interviewees were asked to provide additional information such as associated
future problems. A measurable quantitative attribute were attached to each identified early
warning and will be termed as EWI.
3.4.3
Data collection
To obtain the understanding of relationship between early warnings and project performances,
data had to be collected and analyzed. Fluor Corporation is the primary source of past project
data. Due to the time constrain, date from past eight projects is used for quantitative analysis,
However the each project will provide 8 data collection point, collected at 0% (baseline),
15%, 30%, 45%, 60%, 75% and 95% of actual engineering duration to normalize the project
with different durations. The primary objective of collecting multiple data within one project
is to understand the dynamic relationship between early warning and project performance and
to develop a dynamic prediction model.
Apart from quantitative analysis, part of past projects are studied as case studies understand
the relationships and to differentiate between coincidences and causality of early warning and
project performance.
3.4.4
Exploratory data analysis was performed before establishing statistical relationship between
identified early warnings and project performances. R Project for Statistical Computing will
be used to perform the statistical analysis due to its capability of customization the statistical
techniques and graphical outputs.
Relationship of EWI with project outcomes was established through collection and analysis of
past project data via stepwise multi regression. The conceptual prediction model was validated
using past project data (which were not included in training) and current projects. The results
of the validation test will be analyzed to form recommendations and conclusions.
33
4 LITERATURE STUDY
4.1 SUMMARY
First step in collecting the available literature on the topic of project performance management
and early warning is database research. Various search phrases were used to find the relevant
literature. Google scholar was used as primary internet search tool. For all the relevant
literature that could be identified, an attempt was made to get access. Further references of
many sources were searched to get the more specific literature regarding O&C industry.
The purpose of this chapter is to investigate how project management literature treats the
detection of early warnings during project execution. The chapter first defines the project
success and project performance, followed by adaption of project performance from current
research perspective. The chapter then proceeds to define the concept of early warnings from
theoretical perspective, followed by reflecting on their benefit in project execution control.
34
The professional consulting companies serve their customers globally thus representing their
findings on projects all over the world, along with publishing region based reports. These
consulting companies published mainly through their official publications.
Academic researchers:
Academic research group involves authors from academic institutions like technical and
business universities, research schools and sponsored academic research by organizations. The
focus of authors is to enhance theoretical and scientific knowledge base regarding overall
project management and specific domains of project management. Their research explores the
science and engineering to delineate unknown causes and potential solutions of practical
problems faced by industry and to find their theoretical solutions. The most applicable
findings are further explored and tested by industries before adopting them as practices. The
present research investigates (but do not limit itself) academic publications relevant to early
detection of problems in projects, problems in execution and their performance management.
Limitations of the review:
Terminology in project management is not uniform for early warning indicators of problems.
Some describes them as leading indicators, symptoms, early warnings, problem causes.
Moreover, many authors see actual problems as potential indicators of future problems.
The diverse approaches and many implicit mentioning of early warning indicators could make
the literature study a time consuming activity. Therefore, it is logical and necessary to adhere
to the discussion of more relevant literature, which explicitly deals with early warnings in
context of project execution. This approach will result in only a part of literature that could
possibly be relevant and might bring the risk of leaving block of literature which might
result in extra concepts.
Having a view on what O&C project are, what are their phases and their performance
management, a natural question arises what are successful projects, in other words How do
we perceive success of a project. Before answering this question, it is necessary to
understand the concept of project success. The Figure 9 shows the best-known and most used
representation of project success i.e. iron triangle with time, cost and scope (or
performance/quality) on its corners (See e.g. Freeman & Beale, 1992, Larsen & Gobeli, 1989,
Might & Fischer, 1985) and Oisen, 1971). From perspective of cost, time and scope, the green
colored triangle is seen as a successful project, whereas the dotted red triangle can be termed
as unsuccessful project, due to overrun in three dimensions of success. Although, this
approach has been seen as too narrow and often criticized. See: (Atkinson, 1999), (Raz &
Dvir, 2002) and (W.Hughes, Tippett, & Thomas, 2004)
To widen the concept of project success, Morris and Hough defined three dimensions of
project success (Moris & Hough, 1987):
35
I. Project functionality: to what extent does the project perform financially and or
technically in the way expected by the project sponsors?
II. Project management: how close is the implementation of the project to budget,
schedule and technical specification?
III. Contractors commercial performance: did the contractors have a commercial benefit
in either short or long term?
The project success dimensions comprehend project success from different perspectives of;
the customer, project execution contractors and sub contractor and other stakeholders.
However, in reality the perspectives differ more than they look. A project could be delayed,
but can be termed as a commercial success from client perspective given changes in his
strategic financial goals. This indicates that whether a project is success depends largely on
the perspective from which the project is viewed (Lientz & Rea, 1995).
(Lim & Mohamed, 1999) addressed the differences in perspective of stakeholders and defined
project success into two criteria: project completion criteria and satisfaction criteria. At macro
perspective the criteria involves both the project completion and satisfaction criteria. On the
other hand, the micro perspective only involves completion criteria. This is shown below in
Figure 10 below.
36
As mentioned in the scope of this research project, (see section 3.2) the present research
adopted the perspective of E&C contractors as they have the key responsibility of EPC phase
of project. In other words, the research is focused upon micro level criteria as defined by Lim
and Mohamed (1999).
However, it would be wrong to assume that the perspective of client and subcontractors are
ignored, because to achieve the sustainable success, an engineering and construction
contractor has to work collaboratively with its customer and suppliers by integrating their
perception of success into its own to the possible extent.
The success of a project is determined by evaluating its performance against success criteria
(Wit, 1988), which implies that performance needs to be measured to determine the
successfulness of a project.
Another noteworthy point regarding project performances is that intermediate project
performance varies with the time during project execution. A bad performing project could be
turned around by making necessary strategic changes or a good performing project could turn
into a poor performing project due to multiple reasons. However, final project performance is
static and determines the success or failures of a project.
4.3.2
Having adopted the micro level success, the next step is to develop performance measurement
criteria to measure the success. Menches and Hanna (2006) developed a performance
measurement index with the following six project outcomes:
I. Percentage budget overrun,
II. Percentage schedule overrun,
III. Actual percentage profit,
IV. Change in work hours
37
Input to construction:
38
The milestone for 95 % engineering complete signifies the completion of all major
engineering activities including final issuance for key deliverables (Issue for construction). In
other words, marks the completion of E phase of EPC project. The rest 5 % of engineering
is designated to miscellaneous construction and start-up support, which could extend until
completion of construction or MC (Fluor Corporation, 2012). Therefore, in industry practice
95% engineering completion is seen as finish of engineering efforts. Figure 11 shows the 95 %
engineering milestone on EPC progress curves.
39
contractor and the owner due to the simple fact that TIC is the important determinant factor in
net present value (NPV) of a plant.
In contractual terms, the dynamics of TIC on project economics of owner and E&C contractor
can be illustrated by following model (Berends, 2007):
Where:
P = Actual E&C profit
Pt = Target E&C profit
= E&C sharing cost related profit; 0 1 (Based on contract type)
Ct = Target/as sold TIC
C = Actual TIC cost
Cc = Owner contract cost
From Equation 3, it is evident on higher level that growth in TIC (C) will shrink the profit
margin for E&C contractor and at the same time will increase the cost for owner.
The cost performance in terms of TIC as project outcome can be assessed as follows:
IV) Engineering man-hours
The amount of engineering man-hours in a project can be seen as an indicator of engineering
efforts required in a project. Although from cost perspective, the cost of engineering efforts is
quite small as compared to the cost of equipments and construction (on average varies
between 10-15 % of TIC). In addition, the maximum engineering cost could be as high as 31%
of TIC and as low as 8 % (Bakker, 2012).
However, from project execution perspective engineering is the most important activity. As
the engineering set the basis for equipment, purchase documents and construction drawings
(see section 2.2.1). Any significant variation or a change in engineering man-hours has direct
effect of procurement and construction activities. Therefore, from project performance
perspective, a project has high chances of being a good performing project and successful
project, if it consume more or less the same hours as estimated.
40
Marcus Aureliuss philosophy from days of Roman Empire suits well in the context of project
execution management. However, a balance between obvious and non-obvious needs to be
identified and monitored. In the present research, early warnings are the mixture of nonobvious and obvious, whos monitoring and subsequent appropriate actions based on them
could prevent manifestation of many problems.
The scientific relation of early warnings of problems goes back to 1970s theory of weak
signals proposed by Professor Igor Ansoff. His primary aim was to improve the strategic
planning methods in business environment. As per his proposition those methods did not work
satisfactorily when faced with sudden changes or unanticipated problems. Moreover, the
traditional strategic planning tools of trend monitoring and basing future planning on these
were not efficient neither successful in time of turbulence. Further, he emphasized that the
potential strategic surprises give advance information of themselves through weak signals
(Ansoff, 1984). He defined weak signals as follows:
An imprecise early indicator about impending impactful eventsall that is known is
that some threat or opportunity will undoubtedly arise, and their shape and nature and
source are unknown (Ansoff, 1984; p. 22).
The Ansoff theory of weak signals was advanced by many researches in a wide range of areas
such as communications research (Aberg, 1989), military science and research on
international security (Betts, 1982; Herman, 1996) and predicting bankruptcies (Morris R. ,
1997). However, the occurrence of weak signals in project environment was first addressed by
Lientz and Rea; they suggested that the potential problems have prior symptoms, which are
visible in the project environment (Lientz & Rea, 1995). They presented weak signals as
symptoms of problems. However, these symptoms are implicit in nature and seem to be
embedded in the project environment. Their identification relies heavily on the skills and proactiveness of project manager and team members.
There are other literature sources, which presented implicit discussions about early warnings
regardless of the term used in original text. Lewis presented a long list (10 pages) called
checklist for managing projects (Lewis, 1993, Ch. 24). The absence of elements mentioned
in this list could act is a cause of potential problems in projects, in other words those elements
could act as potential indicators. Kerzner, Cleland and Honko (Kerzner, 1995; Cleland 1995,
Honko, 1982) mention the similar lists or causes.
The first comprehensive and explicit study of early warning phenomenon in project
management was published by Nikander in the dissertation Early warnings: A phenomenon
in project management. His research contrasted on the deficiencies of traditional project
performance and control mechanisms for correct assessment of project performance status.
He went further on the forecasting techniques and questioned their efficacy by highlighting
their incapacity to accommodate possibly changing project circumstances. He proposed that
the potential problems during project execution could be detected beforehand via their early
warnings. The early detection of problems facilitates more informed decision-making and
41
provides an opportunity for the project manager to act proactively. He defined an early
warning as:
An early warning is an observation, a signal, a message or some other item that is can
predict project health. Reveled in timely manner, these indicators allow for proactive
management to influence project outcomes (CII, 2006; p. 20).
The research had two objectives: identification of leading indicators and development of
tool to assess the health of a project based on those leading indicators (CII, 2006). Unlike
Lientz and Nikander, this study was more focused on industrial practices and relied heavily
on the inputs from the owner and E&C companies. The study emphasized on leading
indicators from industrial perspective and how they anticipate their impacts on project
outcomes. However, the research is purely qualitative in nature and the respective tool
utilizes user subjective information (based on perception) for reflecting on the status of a
project.
IPA also provided the O&C project industry with its services through its project evaluation
system (PES). The underlying premise behind the PES is that a relationship exists between
project drivers and the project's outcomes (IPA, 2012). The service by IPA is quantitative in
nature and utilizes the vast database of past projects.
The IPA performs the project audits primarily at FID and at project completion, which are the
battery limits for project execution phase; the dynamics with in project execution is ignored.
The IPA findings at FID could serve as early warnings for project execution i.e. if the project
is not well defined and basis of design is not of good quality. It could act as first warning for
the project manager and the deficient areas of project will most probably have problems
during execution, if no remedial action is taken. IPA being a service provider, their activities
is commercial in nature and only the outcomes and their illustration is known to the project
owners.
42
In the present research, the larger view focuses primarily on the management of project
performance with in project execution; therefore, audit key parameters from IPA could be
used as early warnings for continuous monitoring.
4.4.1
By definition, early warnings are indicators of future development. Following the reasoning of
Nikander (2002), if early warnings of potential project problems are detected at early phases
of project, many options are still available at the discretion of project manager.
How early warnings can contribute to the cost effectiveness of project can be understood by
following simple mathematical expressions:
Where:
COP = Cost of problems in projects/unanticipated risks
C = Actual cost of project
Ct = Target cost
Cm = Cost due to unavoidable events
The cost of problems can be measured as the cost incurred to mitigate the negative impacts of
manifested problems. In the context of present research, the COP can be seen in two different
perspectives:
COPR = Cost associated with reactive approach (cost of corrective strategy after problem has
manifested)
COPA = Cost associated with active approach (Early warning detected and corrective action
taken)
Applying the reasoning of cost influence curve, explained via Figure 5, it can be concluded
that the cost of reactive approach is more significant as compared to the cost of proactive
approach.
43
With the identification of early warnings in projects, we can minimize the COP to COPA and
subsequently minimize the cost over-rum. In other words, EWI can help project in achieving
following state:
Figure 12 Potential benefits of EWIillustrates the relation between early warning and project
success more explicitly. As the project success is a function of project outcomes, undesired
project outcome can be interpreted as unsuccessful projects. Figure 13(i) show a situation in
which the potential problems was not anticipated well before and when it became visible. As a
reaction to the existence of problem, a corrective strategy was formulated to contain the
negative effects of problem, and subsequently implemented.
The implementation of corrective strategy, required additional cost and schedule requirements,
which were may not be anticipated while baseline finalization (COPR). Moreover, it might
cause non-satisfaction among owners, project team or suppliers. Figure 13 (ii) shows a similar
situation where the early warning indicated the presence of same potential problem in future
course of project execution. The project manager has additional time to make a proactive
decision, inform the stakeholders, and form preventive strategies and to implement the chosen
strategy. The preventive strategy utilizes much less resources, thus keeping the cost and
schedule close to desirable project outcomes (COPA). Furthermore, it vitalizes the sense of
confidence and predictability in project manager and project team.
44
45
The first two PO (MCI and TIC) signifies the final cost and schedule of the project and the
rest two (ESI and MHI) can be seen as intermediate project outcomes, yet from an E&C
contractor perspective they do represent an important aspect of projects.
Review of the project management literature advocates the presence of similar notations to
early warnings in projects and addressed as symptoms of problems leading indicators and
even as checklist for managing projects. However, there is a limitation of explicit pragmatic
studies, which provides the evidence of early warnings engaged in management of projects.
The study by Nikander titled Early warning: phenomenon in project management lacks the
utilization capability in actual project work as the study focus on early warnings which are
either purely qualitative or composed of feelings and behaviors of project team. Another
research was conducted by CII titled Leading indicators to project outcomes identified the
leading indicators which might influence the project performance. However, this research is
also qualitative in nature and respective tool depends on user-subjective information. The
methodology adopted by IPA is of limited use, as it only asses static project health at FID and
project completion and does not provide active control mechanism during project execution.
Section 4.4.1 illustrated the potential benefits of early warnings. The detection of early
warnings and forecasting project performance based on them does provide valuable
information and additional time to project manager to correct the problem before its
manifestation. By the means of information and additional time, the reactive approach could
be converted into active approach, thus reducing the cost and schedule overrun of the projects.
46
5.1 SUMMARY
The primary objective of this chapter is to find a set of early warnings in O&C projects, which
could be operationalized to predict the project performance. To achieve this objective, the
present chapter explores the early warnings from various sources (literature, experts) followed
by assessment of identified early warnings based on selection criteria (formulated from
present research perspective) to select fewer early warnings, which will to be carried forward
for further investigation.
The section 5.2 discuses the importance of measurability of early warnings and provides a
formal definition of early earnings used in present research. Section 5.3 presents the early
warning selection criteria with selection parameters and provides the argumentation for
selecting respective parameters. Section 5.4 discusses the further classification and potential
uses of early warnings after their evaluation on section parameters.
Further section 5.5 and Section 5.6 provides the overview of early warnings found in literature
sources and via expert interviews respectively. The comprehensive lists of early warnings are
attached as appendix A. Section 5.7 presents the final list of early warnings as EWI after
analyzing and rearranging the selected early warnings from respective sources.
47
With reference to the type of information presented, early warnings may be separated into two
categories: quantitative and qualitative. The former provides information in the form of hard
data extracted from system state (i.e. numbers) and latter are based on the subjective data
based on some ones perception (i.e. ordinal numbers or qualitative judgments such as good,
sufficient, bad).
The information, which can be treated as early warning in projects is available in both forms
i.e. qualitative (qualitative assessment) and quantitative (variations, percentage delays).
However, there is another set of early warnings, which is expressed as feeling, inter-personnel
behaviors and communications.
Measurable: The early warnings that are measured quantitatively or at least have the
potential to be measured
Early in project - The definition of early for identification of early warnings is somewhat
subjective. The screening of early warnings gives an indication that some of the early
warnings could exist all along the life cycle of projects which include early phases of projects.
48
There is no reference found in literature that could define the early phase of an O&C project.
Therefore, expert opinion was sought and based on the discussions; the period from start of
detailed engineering, up to 2/32 model review was defined as early. This choice seems quite
reasonable and can be argued based on design definition at 2/3 model review stage.
Data availability To establish the statistical relationship between early warnings and
project performance, it is of utmost importance that we can find the data in past projects
corresponding to selected early warnings.
However, there might be few early warnings, which might not have quantitative data attached
to them, yet the written sources of information in project reports could provide an opportunity
to make an inference regarding their existence. Such qualitative early warnings will be
accepted only for analysis during case studies.
Important here refers to the importance given by the sources i.e. multiple literature or interviewee mention
the same early warnings (Frequency)
49
warning and can take necessary steps to make project environment more positive and
constructive.
5.5.1
Academic literature
The academic literature discusses the early warnings from different viewpoints such as
symptoms of problems, problems themselves, complexity factors or non-fulfillment of project
standard procedures (Lientz, 95; Honko, 82; Cleland, 84; Kerzner-95; Bosch-rekveldt, 2011).
From the analysis of early warnings found in this class, it could be readily observed that all of
them contain a clear indication of their warning like character. Explicit or implicit, all the
relevant quotes or elements are mentioned in appendix A.1 (L-1 to L-30).
5.5.2
The CII study titled Leading indicators to project outcome investigated the leading
indicators (LI) of potential problems in projects. Given the accepted definition in chapter 5,
the mentioned LI probably can be seen as early warnings. The study concluded with 43 LI,
which were presented after three levels of evaluations.
All 43 LI were analyzed and after the preliminary screening, 15 top indicators were selected
based on the importance (in terms of score) given to them by E&C contractors. The early
warnings from CII are mentioned in appendix A.1 (L-31 to L-45)
5.5.3
As mentioned in section 4.4 IPA audit findings at completion of front-end design could serve
as early warnings for project execution. IPA project audits are purely quantitative and could
serve as excellent early warning indicators as per our adopted definition. IPA has defined its
own metrics and project database to validate those metrics. Apart from rating the project based
on its characteristics and metrics, they provide a benchmarked comparison of project with
other more or less similar projects. In addition, IPA findings on post project evaluation could
also provide key early variables, which makes difference between good performance and bad
performance of projects and can be seen as early warnings.
In the present research, the underlying principle of IPA PES is of interest, attempts could be
made to find the relevant data from past project and conclusions could be drawn based on the
findings. The early warnings from IPA PES are mentioned in appendix A.1 (L-45 to L-49)
50
5.5.4
Based on the characterization and typology of early warnings found in literatures, it was
observed that the majority of the early warnings are in qualitative in nature (21 Nos, 45 % of
total). In addition, there were 17 nos of quantitative early warnings including 4 nos of
traditional performance indicators which could be seen as early warnings. The classification of
early warnings by literature source could be illustrated by following graph.
Early warnings from literature
30
Feeling, behavior
Quantitative
25
Qualitative
20
7
15
10
13
5
8
6
0
Acedemic Literature
CII
IPA
The early warnings can be further classified into sub categories. Most of the sub categories
consist of various early warnings primarily related to that sub categories. Figure 16 shows the
classification of early warnings into sub categories.
51
25
Qualitative
9
20
3
15
10
15
4
1
2
4
Project Team
development &
integration
Change mgmt
Early Engineering
Procurement
3
Concurrency in
projects
Note that the quantitative nature of early warning reflects the potential to measure and does
not necessary mean that it is measured. For example, early warnings number L-35, which
could be measured by monitoring the experience (Based on the grade), personnel need date of
actually deployed personnel against the planned requirements. However, not every project or
E&C Company measures the experience and skills of the project team quantitatively.
There are 27 early warnings that belong to quality of project team and its integration, which in
a sense reflects the importance of team in a project. The noteworthy point from above analysis
is that majority of early warnings in category team development and integration (24 out of
total 27) are either qualitative in nature or are feeling/behavior of team members.
Based on the selection criteria, following early warnings are selected for further analysis
Table 2: Early warnings from literature sources
Criteria
ID
Early warning
L-1
Monitoring of actual
allocation of resources
against the plan can
provide early warning of
lack of work
Measura
bility
Early in
project
Data
Availability
Remarks
Here it should be noted that the quantitative nature of early warning reflects the potential to measure.
52
L-2
Noticeable change in
performance level for a
team or individual
members reflects
problems.
L-18
L-30
L-32
L-34
L-46
L-48
L-49
53
5.6.1
54
After a brief personal introduction, the interviewee was explained the conceptual design of
research, the concept of early warning indicators, EWIs intended use in present research and
the expected outcomes of the research project. The interviewee was asked to describe the key
early warning indicators in project execution, which they believe leads to the project problems
in project at later stage. Then further questions were asked to better understand of how the
particular EWI influences the project outcomes or key deliverables of the project and how it
enables the project manager to predict future problems.
After the discussing the particular early warning, the interviewee was asked to delineate the
time or instance on the project timeline at which this early warning can be identified along
with the possibility to measure this early warning quantitatively or qualitatively. In addition,
any example from interviewees practical experience was sought. The interview was
concluded with a discussion of a few practical matters on project execution and key
parameters to be kept in notice from a project manager point of view.
All interviews were taped after consent and notes of important points were taken during the
interviews. After the interviews, preliminary analysis was done. The final analysis was
conducted after all the interviews had been completed.
5.6.2
The detailed transcripts provide the overview regarding each early warnings description,
which project outcome it affected, and how it could be measured. Based on the interviews the
list of early warnings from practical perspective is attached as appendix A.2:
5.6.3
Based on the typology of early warnings found from expert interviews, it should be noted that
there is no early warning mentioned as feeling or behavior, which might be explained by the
fact that prior to interviews, every interviewee was made aware of the final objective of the
research i.e. building of quantitative model. Another explanation could be that the interviewee
base included people who had experience as PM or experience in project leadership, (PM,
PCM, Sr. mgmt) and being leaders, they might perceive feeling/behavior aspects as qualitative
to have meaningful interpretations from them rather than just behavior or feelings (Prati et al,
93; Pescolido, 02).
55
Quantitative
16
14
Qualitative
1
12
10
16
8
14
6
11
7
4
2
2
Early Engineering
Procurement
Concurrency in
projects
0
Project Team
development &
integration
Change mgmt
On the contrary, to the early warnings from literature, experts see early engineering as most
prominent category of early warning (31%). In addition, the majority of early warnings in sub
category project team development and integration are qualitative in nature (14 out of 15),
similar to early warnings from literature.
Apart from the classification of early warnings, it should be noted that the early warnings
related to sub category change management and early engineering were mentioned by all
interviewees irrespective of the nature of early warning, implies towards the possible
importance of these two categories to determine project success. The following figure presents
the graphical representation of early warnings (category wise) as mentioned by interviewees.
56
12
13
13
11
10
9
8
0
Project Team
development &
integration
Change mgmt
Early Engineering
Procurement
Concurrency in
projects
Based on the selection criteria, following early warnings are selected for further analysis:
Table 3: Early warnings from Experts
ID
Early warning
I-3
Large
amount/Numbers of
scope changes
(additional items or
details) by clients
I-13
I-15
I-16
I-18
I-20
High Concurrency
level in projects
High Level of client
involvement
Late scope change
Overrun in process
engineering manhours
Delay in process
engineering
Criteria
Measurab
ility
Early in
project
Data
Availability
Remarks
Segregation of change
orders from past projects
at required level of detail is
difficult, therefore the early
warning is mentioned as
nos of changes
Data could be found as
write-up in internal reports
I-23
High Nos of
arguments between
E&C contractor and
owner
I-25
57
I-27
I-31
I-35
I-40
I-43
I-52
Lack of
understanding
regarding project
objectives among
project team
Delay in purchase
order placing
Nos of changes
After P&ID's are
issued for design
due to rework
Changes in major
project objectives
Early engineering
discipline crossing
late curve
Higher numbers of
changes in project
at early stages
Incomplete Front
end design
Same as I-25
Same as I-3
Data could be found as
write-up in internal reports
58
New
ID
LES
Early warning
indicator
Lack of
understanding of
project execution
strategy among
project team
PTE
Project team
lacks experience
required for the
project
Conflicts between
owner and E&C
contractor
COC
Measurability
Possible sources
Qualitative
Project review
meeting, Project
surveys
Qualitative
Deficiency in
required and
actual deployed
resources
Client survey,
review meetings,
contractual
disagreements
Qualitative
Integrated early
warnings
I-26, L-48
L-34
I-15, I-25
Note: In the perspective of present research, the term conflict has been used in negative sense
i.e. the conflicts which manifest the project problems. However, it should be noted that in
conflict is not always dysfunctional or destructive, though it also play constructive role by
generating new norms, and stimulating the engaged parties towards economic gains and
technological advancement (Coser, 1957)
EWI related to change management: The changes during project execution do affect the
project outcomes (often negatively) and this is confirmed by both literature and experts. One
of the basic principles of efficient work processes is that work is done once and done
correctly. However, there might be situations in the project where changes are not avoidable
due to safety, functionality or standards.
Ideally, in an EPC phase of an O&C project any change after completion of FEED stage
should be avoided. Any proposed change should pass the following criteria before
implementation (Bakker, 2012):
I. If the current design is not safe
II. If the current design does not work
III. If the current design is not as per standard
Changes are generally detrimental to the health of a project, as they often have major impact
on cost and schedule. It is also well documented that the cumulative impact of numbers of
small changes is much greater than sum of individual impacts. This is mainly due the fact that
the changes not only brings new addition of work, but also produces out of sequence activities
and disruption effect on ongoing activities.
59
Another noteworthy point is that impact of disruption and out of sequence activities is often
difficult to assess before implementing the change and often become visible at the later stage
of project.
The following EWIs were selected based on the selection criteria.
New
ID
NCO
CCO
Early warning
indicator
Numbers of change
orders
Cost impact of
changes
Measurability
Possible sources
Nos of changes in
project
Cost of
changes/estimated
cost of project
Integrated early
warnings
L-32, L-34, L-49, I-3, I43
L-30, L-32, L-34, L-49,
I-3, I-43
EWI related to early engineering: The early engineering in the EPC phase of an O&C
project can be seen in two perspectives:
1) Completion of FEED
2) Process engineering of EPC phase
The FEED phase of a project produces the basis of design (BOD) for detailed engineering.
Efforts at the FEED phase defines the quality of design basis (BOD), The BOD forms the
starting point for detailed engineering and the project baseline estimate. Therefore, an
incomplete FEED could not only disturb well-defined start of the detailed engineering design
but also affect the accuracy of estimate and resources for the project.
However, if the incompleteness of front-end design has been addressed by project team, it is
of utmost importance that incompleteness has been communicated well with in project team as
well as has been reflected in estimates, management reserves and contingency plans.
Process engineering works on BOD and set the design criteria for further discipline
deliverables. Process engineering delivers the critical documents, which act as the base
documents for subsequent engineering. In a case where process engineering is forecasting
significantly higher numbers of hours due to any reason (changes, missing items, inefficiency
etc,) it implies that the critical documents require additional efforts. The growth in process
hours is an indicator that the subsequent engineering discipline also might need additional
hours (to complete the added or modified work) and additional cost is required for material
and services.
The delay in process engineering implies that the critical documents will be issued with delay,
thus affecting the downstream disciplines in terms of work front and delay in their respective
deliverables. In addition, the delay in process engineering impacts the procurement cycle for
the project and this could result into late delivery of material and ultimately construction
delay.
Any early problems in these two areas should be seen as an early warning. Following EWI
were selected in this category
60
New
ID
FED
Percentage of missing
information in FEED
package
PH
PS
Delay in process
engineering
Measurability
IPA FEL index
Non-conformities
and observations in
Gate-C Review
Forecasted or
actual hours /
Baseline hours
Actual %
completion
Planned %
completion
Possible
sources
Integrated early
warnings
IPA audit
report
Gate C review
I-52, L-46
Progress
reports
I-18, L-2
Progress
reports
EWI related to concurrent engineering: At the start of EPC phase, engineering planning
is carried out based on certain level of concurrency (baseline concurrency levels) via
information available from BOD, schedule and cost targets and PEP.
From the analysis of early warnings by literature and experts, it could be concluded that the
concurrent engineering execution is directly proportional to the amount of assumptions being
made in project. A high concurrency level means that an engineering discipline is either
working on assumptions or sufficient reliable information is available beforehand. In either
case, there exists a risk of assumed information proved wrong at later stage, which will
produce rework not only with in a specific discipline but also with the associated discipline
activities, bulk quantities and changes in vendors scope.
From an early warning perspective, it seems logical to identify the change in concurrency
levels early in the detailed engineering to have a check, if design is being carried on high level
of assumptions.
The engineering discipline process and piping has been identified as indicator of concurrency
in project. The choice has been made because of the following reasons
I. The engineering activities of the piping discipline constitute a significant part of early
engineering efforts are highly dependent on the inputs from process engineering.
II. Piping engineering constitutes of a substantial part of engineering and material cost.
III. The rework in piping could lead to the high impact of total project cost and schedule.
Following EWI were selected in this category:
New
ID
CE
Measurability
Change in concurrency
level between process
and piping engineering
Change in area
between process and
piping progress curves
Possible
sources
Discipline
progress
curves
Integrated early
warnings
I-13, L-32
61
Early warning
Measurability
DPO
Delay in
issuance of
purchase
orders
Possible
sources
Procurement
progress curves
Integrated early
warnings
I-27
62
6 CASE STUDIES
6.1 SUMMARY
The primary aim of this chapter is to assess the presence and predictive capability of selected
early warnings (in section 5.7) in real life projects. To achieve this goal, a case study approach
is adopted. A multiple cases embedded design is used, in which each case represents a
completed project (Yin, 2002). The case study approach was chosen, mainly because the
present research is focused on contemporary real life situation. Therefore, it seemed necessary
to understand, how the suggested early warnings can be identified in projects, what problem
they can predict and how they influence the project outcomes.
In case studies, an effort has been made to recognize the link between early warnings, their
associated project problems and final project outcomes. In other words, the intention here is to
check and understand if an early warning provides a correct picture and illustration of their
intended purpose.
Section 6.2, discusses the basic design of case studies, including the possible data sources.
Section 6.3 describes the factors, which were considered while selecting the projects for case
studies followed by a brief overview of the selected cases. Section 6.4 to 6.7 discusses the
individual case studies by discussing the presence of early warnings, their associated project
problems and final project outcomes. Each case study is concluded by proving an overall case
summary.
The final section 6.8 provides a cross case analysis to argument the prediction capabilities of
early warnings in the contrast set by Successful and less than successful performing case
projects.
63
available) 6) IPA audit reports (if available) 7) Discipline progress curves 8) Procurement
progress curves.
Wherever possible, an effort has been made to investigate more than one source for particular
evidence. On the top of above mentioned sources, informal talks with project team members
were conducted with an intention to understand the itinerary of project execution. The
information from project members also helped in identifying any special or abnormal events
that happened during the project execution, which might have influenced the project
outcomes.
The relationship between detected early warnings, manifested project problems and their
impact on project outcomes has been mapped by following framework.
Figure 18 Framework for mapping the relationship between early warnings, project problems, and project outcomes
In the above framework, the early warnings indicate the possible manifestation of specific
project problem, which cause an undesired growth in project outcome i.e. affects the project
performance. The relationship for each warning in all four case studies is mapped and has
been attached as appendix D.
64
Having said that, three of the four selected case study projects were executed between 2008
and 2012. One case project was executed in year 2004 and 2005, which was selected due to its
extremely successful execution, and therefore could provide useful information while
comparing the cases.
In terms of project characteristics, the case projects are summarized in following table:
Case project
Case project 1
Case project 2
Confidential
Case project 3
Confidential
Case project 4
Confidential
Project type
Contract Type
Chemical-green
field
Refinery upgradebrown field
Chemical-Green
field
Chemical-green
field
Cost-reimbursable
fixed Fee
Cost-plus fixed
Fee
Lump sum
Cost-reimbursable
fixed Fee
Year of EPC
execution
2008-2011
2004-2005
2011-2013
2010-2012
The case number one and four did experience growth all project outcomes. However, the
growth in final project outcomes i.e. TIC and MCI has been relatively low as compared to the
MHI and ESI. The good performance project did not show any significant growth in final
project outcomes i.e. TIC and MCI. The case project number two finished on schedule with
cost under. The case project three finished on schedule with less than 1% growth in TIC. The
case project three did experience growth in MHI, at the same time finished the detailed
engineering ahead of the schedule.
The objective of discussed case project was to construct a new fully continuous integrated
chemical. The E&C contractor worked under fixed fee reimbursable contract.. The period for
EPCm execution of project was from 2008 to 2011. From project documentation, it was
concluded that during project execution, the project drivers were often changed. This is
evident by the multiple cost review and scope reduction exercises. The final project
experienced both cost and schedule overrun. Apart from cost and schedule, the performance
of the project was very good in terms of functionality and safety.
6.4.2
Project evaluation
65
Early warning
Presence
Lack of understanding
regarding project execution
strategy among project team
Yes
Yes
Yes
Moment of
detection
4 months
after EPC
start
2 months
after EPC
start
2 months
after EPC
start
EWI mentioned as
Source
Project
status
report
Project
status
report
Project
status
report
Analysis of the project status reports indicated that the project team integration was a
challenge during project execution. The project team (E&C, Client) had some inexperience
issues. Extreme involvement by few stakeholders amplified the decision-making problems. A
sub-contractor did not seem to be aligned with the project execution strategy, which prompted
the E&C contractor to take corrective actions that resulted into some rework.
The analysis of project reports suggests that the early warning in team development and
integration domain has substantial potential to predict the problems at late engineering and
construction stage. The link between early warnings, their associated future project problems,
and their impact on final project outcomes is attached in appendix D.1.1:
2.) Change management: A strict change management process was placed since the start
of EPC phase, although project execution went through changes both in scope
addition/reduction and in detailing of design. The numbers of change orders were seen as
extremely high by the E&C contractor team and issues were raised.
In addition, the frequency of change orders was quite high, there were 70 change orders
between 15 to 30 % engineering. The significant numbers of change orders not only affected
the engineering efforts, but also affected proper assessment of their impacts.
The status of early warnings is shown in following table.
Early warning
Presence
Detection time in
project
Early warning
mentioned as
Source
Number of changes
Yes
7 Months after
EPC start
Nos of agreed
change Orders
132
Change order
Log
Yes
7 Months after
EPC start
Cost of
Changes/TIC of
plant = 6 %
Change order
Log
The relationship between early warnings related to project changes, project problems and their
impact on final project outcomes is illustrated with figure in attachment D.1.2.
66
It was observed from the study of project archives that such high magnitude of changes
introduced many problems during project execution. The manifested problems include
engineering, cost and schedule related problems.
3.) Early engineering: The assessment of FEED package and process engineering is
explained further below.
I) Completion of FEED As explained in Figure 2 when a project moves from the FEED
phase to EPC, generally a review should be conducted (known as gate review).
Furthermore, the quality of a gate review should be thorough, rather than a tick the box
approach. In the present case project, the gate between FEED and EPC is known as Gate C.
The Gate C review of the FEED package determines the quality of BOD and ideally, all issues
at this stage should be resolved before moving into full scale EPC.
Gate C review of the project indicates that the E&C team identified non-conformities and
observation from organizations standards including few related to design required for
downstream disciplines. Furthermore, due to schedule pressure, the project team made a plan
to deal with non-conformities; however, it was mentioned in project documentation that
project moved into EPC too early. In summary, the real implications of missing information at
FEED stage only became visible at EPC phase and resulted late engineering problems and
enforced E&C contractor to work on concurrent execution.
Early warning
Presence
Detection time in
project
EWI mentioned
as
Source
Percentage of
missing
information in
FEED package
Yes
Start of EPC
Non-conformity
Observations
Gate C review
II) Process Engineering: As per interdisciplinary logics of engineering design, any design
problem in process engineering produces a negative effect on downstream engineering
disciplines.
The case project has few issues with deliverables in process engineering. The extra efforts
made, which had to be made by process engineering to meet with deficiencies from previous
phase enforced few scheduling problems within the discipline and for downstream disciplines
and affected some of the deliverables of the project. The total project team worked cohesively
to deal with problems, thus caused some overrun in manhours.
The status of early warnings at 2/3 model review stage is shown in following table:
Early warning
Presence
Yes
Delay in process
engineering
Yes
Detection time in
project
7 Months after EPC
start
7 Months after EPC
start
Early warning
mentioned as
Growth in process
hours
Process
engineering behind
schedule
Source
Project status
report
Project status
report
67
The relationships between early warnings, project problems (discussed in above paragraphs)
and project outcomes are shown D.1.3 below.
4) Concurrency:
In the case of project 1, the concurrency level in process and piping discipline varied
throughout the engineering execution. It can be concluded that 15 % to 30 % engineering
duration, process forecasted growth in hours, and was behind schedule. There was
downstream effect in piping and to meet the schedule constrains piping engineering made
assumptions. Later, when the piping engineering was not able to sustain the assumption and
could not ignore the risk of high amount of rework, it re-optimized its schedule and forecasted
some delays thus concurrency level decreased, going beyond the planned concurrency.
However, the concurrent execution of piping engineering in the early stage of the project (up
to 45 % engineering completion) resulted into few problems related to cost and schedule. The
re-optimization and some rework affected bulk quantities and their subsequent purchase; in
addition to some delay in project milestones.
5. Procurement:
The case project did not experience much delay (4 %) in issuance of purchase orders for
equipment as due to favorable economic conditions (Buyer market) at early 2008, and major
equipment were already purchased (50 % PO issued at start of EPC). However, from the
project execution perspective the strategy met with some challenges, mainly due to the
changes in the project. The E&C contactor made corrective strategies to deal with the problem
and had some cost and schedule impact on project.
Early warning
Presence
Delay in PO
Yes
Detection time in
project 2/3 Model
review
7 Months after EPC
start
EWI mentioned as
Source
Delay in PO
From the perspective of EWI, the delay in PO did not manifest any major problem during the
project. However when coupled with late changes in the project it did contribute to the cost
overrun.
The case project 2 stared in January 2011 and the planned to be complete in august 2013. The
objective of case 2 project is to build a chemical plant. The project is a part of larger project,
aimed to construct a chemical complex owned by consortium petrochemical companies. The
EPC contract between E&C contractor and owner is on lump-sum basis. The EPC project
duration is 29 months with 14 months for detailed engineering. The peak engineering staffing
68
for project was 160 full time equivalents (FTE). Although the project is still in end of
construction phase, the project performance so far has been very good in terms of cost,
schedule, quality and HSE.
6.5.2
Project evaluation
As explained in section 6.1 (case study design) project evaluations are performed with an
intention to identify the presence and prediction potential of EWI. The present case project is
of lump-sum nature and still in the last phase of execution; therefore, project reporting was
selective and regarded as highly confidential.
1) Team development and integration: The project execution team consisted of owner
and E&C contractor and work-share office within E&C contractor. The project management
team consisted of an integrated management team (IMT) with owner and E&C contractor with
an intention to gather the experience of operations from owner. The IMT approach was
adopted since the FEED phase of the project. The IMT approach did work well and no
significant issues were identified in the team development and integration.
2) Change management: The changes implemented during the detailed engineering phase
are considered significantly low for project of such magnitude. The cost impact of
implemented changes was only 0.6% of estimated TIC, with the largest change being 0.24 %
of TIC in magnitude.
The early changes did produce some rework during engineering, but the project was capable
of absorbing the impact. However, the schedule impact of changes was contained within
schedule and the 95% engineering was achieved ahead of schedule. In addition, no period in
the project experienced relatively high numbers of changes.
The well executed no change policy helped the project in initial stages by not only
minimizing the impact on engineering efforts, but also to assess the impact of changes more
precisely. The figure 25 below underlines the above observations.
The status of EWI related to changes is as follows:
Early warning
Presence
Number of changes
Yes
Timing of changes
Yes
No
No
Detection time in
project 2/3 Model
review
7 Months after EPC
start
7 Months after EPC
start
7 Months after EPC
start
7 Months after EPC
start
Early warning
Source
Change
order Log
Change
order Log
Change
order Log
CO man-hours/original
budget = 0.3 %
Change
order Log
The mapping of EWI, associated project problems (discussed in above paragraphs) and their
impact on project outcomes has been illustrated in appendix D.2.1.
69
I) Completion of FEED: The Gate C review of the project was conducted 3 months prior to
the start of EPC phase with an objective to assess if the BOD and the associated work were
ready to proceed into EPC phase. The overall assessment suggested that the BOD was
sufficiently defined to move into EPC phase.
The results of Gate C review indicated few non-conformity and observations, however the
time available between the FEED and EPC phase allowed for the resolution of all the
observations. All non-conformities were rectified by E&C project team.
Early warning
Presence
Detection time in
project Gate C
Review
EWI mentioned as
Source
Percentage of
missing information
in FEED package
No
Start of EPC
Non-conformity 0
Observations 4
Gate C review
II) Process engineering: The process engineering execution for case project went
extremely well with ahead of schedule and with small person-hour growth. The process
deliverables were well on schedule with no or minimum scope changes at later stage, that
could be validated with only 0.82 % man-hour growth in process engineering caused by
changes. In addition, there were no significant project problems identified.
The status of EWI related to process engineering is shown below.
Early warning
Presence
Detection time in
project 2/3
Model review
No
Delay in process
engineering
No
EWI mentioned as
Growth in process
hours 3%
(negative)
Process
engineering ahead
of schedule (3 %)
Source
Project progress
data
Project progress
data
4) Concurrency: The case project did experience increase in the concurrency between
process and piping engineering. However, the increase in concurrency did not manifest into
any major project problem. The status of CE has been shown in following table.
Early warning
Presence
Detection time in
project 2/3 Model
review
EWI mentioned as
Source
Change in concurrency
level between process and
piping engineering
Yes
Increase in
concurrency by
20%
Discipline progress
curves.
5. Procurement: The case project did experience a delay in issuance of purchase orders for
equipment during early stages. However, the delay was recovered soon enough and purchase
orders were issued well in time to receive the required vendor information. From the
70
perspective of early warning, the delay in PO did not manifest any major problem during the
project.
Early warning
Presence
Detection time in
project 2/3 Model
review
EWI mentioned as
Source
Delay in PO
Yes
7 Months after
EPC start
Delay in PO 0.1
%
The case three project was aimed at up-gradation of a fuel refinery to produce cleaner fuels.
The project was brown field in nature and involved tie-in with existing operation plant. The
contract between E&C contractor and project owner was EPCm under cost reimbursable with
incentive arrangement. The period for project EPC execution was 2004 through 2005 with 13
months for detailed engineering and 19 months to MC. The project performed quite
extraordinary with under run in TIC and on time completion of plant. Furthermore, the project
performance in terms of safety and quality was in line with clients expectations.
6.6.2
Project evaluation
1) Team development and integration: The case project involved a Fluor project team,
client team, and Fluor work share offices. The study of internal status review documents
suggested that were no issues at all with client and internal interfaces. On few instances, the
project literature did mention conflicts with the client during initial stages, but they were
limited to negotiations. Later the contract negotiations concluded with satisfaction of all
parties concerned. The E&C team was open to client audits and there were no major
disagreement issues identified.
Early warning
Presence
Detection time
in project
EWI mentioned as
Source
No
No issues identified
Project status
report, IPA audit
No
Project status
report, IPA audit
No
No issues identified
Project status
report
2) Change management: The project change log reveals that there were no significant
changes in the project due to the no change culture from client and E&C contractor. The
71
successful implementation of a No change philosophy was possible due to the strong and
clear implementation of FEED package, management support and regular feedback to the
team regarding success of No change philosophy. The changes implemented in EPC phase
resulted in net cost saving. The engineering hours in change orders show 4 % growth with
respect to original budget, and the total TIC impact shows 1 % saving. The following
observation is shown by graph.
The EWI identification:
Detection time in
project 2/3
Model review
7 Months after
EPC start
Early warning
Presence
Number of changes
Yes
Timing of changes
Yes
7 Months after
EPC start
No
7 Months after
EPC start
Yes
7 Months after
EPC start
EWI mentioned as
Nos of change
Orders 14
Nos of changes
after 1/3 model
review 14
Cost of
Changes/TIC of
plant = -0.87%
CO manhours/original
budget = 4 %
Source
Change order Log
Change order Log
Change order Log
Change order Log
3) Early Engineering: The early engineering in EPC phase O&C project can be seen in two
perspectives:
I) Completion of FEED: The quality of FEED for case project was significantly high. The
audit by IPA suggested that the FEL index for FEED was 4.5, which is considered as best in
class. The project moved into EPC phase with just three non-conformities (As per Gate C
review). The gate-C audit report of the project concluded that the design is ready to move into
EPC phase. However, the report did recommend that project should wait for 2 weeks for the
start of few activities to include the information in BOD rather than working on assumptions.
No information could be found regarding project action on the recommendations of the audit
report.
Early warning
Percentage of
missing information
in FEED package
Presence
Yes
Detection time in
project Gate C
Review
EWI mentioned as
Source
Start of EPC
Numbers of Nonconformities
FEL Index 4.5
Best in class
Gate C audit
report
IPA Audit report
II) Process engineering: The case project had an exceptional well-executed process
design. As mentioned earlier, this is due to well defined FEED package and No change
philosophy employed in the project. The process engineering deliverables experience no major
changes and timely process deliverables enabled downstream disciplines to execute work as
planned. Furthermore, the process man-hours showed under run of 9 %. This statement could
72
be verified by the zero growth in process man-hours and on schedule execution of process
engineering.
Early warning
Presence
Detection time in
project 2/3 Model
review
EWI mentioned as
Source
Growth in
process manhours
No
Delay in
process
engineering
No
Process engineering
behind schedule 0 %
Presence
Delay in PO
Yes
Detection time in
project 2/3
Model review
7 Months after
EPC start
EWI mentioned
as
Source
Delay in PO 2
%
The case four project was aimed at construction of a chemical processing facility. The project
was green field in nature and was extension of an existing chemical complex. The contract
between E&C contractor and project owner was for EPCm under cost reimbursable with
liabilities. The schedule was the main project driver followed by cost and quality. The period
for project EPC execution was 2010 through 2012 with 11 months for detailed engineering
and 18 months to MC. The project did not perform as per expectations with over run in TIC
73
and schedule overrun. Furthermore, the project performance in terms of safety and quality was
in line with clients expectations.
6.7.2
Project evaluation
1) Team development and integration: The project execution team involved E&C
contractor, client team, and work share offices with in Fluor. The project adopted an
integrated project execution team led by clients project manager. The summary of the
findings on project team development and integration is given in following table:
Early warning
Presence
Lack of understanding
regarding project execution
strategy among project team5
Yes
Yes
Yes
Detection
time in
project
2 months
after EPC
start
2 months
after EPC
start
2 months
after EPC
start
EWI mentioned as
Different perception about project
execution strategy
The project required new software
system for piping design, on which
project team had limited experience.
Different underlying interests of
stakeholders.
Less synergy on impacts of proposed
changes on project
Source
Project
status
report
Project
status
report
Project
status
report
Analysis of the project status reports indicated that the project team integration had a good
start with establishment of integrated project management team, but soon challenges emerged
due to different underlying interest of stakeholders. At few instances, intervention level by
stakeholders was seen as significantly high and was a point discussion among project team.
Moreover, the schedule and cost impact of almost all changes were challenged by project team
with an intention to only accept necessary changes, which ultimately affected relations and
timing of change implementation.
The link between EWI, above discussed project problems and their impact on final project
outcomes is illustrated in attachment D.4.1:
2.) Change management: The basis for development of aggressive project schedule was
no change during engineering, however the project experienced introduction of heavy
changes since the start of EPC phase. Within 5 months of EPC, 51 change orders were issued.
These changes were in direct violation of assumptions made in schedule development. The
possible impacts of such high changes were highlighted by project E&C project management.
At intermediate model review, there were 125 approved changes in the project with a cost
impact of 4 %. The summary of the relevant EWI is presented in following table:
However, there is no specific reference to the difference in understanding of project execution strategy
among project team; the study of project document does suggest that there were differences in confidence
level of team members regarding highly concurrent execution model.
74
Detection time in
project 2/3 Model
review
7 Months after EPC
start
Early warning
Presence
Yes
Timing of changes
Yes
Yes
Yes
EWI mentioned as
Source
Nos of change
Orders 125
Nos of changes
after 1/3 model
review 125
Cost of
Changes/TIC of
plant = 4 %
CO manhours/original
budget = 23 %
Change order
Log
Change order
Log
Change order
Log
Change order
Log
The analysis revealed that the project did acknowledge the impact of changes and their effect
of final engineering hours yet was a point of conflict.
In addition, the frequency of change orders was quite high as there were on average 26 change
orders between 0 to 60 % engineering. The significant numbers of change orders not only
affected the engineering efforts, but also brought inefficiency in assessing the impact of
changes. Moreover, relationship between EWI related to project changes, the project
problems, and their impact on final project outcomes is illustrated by attachment D.4.2
3.) Early engineering:
I) Completion of FEED The Gate C review conducted by project team indicates that the
FEED package had non-conformities and observations of deviations from the standards of
E&C organization. The literature shows there were deficiencies with the received BOD and
was not of good quality to proceed into EPC phase. However, due to schedule requirements
the project continued for a certain time with project team simultaneously improving BOD. In
summary, the real implications of missing information at FEED stage only became visible at
EPC phase and resulted late design changes and concurrency in design disciplines.
Early warning
Presence
Incomplete front
end engineering
design
Yes
Detection time in
project Gate C
Review
Start of EPC
EWI mentioned as
Source
Non-conformity
Observations
Gate C review
II) Process Engineering: The P&IDs during early design went through major changes as
soon as project moved into EPC phase. Because of the incompleteness in the BOD, the
activities had to be reworked. Moreover, the discovery of new work during process
engineering schedule and cost related problems in project. It was observed that the E&C
process engineering experienced & acknowledged the problems at very early stages, and
forecasted growth, which produced cascading effect into downstream discipline and ultimately
total engineering.
75
Early warning
Presence
Yes
Delay in process
engineering
Yes
Detection time in
project 2/3 Model
review
7 Months after EPC
start
EWI mentioned as
Growth in process
hours
Project
report
status
Process
engineering behind
schedule
Project
report
status
Source
The relationships between early warnings, project problems, and their impact on project
outcomes are shown in attachment D.4.3:
4) Concurrency:
In case of project 4, a concurrent project execution model with high concurrency level in
process and piping discipline was planned. An interesting observation was made that the
concurrency level did not change significantly throughout the project engineering duration. It
implies that the during the project execution the E&C team was proactive and acknowledge
the risk of concurrent execution and piping engineering forecasted the realistic progress plans
based on behavior of process engineering. However, at the later stage, the concurrency level
reduced further because frequent changes forced piping engineering re-optimize its planning.
From EWI perspective, the change in concurrency did not reveal any significant problem
mainly because the pre-set level of concurrency very initially high and as the detailed
engineering progressed in the project, the process and piping discipline addressed the early
problems and reforecast their realistic growth.
5. Procurement: The case project experienced delays in issuance of purchase order for
equipment. The delay in PO did affect the downstream disciplines regarding required vendor
information for their design.
Early warning
Presence
Delay in PO
Yes
Detection time in
project 2/3 Model
review
7 Months after EPC
start
EWI mentioned as
Source
Delay in PO
Project status
report
Delayed vendor design information produced a cascading effect and ultimately induced
schedule and cost related problems, which in turn affected the availability of work front for
mechanical and piping contracts at site. From the perspective of EWI, the delay in issuance of
PO resulted few project problems and ultimately contributed to the growth in project
outcomes. The relationship of early warning, project problems, and project outcomes is shown
in attachment in D.4.4.
76
Case 1
Case 2
Case 3
X
X
X
Case 4
X
X
X
Red Tick = Early warning detected and contributed to project problems, Green tick =
early warning detected but did not result into any problem, X = no early warnings was
detected
The effect of COC was observed to be important in all case projects. The conflicts in project
environment not only reduce the effectiveness of decision-making process, but also affect the
management of changes. From efficiency and communication perspective, the early warning
LES did play an important role during execution of case projects. In case of project two and
three, the positive synergy on project execution strategy among project team members did
improve the communication and enhanced the overall efficiency. In addition, the
understanding of project execution strategy among project team does result into a focused
approach towards projects main driver and help the project to adapt in case the project driver
changes (due to external or internal environment). In addition, it was found that incase of
projects with less than successful performance, lack of experience among few domains of
project team members did induced problems.
Change management: All four case projects did declare the use of No change policy at the
start of EPC phase. The no change prohibits the introduction of any change unless and until
the design is either unsafe, does not function or violate any procedure (Bakker, 2012).
However, almost all the projects did implement changes during their execution.
Early warnings
NOC
CCO
Case 1
Case 2
Case 3
Case 4
77
The case project two experienced some rework due to early changes in the project that
resulted into growth in engineering man-hours and made small impact on the TIC. However,
the small nos of changes allowed proper assessment and management of those changes.
In case of project 3, the nos of changes were seen as small by project. In addition, the
majorities of the changes were small in their cost impact and intended towards cost savings
and no major problems were found to be associated with implemented changes during project
execution.
Early engineering: The early warnings FED, PH and PS did show a distinct behavior in
successful and less than successful projects. The results of early warnings in this category are
shown in following table.
Early warnings
FED
PH
PS
Case 1
Case 2
Case 3
X
X
X
Case 4
X
X
Case 1
Case 2
Case 3
Case 4
78
in concurrency level was observed due to the high risk associated with assumptions thus
delaying the deliverables or actual information found to be different from assumptions, which
forced the piping discipline to forecast delay in their schedule.
In case of project two and three, the change in concurrency level was less significant as
compared (20%) to project one. The projects were able to absorb the changes and did not face
any major problems. The ability to absorb the increase in concurrency level could also be
explained by number of changes in these projects, as the project did not experience significant
amount of changes therefore it could be concluded that the assumptions made by piping
engineering did not change significantly.
From change in concurrency perspective, the case project four proved to be an exception. The
project was seen as extremely concurrent by project team members, therefore the piping
engineering discipline was proactive in forecasting its schedule with reference to schedule of
expected inputs from process engineering. The joint change in progress curves of these two
disciplines resulted into low change in concurrency level.
Procurement: The early warning, DPO was detected in all four case projects.
Early warnings
CE
Case 1
Case 2
Case 3
Case 4
79
specific project problems and project problems were analyzed to map their affect on project
performance. By looking at the time aspect of relationship, it was concluded that the early
warnings could predict the project problems and specific project problems manifest because of
problems in domain of these early warnings.
Observatory evidence to predictive capability of early warnings: The contrast set by
project with different performance project played a significant role in highlighting the
predictive capability of early warnings. The differences in detection of early warning and their
associated states (quantitative and qualitative) validate the importance of early warnings
during project execution. It can be concluded that if these early warnings are monitored and
interpreted as symptoms of potential problems, they could provide advance and crucial
information for proactive actions.
80
7 QUANTITATIVE ANALYSIS
SUMMARY
In the previous chapter, it was observed that in case of good performance projects, most of the
early warnings were absent or had quantitative values within acceptable limits (In the view of
project team). The projects with significant cost/schedule overrun exhibited early warnings
with significant high quantitative values. The difference in performance of projects in light of
early warnings became clearer during cross case analysis in section 6.8.
In the present chapter, an attempt has been made to explore the set of early warnings over a
larger number of projects than in chapter 6. As explained in section 5.7, it is not possible to
include qualitative early warnings (LES, COC and PTE) in development of the quantitative
prediction model without introducing user specific subjectivity.
In addition, the early warning completion of FEED could not be implemented, as it is static
in nature and provides information only at the start of EPC phase. Furthermore, if the
deficiencies in FEED are not corrected before diving into EPC phase, the other early warnings
such as process engineering should show warning like behavior.
Section 7.1 discuses the analysis approach at three levels, longitudinal behavior analysis of
early warnings, correlation analysis at 95% engineering completion and longitudinal
correlation analysis of the project. The description of data collected for past projects along
with an explicit description of early warnings as explanatory variables and project outcomes
as dependent variables has been attached as appendix G.
Later in the chapter, section 7.2.1 analyzes the longitudinal behavior of early warnings over
engineering duration. For this purpose, each EWI is plotted over the engineering duration of
the project. Subsequently to have the quantitative evidence of relationship between EWI and
project outcomes, the correlation analysis at 95 % engineering completion is performed in
section 7.3, followed by longitudinal correlation analysis over engineering duration. In the end
of this chapter, section 7.5 provides the discussion on three levels of analysis and conclusions
of this chapter.
81
Based on the finding of cross case analysis in section 6.8, a hypothesis is made that the EWI
do behave differently in case of successful and less than successful projects. The first level of
analysis tests this hypothesis by studying the longitudinal behavior of EWI over engineering
duration of past eight projects. If the results of the analysis are found to be affirmative towards
stated hypothesis, then the findings could be seen as first empirical evidence of predictive
capability of EWI. In addition, the results could validate the correlations found in subsequent
analysis and supports the causality between EWI and project performance.
The second level of analysis, i.e. correlations at 95 % engineering completion could be seen as
complementary to level one analysis and at the same time take the analysis further by mapping
the relationship of each EWI with parameters of project performance i.e. project outcomes.
The main question to be answered by this analysis is that which EWI relates to which project
outcome. As it could be observed in chapter 6 that not all the EWI contribute to all the
project problems and not all the project problems, affect all project outcomes. Therefore, the
second level analysis makes an effort to understand the relationship between EWI and project
outcomes.
The third level performs a longitudinal correlation analysis over the engineering duration.
This analysis could be seen as a statistical combination of first two analyses. In addition, it
provides the dynamic statistical relationship between EWI and project outcomes that will be
used for building of prediction model in subsequent chapter.
82
Note: Moreover, looking at the project outcomes in four projects (in appendix E) with
reference to the study presented by Merrow6 in chapter 1, it can be inferred that the project
execution by Fluor Corporation are well executed. (As no project experienced growth of 30%
in either TIC or MCI)
It could be observed from the figure that the data include a mixed performance projects. For
the classification purposes, the present research consider a project with less than successful
performance, if and only if the project has 15 % growth in either final cost (TIC) or schedule
delay of more than 10% in completion of project (MCI) and combination of both.
The scatter plot matrix was plotted and has been attached in appendix E. It is observed that the
there is a strong relationship between TIC and MHI; this implies that a significant growth in
engineering man-hours leads to the growth in TIC. From cost perspective, the observation
seems obvious as engineering manhours has cost associated with them. As explained in
section 4.3.2 the cost of engineering efforts is quite small as compared to the cost of
equipments and construction (on average varies between 10-15 % of TIC). Therefore, even a
100 % growth in manhours could only affect 10 to 15 % of TIC. However from logical
perspective the above observation implies that the impacts of problems in engineering design
is not limited to engineering itself, in addition they do affect the activities in procurement and
construction, therefore affect total project cost. The above statement could be validated by
found relationships between EWI, project problems and project outcomes in chapter 6.
The above observation is also true for MCI and ESI i.e. delay in completion of detailed
engineering leads to the delay in mechanical completion of the project. As discussed in section
4.3.2 engineering prepare and deliver key documents for procurement and construction
activities, therefore a relationship between MCI and ESI validate that the delay in issuance of
these key documents do impact the progress of procurement and construction activities.
Another important observation is that the growth in engineering manhours does not
necessarily result in improvement of schedule. This could be observed by absence of negative
relationship between MHI and MCI. Therefore, it can be concluded that in most of the past
projects the growth in man-hours was not caused by schedule acceleration, rather rework or
other problems.
7.2.1
The next step in exploratory analysis was to study the behavior of each early warning to
check, if there exists a distinct behavior for projects that performed well and others whose
performance was not as per expectations. To achieve the above goal, each early warning was
plotted over the engineering duration of the project and results are attached as Appendix F and
discussed in following paragraphs.
Process schedule (PS): It is observed that PS showed a distinct behavior in good or
average performance projects and less than successful performance projects. An interesting
O&C industrys 50% project experience growth more than 30% in either cost or schedule.
83
observation is that in most of the projects, which did not perform well, process engineering
was not able to recover from delay until 60 % of the engineering duration. After which, almost
all the projects did show improvement. This could mean that the delay in early deliverables of
process engineering have more impact on the schedule and performance of downstream
disciplines and total project performance7.
Process hours (PH): The results are similar as PS. The PH had significantly low growth
values in good or average performance and high values in less than successful performance
projects. It could be observed from the figure that process engineering start projecting
significant growth after 30% engineering completion. When combined with delay in process
schedule, it can be concluded that process engineering forecast growth in manhours after
actual delay in early deliverables ( average 7.5 % delay in process at 30% engineering
completion) , which might be due to the changes or incomplete FEED.
Number of change orders (NOC): The analysis found that the good performance project did
experience less numbers of changes as compared to the less than successful performance
projects. The analysis indicated that the behavior of cumulative NOC in case of successful
projects (Green and blue) is significantly different as compared to less than successful projects
(Red). However at the early stages in the project, no clear distinction could be found between
projects (Except two), therefore the incremental number of changes were also explored and
difference became clearer as the less than successful performance projects did experience
relative higher number of changes within a short time span during their engineering execution.
Therefore, it can be concluded that changes combined with high incremental changes could
lead to bad performance of a project.
Cost of changes (COC): The costs of changes adopted during project execution do reflect
differences in projects, at-least after 45 % engineering completion. From zero to 45%
engineering duration, it is difficult to segregate between successful and less than successful
performance of the projects (same as NOC). However, the projects, which kept on introducing
changes even after 45 % engineering, ultimately resulted in undesired performance as
compared to the project that experienced little or no additional cost due to changes
Delay in purchase orders (DPO): The behavior of early warning DPO did not show a clear
distinction between the Successful and less than successful performance projects. Along with
successful projects, some less than successful performance project did not experience any
significant delay in issuance of purchase orders.
However, a deeper view on the projects provided more insights into the behavior of DPO.
Some less than successful performance projects issued their PO before the start of engineering
phase of the project, due to either favorable market conditions or the equipments had long lead
times. Despite the project strategic decisions, it can be concluded that the delayed issuance of
84
purchase order does affect the project performance as the related vendor information might be
delayed8.
In addition, the strategic advantage gained by early issuance of PO could be lost if changes
during detailed engineering impact the vendor equipments or services because that will cause
additional cost and schedule claims by the vendor and contribute to the cost and schedule
overrun of a project.
Concurrent engineering (CE): The early warning CE reflects the change in concurrency
level between process and piping engineering, i.e. what is the relative change in behavior of
piping engineering progress w.r.t process engineering. The analysis suggests that the in case
of less than successful performance projects (Shown in Red) the piping engineering progress
curve did came close to process engineering curve during early engineering, which implies
that they worked on concurrent engineering model9. However later, three out of five less than
successful performance projects were not able to sustain the concurrency level and piping
engineering subsequently projected schedule delay.
The conclusion is drawn on the basis of information provided by interviewees and explained in section 5.7
High concurrency implies that the piping engineering has to work on assumptions due to non-availability
of information from process engineering. For detailed explanation please see section 5.7
10
85
Table 4 shows the correlations between project outcomes and PH at 95% engineering
completion. A strong and significant correlation was found between PH and ESI (rs = 0.74, p
= 0.0366), Very strong correlation between PH and MCI (rs = 0.97, p = 0.001), and between
PH and TIC (rs = 0.69, p = 0.058). A moderate correlation with 85 % significance was found
between PH and MHI (rs = 0.55, p = 0.15).
Table 4: Correlation between growth in process engineering and project outcomes
EWI
Project
outcomes
Correlation
strength (rs)
Significance (P)
ESI
0.74+++
0.03
MCI
0.97++++
0.001
MHI
0.550*
0.15
TIC
0.69+++
0.05
PH
Meaning of correlation
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The presence of these correlations with high strength and significance suggested that growth
in process engineering hours indeed affect the project outcomes and ultimately project
performance.
Table 5 below shows the correlations between delay in process engineering (PS) and project
outcomes. The negative sign in correlations means that delay in process results into growth in
project outcomes. A high and significant correlation was found between PS and ESI (rs = 0.68, p = 0.15) and between PS and TIC (rs = -0.75, p = 0.03). Very high and significant
correlation was found between PS and MHI (rs = -0.93, p = 0.007). However no significant
correlation was found between PS and MCI (rs = -0.52, p = 0.18)
Table 5: Correlation between delay in process engineering and project outcomes
EWI
Project
outcomes
Correlation
strength (rs)
Significance (P)
ESI
-0.68++
0.06
MCI
-0.520
0.18
MHI
-0.93++++
0.00
TIC
-0.75+++
0.03
PS
Meaning of correlation
Delay in process engineering results
into delay in 95 % engineering
completion.
Delay in process engineering results
into delay in mechanical completion of
project.
86
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The above correlation suggests that delay in process engineering does affect the project
outcomes (3 out of 4) and ultimately project performance. The absence of significant
correlation between PS and MCI can be interpreted as that delay in process engineering might
be resulted into deployment of additional resources, which resulted into high manhours and
high TIC of the plant but did not impacted the MCI.
Table 6: Correlations between number of changes and project outcomes
EWI
Project
outcomes
Correlation
strength (rs)
Significance
(P)
ESI
0.79+++
0.02
MCI
0.83+++
0.01
MHI
0.6+
0.10
TIC
0.520
0.18
NCO
Meaning of correlation
Higher number of change orders resulted
into delay in completion of 95 %
engineering
Higher number of change orders resulted
into delay in mechanical completion of
project.
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The above Table 6 provides the relationships between number of change orders and project
outcomes. The correlation analysis suggests that there is strong and significant correlations
between NCO and ESI (rs = 0.79, p = 0.02) and NCO and MCI (rs = 0.83, p = 0.01).
Moreover, moderate, low significant correlation between NCO and MHI (rs = 0.6, p = 0.10)
and NCO and TIC (rs = 0.52, p = 0.18).
The correlations strength and values suggests that the high number of change orders does
affect the schedule outcomes i.e. 95 % engineering completion and mechanical completion of
the plant. However, does not have significant relationship with growth in manhours and
growth in TIC of the plant.
The Table 7 below shows the correlations between cost of changes and project outcomes. In
case of COC strong and significant correlation was found between CCO and TIC of the plant
(rs = 0.69, p = 0.05) and moderate and low significant correlation between CCO and MHI (rs =
0.52, p = 0.15). The correlation implies that the cost of changes does affect the project
outcomes associated with cost i.e. MHI and TIC of the project. In addition, it implies that cost
of changes does include additional engineering man-hour cost and cost does not only include
material cost.
Table 7: Correlations between cost of changes and project outcomes
EWI
Project
outcomes
Correlation
strength (rs)
Significance (P)
Meaning of correlation
87
ESI
0.240
0.57
MCI
0.350
0.39
MHI
0.550*
0.15
TIC
0.69++
0.05
CCO
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The Table 8 below shows the correlations between delay in purchase orders and project
outcomes. The negative sign in correlations implies that the delay in purchase orders results
into growth in project outcomes. The analysis found a very strong and highly significant
correlation between delay in DPO and MHI (rs = -0.99, p = 0.001). A strong and significant
correlation was found between DPO and TIC (rs = -0.73, p = 0.03). Moderate the low
significant correlations exits between DPO and ESI (rs = -0.50, p = 0.20) and between DPO
and MCI (rs = -0.49, p = 0.21).
The correlations strength and significance suggests that delay in issuance of purchase orders
resulted into growth in cost aspects of the project i.e. MHI and TIC, which could imply that
the projects intend to stay on schedule by either working on assumption, due to delayed
vendor information. Subsequently, then spend additional manhours in re-verifying the
assumption or re-working in case the assumptions were wrong and accepted high growth in
manhours and final TIC of the plant.
Table 8: correlations between delay in purchase orders and project outcomes
EWI
Project outcomes
Correlation
strength (rs)
Significance (P)
ESI
-0.50
0.2039
MCI
-0.490
0.2199
MHI
-0.99++++
0.001
TIC
-0.73+++
0.0396
DPO
Meaning of correlation
Delay in purchase orders resulted
into delay in completion of 895 %
engineering.
Delay in purchase orders resulted
into delay in mechanical
completion of the project.
Delay in purchase orders resulted
into growth in total engineering
manhours of the project
Delay in purchase orders resulted
into growth in TIC of the project
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The Table 9 below shows the correlations between the final changes (at
completion) in concurrency between process and piping engineering and
The positive sign in the correlations means that positive change (low
planned concurrency level) results into growth in project outcomes.
95 % engineering
project outcomes.
concurrency w.r.t
A moderate and
88
significant correlation was found between CE and MHI (rs = 0.64, p = 0.08). This means that
the delay in progress of piping engineering due to delay in inputs from process engineering
ultimately results into growth in engineering man-hours.
Table 9: Correlation between Change in concurrency and project outcomes
EWI
Project
outcomes
Correlation
strength (rs)
Significance (P)
ESI
0.140
0.73
MCI
0.20
0.62
MHI
0.64++
0.08
TIC
0.380
0.35
CE
Meaning of correlation
Significant change in concurrency between
process and piping engineering results into
delay in completion of 95% engineering.
Significant change in concurrency between
process and piping engineering results into
delay in mechanical completion the project
Significant change in concurrency between
process and piping engineering results into
growth in total engineering man-hours.
Significant change in concurrency between
process and piping engineering results into
growth in final TIC of the project.
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
89
Looking at the incremental changes in EWI, the time duration before and after 45%
engineering completion seemed to be critical. Significant correlations were found between
project outcomes, PH, and CCO. This could imply that most of the project with less than
successful performance did has significant incremental growth in PH and CCO around 45%
engineering duration.
The figure 45 provides the overview of significant correlations per project outcome over
engineering duration. It was observed that the cost related project outcomes i.e. TIC and MHI
have more correlations in general as compared to schedule project outcomes i.e. ESI and MCI.
The phenomena could imply that either the schedule impact of changes, delay in purchase
orders and high concurrent engineering was seen optimistically or ignored until 60 %
engineering completion or schedule was given priority over cost.
The ESI found to be consistently correlated with PH. This could lead to the inference that the
problems in process engineering do produce downstream cascading effect and delay the
subsequent engineering disciplines.
The change in concurrency only become significant after 60 % engineering, And found to be
positively highly correlated with MHI. This could mean that until 60% engineering, the piping
engineering tried to work on assumptions and keep its schedule as planned. However, when it
could not sustain the concurrency due to changes in assumptions (and need rework) and/or
need concrete information to issue documents it project the growth in man-hours.
90
ESI
0.79+++
0.64++
CCO
ESI
ESI
60%Engineering completion
MCI
MHI
TIC
0.83+++
0.67++
ESI
0.74+++
0.67++
0.59+
CE
-0.76+++
DPO
-0.69++
PH
-0.56+
0.6+
PS
-0.74+++ -0.81+++
0.62+
-0.9++++
-0.89++++
0.83+++
-0.75+++
0.66++
0.79+++
-0.66++
-0.86++++ -0.9++++
0.64++
-0.6+
-0.67++
0.65++
0.74+++
0.62+
0.62+
0.83+++
0.88++++ 0.71++
0.83+++
0.62+
0.74+++
0.87++++ 0.57+
NCO
0.74+++
-0.64++
-0.69++
0.62+
0.74+++
-0.62+
0.62+
-0.57+
ESI
ESI
ESI
CCO
ESI
0.88++++ 0.74+++
ESI
0.8+++
0.76+++
CE
-0.6+
DPO
0.69++
0.77+++
0.55+
0.8+++
NCO
0.57+
PH
-0.86++++
PS
++++
+++
++
+
0.001<P value<0.01
0.01<P value<0.05
0.05< P value<0.1
0.1< P value<0.15
-0.79+++
0.76+++
0.83+++
0.67++
1.0
0.9
0.7
o.4
0.66++
0.8+++
0.63++
0.78+++
-0.550
0.9 to
0.7 to
0.4.to
0.2 to
0.66++
Figure 20 Significant correlations between EWI and project outcomes at each prediction moment.
CCO
15
0.79+++
30
0.64++
45
0.83+++
MHI
60
0.74+++
75
15
30
0.74+++
45
60
75
0.64++
0.67++
0.67++
0.59+
0.64++
CE
-0.76+++
DPO
-0.69++
NCO
PS
-0.67++
-0.69++
-0.6+
-0.64++
0.62+
0.74+++
0.74+++
0.62+
0.62+
PH
-0.9++++
-0.75+++
ESI
-0.9++++
0.62+
-0.57+
0.62+
-0.81+++
0.79+++
0.83+++
-0.86++++
15
30
MCI
45
60
75
15
30
45
0.62+
0.57+
0.6+
0.83+++
0.88++++
0.74+++
-0.56+
-0.74+++
-0.62+
0.66++
-0.89++++
60
75
0.65++
0.83+++
0.71++
0.87++++
-0.66++
15
30
45
MHI
60
75
15
30
0.88++++
CCO
45
ESI
60
75
15
30
MCI
45
0.64++
60
15
30
45
0.74+++
60
-0.6+
DPO
0.69++
0.23+++
NCO
0.8+++
0.57+
-0.79+++
++++
+++
++
+
0.67++
-0.86++++
0.001<P value<0.01
0.01<P value<0.05
0.05< P value<0.1
0.1< P value<0.15
0.66++
0.77+++
0.66++
PH
0.9 to 1.0
0.7 to 0.9
0.4.to 0.7
0.2 to o.4
0.8+++
0.76+++
0.83+++
-0.550
Figure 21 Significant correlations of project outcomes with EWI over engineering duration.
92
75
0.8+++
0.76+++
CE
PS
75
0.63++
0.78+++
By analyzing the behavior of behavior of EWI over engineering duration, it was found that the
early warnings do behave differently; however, their visibility is not uniform over all EWI.
The cumulative values of PH along the engineering duration do provide visible difference in
successful and less than successful performance projects. The same is true for PS it was found
that process engineering in projects that experienced significant cost or schedule overrun, fall
behind its schedule consistently until 45 % engineering completion.
In case of number of change orders (NCO), it as observed that in case of less than successful
performance project, cumulative numbers of change orders keep growing at an consistent rate
(consistent slope of the curve in Error! Reference source not found.) However at the early
stages of the project, it is difficult to see the difference between projects. Therefore,
incremental number of change orders was also explored. Via analysis of incremental changes,
it was observed that less than successful performance projects did experience a large number
of incremental changes, on the contrary the successful projects did not experience any higher
incremental changes. The cost of change orders (COC), reflected a similar behavior as NCO,
yet the distinction between projects was found to be clearer than NOC.
The analysis of DPO proved to be complicated EWI, as few of the project that did not
performed well enough, showed a similar behavior as good performance projects. A more
detailed analysis revealed that some of the project do issue purchase orders well before the
start of EPC phase due to reason such as lead-time and market conditions. However, it was
observed good performance projects did not experience any significant delay in issuance of
purchase orders (Not more than 5%).
The EWI CE, which represents the change in concurrency between process and piping
engineering, also showed a distinct behavior in successful and less than successful
performance projects. The analysis revealed that the concurrency level does increase until 45
% engineering and thereafter the concurrency level decreases beyond the baseline concurrency
levels. This implies that in less than successful performance projects, piping engineering do
work on assumptions and try to stay on schedule, yet at the later stage due to changes in
assumptions and subsequent rework the progress of piping engineering get delayed.
From the discussion above, an overall conclusion can be made that the EWI do behave
differently either in absolute numbers or their incremental changes. Therefore, it seemed wise
to look not only at the absolute values of EWI, but at their incremental changes also.
7.5.2
The analysis was performed to find the correlations between each EWI and four project
outcomes at 95% engineering completion. The found correlations between EWI and project
outcomes are discussed in Table 4 to Table 9.
Growth in process engineering (PH) was found to be highly correlated with three project
outcomes (ESI, MCI and TIC) and moderately correlated with MHI. The delay in the process
engineering was found to be highly correlated with ESI, MHI and TIC. The correlation of PS
with project outcomes is negative in nature, implying if process engineering is delayed, the
project outcomes will see growth.
The numbers of change orders was found to be strongly correlated with schedule outcomes i.e.
ESI and MCI and less correlated with cost outcomes. Where as the other aspect of change
orders COC found to be highly correlated with project outcomes related to cost i.e. TIC. This
implies that higher number of changes do affect the schedule of project and when combined
with high cost changes, it will also affect the cost of the project.
The EWI, Delay in purchase orders was found to be highly correlated with cost project
outcomes (TIC and MHI). The possible conclusion can be drawn here that if issuance of
purchase orders is delayed, then extra engineering manhours are required to process the
delayed vendor information, thus results in growth of engineering manhours. Another possible
conclusion could be that due to delay in issuance of purchase orders, projects do place costly
orders, thus results into cost overrun, However there is no sound evidence was found, which
support this conclusion.
The last EWI, change in concurrency level between process and piping engineering CE, was
found to be correlated with growth in engineering manhours. The relationship implies that in
case the piping engineering work on assumptions (negative change in concurrency) during
early engineering, (due to delayed inputs from process engineering) then there are chances that
the assumptions might turned to be wrong or modified, which results in rework and delayed
piping progress (positive change in concurrency).
The table below explores the correlations among the early warnings. A very strong and highly
significant correlation was found between the PS and DPO. The correlation implies that the
projects that experienced delay in process engineering deliverables also issued purchase orders
late, thus validating the importance of process information for PO.
Another strong correlation was found between PH and NCO. The correlation suggests that if
process engineering is projecting significant growth in their manhours, probably the project
will experience high number of changes. In addition, a moderate negative correlation was
found between PS and NCO. Therefore, an inference could be made that if process
information could is frozen on schedule and without rework then there are chances that the
project will experience less number of changes, thus there are higher chances of success.
The correlation between DPO and CE (negative correlation) suggests that the project that
experienced delay in purchase orders also ended up with less concurrency than the baseline,
implying that the planned concurrency level might not be sustainable if the purchase orders
are delayed. The above observation point towards the dependence of piping deliverables on
information form equipment vendors.
Early
warning
Correlations
parameters
CCO
CCO
rs
CE
Early warning
DPO
NCO
PH
PS
94
Sig.
rs
Sig.
0.55
0.16
CE
rs
Sig.
-0.49
0.2166
-0.68+++
0.0621
DPO
rs
Sig.
0.55
0.16
0.5
0.207
-0.57
0.1361
NCO
rs
Sig.
0.38
0.3518
0.21
0.6103
-0.53
0.1796
0.86++++
0.0065
PH
rs
-0.56
-0.48
0.91++++
-0.63++
-0.57*
Sig.
0.1463
0.229
0.0017
0.0909
0.1361
PS
++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level, * significant at 0.15
level, N = 8
7.5.3
Looking at the analysis from previous subsections, it was important and interesting to
investigate the dynamic relationship between EWI and project outcomes over engineering
duration of the project. The relationship was investigated at two levels, between absolute
values of EWI and project outcomes and between incremental values and project outcomes.
All significant correlations are shown in Figure 20 and Figure 21.
The dynamic nature of found correlations during engineering exhibit the similar nature as
behavior of early warnings explained in section 7.2.1. As the engineering progresses the
number of correlations between EWI and project outcomes increases i.e. minimum
correlations at 15 % engineering completion and maximum at 75% engineering completion.
Looking at the individual EWI, the correlations of PH were found to be similar to its behavior
in Error! Reference source not found., and it is true for other EWI. For example, cumulative
value of NCO did not found to be correlation with project outcomes until 60 % engineering,
after which it was found to be strongly correlated with schedule outcomes. In a sense, the
result of the dynamic correlations validates the distinct behavior of EWI in case of less than
successful and good performance projects.
7.5.4
General conclusions
Looking back at the results of quantitative analysis and case studies, there is strong evidence
that the EWI are indeed related with project performance and has the predictive capability to
predict final performance of projects. It is also showing that not all the EWI are correlated
with all the project outcomes at all moment during engineering execution. Therefore a
dynamic prediction model would be required, which can utilize the related EWI at different
moments and predict the future performance based on them.
95
96
Derived from
Reference
Goals
R1
Goals
R2
Knowledge
gaps
stakeholders
R3
R4
stakeholders
G1
97
stakeholders
stakeholders
G2
G3
The performance prediction model should be able to make accurate prediction of future
performance during early phases. The prediction information should be comprehensive in
nature i.e. the potential cost and schedule overrun (if there might be any) along with the
potential future problems that might cause that overrun. The prediction should be continuous
in nature rather than categorical, so that the severity of potential cost and schedule overrun
could be known beforehand. Furthermore, categorical prediction i.e. successful or less than
successful could be made from continuous prediction by setting the threshold for project
outcomes.
The prediction should be objective and completely based on the facts from the previous
projects. The current project controls techniques utilize the forecast made by project team
based on their assessment of project performance thus are susceptible to optimism bias. An
additional unbiased performance prediction could provide an opportunity for project team to
reflect upon their forecast in combination with status of EWI. If case of any warning from
these EWI and undesired performance prediction an investigation and corrective strategy
could be launched to avoid any potential problem.
Furthermore, due to the limited past project data and conceptual nature of the first prediction
model, it seemed necessary to adopt a methodology, which can explain its reasoning for
prediction of future performance. The open knowledge from prediction model could be
applied at a later stage to larger set of past project. In addition, if the quantitative dynamics is
understood at conceptual stage, a more sophisticated prediction methodology could with larger
dataset of past projects.
In addition to the above requirement, few general guidelines are developed which should be
strived for when developing an operational model. The first guideline is that the model should
be easy to use and results should be understandable. The second guiding principle is that the
prediction model should not provide incentives for strategic behavior by project team
members. The strategic behavior into context of prediction model can be described as hiding
or manipulation of information to get desired state of project performance.
98
techniques are linear regression, logistic regression, and probit regression, neural networks,
support vector machines, nave Bayesian classifier.
Furthermore, the predictive techniques can be classified into two classes based on how the
model learns the prediction i.e. supervised learning and unsupervised learning. Supervised
learning refers to a function that maps the relationship between predictors and outcomes (i.e.
their place in function are known), whereas unsupervised learning refers to the techniques
which infers from set of observations without labeling them as predictors or outcomes (e.g.
clustering) (Hastie et al. 2008).
The selection of predictive modeling technique is primarily guided by type of variable to be
predicted and its desired state of information i.e. numeric or categorical. In the present
research, the selection of prediction methodology is guided by the requirements stated in
section 8.1. The requirements R3 state that the prediction model should predict project
outcomes that are numeric and continuous, so that the severity of potential cost or schedule
overrun is known. The requirement R3 rule out the classification prediction techniques that
could classify the project as successful or less than successful such as logistic regression,
nearest neighbors classification, nave Bayes classifiers and support vector machine
classification.
Furthermore, R4 requires that the model should be able to explain its prediction. The
requirement ruled out the so-called black-box prediction techniques such as Kernel
regression, and support vector machine regression. These techniques are known as
nonparametric regression and they fit the model by penalizing the errors in prediction by using
weights proportional to the inverse of variance (Mayers, 1990). In addition, the nonparametric
regression model requires larger sample sizes than parametric models because the data must
supply the model structure as well as the model estimates. In addition, with small data size
there are high chances of model over fitting (Welling, 2010; Hastie et al. 2008; Mayers, 1990)
Looking the associated merits and demerits of above mentioned prediction models and
available data, the linear multi regression models seemed to be reasonable choice. They are
simple and often provide an adequate and interpretable description of relationship between
inputs and outputs. They are based on least squares method, in which the model selects the
variable coefficients to minimize the residual sum of squares (Hastie et al. 2008; pg. 44).
99
visible i.e. the prediction model should be time-dependent. Therefore, the prediction moments
are selected from 15% engineering completion to 75% engineering completion.
Table 10
What to be predicted
When to be predicted
How to predict
- At 15 % engineering completion
EWI at 15
completion
- At 30 % engineering completion
- At 45 % engineering completion
- At 60 % engineering completion
- At 75 % engineering completion
engineering
The answer to third question How to predict is drawn from section 7.2.1 and 7.3. It was
observed that the EWI do behave differently in successful and less than successful
performance projects and in the subsequently section, 7.3 significant statistical relationship
were found between EWI and project outcomes. Therefore, EWI were used as predictors to
predict final project outcomes.
The stepwise approach to develop prediction model is provided in Figure 22 : below. At each
prediction moment, the first step is to establish the correlations between EWI and project
outcomes. Based on the analysis of the correlations, the EWI with at least moderate
correlations (rs = 0.5) and 85 % probability (p=0.15) with project outcomes will be selected.
In the second step, the selected EWI were checked for multicollinearity11 i.e. the correlations
among selected EWI. The aim was to select EWI, which were more independent and highly
correlated with a project outcome. The criteria for multicollinearity is set at minimum rs = 0.7
and p = 0.05. This criterion is selected after a round of investigation into inter-correlations
between EWI.
After the check for multicollinearity, independent and significant EWI were used for multi
regression to predict the final project outcomes. The minimum requirement for EWI was set at
two and due to the less numbers of past projects; the maximum limit for EWI in a regression
model was limited to four. Therefore, in case of more than four EWI, the EWI were selected
11
Multicollinearity refers to a statistical phenomenon in which two or more predictor variables in a multiple
regression model are highly correlated. Due to the inter-correlations, the coefficient estimates may change
erratically in response to small changes in the model or the data. Multicollinearity does not reduce the
predictive power or reliability of the model as a whole, at least within the sample data themselves; it only
affects calculations regarding individual predictors. (Aiken & West, 1991)
100
based on their regression coefficients i.e. the top four EWI with highest correlations with a
project outcome were used.
Establish correlations
between EWI and project
outcomes
Analysis of
correlations
Select EWI
with high and
significant
correlations
Establish correlations
between EWI to check
Multicollinearity
Select
independent EWI
with high and
more significant
correlations
Activities
Decisions
101
Time
R.1
15%
R.2
15%
R.3
15%
R.4
Variables
CCO-15
PS-15
DPO-15
PS-15
DPO-15
NCO-15
PS-15
-
Project
outcome
Model equation
Rsquare
value
Model P
value
TIC
0.01581+18.38801*(CCO-15)0.01946*(PS-15)
0.9567
0.0003906
MHI
0.26452-4.07028*(DPO-15)0.04660*(PS-15)
0.7917
0.01981
MCI
0.148173+2.054449*(DPO15)+0.009458*(NCO-15)0.024970*(PS-15)
0.9449
0.0216
The approach was applied to each prediction moment and the summary of regression models
is attached below.
Table 12: Prediction models at 30 % engineering completion
Model
No
Time
Variables
Project
outcome
Model equation
R.5
30%
DPO-30
TIC
0.001121+13.097283*(CCO-
Rsquare
value
0.9728
Model P
value
0.01104
102
R.6
30%
R.7
30%
R.8
30%
PS-30
CCO-15
PS-15
PH-15
PS-15
PS-30
PH-30
PS-30
PH-30
PS-15
15)-0.243072*(DPO-30)0.016940*(PS-15)0.001786*(PS-30)
MCI
0.03280+0.09575*(PH-15)0.00618*(PS-15)-0.01853*(PS30)
0.6322
0.22
ESI
0.02292+0.41004*(PH-30)0.02487*(PS-30)
0.5772
0.1163
MHI
0.32199+1.16856*(PH-30)0.05965*(PS-15)
0.6966
0.0507
Time
R.9
45%
R.10
45%
R.11
45%
R.12
45%
Variables
CCO-45
DPO-30
DPO-45
PS-45
CCO-45
PH-45
PS-45
DPO-45-1st
PH-45-1st
PS-30
PS-45
DPO-45-1st
PH-45
PH-45-1st
Project
outcome
Model equation
Rsquare
value
Model
P value
TIC
-0.014780+1.948622*(CCO45)-1.696703*(DPO30)+1.126065*(DPO-45)0.002113*(PS-45)
0.9303
0.0440
8
MHI
0.04258+7.37035*(CCO45)+1.13930*(PH-45)0.01938*(PS-45)
0.836
0.0475
6
MCI
0.033925+0.860814*(DPO-451st)+0.407402*(PH-45-1st)0.023325*(PS30)+0.012497*(PS-45)
0.9771
0.0085
42
ESI
-0.004889-0.097507*(DPO1st)+0.671001*(PH45)+0.673656*(PH-45-1st)
0.8699
0.0303
1
Time
R.13
60%
R.14
60%
R.15
60%
R.16
60%
Variables
CCO-60
NCO-60-1st
PS-45
CCO-60
CE-60
NCO-60
PH-60
Project
outcome
Model equation
Rsquare
value
Model P
value
TIC
-0.011655+0.723199*(CCO60)+0.003017(NCO-60-1st)0.014405*(PS-45)
0.6549
0.1957
MHI
0.353659+3.366194*(CCO60)+0.489701*(CE-60)0.004420*(NCO60)+1.711740*(PH-60)
0.9255
0.04854
MCI
-0.034538+9.445831*(CCO60-1st)+0.006913*(NCO-601st)+0.035976*(PH-60)
0.5813
0.2785
0.8307
0.05063
CCO-60-1st
NCO-60-1st
PH-60
CCO-60-1st)
PH-45-1st
ESI
-0.00519+7.91209*(CCO-601st)+0.56122*(PH-45-
103
1st)+0.52772*(PH-60)
PH-60
R.17
Time
75%
R.18
75%
R.19
75%
R.20
75%
Variables
CCO-75
DPO-75
NCO-75
PH-75
CCO-75
CE-75
PH-60
PS-75
NCO-75
PH-75-1st
PH-75
CCO-60-1st
CE-75-1st
PH-60
PH-75
Project
outcome
Model equation
Rsquare
value
Model P
value
TIC
-0.098785-5.979603*(CCO-75)2.748513*(DPO75)+0.002186*(NCO75)+0.245588*(PH-75)
0.8528
0.1288
MHI
0.29162+1.90469*(CCO75)+0.50289*(CE75)+0.45872*(PH-60)0.03505(PS-75)
0.9537
0.02422
MCI
0.0550667-0.0013335*(NCO75)+0.5382666*(PH-751st)+0.4170979(PH-75)
0.8867
0.02315
ESI
-0.04344+22.50128*(CCO-601st)+0.56418*(CE-751st)+0.61536*(PH-60)0.11908*(PH-75)
0.8117
0.1512
104
For the first level analysis, predicted values of project outcomes via prediction model are
compared with values predicted by traditional forecasting methods. For the purpose of
explanation, Figure 23 provides an example in which the delay in mechanical completion of
project is predicted by two methods longitudinally over engineering duration of the project.
The yellow curve represents the actual final delay in mechanical completion of the project.
The blue curve represents the prediction by traditional method. The pink curve represents the
prediction made by prediction model. It can be observed that the pink curve is quite close to
actual curve. Whereas, the forecast made by traditional methods is changing from zero to 40
% and then coming close to actual value.
Prediction of MCI (Project 1)
0.25
1.2
Error by Traditional method
P1 Traditional Method
P1 Actual
0.2
0.8
Percentage growth
0.15
0.6
0.1
0.4
0.05
0.2
0
15
30
45
60
75
Figure 23 Predictive capability comparison of traditional method and developed prediction tool.
The bars in above figure represent the absolute error made in prediction by two methods. The
errors are measured as predicted value minus actual value of project outcome divided over
actual value. The pink bar represents the error made by traditional methods and violet bar
represents the absolute error made by prediction model. The error in prediction model is far
less than traditional method.
The above example is just one case; therefore basing our conclusion on this case is not
practical. The concept of absolute error is applied to all past projects (same as used in building
the model) and following paragraphs discuss the results.
105
The Table 16 below shows the comparison of forecast made by traditional methods and
forecast by prediction model. The signs in table are defined as follows:
+ sign means that the absolute error made by traditional methods is larger than the absolute
error by prediction model, implying forecast made by prediction model is more accurate.
- sign means that the absolute error made by traditional methods is smaller than the
absolute error by prediction model, implying forecast made by prediction model is less
accurate.
Table 16 : Predictive capability comparison of traditional methods and prediction model
Green = Good performance, Blue = Average performance
Red = less than successful performance
Prediction of TIC
Engineering
completion
15%
30%
45%
60%
75%
Engineering
completion
15%
30%
45%
60%
75%
Engineering
completion
15%
30%
45%
60%
75%
Engineering
1
+
+
+
+
+
1
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+
+
+
+
Project Number
4
5
+
+
+
+
+
+
+
+
-
2
+
+
+
+
-
Prediction of ESI
Project Number
3
4
5
+
+
+
+
+
+
6
+
-
7
+
+
+
+
+
8
+
+
+
+
+
2
+
+
+
-
Prediction of MCI
Project Number
3
4
5
+
+
+
+
+
+
+
+
6
-
7
+
+
+
+
+
8
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Prediction of MHI
Project Number
106
completion
15%
30%
45%
60%
75%
1
+
+
+
+
-
2
+
+
+
+
+
3
-
4
+
+
+
+
+
5
+
+
+
+
6
+
+
-
7
+
+
+
+
+
8
+
+
+
+
+
It could be observed that at majority of the prediction moments, the forecast made by
prediction model is more accurate than traditional methods.
Another important observation can be made from the analysis that in case of good
performance projects, the traditional methods were found to be more accurate than prediction
model and reverse is true in less than successful performance projects. Two possible
explanations could be provided for this observation 1) presence of optimism bias in traditional
methods. As discussed by Flyvbjerg and Bruzelius, many forecasts believe that their project is
less prone to problems therefore keep their forecast close to the desired state, until the problem
has already manifested (Flyvbjerg & Bruzelius, 2003). 2) Presence of possible bias towards
less than successful projects in dataset due to relatively high number of less than successful
projects.
8.6.2
Analysis of errors
The regression models developed in above section showed sufficient R-square value. Yet the
R-square was not seen as sufficient explanation of error, therefore the prediction models were
used to predict the project outcomes of same projects used in building of the model.
The errors between predicted and actual values of project outcomes were plotted over
engineering duration of projects. The errors were measures in absolute terms irrespective of
the sign. The errors are measures as difference in predicted value minus actual value of project
outcome divided over actual value. For example, absolute percentage error is TIC is measured
as difference between predicted values minus the actual cost of the project divided over actual
cost of the project.
On similar terms, the errors in project outcomes were plotted and analyzed. The following
paragraphs discuss the errors per project outcomes.
Errors in TIC: The error found in prediction of project TIC is shown in Figure 24, each
curve represents an individual project. The curves reflects that for most of the projects, the
error stays with in 6 % range till 45% engineering completion, Thereafter, absolute error
become as high as 12 % for few projects. This is in direct conflict with correlations found in
chapter 7.
107
0.2
0.18
0.16
Absolute % error
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
15
30
45
60
75
Engineering % Complete
The Figure 25 shows the errors in prediction of ESI. No prediction could be made at 15 %
engineering completion due to insufficient numbers of EWI. The figure shows that the errors
decrease from 30 to 45 % engineering and then remain more of less stable (within range of 8
to 9%). This could be explained by low number of correlated EWI with ESI. Figure 21,
validate the above statement and shows that only PH showed a consistent correlation with ESI
other than few incremental changes in EWI.
Error in prediction of ESI
0.24
0.2
0.16
0.12
0.08
0.04
0
15
30
45
60
75
Engineering % complete
The following Figure 26 shows the errors in prediction of MHI, the errors found to be in
accordance with the established correlations in chapter 7. As, until 45 % engineering
108
0.44
0.4
0.36
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
15
30
45
60
75
Engineering % complete
The same is true for MCI, The Figure 27 shows the errors in prediction of MCI, and it can be
observed that on overall level, the errors seem to be decreasing from 30 % onwards. In
addition, the error stays within 4% ranges (except one project) in prediction after the
completion of 45 % engineering.
109
Error in MCI
0.2
0.16
0.12
0.08
0.04
0
15
30
45
60
75
Engineering % complete
Predication confidence
Average
Min.
Max.
110
P50
P80
tolerance to point
estimate
(growth in TIC)
7.5%
15.5%
tolerance to point
estimate
(growth in TIC)
3%
6.5%
tolerance to point
estimate
(growth in TIC)
14%
30%
The actual value is within or close to P50 range. The maximum difference between lower and
upper range at 60% engineering completion was found to be too wide, however the point
estimate is closest to actual value. From 15 % to 60% engineering completion, the estimated
ranges are quite reasonable as compared to actual value. Longitudinally, the estimates
improve. However, at later stages, the lower bound values show a possibility of under run. In
case of P80, the average difference between the ranges is too wide ( 15.5 %), therefore
considered as less reliable.
The validation results for MCI are shown in Figure 29. The point estimate did improve
longitudinally, but showed a less growth at 75% engineering completion. The actual value is
more or less within P50 range. The statistics of prediction ranges are shown in following table.
Predication confidence
P50
P80
Average
tolerance to point
estimate
(growth in MCI)
6.5%
13.5%
Min.
tolerance to point
estimate
(growth in MCI)
3.5%
7%
Max.
tolerance to point
estimate
(growth in MCI)
11%
22%
111
Looking at the point estimate, the prediction does seem to improve longitudinally. However
goes below the actual value after 45% engineering completion and predict a lower growth at
60% and 75% engineering completion.
The validation results from ESI are shown in Figure 30. No prediction model was found at
15% engineering completion. After the 30% engineering completion, the point estimate for
growth in ESI proved to be on lower side of actual growth. The P50 range estimate covers the
actual growth and upper estimates are close to the actual value, however overall range is
found to be too wide to make any policy decision.
The prediction ranges for P50 and P80 are shown in following table:
Predication confidence
P50
P80
Average
tolerance to point
estimate
(growth in ESI)
11.5%
24.5%
Min.
tolerance to point
estimate
(growth in ESI)
10%
21%
Max.
tolerance to point
estimate
(growth in ESI)
13%
28%
112
For prediction of MHI, the project proved to be exceptionally different from dataset. In the
data set utilized to build the model, for average 20% growth in TIC, 77% average growth was
observed in MHI, whereas in case of this project the final growth in MHI was found to be
21%. The errors in the prediction of MHI are found to be high. The actual value is outside the
prediction ranges of P50.
113
Requirements:
is accurate, timely, and comprehensive information to assist
management in making decisions to prevent future problems
quantitative and continuous prediction so that the severity of
potential cost and schedule overrun is known
free from optimism bias
The prediction model should be able to explain its reasoning of
prediction
Evaluation
(Meet the
requirement?)
Partially
yes
yes
yes
The most important requirement for prediction model is to predict the future performance that
is accurate, timely and provide comprehensive information to guide the interventions during
project execution. The current form of prediction model partially meet this requirement, as the
model predict the future performance at early stages, thus allowing the time for proactive
decision-making. However, the predictions made by current form of model have limited
accuracy (errors) and reliability (low confidence level of predictions). In spite of limited
114
accuracy and reliability, the stepwise development model does provide a useful framework to
enhance the accuracy and reliability of model by adding more projects.
In addition, the level of comprehensive information could be further improved by adding the
potential problems associated with EWI. Few of the potential information have been identified
in chapter 6 and an effort has been made for their integration in prediction model during
formation of implementation strategy in section 8.9.
The second and third requirements are fully meet by pilot prediction model. The predictions
made by model are continuous and quantitative, thus do have the capability to reflect upon the
severity of cost of schedule overruns. The data used for training the model is purely
quantitative and objective. In addition, the data set is based on the previous completed
projects; therefore predict the future performance from actual situation in similar projects and
could be considered as free from optimism bias.
The fourth requirement is met by using the multi-regression as prediction technique. The
mathematical models provide the explicit relations between EWI and project outcomes, thus
the effect of each EWI in a model could be understood by magnitude of its coefficient and its
sign in model.
Potential problems
NCO
CCO
115
EWI
Potential problems
PH & PS
Potential problems
CE
Potential problems
DPO
Prediction Moment
PS = 6 % behind schedule
CCO = 5% of estimated TIC
DPO = 6% at 30% engineering completion
=4% at 45 engineering completion
116
Based on the EWI values and associated potential problems, following problems could
manifest and drive the TIC to predicted growth:
Problem
predictor
PS
Potential problems
- Delay in critical information to downstream disciplines, procurement activities
- Concurrent engineering execution
CCO
DPO
In the above illustration, the undesired growth in TIC is predicted, which can triggered the
investigation into potential problems associated with EWI and could enable the informed
decision making to avoid above mentioned problems.
117
The results of external validation reflected the limited reliability of pilot prediction model. On
average, the point predictions of project outcomes except MHI were found to be close to
actual values, however lacks consistent behavior. The prediction ranges with 50% confidence
(P50) embrace the actual value and could be seen as reasonable prediction ranges. The
prediction ranges with 80% confidence interval were found to be too wise and in current state
could not be used for guiding the decision-making.
Based upon the findings in this chapter, it should be concluded that the accuracy level of
prediction model developed by this research need more preciseness and reliability. To have
more preciseness, a larger dataset with balance of Successful and less than successful
performance project is required. Nonetheless, it can be concluded that if the approach
developed by this research is applied to a larger and rich dataset, a more precise and better
prediction model could be developed.
118
119
continuity of delay was not observed neither any significant early delays. The above insight
imply towards the criticality of early process deliverables.
2. In projects with of less than successful performance, process engineering forecasted
significant growth in their hours (on average 25%) from 15% to 45% engineering completion,
this phenomenon was not observed in successful projects.
3. A high numbers of incremental changes were observed in projects with less than successful
performance. In addition, the time of high numbers of incremental changes does affect the
severity of cost and schedule over run. It was observed that the projects in which high
incremental changes were implemented late (after 45% engineering completion) they
experienced more severe overruns.
4. Almost all the projects did experience increase in concurrency until 45% engineering
completion, however less than successful projects did experience positive change (piping
projecting schedule delay) of more than 30% in concurrency between 45% and 75%
engineering completion.
Based on the above insights, following general recommendations could be made:
1. Focus on status of early process deliverables for example critical unit P&IDs and reasons
for growth in process engineering hours.
2. The growth in proposed changes should be monitored both commutative and incrementally
and in case of high number of changes within one or two reporting cycles, efforts should be
made to first avoid the changes that could be avoided, secondly proper assessment of accepted
changes should be performed (may be add additional temporary resources). The assessment of
changes should not include only the cost and schedule impact, but also the cost and hours of
rework, impact on associated vendors and contracts.
3. The concurrency between process and piping engineering disciplines should be monitored
by tracking the relative change in their progress curve. Any significant change in concurrency
should be scrutinized by checking the level and types of assumptions made by piping
engineering disciplines.
120
9.2.1
The total results of the study, prediction model, EWI, potential problems can apparently be
considered as new, additional project control and management method created for the field of
project control. The usability areas of the research results can be examined both as separate
results that obtained from research phase one to five and as total, integrated prediction model.
In most simplistic form, the use of prediction model is shown by following figure:
Decision maker
For any current project, and at any prediction moment t1, the prediction model user collects the
data regarding status of EWI. The quantitative values for EWI are then used to predict the
future performance of project outcomes. In case any undesired performance, the relevant
explicit relation between EWI and potential problem and between potential problem and
project outcomes is mapped. Subsequently, decision-maker himself or with support from key
stakeholder plan and implement proactive response to mitigate the impact of potential project
problems.
9.2.2
Short-term implementation
The goal of short-term implementation is two fold. 1) The aspects of the research that is to be
implemented within the short term 2) The recommendations for improvement and further
development of prediction model and its use as a new method for management of project
execution.
121
The aspects of the research that could be implemented in 0-9 month duration are the use of
EWI, their relation with potential problems (identified in chapter 6 and again comprehended in
section 8.9). This can be achieved by mapping the behavior of EWI in current projects with
the behavior of EWI from past projects. Incase the behavior and status of EWI was found to
be similar to one of the lass than successful projects, an outside reflection could be provided
on potential overrun/under runs in project outcomes.
Before the prediction model is implemented as actual practice, additional efforts are required.
These efforts are formed as recommendations and categorized into two domains 1) Technical
2) Institutionalization.
Technical design: The recommendations in the technical domain are focused on design of
prediction model as a system. The recommendations for technical design of prediction system
are focused on associated aspects (can be termed as subsystems) and their interactions that are
necessary to act as whole.
1) Automated data collection process: The data collection process from project should be
automated and standardized. The standardization should progress in two directions i.e.
collection from current projects 2) collection from past projects. An automated excel sheet
could be developed which can be send to different Fluor offices (that execute similar types of
projects) across the projects to collect more data for increasing the accuracy and reliability of
prediction model.
2) Platform and technical architecture to make prediction model more automated: In the
current research, R project has been used as main statistical tool for predication modeling in
combination with Microsoft excel. From usability perspective, R project does not seem to be a
most preferred approach, however Microsoft excel is used extensively and considered as a
preferred option.
The prediction algorithms and development approach could be transferred to Microsoft excel
as Visual basic (VB) codes. The technical architecture for prediction model includes series of
processes and decisions moments. On higher level, the main processes that should to be
modeled are:
I. Prediction algorithms that can guide the selection of specific model and required EWIs
based on prediction moment.
II. Automated upgrading process that could provide the capability to enter new data from
past project and can upgrade the prediction model based on the new information
III. Database of relationships between EWI, project problems and project outcomes. The
system should be able to extract the relationships based on the status of EWI and
predictions of project outcomes.
2) Institutionalization: For successful implementation of prediction system, it is necessary to
institutionalize and foster the right incentives to its users and project managers. Following
recommendations could be made to institutionalize the prediction in short term.
122
I. Create and sustain the top management commitment: Commitment from top
management is a necessity for successful implementation of prediction system. In
present case, the dynamic prediction system is relatively new, therefore commitments
from top management need to be created and sustained. A way to do that is to make
the role of dynamic prediction of project performance more clear and highlight the
added value.
II. The second recommendation is to gather the support of users and project team
members. The prediction model and associated knowledge base should be distributed
among it potential users and project managers. The feedback from users and project
mangers could be accommodated into design of prediction system.
III. To provide the right incentives to its users and affected stakeholders, it is necessary to
build an open and constructive environment for participation. The message that should
be received by users, project managers and project team as whole is that they are not
just at the receiving end of prediction system. However, their contribution and
participation is critical for successful management of project execution. In addition,
the positive antecedent for their desired state of behaviors should be mapped and
implemented.
9.2.3
Once the prediction system has been developed, test and being institutionalized within
organization system and is supported by its sponsors, top management; prediction system
could be embedded within projects of Fluor. By this time, the following products are to be
available:
I. Prediction model equipped with sufficient dataset of part project
II. Second automated pilot of prediction system Excel based
III. Organizational requirements and draft procedure for implementation of prediction
system
The long-term implementation contains the further improvement in prediction system and its
introduction as independent system for management for project execution. The long-term
implementation will possibly induce additional improvement cycles. The Possible
recommendations for long-term implementation can be summarized as follows.
I. A transition from excel based prediction system to more sophisticated platform such as
web-based system with more robust database of past projects
II. A self-learning design to add current project data along with its predictions
III. Customized reporting capability to assess and quantify the impacts of interventions in
project execution, Furthermore automatic storing of interventions along with its
impacts, applicability, reference projects; to offer the decision-makers, a
comprehensive list of corrective actions.
123
IV. Capability to extract quantitative information for benchmarking and planning purposes
V. Integration with lesson learned framework to receive quantitative information along
with lessons.
VI. Sustain the top management commitment
In addition to further institutionalization of prediction system, a training module could be
developed to make the staff more familiar with prediction system, tools, its principles and
changes to the current procedures.
To widen the acceptability of prediction system beyond Fluor Corporation and gain the
acceptance of Fluors main clients, a joint research project could be setup. This will provide a
dual benefit for Fluor; 1) more projects could be added to the prediction system, therefore
increasing the accuracy and reliability of its predictions, 2) the acceptability of prediction
model will increase by incorporating the client needs, thus improve the project management
capabilities and competitive advantage of Fluor Corporation in E&C industry.
124
125
their use during EPC was found to be limited. IPA conducted its audits primarily at FID
(before the start of EPC phase) and at project completion and therefore are static in nature,
whereas the presented research is focused on dynamic performance management over EPC
phase i.e. project execution.
From literature review, we adopted our own definition of early warnings, which defined early
warnings as follows and therefore answer the second research question.
Early warnings are inherited measurable project characteristics and/or observations that
indicate towards the development of potential problems in future course of project
execution.
Subsequently, in chapter 4, the potential benefits of early warnings were presented. If well
detected in time, the early warning could provide an opportunity for the project manager to act
proactively and mitigate the problems, before they actually affect the project cost and
schedule.
RQ.3 What early warnings could be identified in project execution?
Having defined the early warnings, the next task was to identify the early warnings during
project execution. For that purpose, two main type of sources were selected, the literature
(academic as well as professional) and experts from industry. A selection criterion with three
parameters was designed.
Thereafter, the early warnings from literature and experts were selected irrespective of their
nature i.e. qualitative or quantitative. The analysis of early warnings (section 5.5.4 and section
5.6.3) from both literature and experts revealed that there are early warnings related to the
quality of project team and its integration that are critical for project success. All the early
warnings were evaluated on three selection parameters and finally ten early warnings were
selected for further analysis.
Therefore, answer to research question three is two fold, the comprehensive list in appendix A
provides the full list of all the early earnings that could be identified in project execution and
the list of selected early warnings from perspective of present research.
RQ.4 what early warnings can be operationalized to build a performance prediction model?
In an attempt to find answer to above research question early warning selection criteria was
developed. As the main objective was to have a quantitative model that could use the data
from past project, therefore measurability and data availability was two of three selection
parameters.
After the careful evaluation of each identified early warning on selection criteria, section 5.7
provides a list of early warnings (with their detailed explanation) that were selected for further
analysis. The following table provides the list of selected early warnings.
New ID
126
LES
PTE
COC
NCO
CCO
FED
PH
PS
CE
DPO
However, from operationalization perspective, few exceptions were made. The early warnings
related to project team development and integration (LES, PTE and COC) were included
because of their importance stated by both literature and experts. Therefore, a decision was
taken to explore them further at least during case studies. Another early warning FED
(percentage of missing information in FEED package) was selected for the same reasons.
RQ.5 What are the dynamics between early warnings and project performance?
Having selected the early warnings of future potential problems, a logical question arises that
how early warnings that belongs to the early phase of engineering can predict the problems in
late engineering, procurement or even construction. The above question has three parts, early
earnings, project problems and project performance (assessed via project outcomes). The first
part i.e. the research questions has been provided in the previous section of this search
(Section 5.7)
To connect the future project problems and project performance, a case study approach was
adopted. Chapter 6 explored the four past projects in detail (section 6.4 to section 6.7) and
provided the answer to research question five. The case study investigation found that there is
a relationship between early warning indicators, project problems and project performance.
For example, the problems associated with early warning indicator, number of changes (NCO)
are mentioned in Figure 32. The past project literature mentions those identified problems as
main reason for their less than successful performance.
127
Figure 32 future project problems and project outcomes associated with NCO
128
i.e. eight projects. Therefore, to have more concrete conclusion and evidence, more data
would be required.
Based on the results in chapter 7, chapter 8 explored the conceptual development of prediction
model with a well structured development approach. On the overall level, it was found that
predictions by prediction model are more accurate than traditional forecasting methods.
However, it is important to remember that the results are very much limited to small dataset
used for development of prediction model.
Furthermore, the analysis showed another limitation of dataset, the bias towards less than
successful performance projects. Therefore, to have a more accurate prediction in good
projects, a dataset is required which contains a balance of Successful and less than successful
performance projects.
Based upon the findings in this chapter, it should be concluded that the accuracy level of
prediction model developed by this research need more preciseness and validation.
Nonetheless, it can be concluded that if the approach developed by this research is applied to a
larger and rich dataset, a more precise and better prediction model could be developed.
Therefore, the chapter provided a limited answer to our last research question.
The research question one to four deals with the project performance and early warnings that
are used as main instrument for predicting future problems and final project performance via
129
literature and experience of experts. Answer to research question five provided the
observatory evidence of relationship between early warnings, future problems and project
performance using past projects. The answer to final sub-question six utilized the past projects
quantitative knowledge and developed an approach with first conceptual prediction model.
Based on so far research it can be concluded that early warnings can be used for predicting the
future performance of the project. In addition, the potential future problems associated with
early warnings could also be known before they affect the project performance.
However, the above answer to main research question is partial in nature due to very limited
dataset used in this research; therefore limit the generalization of findings. To have more
concrete conclusions and better prediction model, a larger dataset consisting of more projects
with similar nature should be used. Even though, the findings and prediction model
development approach presented in this research can be used to build a useful model and
applied in O&C project industry.
130
warnings of project problems and chapter 6, provided the observatory evidence of relationship
between early warnings and project problems. Further, the chapter 8 used the indirect
relationships (proved in chapter 6) between early warnings and project performance to build a
conceptual prediction model. The presented approach could provide the opportunity for
project managers to act well in advance incase the early warnings shows undesired
performance.
131
132
the research. When reflecting on the desk research (in phase one and two) it would have been
better to limit the literature to select early warnings. From the preliminary analysis of
literature, it became evident that the there is a lack in the scientific literature on this topic,
therefore expert interviews could have used earlier. The expert interview proved to effective
and efficient method to get the early warnings in project execution. The notion of
measurability in early warnings could have been defined more preciously; the interviews
resulted into many early warnings that could be measured, however not necessarily being
measured in actual practices.
The case study analysis as a research method and its execution did prove to be highly
beneficial to overall research. The in-depth case study analysis revealed the dynamics between
EWI and project performance i.e. the potential project problems. The cross-case analysis set
the background in which the evidence for prediction capability of EWI was obtained.
The data collection from past project proved to be most problematic and cumbersome task.
The corresponding data for each EWI has to be collected at 8 moments in project execution
and there were no standard reports were found from which the data could be readily obtained.
Moreover, data for few of the project had to be retrieved from archives and was available in
size of several GB. Nonetheless, to keep the quality of data high, the efforts were made to
collect the data from more than one source.
The statistical research methods i.e. correlation analysis and multi regression could be termed
as reasonable choice, however subjected to limited data availability. Furthermore, the data
from past projects could not be normalized (except the duration), because the variable that
cannot be predicted, it is not possible to normalize them. However, the similarity in nature of
projects and their sizes could be compensating for the absence of normalization.
133
Reflection on the gathered knowledge could be made by looking at the contributions made by
present research to practical and scientific domains. From scientific perspective on project
management, it can be concluded that the research does add to the dynamic control of project
execution. By highlighting the early warnings of potential problems and establishing its
relationship with overall project performance, the research does mark a shift prom reactive
project management practices to proactive project management. In addition, the research
provides a constructive use of past project, by learning from factual dynamic situation to
reduce the future uncertainties in current projects. From practical perspective, it can be
concluded that the prediction model is still far from being off the shelf method, thus have
limited usability in current form. However, as stated earlier the current research project sets
fertile ground for improvement in accuracy and reliability of prediction model.
134
11 REFERENCES
1. A.J.Shenhar, & Dvir, D. (1996). Towards a typology theory of project management.
Research policy, 25, 607-622.
2. Aberg, L. (1989). The facet theory of communication. University of Helsinki.
3. Anderson, D. R., Sweeney, D. J., & Williams, T. A. (2007). Essentials of Modern
Business Statistics with Microsoft Excel. Cengage learning.
4. Ansoff, I. H. (1984). Implanting Strategic Management. Prentice/Hall International.
5. Artto, K. A., Lehtonen, J.-M., & Saranen, J. (2001). Managing projects front-end:
incorporating a strategic early view to project management with simulation.
International Journal of Project Management, 19(5), Pg. 255264.
6. Atkinson, R. (1999). Project management: cost time and quality - two best guesses
and a phenomenon, Its time to accept other success criteria. International journel of
project management, 17(6), 337-342.
7. Barber, E. (2004). Benchmarking the management of projects: a review of current
thinking. International Journal of Project Management, 22(4).
8. Berends, T. (2007). Contracting economics of large engineering and construction
projects. TU Delft.
9. Betts, R. K. (1982). Surprise Attack, Lessons fro defence planning. Washington
D.C: The Brookings institution.
10. Booz Allen & hamilton. (2008). Capital project execution in the oil and gas industry.
11. Choi, J., Anderson, S., & Kim, S. (2006). Forecasting potential risks through leading
indicators to project outcome. Construction industry institute.
12. CII. (2004). Model Planning and controlling system for EPC of industrial projects.
Construction industry institute.
13. CII. (2012). Retrieved from https://www.constructioninstitute.org/scriptcontent/basics.cfm?section=aboutcii
14. CII. (2012). Manageing EPC projects in oil and gas.
15. Cleland, D. I. (1994). Project Management, Strategic Design and Implementation,
(2nd ed.). McGraw-Hill Inc. USA.
135
16. Coser, L. A. (1957). Social Conflict and the Theory of Social Change. The British
Journal of Sociology, 8(3), pp. 197-207.
17. Crawford, L., Pollack, J., & England, D. (2006). Uncovering the trends in project
management: Journal emphases over the last 10 years. International Journal of
Project Management, 24(2), 175-184.
18. Davies, H., & Ellis, P. (2000). Porters Competitive Advantage Of Nations: Time
For The Final Judgement? Journal of Management Studies, 1189-1214.
19. Fluor Corporation. (2012). Standard engineering work process.
20. Fluor, C. (2010). Guidlines for Detailed engineering. Guideline 000 200 1505.
21. Flyvbjerg, B., & Bruzelius, N. (2003). Mega project and risk - An Anatomy of
ambition. 207.
22. Freeman, M., & Beale, P. (1992). Measuring project success. Project management
journal, 1, 8-17.
23. Harpum, P. (2004). Project control in The Wiley Guide to Managing Projects (eds P.
W. G. Morris and J. K. Pinto. John Wiley & Sons.
24. Herman, M. (1996). Intelligence power in peace and war. Cambridge university
press.
25. Hughes, S., Tippet, D., & Thomas, W. (2004). Measuring project success in
construction industry. Engineering management journal, 16(3), 31-37.
26. IPA. (2012). Retrieved from http://www.ipaglobal.com/
27. Kerzner, H. (1994). Project Management, A Systems Approach to Planning, (Fifth
ed., Vol. Fifth addition). Van Nostrand Reinhold Co, USA.
28. Koontz, H. (1958). A PRELIMINARY STATEMENT OF PRINCIPLES OF
PLANNING AND CONTROL. 1, 45-61.
29. L.Lientz, B., & P.Rea, K. (1995). Project management for the 21st. Academic Press,
USA.
30. Larsen, E., & Gobeli, D. (1989). Singificance of project management structure on
project success. IEEE Transactions on engineering management, 36(2), 119-125.
31. Lientz, B., & Rea, K. (1995). Project management for 21st century. Academic press.
32. Lim, C., & Mohamed, M. (1999). Criteria of project success: an exploratory reexamination. International Journal of Project Management, 17(4), Pg. 243-248.
136
33. Merrow, E. w. (2012). Oil and gas industry mega projects:Our recent track record.
Society of petroleum engineers.
34. Might, R., & Fischer, W. (1985). THe role of structural factors in determining
project management success. IEEE Transactions on engineering management, 32,
71-77.
35. Morris, P., & Hough, G. (1987). The Anatomy of Major Projects. Wiley guide of
project management.
36. Morris, R. (1997). Early warning indicators of corporate failure, A critical review of
previous research and further empirical evidence. Ashgate publishing Limited.
37. Newcombea, R. (2003). From client to project stakeholders: a stakeholder mapping
approach. Construction Management and Economics, 21(8), Pg. 841-848.
38. Nikander, I. O. (2002). Early warning: A phenomenon in project magement.
Helsinki university of technology.
39. Nikander, I. O., & Eloranta, E. (2001). Project management by early warning.
International journal of project management, 19, 385-399.
40. Norwegian Technology Standards Institution. (1999). MECHANICAL
COMPLETION AND COMMISSIONING. Retrieved from
http://www.standard.no/pagefiles/965/z-007.pdf
41. OCED. (2011). Environmental Outlook to 2050: Draft Health and Environment
Chapter. OECD.
42. Oisen, R. (1971). Can project management be defined. Project management
Quarterly, 2(1), 12-14.
43. P.Lewis, J. (1993). The Project Managers Desk Reference. Probus Publishing Com.
44. PMI. (2012). Project management body of knowledge. Project management institute.
45. R.H.Mayers. (1990). Classical and modern regression analysis with applications.
PWS-Kent.
46. Raz, T. S., & Dvir, D. (2002). Risk management, project success and technological
uncertanity. R&D Management, 32(2), Pg-101-109.
47. Schlumberger business consulting. (2012). E&P Mega projects. Schlumberger
business consulting.
48. Thamhain, H., & Wilemon, D. (1986). Criteria for controlling projects according to
plan. Project management Journal, 17(2), 75-81.
137
138
APPENDIXA.1
EWI from literature
ID
L-1
L-2
L-3
L-4
L-6
L-7
L-8
L-9
L10
early warnings
Monitoring of
actual allocation
against the plan
can provide early
warning of lack of
manpower or lack
of work
Noticeable change
in performance
level for a team or
individual
members reflects
problems.
Changes in cost
estimates of
management
reserve, cost
variance, budget
slips
Everyone is
writing protection
memos, before
everything was
verbal
Hidden feelings,
conflicts avoidance
at all costs
Undefined clear
objectives and
fluid commitment
levels from key
participants
Duplication of
efforts
Undefined
deliverables
Argumentive
sessions at
meetings
Source
Bent - 82
Feature
Qualitati
ve
Quantitativ
e
Feeling,
Behavior
Typology
Descripti
ve
Performan
ce
indicator12
Cleland 84
Fleming83
Kerzner95
Kerzner95
Kerzner95
Kleim and
Ludin-94
Kleim and
Ludin-94
Kleim and
Ludin-94
12
Performance indicators are indicators with pre-established thresholds and deviation from thresholds is
noticed and reported on project level (for more information, see section 5.4.2).
139
L11
L12
L13
L14
L15
L16
L17
L18
L19
L20
L21
L22
L23
L24
L25
Project objectives
are cut back.
Project team
members are
reassigned.
Contingency plans
to project surface
No discussion on
task status
Difficulty in
getting resources
for project
High turnover rate
of personnel
High rate of
rework
High rate of
changes in part
prompted by
mistakes and
errors
Confusion and loss
of enthusiasm and
motivation
People withhold
information on a
problem in a hope
that the problem
will go away
An excess of no
cost no time
effects
High Level of
subcontractors
claims and
extension of time
claims
When people work
too much or too
little
Letters exchanged
between main
parties showing
there was not a
common definition
of what one partys
responsibility
is
Continually
unfulfilled
promises,
frequently
changing
decisions, or lack
of commitment to
make decisions
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Obradovit
ch - 90
Obradovit
ch - 90
Obradovit
ch - 90
Smith-83
Williams
et al. 2012
Williams
et al. 2012
Williams
et al. 2012
Williams
et al. 2012
Williams
et al. 2012
140
L26
L27
L28
L29
L30
L31
L32
L33
L34
L35
L36
Unaligned goals
Conflicting norms
and standards
Lack of trust in
project team
Lack of companys
internal support
Lack of experience
in project team
Owner and
contractor is
requesting an
excessive number
of contract
changes during
project execution
Significant project
scope items are
omitted from bid
packages
Project is
experiencing a
high level of
engineering/design
/ specifications
errors and changes
The project team is
frequently asking
vendors, suppliers
and service
providers to
perform functions
outside their area
of expertise and
experience
The project team is
lacking in the
necessary
expertise,
experience, breath
and depth to
successfully
execute the project
Design reviews fail
to include qualified
personnel who can
analyze safety and
loss prevention
features of plans
and specifications
Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011
CII-2006
CII-2006
CII-2006
CII-2006
CII-2006
CII-2006
141
L37
L38
L39
L40
L41
L42
L43
CII-2006
CII-2006
CII-2006
CII-2006
CII-2006
CII-2006
CII-2006
142
L44
L45
L46
L47
L48
L49
L49
CII-2006
CII-2006
IPA, 2012
IPA, 2012
IPA, 2012
IPA, 2012
IPA, 2012
143
APPENDIX A.2
Early warnings from Experts
Feature
ID
I-1
I-2
I-3
I-4
I-5
I-6
I-7
I-8
I-9
I-10
I-11
I-12
I-13
I-14
I-15
I-16
I-17
I-18
Early warning
Delay in Critical unit final
P&ID,s,
Lack of understanding of
execution strategy by project
team
Large amount/Numbers of
scope changes (additional
items) by clients
Large amount/Numbers of
scope changes (In details)
by clients
Lack of communication
between engineering
disciplines regarding
critical interfaces
Lack of communication
between engineering
disciplines regarding
assumptions and their
verifications
Low productivity of project
team members
Delay in 1/3 Model review
Delay in 2/3 Model review
Delay in Vendor design
information for critical
equipments
higher nos of 1/3 Model
review tags
Higher numbers of 2/3
Model review tags
High Concurrency level in
projects
High ratio of open changes
vs. approved changes
High Level of client
involvement
Late scope change
Variation in contingency
rundown rate
Overrun in process
engineering man-hours
Qualitative
Typology
Quantitative
Feeling,
Behavior
Descriptive
Performance
indicator
144
I-19
I-20
I-21
I-22
I-23
I-24
I-25
I-26
I-27
I-28
I-29
I-30
I-31
I-32
I-33
I-34
I-35
I-36
I-37
I-38
I-39
I-40
I-41
I-42
Overrun in piping
engineering man-hours
Delay in process
engineering
Ration of changes induced
by client vs. casual changes
Forecasted growth in
engineering man-hours
High Nos of arguments
between E&C contractor
and owner
Limited Level of risk
identification
Low level of team
integration
Lack of understanding
regarding project objectives
among project team
Delay in purchase order
placing
Growth in man-hours per
equipment
Growth in man-hours/
P&ID
The distance between peaks
staffing of engineering
disciplines is diminishing
Nos of changes After
P&ID's are issued for
design
The percentage of total RFQ
delayed over planned RFQs.
Growth in man-hours per
key quantities
Large increase in numbers
of engineering drawings
Changes in major project
objectives
Delay in critical equipment
specifications and data
sheets
Low progress earned for
critical Activities
Unrealistic monthly
incremental schedules
Delay in interface milestone
of engineering and
procurement
Early engineering discipline
crossing late schedule curve
Undue importance or focus
of parts of facility, while
ignoring the other parts
Presence of complex
interfaces with in project
145
I-43
I-44
I-45
I-46
I-47
I-48
I-49
I-50
I-51
I-52
I-53
I-54
I-55
I-56
I-57
execution methodology.
Higher numbers of changes
in project at early stages
Low motivation of project
team especially when its
induced by numerous
changes
High numbers of project
deviation notices
Delay in vendor design
information for detailed
engineering
Quality of project team in
terms of technical
experience and experience
with adopted procedures
Opportunistic behavior by
vendors
over optimistic monthly
incremental schedules
early spending of
Contingency (faster
rundown rate)
Changing project drivers
during execution
Incomplete Front end design
Failure to incorporate
technical complexity while
estimating the budget hours
for engineering
Undue delay in approval or
rejection of proposed
changes
Failure to build and
harmonized project team,
addressing cultural
differences
Low confidence of
engineering leads towards
schedule and budgeted
hours.
High variation in
construction quantities w.r.t
original estimate
146
ID
Early warning
I-1
I-2
I-8
I-9
I-10
I-11
I-12
I-21
I-24
I-28
I-29
I-30
I-34
I-36
I-37
I-42
I-54
I-56
Measurab
ility
Criteria
Early in
project
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Data
Availability
Status
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
Recommendation
147
Cost
Schedule
Quality
Functionality
Safety
3. Key definitions:
3.1) Early warning indicators (EWI)
Early warning indicators are inherited project characteristics and/or observations that indicate towards the
development of potential problems in future course of project execution. Reveled in timely manner, these
indicators allow proactive management to influence project outcomes.
148
3.3.3) Quality Quality performance is viewed as the amount of rework being done on project. Secondary
quality performance can include more document review cycles than target.
3.4.4) Functionality: Functionality performance can be seen as the actual numbers or cost of change orders
processed during project versus target.
3.5.5) Safety: Safety performance is viewed as OSHA incident rate (Actual Vs Zero)
4. Further steps: The selected EWI from literature study and expert interviews will be evaluated in the
context of project reality via quantitative analysis. The data related to selected EWI and project outcomes
will be collected from past-completed projects to establish the correlation between these EWI and project
outcomes. Furthermore, the correlations will be used in modeling of performance prediction tool.
The complete approach for this research is attached below.
149
150
APPENDIX D CONFIDENTIAL
APPENDIX E : CONFIDENTIAL
151
Variable
ID
PH
Delay in Process
hours
PS
Cost of change
orders
CCO
Numbers of
change orders
Delay in purchase
order
NCO
Change in
concurrency
between process
and piping
engineering
CE
Project outcomes
as dependent
variables
ID
Description
Measurement units
Total installed
cost
TIC
Percentage growth
+ve = Cost over-run
-ve = Cost under run
Engineering Man
hours
MHI
95 % engineering
schedule
ESI
Mechanical
completion
MCI
Percentage delay
+ve = Delay
-ve = ahead
Percentage delay
+ve = Delay
-ve = ahead
DPO
Description
Measurement units
Process engineering is
consuming or forecasting
more/less hours than baseline
hours.
Process engineering is
behind/ahead of baseline
schedule
The cost impact of implemented
or to be implemented changes
Numeric
+ve = ahead
-ve = delay
Percentage of estimated TIC
+ve = Growth
-ve = reduction
Numeric
Percentage delay
+ve = ahead
-ve = delay
Percentage change
+ve = Low concurrency w.r.t. baseline
-ve = High concurrency w.r.t. baseline
152
153
APPENDIXG:
G.1 Correlations among EWI and project outcomes at 15 % engineering completion
Early warning
indicator
Correlations
parameters
rs
CCO
Sig.
CE
Sig.
DPO
Sig.
rs
rs
rs
NCO
Sig.
PH
Sig.
rs
rs
PS
Sig.
Project outcome
ESI
MCI
MHI
TIC
-0.10
0.8225
0.130
0.7558
0.330
0.4198
0.79+++
0.0208
-0.050
0.9087
-0.10
0.8171
0.340
0.4076
0.120
0.7735
0.190
0.6514
0.040
0.9327
-0.50
0.207
0.430
0.207
0.140
0.7358
0.40
0.3325
0.310
0.4556
0.520
0.1827
-0.150
0.719
-0.56+
0.1476
-0.340
0.4063
-0.320
0.4441
-0.450
0.2604
-0.74+++
0.0349
-0.81+++
0.0149
-0.9++++
0.002
154
Project outcome
MCI
MHI
Correlations
parameters
ESI
rs
0.120
0.360
0.260
0.64++
CCO-30
Sig.
0.7789
0.3821
0.5309
0.0856
rs
CE-30
Sig.
0.170
0.6915
-0.230
0.5754
0.250
0.5479
-0.310
0.4528
DPO-30
Sig.
-0.290
0.4927
-0.240
0.5678
-0.50
0.207
0.64+++
0.028
NCO-30
Sig.
-0.220
0.6081
-0.050
0.8985
0.160
0.7128
0.260
0.5284
0.6+
0.1195
0.410
0.3167
0.62+
0.1017
0.380
0.3518
-0.540
0.1681
-0.89++++
0.0029
-0.540
0.1681
-0.75+++
0.0305
rs
rs
rs
PH-30
Sig.
rs
PS-30
Sig.
TIC
Correlations
parameters
rs
CCO-1st-30
Sig.
CE-1st-30
Sig.
DPO-1st-30
Sig.
st
rs
rs
rs
NCO-1 -30
Sig.
PH-1st-30
Sig.
PS-1st-30
Sig.
rs
rs
ESI
Project outcome
MCI
MHI
TIC
0.210
0.6103
0.190
0.6494
0.450
0.2604
0.290
0.4927
0.140
0.7358
-0.240
0.5678
0.140
0.7358
-0.480
0.2329
-0.520
0.1827
-0.350
0.3993
-0.330
0.4198
-0.6+
0.1195
0.20
0.6265
0.120
0.786
0.230
0.5855
0.270
0.5258
0.40
0.3199
0.420
0.3013
0.57+
0.139
0.290
0.4927
-0.050
0.9098
-0.020
0.9659
0.510
0.2007
0.230
0.5855
155
Project outcome
Correlations
parameters
ESI
MCI
MHI
TIC
rs
0.260
0.440
0.64++
0.83+++
Sig.
0.5309
0.2715
0.0856
0.0102
rs
0.40
0.230
0.260
-0.260
CE-45
Sig.
0.3199
0.5878
0.5309
0.5309
rs
DPO-45
Sig.
0.050
0.9108
0.020
0.9551
-0.380
0.3518
-0.69++
0.058
NCO-45
Sig.
-0.050
0.9087
00
1
0.340
0.4076
0.220
0.6013
0.83+++
0.0102
0.66++
0.0757
0.79+++
0.0208
0.520
0.1827
-0.550
0.16
-0.66++
0.0757
-0.86++++
0.0065
-0.9++++
0.002
rs
rs
PH-45
Sig.
rs
PS-45
Sig.
Correlations
parameters
CCO-1st-45
Sig.
CE-1st-45
Sig.
rs
rs
rs
DPO-1st-45
Sig.
NCO-1st-45
Sig.
PH-1st-45
Sig.
PS-1st-45
Sig.
rs
rs
rs
Project outcome
ESI
MCI
MHI
TIC
0.140
0.7358
0.460
0.2572
0.64++
0.0856
0.88++++
0.0039
0.120
0.7789
0.070
0.8657
-0.070
0.8665
-0.050
0.9108
0.69++
0.058
0.77+++
0.0265
0.550
0.16
0.40
0.3199
0.070
0.8657
0.130
0.7544
0.470
0.2433
0.23+++
0.028
0.76+++
0.028
0.83+++
0.0114
0.67++
0.071
0.570
0.5878
0.020
0.9554
0.320
0.4346
-0.330
0.4198
-0.14+
0.139
156
Correlations
parameters
rs
CCO-60
Sig.
CE-60
Sig.
DPO-60
Sig.
NCO-60
Sig.
PH-60
Sig.
rs
rs
rs
rs
rs
PS-60
Sig.
ESI
0.310
Project outcome
MCI
MHI
0.40
0.67++
TIC
0.74+++
0.4556
0.070
0.8657
-0.380
0.3518
0.3325
0.150
0.7219
-0.20
0.6287
0.071
0.59+
0.1262
-0.6+
0.1195
0.0366
0.360
0.3821
-0.67++
0.071
0.520
0.1827
0.88++++
0.0039
-0.490
0.2166
0.65++
0.0831
0.71++
0.0501
-0.190
0.6474
0.74+++
0.0366
0.83+++
0.0102
-0.420
0.3013
0.62+
0.1017
0.62+
0.1017
-0.50
0.2039
Correlations
parameters
rs
CCO-1st-60
Sig.
CE-1st-60
Sig.
DPO-1st-60
Sig.
rs
rs
rs
st
NCO-1 -60
Sig.
PH-1st-60
Sig.
PS-1st-60
Sig.
rs
rs
ESI
Project outcome
MCI
MHI
TIC
0.74+++
0.0366
0.8+++
0.0165
0.290
0.4927
0.310
0.4556
-0.190
0.6514
0.020
0.9551
0.020
0.9554
0.450
0.2604
-0.070
0.8665
0.010
0.9775
-0.10
0.8225
0.020
0.9554
0.440
0.2715
0.8+++
0.0183
0.66++
0.0757
0.8+++
0.0165
0.10
0.8225
0.140
0.7342
0.070
0.8665
0.450
0.2604
-0.550
0.16
-0.30
0.4713
0.10
0.8225
-0.020
0.9554
157
Early warning
indicator
Correlations
parameters
CCO-75
Sig.
CE-75
Sig.
DPO-75
Sig.
NCO-75
Sig.
PH-75
Sig.
rs
rs
rs
rs
rs
rs
PS-75
Sig.
ESI
Project outcome
MCI
MHI
TIC
0.310
0.4556
0.40
0.3325
0.67++
0.071
0.74+++
0.0366
0.140
0.7358
0.20
0.6287
0.64++
0.0856
0.380
0.3518
-0.360
0.3851
-0.20
0.6287
-0.64++
0.0856
0.50
0.058
0.62+
0.1017
0.83+++
0.0114
0.62+
0.1017
0.74+++
0.0366
0.74+++
0.0366
0.87++++
0.0045
0.57+
0.139
0.62+
0.1017
-0.480
0.2329
-0.140
0.7342
-0.62+
0.1017
-0.57+
0.139
Correlations
parameters
CCO-1st-75
Sig.
CE-1st-75
Sig.
DPO-1st-75
Sig.
rs
rs
rs
rs
NCO-1st-75
Sig.
PH-1st-75
Sig.
PS-1st-75
Sig.
rs
rs
ESI
Project outcome
MCI
MHI
TIC
0.480
0.2329
0.240
0.5678
-0.070
0.8665
0.020
0.9554
0.76+++
0.0274
0.66++
0.0758
0.460
0.2471
0.380
0.3505
0.020
0.9551
-0.160
0.7111
-0.410
0.3167
-0.280
0.5091
0.380
0.3518
0.63++
0.0909
00
1
0.450
0.2604
0.360
0.3851
0.78+++
0.0229
0.330
0.4198
0.430
0.2894
0.210
0.6103
0.410
0.3167
0.050
0.9108
0.330
0.4198
158