How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

Robert A. Zeigenfuse President Advanced Automation Associates, Inc. Exton, PA 19341

KEYWORDS
Conditional Bid, Decision Analysis Tool, Proposal, Risk Management, Control Systems Integrator

ABSTRACT
The review, evaluation and selection of multiple complex technical proposals can be a frustrating and daunting task. All too often this results in an emotional, gut-feel contract award, significantly increasing the risk for a disastrous outcome. This paper will outline the use of a decision analysis tool for a logical, fair and organized approach to evaluate proposals from Control System Integrators. This decision analysis tool will aid in establishing criteria, by first classifying these criteria by their relative importance. Then through outlined structured steps, evaluations and judgment are added and organized to reach a conclusion. Criteria will be suggested to aid in evaluating the content of the Control System Integrators proposal as well as their proposal execution performance. Finally, since no decision is perfect, this paper will show how to both identify risks and then manage them.

INTRODUCTION
As a client goes through the process of preparing an RFQ (Request for Quotation), then selecting two to four CSIs (Control Systems Integrators) to provide proposals, they are left with a most difficult task of reviewing, evaluating and selecting the winning bid from multiple complex technical proposals. A popular approach is to go right to the price and select the lowest bid since all bidders were qualified. The risk in this approach is associated with all the conditions, the basis, exceptions and terms stated in the bidders conditional bid. For those who find this risk unacceptable, this paper will outline the use of a decision analysis tool for a logical, fair, and organized approach to evaluate proposals from CSIs.

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

ANALYSIS TOOL
Kepner-Tregoe recognized that human beings could only juggle so many criteria and solutions in their head before getting confused and frustrated, all too often resulting in an emotional and gut-feel decision. This decision to eliminate ones pain is usually a poor one. So Kepner-Tregoe developed an analysis tool as an aid when making a decision with many complex criteria and multiple solutions to pick from. Like most successful tools, the beauty is in its simplicity. Kepner-Tregoe developed a decision analysis methodology that establishes decision criteria, classifying these criteria by their relative importance, then through structured steps, evaluations and judgments are added and organized to reach a conclusion all on one simple chart that could fit on one page or two at most.

STRUCTURED STEPS
The structured steps associated with the Kepner-Tregoe decision analysis tool are listed below with simple examples relative to selecting a CSIs bid: 1) State the decision: Write a short statement describing the intended result of a decision. Example: Select a CSI to implement your system. 2) Develop Criteria: Identify criteria that will influence the choice. Example Criteria: Services Exceptions Options 3) Classify criteria into "musts" (M) and "wants": Determine the role criteria will play in the decision Example Criteria: Services Exceptions Options

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

4) Weigh the "wants" (1-10): Assign relative values to "want" criteria. Example Criteria: Services Exceptions Options

9 M 6

5) Screen CSI through the "musts": Determine if CSI meets the "musts". Example: CSI A Acceptable Exceptions Go CSI B Acceptable Exceptions Go CSI C Unacceptable Exceptions No Go (CSI C is no longer considered.)

6) Evaluate and score CSI against the "wants": Evaluate relative performance of CSI (0-5). Example: CSI CSI A Performance Score

(9) Service (4) 9x4 = 36 (6) Options (3) 6x3 = 18 TOTAL SCORE = 54 CSI B (9) Service (3) 9x3 = 27 (6) Options (4) 6x4 = 24 TOTAL SCORE = 51

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

7) Assess risks for the client to manage: Every project involves risk to the client that the client must manage. Identify risks that need to be managed and evaluate your ability to manage them. Hint: Low performance score could indicate risk. Example: CSIs CSI A CSI B 8) Make decision: Select a CSI that has the best balance in meeting criteria (TOTAL SCORE) with acceptable and manageable risks.

Evaluate value of options. Manage missing service component.

PERFORMANCE SCORING OF CRITERIA

The user may define a scoring system that will help the decision- makers evaluate and score the CSIs in a fair and consistent manner. Since the range of criteria is so broad, any one scoring system will have its moments of questionable applicability to the criteria in question. The decision-maker(s) should use their best judgment and move on. To be consistent with the scoring system used in selecting the candidates, it is suggested to use 0-5 with 5 being the most favorable rating.

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

ANALYSIS CHART An analysis chart can be as simple as the one shown below. Criteria List MUSTS 1. Exceptions 2. 3. 4. 5. WANTS 1. Service 2. Options 3. 4. 5. 6. 7. TOTAL Percentage Weight GO or NO GO CSI 1 CSI 2 CSI 3

GO

GO

NO GO

1-10 9 (45) 6 (30) 4 (36) 3 (18) 3 (27) 4 (24) ---

(75) 100%

(54) 72%

(51) 68%

---

Appendix A shows an analysis chart with suggested criteria, suggested weights, and a fictitious evaluation/judgment of three CSIs. Appendix B is a blank chart in which one could fill-in its own criteria, weights and evaluation/judgments. DECISION AND RISK MANAGEMENT HINTS Tools are only as good as those that use them. If one picks criteria that heavily favors one aspect of a proposal and/or scores unfairly, this tool is being used to justify a pre-selected proposal. If the scores on two proposals are very close, youre in luck, since either one should do. However, you should pick the proposal and associated risks that you will be most comfortable in managing. Why manage risks that you are not comfortable with? Another option is to view the proposal process as a mini-project. If a CSI cant perform on a mini-project, how well would they do on the real one?

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

A simple chart can also be used to manage risk. Identify the risk item, weigh the risk from 0 to 10, assign a probability from 0 to 10, multiply together for a risk factor, state action plan to manage risk and back-up plan if risk comes to fruition. Sample risk management chart: Risk Item Critical Response Time "Live" Cutover Inexperience SCADA Inexperience Lack of Ongoing Support Weight Prob. 10 10 7 10 5 8 6 4 Risk Action Plan Factor 50 Prototype 80 42 40 Develop plan Early look and feel demo Training Plan Back-Up Plan Add independent controller Add 2nd shift resources Hire vendor expert Hire local CSI

CONCLUSION
After learning about this decision analysis tool, its difficult to imagine making a complex decision any other way! The logic is in the decision statement and decision criteria. The fairness is in the criteria all having the same relative impact adjusted by the weight factor and uniform scoring of each proposal against the criteria. The organization is in a simple visual chart that is a single reference of all the structured steps. Go ahead and give it a try. See if the resultant decisions dont provide an increase in the success that you currently enjoying. But, don't forget to manage the risks!

ACKNOWLEDGEMENTS
Sam Bernstine Kepner-Tregoe Phone No. 330-868-5045

REFERENCES
1. Control System Integrators' Association, "Proposal Review", Guide for Selecting & Working with a Control Systems Integrator, Volume 2, CSIA, Exton, PA 19341, 2000, Pages 9 - 15. 2. Reprinted with the permission of Kepner-Tregoe, Princeton, New Jersey. All rights reserved. Kepner-Tregoe, Inc. is an international management training and consulting firm. For more information about Kepner-Tregoe, visit www.kepner-tregoe.com.

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

APPENDIX A SAMPLE ANALYSIS CHART (Assumes CSI was Prequalified to Bid)
CRITERIA Price not over budget Meet schedule Unacceptable exceptions Unacceptable terms Proposal Performance User Musts "M" M M M M M M User Wants (1-10) Weight 10 9 7 9 10 8 8 7 10 9 8 6 5 4 7 8 9 10 6 8 Proposal A Performance (Go-No Go) GO GO GO GO GO Score Performance (0-5) Weighted Total Proposal B Performance (Go-No Go) GO GO GO GO GO Score Performance (0-5) Weighted Total Proposal C Performance (Go-No Go) GO GO NO GO GO NO GO

Proposal: Cost Commercial policies Terms and conditions Bid conditions/exceptions Delivery Project approach Project management Engineering services Programming services Documentation/deliverables Solution aspects Options Equipment supply Hardware/panel supply Installation services Commission services Training/ongoing support Project team/personnel Intangibles CSI learn curve Proposal Execution: Meet commitments Follow directions Attention to detail Comfort level TOTAL Max. Total (190 x 5) PERCENTAGE

4 4 5 4 5 5 2 4 4 3 5 5 3 4 5 3 5 4 4 10

(40) (36) (35) (36) (50) (40) (16) (28) (40) (27) (40) (30) (15) (16) (35) (24) (45) (40) (24) (80)

5 4 4 3 5 5 4 4 5 4 4 4 5 3 3 4 5 5 4 2

(50) (36) (28) (27) (50) (40) (32) (28) (50) (36) (32) (24) (25) (12) (21) (32) (45) (50) (24) (16)

NO GO equals disqualified -no further evaluation necessary

8 6 8 10 190 950 100%

5 4 3 4

(40) (24) (24) (40) 825 825 87%

3 3 2 3

(24) (18) (16) (30) 746 79%

How to Evaluate a Control Systems Integrators Proposal Using Kepner-Tregoe Analysis

APPENDIX B "BLANK" ANALYSIS CHART
CRITERIA User Proposal A Proposal B Proposal C Musts Performance Performance Performance "M" (Go-No Go) (Go-No Go) (Go-No Go) M M M M M M User Score Weighted Score Weighted Score Weighted Wants Performance Total Performance Total Performance Total (1-10) (0-5) (0-5) (0-5) Weight

TOTAL PERCENTAGE 100%