Expert Systems with Applications 39 (2012) 65036513

Contents lists available at SciVerse ScienceDirect

Expert Systems with Applications

journal homepage: www.elsevier.com/locate/eswa

Action research contextualizes DEA in a multi-organizational decision-making process

Muhittin Oral
University of the Thai Chamber of Commerce, Bangkok 10400, Thailand in University, Graduate School of Business, Altunizade, Istanbul 34662, Turkey zyeg

a r t i c l e

i n f o

a b s t r a c t
The theory of participatory action research (PAR) grew out of the practice of problem-solving in groups and organization by involving the participation of all pertinent stakeholders in decision-making process through empowerment. The ontological assumption is that the world out there is dened by the participating stakeholders as they understand and perceive it. The actionable knowledge thus produced is mostly used subjectively in the hope of a favorable organizational change or a transformation for betterment. On the other hand, Management Science/Operations Research (MS/OR) is more concerned with epistemological objectivity in identifying and dening managerial issues and nding solutions. In this paper we shall show that MS/OR can benet from and contribute to PAR in complex decision making contexts. This kind of mutual benet will be illustrated through a study reported in the area of R&D planning. It will be shown that PAR contextualizes problem structuring whereas MS/OR optimizes the consensual decision-making process in a multi-organizational context. 2011 Elsevier Ltd. All rights reserved.

Keywords: Participatory action research Enhanced Data Envelopment Analysis R&D project selection Multi-organizational decision making Multi-methodology

1. Introduction Decision making process is a complex phenomenon, especially when it relates to hundreds of stakeholders representing several organizations with different and sometimes opposing values and conicting interests. Such complex decision making processes, which are not rare in todays business environment due to business networking all around the world, invoke a series of issues to be dealt with: (1) identifying the stakeholders that relate to the decision making process, (2) how the reality is perceived and dened by the stakeholders ontology, (3) what kind of knowledge is to be produced and how epistemology, and (4) how the knowledge produced is to be used by the stakeholders axiology. We argue that ontological subjectivity of participative action research (PAR) can contribute to epistemological objectivity of Management Science/Operations Research (MS/OR) through its better and more legitimate problem structuring for action nature. The theory of PAR grew out of the practice of problem-solving in groups and organization by involving the participation of all pertinent stakeholders in decision-making process through consciousness and empowerment. The ontological assumption is that the world out there is something to be dened by the participating

Address: School of Business, University of the Thai Chamber of Commerce, Bangkok 10400, Thailand. Tel.: +90 533 818 2607. E-mail addresses: muhittin_ora@utcc.ac.th, Muhittin.Oral@ozyegin.edu.tr
0957-4174/$ - see front matter 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.12.054

stakeholders as they understand and perceive it. This distinguishing feature of PAR leads to a more relevant problem structuring from the perspective of the participating stakeholders (Dick, 2007). The actionable data, information, and knowledge thus produced are however mostly used subjectively in the hope of a favorable organizational change or a transformation for betterment. MS/OR is, on the other hand, more concerned with ontological and epistemological objectivity in identifying and dening managerial issues and nding solutions. MS/OR mostly favors, based on articles that have appeared in the literature (Mingers, 2003), both ontological and epistemological objectivity. In MS/OR studies, however, ontological issues are not usually made much of a concern because recorded or explicit data are assumed to dene them objectively. Therefore, emphasis is more on the production of knowledge or solution through objective epistemological assumptions. But explicit data or knowledge, without the tacit knowledge and preferences of stakeholders involved, can only explain a small part of the managerial or organizational reality. Because of this limitation, management scientists and operations researchers face with the issues of validity (Landry, Malouin, & Oral, 1983; Oral & Kettani, 1993) and legitimization (Landry, Banville, & Oral, 1996) in order to show that they are dealing with relevant issues and the knowledge they produce is actionable. This is in a sense an effort on the way claiming the oneness of theory and practice in MS/ OR (Van de Ven & Johnson, 2006). In the case of action research, the issues of validation and legitimization are somewhat dealt with implicitly through participation of the stakeholders. Through

6504

M. Oral / Expert Systems with Applications 39 (2012) 65036513

participation processes, subjectively dened reality (ontological subjectivity) and actionable knowledge produced (epistemological subjectivity) are in a sense legitimized simply because both are the outcomes of the stakeholders. However, the knowledge produced by participants in an action research process is situated-knowledge or local knowledge at best and cannot be generalized most of the time (Genat, 2009). In other words, the issue of epistemological validity for knowledge generalization remains as an important issue to be dealt with in action research, as attested by several action researchers themselves (Dick, Stringer, & Huxham, 2009; Gustavsen, 2003, 2008; Sommer, 2009). The issues and methods of legitimization of organizational and institutional change have been also a concern for organization researchers as well (Vaara & Tienari, 2008). In this paper we shall show that MS/OR can benet from and contribute to PAR in complex decision making contexts. MS/OR can contribute to PAR by producing epistemologically objective solutions whereas PAR can contribute to MS/OR by grounding problem denition in reality as well as serving as an instrument for verifying and legitimating the solutions offered by MS/OR. This kind of mutual benet will be illustrated through a study reported in the area of R&D planning. The R&D project data produced subjectively by PAR are objectively used by an MS/OR modeling process for evaluating and selecting R&D projects for funding. The MS/OR modeling process makes use of both self-efciency DEA and cross-efciency DEA models; which are together called Enhanced Data Envelopment Analysis, or shortly E-DEA (Oral, 2010). Following the two strategies for inductive reasoning in organizational research (Kettovi & Mantere, 2010), we employ PAR for the contextualization of decision making process, and MS/OR for the idealization through optimization of decision making itself. We argue that contextualization legitimates the reality construction or problem structuring of stakeholders (subjective ontology of PAR) whereas idealization through optimization, on the other hand, validates the knowledge produced about the constructed reality (objective epistemology of MS/OR). Through such a multi-methodological conceptualization we hope to make an objective sense of the subjective experience.

 Our decision making context relates to three integrated iron and steel complexes, located in different regions of a country. The combined number of engineers and managers run around thousands and total number of employees over ten thousands as a group of stakeholders.  A heuristic reasoning suggests that there is gap between the existing technology level and the desired or required technology level, a serious obstacle perceived to be on the way to meet the current and future demands. This perceived technological gap needs to be partially reduced by R&D activities. More specically, technological gap indicates what sort of R&D activities are to be conducted. Among these are: (1) overcoming technical difculties encountered in production units, (2) solving likely technological problems in the future, (3) improving the existing technology, and (4) developing new technologies.  There is a large network of stakeholders. (1) There are three integrated iron and steel complexes, each of which is an independent entity, administratively speaking. Each integrated complex necessitates the inclusion of more than hundred managers, engineers, and researchers as participants in dening and developing R&D projects, (2) a government planning ofce that decides and approves which R&D projects are to be sponsored in the iron and steel industry. Although they have the nal say in allocating government funds to the R&D project proposed, they act more as overseers of the R&D planning process, and (3) an outsider research team, called the core research team, that plays the role of professional planners to facilitate lu & Rav, the R&D planning process (Ackoff, 1977; Babrog 1992). This research team possesses the necessary background and experience related to iron and steel industry.  It is required that the stakeholders are to be included, not necessarily in all stages though, in the processes of identifying technological needs, formulating, developing, score assigning, evaluating, and selecting R&D projects for the iron and steel industry for government funding considerations. The primary objective of this paper is to meaningfully integrate, particularly with respect to the voice dimension of PAR (Chandler & Torbert, 2003), the relevance offered by PAR through its contextualization approach with the rigor of MS/OR with its idealization through optimization tradition within the framework of the decision making context that has been just briey discussed above.

2. Decision making context We describe a complex decision making context which is based on and inspired from an actual case study reported in the literature (Oral, Yetis , & Uygur, 1981). Although the decision making context we describe here is hypothetically constructed, it reects nevertheless the characteristics of an actual R&D planning process used for rst identifying the needs and concerns in an industry nationwide, then formulating a number of R&D projects for consideration, and nally score assigning, evaluating, and selecting a set of R&D projects for government funding. The decision making context relates to the iron and steel industry in a country and has the following major characteristics:  Any iron and steel industry is a very capital-intensive sector requiring large plant areas, involving numerous processes, using massive and complex equipment, and employing thousands of people. Iron and steel plants can be classied as, (1) non-integrated steelworks, where the plant comprises only rolling mills and nishing facilities, (2) semi-integrated steelworks, where steel making plant is installed in addition to rolling and milling facilities, and (3) integrated iron and steel complexes, which embraces all facilities, commencing with iron ore and coal as the two main materials and proceeding through rolling and nishing operations (Considine, 1978).

3. Contextualization through PAR The relevance of any research depends on how it is contextualized. Contextualization seeks to establish the relevance authenticity of reasoning. Given our decision making context, any reasoning is to be viewed as a context-dependent process, focused on arriving at what the stakeholders judge to be the best explanation for the R&D data gathered related to the decision making context. Three types of contextualization are identied (Kettovi & Mantere, 2010). The rst, subjective contextualization, is based on the premise that all stakeholders have personal backgrounds, value systems, and knowledge bases, which are reected in their reasoning to understand the world around them. The strategy of subjective contextualization is to shift the epistemological focus from knowledge to practice of knowing (Van de Ven & Johnson, 2006), a shift that is needed and required by any PAR. The primary objective of subjective contextualization is to narrow the gap between theory and practice. The second kind, empirical contextualization, is aimed at providing the stakeholders with maximal informational access to the empirical context. Empirical contextualization is an important and powerful tool in our decision making context for the stakeholders to see

M. Oral / Expert Systems with Applications 39 (2012) 65036513

6505

the whole picture better through beneting from the knowledge and experience of the stakeholders as much as possible, acknowledging the fact that they are familiar with the systems they are a part of. The third kind, theoretical contextualization, seeks warrants through establishing the relevance of claims with respect to a particular theory, knowledge, or thinking. Any organizational research is expected to contribute to a theoretical discourse (Kettovi & Mantere, 2010), so is assumed and claimed by those in PAR. These three types of contextualization are to be taken into consideration for a better understanding of the research issues that will make R&D project ndings relevant, usable, and useful. This is more so in the case of PAR. There are many approaches, both in theory and practice, to action research. The most frequently cited denition of action research is due to Reason and Bradbury (2008: 4) and they dene it as follows: Action research is a participatory process concerned with developing practical knowing in the pursuit of worthwhile human purposes. It seeks to bring together action and reection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the ourishing of individual persons and their communities. The above denition of action research suggests certain characteristics, principles, and assumptions that need to be taken into consideration while doing R&D planning:  Actionable knowledge is to be produced to deal with the pressing concerns to people or to organizations narrowing the perceived technological gap in our case.  The participants or the stakeholders are the ones who identify so called the pressing concerns or dene the reality ontological subjectivity.  The actionable knowledge is to be produced by the participants or stakeholders through a participative process epistemological subjectivity.  The actionable knowledge produced through participative process is to be put into practice to deal with the pressing concerns or problems axiological subjectivity.  The oneness of theory and practice is to be established. A conceptual typology of 27 different avors of action research, underpinned by three dimensions identies (Chandler & Torbert, 2003): (1) the voice dimension: rst-person subjective, second-person multiple subjective, and third-person generalized voice, (2) the practice dimension: rst-person, second-person, and third-person practice, and (3) the time dimension: past, present, and future. Chandler and Torbert (2003) argue that action research studies that include a greater proportion of the 27 types of modes are likely to account for more of the reality construction of managerial or organizational situations than does the traditional positivist approach, i.e., mostly model-based MS/OR. In this paper, we shall show how some of the 27 types of modes are used in the decision making context I described employing a model-based approach, that is, an MS/ OR method. The primary dimension of concern, however, will be that of voice. Regardless of the number of considerations that are included as part of a study, there are certain characteristics that are present in any action research (Chandler & Torbert, 2003). First, a primary purpose of PAR is to deal with practical organizational and managerial issues and concerns. So PAR is about working toward practical outcomes, and also about creating new forms of understanding and perception of the world around us. Second, PAR aims to pro-

duce actionable knowledge and knowing in order to deal with the practical issues and concerns of the organization. Third, as we search for actionable knowledge and freeing ways of knowing, working with stakeholders, we can also see that PAR is truly participative and transparent process. PAR is only possible with ideally involving all stakeholders both in the inquiring and sense-making processes (Weick, 1995) that shapes the very nature of the research to be conducted. Fourth, PAR is of the emancipatory nature in the sense that it leads not just to creation of new actionable knowledge, but also to development of new capabilities to create knowledge, hence contributes to the development of organization. Fifth, as a consequence of the previous four interdependent characteristics of PAR, organization is expected to ourish in terms of performance and ideals. Although the contextualization of the R&D planning process can be done according to the three dimension of PAR, in this paper we shall surface its voice dimension more profoundly in comparison with the other two. 3.1. Pragmatic rst-person voice: participatory action research The pragmatic rst-person voice assures that each and every stakeholder has a say as to the nature of things being studied or investigated and the action to be taken. In a sense, all stakeholders have individually an inuence on the construction and denition of their common reality and possible courses of action pertaining to the decision making context of interest. In the context of R&D planning process, this statement translates itself into:  Managers, engineers, and researchers of all three iron and steel complexes are to be called direct stakeholders. They identify those needs that will contribute to narrowing the gap between the perceived desired or required technology and the existing levels in their own departments or units.  The direct stakeholders convert the needs into a tentative set of R&D projects and propose them for funding consideration. Each R&D project proposed has a title, scope, methodology, resources required, and likely contributions if funded.  The direct stakeholders suggest a set of criteria according to which scores can be assigned to the proposed R&D projects. This set of criteria is to be nalized later when the pragmatic secondperson voice is activated.  The core research team employs its pragmatic rst-person voice in facilitating and guiding the direct stakeholders in identifying technological needs that can be translated into a set of R&D projects. In this sense, the core research team is an insider (Poonamallee, 2009). Through the pragmatic rst-person voice step above the direct stakeholders and the core research team contextualize the nature of the decision making situation by soliciting and seeking rst hand contributions from the interested and informed stakeholders on individual, unit and departmental levels (see Fig. 1). 3.2. Pragmatic second-person voice participatory action research The pragmatic rst-person voice is necessary but not sufcient in decision making contexts where transparency and democracy are at play and are important inevitable issues. The pragmatic rst-person voice is to be complemented by pragmatic second-person voices. Through pragmatic second-person voices, each stakeholder has a say as to the nature of says proclaimed by the pragmatic rst-person voices. In other words, ones own idea is crucial, but the others ideas of ones idea are also equally important and are to be taken into consideration. In the context of our

6506

M. Oral / Expert Systems with Applications 39 (2012) 65036513

PRAGMATIC FIRST-PERSON VOICE : PAR

The core research team studies iron & steel industry from written documents, The core research team interviews direct stakekholders managers and engineers Each and every direct stakeholder defines, develops, and justifies R&D projects, Direct stakeholders propose a set of criteria to be used in valuating R&D projects. OUTPUT: A tentative set of R&D projects for consideration.

PRAGMATIC SECOND-PERSON VOICE : PAR

Each and every stakeholder examines the R&D projects proposed by the others, Each and every stakeholder assigns scores to the R&D projects proposed by the others, Scores to the R&D projects are assigned with respect to each criterion in the set of criteria previously decided. OUTPUT: A tentative set fo R&D projects for consideration with their scores.

PRAGMATIC THIRD-PERSON VOICE : PAR

The core research ream studies the tentative set of R&D projects for consistency, relevance, usefulness, and usuability, The government planning office varifies the R&D projects in terms of resource requirements from the perspective budgetary constraints. OUTPUT: A definite set of R&D Projects for funding considertation.
Fig. 1. The voice dimension of PAR.

R&D planning process, the pragmatic second-person voice is to be activated in the following manner:  The direct stakeholders examine, question, criticize, and contribute to the tentative set of R&D projects proposed by all participants.  The direct stakeholders decide on the set of criteria to be used in assigning scores to each R&D project proposed.  Every member of the direct stakeholders assigns scores to each and every R&D project with respect to each criterion in the set.  The core research team facilitates the score assigning sessions to assure their outputs are obtained according to the principles of PAR. In these sessions, the core research team play the role of an outsider (Poonamallee, 2009).  At the end of the pragmatic second-person voice practice, the tentative set of proposed R&D projects are examined, modied, and valued by assigning scores with respect to the set of criteria that is decided by the direct stakeholders. 3.3. Pragmatic third-person voice: participatory action research The pragmatic third-person voice is to nalize and formalize the denite set of R&D projects that will be considered for government funding. The primary objective is to re-examine each and every R&D project from an independent perspective that also reects the views of non-direct stakeholders in terms of the set of criteria to be used, scoring system employed, resources required, and likely contributions of each R&D project if funded. In the context of our decision making context, this statement corresponds to:  The core research team veries and assures that each R&D project is what it is claimed to be in terms of resources demanded and contributions envisaged. The core research team here again plays the role of an outsider (Poonamallee, 2009).  The core research team conrms with the government planning ofce the denite set of R&D projects to be considered for government funding. The output of the pragmatic third-person voice practice is a set of R&D projects, denoted by P = {1, 2, . . . , N}, where N is the total

number of R&D projects under consideration. Every R&D project has scores with respect to each criterion. Let Sik be the score assigned to R&D project-i with respect to criterion k, where k = 1, 2, . . . , K, and K is the number of criteria being used. Every R&D project has utility or value as a function of the scores assigned to it. Let the utility or contributive value of R&D project-i be denoted by Vi = Pi(Si1, Si2, . . . , SiK), where the function Pi maps the scores of R&D project-i onto a real number. Had we have such a convenient mapping or function readily available it would have been of course a much easier task to evaluate and select a set of projects for government funding. But such a mapping is not conveniently in sight and therefore needs to be developed. This is where PAR gives its way to MS/OR for a formal treatment of evaluation and selection of R&D projects in general and the determination of the form of function Pi in particular. 4. Idealization through optimization of MS/OR Idealization is dened as one of the two strategies for inductive reasoning in organizational research and it is based on the classical epistemic virtues of objectivity and truth (Kettovi & Mantere, 2010). The term idealization in our decision making context, however, refers to a process where the voice dimension is optimally represented (therefore idealization through optimization henceforth), certain resentment avoidance principles are complied with, budget constraints are respected, and a highest level of consensus possible is achieved. The idealization strategy, in our case too, as stated by Kettovi and Mantere (2010), will be supported by normative epistemology and methodology of MS/OR. 4.1. Idealized through optimization rst-person voice: Self-efciency DEA model The rst step in idealization through optimization is its connection with the MS/OR version of rst-person voice. The proposers or owners of R&D project-i should have a direct rst say in the way their project is being evaluated. In our notation, this correspondence to the determination of the form of the function Pi in Vi = Pi(Si1, Si2, . . . , SiK). To suggest a form for the function Pi we rst make some observations:

M. Oral / Expert Systems with Applications 39 (2012) 65036513

6507

 For R&D projects, the set of criteria can be divided into two mutually exclusive and totally exhaustive sub-sets: (1) inputrelated criteria, and (2) output-related criteria. Any R&D project requires inputs to produce outputs or contributions. Therefore it is most natural to consider these two types of criteria in this case. For instance, one can consider direct economic, indirect economic, technical, scientic, and social contributions as output-related criteria whereas budget required as inputrelated criterion (Oral et al., 1981). To be consistent with this portioning of the criteria set into two, let the scores (Si1, Si2, . . . , SiK) of R&D project-i are re-organized to have (Oi1, . . . , Oik) as the output-related scores and (Ii1, . . . , Iim) as the input-related scores.  In the presence of both input-related and output-related criteria, an R&D project can be considered as a system which produces outputs using inputs. This observation suggests that the performance of an R&D project can be thought of as efciency and can be expressed as a ratio of total outputs to total inputs. In this case, we can write

P r wir Oir Ei Maximize P ; c v ic I ic

subject to

P r wir Ojr P 6 1 for j 1; 2; . . . ; N; c v ic Ijr wir P 0;

3 4

8r and

v ic P 0; 8c:

P r wir Oir V i Pi Si1 ; . . . ; SiK P c v ic Iic

as the performance or efciency of R&D project-i. Note that Oirs and Iics in (1) are the data produced by PAR, whereas wirs and vics are the coefcients of which the values are to be determined, one way or another. But who is going to determine them and how?  The rst-person voice dimension of PAR requires that the values of wirs and vics are to be determined by the proposers or owners of R&D project-i themselves. We assume that the owners of R&D project-i propose that project because they see a value in it and therefore they would like to have it evaluated in the most favorable way possible. A way of realizing such a favorable evaluation is to let the owners determine the values of wirs and vics themselves.  When the owners of R&D project-i are given the right to determine the values of wirsand vics, we assume that they will do so by choosing those values that will make

 The optimization model described in (2)(4) is nothing but the conventional self-efciency DEA model (Charnes, Cooper, & Rhodes, 1978; Cook & Seiford, 2009) and is the idealized through optimization rst-person voice in the context of our decision making process for at least two reasons: (1) It is idealized through optimization in the sense that it involves a simplication of a complex phenomenon in an attempt to make it tractable (Kettovi & Mantere, 2010), and (2) it is the best advocate for the owners of R&D projects since the model provides those values of wirs and vics that are the most favorable. The owners themselves of R&D projects could nd the favorable values wirs and vics by a trial and error method in the absence of the formal formulation in (2)(4), but it could be very long, cumbersome, and possibly intractable.  The right given to R&D project-i in determining its own values of wirs and vics should be accorded to all other R&D projects in the same way. In a sense, this is a particular way of assuring the practice of democratic rights in evaluating R&D projects and we call it rst-level democratic right. Then the idealized through optimization rst-person voice model becomes, now in more precise notations, as:

P r wir Oir Eii Maximize P ; c v ic I ic

subject to

P r wir Ojr P 6 1 for j 1; 2; . . . ; N; c v ic I jr wir P 0;

3 4

P r wir Oir V i Pi Si1 ; . . . ; SiK P c v ic Iic

as large as possible. The meaning of this self-evaluation is to nd those values of wirs and vics that maximizes the performance or efciency Ei of R&D project-i; that is,

8r and

v ic P 0; 8c:

P r wir Oir Ei Maximize V i Maximize P : c v ic Iic

It should be noted here that the numerical values of wirs and vics also reect, in a sense, the value system or preference system of the owners of R&D project-i. This is so simply because the coefcients wirs and vics are the weights assigned or importance to be attached to outputs and inputs, respectively. This concept of value system will surface again in the case of second-person voice practice in the next section.  The same right is to be given not only to the owners of R&D project-i, but to all other R&D projects in order to create a decision making process where transparency and stakeholder participation is democratically assured. Then we have

 The meaning of the normalization constraint in (3) is that while R&D project-i is performing self-evaluation, none of the other R&D project can have an efciency score greater than 1 with the values wirs and vics of R&D project-i. Such a normalization makes the comparison much easier once we know that the value of Eii = 1.0 implies that R&D project-i is efcient; and Eii < 1.0 implies that it is inefcient.  The idealized through optimization rst-person voice model above can be also considered as a reliable, devoted and trustworthy partner of the owners of R&D project-i since it provides a computational process that offers the most favorable result.  In the conventional DEA terminology, maximum Vii is called efciency and denoted by Eii. We follow the same tradition henceforth. Through the idealized through optimization rst-person voice model we obtain the diagonal elements Eii of a matrix E = [Eij], where Eij = the relative performance of R&D project-j as perceived by the owners of R&D project-i and i, j = 1, 2, . . . , N. The values of Eijs are in fact the optimal and idealized deliberations of second-person voice (see Fig. 2). 4.2. Idealized through optimization second-person voice: Crossefciency DEA model The second-person voice of PAR addresses itself to our ability to inquire face-to-face with others into issues of mutual concern

P r wir Oir Ei Maximize P c v ic Iic

for i 1; 2; . . . ; N:

 R&D project-i is to be eventually evaluated relative to all other R&D projects. Therefore there is need for a normalization process in some mathematical sense so that R&D projects can be compared with one another easily. The most favorable relative performance of R&D project-i then becomes

6508

M. Oral / Expert Systems with Applications 39 (2012) 65036513

OPTIMIZED FIRST-PERSON VOICE : MS/OR

Optimal evaluation of each R&D project through a self-efficiency DEA model, Self-efficiency DEA model uses the score data obtained through PAR, Self-efficiency DEA model produces relative efficiency values for each and every R&D project under consideration. OUTPUT: Self-evaluation values for each and everry R&D projects under consideration. The diagonal elements of matrix E.

OPTIMIZED SECOND-PERSON VOICE : MS/OR

Optimal evaluation of each R&D project through a cross-efficiency DEA model, Cross-efficiency DEA model uses the score data obtained through PAR, Cross-efficiency DEA model produces relative efficiency values of each R&D project from the perspective of all other R&D projects, OUTPUT: Cross-evaluation values for each and every R&D projects from the perspective of other R&D projects. The off-diagonal elements of matrix E..

OPTIMIZED THIRD-PERSON VOICE : MS/OR

Resentment avoidance principles are introduced and respected, A non-DEA model that fully utilizes the information content of matrix E, Consensus level is maximized through the non-DEA model OUTPUT: The selected set of R&D projects for government funding. .
Fig. 2. The corresponding voice dimension in MS/OR.

(Reason & Bradbury, 2008). In our R&D planning context, this translates itself into a situation where each and every R&D project assesses one another. When an R&D project assesses another R&D project, we assume the existence of the following properties:  Any R&D project will maintain, we assume, its value system while assessing the other R&D projects. We, as human beings, do not easily and frequently change our value systems. This feature is most natural and sensible and in alignment with PAR approach. Particularly in the context of PAR, the stakeholders or participants will make use of their mental models, which also include their value system (Senge, 1990), and becomes operative while assessing others. In our context of R&D planning, however, the value system of an R&D project is assumed to be summarized in and represented by the coefcients wirs and vics that optimize its own efciency. In other words, an R&D project will use those coefcients in assessing others that will keep its own efciency level unchanged.  The self-efciency DEA model in (2)(4) might produce several sets of optimal coefcients, which is the case most of the time, if not all the time (Liang, Cook, & Zhu, 2008). In the presence of several sets of optimal coefcients wirs and vics, one needs to decide which set to use in assessing which R&D project. We opt for the philosophy of appreciative inquiry. Appreciative inquiry is a philosophy that incorporates an approach for engaging stakeholders or participants to produce positive change or results (Cooperrider, Whitney, & Stavros, 2008). This approach translates itself into a practice of using the set that gives the best result for the R&D project that is being evaluated. More precisely, let X i = {Xi1, Xi2, . . . , XiT} be the set of T number of optimal solutions obtained from the self-efciency DEA model corresponding to R&D project-i, where Xit = {wirt, "r, and, vict, "c}, t = 1, 2, . . . , T. Which optimal solution of the set Xiis to be used in evaluation R&D project-j? Appreciative approach requires that the set that gives the best result for R&D project-j is to be used. Let Xij 2 Xi be the optimal set that gives the best result for R&D project-j. Then one must use only Xij 2 Xi, not any other optimal set, in calculating the cross-efciency Eij of R&D project-j from the perspective of R&D project-i.

An idealized through optimization second-person voice model that reects the above characteristics can be, for any given pair of i j, "i, j, formulated as:

P r wijr Ojr Eij Maximize P ; c v ijc Ijc

subject to

P r wijr Oir P Eii ; v ijr Iic Pc r wijr Otr P 6 1; 8t; c v ijc Itc wijr P 0 and

6 7 8

v ijc P 0; 8r; c:

 The model in (5)(8) is called cross-efciency DEA model and corresponds to idealized through optimization second-person voice (Oral, Kettani, & Lang, 1991). The constraint in (6) warrants that the self-efciency score Eii of R&D project-i obtained from the idealized rst-person voice model is maintained as unchanged in the process.  The cross-efciency DEA model leads to the exercise of another type of democratic right, which might be called second level democratic right, since every R&D project is permitted to evaluate all others.  The cross-efciency scores Eij, "i j and the self-efciency scores Eii, "i form a matrix called E = [Eij]. Matrix E includes the evaluation scores resulting from the exercise of the rst level democratic right, Eiis, and the second level democratic right, Eijs. Matrix E is called Enhanced DEA for its rich information content (Oral, 2010). The task is now to make a complete use of matrix E, thus putting the rst and second level democratic rights into practice, in selecting a set of R&D projects for government funding. Before this however a couple of comments are in order with respect to the cross-efciency DEA model and the cross-efciency scores obtained from it. First, the cross-efciency scores obtained from the DEA model in (5)(8) are also a better advocate for those R&D projects being

M. Oral / Expert Systems with Applications 39 (2012) 65036513

6509

evaluated by other R&D projects. This formulation is in alignment both with appreciative approach (Dick, 2009) and the very spirit of DEA (Cook & Seiford, 2009). The concept of cross-efciency has appeared in the DEA literature since 1986 (Doyle & Green, 1994; Green, Doyle, & Cook, 1996; Liang et al., 2008; Sexton, Silkman, & Hogan, 1986; Wu, Liang, & Ying-chun, 2008). In the DEA literature, however, the concept of cross-efciency is not in the sense formulated in (5)(8), but is based on central tendency measures, mostly on an average. More precisely, using our notation, cross-efciency score is given by

Ej

n 1X Eij ; N i1

where the cross-efciency scores Eijs are not obtained from a model like the one in (5)(8), but found by simply substituting the optimal solutions obtained from the self-efciency DEA in the constraints (3). There are at least four pitfalls of using the Expression (3) or Expression (9), rather than the cross-efciency DEA model given in (5)(8), from the perspectives of both PAR and DEA: 1. Ambiguity: The non-uniqueness of the DEA optimal coefcients obtained from the self-efciency DEA model creates a confusion in applying expression (9) as to which set of the optimal coefcients of R&D project-i is to be used in nding the cross-efciency score for R&D project-j. Which of these non-unique multiple optimal solutions then will be used in nding the cross-efciency for R&D project-j? Is it the same optimal solution to be used for every R&D project or the one that gives the best result for R&D project-j? If it is the same optimal solution used for all other R&D projects, then one is against the philosophy of appreciative approach (Dick, 2009) by not giving the best chance. The basic idea of giving the benet of doubt of DEA to every R&D project is also violated if one uses the same optimal solution for all R&D projects. This kind of ambiguity is resolved through a formulation as given in (5)(8). 2. Aggregation: PAR differs from most approaches in the sense that majority of social and management science studies are aimed at aggregating data about many individuals, organizations or events irrespective of the particulars (Chandler & Torbert, 2003). Expression in (9) is a particular way of aggregating the cross-efciency scoresEij. Rather than using every single cross-efciency score in decision making process, one is left with a simple average. Therefore, it is not a procedure favored because every voice in PAR tradition matters and counts. It is also against the very idea of DEA which relies on best practices rather than on the one that is only an average. The very philosophy of DEA is to work with a concept related to efciency frontier, not with average or any other central tendency measure. This fact has also been recognized by some of the users of cross-efciency concept as formulated in Expression (9) and they tried to remedy the weakness of that expression (Doyle & Green, 1994; Liang et al., 2008; Wu et al., 2008). 3. Loss of information: Another way interpreting expression (9) is that R&D projects do not really matter as individual projects or units, only their average counts, rather a very limiting way of considering R&D projects in the process of evaluating and selecting for funding. Moreover, aggregating through Expression (9) causes another type of pitfall: loss of information, in addition to not exercising the second level democratic right. Rather than using the entirety of the matrix E, which has N2 elements, Expression (9) reduces this number to2N, N number of averages as given by Expression (9) plus N number of selfefciency scores obtained from the rst-person voice DEA model, thus reducing the use of information considerably as N

increases. Perhaps it is a good idea to dene a measure called degrees of lost information. Let us dene it as the number of elements of matrix E not used in decision making. Then the consequence of using Expression (9) is N2 2N = N(N 2) as the degrees of lost information by making a small calculation to observe its consequences more clearly. For instance, if N = 50 then the size of lost information is N(N 2) = 2,400, which corresponds to 98% of the total pieces of information available 2,500. 4. Deducibility and matrix E: As a corollary to the previous paragraph, we consider the notion of the best explanation, a notion that is important in deciding which set of information is to be used in selecting R&D projects. Let us consider our matrix E and the set F, the elements of which are composed of the diagonal elements Eiis of matrix E and Ej s obtained from Expression (9). Let DE be the selected set of R&D projects if matrix E is used, and DF be the set if the set F is used. According to the concepts of inference to the best explanation (IBE for short), accept DE given the evidence matrix E which has a much more information content than that of F (Niiniluoto, 1999). The above discussions and arguments suggest that we need to use the entirety of the information included in matrix E, thus putting both rst and second level democratic rights into practice, in evaluating and selecting R&D project for funding. This is the rst requirement from the perspective of both PAR and DEA. The second one is complying with some resentment avoidance principles so that R&D projects not selected will not question the fairness and rightness of those that are selected for funding. The third requirement is that the total budget needed for the selected R&D projects will not exceed the available budget. And the last and the fourth requirement is that the selection process is to be realized with a highest consensus level possible. Such a selection procedure is given below.

4.3. Idealized through optimization third-person voice: Consensus optimizing model The third-person voice model, which is a non-DEA model but uses the results of the previous DEA models, will be explained according to the four requirements discussed above, More specically, we shall be (1) using the entirety of the matrix E, (2) achieving a highest level of consensus possible, (3) two resentment avoidance principles, and (4) respecting budgetary constraint. Although different models can be considered or thought for this purpose, we follow the concepts of a particular modeling process appeared in the R&D literature (Oral et al., 1991).

4.3.1. Entirety of matrix E The entirety of matrix E is converted into another matrix called P concordance matrix C = jCijj, where C ij 1=n k /ijk and /ijk = 1 if Eik P Ejk and /ijk = 0 otherwise. With these denitions,Cij is then the ratio of the superiority of R&D project-i over R&D project-j as perceived by all the R&D projects under consideration for government funding. For example, if Cij = 0.75, then according to 75% of the R&D projects, R&D project-i is superior to R&D project-j. Or, in multiple criteria analysis terminology (Roy, 1985), R&D project-i outranks R&D project-j at the concordance level of 0.75. In a sense, the matrix C provides pair wise comparisons between the R&D projects. Now the question is what minimum level of concordance one would like to accept for selecting R&D projects? This question leads to denition of consensus level, denoted by h. For a comparative analysis of DEA with as a discrete alternative multiple criteria decision tool the reader is referred to Sarkis (2000).

6510

M. Oral / Expert Systems with Applications 39 (2012) 65036513

4.3.2. Consensus level R&D project-i is said to outrank R&D project-j at the consensus level h if Cij P h. If R&D project-i outranks R&D project-j then we dene an indicator variable aij(h) as aij(h) = 1 if Cij P h, and aij(h) = 0 otherwise. This denition permits us to identify the pairs of R&D projects between which there is an outranking relationship at the consensus level of h. In other words, for a given value of h, aij(h) s completely determine all the existing outranking relationships between the R&D projects. The totality of these outranking relationships is given by the following expressions:

nization proposing these projects. In order to avoid possible resentments we need to introduce some principles by which selection can be made with as little as resentment possible. 4.3.3. Resentment avoidance principles The transparency principle of PAR and consensual decision making context requires that we must obey some rules that are enforced or suggested by participating stakeholders. In our R&D selection process we shall comply with two resentment avoiding principles that were proposed by Roy (1985): external consistency and internal consistency. 4.3.3.1. External consistency. The set of rejected R&D projects should include those R&D projects each of which is outranked by at least one of the selected R&D projects. This principle implies that an R&D project that does not have a dominant competitor should be given the benet of doubt. It also asserts that an R&D project being outranked, as long as this outranking is not originated from a selected R&D project, is not a reason for its exclusion. As an example consider the following situation: There are six R&D projects under consideration for funding. At the consensus level of h assume that they have the following outranking relationships (see the left side of Fig. 3): R&D project A outranks R&D project D; R&D project B outranks both R&D project D and E; R&D project E outranks R&D project D and F; and R&D project D outranks R&D project C. Suppose we select R&D project A and B for funding and reject the others. Such a selection violates the external consistency principle because R&D project F is not outranked by either the selected R&D project A or B. R&D project F will resent the selection, for it is not outranked by any of the selected R&D projects. It is true that R&D project F does not outrank any project either in the set, but this does not matter, for R&D project F will always compare itself with the ones selected, not with the ones not selected. Although R&D project F is outranked by R&D project E, the owners of R&D project F will still question the selection process because their project is not going to be funded; a kind of bounded rationality (Simon, 1991). So why is not R&D project F in the selected group? Our formulation should avoid the occurrences of such selections.

h aij h 6 C ij 1; h aij h P C ij e;

8i ; j ; 8i; j;

10 i j; 11

where e is a sufciently small positive number, used to actually enforce a strict inequality (Oral et al., 1991). Since h could take on values only in the discrete set of {0, 1/N, 2/N, . . . , N/N} it is readily veried that any value in the interval of (0, 1/N) is appropriate value for e. The important point behind the above expressions is that outranking relationships between R&D projects can be analytically formulated and analyzed. Such a mathematical formulation implies the transparency principle of PAR and the objectivity of MS are maintained simultaneously. A couple of points are to be observed here: (1) A higher level of consensus will tend to decrease the number of outrankings; that is, the occurrence of aij(h) = 1s will be smaller as the value of h increases. (2) Outranking preserved for some value of h are not transitive all the time. For instance, the superiority of R&D project-j over R&D project-k and the superiority of R&D project-k over R&D project-i do not necessarily imply the superiority of R&D project-j over R&D project-i. In a sense, some presence of bounded rationality (Simon, 1991) is acknowledged in such a process. Therefore, it is assumed that there does not exist a simple ordering of R&D projects on which a selection process can be based. Admitting the existence of a non-transitive outranking relationship, a form of bounded rationality, is to assume that the selection process cannot be perfectly rational. In the presence of many stakeholders one might face such a non-rational situation, as the case in most PAR studies. Any selection that is not perfectly rational might create, on the other hand, resentments between R&D projects or orga-

Fig. 3. Violation of external and internal consistency rules.

M. Oral / Expert Systems with Applications 39 (2012) 65036513

6511

4.3.3.2. Internal consistency. The internal consistency principle dictates that the set of selected R&D projects should include only those R&D projects that are not outranked by any other selected R&D project. This principle implies that the set of selected R&D projects should not include any R&D project that is outranked by another R&D project in it. The violation of this principle might mitigate certain kinds of resentment. As an example, consider the following situation. There are ve R&D projects under consideration for funding consideration. At a certain of consensus level h, suppose that the outranking relationships between R&D projects are as given in the right side of Fig. 3. R&D project outranks R&D projects B, D, and E. R&D project B outranks R&D projects C and D. Again assume that R&D projects A and B are selected for funding and the others are rejected. This is a violation of the internal consistency principle because selected R&D project B is outranked by selected R&D project A. The stakeholders of R&D project E will bring the following legitimate point: Why is R&D project B selected and not R&D project E, given the fact that they are both in the same position; that is, both being outranked by the same selected R&D project A. There is no satisfactory response to this argument and this situation becomes a source of resentment, and therefore to be avoided. The above verbal and pictorial descriptions of the internal and external consistencies can be mathematically expressed as

X
j

bj bj 6 B;

14

where bj is the budget required for the realization of R&D project-j and B is the total budget available for funding. Putting all these formulations together we obtain the idealized through optimization third-person voice model that leads to an optimal R&D project selection model with the highest consensus level as follows:

Maxh;
subject to

h aij h 6 C ij 1;

8i; j;

i j; i j;

10 11 12 13 14 15

h aij h P C ij e; 8i; j; X aij hbi bj P 1; 8j; X X

j ij ij

aij hbi N 1bj 6 N 1; 8j;

bj bj 6 B;

bj 0; 1;

8j :

X X
ij ij

aij hbi bj P 1; 8j; aij hbi N 1bj 6 N 1; 8j;

12 13

where bi = 1 if R&D project-i is selected for funding, and bi = 0 otherwise (Oral et al., 1991). The Expressions (12) and (13) assure that R&D project selection is done without violating the external and internal consistencies at the consensus level of h. Because of these two explicit expressions, the transparency requirement of PAR is maintained in the election process. Moreover, the rigor and objectivity of MS/OR is also assured through a third-person voice. 4.3.4. Budgetary constraint The budget needed for the realization of the selected R&D projects cannot exceed the available budget. The mathematical expression of this constraint is rather straightforward:

The way the voice dimension of PAR is reected through E-DEA and non-DEA models and their interdependence are given in a summary form in Fig. 4. The idealized through optimization third-person voice model needs to be used repeatedly or in iterative manner until the available budget B for funding R&D projects is exhausted. Looking at the model gives the impression that it is a complicated one to get solutions from. This is due to the fact that there are two sets of quadratic 01 integer expression as constraints

X
ij

aij hbi bj P 1; 8j and

6 N 1;

X
ij

aij hbi N 1bj

8j

in the model and these expressions need to be converted into an equivalent sets of linear expression through a linearization method (Oral & Kettani, 1992). This computationally difcult problem can in fact be resolved easily by observing the possible values h can take on. Although we do not know the optimal value of h, we do know

Fig. 4. Oneness theory of PAR and MS/OR.

6512

M. Oral / Expert Systems with Applications 39 (2012) 65036513

however which values it can take on in the context of our R&D project selection process. We observe that h can take on values only in the discrete set of {0, 1/N, 2/N, . . . , N/N} because of the outranking possibilities that exist between N number of R&D projects. Note that once we know the value of h, we also know the value of aij(h) as either 1 or 0. Then the procedure to be used in R&D project selection becomes rather straightforward. First, set h = 1and nd the corresponding aij(h)s. Then substitute these values of aij(h) in the constraints of the selection model. Any feasible solution in bjs is an optimal solution. If there is no feasible solution then set h = (N 1)/N, the next maximum value of h. And repeat the same procedure until a feasible solution is found. Even a simple spreadsheet model developed at the excel platform will sufce to nd the optimal solution giving the highest consensus level.

5. A theory of oneness for PAR and MS/OR One can consider different kinds of oneness. Oneness of mind and body as in Eastern philosophy, and now in neuroscience; oneness of individual and society as in Japanese social fabrics; oneness of theory and practice or knowledge and action as sought by many disciplines, including both PAR and MS/OR; oneness of nature and creatures in biology are some examples that are put forward in the hope of avoiding the inherited dualism in the Western way of thinking (Mead, 1962). As the last point of this paper, we would like to claim that oneness of ontology and epistemology in understanding and dealing managerial issues in organizations can contribute to the oneness of knowledge and action or theory and practice. We claim that the way PAR and MS/OR has been conated in the previous sections constitutes a theory of oneness in the sense that one without the other is less meaningful and useful. PAR is struggling for academic recognition from conventional scholars whereas MS/OR is trying to convince others for its relevance to organizations and communities. PAR is relevant but claimed to be lacking rigor, and MS/OR is rigorous but its relevance is challenged. Both are trying to eliminate their weaknesses through different methodological improvements, but mostly staying in their own domains, without much crossing of the boundaries of other disciplines. The oneness theory of PAR and MS/OR, as can be observed from Fig. 4, constructs a synergy, not an opposition, between PAR and MS/OR. According to this theory, PAR contextualizes the issues and produces data, information, and knowledge; and MS/OR uses the outputs of PAR in an idealized through optimization process in a multi-organizational decision making context. In this theory, MS/OR cannot exist without PAR, if relevance is an important consideration. MS/OR, on the other hand, complements and contributes to PAR by producing actionable knowledge that is epistemologically objective, yet in the very spirit of PAR with respect to its voice dimension. There is a considerable body of literature that strongly supports the need for paradigmatic improvements including ontological, epistemological, axiological, and methodological assumptions, both in PAR and MS/OR elds. On the side of PAR, the interplay between practical challenges and theoretical discourse has been always a point of departure for research. PAR researchers claim that an interplay can exist when action research works within the same practical context where changes in theoretical orientation occurs. Although claimed as a form social research, action research generally has difculty gaining academic acceptance, partly because it is seen by many conventional academics as an esoteric kind of research (Gustavsen, 2008). The importance of explicit theory building and testing as an integral part of action research practice has been advocated by several (Friedman & Rogers, 2009; Genat, 2009). As related to pragmatism, action research may have

enjoyed advances and even some elements of prestige in some academic circles, though the overall story of action research is far from one of undisputed success (Greenwood, 2002). However, there are articles which claim that action research is scientic because it has its foundations in open systems thinking that relates to the scientic method under certain conditions (Barton, Stephens, & Haslett, 2009). There has been also the suggestion that action research methodology can be enriched by using the concepts of normative lu & Rav, 1992). The primary concern has been planning (Babrog always to improve the status of actions research with its own philosophy of research remaining mostly intact. On the side of MS/OR, the articles questioning the relevance of MS/OR are more than abundant (Kirby, 2007; Silverman, 1994). To remedy the weaknesses of MS/OR in this respect, researchers in the eld have developed many methods to structure a relevant problem in a variety of managerial situations or messes. One can even claim that there is a sort of well established subeld now called problem structuring methods, or shortly PSM. The main objective of PSM is to ground problem denition in reality as much as possible to claim its relevance. The most established approach is the one called soft systems methodology (SSM) and has been widely in use since its inception almost 30 yr ago (Checkland, 1981; Checkland & Scholes, 1990; Checkland & Winter, 2006; Reisman & Oral, 2005). It is interesting that SSM has emerged as a methodology in the course of action research activities. Nevertheless the primary objective of SSM is to make the classical paradigm of MS/OR and SSM to as complementary to one another to the extent they are more powerful in combination than they are alone. In addition to different versions of unifying hard and soft MS/OR, there has been considerable effort to legitimate the integration of different methods under the name of multi-methodology. Multi-methodology is considered desirable in MS/OR studies for four major reasons: (1) the real world appears to be multidimensional and each study should take into consideration material, social, and personal aspects, (2) most major MS/OR studies might have distinct phases that require different methods, (3) the use of several methods might improve the reliability of ndings through triangulation, and (4) using several methods in an MS/OR study increases the richness and variety of possible results (Mingers, 2003). It is also interesting to note that both PAR and MS/OR have sought help from the eminent philosophers of the past. PAR offers an interpretation of Aristotelian phronesis and its relevance for action research and concludes that phronesis is very important for not only action research but also for other disciplines as well. MS/OR has also established such connections. Critical realism is claimed to be suitable as an underpinning philosophy for MS/OR (Mingers, 2000). Pragmatism as a philosophical approach has an important place both in MS/OR and PAR. It is suggested that the practical commonsense and scientic approach embedded in pragmatism resonates with MS/OR as practiced. (Ormerod, 2006). The emphasis of Marxism on the dynamics of change is claimed to have relevance today for MS/OR (Ormerod, 2008). The critical rationalism of Karl Popper has also implications for MS/OR, because both natural and social sciences, including MS/OR, are claimed to have progressed through problem solving using trial and error. Progress has been made by subjecting potential theories to vigorous testing and criticism; and falsied theories are rejected (Ormerod, 2009). The unifying framework for PAR and MS/OR given in Fig. 4 as a theory that includes three modes of inference: abduction, deduction, and induction. Abductive stage, the upper part of Fig. 4, corresponds to PAR and its output. Abductive reasoning produces an inference of the case from the rule and result (Niiniluoto, 1999). Abductive reasoning is present in our R&D planning process because the stakeholders conclude that there is a technological gap (case) given the desired technological level (rule) and the existing

M. Oral / Expert Systems with Applications 39 (2012) 65036513

6513

technological level (result). Deductive and inductive reasoning are dominant in the MS/OR activities, the lower part Fig. 4. Deduction is an inference of a result from a rule and a case (Niiniluoto, 1999). The explaining syllogism in our R&D planning process is that the selected R&D projects (result) need to be sponsored if a higher level of consensus (rule) is to be achieved (case). Induction, on the other hand, is the inference of the rule from the case and result (Niiniluoto, 1999). Again referring to our R&D planning process, to achieve the highest level of consensus possible (rule) one needs to nance the selected set of R&D projects (result) through the process of optimization (case). It should be noted that PAR is mostly dominated by abductive reasoning whereas MS/OR by deductive and inductive reasoning. 6. Concluding remarks PAR and MS/OR converge to one another in the sense that they both seek to produce knowledge for action and change in organizations or communities. The ultimate goal for both is the conation of theory and practice for the betterment of organizations or communities. All organizations are created and managed by humans and therefore they reect the preferences, values, desires, and capabilities of their designers and managers. The world of organizations out there is not something given by an independent creator but by humans and therefore is subject to change constantly to respond to new needs and desires. The nature of the world of organizations is human-based and the way it is perceived by its stakeholders does matter for all kind of researchers. The ontological subjectivity of PAR makes a contribution in this regard. Once the nature of the world of organizations is established however the issues needs to be addressed as objectively and consistently as possible to assure some epistemological objectivity. From this perspective, MS/OR is there to contribute. One can extend this kind of approach to unifying different paradigms in operations management research. For example, one can study how PAR can be integrated with AHP (Analytical Hierarchy Process), a methodology in MS/OR, so that better parts of both approaches are put into use in a decision making process for more relevant, legitimate, and valid results. References
Ackoff, R. L. (1977). National development planning revisited. Operations Research, 25, 207218. lu, O., & Rav, I. (1992). Normative action research. Organization Studies, 13, Babrog 1934. Barton, J., Stephens, J., & Haslett, T. (2009). Action research: Its foundations in open systems thinking and relations to the scientic method. Systemic Practice and Action Research, 22, 475488. Chandler, D., & Torbert, B. (2003). Transforming inquiry and action: Interweaving 27 avors of action research. Action Research, 1, 133152. Charnes, A., Cooper, W. W., & Rhodes, E. L. (1978). Measuring the Efciency of Decision Making Units. European Journal of Operational Research, 2, 429444. Checkland, P. (1981). Systems thinking, systems practice. Chichester, UK: John Wiley & Sons. Checkland, P., & Scholes (1990). Soft systems methodology in action. Chichester, UK: John Wiley & Sons. Checkland, P., & Winter, M. (2006). Process and content: two ways of using SSM. Journal of the Operational Society, 57, 14351441. Considine, D. M. (1978). Van Nostrands scientic encyclopedia. New York: Van Nostrand Reinhold, pp. 1399-1408. Cook, W. D., & Seiford, L. M. (2009). Data envelopment analysis (DEA): Thirty years on. European Journal of Operational Research, 192, 117. Cooperrider, D. L., Whitney, D., & Stavros, J. M. (2008). Appreciative inquiry handbook: For leaders of change (second ed.). Brunswick, Ohio: Crown Customs Publishing, Inc. Dick, B. (2007). Action research as an enhancement of natural problem solving. International Journal of Action Research, 3, 149167. Dick, B. (2009). Action research literature 20062008. Action Research, 7, 423441.

Dick, B., Stringer, E., & Huxham, C. (2009). Theory in action research. Action Research, 7, 512. Doyle, J., & Green, R. (1994). Efciency and cross efciency in DEA: Derivations, meanings, and uses. Journal of the Operational Research Society, 45, 567578. Friedman, V. J., & Rogers, T. (2009). There is nothing so theoretical as good action research. Action Research, 31, 3147. Genat, B. (2009). Building emergent situated knowledge in Participatory Action Research. Action Research, 7, 101115. Green, R. H., Doyle, J. R., & Cook, W. D. (1996). Preference voting and project ranking using DEA and cross evaluation. European Journal of Operational Research, 90, 461472. Greenwood, D. (2002). Action research: Unfullled promises and unmet challenges. Concepts and Transformation, 7, 117139. Gustavsen, B. (2003). New forms of knowledge production and the role of action research. Action Research, 1, 153164. Gustavsen, B. (2008). Action research, practical challenges and the formation of theory. Action Research, 6, 421437. Kettovi, M., & Mantere, S. (2010). Two strategies for inductive reasoning in organizational research. Academy of Management Review, 35, 315333. Kirby, M. W. (2007). Paradigm change in operations research: Thirty years of debate. Operations Research, 55, 113. Landry, M., Banville, C., & Oral, M. (1996). Model legitimisation in operational research. European Journal of Operational Research, 92, 443457. Landry, M., Malouin, J.-L., & Oral, M. (1983). Model validation in operations research. European Journal of Operational Research, 14, 207220. Liang, L., Cook, W. D., & Zhu, J. (2008). The DEA game cross-efciency model and its Nash equilibrium. Operations Research, 56, 12781288. Mead, G. H. (1962). Mind, self, and society from the standpoint of social behaviorist. Chicago, US: The University of Chicago Press. Mingers, J. (2000). The contribution of critical realism as an underpinning philosophy for OR/MS. Journal of the Operational Research Society, 51, 12561270. Mingers, J. (2003). A classication of the philosophical assumptions of management science methods. Journal of the Operational Research Society, 54, 559570. Niiniluoto, I. (1999). Defending abduction. Philosophy of Science, 66, S436S451. Oral, M. (2010). E-DEA: Enhanced Data Envelopment Analysis. European Journal of Operational Research, 207, 916926. Oral, M., & Kettani, O. (1992). A linearization procedure for quadratic and cubic mixed-integer problems. Operations Research, 40, S109S116. Oral, M., & Kettani, O. (1993). The facets of modeling and validation process in operations research. European Journal of Operational Research, 66, 216234. Oral, M., Kettani, O., & Lang, P. (1991). A methodology for collective evaluation and selection of industrial R&D projects. Management Science, 37, 871885. Oral, M., Yetis , N., & Uygur, R. K. (1981). Participatory planning of industrial R&D activities. Technological Forecasting and Social Change, 19, 265277. Ormerod, R. J. (2006). The history and ideas of pragmatism. Journal of the Operational Research Society, 57, 892909. Ormerod, R. J. (2008). The history and ideas of Marxism: The relevance for OR. Journal of the Operational Research Society, 59, 15731590. Ormerod, R. J. (2009). The history and ideas of critical rationalism: The philosophy of Karl Popper and its implications for OR. Journal of the Operational Research Society, 60, 441460. Poonamallee, L. (2009). Building grounded theory in action research through the interplay of subjective ontology and objective epistemology. Action Research, 7, 6983. Reason, P., & Bradbury, H. (2008). Handbook of action research (second ed.). London: SAGE. Reisman, A., & Oral, M. (2005). Soft systems methodology: A context within a 50year retrospective of OR/MS. Interfaces: An International Journal of the Institute for Operations Research and The Management Sciences, 35(2), 164178. Roy, B. (1985). Mthodologie Multicritre dAide la Dcision. Paris: Economica. Sarkis, J. (2000). A comparative analysis of DEA as a discrete alternative multiple criteria decision tool. European Journal of Operational Research, 123(3), 543557. Senge, P. M. (1990). The fth discipline: The art and practice of the learning organization. New York: Doubleday Currency. Sexton, T. R., Silkman, R. H., & Hogan, A. J. (1986). Data envelopment analysis: Critique and extensions. In R. H. Silkman (Ed.). Measuring Efciency: An Assessment of Data Envelopment Analysis (Vol. 32, pp. 73105). San Francisco: Jossey-Bass. Silverman, B. G. (1994). Unifying expert systems and the decision sciences. Operations Research, 42, 393413. Simon, H. (1991). Bounded rationality and organizational learning. Organization Science, 2, 125134. Sommer, R. (2009). Dissemination in action research. Action Research, 7, 227236. Vaara, E., & Tienari, J. (2008). A discursive perspective on legitimation strategies in multinational corporations. Academy of Management Review, 33, 985993. Van de Ven, A. H., & Johnson, P. E. (2006). Knowledge for theory and practice. Academy of Management Review, 31. 802-801. Weick, K. E. (1995). Sensemaking in organizations. Sage Publications. Wu, J., Liang, L., & Ying-chun, Z. (2008). Determination of weights of ultimate cross efciency based on the solution of nucleous in cooperative game. Systems Engineering, 28, 9297.