UTILIZING KPIS IN EVALUATING ACADEMIC PROGRAMS

Prof. Shaker Rizk Suez Canal University shaker115@hotmail.com

Abstract In order to improve its academic programs, Ittihad University (IU) has launched a two phase benchmarking project that aims to evaluate the various aspects of performance in Higher Education Institutions (HEIs). By the end of phase one, IU has developed some key performance indicators (KPIs) that are being utilized internally to evaluate and improve quality in its existing programs. This paper is intended to report on the progress that has been achieved in phase one of the project in which IU has piloted, standardized, and implemented those KPIs internally in preparing for the reaccreditation of its programs. The developed KPIs can be considered a consistent set of statistical objective measures of how academic programs at IU are performing. They facilitate comparison among the different programs at IU and between individual institutions of a similar nature in the UAE. They are intended to enable HEIs to benchmark their performance and provide data necessary for policy developers and decision makers to assess the progress and success of their missions, and pave the way for achieving HEIs goals and objectives. The developed KPIs have covered various aspects that include: Teaching and Learning, Student Outcomes, Educational Support Resources, Research and Training, Community Service / Engagement, Organizational Leadership, Staffing, Institutional Infrastructure, and Information and Communication. Findings of the project will be disseminated to other countries in the Arab World hoping that they set the foundations for homogeneous standards and performance indicators in their HEIs. Key words: Performance indicators, benchmarking, program assessment, quality assurance, higher education

1. INTRODUCTION Key Performance Indicators (KPIs) are a set of statistical indicators designed with the purpose of offering an objective measure of how a higher education institution (HEI) is performing. They are basic elements of the overall planning and monitoring system and are usually derived from the Higher Education Institutions (HEI) mission. They are integrated collections of components that help HEIs assess their performance in accomplishing their missions. According to (Hfner, 1991), quoted by Pereira and Tavares (2002:84), there are four categories of indicators: 1. Internal performance indicators which are based on the internal information produced by the institution, such as the pass rate, the graduate rate, the number of dissertations accepted, the average duration of the course, and the assessment of the professors by the students, amongst others, 2. Indicators of operational performance that refer to the internal activities of the departments, e.g., student/professor ratio, student/staff ratio, and the unit costs, size of the classes, ratio of employees, and student/computer ratio, 3. External performance Indicators which refer to information external to the university, such as, employability of the graduates and reputation of the university in the work market, and 4. Research Performance Indicators which aim at assessing the research activities of the university, e.g., the number of publications, percentages in the research contracts, the number of dissertations accepted, consultations provided, inventions and/or patents, invitations to relevant scientific conferences, awards and distinctions. In the present study the four categories will be accounted for in order to get a full picture of the institution in its path towards future planning and expansion. 1.1. Why use KPIs? In the IU project, measuring performance of faculty and staff is mandatory so as to equip leadership of the institution with information that helps it make informed decisions that

lead to improvement. For this particular reason, KPIs are designed to assist IU leadership carry out real and authentic measurements of its performance. Key Performance Indicators are consistent measurement tools that provide information used by HEIs management to monitor and track their progress against their suggested strategic goals , and know whether they are succeeding in achieving their missions or not. They also permit comparisons between similar institutions through benchmarking, and assist policy makers, planners, and decision makers with reliable data that can help them decide on their progress and achievements. KPIs are considered communication vehicles that enable leadership and top management transmit their missions and goals to their employees ensuring that everyone in the institution is following the right direction that lead to its development and progress. In addition, KPIs are mainly used in the benchmarking (BM) process where different programs can be compared against each other or against specific quality assurance standards. They are indicators that are used to establish good practices, determine strengths and weaknesses, and implement best practices that improve quality in academic programs and management policies. Together with BM, KPIs are used for measuring and comparing the work processes of one organization to those of another, by bringing an external focus to internal activities, functions, or operations (Kempner 1993). They provide decision makers and leaders in HEIs with an external standard for measuring the quality and cost of internal activities, and to help identify areas of improvement that may take place in such institutions. Furthermore, KPIs and BM are used for comparison with those of competitors and others considered best-in-class in a specific function (Dervitsiotis, 2000), comparing a product or process against others, with reference to specified standards (Pepper, Webster & Jenkins 2001), a systematic way of learning from others and changing what you do (Epper 1999), imitating successful behaviour (Karlf and stblom 1993), or a learning process, which requires trust, understanding, selecting and adapting good practices in order to improve (ENQA workshop 2002).

2. DESIGN OF KPIS This study adopts an eclectic approach that determines different areas in a certain hierarchical order moving from the institutions mission to its goals, objectives, strategy, indicators, measures, target, and ending with its results. Goals and objectives are related in a way where one goal is interpreted in many objectives, whereas each measure has only one target. First, mission and goals have been specified. Then Objectives are defined as what IU is trying to achieve; and one objective may have more than one indicator and strategies on how to achieve it. Second, Indicators specify what is to be measured and are used to assess the present state of progress and to suggest appropriate procedures of action. Third, Measures are quantitative and qualitative data and tools relating to input, process and output that can be used in the measurements, while Targets are the expected results that IU may want to achieve. Finally, Results mean what the institution has actually achieved. 2. 1. Previous KPIs Designs Kennedy, C. (2010) designed some KPIs in higher education under certain perspectives and objectives. Perspectives include academic, financial, research, and campus and community aspects, while objectives include better recruitment of students, graduating better students, providing quality academic programs and support services, attracting and retaining outstanding faculty/staff , optimizing expenditures, increasing funded research, providing adequate research facilities, increasing faculty scholarly activities, improving campus life, supporting integrity and service to community, supporting superior athletics, and promoting and enhancing sustainability. Moreover, the Higher Education Statistics Agency (HESA) has produced and published the Performance Indicators since 2002/03. These PIs provide comparative data on the performance of institutions in widening participation, student retention, learning and teaching outcomes, research output and employment of graduates. They cover publiclyfunded higher education institutions in the UK. On the other hand, da Costa (2001) suggests a group of management indicators that can support the university management process in the Portuguese State Universities. Those Management indicators can be used

by state universities in exercising economic efficient management control. New types of management that will improve motivation, productivity and obtain results that are real benefits for society, have to be introduced. Bunting (2004) offers a country case study of the development of KPIs in the context of international experience of the development and role of performance assessment and indicators. He suggests some PIs that fit the African context making use of International Perspectives on KPIs. Knight (2006), on the other hand, developed a planning process based on best practices at Bowling Green State University where he utilized it in specifying some KPIs that fit HEIs. In 2006, the Committee of University Chairmen (CUC) has built a framework for monitoring institutional performance around the concept of (KPIs). At the top level, the CUC framework proposes the use of ten high-level KPIs which can be used to assess all the aspects of institutional performance which are of fundamental concern to governors. These ten levels include: institutional sustainability, academic profile and market position, the student experience and teaching and learning, research, knowledge transfer and relationships, financial health, estates and infrastructure, staff, governance, leadership and management, and institutional projects Suryadi (2007) has developed a model of KPI measurement in HEIs based on a combination between Analytic Hierarchy Process (AHP) trend analysis and comparative data. KPIs are determined as a description of key success factors. These KPIs are categorized into academic, research and supporting KPIs. The proposed model contributes in measuring and explaining institution success using multi dimensions of KPI, and it is considered a tool for organizational self-assessment. Ishak et al (2008) have developed KPIs to measure performance of academic staff in private higher learning institutions in Malaysia. Universiti Tenaga Nasional is chosen as the case study, where 14 key performance indicators were developed under 5 main headlines, teaching and supervision, research and innovation, publication, consultancy, and services.

In 2011, the American Academy for Liberal Education (AALE) has developed some KPIs for a streamlined system for financial and institutional data collection and analysis, in order to minimize the time and manpower required to establish the viability of educational programs. The Academy uses the KPI process in its reviews of new applicants for accreditation, and in monitoring the continuing performance of member institutions. It emphasizes the compactness and reliability of the set of indicators used in the KPI review, arguing that they provide accreditors with accurate "dashboard indicators" of institutional health without the intrusiveness and massive documentation that characterize the reporting systems. The present study followed a frame focusing on the extent to which outcomes-based objectives have been achieved and the extent to which strategies for achieving the objectives are being successfully implemented. This means that there may be a separate KPI for objectives and their associated strategies. In addition, tools for analysing a KPI have been SMART, i.e. (Specific, Measurable, Aligned, Realistic, and Timely). In order to design good KPIs, IU has aligned them to the Standards of the Commission for Academic Accreditation (CAA), tested their validity and reliability, reviewed them to make sure that they are really keys, benchmarked its five accredited programs, simplified them for stakeholders , and finally used their results for change. IU has specified ten areas for IU KPIs including: Teaching and Learning, Student Outcomes, Educational Support Resources, Research and Training, Community Service, Organisational Leadership, Staffing, Institutional Infrastructure, and Information and Communication, but the present study is reporting on the teaching and learning areas.

3. THE IU KPIS In the light of the preceding importance of KPIs in assessing HEIs in general and in applying for reaccredidation of its programs in particular, IU has designed KPIs in order to benchmark internally its five accredited programs and get prepared for the 2012 reaccreditation application (see Rizk and Al Alusi 2011). The study followed a frame

focusing on achieving outcomes-based objectives and the strategies that enable IU achieve them. The process followed the following procedures: 1. Stating the IU mission and goals, 2. Defining the objectives in light of what IU is trying to achieve; 3. Specifying the indicators that show what to be measured, 4. Assessing IU progress, 5. Suggesting actions to be taken, 6. Specifying the measures and tools that are used with the quantitative and qualitative data, 7. Specifying the expected results, i.e., the targets, 8. Arriving at the Results in order to determine what IU has actually achieved. 9. Suggesting action plans to fill the gaps or modify adopted plans and actions, and 10. Recommending possible tools and means of carrying out the actions before IU applies for the reaccreditation. 3.1. IU Learning Domain KPIs In each set of the IU designed KPIs, the goals, objectives, and strategies are tabulated in one table and the indicators, measures and targets are set in another. Table 1 below shows the Goal, Objectives and Strategies for Student Learning domain. The major goal is for students success and educational effectiveness, IU has to embrace a culture of excellence and performance as the hallmarks of learning and student success and to provide a solid foundation for their careers. This broad goal has two objectives: 1. To achieve a shared institutional culture that makes student learning and success the responsibility of all. 2. To achieve a shared institutional culture that promotes diversity, respect for others, and continuously strives for outstanding performance. These objectives have been set into different strategies that can be carried out to achieve the goal. Ten major strategies and some minor strategies have been developed for this purpose. Table 1 Goal, Objectives and Strategies for Student Learning
Goal Objectives Strategies

Design and implement an effective enrolment plan to improve the entry, retention, and success of diverse student population. Develop programs to attract high-achiever students, assist them in their transition to university study. Ensure that students experience a transforming education by: Providing access to fully articulated, integrated, and comprehensive academic programs. Engaging students in active and autonomous learning. Educating students to rise above ethnic and other prejudice. Using technology to support learning. Enhancing the quality and efficiency of student services. Assessing student outcomes, conducting institutional research on curriculum and pedagogy, and using results to improve student learning. Addressing the need for remedial and developmental programs. Facilitating the faculty role as coach, mentor, and tutor. Enhance the involvement of undergraduate students in the creation and transfer of knowledge through research-intensive courses, student research opportunities and related employment, joint faculty-student publications, and service learning opportunities. Enhance the student experience by implementing: Student leadership training. Community service opportunities and career networks. A student database that supports a positive relationship with students from recruitment to alumni status. Learning communities that connect students to one another. Improve the structure of decision-making, promote core values, and build quality conversations among different parts of the University. Develop processes that enable students, faculty, and administrators to evaluate faculty teaching effectiveness and encourage faculty to improve their pedagogies and curricula throughout their careers; carry out complementary processes to assess administrative effectiveness. Continue to give admission preference to better qualified students and actively recruit non-Ras Al-Khaimah/UAE students. Continue to recruit qualified faculty and support staff. Continue to recruit and support academic high achievers by providing financial support and opportunities.

To achieve a shared institutional culture that makes student learning and success the responsibility of all. Embrace a culture of excellence and performance as the hallmarks of learning and student success and to provide a solid foundation for their careers.

To achieve a shared institutional culture that promotes diversity, respect for others, and continuously strives for outstanding performance.

Table 2 below shows the indicators, measures and targets for students learning. The indicators include: number of newly enrolled students (nationality, gender), retention rate, and number of students enrolled with 80% and above, students' satisfaction rate, students' training and research, and students' academic and social performance progress. To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine: % new enrolment to total RAK leavers, % graduating students rate per nationality and per gender, % High School leavers enrolled

with 80% +,% CGPA of 3.0 and above, Course Evaluation domains, Exit (Advising, Pos, Table 2 KPIs for Student Learning Indicators Measurement/Tools

Targets
15% 80% 50% 50% 30% 20% 4 out of 5 4 out of 5 4 out of 5 4 out of 5 4 out of 5 4 out of 5 80% 80% 80% 5% 75% 40% 10% 3 15% 10%

# of newly enrolled students % new enrolment to total RAK leavers (nationality, gender) % graduating students rate % per nationality % per gender retention rate % HS leavers enrolled with 80% + enrolled with 80% above % CGPA of 3.0 and above students' satisfaction rate

students' training and

research

CTE (Course Evaluation domains) Exit (Advising, curriculum, POs) CFR ( students written remarks) Academic Advising Survey Alumni Survey (POs) Training Evaluation by Student Survey Graduation Project (BIS, EL&T, CS,CE) TPI 1 and 2 Portfolios Trainee Performance Reports

students' academic and

Academic warning (students at risk) social performance progress Semester GPA Scholarships 4 categories: o 20% 80-89 HS Av o 50% 90+ o Full Top 10 (UAE, RAK) o 20% Top 10 in each program o Financial Aid (social:4-12%, brothers)

curriculum), Course File Report ( students written remarks), Academic Advising Survey, Alumni Survey, Training Evaluation by Student Survey, Graduation Project (BIS, EL&T, CS,CE), Teaching Practice I and II Portfolios, Trainee Performance Reports, Academic warning (students at risk), Semester GPA, Scholarships 4 categories ( 20% 80-89 HS Av, 50% 90+, Full Top 10 (UAE, RAK), 20% Top 10 in each program), and Financial Aid (social:4-12%, brothers). Targets for these measures are set and the five accredited programs have been benchmarked accordingly.

3.2. IU Teaching Domain KPIs Table 3 below shows the Goal, Objectives and Strategies for Faculty Teaching. The major goal for faculty and staff is: Providing an inspiring work environment for faculty and staff. This goal has been put in two objectives: 1. to develop faculty and staff

competencies and skills, and 2. to create a culture of excellence through recruiting, rewarding, and empowering highly qualified faculty and staff. Four strategies have been set to achieve the goal and objective of this domain. These include: Faculty Professional development is of prime priority within IU, Faculty and staff have access to a professional development plan, Recruiting faculty & staff in posts relating to their college and university strategic plan, and Conducting exit survey and /or interviews with faculty and staff leaving IU. Table 3 Goal, Objectives and strategies for Faculty Teaching
Goal Providing an inspiring work environment for faculty and staff Objectives
1. Develop faculty and staff

Strategies

Faculty Professional development is of prime priority within IU Faculty and staff have access to a professional development plan 2. Create culture of Recruiting faculty & staff in posts relating to excellence through their college and university strategic plan recruiting, rewarding, and Conduct exit survey and /or interviews with empowering highly faculty and staff leaving IU qualified faculty and staff competencies and skills

Table 4 shows the indicators, measures and targets for faculty teaching. The indicators include: Faculty Professional development activities, Faculty turnover rate, and IU prospective students choice of courses. Table 4 KPIs for Faculty Teaching Indicators
Faculty Professional development activities

Measurement/Tools
% of faculty participating in Prof Dev. # publications, seminars, conferences, workshops # thesis & disserts supervised/examined # Consultations & Reviews No. of Faculty leaving IU annually (probation) No. of Faculty leaving IU within 3 years (contract) # of students registered with faculty

Targets
60% of the total At least once a year per faculty At least once a year per faculty At least once a year in each program No more than 20% of the total faculty Reduce by 5% annually At least 80% of students in compulsory courses

Faculty turnover rate IU prospective students choice of courses

To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine: % of faculty participating in Professional Development, # publications, seminars, conferences, workshops, # theses & dissertations supervised/examined by IU faculty, # Consultations & Reviews, # Faculty leaving IU annually (probation), #of Faculty leaving IU within 3 years (contract), and # of students registered with faculty. Targets for these measures are set and the five accredited programs have been benchmarked accordingly. 3.3. IU KPIs for other Domains Following are the KPIs for the other domains that will be used in the analysis of data that cover the academic domains in the five IU accredited programs. These domains include Curriculum, Research and Training, Community Services, and Resources Management. Table 5 below shows the Goal, Objectives and Strategies for Curriculum. The major goal for this domain is to provide viable Academic Programs. This goal has been put in one objective, i.e., Academic Programs are relevant to the market needs. Two strategies have been set to achieve the goal and objective of this domain. These include: attracting high achievers high school leavers through marketing IU academic programs and forming an advisory board for each IU program that includes potential employers. Table 5 Goal, Objectives and strategies for Curriculum
Goal Provide viable Academic Programs Objectives Academic Programs are relevant to the market needs Strategies
1. attract high achievers high

school leavers through marketing IU academic programs 2. form an advisory board for each IU program that includes potential employers

Table 6 shows the indicators, measures and targets for the curriculum domain. These indicators include: New Students enrollment rate, Graduate employment rate, and Program Accreditation by the CAA. To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine five measures: % of student enrollment in each program, % of Top 10 RAK and UAE high

school leavers, % of graduates in full-time employment, Academic Accreditation and Feedback from employers (Employer Survey). Targets for these measures are set and the five accredited programs have been benchmarked accordingly. Table 6 KPIs for Curriculum
Indicators New Students enrollment rate Graduate employment rate Program Accreditation by the CAA

Measures % of students enrollment in each program % of Top 10 RAK and UAE high school leavers % of graduates in full-time employment Academic Accreditation Feedback from employers (Employer Survey)

Targets > 20% > 75% Grant program Initial, Full, and reaccreditations

Table 7 below shows the Goal, Objectives and Strategies for Research and Training. The major goal in this domain is: Encourage Research, and Training Engagement. This goal has been put in three objectives: 1. to encourage faculty in Institutional research and external applications, 2. to pursue postgraduate studies by IU graduates and 3. to encourage Institutional and external training programs Five strategies have been set to achieve the goal and objective of this domain. These include: 1. Faculty conduct research collaboratively across disciplines utilizing library resources, 2. Faculty can access professional development budget, 3. Fostering partnership with national & international HEIs to provide grants, 4. Initiating postgraduate studies in current disciplines, and 5. Providing training opportunities internally & externally in workplaces. Table 7 Goal, Objectives and strategies for Research and Training
Goal Objectives Strategies Encourage faculty in Faculty conduct research collaboratively across Encourage Institutional research and disciplines utilizing library resources Research, external applications and Training Pursue postgraduate studies Faculty can access professional development Engagement by IU graduates budget Foster partnership with national & international HEIs to provide grants Encourage Institutional Initiate postgraduate studies in current disciplines and external training Provide training opportunities internally & programs externally in workplaces Encourage Institutional

and external training programs

Table 8 shows the indicators, measures and targets for research and training domain. The indicators include: Research projects conducted annually, Annual expenditures on professional development, Graduates successfully completing postgraduate degrees, # of postgraduate studies initiated, Trainees successfully accomplish training program. To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine: # articles published in journals, conferences, and other academic activities, # projects conducted institutionally and externally, proportion of used budget to be allocated to professional development budget & IU budget, proportion of an IU program graduates completing postgraduate degrees to the program/IU graduates annually, # registered graduate students, and #faculty accomplish training program to the total IU faculty. Targets for these measures are set and the five accredited programs have been benchmarked accordingly.

Table 8 KPIs for Research and Training

Indicators Research projects conducted annually

Measures No. of articles published in journals, conferences, and other academic activities No. of projects conducted institutionally and externally

Targets 50% of full time faculty publishing articles, books, participating in conferences/workshops/s eminars 50% success 50% of the allocated budget for faculty professional development budget 05%

Annual expenditures on Proportion of used budget to professional development be allocated to professional development budget & IU Graduates successfully budget completing postgraduate Proportion of an IU program degrees graduates completing postgraduate degrees to the program/IU graduates annually No. of postgraduate No. of registered graduate studies initiated students Trainees successfully No. of faculty accomplish accomplish training training program to the total IU program faculty

05% 25%

Table 9 below shows the Goal, Objectives and Strategies for Community Services. The major goal for this domain is to: Enhance Community and Service Engagement. This goal has been put in two objectives: 1. to provide leadership to local community, and 2. to increase the number of intake in the local region. Table 9 Goal, Objectives and strategies for Community Services
Goal Enhance Community and Service Engagement Objectives Provide leadership to local community Increase the number of intake in the local region Strategies Faculty and staff are encouraged to participate in community activities Enhance high schools marketing program through scheduled visits

Two strategies have been set to achieve the goal and objective of this domain. These include two strategies: 1. Faculty and staff are encouraged to participate in community activities, and 2. Enhancing high schools marketing program through scheduled visits. Table 10 shows the indicators, measures and targets for faculty teaching domain. The indicators include: participation in community activities, (committees, boards), and number of newly admitted students to IU To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine: % of faculty & staff participating in community services activities, Proportional of local region new enrollment against IU total new enrollment, and Proportional of local region new enrollment against total of high school leavers. Targets for these measures are set and the five accredited programs have been benchmarked accordingly. Table 10 KPIs for Community Services
Indicators Participation in community activities, (committees , boards) No. of newly admitted students to IU

Measures % of faculty & staff participating 20% in community services activities

Targets

Proportional of local region new 30% in 2009 increasing 5% enrollment against IU total new every year until 2012 enrollment. (reaccreditations) Proportional of local region new enrollment against total of high (a) 10% of RAK high school school leavers leavers

(b) 50% more than other

RAK HEIs.

Table 11 below shows the Goal, Objectives and Strategies for Resources Management domain. The major goal for is: To Enhance Effective Resources Management and Future Expansion. This goal has been put in four objectives: 1. Allocate & manage resources that improve organizational processes, 2. Generate non-traditional revenue strategies 3. Establish new undergraduate & postgraduate academic programs, and 4. Expand the physical infrastructure to meet the growing demands.

Table 11 Goal, Objectives and strategies for Resources Management

Goal To Enhance Effective Resources Management and Future Expansion Objectives Strategies Allocate & manage resources that Invest in change to reduce improve organizational processes cost, generate revenue, and maintain quality Conduct annual and 5-year holistic reviews Develop, fund, execute a comprehensive marketing plan in the UAE & beyond Generate non-traditional revenue strategies Establish new undergraduate & Initiate new undergraduate & postgraduate academic programs postgraduate programs based on market surveys to assess the market needs Expand the physical infrastructure Establish new premises to meet the growing demands

Five strategies have been set to achieve the goal and objective of this domain. These include: Invest in change to reduce cost, generate revenue, and maintain quality, Conduct annual and 5-year holistic reviews, Develop, fund, execute a comprehensive marketing plan in the UAE & beyond, Initiate new undergraduate & postgraduate programs based on market surveys to assess the market needs, and Establish new premises. Table 12 shows the indicators, measures and targets for Resources Management domain. These include: Revenue increase & cost reduction, Enrollment increase, and Programs accreditation and Physical expansion of IU premises. To measure the progress in achieving the goals and objectives, those indicators are measured using different tools that combine: Ratio of expenditure/cost against revenue, Initial Accreditation (IA), Full Accreditation (FA), and Proportion of area on newly build premises against current premises. Targets for these measures are set and the five accredited programs have been benchmarked accordingly.

Table 12 KPIs for Resources Management

Indicators Measures Targets

Revenue increase & cost Ratio of expenditure/cost against 5% Cost reduction every reduction revenue year 2009-2012 10% revenue increase every year 2009-2012 Enrollment increase Programs accreditation IA One academic program FA annually A postgraduate program every 3 years nd Physical expansion of IU Proportion of area on newly Structure the 2 stage of IU premises build premises against current campus when student premises population reaches 1000.

4. RESULTS OF THE LEARNING DOMAIN Results of this domain are shown in tables 13 and 14 below (for more details see Rizk et al 2011). The tables show that averages in all items of the domains are approximately close. This indicates stability of the program offerings and policies. In year 2009/2010 there is an increase of total number of registered students in the CE program, and the scholarships over 90% (this means that the program attracts top high school graduates due to its reputation among local students of the RAK Emirate). On the other hand, a decrease of total number of registered students in the CS program, and the scholarships over 90% (none), the CGPA (11 versus 13.3), Male Graduates (1 versus 3), and Scholarship top 10% (2 versus 2.3). Table 13 Enrollment, GPA, and Graduates (ALL Programs 3-Year Average)
New Enrolment New Enrolment with >80% Total Registered Students With Academic Warning Average Sem. GPA CGPA > 3.0 Graduates Female Graduates Male Full Scholarship Scholarship 90 % CE 13.67 12.67 37.00 0.67 3.29 18.33 0.67 2.00 2.33 14.33 CS 11.67 6.00 50.33 7.67 2.33 13.33 5.67 3.00 0 11.00 BIS 82.67 29.33 312.67 39.33 2.44 72.33 19.33 17.00 4.67 21.00 ELT 16.00 6.67 71.67 6.67 2.61 20.33 11.00 0 0.33 7.33 EL&T 10.33 5.00 57.00 4.00 2.56 19.00 9.67 0 2.33 7.33

Scholarship 80 % & < 90 % Scholarship Top 10 % (3.5) Financial Aids

8.00 1.33 7.33

4.67 2.33 16.00

23.67 21.33 75.00

4.00 3.00 17.00

4.00 3.00 14.00

This indicates that the program is undergoing some problems that require practical actions to improve it. In the BIS there is a decrease in the new enrollment in 2009 due to opening a new program in IU. Table 14 Enrollment, GPA, and Graduates (ALL Programs Average per Year)
New Enrolment New Enrolment with >80% Total Registered Students With Academic Warning Average Sem. GPA CGPA > 3.0 Graduates Female Graduates Male Full Scholarship Scholarship 90 % Scholarship 80 % & < 90 % Scholarship Top 10 % (3.5) Financial Aids 2007-2008 26.2 10.4 97.2 9.8 2.758 28 6.4 3.2 1.4 9.8 0 7.6 33.4 2008-2009 29.6 13.8 110.8 14 2.664 27.2 10.8 5.2 1.4 11.8 8.4 5.2 24.6 2009-2010 24.8 11.6 109.2 11.2 2.516 30.8 10.6 4.8 3 15 18.2 5.8 19.6 Overall Average 26.87 11.93 105.73 11.67 2.65 28.67 9.27 4.40 1.93 12.20 8.87 6.20 25.87

There is also a big increase in the Male graduates in 2009. In the ELT and EL&T programs there is a decrease in most items of the domain in 2009. This might be due to opening the new Teacher of Arabic and Islamic Studies Program which requires no TOEFL score, and eventually attracted newly enrolled students over the past two years. There is also an increase of students with academic warning.

4.1. Results of the Learning Indicators Indicator #1 Table 15 below shows that the new enrolment average is relatively close compared to the overall average. Compared to the target, this item has been achieved over the five programs. The percentages and averages of graduating students per gender show that female students outnumbered male students graduating in all programs. It is almost

double ratio 2 to 1 (9.27 versus 4.4). The new enrollment with > 80% has reached @12 (11.93) which exceeded the 30% target. It means that the high school leavers with 80% and above have joined Ittihad University. This indicates that IU has attracted high quality students. The average semester GPA is 2.65 with a percentage of 66% or D+ Grade. This is below average and indicates that the overall average of GPA needs to be assessed and analyzed carefully to account for its reasons and find ways to improve and upgrade it. Achieved CGPA above 3 has reached 21.5% over the three academic years which indicates that the percentage of very good students has achieved the target. Table 15 Indicator #1 of Student Learning Domain
Indicator I
# of newly enrolled

Measurement/Tools

2007-2008 26.2 97.2 3.8 5.2 6.4 3.2 10.4 2.758 28

2008-2009 29.6 110.8 8.6 7 10.8 5.2 13.8 2.664 27.2

2009-2010 24.8 109.2 8 7.2 10.6 4.8 11.6 2.516 30.8

New Enrolment Total Registered Students % graduating students rate National no national Female Male enrolled with 80% above New Enrolment with %08> Average Sem. GPA Achieved CGPA 0.8above students (nationality, gender) retention rate

Overall average 26.87 105.73 6.8 6.47 9.27 4.4 11.93 2.65 28.67

Indicator #2 Table 16 below shows the average of student satisfaction with curricula in the CTE (4.3). This indicates that most students in all programs are satisfied with the courses offered. T Table 16 Average (out of 5) of Course Evaluation
Program CS & CE BIS ELT EL&T IU Average 2007-2008 4.23 4.02 4.41 4.47 4.28 2008-2009 4.28 4.31 4.23 4.37 4.30 2009-2010 4.35 4.21 4.45 4.39 4.35 3-year Av 4.29 4.18 4.36 4.41 4.31

Table 17 below shows that 88% of students are satisfied with the teaching process. This percentage is almost 82%+ among students in all programs. Table 17 Average of Teaching Evaluation

Program CS &CE BIS ELT EL&T IU Average

2007-2008 4.23 4.02 4.41 4.47 4.28

2008-2009 4.38 4.43 4.34 4.39 4.38

2009-2010 4.35 4.26 4.53 4.96 4.52

3-year Av 4.32 4.23 4.43 4.60 4.40

Table 18 below shows that students are positively satisfied with the faculty members. On the other hand, students have shown highly negative attitudes toward curriculum and methodology, exams, and other general issues. Overall average indicates that IU leadership on the program level and the deanship level has to consider students comments and work on finding mechanisms to improve the negative aspects in the areas of exams, curricula and teaching methods. Table (18) CFR Students Written Remarks
Category Faculty members Curriculum & methodology Exams General Total Average 2007-2008 For + 249(86.5%) 5 (13.16) 2 (18.2%) 1(16.7%) For _ 36(13.5%) 28 (86.84) 15(81.8%) 13(83.3%) 2008-2009 For + 141(87%) 7 (15.4%) 0 2 (13%) For _ 18 (13%) 49 (84.6%) 2 (100%) 13 (87%) 2009-2010 For + 242 (81%) 23 (74 %) 1 (8.4%) 4 (31%) For _ 56 (19%) 29(26%) 11(91.6%) 19 (69%) 72.3% 27.7% Average to Total For + For _ 68.7% 10.2% 3.24% 0.28% 12.3% 5.2%

Graph 1 below shows the average of student satisfaction with Advising, Curriculum, and POs in the Exit Survey in all programs. The average of the three items among students in all programs achieves the target. Graph 1 Students Satisfaction (Exit Survey)

Three-year Average Graduating/s enior Students ' Satis faction per Program (exit) 5.00

Scores out of 5

4.00 3.00 2.00 1.00 Advis ing Curriculum POs CE 4.39 3.68 3.67 CS 4.15 3.64 4.06 BIS 4.47 4.17 4.33 Dom ains ELT 3.72 3.66 4.00 ELAT 4.50 4.20 4.31

Graph 2 below that students are satisfied with the registration, but they are less satisfied with advising on academic specialization, and communication with advisor (76%). Overall average shows that communication with faculty and staff in the advising survey is satisfactory and so close to the target. Graph 2 Students Satisfaction (Academic Advising Survey)
Overall (all Programs) Average Graduating/Senior Students' Satisfaction (advising 5 3 1 -1
Registration Adv ising on Academic Specialization Communication with adv isors 2007-2008 4.09 3.716 3.654 2008-2009 4.156 3.944 3.724 2009-2010 4.222 4.124 3.982 ov erall Av 4.16 3.93 3.79

Scores out of 5

Doamins

Indicator #3 Table 19 below shows that: 1) the overall Grade average is 3.72 or (93%) and student rating (85.4%), 2) All programs exceeded the target percentage (80%), and 3) the BIS program is rated first (97.7%) followed by CE, EL&T, CS. The ELT is rated last (88%).

Table (19) Training Evaluation Survey & Final Grades

Students' Rating out of 5
Program CE CS BIS ELT EL&T Year Average 2007-2008 4 4 4.46 0 4.25 4.18 2008-2009 4.42 4.42 3.95 0 4.19 4.25 2009-2010 4.33 4.33 4.57 0 4.45 4.42 3-Year Av. 4.25 4.25 4.33 0 4.30 4.28 3-Year Grade Av.

3.91 3.56 3.92 3.52 3.68 3.72

Table 20 below shows the averages of the Graduation Projects grades in all programs. It indicates that the overall average is 3.3 or (82.5%) which is higher than the target grade. It also shows that the majority have learned and benefited from the programs. All programs exceeded the target percentage, and the CE program is rated first followed by EL&T, BIS, and CS. The ELT program is not included because it does not have any graduation project. The results of this indicator show that student learning has been satisfactory and the theoretical background knowledge they have learned has been applied and practiced effectively in a satisfactory percentage. Table (20) Graduation Projects
Program CE (Av SPI,SPII) CS(Av SPI,SPII) BIS EL&T Year Average 2007-2008 2.83 3.38 3.08 3.32 3.15 2008-2009 4 3.05 3.55 3.79 3.60 2009-2010 3.64 2.79 3 3.2 3.16 3-Year AV 3.49 3.07 3.21 3.44 3.30

Indicator #4 Table 21 below shows that the average Academic warning in the university is almost 2% in the three years. Scholarship for students with 90% score in high school is 2.3%. The overall scholarship has reached 10% and the financial aids reached 20% of the total registered students. This indicates that students academic progress and performance was

high compared to the Academic warning. The 2.65 SGPA is a good indicator on the students academic progress. Table (21) Academic Progress& Financial Aids
Overall Average Students' academic and social performance progress Academic Warning Average Sem. GPA Full Scholarship Scholarship % 08 Scholarship 90 % Scholarship 80 % & < 90 % Financial Aids 2007-2008 9.8 2.758 1.4 9.8 0 7.6 33.4 2008-2009 14 2.664 1.4 11.8 8.4 5.2 24.6 2009-2010 11.2 2.516 3 15 18.2 5.8 19.6 Overall average 11.67 2.65 1.93 12.2 8.87 6.2 25.87

4.2. Results of the Teaching Indicators Indicator #1 Table 22 indicates that faculty publications and participation in conferences, seminars, and workshops have been within the targeted percentages. It also shows that two faculty members instead of one have had access to at least a criterion of development. The second criterion is not applicable because IU has not started graduate programs yet, the percentage given applies to one faculty who is seconded to IU and has access to graduate programs at his home university. The third criterion indicates that faculty participation in specialized consultations and reviews have been within the targeted percentages. It also shows that1.2 faculty members instead of one have participated. Overall this criterion indicates that IU has very good reputation and cares for faculty development. Table (22) Faculty Professional Development
Professional Development Publications/ Conferences/ Seminars/ Workshops Thesis &Dissertations Reviews & Consultations 6 07-08 6 08-09 12 1 3 09-10 12 1 3 3-Year AV 10 (2 Fac/Progr) 6 (.12 Fac/Progr) 4 (.8 Fac/Progr) % 60% NA 80%

Indicator #2 Table 23 indicates that faculty retention rate is high and the percentage of those leaving IU after the probation year is only 5%, and the percentage of those who leave IU after 3-

year contracts is 25%. This means that IU attracts faculty and keeps them until they end their terms of contracts. Table (23) Faculty Turnover Rate
Professional Development No. of Faculty leaving IU annually (probation) No. of Faculty leaving IU within 3 years (contract) 07-08 0 4 08-09 0 4 09-10 1 4 3-Year AV .33 4 % 5% 25%

Indicator #3 Table 24 indicates that faculty teaching is highly appreciated by the students. This is almost the same as that indicated in Table 17 above in the written comments of students concerning their opinions of faculty. Table (24) Student Choice of Courses
Professional Development # of students registered with faculty 07-08 82.1% 08-09 85.7% 09-10 86.6% 3-Year AV 84.8%

5. CONCLUSION The results of this phase of the IU project have come out with a number of KPIs that have been used in benchmarking the performance of the institution over the past four years so as to equip its leadership with data necessary for making decisions that will lead to improvement in the five accredited programs applying for reaccreditation in 2012. They are to be used to set operational targets for change and involve careful planning to incorporate new practices in the operations, develop action plans; and communicate findings to all institutional levels to obtain support and commitment. In addition, those KPIs provide reliable information, consistent tools for performance measurement, and authentic assessment of performance of the IU programs. The designed KPIs have been used in the benchmarking (BM) process of IU accredited programs through comparing them against each other or against specific quality

assurance standards (CAA Standards). After benchmarking IU five programs using the designed KPIs of the project, certain issues have been reported as necessary to reconsider before the institution applies for the reaccredidation of these programs in 2012. These include: 1. IU needs to enhance the implementation of the process of Quality Assurance. 2. A general Education program needs to be established with specific, mission, goals, objectives, outcomes, and measurement tools. 3. The admissions policies requirements are to comply with the CAA Standards. In this respect, IU has to emphasize that students pass the TOEFL test, or an equivalent placement test, and score not less than 500. 4. Results from surveys are to be disseminated through appropriate channels in accordance with IU quality assurance of the Policy and Procedures Manual. They need to be analyzed for senior management to make decisions and take corrective actions when necessary. 5. Courses offered need to be revised in a comprehensive and complete way, and suggestions for change/improvement are to be carefully discussed in college and University meetings. 6. Reports in Course File Reports need to be written in compliance with CAA 2011 Standards. 7. Faculty and staff evaluation need to be done in accordance with the IU Evaluation Policies, and faculty teaching load has to be implemented properly. 8. Students are to be involved in the university decision making through student associations whose function is to report students opinions and suggestions regarding academic and service issues to the university senior management through the Director of the Students Affairs. This is to ensure that students voices are heard by the university management. 9. IU has to convert benchmark findings, and operational principles based on them, to specific actions to be taken. IU has to put in place a periodic measurement and assessment of achievement. Use the creative talents of the people who actually perform work tasks to determine how the findings can be incorporated into the work processes.

10. Best practices have to be incorporated in all academic and administrative processes at IU in order to achieve maturity and thus ensuring superiority.

References

AL- Alusi, A. & Rizk, S. (2009). A Continuous Improvement Cycle for Academic Programs: Ittihad University Case Proceedings of INQAAHE Conference. Abu Dhabi, UAE. American Academy for Liberal Education (2011). Key Performance Indicators at the WASC Data Elements Task Force Conference. Retrieved January 2, 2011 from: http://www.aale.org/aale/kpiwasc.html Committee of University Chairmen (2006). Report on the Monitoring of Institutional Performance and the Use of Key Performance Indicators. Retrieved January 2, 2011 from: http://www2.bcu.ac.uk/docs/cuc/pubs/KPI_Booklet.pdf da Costa Marques, M. (2001). Key performance indicators in Portuguese public universities. Retrieved January 2, 2011 from: http://www.aabri.com/manuscripts/09240.pdf Dervitsiotis, K. N. (2000). 'Benchmarking and business paradigm shifts, Total Quality Management. 11, pp. 64146. Eckerson, W. (2011). Ten Characteristics of a Good KPI. TDWI. Retrieved January 2, 2011 from: http://tdwi.org/pages/misc/columns/ten-characteristics-of-a-good-kpi.aspx Edith Cowan University in Australia. Annual Reports (2004 - 2010). Retrieved May 22, 2011 from:http://www.ecu.edu.au/__data/assets/pdf_file/0020/208523/ECU-AnnualReport-2010.pdf ENQA (2011). Annual ENQA International Quality Assurance (IQA) Seminar (16-17 June 2011), Helsinki, Finland. Retrieved July 28, 2011 from: http://www.enqa.eu/pubs_workshop.lasso Epper, R., (1999). 'Applying benchmarking to higher education', Change, pp. 2431. Higher Education Statistics Agency (HESA) (2010). Performance indicators in higher education in the UK. Retrieved March, 16, 2011 from: http://www.hesa.ac.uk/index.php?option=com_content&task=view&id=2072&Itemid141 Ian Bunting, I. and Cloete, N. (2004). Developing Performance Indicators for Higher Education. Retrieved January 2, 2011 from: http://chet.org.za/books/developingperformance-indicators-higher-education

Ishak, M., M.S. Suhaida and M.Y. Yuzainee (2008). Performance Measurement Indicators for Academic Staff In Malaysia Private Higher Education Institutions: A Case Study. Retrieved January 2, 2011 from:www.pma.otago.ac.nz/pma-cd/papers/1115.pdf Karlf, B. & stblom, S. (1993). Benchmarking: a signpost to excellence in quality and productivity. Chihester Wiley. Kempner, D.E. (1993). The Pilot Years: The Growth of the NACUBO Benchmarking Project. NACUBO Business Officer, 27(6), 21-31. Knight, W. (2006). Key Performance Indicators in Higher Education. Retrieved May 24, 2011 from: http://www.bgsu.edu/downloads/finance/file21254.pdf Pepper D.; Webster F.; and Jenkins A. (2001). Benchmarking in Geography: some implications for assessing dissertations in the undergraduate curricula, Journal of Geography in Higher Education, Volume 25, Number 1, pp. 23-35. Rizk. S. (2010). Improving an ELT Program through Self-Assessment Proceedings of HEIC Conference. Rizk, S. and AlAlusi, A. (2011). Improving Quality in Higher Education through Internal Benchmarking (IBM). Proceedings of INQAAHE 2011 Conference, Madrid, Spain. Rizk, S. & AL- Alusi, A. (2009). Promoting a Quality Assurance Culture in Higher Education, Proceedings of HEIC Conference. Schofield, Allan (2009). What Can Performance Measurement and Key Performance Indicators Do For Higher Education Institutions? A UK Perspective. Retrieved January 2, 2011 from: http://www.eua.be/eua/jsp/en/upload/Allan%20Schofield.1164100168269.pdf Suryadi, K. (2007). Framework for Measuring Key Performance Indicators for Decision Support in Higher Education Institution. Retrieved January 2, 2011 from: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.137.6940