Vol 01, Issue 02, December 2012 http://iirpublications.

com

International Journal of Web Technology ISSN: 2278-2389

MEASUREMENT-BASED SOFTWARE ENGINEERING EDUCATION

Julias Ceasor A., Sundaram M., Gregory Thaddeus S.

Department of Computer Science and Applications, BIIT, Yelagiri Hills, Tamilnadu-2012.

{ juliasceasor@biit.edu.in, ms@biit.edu.in, thad@biit.edu.in }

Abstract: This study expounds the impact of the evaluation of academic projects with and without
metrics. It is based on real-time data gathered from term-end academic projects of six batches of students at BIIT, Yelagiri Hills. It validates its result with tables and graphs to show the distinction between the two types of evaluation in software academic projects. The data for three years without measurement and with measurement confirm that the evaluation with measurement improves the students performance in their project in terms of timeframe, output and quality. It concludes that measurement - based software engineering education helps the learners and trainers to identify learners strengths and improve further.

Keywords: Software Projects, Evaluation of projects, Software Measurements

1. INTRODUCTION In Indian IT education scenario, software engineering is more theory-based than practice. To bridge the industrial needs with academics, it is suggested that the learners adapt PSP (Personal Software Process) [Cea11] to identify the performance level and take corrective actions. [Mas97] suggests incorporating PSP along the curriculum of Computer Science. PSP helps a software engineer to plan and schedule ones work and analyze his / her own performance in terms of quality, productivity and error prediction [Hum01]. If this provides young software professional to improve, the outcomes of the software projects are evaluated simply from the mere program coding without any criteria followed. Hence the quality of a graduate is not determined / graded as per their quality of performance and knowledge of what they really are. So that setting up clear-cut layers to analyze the software projects with software engineering measurements such as Problem Statement, Requirements, Design (DFD, ERD, etc.), Use case, Test case, Software Build and so on, will help to scale a person on quality, by which one can identify oneself in which area one is strong. Then improving this strength will further make them more professional in a particular domain. Hence the problems described by [Ivi00] may be possible to overcome in the course of time. Integrated Intelligent Research (IIR) 2. RELATED WORKS Many works have been done on software engineering education in different universities to incorporate the requirements of the industry in their teaching. Many found that there is a wide gap between industrial practice and software engineering education in the universities. [Ivi00] states that preparing a software engineer is cumbersome, unless the software engineering discipline combines with the realistic examples. And also, [Ivi00] lists out various difficulties in teaching software engineering course, which is highly challengeable with the industrial expectations. Further [Ivi00] points out, how a project should be chosen to practice software engineering discipline, since the course goes along with project, to gain practical experience of concepts. [Car05] describes the skills required by a software engineer. It is necessary to concentrate on the required software engineering skills. To verify/test a software engineer, one must evaluate ones software project objectively not subjectively. Therefore, this paper shows the experiment done on measurement-based software project evaluation, which helps to identify the required quality while they learn and practice software engineering education. [Den04] says that industries need quality professionals of software engineering; but young graduates lack in recent trends and practice. Some universities provide hands-on experiments 60

Vol 01, Issue 02, December 2012 http://iirpublications.com concerning various software engineering concepts to practice it. Even though it is practiced, the evaluation of their skills in the software engineering disciplines is more subjective; that is, not able to specify where the students have got more skill a concepts. It gives only overall grade; but not in each phases of the software project. 3. BACKGROUND The real background of this experiment is to study and analyze the effects of software engineering metrics, which are considered as measurement units to evaluate software projects of the students. The scope of this study is restricted to BIIT final year students of last six batches (2004 to 2009) for experimental purpose. The intention of this experiment is to study the effects of software measurement-based evaluation of students projects by creating stimuli variables in the process of evaluation. Bosco Institute of Information Technology (BIIT) is a professional educational college offering computer science courses to undergraduate students to shine in IT field. With 15th year of its existence, BIIT is promoting underprivileged rural youth to have decent societal standing. Every year, the finalyear students execute software project for completion of the programme. The students are motivated to initiate the project with project plan and proposal on-time. In every project review meeting they are demanded to apply software engineering discipline discussed in software engineering course to get into the real life of software development. It is ensured by the faculty/guides during the review meeting and directed to move towards better direction. During the project run, they are called for mid-project evaluation to verify the status and progress. Finally, they are asked to submit their project as bound volume and it is evaluated to be included in the final grade. 4. EXPERIMENT The experiment was conducted after analyzing the performances of the students in their final year projects for more than three years. It is found that the students who scored good grade in their projects evaluation have no quality software build at the end. They were not able to identify themselves of their ability in phases Integrated Intelligent Research (IIR)

International Journal of Web Technology ISSN: 2278-2389 of software engineering concepts, which are essential to fix their roles in future. Here, the evaluation of the projects is not the objective. A standard template was used to perform the scoring of the project. But this form of scoring was not relevant to the practices of software engineering, which students learnt in their course of study. 4.1. Procedure The projects with measurements-based evaluation are totally different in procedures from the evaluation without measurements. The slight modification of approach in final year project yielded lot of changes in the process as well as growth of the learners and guides. The steps given below were followed in the experiment to analyze the effects. i. The coordinator of the project explains the go about of the project during the semester. And it explains various project templates to prepare artifacts during the project tenure. The coordinator divides the group and assigns the guide based on the project category as a team or as an individual. Coordinator asks the students to prepare the project plan with number of hours required for each phases of software engineering and scheduling it with a timeframe. The project plan, once is approved by the project guide will be recorded in the actPSP [Cea11] tool for performance appraisals. Then the students initiate the project with work breakdown structure (WBS) to track the event and launch into the SDLC process. The total hours spent in each and every phases of software engineering is recorded into the actPSP tool after meeting the project guide as per the schedule. Both students and guide update the status of the project often in the tool. Finally, the total hours spent for the project, maximum and minimum 61

ii.

iii.

iv.

v.

vi.

vii. viii.

Vol 01, Issue 02, December 2012 http://iirpublications.com hours spent in particular phase are generated as report for the analysis of students performance at the end. ix. The project conducted at documentation with various measurements. evaluation will be the end for both and software build, software engineering

International Journal of Web Technology ISSN: 2278-2389

S.No Artifacts Marks

4.2. Measurements The following criteria were followed to evaluate the project very objectively. Scheme of Software Project Evaluation: BIIT - Yelagiri Hills.
Internal Evaluation # 1 2 3 4 5 Particulars Mid-evaluation Review Record Software Requirement Specification System Design Test case Design (5 x 3 = 15) Total I Marks 20 5 10 10 15 60

Problem Statement 5 System Study (DFD, ERD) 20 SRS (Vision, Use cases) 20 System Design (Arch., EL,) 15 Test Cases 10 Presentation 5 Total 75 Note: Weight : 15% Table 3: Scheme for Document Evaluation Note: 1. 2. 3. Minimum requirement-based five Use cases At least five Test Cases for each Use cases 60% of software requirements must be realized in product.

1 2 3 4 5 6

It defines various measurements for each and every aspect of software engineering disciplines, which are essential in software development lifecycle. 4.3. Tool Adoption Personal Software Process (PSP) [Hum97] [Hum01] mentioned above, helps an individual to know their performance in programming / software development. [Sun04] shows the result of practicing PSP in programming laboratory that yielded considerable growth in students performance. [Sun04] and [Cea11] points out the successful stories of implementing actPSP in software engineering education in Indian academic setup. Hence, actPSP is used as agent to collect data during the project execution. 4.4. Data Gathering The BIIT students were doing their programming projects (C, C++ and JAVA) in actPSP tool environment which capture the data automatically as they progressed in their project. This same tool is used for data collection for the past three batches (20092010, 2010-2011, and 2011-2012), which defines the phases of software engineering with measurement and weightage. At present, actPSP used only for data gathering during the project and taking report at the end for the analysis. In the future, it will be integrated with software engineering activities and actual software development for better usage.

Table 1: Scheme for Internal Evaluation

Technical Evaluation S.No Cases Marks

1 2 3 4

UI Standard Coding Standard Test Case Execution Presentation Total - II Total I & II Table 2: Scheme for Build Evaluation

5 5 15 15 40 100

Integrated Intelligent Research (IIR)

62

Vol 01, Issue 02, December 2012 http://iirpublications.com

International Journal of Web Technology ISSN: 2278-2389

Figure 1: Screenshot of actPSP

The screenshot (figure 1) is used to enter project plan before starting the project and keep on updating the progress status of the project by both students and their project guide. 4.5. Analysis The students were oriented towards the new evaluation scheme and they were motivated and followed keenly to apply these aspects in their project.

After applying measurement based evaluation: Experiments were observed for three years and its pass percentage, minimum mark, and maximum mark of each batch is listed in table. Similarly, three years of data in same category is taken for each batch, before applying a new scheme of evaluation. Hence, this shows us the clear difference and many observations, which support measurement-based project evaluation that helped in the growth of students, in terms of quality of an individual and evaluation.

Integrated Intelligent Research (IIR)

63

Vol 01, Issue 02, December 2012 http://iirpublications.com

International Journal of Web Technology ISSN: 2278-2389

The outcome of this experiment is presented below:

Before Applying Software Measurement Pass % 95 96 94 95 Min. Mark (100) 79 70 72 73.67 Max. Mark (100) 95 98 97 96.67 Avg. Mark

After Applying Software Measurement Pass % 89 90.6 98.3 92.63 Min. Mark (100) 25 32 32 29.67 Max. Mark (100) 75 80 87 80.67 Avg. Mark 52.05 55.32 63.69 57.02

Batch

Batch

2004 2005 2006 Average

72 74 70 72

2007 2008 2009 Average

Table 4-5: Analyzed Report

Figure 2 & Figure 3 show the differences among the evaluation as graph.

Figure 2 & 3: With Measurements

Integrated Intelligent Research (IIR)

64

Vol 01, Issue 02, December 2012 http://iirpublications.com 5. RESULT With comparative study the followings: The pass percentage before applying software measurements for evaluation is high compared to after applying it. The minimum grade and maximum grade of last six batches shows the significant differences between before and after applying the measurements. Before, the criteria for the evaluation are not more objective, rather it was subjective.

International Journal of Web Technology ISSN: 2278-2389 the students decreased in measurement-based evaluation, it is done more objectively to help the students to improve their performance. 8. BIBLIOGRAPHY [Cea11] Julias Ceasor, Sundaram, BIIT, Yelagiri, Tamilnadu, A Practical approach to Software Engineering Education Conference Proceeding, National Conference on IQAC, Sacred Heart College, Tirupattur, 2011 [Sun04] Sundaram M, Thaddeus S, Peter K, PSP-driven programming laboratory for institutes for IT Education in India Conference Proceeding, National Conference, Sacred Heart College, Tirupattur, 2004. [Ivi00] Ivica Crnkovic, Magnus Larsson, Frank Luders, Implementation of Software Engineering Course for Computer Science Students Proceedings of the 7th Asia-Pacific Software Engineering Conference, IEEE, 2000 [Car05] Carlo Ghezzi, Dino Mandrioli, The Challenges of Software Engineering Education ICAC 05, St Louis, Missouri, USA, 2005 [Gre03] Gregory W. Hislop, Heidi J. C. Ellis, Paul E. MacNeil, Cynthia D Tanner, Challenges in Teaching Software Engineering Professionals Online, 33rd ASEE/IEEE Frontiers in Education Conference,2003 [Hum06] Watts S. Humphrey, Teaching Modern Software Engineering, Software Engineering Institute, Carnegie Mellon University, , 2006 [Den04] Dennis J. Frailey, Software Engineering Education 2010: A Challenge to Educators, Proceedings of the 17th Conference on Software Engineering Education and Training (CSEET04), 2004 [Vic96] Victor R. Basili, The Role of Experimentation in Software Engineering: Past, Current, and Future, Proceedings of the 18th International Conference on Software Engineering (ICSE96), University of Maryland, 1996 [Tak93] Takeshigie Miyoshi, Motoei Azuma, An Empirical study of Evaluating Software Development Environment Quality Member 65

Hence, this result makes sense that the subjectivity may boost the grade of one without pinpointing the strength. But the objective evaluation leads to growth of an individual and others even it reduces the grade at overall. 6. RECOMENDATIONS Since this study points out some essential advantages of the project evaluation with some software measurements, any organization can adapt and customize its measurement templates to meet the requirements of an individual organization. Before applying this method of evaluation to an existing project execution system, the students must be motivated and imparted well in knowledge of various software cases (such as problem statement, system study [DFD, ERD, class diagram], system design [architecture, UI, etc.] use cases, test cases, and coding standards etc.) discussed in this report. 7. CONCLUSION The result findings states that implementing various measurements-based evaluation of students software projects will definitely enrich the quality of evaluation of a student performance and help one to find out the most prominent and important skills in software engineering disciplines. The subjective evaluation system on the contrary gave better grade system to the students; but critically fails to identify the student professional who is actually strong enough in fields of the software engineering disciplines. Though the performance grade of Integrated Intelligent Research (IIR)

Vol 01, Issue 02, December 2012 http://iirpublications.com IEEE, Transaction on Software Engineering, Vol. 19, No. 5, May 1993, 1993 [Hum97] Humphrey, Watts S, Introduction to the Personal software Process, AddisonWesley Professional, 1997. [Hum01] Humphrey, Watts S. The Personal Software Process Overview, Practice and Results, Software Engineering Institute, Carnegie Mellon University, Pittsburg, 2001. [Phi03] Johnson Philip M. et al, Beyond the Personal Software Process: Metrics collection and analysis for the differently disciplined, Collaborative Software Development Laboratory, Department of Information and Computer Sciences, University of Hawai', 2003 [p641-646]. [Mas97] Massood Towhidnejad, Thomas Hilburn, Integrating the Personal Software Process (PSP) across the Undergraduate Curriculum, Department of Computer Science, Embry-Riddle Aeronautical University, Daytona Beach, 1997.

International Journal of Web Technology ISSN: 2278-2389

Integrated Intelligent Research (IIR)

66