Pakistan Association of Anthropology, Islamabad, Pakistan

Special Issue
Sci.Int.(Lahore),28(2),2031-2035,2016 ISSN 1013-5316;CODEN: SINTE 8 2031
IMPACT OF HIGHER ORDER THINKING SKILLS OF UNIVERSITY STUDENTS ON
THEIR ACADEMIC PERFORMANCE
Aabida Lateef 1, Muhammad Arshad Dahar 1, Khalida Latif 2
1
Division of Continuing Education, PMAS Arid Agriculture University, Rawalpindi, Pakistan
2
Department of Psychology, Preston University, Islamabad Campus
Corresponding Author‟s Email: drarshad1969@uaar.edu.pk
ABSTRACT: Higher order thinking skills (HOTS) play an important role in enhancement of students’ academic
performance. The major purpose of this paper was to find out the impact of higher order thinking skills of university students
on their academic performance. Multistage random sampling technique was used in this study. Two universities were selected
randomly. From each university 50 Undergraduates and 50 graduates were selected. Total sample comprised of 200 students.
Marzano’s level of higher order thinking skills were used in the development of the questionnaire. For analysis Mean, Standard
deviation, Regression and T-test was applied. Results showed the ‘medium level’ of undergraduates and ‘high level’ of
graduates on Marzano’s HOTS. It further explained the combine strong impact of all items of HOTS on students’ academic
performance. Individually all items were insignificant with no impact on students’ academic performance except ‘comparing’,
‘analyzing perspectives’, ‘decision making’, ‘experimental inquiry and invention’ as P<.05. It was recommended that HOTS
can be enhanced by employing different learning activities during class.

Keywords: Higher order thinking skills, undergraduate and graduate students, academic performance

INTRODUCTION achievement level of students [19]. Horton et al., compared

Thinking skills are of utmost importance in the process of
education. As every person thinks in a different way and it
influences the learning ability. So, in this way we may say
that thinking skills are related to the process of learning.
Resnick‟s study findings in this regard reflect improvement in
students‟ grades. Thinking skills help students to improve
errors in thinking [1]. Higher order thinking skills are defined
to meet the new challenges [2]. Different authors have
showered light on the concept of higher order thinking skills
[3-7]. According to Underbakke et al. and Welingsky higher
order thinking skills(HOTS) have been described as the
capability for problem solution, negotiation of the issues and
analyzing the arguments [8,9]. Vockell et al. emphasized on
the role of higher order thinking skills in the modern world
by not only considering it advancement in facts and concepts,
but to a certain extent it is the capability of an individual to
solve problems for the contribution in society [10]. Lewis &
Smith states higher order thinking skills that involve thinking
in a critical and creative way, solving problems and to make
decisions [11]. Crawford and Brown further describe that
HOTS is based on content, critical and creative thinking [12].
In Higher education thinking skills have been given due
importance due to its utmost role played in the enhanced
learning of students [13,14].
Barba and Merchant examined the higher order skills of
students and found that students performed much better in
science subject who were having higher order skills [15].
Bransford et al., investigated students gain in academic
achievement that possesses higher order thinking skills.
Students shown better performance in reading and writing
skills in the subjects of English and math [16]. Crump et al.
presented the study results after the assessment of students‟
thinking skills that there was a gain in academic achievement
of students who involved their thinking skills [17]. Freseman
findings depicted students‟ positive outcome in the subject of
geography who used divergent thinking skills [18]. Hansler
described the effectiveness of the activities that enhanced
cognitive skills of students and side by side increased the
March-April
the students‟ performance and found that students who
performed cognitive tasks were better in learning than the
students who did not perform cognitive tasks well [20].
Hudgins et al., investigated the effects of critical thinking
skills and found the improved performance of students [21].
Kaganreported the positive effects of higher order thinking
skills [22]. Considerable gains in achievement were made by
the students. Matthews reported the findings of a study that
those students having cognitive abilities performed better on
posttest of experimental group [23]. Mid-Continent Regional
Educational Laboratory examined that higher order thinking
sub-skills improved students‟ learning skills [24]. Pogrow,
described the apt use of HOTS (Higher Order Thinking
Skills) program for the enhancement of learning skills [25].
Robinson significant improvement in the students‟
educational attainment was observed by utilizing the thinking
skill activities [26]. Snapp and Glover found that students‟
output was remarkable in experimental groups as they have
high level of higher order thinking skills as compared to the
control group [27]. Tenenbaum investigated the effectiveness
in Math and science subjects, striking differences were found
in the higher mental processes [28]. Whimbey reviewed that
thinking skills not only improved the performance of
students, but also instilled the skill of problem solving to be
applied in and out of the school [29]. According to Marier‟s
view students who are good at reading use higher order
thinking skills and students who are low achievers are bad at
reading [30]. Cotton suggests that teachers should use various
instructional approaches to enhance thinking skills [31].
Objectives
1.To identify the levels of Higher order thinking skills in
undergraduate and graduate students.
2.To compare higher order thinking skills in undergraduate
and graduate students.
3.To find out the impact of higher order thinking skills on
students‟ academic performance.
Hypotheses
 Pakistan Association of Anthropology, Islamabad, Pakistan
Special Issue
2032 ISSN 1013-5316;CODEN: SINTE 8 Sci.Int.(Lahore),28(2),2031-2035,2016
Hₒ1=There is no significant difference between higher order Abstracting 2.7 1.0 2.7 1.1 4.0 0.8 4.0 0.7
thinking skills in undergraduate and graduate students. decision making 2.9 1.2 3.1 1.2 3.6 1.2 3.9 1.1
Hₒ2=There is no significant difference between students‟ investigation 2.9 1.2 2.7 1.1 3.9 1.0 4.1 0.9
higher order thinking skills and their academic problem solving 2.9 1.1 2.7 1.1 3.6 1.1 3.6 1.0
performance. inquiry 2.7 1.1 2.7 1.1 4.1 0.8 4.0 0.8
Delimitations of study invention 2.9 1.1 2.7 1.1 4.0 1.0 4.3 0.7
1. 2016-2017 session
2. Undergraduate and Graduate students Above table reflects the gender wise mean scores of
3. PMAS-AAUR and Sargodha University undergraduates and graduates. Both male and female
4. CGPAs of last semester undergraduate students have medium level of higher order
thinking skills as their mean scores lie between 2.50-3.49.
Whereas mean scores of graduate students of both male and
METHODOLOGY
female lie between 3.50-5.00 that shows high level of higher
A descriptive research design was applied in the current
order thinking skills. These mean scores are discussed
study. according to the Wierma mean score range [32].
Population Table 3: Impacts of HOTS on academic performance of students
The population of the study consists of undergraduate and (Linear Regression)
graduate students of AAUR and Sargodha University. Sr. Statements M S.D t Sig.
Sample 1 Comparing 3.20 1.29 2.16 .032
Total sample comprised of 200 university students, detail is 2 Classifying 3.25 1.30 1.25 .211
given below: 3 inductive reasoning 3.57 1.30 1.72 .086
Selected Undergraduate Graduate Total 4 deductive reasoning 3.73 1.25 1.30 .193
Universities students students Sample 5 analyzing errors 3.29 1.13 0.98 .328
PMAS-AAUR 50 50 100 6 constructing support 3.18 1.15 0.27 .787
Sargodha 50 50 100 7 analyzing perspectives 3.34 1.24 4.56 .000
University 8 Abstracting 3.36 1.14 1.35 .176
Total Sample 100 100 200 9 decision making 3.39 1.26 3.36 .001
Instrument 10 Investigation 3.42 1.24 1.01 .310
In order to check the levels of Higher order thinking skills, 11 problem solving 3.25 1.18 -1.68 .093
Questionnaire based on Marzano HOTS was developed. 12 experimental inquiry 3.38 1.18 2.56 .011
Questionnaire consisted of 26 items. Students‟ academic 13 Invention 3.50 1.20 2.25 .025
performance was measured through their last semester Above table shows insignificant difference on all items of
CGPAs. Instrument was validated by the experts and it was Marzano‟s HOTS as P>.05. Table further depicted that P<.05
pretested in Islamic International University, Islamabad with on some items of HOTS(comparing, analyzing perspectives,
the sample having same characteristics. Reliability was found decision making, experimental inquiry and invention) that
to be 0.08. shows significant impact on students‟ academic performance.
Data Collection
Questionnaires were administered to undergraduate and Table 4: Impact of HOTS on academic performance of students
graduate level students of PMAS-Arid Agriculture University (ANOVA)
SS Df MS F Sig.
and Sargodha University.
Regression 43.13 13 3.318
DATA ANALYSIS Residuals 26.06 186 0.140 23.679 .000
Mean, standard deviation, regression, t-test and ANOVA was Total 69.19 199
applied for data analysis. Above table reflects that p<.05 that clearly pin points the
Table 1: Range of Mean scores for Levels of HOTS overall impact of all variables of HOTS on academic
Mean Score Range Levels of interpretation
performance of students.
1.00-2.49 Low
2.50-3.49 Medium
3.50-5.00 High
(Source: Adaptation from Wiersma, 2000) [32].
Table 2: Detailed gender wise comparative analysis of HOTS of
Undergraduates and Graduates at University level
Undergraduates Graduates
Statements Males Females Males Females
M S.D M S.D M S.D M S.D
Comparing 2.7 1.1 2.6 1.1 3.8 1.0 3.6 1.3
Classifying 2.6 1.0 2.8 1.3 4.0 0.8 3.7 0.9
Inductive 2.7 0.9 2.5 1.1 4.5 0.7 4.5 0.6
deductive 3.0 1.1 2.9 1.3 4.3 0.7 4.7 0.5
analyzing 2.8 1.1 3.0 1.2 3.5 1.0 3.7 0.9
support 3.0 1.1 2.7 1.0 3.5 1.2 3.4 1.1
analyzing 2.7 1.1 2.6 1.1 4.0 0.9 3.9 1.0
March-Apri
 Pakistan Association of Anthropology, Islamabad, Pakistan
Special Issue
2033 ISSN 1013-5316;CODEN: SINTE 8
Table5: Comparative analysis of HOTS of undergraduates and
Graduates
Students’ scores M S.D Levenes’s t-test for equality of
Test for means
equality of
variance
F sig t df sig
Under Graduates
2.80 .33 98
Male=50
3.79 .05 .161 .873
Under Graduates
2.79 .42 93
Female=50
Graduates .24
Male=50
3.92 98
1.72 .19 -.962 .338
Graduate Female=50 .28
3.97 96
Table shows that the results of Levene‟s test depicts the
F=3.79 that is equal to the p value (p=0.05). It reflects that
undergraduates HOTS have no impact on academic
performance. A minor difference is seen in the mean value of
male and female undergraduates. For graduates Levene‟s test
shows that F=1.72 and P>.05, it clearly describes that there
is no impact of HOTS on academic performance. Mean score
value of graduates reveals that females have little bit more
mean score than males. Over all, graduates have more mean
scores than undergraduates.
DISCUSSION
Current study results have shown that there is an overall
impact of higher order thinking skills on the academic
performance of university students. So, we may say by
rejecting the first hypothesis that there is a significant
difference between higher order thinking skills in
undergraduate and graduate students. First, levels of higher
order thinking skills were measured through their mean
scores that showed “high” mean score range of both male and
female graduate students. Mean scores range of higher order
thinking skills of both male and female undergraduates was
medium. Study conducted by Mahiroğlu was in line with the
current study as it reflects that higher order thinking skills of
students were found to be „average‟ [33]. Some items of
higher order thinking skills shown an insignificant impact on
academic performance and some reflected significant impact.
Second hypothesis is also rejected as there is difference
between students‟ higher order thinking skills and their
academic performance. Heonga et al., investigated that male
and female students have an equal level of higher order
thinking skills in higher education [34]. Comparison was
done and found that there is no significant impact of higher
order thinking skills on the academic performance of
undergraduates and graduates. Several factors affects their
higher order thinking skills such as environment of the
classroom, characteristics of the family and physiology and
intelligence [35-42]. This may be the one major reason for
insignificant impact of HOTS on students‟ academic
performance.
However, the HOTS mean scores of graduates (both male and
female) were more than the undergraduates (both male and
female). Study conducted by King et al. on graduates and
undergraduates reflected the „high‟ mean scores of higher
order thinking skills in males than females [43]. Heong et al.,
March-Apri
Sci.Int.(Lahore),28(2),2031-2035,2016
findings indicated there was a very low positive relationship
between the levels of Marzano‟s higher order thinking skills
with gender, academic achievement and socio economic
status. There exists no significant differences in thinking
levels between male and female students; neither among their
academic achievements [44].
CONCLUSION
Present study results depict that there is a room for
improvement of higher order thinking skills in students of
undergraduate level. So, due attention should be given for the
improvement of higher order thinking skills of students. In
third regard, teachers can play vital role in enhancing higher
order thinking skills by engaging students in task oriented
work [45]. Moreover, if students internalize competencies
their thinking will be enhanced and they will become more
self-directed, self-disciplined, self-monitored thinkers [46].
Students‟ learning can be facilitated by enhancing higher
order thinking skills [47-49]. Moreover, higher order thinking
skills may be enhanced by applying learning activities during
the class so that students may indulge in in-depth thinking
process [50].
REFERENCES
1. Resnick, L.B. “Education and learning to think,”
Washington, DC: National Academy Press, (1987)
2. Rajendran, N.S. Teaching & Acquiring Higher-Order
Thinking Skills: Theory & Practice. Penerbit Universiti
Pendidikan Sultan Idris. (2008)
3. Lawrenz, F. Science teaching techniques associated with
higher order thinking skills. Journal of Research in
Science Teaching, 27(9): 835-47(1990)
4. Callison, D. Thinking (higher order) skills. School
Library Media Activities Monthly, Abstract retrieved
September 12, 2005, from Proquest database. 18(8): 38-
40(2002)
5. Hernandez, D. F. Developing and assessing higher order
thinking skills. Monograph 46.Quezon City: Institute for
Science and Mathematics Education Development.
(1991)
6. Zoller, U. Are lecture and learning compatible? Journal
of Chemical Education, 70(3): 195-197(1993)
7. Zoller, U., Lubezky, A., Nakhleh, M. B., Tessier, B. &
Dori, Y. J. Success on algorithmic and LOCS vs.
conceptual chemistry exam questions. Journal of
Chemical Education, 72(11): 987-989(1995)
8. Underbakke, M., Borg, J. M., & Peterson, D.
Researching and developing the knowledge base for
teaching higher order thinking. Theory into practice,
32(3): 138-146(1993)
9. Wenglinsky, H. "The link between teacher classroom
practices and student academic performance." Education
policy analysis archives 11:12(2002)
10. Vockell, E. & van Deusen, R. M. The computer and
higherorder thinking skills. California: Mitchell
Publishing, (1989)
11. Lewis, A., & Smith, D. Defining higher order thinking.
Theory into Practice, 32(3): 131−137(1993)
12. Crawford, C. M., & Brown, E. Focusing upon higher-
order thinking skills: Web Quests and the learner-
 Pakistan Association of Anthropology, Islamabad, Pakistan
Special Issue
2034 ISSN 1013-5316;CODEN: SINTE 8
centered mathematical learning environment. (ERIC ED
474086). 6(2002)
13. Dewey, J. How we think. New York: DC Heath & Co.
pp.27-28(1910)
14. Tsui, L. Fostering critical thinking through effective
pedagogy: Evidence from four institutional case studies.
The Journal of Higher Education, 73(6): 740-763(2002)
15. Barba, R. H., and Merchant, L. J. "The Effects of
Embedding Generative Cognitive Strategies in Science
Software." Journal of Computers in Mathematics and
Science Teaching, 10(1): 59-65(1990)
16. Bransford, J. D.; Burns, M. S.; Delclos, V. R.; and Vye,
N. J. "Teaching Thinking: Evaluating Evaluations and
Broadening the Data Base." Educational Leadership,
44(2): 68-70(1986)
17. Crump, W. D.; Schlichter, C. L.; and Palk, B. E.
"Teaching HOTS in the Middle and High School: A
District-Level Initiative in Developing Higher Order
Thinking Skills." Roeper Review 10(4): 205-211(1988)
18. Freseman, R. D. Improving Higher Order Thinking Of
Middle School Geography Students By Teaching Skills
Directly. Fort Lauderdale, Fl: Nova University, (Ed 320
842) (1990)
19. Hansler, D. D. Studies On The Effectiveness Of The
Cognition Enhancement Technique For Teaching
Thinking Skills, (Ed 266 432) (1985)
20. Horton, J., and Ryba, K. "Assessing Learning with Logo:
A Pilot Study." The Computing Teacher 14(1): 24-
28(1986)
21. Hudgins, B., and Edelman, S. "Teaching Critical
Thinking Skills to Fourth and Fifth Graders Through
Teacher-Led Small-Group Discussions." Journal Of
Educational Research 79(6): 333-342(1986)
22. Kagan, D. M. "Evaluating a Language Arts Program
Designed to teach Higher Level Thinking Skills."
Reading Improvement 25(1): 29-33(1988)
23. Matthews, D. B. "The Effect of a Thinking-Skills
Program on the Cognitive Abilities of Middle School
Students." Clearing House 62(5): 202-204(1989)
24. Mid-Continent Regional Educational Laboratory. Report
of Thinking Skill Instructional Activities. Denver, CO:
Mid-Continent Regional Educational Laboratory, Inc.,
(1985)
25. Pogrow, S. "HOTS: A Thinking Skills Program for at
Risk Students." PRINCIPAL 67(4): 19-24(1988)
26. Robinson, I. S. A Program to Incorporate High-Order
Thinking Skills into Teaching and Learning for Grades
K-3. Fort Lauderdale, FL: Nova University, (1987)
27. Snapp, J. C., and Glover, J. A. "Advance Organizers and
Study Questions." Journal Of Educational Research
83(5): 266-271(1990)
28. Tenenbaum, G. "The Effect of Quality of Instruction on
Higher and Lower Mental Processes and on the
Prediction of Summative Achievement." Journal Of
Educational Research, 80(2): 105-114(1986)
29. Whimbey, A. "Test Results From Teaching Thinking." In
A. L. Costa (ed.). Developing Minds: A Resource Book
for Teaching Thinking. Alexandria, VA: Association for
Supervision and Curriculum Development, 269-
271(1985)
March-Apri
Sci.Int.(Lahore),28(2),2031-2035,2016
30. Marier, R. Reading Comprehension Techniques for
Improving Students' Success in Extracting Useful
Knowledge from Text. New Jersey: Pearson Education,
Inc. (2000)
31. Cotton, K. Teaching Thinking Skills. School
Improvement Research Series (SIRS). Northwest
Regional Educational Laboratory. (2001)
32. Wiersma, W. Research Methods in Education: An
Introduction. 7th ed. Boston, Allyn &Bacon, (2000)
33. Mahiroğlu, A. Teachers‟ Opinions on Students‟ Higher
Order Thinking Skills Gazi University, Turkey.Online
submission. (2007)
34. Heong, Y.M., Yunos, J. B., & Hassan,R. B. The
perception of the level of higher order thinking skills
among technical education students. Paper presented at
the 2011 International Conference on Social Science and
Humanity, IPEDR. 5: (2011)
35. Chini, J.J., Carmichael, A., Rebello, N.S. and
Puntambekar, S. Does the teaching/learning interview
provide an accurate snapshot of classroom learning?. In
Physics Education Research Conference, 29-30(2009)
36. Fearon, D.D., Copeland, D. and Saxon, T.F. The
relationship between parenting styles and creativity in a
sample of Jamaican children. Creativity research
journal, 25(1): 119-128(2013)
37. Horan, R. The relationship between creativity and
intelligence: A combined yogic-scientific approach.
Creativity Research Journal, 19(2-3):179-202(2007)
38. Lather, A.S., Jain, S. and Shukla, A.D. Student‟s
Creativity in Relation to Locus of Control: a Study of
Mysore University, India. International Journal of
Indian Psychology, 2(1): 146-165(2014)
39. Lim S, Smith J. The Structural Relationships of
Parenting Style, Creative Personality, and Loneliness.
Creativity Res. J. 20(4):412-419(2008)
40. Pannells, T.C. and Claxton, A.F. Happiness, creative
ideation, and locus of control. Creativity Research
Journal, 20(1): 67-71(2008)
41. Pascarella, E.T., Wang, J.S., Trolian, T.L. and Blaich, C.,
How the instructional and learning environments of
liberal arts colleges enhance cognitive development.
Higher Education, 66(5):569-583(2013)
42. Silvia, P.J., Creativity and intelligence revisited: A latent
variable analysis of Wallach and Kogan (1965).
Creativity Research Journal, 20(1): 34-39(2008)
43. King, P.M., Wood, P.K., & Mines, R.ACritical thinking
among college and graduate students. The Review of
Higher Education. Winter 1990, 13(2):167‐186(1990)
44. Heong,Y.M., Widad B. O., Yunos, J. B., Tee T. K.,
Razali B. H., and Mimi M. B.M. The Level of Marzano
Higher Order Thinking Skills among Technical
Education Students. International Journal of Social
Science and Humanity, 1(2): 121-125(2011)
45. Zohar, A., & Y. Dori. Higher order thinking skills and
low‐achieving students: are they mutually exclusive?
Journal of the Learning Sciences, 12(2):145–181(2003)
46. Paul, R. & Elder, L. Guide for educators to critical
thinking competency standards: Standards, principles,
performance indicators, and outcomes with a critical
thinking master rubric. Foundation for Critical Thinking.
(2005)
 Pakistan Association of Anthropology, Islamabad, Pakistan
Special Issue
2035 ISSN 1013-5316;CODEN: SINTE 8 Sci.Int.(Lahore),28(2),2031-2035,2016
47. Donald, J. G. Learning to think: Disciplinary Thinking in a Blended Course. Online Learning: Official
perspectives. The Jossey-Bass Higher and Adult Journal of the Online Learning Consortium, 18(4):
Education Series. San Francisco, CA: Jossey-Bass. (2014)
(2002) 50. Donawa, A. M. Critical thinking and the impact of
48. Perkins, D., Jay, E., & Tishman, S. New conceptions of critical thinking instruction on minority engineering
thinking: From ontology to education. Educational students at a public urban higher education institution,
Psychologist, 28(1): 67-85(1993) (2003)
49. McDonald, P., Straker, H., Schlumpf, K. and Plack, M.,
Learning Partnership: Students and Faculty Learning
Together to Facilitate Reflection and Higher Order

March-Apri