Academic Ranking of World Universities

(ARWU)

February 16, 2006

By Professor Nian Cai Liu

Institute of Higher Education and Center for World-Class Universities

Shanghai Jiao Tong University, China

1

Outline
Purposes of ARWU
Methodologies of ARWU
Results of ARWU
Problems with ARWU
Future Efforts of ARWU
2

Purposes of ARWU

Dream of Chinese for WCU

WCU is a dream for generations of Chinese.

Its not only for pride, but also for the future
of China.

Recently, Chinese government has launched

several initiatives for research universities.
The best-known one is specially designed to
build WCU.
4

Extra Funding in 985 Project (2001-2003)

Institution
Peking Univ.
Tsinghua Univ.

Funding (Billion RMB)

1.8

Fudan Univ.
Nanjing Univ.
Shanghai Jiao Tong Univ.
Zhejiang Univ.

1.2-1.4

Univ. Sci. Tech. China (CAS)

Xian Jiao Tong Univ. (Shanxi)
Harbin Inst. Tech. Heilongjiang)

0.9-1.0
5

Goals of Top Chinese Universities

Many top Chinese universities have setup

their strategic goals as WCU.

Most of them has also set a time table for

reaching the goal of WCU. For example:
2016 for Peking University
2020 for Tsinghua University
6

Questions About WCU

Is there a clear definition for WCU?

Are there any common characteristics for WCU?

How many WCU should there be in the world?

What are the positions of Chinese universities in

the world?

How can Chinese universities improve

themselves to reach the goal of WCU?
7

Characteristics of WCU

There is no clear definition for WCU. Nevertheless,

there seem be to common characteristics:

Top departments/programs of the world

Excellent research of international standard

Best professors and students from the world

Academic freedom and atmosphere of innovation

Adequate financial resources and facilities

Excellent leadership and effective governance

-----8

Ranking of World Universities

Our original purpose of doing the Academic

Ranking of World Universities (ARWU) was to find
out the position of Chinese universities in the
world and the gap between them and WCU.

ARWU was put on the internet upon on the

encouragement of colleagues from all over the
world. There have been more than 2,000,000
visitors since 2003, an average of 2000 every day.
9

Features of ARWU

Its done for pure academic interests, without

any external support. It has nothing to do
with any commercial activities.

Only non-subjective indicators and

internationally comparable data that everyone
could verify in some way were used.

10

Cautions About Ranking

The quality of universities may not be precisely

measured by mere numbers.
The quality of world universities may not be
accurately compared because of the huge
differences of various types of universities in
different countries.
The choice of indicators and their weights make
significant differences to the final ranking
results.
11

Methodologies of ARWU

12

Selection of Universities

Any university that has any Nobel Laureates, Fields

Medals, Highly Cited Researchers, or papers published
in Nature or Science.

Major universities of every country with significant

amount of papers indexed by SCIE, SSCI and AHCI.

Number of universities scanned: >2000

Number of universities actually ranked: >1000

Number of ranked universities on our web: 500

13

Design of Ranking Criteria

It would be impossible to rank the quality of

education, administration, campus culture, and
national contribution etc.

We chose to rank universities worldwide by their

academic or research performance, which is a
good indication of its international reputation.

14

Ranking Criteria and Weights

Criteria

Quality of
Education

Quality of Faculty

Indicator

Code

Weight

Alumni of an institution winning Nobel

Alumni
Prizes and Fields Medals

10%

Staff of an institution winning Nobel

Prizes and Fields Medals

Award

20%

Highly cited researchers in 21 broad

subject categories

HiCi

20%

Articles published in Nature and Science*

N&S

20%

Articles in SCIE SSCI and AHCI

SCI

20%

Academic performance with respect to

the size of an institution

Size

10%

Research Output

Size of Institution
Total

100%

For institutions specialized in humanities and social sciences such as London School of Economics,
N&S is not considered, and the weight of N&S is relocated to other indicators.

15

Scoring Procedures

For each indicator, the highest scoring institution is assigned

a score of 100, and other institutions are calculated as a
percentage of the top score.
The distribution of data for each indicator is examined for
any significant distorting effect; standard statistical
techniques are used to adjust the indicator if necessary.
Scores for each indicator are weighted to arrive at a final
overall score for an institution. The highest scoring
institution is assigned a score of 100, and other institutions
are calculated as a percentage of the top score.
The scores are then placed in descending order. An
institutions rank reflects the number of institutions that sit
above it.
16

Definition of Indicator: Alumni

The total number of the alumni of an institution winning

Nobel Prizes and Fields Medals.

Alumni are defined as those who obtain bachelor,

Masters or doctoral degrees from the institution.
Different weights are set according to the periods of
obtaining degrees. The weight is 100% for alumni of
1991-2000, 90% for alumni of 1981-1990, 80% for
alumni of 1971-1980, and so on.

If a person obtains more than one degrees from an

institution, the institution is considered once only.
17

Definition of Indicator: Award

The total number of the staff of an institution winning Nobel prizes

in physics, chemistry, medicine and economics and Fields Medal in
Mathematics.
Staff is defined as those who work at an institution at the time of
winning the prize.
Different weights are set according to the periods of winning the
prizes. The weight is 100% for winners since 2001, 90% for
winners in 1991-2000, 80% for winners in 1981-1990, 70% for
winners in 1971-1980, and so on.
If a winner is affiliated with more than one institution, each
institution is assigned the reciprocal of the number of institutions.
For Nobel prizes, if a prize is shared by more than one person,
weights are set for winners according to their proportion of prize.
18

Definition of Indicator: HiCi

The number of highly cited researchers in 21 broad

subject categories in life sciences, medicine,
physical sciences, engineering and social sciences.

The definition of categories and detailed procedures

can be found at the website of Institute of Scientific
Information.

The total number of HiCi is about 5000, about 4000

of which is university staff.
19

Definition of Indicator: N&S

The annual average number of articles published in

Nature and Science in the past five years.

To distinguish the order of author affiliation, a weight of

100% is assigned for corresponding author, 50% for
first author (second author if the first author is the same
as corresponding author), 25% for the next author, and
10% for other authors.

Only publications of article type are considered.

20

Definition of Indicator: SCI

Total number of articles indexed in Science

Citation Index-expanded (SCIE), Social Science
Citation Index (SSCI), and Arts & Humanities
indices (AHCI) in the past year.
A weight of 2 is assigned to articles indexed in
SSCI and AHCI to compensate the bias against
humanities and social sciences.
Only publications of article type are considered.
21

Definition of Indicator: Size

The sub-total scores of the above five indicators

divided by the number of full-time equivalent
academic staff.

If the number of academic staff for institutions of a

country cannot be obtained, the total scores of the
above five indicators is used.

For ranking 2005, the number of full-time equivalent

academic staff is obtained for institutions in USA,
China, Australia, Italy, Netherlands, Sweden, and
Belgium etc.
22

Main Sources of Data

Nobel laureates:
http://www.nobel.se

Fields Medals:
http://www.mathunion.org/medals/

Highly cited researchers:

http://www.isihighlycited.com

Articles published in Nature and Science:

http://www.isiknowledge.com
Articles indexed in SCIE, SSCI, and AHCI:
http://www.isiknowledge.com
23

Results of ARWU

24

Top 500 Universities by Region

Region

Top 20

Top 100

Top 200

Top 300

Top 400

Top 500

Americas

17

57

100

140

165

198

Europe

35

79

123

168

205

AsiaPacific

23

36

65

93

300

400

500

Africas
Total

20

100

202

25

Top 500 Universities by Country

Country

Top 20 Top 100 Top 200 Top 300 Top 400 Top 500

United States

17

53

90

119

140

165

United Kingdom

11

14

30

36

40

Japan

13

24

34

Germany

16

23

33

40

Canada

17

19

23

France

13

19

21

Sweden

11

11

Switzerland

Netherlands

11

12

10

Australia

10

14

19

China

15

18
26

Distribution of Total Score

27

Distribution of Scores of Indicators

28

Pearson Coefficients
Total
Score

Alumni
Score

Award
Score

HiCi
Score

N&S
Score

SCI
Score

Total Score

1.00

Alumni Score

0.80

1.00

Award Score

0.84

0.76

1.00

HiCi Score

0.90

0.60

0.65

1.00

N&S Score

0.93

0.67

0.70

0.86

1.00

SCI Score

0.81

0.55

0.50

0.68

0.74

1.00

Size Score

0.83

0.68

0.73

0.70

0.77

0.56

Size
Score

1.00

29

Performance of Selected Universities

Range of
Universities

Top 20 21-100 101-200 201-300 301-400 401-500

Peking Tsinghua
(200-300) (150-200)

Average No. of
Alumni

8.5

1.6

0.4

0.2

0.2

0.0

0.0

0.5

Average No. of
Award

4.4

0.6

0.1

0.0

0.0

0.0

0.0

0.0

Average No. of
HiCi

56.5

13.6

5.3

2.3

1.0

0.8

0.0

0.0

Average No. of
N&S

27.6

6.7

2.7

1.5

0.7

0.4

1.0

0.8

2400

1600

1100

800

600

2500

2800

Average No. of
4300
SCI

30

Problems with ARWU

31

Methodological: Social Sciences

Many well-known institutions specialized in humanities

and social sciences are ranked relatively low.
Nevertheless, if a university specialized in social sciences
and humanities had Nobel Laureates in economics and
Highly Cited Researchers in social sciences, it should have
good standing.
Since 2004, the indicator of N&S is not considered for
institutions specialized in humanities and social sciences,
its weight is relocated to other indicators.
Since 2005, a weight of 2 for articles indexed by SSCI and
AHCI is considered.
32

Methodological: Language Bias

English is the language of international academic

community.

Any ranking based on academic performance will

be biased towards institutions in Englishspeaking countries.

Possible solution: papers published in non-native

languages are offered a special weight.

33

Methodological: Award and Alumni

Universities started after the 1911 do not have a fair

chance.
Disciplines not related to the awarding fields do not
have a fair chance. Other important awards include
Abel, Pulitzer, Turing, Tyler, etc.
Institutions for winning awards and those for doing
the researches may not be the same.
Institutions for obtaining degrees and those for
pursuing the studies may not be the same.
Postdoctoral training is not considered.
34

Methodological: Per Capita Performance

The weight of the Size indicator for per capita

performance is rather low. Large institutions have
relatively high positions in the ranking.

However, its very difficult to obtain internationally

comparable data on the number of academic staff.

The types of academic staff: such as purely teaching

staff, teaching and research staff, purely research staff.

The ranks of academic staff: such as professor, associate

professor, reader, lecturer, instructor, scientist etc.
35

Technical: Attributions

Many universities have more than one commonly used

names: such as Virginia Tech and Virginia Polytechnic and
State University.
Variations due to translation: such as Univ Koln and Univ
Cologne, Univ Vienna and Univ Wien.
Abbreviated names: such as ETH Zurich for Swiss Federal
Institute of Technology Zurich.
Some authors only write their departmental or institute
name without mentioning their university name.
36

Technical: Definition of Institution

University systems: such as Univ California

system, Univ London system.
Affiliated institutions and research organizations:
such as Ecole Polytechnique Montreal (affiliated
to University of Montreal), CNRS Labs (affiliated
to French universities).
Teaching and affiliated Hospitals: complex!
Our answer: according to authors expression.
37

Other Technical Problems

Merging, splitting, inheriting, discontinuing, namechanging of institutions such as:

Univ Kwazulu-Natal in South Africa merged from Univ

Natal and Univ Durban-Westville.
University of Innsbruck in Austria splitted into Univ
Innsbruck and Innsbruck Medical Univ.
Humboldt Univ Berlin and Free Univ Berlin inheriting the
Nobel Prizes of the Berlin University before world war II.

Vrjie Universiteit Brussel and Universite Libre Bruxelles

share the same English name of Free University of
Brussels.
38

Future Efforts for ARWU

39

Future Efforts

Study all the above mentioned problems and

continuously improve the ranking methodologies.

Establish more comprehensive databases of WCU.

Update ARWU annually (in August) .

Provide ranking of broad subject areas such as

physical sciences, social sciences, life sciences,
clinical medicine, and engineering etc.
40

Future Efforts (Conted)

Provide ranking of universities specialized or

oriented in engineering, medicine, etc. based on
the scientific classification of world universities.

Provide ranking with a much higher weight of

the Size indicator once internationally
comparable data on the number of academic
staff were obtained.

41

Thank you very much

for your attention!
http://ed.sjtu.edu.cn/ranking.htm
http://ed.sjtu.edu.cn/en/

42