Tuesday, March 19, 2013

Some More on the THE World University Rankings 2


I have calculated the mean scores for the indicator groups in the 2012-13 Times Higher education. world university rankings. The mean scores are for the 400 universities included in the published 2012-13 rankings are:

Teaching   41.67
International Outlook 52.35
Industry Income 50.74
Research 40.84
Citations 65.25

For Industry Income, N is 363 since 37 universities, mainly in the US, did not submit data. This might be a smart move if the universities realized that they were likely to receive a low score. N is 400 for the others.

There are considerable differences between the indicators which are probably due to Thomson Reuters' methodology. Although THE publishes data for 200 universities on its website and another 200 on an iPad/iPhone app there are in fact several hundred more universities that are not included in the published rankings but whose scores are used to calculate the overall mean from which scores for the ranked universities are derived.

A higher score on an indicator means a greater distance from all the institutions in the Thomson Reuters database.

The high scores for citations mean that there is a large gap between the top 400 and the lesser places outside the top 400.

I suspect that the low scores for teaching and research are due to the influence of the academic survey which contributes to both indicator clusters. We have already seen that after the top six, the curve for the survey is relatively flat.

The citations indicator already has a disproportionate influence contributing 30 % to the overall weighing. That 30 % is of course a maximum. Since universities on average are getting more for citations than the the  indicators, it has in practice a correspondingly greater weighting.

Friday, March 15, 2013

Some More on the THE World University Rankings 2012-13

Here are some observations based on a simple analysis of the Times Higher Education World University Rankings of 2012-13.

First, calculating the Pearson correlation between the indicator groups produces some interesting points. If a ranking is valid we would expect the correlations between indicators to  be fairly high but not too high. If the correlations between indicators are above .800 this suggests that they are basically measuring the same thing and that there is no point in having more than one indicator. On the other hand it is safe to assume that if an indicator does measure quality or desired characteristics in some way it will have a positive relationship with other valid indicators.

One thing about the 2012-13 rankings is that the relationship between international outlook (international faculty, students and research collaboration)  and the other indicators is negative or very slight. With teaching it is .025 (not significant), industry income .003 (not significant), research .156 and citations 158. This adds to  my suspicion that internationalisation, at least among those universities that get into the world rankings, does not per se  say very much about quality.

Industry income correlates modestly with teaching (.350) and research (.396), insignificantly with international outlook (.003) and negatively and insignificantly with citations (-.008).

The correlation between research  and teaching is very high at .905. This may well be because  the survey of academic opinion contributes to the teaching and the research indicators. There are different questions -- one about research and one about postgraduate supervision -- but the difference between the responses is probably quite small.

It is also very interesting that the correlation between scores for research and citations is rather modest at .410. Since volume of publications, funding and reputation should contribute to research influence, which is what citations are supposed to measure, this suggests that the citations indicator needs a careful review.

Teaching, research and international outlook are composites of several indicators. It would be very helpful if THE or Thomson Reuters released the scores for the separate indicators.

Sunday, March 10, 2013


The California Paradox

Looking at the Times Higher Education reputation rankings, I noticed that there were two Californian universities in the  superbrand six and seven in the top 50. This is not an anomaly. A slightly different seven can be found in the THE World University Rankings. California does even better in the Shanghai ARWU with three in the top six and 11 in the top 50. This is a slight improvement on 2003 when there were ten. According to ARWU, California would be the second best country in the world for higher education if it became independent.
California’s performance is not so spectacular according to QS who have just four Californian institutions in their top fifty, a fall from 2004 when they had five (I am not counting the University of California at San Francisco which, being a single subject medical school, should not have been there). Even so it is still a creditable performance.
But, if we are to believe many commentators, higher education in California, at least public higher education, is dying if not already dead.

According to Andy Kroll in Salon:

"California’s public higher education system is, in other words, dying a slow death. The promise of a cheap, quality education is slipping away for the working and middle classes, for immigrants, for the very people whom the University of California’s creators held in mind when they began their grand experiment 144 years ago. And don’t think the slow rot of public education is unique to California: that state’s woes are the nation’s".

The villains according to Kroll are Californian taxpayers who refuse to accept adding to a tax burden that is the among the highest in the world. 
It is surprising that the death throes of higher education in California have gone unnoticed by the well known international rankers.
It is also surprising that public and private universities that are still highly productive and by international standards still lavishly funded exist in the same state as secondary and elementary schools that are close to being the worse in the nation in terms of student performance. The relative and absolute decline in educational achievement is matched by a similar decline in the overall economic performance of the state.

It may be just a matter of time and in the coming decades Californian universities will follow primary and secondary education into irreversible decline.

 

 

 
Preserving data

Times Higher and QS have both renovated their ranking pages recently and both seem to have removed access to some data from previous years. THE used to provide links to the Times Higher Education (Supplement) - Quacquarelli Symonds rankings of 2004-2010 but apparently not any more. QS do not seem to give access to these rankings before 2007. In both cases, I will update if it turns out that there is a way to get to these rankings.


There is, however,  a site which has the rankings for the top 200 of the THES - QS Rankings of 2004-2007.

Wednesday, March 06, 2013

The THE Reputation Rankings

Times Higher Education have published their reputation rankings based on data collected from the World University Rankings of 2012.

They are not very interesting. Which is exactly what they should be. When rankings show massive changes from one year to another a certain amount of scepticism is required.

The same six, Harvard, MIT, Stanford, Berkeley, Oxford and Cambridge are well ahead of everybody else as they were in 2012 and in 2011.

Taking a quick look at the top fifty, there is little movement between 2011 and 2013. Four universities for the US, Japan, Netherlands and Germany have dropped out. In their place there is one more from Korea and from the the UK and two more from Australia.

I was under the impression that Australian universities were facing savage cuts in research funding and were going to be deserted by international students and researchers..

Maybe it is the other universities that are being cut. or maybe a bit of blood letting is good for the health.

I also noticed that the number of respondents went down a bit in 2012. It could be that the academic world is beginning to suffer from ranking fatigue.

Saturday, March 02, 2013

GRE Country Ranking: Verbal Reasoning

Arranging the mean scores for the 2011-12 GRE Verbal Reasoning test, we can see that the bottom looks rather similar to  the Quantitative Reasoning test. It comprises African and Arab countries. The top is very different, with five out of six places held by countries where English is currently the native language of a majority of the population.




1Australia158.40
2New Zealand157.30
3=Singapore157.10
3=Ireland157.10
3=UK157.10
6Canada156.00
7Netherlands155.50
8Belgium155.00
9US White154.10
10Switzerland153.70
11Romania153.50
12=Sweden153.30
12=South Africa153.30
14Bulgaria153.20
15Norway153.10
16USA152.90
17Argentina152.80
18France152.70
19US Asian152.60
20Austria152.50
21Germany152.30
22Denmark152.30
23Italy152.20
24Croatia151.70
25Finland151.70
26Uruguay151.60
27US American Indian151.50
28=Czech Republic151.40
28=Israel151.40
28=Trinidad151.40
31Hungary151.20
32=Portugal150.90
32=Spain150.90
34Poland150.40
35Lithuania150.30
36US Hispanic150.20
37Iceland149.80
38US Mexican149.70
39=Malaysia149.50
39=Barbados149.50
41Greece149.40
42=Costa Rica149.10
42=Philippines149.10
44Guatemala149.00
45Brazil148.90
46=Zimbabwe148.80
46=Jamaica148.80
48US Porto Rican148.70
49=Georgia148.60
49=Bosnia - Herzogovina148.60
51Guyana148.60
52Moldova148.40
53Macedonia148.30
54=Peru148.20
54=Mexico148.20
54=Bahamas148.20
57Chile148.00
58Belarus147.90
59Russia147.80
60=Latvia147.70
60=Colombia147.70
62Albania147.60
63=Estonia147.60
63=Venezuela147.60
63=El Salvador147.60
63=Cuba147.60
63=St Lucia147.60
68=Hong Kong147.50
68=South Korea147.50
70=Ukraine147.40
70=Bahrain147.40
72=Serbia147.30
72=Bolivia147.30
74=Eritrea147.20
74=Honduras147.20
76=Zambia147.10
76=Afghanistan147.10
78Pakistan147.00
79Panama146.80
80US Black146.70
81=Ecuador146.50
81=Kenya146.50
83Nigeria146.40
82=Morocco146.30
82=Senegal146.30
82=Nicaragua146.30
87=Cyprus146.10
87=Dominican Republic146.10
87=Sierra leone146.10
90Uzbekistan146.00
91China145.90
92Mongolia145.80
93=Vietnam145.70
93=Myanmar145.70
95Kazakhstan145.60
96=Togo145.50
96=Ghana145.50
98=Tunisia145.20
98=Uganda145.20
100Niger145.10
101=Burkina Faso145.00
101=Malawi145.00
103Kyrgyzstan144.90
104=India144.70
104=Indonesia144.70
106Cote d'Ivoire144.60
107=Japan144.50
107=Nepal144.50
107=Haiti144.40
110=Taiwan144.20
110=Bangladesh144.20
110=Ethiopia144.20
110=Benin144.20
110=Congo DR144.20
115Turkey144.10
116=Egypt143.80
116=Azerbaijan143.80
118Armenia143.70
119=Turkmenistan143.50
119=Cameroon143.50
121=Macao143.40
121=Sri Lanka143.40
121=Tanzania143.40
124Thailand142.80
125=Syria142.70
125=Rwanda142.70
127Qatar142.50
128Algeria141.60
129Jordan141.40
130Oman141.30
131Yemen141.00
132Congo Republic140.90
133Kuwait140.80
134Sudan140.60
135=UAE140.30
135=Mali140.30
137Namibia140.20
138Saudi Arabia137.40