Measuring Quality Outcomes in Higher Education

Transcription

1 Australian Government/ HEFCE/ European Commission 16 October 2015, Singapore Measuring Quality Outcomes in Higher Education Simon Marginson Professor of International Higher Education Director, ESRC/HEFCE Centre for Global Higher Education (Nov 2015) UCL Institute of Education University College London, UK

2 Coverage High Participation Systems (HPS) of higher education: Participation and system stratification Research environment What are the systemic drivers of quality operating at global level? Present global rankings The lacuna: Credible comparative metrics on learning in higher education Problems of proxies and indirect measures Post-AHELO landscape

3 A time of movement and change in world higher education

4 Participation is growing at 1% a year Gross Tertiary Enrolment Ratio (UNESCO 2015) World North America and Western Europe Sub-Saharan Africa

5 Same trend across the globe: GTER by world region, 1995/2012 North America & Western Europe Central & Eastern Europe Latin America & Carribean East Asia & Pacific Arab States Central Asia South & West Asia 6 23 Sub-Saharan Africa 4 8 WORLD

6 Growth in participation to come World and Asian middle class (billions) Source: Brookings / OECD projection in World Asia Middle class persons are defined as persons living on USD $ per day, PPP

7 Tendency towards bifurcation and stratification of HPS Elite HEIs student selecting upward push (aspirations) downward pull (scarcity of resources and status) Middle Sector Non-elite demand absorbing

8 Stratifying effects in system design Competition World-Class University movement Under-regulation of quality in mass HE sectors More fragmented and diverse offerings, cross-border etc Indifference to equity issues (access to elite HEIs, cognitive formation in mass sector, etc) BUT POLICY COUNTER-WEIGHTS Horizontal system design features, system architectures Common mission designations ( research university ) Sector-wide promotion nationally and offshore Funding parcels, nuanced specialisations sustain middle sector

9 Students enrolled outside their country of citizenship, millions, OECD data,

10 Students enrolled outside their country of citizenship, millions, other OECD, 8% Belgium, 1% Switzerland, 1% Korea, 1% Netherlands, 1% South Africa, 2% Austria, 2% Italy, 2% 0.8 New Zealand, 2% China, 2% Spain, 2% OECD data, 2014 other non-oecd, 17% 1.3 Russia, 4% Japan, 3% 1.7 Canada, 5% France, 6% USA, 16% UK, 13% Australia, 6% Germany, 6%

11 Leiden Ranking world top 15 Universities ranked 1-15 in world Papers % papers in top 10% Number of papers in top 1% top 10% 1 Harvard U USA 31, Stanford U USA 14, U Toronto Canada 19, U Michigan USA 17, U California, Berkeley USA 11, MIT USA 10, U California, Los Angeles USA 14, Johns Hopkins U USA 14, U Oxford UK 12, U Washington, Seattle USA 13, U Pennsylvania USA 12, U California San Diego USA 11, U Cambridge UK 12, Columbia U USA 11, U California, S. Francisco USA 10,

12 Stronger research universities in Europe European universities in ARWU top 80 in 2004 (14) Fed Instit Tech Zurich SWITZERLAND U Utrecht NETHERLANDS Paris 6 P&M Curie FRANCE TU Munich GERMANY Karolinska Instit SWEDEN Paris 11 Sud FRANCE U Munich GERMANY U Zurich SWITZERLAND U Copenhagen DENMARK Leiden U NETHERLANDS U Oslo NORWAY U Helsinki FINLAND Uppsala U SWEDEN U Goettingen GERMANY European universities in ARWU top 80 in 2015 (19) eq 58 eq 61 eq 67 eq Fed Instit Tech Zurich SWITZERLAND U Copenhagen DENMARK Paris 6 P&M Curie FRANCE Paris 11 Sud FRANCE Heidelberg U GERMANY Karolinska Instit SWEDEN TU Munich GERMANY U Munich GERMANY U Zurich SWITZERLAND U Utrecht NETHERLANDS U Geneva SWITZERLAND U Oslo NORWAY Uppsala U SWEDEN U Helsinki FINLAND Ghent U BELGIUM Ecole Normale Superieure FRANCE Aarus U DENMARK U Groningen NETHERLANDS Stockholm U SWEDEN

13 Dynamism in China and Singapore: top 10% papers to (Leiden) NU Singapore Nanyang UT Tsinhgua U Zhejiang U Peking U

14 Growth top 10% papers, to university system growth NU Singapore SINGAPORE % Nanyang TU SINGAPORE % Tsinghua U CHINA % Zhejiang U CHINA % Peking U CHINA ,0% Shanghai JT U CHINA % Fudan U CHINA % U S&T China CHINA % U Hong Kong HONG KONG % Seoul National U KOREA % National Taiwan U TAIWAN % MIT USA % U Cambridge UK %

15 What drives quality? Top end competition for research rankings (takes in only top 500, i.e. T1 universities and aspirants) generates continuous increase in paper outputs and improved cite rates Competition for foreign students, plus national QA and management of student satisfaction generates continuous improvement in services for foreign students National competition, QA and management of student satisfaction generates improvement in student servicing in some HPS but absence of a global dynamic means this is weaker than research effects Nothing drives continuous improvement in student learning

16 Global rankings and quality ARWU big science. Works. Leiden and Scimago allow fine-tuning of research management (REAL METRICS MATTER) Times Higher more than two thirds research driven QS and TH surveys no link to performance. No inherent quality driver (there s a marketing driver) Multi-indicator rankings of QS and TH incoherent, no validity, no link to performance (separate indicators would be OK) Teaching proxies are staffing ratios (???) and surveys of who is good at teaching (?????? Why do we tolerate this?)

17 Lacuna in learning measures in HE Impact of PISA as a driver of performance Not transparency in learning function No useful information to guide choice No performance drivers for T2 and T3 HEIs In the context of research-led competition and absence of learning achievement measures talk about parity (or primacy) of teaching is vapid Proxies create wrong incentives and we have no way of telling they create right incentives Real, grounded metrics matter. Nothing else works

18 Proxies don t get us there Resource measures no necessary relation to quality Graduate employment data are important but do not give us learning quality (human capital theory is a metaphor not an individualisable technology and it cannot be reverse engineered) Indirect outcomes assessment (student surveys of learning behaviour, student satisfaction, student engagement) etc a small part of what we need, BUT Management by satisfaction drives down cognitive formation, standards (Arum & Roksa 2014, Armstrong & Hamilton 2014): most students now believe it is not what you know it is who you know Cognitive formation does not sit well with consumer sovereignty but is crucial to bright poor student

19 Measuring learning outcomes Three aspects - general cognitive formation, CLA-type tests - discipline-based learning - work-related generic skills, nested in each occupation Robust faculty cultures to build discipline measures and work skill measures on crossborder basis Measures cannot provide holistic/total outcomes Use separate indicators and resist the pressure for combined indicator-based tables Don t combine with research measures in single tables (unless through user customization)

20 Post-AHELO landscape Moving on Individual university route can t work like governing on the basis of a UN Security Council with 100 permanent members who all want to say no Governments are crucial but only some can deliver Move on CLA-type competency tests in advance of the rest of the package Move in selected countries/regions (East Asia/Singapore?) but crucial to include WCUs