Research evaluation in Sweden FOKUS

Transkript

1 Research evaluation in Sweden FOKUS Swedish Research Council Hearings May 2014

2 Content Background Working process Proposed draft model Future work Timetable Questions and discussion

3 UoH research income 2013 Swedish businesses 4% Foreign org. & businesses 2% Other 1% Performance based Non-profit org. 13% EU 5% Direct governnment funding 40% Local government 4% ALF-funds (county) 4% Other gov. agencies 9% Research councils 18%

4 Background Previous government commissions The Research and innovation bill 2012 Government commission March 2013

5 The Government Commission (14 March 2013): The Government commissions the Swedish Research Council to investigate and submit in consultation with Forte, Formas and VINNOVA a proposed model for resource allocation to universities and university colleges involving peer review of the quality and relevance of research Enable resource allocation that rewards quality and performance in research Comprise both academic assessments and assessment of the societal relevance and impact of the research May provide a good platform for long-term planning at higher-education institutions Should be done bearing in mind the preconditions for the respective research area All research is regularly assessed in subject-area based evaluations in a cycle of four to six years May include indicators To be submitted by December 31, 2014

6 Our guiding principles Improve the quality of research All data should be subject to panel assessments Understandable as simple as possible Transparent Fair Include all research areas Broad interpretation of societal impact Balance between two principles: Efficient Informative

7 Working process Fact finding study Nov 2012 May 2013 Continous dialogue and consultation with stakeholders within and outside academia Advisory boards Discussion with other Government agencies Meetings and open hearings with HEIs (8, 12 and 16 of May)

8 Elements of evaluation and suggested weigthing FOKUS Research evaluation in Sweden Purpose: To improve the quality of research and ensure that high quality research is of benefit to society Background information (not to be graded) 70 % Scientific quality Quality enhancing factors Impact outside academia All research to be evaluated in a cohesive manner every sixth year

9 Units of assessment 5 fields of science N T M S H 24 research areas A B C D E F B H I J K L M N O P Q R S T U V W X Based on existing classification and statistics Consideration given to interdisciplinary research

10 Panel structure Subpanels Main panels 24 X X 5 Scientific quality Quality enhancing factors Impact outside academia

11 Background information to the panels FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Description of: HEI s vision and strategy research profile organisation, administration and management Quantitative data: financial resources (total income) research staff education (number of FTE students incl PhDs) publication volume and activity

12 Scientific quality FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Weight: 70 %, based on the quality of research results per research area Criteria: novelty and originality, significance of the research in its field and scientific rigour The panels assessments should include research excellence as well as spread (i.e. top results and average quality)

13 Scientific quality Basis for the assessment FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Research statement Publishing profile Evaluating excellence: - peer review (ie. review of outputs) of research outputs in all areas, done by the panels Alternatives to evaluate average quality: - external peer review, done by external reviewers - citation analysis

14 Scientific quality Basis for the assessment FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin The basis for evaluating average quality may vary. Within each research area the basis for evaluation should be the same.

15 Quality enhancing factors FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Weight:, assessment at the level of Field of Science Criteria: potential for renewal and sustainability Quality enhancing factors: Renewal turnover of young researchers incl. PhD students International and national collaboration within academia Mobility Gender equality Integration of research and education Access to infrastructure

16 Quality enhancing factors Basis for the assessment FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Quantitative data Short description with comments in relation to the HEI s overaching strategies

17 Impact outside academia FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Definition: a successful application of research in order to achieve social, economic, environmental and/or cultural effects Weight:, assessment at the level of Field of Science Criteria: reach and significance

18 Impact outside academia Basis for the assessment FOKUS Forskningskvalitetsutvärdering i Sverige Övergripande syfte: Kvalitetsdrivande och verka för att forskningen ska bidra till samhällets utveckling Bakgrundsinformation (betygssätts ej) 70 % Vetenskaplig kvalitet Kvalitetsutvecklande faktorer Genomslag utanför akademin Case studies, eg. two per 50 FTEs Case studies may include indicators Description of HEI strategy and resources for working with dissemination and the promotion of use outside academia, existing support functions regarding dissemination of research results, collaboration between researchers, community, business interests etc.

19 How to collect data Main principles Minimize work load for individual researchers Use as much existing data as possible Data Existing data Staff Specific data required for FOKUS Eg. descriptions, case studies and strategies PhDs and PhD exams Finances Publications (SwePub database, WoS)

20 The focus of the evaluation

21 The result of the evaluation of a single HEI

22 Work process until December 2014 Composition of panels How to translate evaluation results to resource allocation? Cost estimates incl. cost-benefit Analyses of the consequences

23 Timetable SRC proposal Consultation Preparations Commission from Govt. Autumn 2015 Evaluations Results Budget Bill 2018 Allocation of funds starting 2018

24 Thank you! More information: Contact: