6 Eric Archambault 1

Slide1 : SCIENCE AND TECHNOLOGY EVALUATION Éric Archambault, Ph.D. Workshop on Performance Assessment of Public RTD Programs Bruxelles, Belgique, 17-18 juin 2004 Use of multiple scientometric and technometric indicators for benchmarking: the case of leading countries in genomics


Slide2 : Comparative assessment of countries' performance in S&T Objective: to describe the wealth of situations encountered by heterogeneous countries in genomics S&T Problems: lack of comparability of data gathered at the national level – - data compiled at the national level often lacks coherence at the international level; - countries are of different sizes and different S&T capabilities and raw output should not be the only consideration. Solution: use of a common source of data and of relative indicators – allows comparison of the scientific performance of heterogeneous countries Benefits: these data provide added value information for evaluation purposes as well as for strategic planning


Slide3 : Methods Scientific publication data: compiled by Science-Metrix from the SCI expanded database Patent data: compiled by Science-Metrix from the USPTO database Dataset: built using keyword in title as well as in abstract in the case of patents – keywords were selected in collaboration with Genome Canada’s Scientific Committee – 95% of papers in the journal Genomics were retrieved Specialization index: provides a measurement of a country’s relative effort in a field relative to the world Average relative impact factor (ARIF): provides a measurement of the expected citation rate of a given country relative to the world Average relative citation (ARC): provides a measurement of the citation rate of a given country relative to the world


Slide4 : Scientific papers and patents in genomics, 1991-2002


Slide5 : Scientific papers (1991-2002) and growth (1999-2002)


Slide6 : Scientometry, 1991-2002


Slide7 : Scientometry, 1991-2002


Slide8 : Technometry, 1991-2002


Slide9 : Technometry, 1991-2002


Slide10 : Strengths and weaknesses of Canada in genomics, 1991-2002


Slide11 : Agriculture & Food science Analytical Chemistry Biomedical Engineering Biophysics Cell biology, Cytology & Histology Dairy & Animal Science Dermatology & Venereal Diseases Environment & Occupation Health Environmental Science Gastroenterology General chemistry General physics General & Internal Medicine Geology Immunology Inorganic & Nuclear Chemistry Marine Biology & Hydrobiology Miscellaneous Biology Miscellaneous Biomedical Research Nephrology Neurology & Neurosurgery Obstetrics & Gynaecology Ophthalmology Parasitology Pharmacology Pharmacy Strengths and Weaknesses of leading countries, 1991-2002 Physiology Respiratory System Surgery Log (ARC) Log (SI)


Slide12 : Conclusion The US dominates the field of genomics both in terms of scientific output and technological output – it is therefore likely to be the predominant actor within worldwide markets when commercial applications will mature Germany and Japan, while very active in absolute terms, are being overtaken by smaller countries relative to their population size Small countries with good performance relative to their population size include Switzerland, Sweden, the Netherlands and Canada Developing countries like China and the Republic of Korea are closing the gap with the most active countries and are likely to enter the pack of leading countries in the not too distant future The use of a common source of data and of relative indicators allowed us to successfully conduct a comparative analysis of the state of S-T at the international level