This talk deals with the challenge of building and using bibliometric indicators in an evaluative context. The aim is to show how valid metrics for scientific information and research assessment can be developed and applied and what the main possible pitfalls in this process are. The talk is organized in seven chapters proceeding from the ‘perspective change’ that scientometrics has undergone in the last decades of the 20th century and the strong interaction between metrics and information retrieval. The first two chapters deal with the use of suitable data sources and the quality and cleanness of data, including data retrieval and subject delineation. The following three chapters deal with methodological issues and are devoted to the explanation of important criteria of soundness of methodology, the discussion of the effect of subject granularity and aggregation level and finally to the extension of traditional indicators towards performance profiles. Special focus is laid on the use of composite indicators, notably in the context of ranking exercises (sixth chapter). The last chapter introduces the challenge of measuring impacts beyond scholarly communication including issues regarding the appropriate integration of data originated from heterogeneous sources.
The Science of Science in the Spotlight Lecture Series provides a forum for multidisciplinary exchange and dialogue across the fields of higher education, bibliometrics, and science studies in Luxembourg. Through international visits and collaboration with leading experts in the relevant research fields and disciplines, the lecture series offers an ideal platform to bolster Luxembourg’s growing research expertise and current initiatives in the « science of science”, including diverse science and policy communities and the wider public. At the interface of different strands of research that have grown in Luxembourg over the last decade, the lecture series showcases emerging collaborations within Luxembourg, across Europe, and globally. It provides opportunities to discuss cutting-edge research results in this highly innovative multidisciplinary field of research. The lectures will stimulate debates on theoretical and methodological approaches and on data acquisition and analysis. Leading researchers visiting Luxembourg will share their knowledge at the intersection of higher education research and science studies, bibliometrics, embedded in social and computational sciences more generally.
The lecture series will foster discussions relating to international higher education developments, science capacity-building, scientific knowledge production, research evaluation, collaborative networks, and researcher mobility. Structured as a multidisciplinary series of events, the lectures provide diverse and integrated insights and will help us to solidify Luxembourg’s network in this field of research, to build new national and international relationships, and to encourage a multidisciplinary, interfaculty, and interorganizational exchange with leading researchers from abroad.