Natalka Cmoc – Director General of Science and Policy, Science and Parliamentary Infrastructure Branch, Public Services and Procurement Canada
Fred Wrona – Professor in Integrated Watershed Science, University of Calgary
Jacqueline Ottman – Professor, Vice-Provost Indigenous Engagement, University of Saskatchewan
David Fortin – Associate Professor and Director of School of Architecture, Laurentian University
Melody Lepine – Director, Mikisew-Cree First Nation Government
Context: Advancing Canada’s federal science delivery requires a more holistic, integrated, inter- and transdisciplinary approach that from the onset recognizes the benefits of braiding multiple knowledge systems. Experts on this panel shared their experiences and perspectives from both the western science and Indigenous knowledge systems to discuss approaches on how to best advance the application of multiple knowledge systems into the development and delivery of federal science and knowledge generation in Indigenous communities. This includes looking at how facilities and laboratories extend into the traditional territory of Indigenous communities that participate in the knowledge creation.
Indigenous knowledge provides a broader view and its incorporation/inclusion has a significant impact on reconciliation.
Knowledge gaps and lack of training limits appropriate use and applications of indigenous knowledge.
Indigenous research belongs to the community.
Traditional knowledge from the Mikisew Cree First Nation influenced the development of a water management strategy for the Athabasca River.
An efficient design for new buildings is one that incorporates indigenous teaching with contemporary building science.
The “3 Rs” – relationships, respect, and reciprocity – are critical to collaborating with communities, learn their values, and integrating them into research/training programs.
How do we bring the indigenous perspectives, and be more inclusive and equitable, so that everyone can benefit from ideas that indigenous communities offer?
Recognize and find ways to benefit from indigenous knowledge and develop policies accordingly.
Merge conventional western science and indigenous knowledge to advance our knowledge and develop policies that support this.
Establish a new way of working together. Approach indigenous communities with appropriate ethical and cultural ethos such to avoid a lack of understanding, mistrust, and misuse of indigenous knowledge.
Anna Hatch – Program Director , Declaration on Research Assessment (DORA) Leonie van Drooge – Senior Researcher, Dutch Rathenau Institute Lin Zhang –Professor, Wuhan University Erika Kraemer-Mbula – Professor, University of Johannesburg
Stephen Pinfield – Professor of Information Services Management, University of Sheffield
Context: The concept of research excellence is changing. In Canada and internationally, there is increasing recognition that valuable knowledge is produced and disseminated by different peoples, using different methodologies, in different formats and languages. Funders are recognizing the limitations of performance indicators, which neither represent the breadth of valuable contributions undertaken by researchers, nor account for biases that disproportionately impact under-represented, equity-seeking groups. This panel of experts reflected on the concept and assessment of “excellence”, and how these are or should be changing around the world.
The over-emphasis on traditional, quantifiable metrics of research excellence (e.g., number of publications, h-index, and journal impact factors) has led to a high level of pressure for researchers due to heavy workloads and hyper-competition for funding. It can also create “islands of excellence” that discourage creativity and restrict the equitable participation of all people in the research enterprise.
The concept of “excellence” shouldn’t be taken for granted. It is pluralistic, and should not be defined without considering the context in which it is applied. If the metrics of excellence are not sufficiently transparent and/or not tailored to the right context, this can create tension. E.g., in South Africa researchers are often ranked on their ability to publish internationally in high-impact peer-reviewed journals, but this is not always compatible with the pressure to conduct research that is locally relevant (e.g., designed to address socioeconomic issues).
China’s Ministry of Science and Technology has announced many changes to their research assessment practices, including a farewell to the use of science citation index, a shift from publication-based assessment to more comprehensive peer review, and new priority towards local relevance. As these changes are being implemented, this raises challenges for finding a balanced approach to funding diverse kinds of research, across disciplines, types of research, development stage, global and/or local relevance, strengths and career stage of researchers, and individuals versus groups of researchers.
A 30-year review of excellence policy by the Netherlands’ Rathenau Instituut found that the concept of “excellence” enables autonomy and flexibility of researchers but also has unintended consequences where researchers face pressure to prove their value in competitions with limited funds and limited approval rates, that too often under-appreciate measures of impact beyond publications. It also raised many fundamental questions such as: Is excellence a zero-sum game, where only some can be excellent? If so, how do we describe the rest of the research community? Or is excellence more of a threshold, where anyone can be excellent if they meet a certain standard? (Dutch research funders together with universities and of university medical centres have since developed a new approach, “Room for everyone’s talent: towards a new balance in the recognition and rewards of academics”.)
UK funders have also encouraged more co-design and co-production of research, to better ensure that results are put into practice. The Research on Research Institute established a consortium of international partners (including the Canadian Institutes of Health Research) to gather evidence and experiment with different methods of research assessment, which can enable better evidence-based decision making by funders worldwide.
Assessments of excellence should be context-dependent (assessing individual researchers vs groups, early vs late career researchers, fundamental and applied research, local vs global relevance) and should consider how to address longstanding imbalances and inequities.
We need to recognize and reward academics based on a more diverse set of indicators, to foster different academic talents while continuing to focus on quality. This can be achieved by embracing the diversification of career paths as well as strengths in other areas (open science, academic leadership, community engagement).
Fundamentally, does every researcher need to be “excellent,” and is it wrong with simply be “good”? Research assessment systems should build capacity for the whole of the research community, and should avoid promoting “islands of excellence”.
We need evidence! By analyzing how excellence is measured across funding organizations internationally, we can build a better picture of excellence in practice, in order to develop better evidence-based frameworks for the future.
The biggest challenge is not policy change, it is behaviour change. Therefore, reforming research assessment is a systems problem, where all stakeholders in the research ecosystem need to work together to enact culture change at all levels: funders, universities, education programs, publications. Both bottom-up and top-down approaches are needed – there is no one silver bullet.
Takeaways and recommendations, CSPC 2019 plenary on The Future of Research Excellence: A Conversation with Canada’s Granting Agencies. Organized by: Canada Foundation for Innovation (CFI), Canadian Institutes of Health Research (CIHR), Natural Sciences and Engineering Research Council of Canada (NSERC), Social Sciences and Humanities Research Council of Canada (SSHRC) with support from the Canada Research Coordinating Committee Secretariat (CRCC).
Sethuraman Panchanathan – Director, National Science Foundation Sudip Parikh – Chief Executive Officer, American Association for the Advancement of Science (AAAS)
Mona Nemer – Chief Science Advisor to Canada’s Prime Minister, Government of Canada
Mehrdad Hariri – Chief Executive Officer & President, Canadian Science Policy Centre
Context: This year’s annual CSPC fireside chat brought together two of the United States’ most recognized science leaders for an informal discussion about their experience in the U.S research landscape. Panelists discussed the importance of good public policy in maintaining research excellence, the increasing need for open science and multidisciplinary collaboration in responding to global crises, the role of science in helping to diminish socioeconomic inequality, and science communication for building public trust.
Scientific communications have changed dramatically with the restrictions brought on by COVID-19, making it even more urgent to create science that is accessible to the public.
As illuminated by grand challenges such as COVID-19, scientific knowledge has increased in value within the political and public landscapes.
Diversity within STEM is a societal issue, as the issues transverse from STEM education, to STEM research and opportunities.
In terms of public opinion and trusting science, COVID-19 has illuminated that science and scientists must work to bridge the gaps between scientific understanding and the general public.
The challenge of our time is how to incorporate science in policymaking without over-emphasizing or under emphasizing it.
Science communication must continue to be accessible to a wider variety of audiences. This is particularly urgent given that individuals and organizations outside of scientific domains are now using open access platforms to understand current scientific advances.
More funding for STEM is necessary, as public leaders, scientists, and policymakers continue to collaborate on grand challenges such as the COVID-19 pandemic.
Leaders in STEM must ensure diversity, inclusion and accessibility policies are in place, as well as global science leadership, and increased partnership within each level.
Re-building trust between science and the general public is a top priority. These efforts must be creative and effective, such as increased and diverse outreach to locally trusted sources, active members of communities, and strategic and impactful settings.
The key to balancing science within policy is the presence of evidence-based programs. These programs ensure a continuous dialog between science and policy so that when grand challenges arise, valuable time can be better spent on accessing and utilizing experts and evidence rather than building relationships.
Janik Cazabon – Senior Manager, Stakeholder Engagement and Outreach Science and Parliamentary Infrastructure Branch, Public Services and Procurement Canada
Claudie Roy – Research Officer, National Research Council of Canada (NRC)
Robert Black – Research Officer, National Research Council of Canada (NRC)
Josué Jautzy – Research Scientist, Geological Survey of Canada, Quebec City