ASIRPA : from ex post to real-time evaluation

ASIRPA Real-Time (RT) is a formative evaluation approach. It aims to support researchers, together with other actors, to orient their research activities towards a collectively defined direction. This approach builds on more than 60 ex post case studies conducted within ASIRPA ex post. These case studies have highlighted:

• the length and complexity of transformation processes;
• the importance of the network of actors involved;
• the role of the context, researchers and other stakeholders;
• the need to identify and overcome blocking factors.

Whereas ASIRPA Ex Post aims to assess societal impacts a posteriori, and therefore to learn lessons from the past, ASIRPA RT consists of envisioning desired futures and collective learning to anticipate and orient towards these desired futures.

Overcoming societal challenges implies defining a direction to follow. The pathways to contribute to addressing these challenges are complex, uncertain and unknown. The objective of ASIRPA RT is to equip project and programme managers so as to navigate towards their desired futures. It is a formative evaluation approach that fosters learning and promotes the construction of network of actors in order to generate societal impacts.

A societal impact results from a change of state of a system. In other words, what we observe today will not be the same in 2040 (Box 1). For an R&I process to contribute to positive impacts, the question to ask is: who and what will have changed if the alternative solutions studied today are implemented and scaled by 2040?
Reflecting on this question enables to better understand the potential impacts of R&I and to anticipate the transformations needed to achieve them.

Why reflect on the societal impact of research?

Researchers play a key role in studying alternative solutions to address complex societal challenges. Generating a technical solution is not enough; it must also be adopted, used, and fully appropriated by the relevant actors. This often brings about other transformations in society, such as changes in regulations, consumption patterns, habits, mindsets and/or the organisation of value chains.

Achieving a desired future that is beneficial for society is not straightforward. It is not a direct and automatic result of research. It is a complex process, often uncertain, that takes time, additional resources and the collaboration of many actors. It implies working with non-academic partners, co-developing solutions and moving forward collectively. Thus, it is very important to build a clear vision of how that future could become reality.

However, to address current challenges, researchers and programme managers cannot limit themselves to setting new, ambitious scientific objectives. They must also reflect on the transformation of unsustainable systems and on how the solutions they are studying can contribute to it. But to meet this challenge, researchers must deal with an essential reality: while societal challenges define a common problem, the pathways to address them are complex, uncertain and take place over the long term. They require the progressive mobilisation of many actors within networks. This is not a linear process in which research provides a solution that will immediately be adopted by society. Instead it is a process based on dialogue, participation and co-construction of a shared vision of the future.

Go to subsection: How to reflect on impact?