Despite all advances in our knowledge of social innovation, what remains an important open question is how to capture its impact. With 33 participants worldwide, the interactive ESSI webinar »Social Innovation Measurement: How to capture the impact of social innovation?« was a great success with insightful and inspiring discussions.
The two short inputs by Judith Terstriep (Institute for Work and Technology, Westphalian University Gelsenkirchen) and Georg Mildenberger (Centre for Social Investment and Innovation, University of Heidelberg) on SI impact measurement and SI indicators, followed by two rounds of breakout sessions, stimulated lively discussion on how to measure SI impact at the systemic level. That SI impact goes beyond the social to include environmental and other effects while accounting for its ambivalence (positive/negative, intended/unintended), participants were unanimous. Less conclusive, however, was the question of whether it makes sense and is possible to measure the impact of social innovations at the systemic level. Only 35% agreed that it is possible to elaborate a common impact model (see figure below). Although SGDs could function as a joint reference frame, an integrated impact model is open for adding context-specific aspects. The first session discussed, among other things, whether SI impact measurement is approached from a functional or transformative perspective. It was highlighted that while the aspect of »transforming society« could become more standardised, the problem-solving part (functional perspective) might be more diverse. By applying innovation streams or practice fields as criteria developing baseline models could become possible. Also, a matrix approach integrating innovation types/streams and the six »I’s« was emphasised as beneficial.
Envisioning an integrated SI impact model calls for suitable indicators available across various interventions and regions. Yet, we find a plurality of custom-made instruments and methods with individual matrices that do not necessarily allow for aggregation. Nevertheless, participants voted 59% to 41% in favour of the possibility of aggregating available indicators at the micro-level of single SIs to capture impacts at a societal (macro) level. Fairtrade, as an example, illustrates the challenges in elaborating indicators at the systemic level as it concerns different geographies; social change occurs not inevitably where the products are consumed but where they are produced, resulting in a diversity of contexts to be considered. It was posited that the guiding question in indicator development should be on how bottom-up social innovations can contribute to systemic level change. That is, explicating the assumptions of an integrated impact model enables assessing the assumptions rather than the impacts. In addition, it was lively discussed whether there is a learning effect from aggregating data? And if so, what can we learn? That requires scholars to leave their comfort zone to overcome silos and find common ground to aggregate data while assuring data interoperability. Also, it was cautioned that we might lose relevant information on the SIs impact when limiting ourselves to quantitative data. Instead, the focus should be on a proper mix of quantitative and qualitative data.
Summing up, developing an integrated impact model and meaningful indicators is about undertaking a journey of joint exploration, discovery, weighing and decision-taking. An exploration phase should result in a consolidation and evaluation phase where the community decides on the relevance of the SI impact model(s) and its indicator sets. Hence, the webinar is seen as the first step of this journey.