Theory of Change

Creating the CAPE Theory of Change: an exercise in collaboration

Theories of change (ToC) are visual road maps. They help show the connections between the activities and resources of a project, and the impact that project seeks to achieve. For CAPE, our ToC sets out how we think our activities will translate into the change we want our project to have in improving academic-policy collaboration across England.

As a consortium of universities, policy partners, and knowledge brokers, we knew we had a monumental ambition: to align our project structure, objectives, activities, roles and responsibilities of each institution across the consortium. With a view toward generating actionable insight that we can share and replicate across the wider sector, creating our ToC was crucial to build a foundation for our programme of work, collectively establishing the parameters of our project, and for steering our evaluation. Interestingly, the process of creating our ToC mirrored the process for improving academic-policy engagement – a focus on practice alongside theory, co-creation, aligning mutually agreed areas of focus, and combining our collective strength to design and implement our initiatives.

We took some time to reflect on the process of creating the ToC and what we’ve learnt.

Situating CAPE and finding our priorities

We started by examining the existing evidence base with our evaluation team Transforming Evidence. In mapping current initiatives within academic-policy engagement, we were able to situate CAPE within an evolving sector, to draw learning from partners implementing similar activities across the sector, and to see where CAPE could add value both in the provision of activities, and in steering our evidence generation efforts to fill core knowledge gaps.

The more we dug into problems and potential solutions from across the nexus of academic policy engagement, the more we became aware of the myriad cultures, incentive systems, organisational contexts, feasibility constraints, and opportunities that exist to drive change within the sector. By mapping this complexity, we were able to consider, rank, and refine where our priorities lay against the context of the existing evidence base – and determine where and how CAPE’s unique strengths as a multi-institutional consortium could contribute the greatest added value. We also wanted to understand how to work in partnership with others working in the space, to be able to leverage our insights to inform our actions going forward in partnership with others, from the Universities Policy Engagement Network, to the British Academy, and UKRI.

Questioning our assumptions behind ‘good’ academic policy engagement

At the same time, creating the ToC caused us to question and explore more deeply the assumptions underpinning the feasibility, effectiveness, and underlying mechanisms that would allow for the success of our activities across a diverse geographic and political system: how do we know that training for researchers improves policy engagement capability (as we had assumed)? What are the conditions that create meaningful knowledge exchange in one university – and do this hold in other university contexts? How do we define and measure meaningful knowledge exchange? What does ‘good’ co-production look like? How can we ensure that we don’t reproduce unequal power dynamics in our engagement initiatives?

Surfacing the assumptions that exist across our activities allows us to identify where and which assumptions are evidence-led, or evidence-generating- helping us to steer the design, implementation, and evaluation of our mechanisms in a way that allows for both greater impact, and greater opportunities to contribute unique learning that can be shared with the wider sector through evaluation.

Moving beyond the ‘usual suspects’ to a more inclusive landscape

Through the process of creating a ToC, we were also able to surface a core CAPE priority: creating a more diverse and inclusive policy landscape. Previous initiatives aimed at improving academic-policy engagement have often failed to systematically examine equality, diversity and inclusion in the context of knowledge exchange, and this came to the fore as both an evidence and provision gap and an important ethos for the project. It is important to us that we are working towards a knowledge exchange ecosystem that re-examines who has a seat at the table.

Through CAPE, we want to ensure that the design, implementation, and evaluation of our initiatives are done in an inclusive way: by ensuring the inclusion of both people from historically underrepresented backgrounds whose experience and expertise are often excluded from agenda setting, and the promotion of locally relevant knowledge for decision-making that celebrates the diversity of thought from across England, as opposed to the dominant London-centric circle of knowledge-for-policy. Notably too, the importance of creating this inclusive landscape also responds to something we increasingly hear from our policy stakeholders in our conversations with them.

We learnt what was important by listening to each other

It has been as much about familiarising ourselves with the language needed to navigate the ToC world as it has been about narrowing down our focus – by going through the process together, we have been learning and developing collaboratively
Kayleigh Renberg-Fawcett, UCL CAPE Coordinator

Our theory of change process wasn’t just about the product itself either, but mirrored our commitment to co-producing activities that allow for the mutually beneficial alignment of purpose, activities, and impact. In working together and building upon our collective long-standing experience of academic policy engagement, both in knowledge brokerage and policy roles, we were able to learn from one another and build relational trust across our team.

Listening to each other’s experience undoubtedly shaped our understanding of CAPE priorities. For example, when our project coordinators joined the CAPE team for example (they were not in post at the start of the process due to COVID-19 hiring delays), their awareness of the importance of knowledge brokerage functions within universities allowed us to hone our focus on the required structures and systems that allow for the sustainability and embeddedness of initiatives that promote policy engagement.  

How we’re using our Theory of Change

Within a context ripe with opportunities for improving academic-policy engagement, our ToC has provided us clarity on the unique contributions that CAPE can add to the sector, as well as where others are better suited to drive forward change. Importantly, we’ve co-created consensus about the boundaries of our project, whilst celebrating the spirit of exploration and experimentation which is central to CAPE’s approach to our activities. As we continue to scope new projects with our partners, the ToC allows us to explore how collaborations contribute to our goals and which activities (and correspondingly their design and implementation), are likely to be impactful and knowledge generating, while being sensitive to the complexities, contexts and assumptions of our work.

In the national evaluation of CAPE by the Transforming Evidence team, we will use the ToC to guide data generation and analysis. By exploring its components in qualitative interviews, we will develop nuanced understandings of local mechanisms and can test out the assumptions across sites.

At a more local level, our ToC will be used as a reference point for developing additional theories of change as aligned with our varying activities and institutional contexts, allowing us to generate a better understanding of how CAPE’s activities can account for unique geographies and distinct institutional characteristics of our partner organisations. As a consortium it’s important for us to learn and share how our activities can be delivered and be responsive to the unique needs and operating environments of high education institutions and policy organisations alike: to ensure the parts speak to the whole and vice versa.  

Finally, we recognise that theories of change are living, breathing, and adaptable. As our knowledge of what works across our initiatives improves, as we start to see what emerges that we may not have anticipated, and as the sector continues to develop, we’re able to integrate this learning into our work and adjust our course accordingly. We’ve included feedback loops for this very reason, and we will revisit our ToC regularly throughout the rest of the project. 

Have any questions about our ToC or want to chat about how you can be involved with CAPE? Get in touch [email protected]  