Evidence: from engagement to impact
⌚ Estimated reading time: 10 minutes
In this blog Ben Hepworth from the Ministry of Justice describes how civil servants working in knowledge exchange roles across government have been supported by CAPE to convene a forum – the Policy Knowledge Brokering Forum (PKBF) – to discuss topics of mutual interest. The substantive content of this blogpost explores and summarises one session of the PKBF that discussed evidence usage, academic engagement, and ‘impact’.
Ben Hepworth
Ministry of Justice, Evidence and Partnerships Hub
Context
The UK government makes a lot of decisions. Decisions that underpin policy development and operational practice that affect the lives of every UK citizen. To make these decisions, the government needs evidence, including academic research in all its shapes and forms. Academic research that has ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’ is said to have ‘impact’ and is the cornerstone of the Research Excellence Framework (REF). In government it is crucial that departments can access the right (research) evidence, at the right time, to support decisions affecting public policy and services, but ‘impact’ is not under the same spotlight as in academia. If research is to flow from academia to government – whether it is formally acknowledged as impactful or not – then it must cross an interface between these two worlds. Knowledge must be exchanged.
Knowledge exchange can be aided by individuals in both government and academia working in roles (as knowledge brokers) to operationalise the exchange of evidence and expertise. Such roles have become common in academia in the last decade and many universities now have a team that actively supports ’policy engagement’. There are also significant benefits from the organisation of individuals and institutions through, for example, initiatives such as Transforming Evidence and the Universities Policy Engagement Network (UPEN) respectively.
Whilst not common, there are also specialist knowledge brokerage roles in many government departments that facilitate the use of academic evidence in policy and operational decision-making. I do this in the Ministry of Justice in a knowledge exchange unit called the Evidence and Partnerships Hub. Seeing the collective benefit of networks in academia, individuals from across government and parliament have convened their own – the Policy Knowledge Brokering Forum (PKBF). The PKBF has been facilitated by the Capabilities in Academic Policy Engagement (CAPE) project as a year-long pilot. The aim is to assess the effectiveness of convening departments to discuss overlapping topics of interest related to government-academic knowledge exchange. An initial selection exercise was run before holding different forums on different topics.
The remainder of this blog explores and summarises one session of the PKBF that discussed evidence usage, academic engagement, and ‘impact’ – what that means inside and outside government and good practice advice on how to achieve it. The blog’s structure is based around themes that came out of the forum with relevant quotes taken from an online post-it note exercise.
Complexity
It could be intuitive to assume that generating impact is simple and linear – an academic has some new research evidence, they engage with government, it enables a decision and policy/operations change as a result (impact) – but this is far from true. Academic-government engagement is not simple, it is a complex process involving multiple actors; nor is it linear, the process involves back-and-forth, feedback, and doesn’t always start at the ‘beginning’. In terms of that first encounter between an academic and a government official, this could stem from a structured workshop involving tens of participants (formal) to a single ad hoc conversation between two individuals (informal). Often by doing the former you create the space and conditions for the latter and this is what can eventually lead to true impact. It can be helpful to think of what are the necessary and sufficient conditions that lead to impact (borrowing terminology from mathematical logic). Whilst acknowledging that the process is complex, the research that can unlock a topic need not be – even the smallest piece of evidence can have a huge impact.
“A nugget of information that sticks”
Engagement doesn’t always have to originate from a transaction of evidence, it can be speculative or lead to the suggestion of new research to satisfy an unmet need. This is the basis of the government Areas of Research Interest (ARI) initiative and the engagement workshops that many departments (cf. DWP and Defra) have instigated – seeking to fill evidence gaps through proactive research engagement. Research engagement may also highlight where impact has already been tentatively realised.
Whilst the need to underpin decisions with evidence is embraced, working in a fast-paced government department means that utilisation of research isn’t always acknowledged. From outside it may seem like an obvious and simple step but when it comes to impact tracking inside government, at a practical level, there is the question of what is feasible. For example, in a monthly academic seminar series attended by 50-100 civil servants, what is proportionate monitoring of what individuals do with that information? It’s feasible that someone will do something, at some point, but it’s impossible to follow up with everybody, at regular intervals, over time – so how can outcomes and impact be tracked? Inside an organisation of tens of thousands of people this is a significant task. This is not always appreciated in academia where faculties, departments, and research groups are much smaller. Even with the will, often there is not a way, and impact that is realised is serendipitous.
“Windows of opportunity”
The structures of both government departments and universities as organisations mean that silos are unavoidable. Sometimes a well-informed researcher can have better contacts at – and knowledge of the work in – a government department than a civil servant from that department (and vice versa). Such are the overlaps and complexities of the government-academic interface and the many actors at play. This is why relationships at the boundaries of these organisations are crucial and the role of knowledge brokers is paramount.
Relationships
Government and academia are different in many ways, but there is a stark contrast in the imbalance of (publicly available) information on both parties. Search online for an academic and you can find out lots about them and even what they look like; it’s often difficult to find the name of a government official. In the past this made it challenging for academics to make the right connections, but as knowledge brokerage has grown as a ‘profession’ the onus has moved away from the producers and users of research and onto those in the middle-ground.
Networks and communities such as UPEN and Transforming Evidence, with machinery such as regular newsletters and events mean this information imbalance can be redressed. Certainly, on the government side, there has been a change in recent years – the ARI initiative serving as a catalyst – with more individuals taking the time to communicate how their departments want to collaborate. Building on this requires knowledge brokers from both government and academia taking the time to understand each other’s worlds and figuring out where they can be aligned. Sometimes this is obvious and immediate, sometimes this means making speculative connections that can be drawn on in the future. Many people report keeping in regular touch with a (government or academic) counterpart, even when not supporting a specific project, to ensure relationships are built and changing priorities are captured.
“Cultivating positive relationships”
These relationships also serve the greater good of promoting and improving diversity. Historically when individuals have required specific evidence or expertise, they have typically gone to someone that has helped them before and they know can help them again. However, without diversity of thought and voice, existing biases and beliefs can be perpetuated which can lead to narrow policy options and decisions. This also encompasses academic diversity and not being constrained by (artificial) disciplinary boundaries. What is taken for granted in one domain can be transformational in another, particularly when it comes to the application of research methods and techniques to unlock knotty questions. Some of the most exciting and impactful collaborations are where boundaries are transcended, for example like the DWP Methods Advisory Group. However, not all government departments or even universities have the people and systems in place to forge these relationships, so it is always worthwhile questioning what is missing – even if engaging with a network or community. And it’s crucial to acknowledge that even by creating and fostering the right relationships, the pathway to impact can be a long and winding one.
Time
Timing is significant, the right evidence at the wrong time will likely not be impactful, but often the timescales of government and academia are out of sync – both in terms of priorities and people. It is generally accepted that engaging early in the academic lifecycle of a research project is best – even as part of the funding process. This can be to advise with question framing and strategic priorities, as well as plan for continued engagement if a project is subsequently funded – for example via research steering group membership. Early engagement also ensures that research can be fed in as it becomes available, and support decisions when they’re required to be made, rather than waiting until a research project has finished. Early is best for both parties but we are some way from this being the norm; even if initial contact is established, by the time a research project is funded and delivers (some) findings, the civil servant initially involved may have moved on.
“Right people, right times”
The typical tenure of a government official’s role is often measured in years – sometimes even months – whereas academics can be in post for decades. This ‘churn’ can lead to poor institutional memory within government, and therefore good systems and structures are crucial. Many departments have developed libraries and digital tools to improve the storage and curation of evidence, meaning it can be available to support decision-making when it is required. There is also a huge demand for evidence synthesis, and this doesn’t always have to mean a lengthy systematic review. Rapid reviews can play an important role where optimality, in lieu of perfection, is still good enough. All of this can help ensure that there is a greater chance of impact, but there also needs to be a (mutual) incentive to get there.
Government decision-makers operate within parameters not always factored into academic research and the practical constraints of implementing some policies – even with the will to do so – can be insurmountable. This is why initiatives such as fellowships and secondments can be so powerful as they allow academics to conduct research within the right environment. This means decision-making parameters are factored into research and there is a more obvious pathway to impact from the start. Thankfully there are increasing opportunities for this type of collaborative working and many departments are seizing them. Fewer are opportunities in the other direction but even this is improving, allowing civil servants to develop themselves and their thinking alongside – or away from – their day-jobs.
Conclusion
Underscoring everything that has been written is the ‘so what?’ factor. Acknowledging the complex system and pathways to impact, the relationships required, and the time needed to be invested on both sides, what is it that ultimately leads individuals to pursue it? Optimistically, it is shared desire to make the world a better place through research that supports evidence-based decisions and stronger policies and practice as a result. Below that, and differing inside academia and government, are precise motivations and incentives on both sides, as well as challenges to overcome. Through the support of knowledge brokers on both sides and networks – such as the Policy Knowledge Brokering Forum (PKBF) – working with dedicated academics and civil servants, the gap between the two worlds is narrowing. The pathway to impact may be winding but the details on the map are becoming clearer.