Narrowing the gap between research and policy: using rapid evaluation during the COVID-19 crisis

IntroductIon an evidence base is vital to ensure well-informed policies with systematic processes.1,2 However, a research-to-policy gap exists that is widened by the discrepancy between the extensive time it takes to conduct conventional academic research and the short timescale over which policy-makers are often required to make decisions.2 this is especially concerning in times of crisis, such as the cOVID-19 pandemic, that demanded an avalanche of data, analysis, and interpretation to be provided over a very short period of time. rapid evaluation can be used to generate research-based evidence under pressure to inform decisionmaking and policy. existing literature clearly details the steps involved in conducting rapid evaluation3; yet this literature contains little practical knowledge about how best to carry out such research. In this article, we reflect upon the practical implementation of rapid evaluation for an urgent project during the crisis of cOVID-19.4

IntroductIon an evidence base is vital to ensure well-informed policies with systematic processes. 1,2 However, a research-to-policy gap exists that is widened by the discrepancy between the extensive time it takes to conduct conventional academic research and the short timescale over which policy-makers are often required to make decisions. 2 this is especially concerning in times of crisis, such as the cOVID-19 pandemic, that demanded an avalanche of data, analysis, and interpretation to be provided over a very short period of time. rapid evaluation can be used to generate research-based evidence under pressure to inform decision-making and policy. existing literature clearly details the steps involved in conducting rapid evaluation 3 ; yet this literature contains little practical knowledge about how best to carry out such research. In this article, we reflect upon the practical implementation of rapid evaluation for an urgent project during the crisis of cOVID-19. 4

BrIef Project Background and reflectIon Process
Between June and October 2020, a cOVID-19 Saliva testing Programme was piloted, where findings were reported to the UK Department of Health and Social care (DHSc). 5 a rapid evaluation was conducted with participants of the testing programme to generate insights that would inform the testing's design and modification, and the next phase of future mass-testing to the UK DHSc. 4 For further details on the project, see our published paper. 4 We used Gibb's reflective cycle 6 Noting the leadership skills of your project lead may help you vicariously develop your own leadership skills; look out for the following to help you: • Methods used to communicate information • Storage of information • How they use technology to support the project • How they interact with team members and the type of support they offer

Access to Rapid Evaluation Methods and Digital Technology
Understand the principles of a rapid approach and share them with the team at the beginning of the project to set expectations Familiarise yourself with the principles of a rapid approach and be prepared to meet its expectations clearly communicate deadlines and expectations of work timescales to your team at the start; do this by: • Storing deadlines and timelines on a central storage system that is easily accessible • ensuring every team member knows where and how to access this information • emphasising task expectations by outlining specific task details and workload involved • Setting clear expectations of the work schedule and arranging working hours based on your team's availability ensure you are prepared to reply to communications from your project lead and team members quickly to ensure task deadlines are met Balance your rapid project with your other work commitments by setting specific hours and days to work on specific projects communicate your specific working hours to your project lead to set communication expectations Update project information as soon as possible and communicate these updates to your team; using a real-time reporting system (e.g. on Microsoft teams) will greatly assist this communication create a plan that can be flexible, based on your deadlines, your team's working hours, and skills of team members (4) access to rapid evaluation methods and digital technology.
The value of a shared vision the pandemic crisis gave the work an urgency and significance felt by the team, resulting in both a personal and professional commitment to the project. team members felt positive about the contribution they were making to the national campaign to reduce spread of the infection. this drive facilitated a proactive and supportive work culture, where team members responded to communications quickly and worked collectively to solve problems. all team members listened and valued opinions from other members and were quick to take up responsibilities if others did not have the capacity. a sense of trust and confidence were quickly built among the team members, who easily relied on each other to complete tasks and receive advice. the culture enabled the project to collect data beyond its target and produce outputs that were valued by DHSc for their timeliness and insights.

Effective project management
effective project management and accessible information sharing and communication were key to generating project insights within the tight deadlines required by the stakeholders. the team leader communicated the project's goals and objectives at the beginning of the project, specifying how the project was to be completed. She also held weekly meetings to discuss experiences or challenges and real-time digital summaries accessible to all on Microsoft (MS) teams. the well-organised communication of the project ensured all team members were up-to-date on the project and knew its immediate next steps. the team also had immediate access to all project-related documents and data via MS teams, such as project protocols, task delegation spreadsheets, deadlines, and result summaries.
The supportive leadership style the team leader initially identified the strengths, skills, backgrounds, and experience of team members in order to delegate tasks accordingly. the team leader's in-depth knowledge of her team led to inherent confidence in each member's ability to complete the project's tasks on time. the team consisted of 12 researchers from varied backgrounds and seniority that required active management from the team leader by listening to their opinions and concerns, and making members feel valued. Some team members were inexperienced with rapid evaluation methods and were initially worried about their contribution to the project. the team leader reassured them by stating why they were asked to be on the team and provided appropriate support where needed. When team members had opposing perspectives on aspects of the project, the team leader made the final decision through team discussion -a process made easy due to the trust built over the course of the project. encouraging team members to actively participate and support each other eased the management burden of a large team for the team leader, which prevented her feeling overwhelmed with project speed and enabled her instead to enjoy the process. the team leader was open about gaps in her knowledge and sought help when needed. these leadership skills were respected by the team, who felt they vicariously learned about effective leadership and believed they would be employing these skills in future team management activities.
the relationship between the topics discussed in the above sections is shown in Figure 1.

Access to rapid evaluation methods and digital technology
Successfully implementing rapid evaluation requires access to the right resources that can be summarised as having a large team, using rapid assessment procedure ('raP') sheets to analyse findings iteratively, and providing ongoing immediate feedback to stakeholders. 8 Figure 1 diagram showing the interaction between factors that contributed to teamwork; a shared vision bridges leadership skills and the large team with varied backgrounds, which is embedded in the working culture and project management Our project was entirely virtual due to the cOVID-19 lockdown and therefore demanded reliance on technology. We used the synchronous features of MS teams to share documents, communicate quickly, and keep track of tasks and deadlines. Our team were familiar with MS teams but it is important to consider time to support any team members who are not familiar with the technology used for any project.
all team members were working remotely, some of whom had caring duties and other work responsibilities, which resulted in a flexible working timetable. clear and strict deadlines were communicated in advance, and responsibilities were delegated based on skillset and availability as indicated on a live team calendar -a crucial resource for our project.
challenges created By the raPId evaluatIon Process team members felt pressured to be on alert for communications and reply quickly, even outside of their normal working hours due to the flexible work schedule. Some data collection tasks were scheduled quickly, which was sometimes too abrupt for some members who could not make those timings. the team leader felt particular pressure as she had to juggle leading this project with other work responsibilities. Finally, this project was ethically approved as a service evaluation and not as research because service evaluation ethics was quicker to attain. this restricted dissemination of this project's findings and delayed publication. Despite these challenges, team members openly discussed and overcame these challenges because of the culture of trust and support built from the start of the project.

conclusIon
Our experience illustrates how a rapid yet rigorous evaluation to inform policy can generate actionable results under time pressure that directly informed government policy and practice. this demonstration of how these outcomes can be achieved using entirely online methods suggests that a rapid approach to programme evaluation can be used irrespective of the location of research or public health partners, while also accommodating needs for flexible working. Furthermore, this approach makes the most of short contracts that govern much commissioned research in the UK. We have generated a list of actionable points based on our reflections on the conduct of this project which we hope could be used by anyone planning to conduct a rapid evaluation project to inform policy and practice regardless of the field of study (table 1).