Co-producing evaluations: Sharing a journey of discovery

Ruth Unstead-Joss from the Evaluation Exchange shares her experience of supporting projects as they co-produce evaluations of their work through the Measuring Success in Co-production programme.

A need to better evidence the value of co-production

There’s relatively little formal evidence of the value of co-production. The Value of Co-Production Research conducted by Co-Production Collective in 2022 identified a need to learn about co-production processes and better demonstrate how co-production impacts projects and everyone involved.

In response, Co-Production Collective invited me and my Evaluation Exchange colleagues to be involved in co-producing the Measuring Success in Co-Production programme that aims to address this gap. Specifically, we were asked, as part of the programme’s wider team, to support eight funded projects as they co-produce evaluations of their work. The programme team included lived experience co-producers, staff from Co-Production Collective, UCL’s Grand Challenges, the Academy of Medical Sciences and ourselves from the Evaluation Exchange.

What evaluation is and the importance of inclusion

In simple terms, ‘evaluation’ is the process of:

• Checking if you've done something the best way
• What has happened as a result of it
• Seeing what improvements could be made to the way you did it.

A lot of language commonly used in evaluation can feel alien to those who aren’t familiar with it. There’s also a perception that it’s a technical process that requires specialist training and education. However, for evaluation to be useful and valid, it should be inclusive.

Barriers need to be broken down to ensure a diversity of people play key roles in evaluations and to challenge common perceptions of:

• Who decides what should be evaluated
• How evaluations should be done
• Who is analysing and sharing information
• What is done with evaluation findings.

Who are the Evaluation Exchange and why did we want to be involved?

The Evaluation Exchange builds skills and confidence in evaluation. We are a collaboration between University College London (UCL) and the community organisation Compost London. We are also a group of people passionate about inclusive and accessible evaluation approaches that work in the real world. So, we were delighted to be part of a programme that aims to better evidence the value of co-production and share learning about co-producing evaluations.

Developing a programme of learning around evaluation in co-production

My first task from June 2023 was to co-ordinate a small group of people that aimed to design how the eight funded projects would be supported to co-produce evaluations. We worked closely with the rest of the Measuring Success team to align with other aspects of the programme such as communications, funding decisions and the programme’s wider purpose.

Together, we settled on a structure of support for the teams including four ‘training’ sessions over six months and agreed that the sessions would focus on different aspects of co-producing evaluations, with some ad-hoc coaching and guidance available to teams if needed. We embraced the name ‘Deepening Practice’, suggested by one of our lived experience co-producers, to describe the sessions.

Guidance and encouragement in co-producing evaluation

From January to June 2024 my role was to coordinate delivery of the Deepening Practice sessions. Each of the four half-day sessions were attended by 25-30 participants from the eight funded projects. By exploring participants’ different needs prior to the sessions, we discovered that, whatever their background, there were varying degrees of confidence in evaluation or co-production for different reasons.

The four sessions covered planning evaluations, exploring different evaluation methods, tackling challenges and analysing and sharing their findings. Co-designing the sessions, we explored the best way to cover chosen topics whilst ensuring that the diversity of people participating felt comfortable in the sessions and found them useful.

Co-production can sometimes feel messy and unearth different perspectives that can feel uncomfortable. We spent time considering how we could create an environment in which people felt comfortable to come together and navigate conversations about evaluation and co-production when these tensions might be present in their team.

It was important to us all that we created an environment in which every project team member felt they had a valued role to play in co-producing their evaluations. We also felt it was important for us to not dictate the evaluation approach each team should use. Our aim was to help teams explore different methods so they could decide their preferred approach. We felt strongly that teams would know best their own needs, interests, context and people they were working with.

Positive feedback and valuable lessons

Feedback on the sessions was really positive. Not only did participants appreciate the range of content and different ways we tried to make sessions engaging, they also mentioned how supportive and caring the environment was. This was testament to everyone involved in session design and delivery who took great care to think through session ideas and how best to facilitate activities.

There are always valuable lessons to learn. Back in June last year, one of the things I had looked forward to most was working with people with a diversity of perspectives and different experience to me. A year later, I’ve really valued feedback from the teams on how we could adjust session content to create more space for exploration and practical examples from them to further improve accessibility. I’ve also learnt more about co-production, particularly different people’s perceptions of what co-production means and how you might work with this and to your project’s advantage.

What I have learned from being involved

It’s been uplifting to see how teams have taken ideas from the Deepening Practice sessions and developed them to suit their own needs. For example, the Aphasia New Music Group adapted a simple ‘bull’s eye’ tool that we shared in a session exploring creative ways to capture information. Using the tool, they developed an activity that was accessible for people with aphasia who may have difficulties communicating. I was also inspired by them telling us their music has changed since their involvement in the programme and they shared their reflections on whether their approach to co-producing their evaluation had had a role in this.

I’ve also been reassured to discover that we can learn from others’ experience of co-producing evaluations from across the UK and internationally. For example, in our Deepening Practice sessions, we shared relevant evaluation resources (a selection of which are listed at the bottom of the blog). We also invited others to share their experience with the teams. These included Jeremy Spafford from the Old Fire Station in Oxford who joined our session exploring storytelling as an evaluation approach, and Lynn Laidlaw who shared her personal experience as a lived experience co-producer of co-producing research . Participants told us how valuable it was to hear from others with similar experiences to them and to be inspired by others’ ways of working.

‍

Thank you!

I’ve been grateful for an opportunity that’s built my own understanding of more inclusive approaches to evaluation, and to work with new people to deliver evaluation and co-production support. Thank you to everyone in the Measuring Success in Co-Production team and beyond!

‍

Find out more - learn and connect with others

If you want to learn more about co-producing evaluations, keep an eye on the insights emerging from the Measuring Success in Co-production programme.

Coming up the project teams will be sharing their learnings with us in September, reporting on their projects and we will be sending an invitation to join an online session to celebrate the learnings in late November/early December.

Connect with the Evaluation Exchange

If you would like to learn more about the Evaluation Exchange’s work with the Co-Production Collective or our work more broadly, get in touch via:

Email: r.unstead-joss@ucl.ac.uk, LinkedIn or the ‍Webpage

Useful resources exploring coproducing evaluations ‍

• ‍Co-production Network for Wales have trialled ways to evaluate co-production
• Scope, with their commitment to tackling inequalities, have co-produced evaluations of their work.
• Influencing how evaluations are commissioned, Queen Mary University of London’s ‘Evaluation Evolution’ encourages inclusive approaches to commissioning evaluations.
• There are international movements advocating for ‘decolonising evaluation’ that challenge power dynamics found in traditional evaluation approaches.
• UK groups such as the Equitable Evaluation Collective are tackling inequities and bias.

‍

Headline photo credit: Photo by Rido Alwarno on Pexels

Other images: From slides prepared by Evaluation Exchange for Deepening practice sessions 3 and 4