Measuring Success in Co-production - Celebration and Learning Event

On 5 December 144 people joined our online Measuring Success in Co-production: Learning by Doing Event on Zoom. The event celebrated the experiences and outputs of the eight project teams funded in the programme. Alongside our Measuring Success Team, Co-Production Collective staff members, co-producers with a range of lived experience and The Evaluation Exchange (a collaboration between UCL and Compost London) and representatives from two of our co-funders, UCL Grand Challenges and The Academy of Medical Sciences.

All eight funded project teams shared some of their experiences and collective learning gained during this journey. All of this was captured visually by our amazing Graphic Facilitator Anna Geyer. If you missed it, you can catch up by listening to the recording.

Background and context

The Measuring Success Programme is funded by Research England participatory research funding, UCL Grand Challenges & The Academy of Medical Sciences. It follows on from our previous Co-production Pilot Projects and the Value of Co-Production Research Project published in October 2022. We’ve documented our journey in blogs (which can be found on the Further information tab of our Measuring Success webpage) over the past 18 months if you want to find out more about how we got to this point!

The aim for this programme was to gain a better understanding from the eight project teams of the conditions required to optimise co-production and evaluation across different mental health, wellbeing and/or climate change contexts and projects. Considering the findings from the Value of Co-production work, we wanted to not only focus on outcomes, but also how all team members are affected during their journeys.

We define evaluation as ‘how we go about measuring and understanding the difference your programme, project or process is making, learning about what does, and doesn’t work, and sharing these findings to help change or improve something. We describe this as a continuous process.’ You can find out more about our definitions for ‘measuring’, ‘success’ and ‘conditions’ on the Measuring Success webpage.

Our sense of purpose

As our programme comes to a close, we thought it would be helpful to share our sense of purpose in responding to our aim.

• Open to understanding and thinking and doing differently to enable change to happen.
• Mindful that what we develop is helpful and accessible to support others to start, plan and apply values and principles (or your own frameworks) to co-production - for different people, place and contexts.
• Curious but sensitive when measuring the ‘process’ and ‘conditions’ - to do things that are helpful and impactful when working in this more relational way.

Sharing experiences from project teams

Our motivation for this session was celebrating understanding the experiences of project teams from this ‘Learning by Doing’ approach. Our eight teams provided brief summaries of their co-production experiences on projects and what they learnt.

1. Mandy, Sophie and Ross shared excerpts from the podcast they created about Homelessness and the Dual Diagnosis of substance misuse and mental health problems. Their reflections explored how being involved in a co-production programme had impacted them, helping them to harness the benefit of lived experience to help others.
2. Katerina from the Fostering Equality and Diversity project at UCL talked about how their project had helped them to co-produce strategic priorities for the newly established Centre for Equality Research in Brain Sciences at UCL.
3. Gemma and Debora from the Meta Co-production project demonstrated their co-designed evaluation and learning tool. This is based on evidence-based lived experience-centred standards, for use in mental health co-production. You can find out more about the project in this report.
4. Tom from the Improving Mental Health project at the University of Warwick talked about the journey they had been on to review the resources and support available to students and the #GetWarwick Talking campaign they came up with to encourage staff and student to talk about mental health.
5. Six members of the Aphasia New Music Group shared a video they had co-produced which included footage of their songs. They outlined the process they went through to evaluate the creative activities of their music group.
6. Claire and Becky from the Neurodivergence, climate action & eco-hope project talked about how they recruited a new group of people to be part of their co-production team. Their project examined how neurodivergence interacts with taking climate action and they developed ways to support neurodivergent people who work in sustainability-related fields. You can find out more about this project in this video.
7. Lilliana, Siobhan, Nicci, Susan, Lou and Kate from the Co-Production from the Inside Out project shared their work. They combined co-production with their beliefs in ‘Innate Health’ in the context of mental health. This is the idea that everyone has access to their innate mental wellbeing and wisdom regardless of circumstances and that our perceptions of life are shaped by moment-to-moment thinking’. They co-produced mental health and wellbeing support for the staff at a women’s refuge. You can read more about how they did this, and the journey Siobhan has taken in her blog.
8. Maggie from the East of England psychological professions co-production group talked about their project with people with lived experience that co-produced guidance for mental health professionals that encourages them not to be over reliant on using assigned labels to define people. This will be incorporated into clinical training materials for professionals.

‍

Participants then had the opportunity to attend two breakout sessions with any of the projects they were interested in hearing more about. This gave them the opportunity to ask questions and gain a deeper understanding of what they learned from being part of the programme. It also enabled them to share their own reflections on co-production and evaluation if they wanted to.

What the Measuring Success Team have learned

Reflecting on the guiding questions we wanted to explore in the programme, our Measuring Success team gave a flavour of some of the learning that will feature in our Collective Output in January.

While making sense of the learnings from the eight projects and the programme as a whole, the team has been reviewing different sources of information collected at different stages in the programme. These data sources were self-reported (generated within our team), from an independent analysis with People’s Voice Media, and drawn from the project team reports.

‍

Having gathered all of this information from multiple sources we approached this stage of the evaluation with sensitivity to try and uncover the conditions for co-production and evaluating co-production. Managing the potential biases we all bring, was a challenging, reiterative learning process. Throughout the process, we needed to make the amount of information manageable and be pragmatic and realistic about our own expectations with the resource we had. While we wanted to know how our findings around the conditions related to the principles of co-production, it was important not to be constrained by them.

“Most of all, we wanted to retain the authentic experiences (and nuance) already captured in the data from ourselves and the project teams. Our aim was to balance the volume and complexity of information to understand the conditions without being too reductive or adding layers of interpretation, and handling sensitivities appropriately.” Vanessa, Co-Production Collective

Learning about evaluation in co-production

Many of these challenges relate to the evaluation themes that we discovered throughout the programme. At the start of this event, we gave participants a brief introduction to some of these themes and asked them to reflect on their experience of these while listening to our project teams. We re-visited this at the end of the event!

These themes were outlined in our ‘Listening guide’ and included considering:

• Dealing with the context of uncertainty in co-production
• Reflecting and adapting throughout co-production and the evaluation process
• Making sure an approach to evaluation is appropriate and proportionate‍
• Ensuring that we accept and acknowledge multiple truths and perspectives
• Capturing the different layers and complexity in co-production
• Nurturing inspiration and offering support

We will expand on these further in the forthcoming report and other materials accessible to a wide variety of different groups of people that we will share in early 2025.

A few reflections from our team about the programme

“Each co-produced project is different. However, by bearing in mind these key themes, we hope it is possible to tackle commonly faced challenges and find helpful strategies that can inform ways to co-produce evaluations and evaluate co-production.” - Ruth from Evaluation Exchange

“I’ve learnt that reflection and reflective learning is at the heart of co-production. It spans across all the principles as we know them at Co-production Collective and helps us to adjust our ways of working and learn how to do better.” - Lisa Richardson

“This whole process has been really interesting, but not easy! Like the 8 teams we did our learning by doing … and in the end, we pulled in all the different ideas and perspectives to help us to find ways that worked well enough for everybody.” - Anne Crisp

A taste of what’s to come

We shared some insights into the complexity and messiness of co-production and evaluation in this context. In early 2025, we will share more on the conditions for co-production and considerations when evaluating, along with some resources that address the learning points we outline below, which are:

• Our learning around conditions for co-production and evaluation themes lend support to the fact that an evaluation framework is unlikely to be as helpful as adopting a 'thinking and feeling' process.
• Open questions that help us to delve into the conditions (and principles) may support a flexible 'learning by doing' approach that can be tailored to different contexts.

Blog title image credit: Canva