Nova Scotia Poverty Reduction Blueprint Evaluation

The Work is Messy: An Evaluation Process that Hits it out of the Park!

Written by Cari Patterson, Director of Research and Evaluation

Inspiring Communities is partnering with the Province of Nova Scotia to evaluate projects funded through its Poverty Reduction Blueprint work.

The Province is funding action-oriented initiatives that test innovative ways of addressing poverty, through three funding streams:

• community projects
• strategic government projects
• social innovation labs

The learnings from these projects will inform the Poverty Reduction Blueprint for the Province.  The evaluation approach is developmental and appreciative, and is designed to build people’s comfort and capacity with evaluation itself. 

Working in partnership with government funders and trusted local community organizations, our team went to communities and invited them to reflect on their work together through a series of ‘community conversations’ held across Nova Scotia. One of the main data collection tools we used was Success, Evidence, Strategies (SES), developed by Kataraina Pipi, Nan Wehipeihana & Kate Mckegg. The process yielded comprehensive data from a range of perspectives. It also allowed stakeholders to step back and look at the bigger picture of their work together, see and celebrate their achievements, generate relevant indicators, identify critical success strategies that can inform other communities, and reflect on the implications of their work going forward. By coming together for these conversations, they also built new connections and planted seeds for new partnerships. Finally, this process and the SES tool helped demystify evaluation, and offered people a practical process they can use on an ongoing basis to evaluate and reflect on their community development practice.  

Lessons Learned from this evaluation process:

• Participants in the community conversations felt valued because we ‘came to them’ to hear their stories. Many said they were happy that we were coming to hear their stories, and not asking them to ‘fit their stories in check-boxes on a form’. They were eager to talk about their projects, and proud of their roles and capabilities.
• Asking about the supports needed for people to participate – and providing them – facilitated more inclusive participation. Through this process, we were able to provide supports for childcare expenses, travel costs, translation, and other needs people identified. We also provided cash honouraria (based on a living wage) for all participants, and we had refreshments at all sessions. In more than one case, participants offered their honouraria to others who were going through difficult challenges.
• Government funders understanding the importance of providing sufficient funding to address barriers to participation is critical. So too, is their knowledge of trusted organizations in each area who could act as co-hosts for the community conversations, and offer insights and advice about helpful supports. 
• The intention of bringing people together to learn from each other and work together in the future was successful. Although participants were from the same geographical areas, they work in different sectors, and many did not know each other. The facilitators observed a great deal of cross-fertilization and relationship building among groups. Participants expressed gratitude for hearing each other’s views – they were very pleased to learn from each other, learn about other services in their communities, and connect with potential partners for their ongoing work. At the end of some sessions, people did not leave their seats, and instead kept the conversations going in a circle. They discussed future collaborations. Many exchanged contact information after the sessions.
• We went beyond traditional evaluation by engaging project leads, partners, and participants to reflect together on their work, and on their impact. They expressed  appreciation for having these conversations together. Many expressed that the discussions helped them understand each other’s perspectives, and better understand the bigger context in which the work takes place. Some said that the act of reflecting together was helpful for them in determining future priorities and direction in a meaningful way. 
• The SES tool was accessible and empowering. Completing the reflection process helped people articulate their shared sense of accomplishment, honoured their knowledge and different ways of knowing, and helped them realize that they were implementing a number of critical strategies for bringing about their achievements. This validated their ways of working, and helped them realize that they knew a lot about how to do the work.
• Modeling a user-friendly process (and providing copies of the SES tool) gave people something they could use to evaluate their own work. The appreciative process demonstrated that evaluation can be a useful learning experience without being scary. Many participants left the community conversations with the intention of using the SES tool – and some have reported back that they had very good experiences using it.