By Sarah Lawrason and Emilie Michalovic
Persons living with disability are up to 60% less likely to meet recommendations for physical activity participation (Carroll et al., 2014). The lack of physical activity participation among persons with disabilities is largely related to barriers such as societal attitudes, knowledge gaps among organizations (e.g., how to adapt or design appropriate programming), accessibility of facilities, and financial constraints (both personal and organizational; Martin Ginis et al., 2016). Community organizations across Canada play an important role in supporting physical activity opportunities, particularly those that work to remove barriers and alleviate inequities experienced by Canadians with disabilities.
To successfully support their mission and deliver programming to their community members, most community organizations need to conduct thorough evaluation. This evaluation helps them to:
- identify gaps to fill,
- make improvements and raise the quality of services,
- report back to sponsors, donors, and other funders,
- or stay accountable to their strategic plan.
However, evaluation can come with many challenges and require capacity in terms of people power, dedicated evaluation resources, or expertise in program evaluation. Conducting an impactful evaluation can take a lot of time and involves a lot of decision-making. Staff members at organizations have to decide what they would like to capture, who they can go about understanding that information or asking people about those topics, and then decide how they will be able to gather everything together to share back to funders and community members. These challenges can unfortunately compromise organizations’ opportunities to acquire funding or grow their programming and impact.
At Abilities Centre, we also evaluate all our programming for the same reasons identified above. We are fortunate to have a dedicated research department that supports the evaluation of our programming, but it also requires time and commitment from people all across the centre to continuously identify gaps, find solutions, and share back experiences to the community.
Recently, a group of community organizations and academic researchers came together to find a solution that would make evaluation more accessible for all organizations, even those that do not have dedicated evaluation team members. This group worked to gather feedback from the community and create an easy-to-use toolkit that would ease resource and decision burdens for evaluation. The toolkit was specifically designed to include evidence-based indicators and practical measures that will streamline evaluation for disability physical activity organizations. By using this toolkit, the creators hope that organizations will be better able to assess their programs, identify improve actions, enhance programming for the community, and ultimately, make a meaningful impact on the people they serve.
How was the toolkit created?
The toolkit was created by a team of researchers and representatives of community organizations. Researchers from The University of British Columbia (UBC), Queen’s University, and McGill University were involved alongside individuals from Abilities Centre, Revved Up, The Steadward Centre, Pickering Football Club, Rocky Mountain Adaptive, Active Living Alliance, and BC Wheelchair Sports. This group wanted to ensure that the final toolkit would fill gaps and be an effective support for organization. This process they used was thorough and happened over 4 phases:
1 - The project team reviewed the literature in the field of disability and health programming. The combed through the available information on the topic to gather a list of possible indicators of program success or program elements for assessment. The list was then reviewed by the community members on the project team and other disability sport programmers to determine if the indicators were a) relevant and b) helpful for program evaluation. After review, there was a list of 57 indicators for the team to consider for the toolkit development during step 2.
2 -The 10 project partners provided feedback on the usability of the possible indicators for the toolkit based on their own personal and professional experiences. The goal of this step was to understand how the indicators could be adapted to specific local organizations. This step also involved considering how the various elements could be measured or captured during evaluation. Through this process, measures that align with the 57 indicators in the toolkit and meet the needs of the disability physical activity organizations were identified.
3 - The project team conducted interviews with community partners to hear their insights and gather valuable feedback on the updated indicators and toolkit. Their feedback included user interface suggestions, specific content modifications, and ways to enhance the toolkit's overall user experience and relevance.
4 - The goal of this project was not to create something that was just theoretical but something that could be broadly and effectively implemented. Once all the feedback was addressed and a user-friendly toolkit was created, the project team created a plan to share the toolkit widely for use by organizations and programs across Canada. To help with sharing, information packages, interactive webinars, and targeted publications highlighting the toolkit’s resources and knowledge were created.
Where is the toolkit now?
The program evaluation toolkit is now available and accessible at https://et.cdpp.ca.
This toolkit is as a practical resource for program evaluation and is designed specifically for disability physical activity organizations. It provides a structured approach to assess program initiatives as well as offering tailored indicators, feasible measures, and valuable insights garnered from community interactions.
- Users can learn about how to use the toolkit and conduct program evaluations through the resources on the website.
- Each indicator is provided with a definition, a measure, the type of measure (e.g., interview, survey), and the time it takes to complete that measure.
- When finished, the website provides a downloadable or emailed PDF that details the evaluation plan and provides a link to each measure.
Information on this toolkit and its creation was originally published in Abilities Magazine (
https://www.abilities.ca/). Find the full article here (
https://www.bluetoad.com/publication/?m=27341&i=810088&p=26&ver=html5)
About the authors:
Sarah Lawrason completed her PhD in Kinesiology in 2022 from the University of British Columbia (UBC). She is currently a Postdoctoral Fellow at UBC and a Research Analyst at the Public Health Agency of Canada. Her research interests include physical activity and disability, knowledge translation, program evaluation, and digital health.
Emilie Michalovic (she/her) is the Director of Research at Abilities Center. She holds a PhD in Kinesiology Sciences from McGill University, where she conducted research on program implementation and evaluation with individuals living with chronic obstructive pulmonary disease. Emilie's other research interests include behaviour change and motivation for physical activity among persons with disability.
References:
Carroll, D. D., Courtney-Long, E. A., Stevens, A. C., Sloan, M. L., Lullo, C., Visser, S. N., ... & Dorn, J. M. (2014). Vital signs: disability and physical activity—United States, 2009–2012. MMWR Morb Mortal Wkly Rep, 63(18), 407-413.
Martin Ginis, K. A., Ma, J. K., Latimer-Cheung, A. E., & Rimmer, J. H. (2016). A systematic review of review articles addressing factors related to physical activity participation among children and adults with physical disabilities. Health psychology review, 10(4), 478-494.