CBR Institute Participatory Evaluation August 8, 2011 Enzo Verrilli Chiara Airaldi Rebecca Houwer
Welcome and Overview • Review last week • Realworld evaluation- introduction and warm-up activity • International example – Manta, Italy • Long break (30 min) • Participatory evaluation plan for CBR institute • Session evaluation • Short break (15 min) • Mentor group meetings
Homework Review • • • • •
Averages Mean, Median, Mode X, Y axis Strength of correlation Significance
Experiencing Evaluation • What are your experiences with evaluation? – – – – –
Workplace School Product Performance Cost/Benefit
Dimensions of Variation Giving
voting
ttc
No Influence (no power)
Influence (have power) employee review
Receiving
?
Realworld Evaluation • Chocolate Chip Cookie Evaluation – 3 kinds • What makes a good chocolate chip cookie? • Indicators: • Rating: • Stakeholder perspective:
International Context – Evaluation in Italy • See Chiara’s presentation (separate pdf)
The first Rule of Evaluation
… there will be RESISTANCE to evaluation!
The first Rule of Evaluation (continued)
… Why RESISTANCE ?
Goal of this session • De-mystify “evaluation” by learning more about it • Collaboratively plan an evaluation of the ACT for Youth Summer CBR Institute to be conducted in the session “Putting it all together” (in 2 weeks)
What is the evaluation for? Improve the program: Identify strengths and weaknesses of program activities. Build organizational capacity: Improve the ability to plan and implement programs. Document progress toward meeting program goals. Inform and refine community change efforts: Identify unmet community needs and assess the impact of social change efforts Enhance personal growth and development among staff and participants: Provide feedback to staff and participants about their work. Recognize accomplishments and provide suggestions for improvement. Provide evidence of program accomplishments: Report to boards, community members, and funders about program effectiveness.
Who wants to know? • • • • • • • • • • •
Students? Family members and friends? ACT for Youth Staff? Partner organizations?? Funders? Mentors and session facilitators? CBR Institute Coordinator and assistants? Rob Ford? Your city councillor or Antonius Clarke? Big shots at York University? Others?
Participatory Evaluation • To • For • With • • • •
Experience with each kind of evaluation? Which is participatory? Qualities of?
Youth as Objects Youth as Recipients Youth as Resources Youth as Partners
Different perspectives!
Kinds of evaluations FOCUS
FORM
Products/services (things) • cookies/ttc
Needs assessment
Performance (people) • movie/athlete/report card)
Evaluability assessment • logic models or “theory of change”
Programs/projects • Pathways to Education/ACT CBR Institute
Process Evaluation • what’s happening? (weekly session feedback forms)
Outcome Evaluation • what happened? Efficiency
Outcome evaluation • • • • • • •
What were our goals? What did we do to TRY to achieve our goals? Did we achieve our goals? How will we know if we achieved our goals? What were our challenges? What worked really well? What should we consider changing?
…. What do we recommend for the next time around?
GOAL of CBR summer institute • ACT for Youth’s Primary Objective:
» “Increase knowledge of and familiarity with community-based research”
LOGIC MODEL
INPUTS - Resources -Ingredients - Recipe, stove
OUTPUTS - Activities - Make and bake the Cookie dough
OUTCOMES - Results - Delicious cookies
Everyday example
H E A D A C H E Situation
Get pills
Take pills
Feel better
INPUTS
OUTPUTS
OUTCOMES
19
Logic Model Goal: Increase knowledge of and familiarity with community based research Inputs Outputs Outcomes Resources we need
- Hired Youth Researchers -Secured stipend funds -- mentor group research projects
Youth researchers increase knowledge of and familarity with CBR…
Before and after…
What does learning about CBR look like? Feel like? What is missing? What inputs were great? Which ones can we forget about?
ACTIVITY RECIPE for Excellent CBR Summer Institute What are the qualities of an excellent CBR Summer Institute according to each of the stakeholders (taste testers)? What ingredients (inputs) do we need to make an excellent CBR Summer Institute? What is the recipe?
RECIPE
Evaluation Plan
RECIPE Inputs (ingredients) – do we have those? What ingredients do we have? Which ones should we change? Outputs (activities) – what activities lead to the most excellent CBR Summer institute? What activities did we do? How were they? What worked and what can be improved? Outcomes (perfect cookie) – was it the most excellent CBR summer institute? If yes, why? If not, what are the constraints/challenges?
Taste-testing the recipe OUR evaluation plan How will we find out if our recipe worked?
