hero image

Assessing Program Grants: Are They Making an Impact? [Part 2 of 2]


Author: Nicholas Randell

PART 2

[This second (and final) installment takes a look at recent Tower grants and their impact on the target population, grantee organization, and the field at large.]

Once again, we wanted to take a look at the somewhat broader impact of our grantmaking. On our lessons learned forms, we also scored grants for impact (whether “significant,” “some,” or “no impact”).

We looked at three kinds of impact:
1) impact on target population (was a clinic able to see more young patients in a given month?);
2) impact on organizational capacity (did an agency develop new in-house training capacity?);
3) impact on the field (did the work help convince a state agency to expand reimbursements to a new service delivery model?).

Below are three pie charts showing what we found for the 47 grants we looked at. I’ve tried to capture some key features of projects that demonstrated “significant” impact on the target population. Here are some things that were true of grants with target population impact rated “significant.”

IMPACT ON TARGET POPULATION

  • Job skill development actually aligned with jobs in community
  • Behavioral screening in Kindergarten and/or Pre-K becomes standard operating procedure (cited twice)
  • Counseling services maintained at high school
  • Agency has capacity to offer new intervention to youth (cited three times)
  • Integrated care model will reduce wait times for mental health clients
  • Follow up support model now available for youth with psychosis
  • More in-home therapy providers available in Essex County
  • Prevention curriculum implemented in area school district
  • Expanded access to recovery programs for youth
  • Teachers feel more confident in ability to help students
  • Youth and families involved with Juvenile Court get more responsive support services
  • Validation of effectiveness of after-school intervention for ASD (Autism Spectrum Disorder) youth can lead to more evidence-based programming generally available
  • Social engagement programming for DD/IDD youth sustained
  • School district culture change creates more welcoming/supportive environment
  • First responder training will improve interaction with youth that exhibit challenging behaviors
  • Increased capacity to support readers with print-based learning challenges
  • Work on college inclusion model equips colleges to improve experience for youth, improve advising, improve integration with vocational supports, and improve transition from high school
  • Art programming builds confidence in trying new things


IMPACT ON ORGANIZATION

Grants that demonstrated significant impact on organizational capacity were characterized by:

  • Evidence-based intervention implemented (cited four times)
  • Clinic processes and efficiencies improved
  • Mental health screening a standard practice in the school/early childhood center
  • Screening protocols in place in outreach program
  • Client intake processes streamlined
  • Teamwork culture improved
  • Trauma informed training completed
  • Implementation of behavioral intervention was a culture changer for childhood center
  • Expanded capacity to provide in-home therapy
  • Program expanded volunteer base and created Family Advisory Council
  • Successful research (and publication) elevated profile of center
  • University developed capacity for first-responder training
  • Drug recovery program has integrated behavioral supports
  • Arts program better equipped to engage participants with intellectual disabilities


IMPACT ON FIELD

We expect the field-level impact of our Core Programs and Services grants to be fairly modest, simply because they tend not to have field-level objectives up front. Nonetheless, about 58% had at least some potential impact on the field.

Here are a few factors that seem to contribute to field level impact:

  • A program supporting vocational skill development for individuals with intellectual disabilities is up to 20 sites in NYS
  • Commitment to broad-ranging mental health supports in schools is snowballing
  • Publication of research findings on brain-based formats for reading interventions
  • Organization-wide commitment to trauma-informed training reinforces trend
  • Agency received several national awards that reflect validation of recovery coach model
  • Model for juvenile court supports is spreading across MA, potential model for additional replication
  • Documentary, publication, and national presentations promote success of family driven planning program focused on foster care
  • Promising program now ready for dissemination; another developing program ready for Phase III evaluation (with federal grant pending)
  • Trauma model in schools is shared/promoted at national level, program director is a co-chair of a state commission on safe and supportive schools
  • Highly successful MTSS implementation offers lessons and a template to other districts
  • Model for in-home therapy has been connected to a statewide initiative for replication
  • Blueprint has been created for college inclusion that may achieve better advising, and improved support of academic performance, life skills development, and vocational skill growth