Methods for Successfully Adopting Evidence-Based Programs

Implementing Evidence-Based Practices and Measuring Results:

Logic Models: The Logic Model provides an effective method for charting progress from initial and short-term outcomes toward intermediate and long-term outcomes. A Logic Model focuses on identifying the logical links between the outcomes you desire, your program assumptions or theories, and your program strategies or services.

How to Keep to the Fidelity of Evidence-Based Programs

What is Fidelity? - Implementing a model program as designed with the recommended techniques, in the recommended setting, to the recommended population, for the recommended time and duration.

Why is Fidelity important?

  • Anticipated research-based outcomes are based on adherence to the program model and design.
  • Participation can be impacted.
  • Anticipated outcomes may be negatively impacted, or unanticipated outcomes may occur.

What are Fidelity criteria components?

  • Written Curriculum
  • Key Replication Components
  • Standardized Training
  • Defined Target Population
  • Defined Setting
  • Defined Number and Duration of Sessions.

Who Develops the Criteria?

  • The developer of the evidence-based program usually provides the fidelity criteria.  This information is often provided on the developer’s web site. 
  • Federal agencies that identify or endorse evidence-based programs may also have this information.
Fidelity Issues: Questions to Ask
  • What training requirements does my staff need?
    • Is it one-time training or on-going training?
  • Are there frequency/intensity/dosage requirements for implementation?
  • Are there caseload limits for staff?
  • Is there a curriculum manual that is used in association with this program?
  • Is there technical assistance available to us while we implement the program?
  • Are there fees associated with my agency using this program?
  • Are there specified evaluation tools that we need to use?

Adaptation and Fidelity

Adaptation can be defined as changing an aspect of the evidence-based program:

  • To be culturally relevant
  • To meet community norms or standards
  • To meet partner or key stakeholder expectations
  • To better meet participant needs
  • To adapt to facilitator/trainer knowledge and skills.

Research shows that adaptations are more effective when:

  • Underlying program theory is understood;
  • Core program components have been identified;
  • Both the community and needs of the population have been carefully defined.

(Source:  Achieving Outcomes: A Practitioner's Guide to Effective Prevention SAMHSA, 2002.

The ultimate goal is to maintain the basic integrity of a program while matching the innovation to the unique features of the setting and the preferences/reactions of the relevant population. (Jason, Durkal & Holton-Walker, 1984)

To Adapt or Not to Adapt?
The decision to adapt a program is difficult and certain steps are recommended:

  • Familiarize yourself with the program
  • Contact the program developer
  • Check needs assessment data
  • Identify adaptation needs
  • Consult with key stakeholders
  • Develop a plan.

Source: Achieving Outcomes: A Practitioner's Guide to Effective Prevention SAMHSA, 2002.

Why Self-Assess for Program Fidelity?

  • In order to ensure the best program outcomes, it is important to assess that the program is being implemented as designed.
  • To help determine whether the program you are implementing fits your program goals and target population.
  • To ensure the program is being implemented as designed.
Techniques for Self-Assessing Fidelity
  • Conduct random visits of sessions
  • Attend facilitator training to verify training is consistent with the program design
  • Document the length and duration of sessions
  • Interview the participants or facilitators
What to do with the Results of Self-Assessment?
  • Monitor adherence to intervention as originally developed and evaluated
  • Identify significant deviations from key components as they arise
  • Correct deviations by providing:
    • “Coaching”
    • Helpful resources
    • Booster training

Fidelity Resources: