Administration on Children and Families/Children’s Bureau

Community-Based Child Abuse Prevention (CBCAP) Program Draft Definitions for the Levels of Evidence for Evidence Based Practice/ Evidence Informed Practice

Level I - Emerging Programs and Practices

PROGRAM CHARACTERISTICS

  • The program can articulate a theory of change, which specifies clearly identified outcomes and describes the activities that are related to those outcomes.  This may be represented through a program logic model or conceptual framework that depicts the assumptions for the activities that will lead to the desired outcomes. 
  • The program may have a book, manual, other available writings, and training materials, OR may be working on documents that specify the components of the practice protocol and describe how to administer it.
  • The practice is generally accepted in clinical practice as appropriate for use with children and their parents/caregivers receiving child abuse prevention or family support services.

RESEARCH & EVALUATION CHARACTERISTICS

  • There is no clinical or empirical evidence or theoretical basis indicating that the practice constitutes a substantial risk of harm to those receiving it, compared to its likely benefits.
  • Programs and practices may have been evaluated using less rigorous evaluation designs with no comparison group, including “pre-post” designs that examine change in individuals from before the program or practice was implemented to afterward, without comparing to an “untreated” group – or an evaluation may be in process with the results not yet available.
  • The program is committed to and is actively working on building stronger evidence through ongoing evaluation and continuous quality improvement activities.

Level II - Promising Programs and Practices

PROGRAM CHARACTERISTICS

  • The program can articulate a theory of change, which specifies clearly identified outcomes and describes the activities that are related to those outcomes.  This is represented through presence of a program logic model or conceptual framework that depicts the assumptions for the activities that will lead to the desired outcomes. 
  • The program may have a book, manual, other available writings, and training materials that specify the components of the practice protocol and describe how to administer it.  The program is able to provide formal or informal support and guidance regarding program model.
  • The practice is generally accepted in clinical practice as appropriate for use with children and their parents/caregivers receiving child abuse prevention or family support services.

RESEARCH & EVALUATION CHARACTERISTICS

  • There is no clinical or empirical evidence or theoretical basis indicating that the practice constitutes a substantial risk of harm to those receiving it, compared to its likely benefits.
  • At least one study utilizing some form of control or comparison group (e.g., untreated group, placebo group, matched wait list) has established the practice’sefficacyover the placebo, or found it to be comparable to or better than an appropriate comparison practice, in reducing risk and increasing protective factors associated with the prevention of abuse or neglect.  The evaluation utilized a quasi-experimental study design, involving the comparison of two or more groups that differ based on their receipt of the program or practice.  A formal, independent report has been produced which documents the program’s positive outcomes. 
  • The local program is committed to and is actively working on building stronger evidence through ongoing evaluation and continuous quality improvement activities.  Programs continually examine long-term outcomes and participate in research that would help solidify the outcome findings.
  • The local program can demonstrate adherence to model fidelity in program or practice implementation.

Level III - Supported Programs and Practices

PROGRAM CHARACTERISTICS

  • The program articulates a theory of change, which specifies clearly identified outcomes and describes the activities that are related to those outcomes.  This is represented through the presence of a detailed logic model or conceptual framework that depicts the assumptions for the inputs and outputs that lead to the short, intermediate and long-term outcomes.
  • The practice has a book, manual, training, or other available writings that specify the components of the practice protocol and describe how to administer it.
  • The practice is generally accepted in clinical practice as appropriate for use with children and their parents/caregivers receiving child abuse prevention or family support services.

RESEARCH & EVALUATION CHARACTERISTICS

  • There is no clinical or empirical evidence or theoretical basis indicating that the practice constitutes a substantial risk of harm to those receiving it, compared to its likely benefits.
  • The research supporting the efficacy of the program or practice in producing positive outcomes associated with reducing risk and increasing protective factors associated with the prevention of abuse or neglect meets at least one or more of the following criterion:
    • At least two rigorous randomized controlled trials (RCTs) in highly controlled settings (e.g., university laboratory) have found the practice to be superior to an appropriate comparison practice. The RCTs have been reported in published, peer-reviewed literature.  

    OR

    • At least two between-group design studies using either a matched comparison or regression discontinuity have found the practice to be equivalent to another practice that would qualify as supported or well-supported; or superior to an appropriate comparison practice.
  • The practice has been shown to have a sustained effect at least one year beyond the end of treatment, with no evidence that the effect is lost after this time.
  • Outcome measures must be reliable and valid, and administered consistently and accurately across all subjects.
  • If multiple outcome studies have been conducted, the overall weight of evidence supports the efficacy of the practice.
  • The program is committed and is actively working on building stronger evidence through ongoing evaluation and continuous quality improvement activities.
  • The local program can demonstrate adherence to model fidelity in program implementation.

Level IV - Well Supported Programs and Practices*

PROGRAM CHARACTERISTICS

  • The program articulates a theory of change, which specifies clearly identified outcomes and describes the activities that are related to those outcomes.  This is represented through the presence of a detailed logic model or conceptual framework that depicts the assumptions for the inputs and outputs that lead to the short, intermediate and long-term outcomes.
  • The practice has a book, manual, training or other available writings that specify components of the service and describe how to administer it.
  • The practice is generally accepted in clinical practice as appropriate for use with children and their parents/caregivers receiving child abuse prevention or family support services.  

RESEARCH & EVALUATION CHARACTERISTICS

  • Multiple Site Replication in Usual Practice Settings: At least two rigorous randomized controlled trials (RCT's) or comparable methodology in different usual care or practice settings have found the practice to be superior to an appropriate comparison practice. The RCTs have been reported in published, peer-reviewed literature.
  • There is no clinical or empirical evidence or theoretical basis indicating that the practice constitutes a substantial risk of harm to those receiving it, compared to its likely benefits.
  • The practice has been shown to have a sustained effect at least one year beyond the end of treatment, with no evidence that the effect is lost after this time.
  • Outcome measures must be reliable and valid, and administered consistently and accurately across all subjects.
  • If multiple outcome studies have been conducted, the overall weight of the evidence supports the effectiveness of the practice.
  • The program is committed and is actively working on building stronger evidence through ongoing evaluation and continuous quality improvement activities.
  • The local program can demonstrate adherence to model fidelity in program implementation.