Introduction
Child welfare systems must comply with federal, state, local, and judicial mandates while also attempting to achieve positive safety, permanency, and well-being outcomes for children and families. To support these varied goals, Public Knowledge® works with agencies on system redesign and improvement initiatives to increase efficiency, effectiveness, and quality of services to children and families. The implementation process itself is critical to ensuring that the improvement initiatives are executed with fidelity to the desired initiative or design, that the groundwork and planning to support the initiatives are in place, and that the process of implementation occurs at a pace that allows for monitoring and adjustments along the way.
Public Knowledge® Has Experience Using Implementation Science in Child Welfare and We Rely on That Experience in Every Engagement
By basing our work with agencies in implementation science, we ensure that the implementation of improvement initiatives and their related components is conducted in a way that minimizes barriers, promotes successful implementation, leads to desired outcomes, and helps guarantee the sustainability of the initiatives. In Public Knowledge®’s experience in implementing major improvement initiatives, we have followed four interconnected phases: beginning with development, then planning and preparation, initial implementation, and, finally, full implementation. Public Knowledge® was part of the team that created the Development, Implementation, and Assessment Approach, which focuses on best practices in implementation science and has been tested and refined in real-world child welfare settings. Although a rigorous implementation strategy may seem time-consuming initially, investing in a carefully considered strategy will help to avoid later pitfalls associated with lack of readiness, capacity to implement, or lack of clarity about what is to be implemented.
As defined by the National Implementation Research Network (NIRN), implementation is a specified set of activities designed to put into practice an activity or program of known dimensions. Implementation science is an emerging field of science designed to understand the many elements and components of implementing an initiative successfully into practice in a sustainable, valuable way. Technically, implementation science focuses on the implementation of evidence-supported initiatives (ESI), or initiatives that are well-operationalized and that have been evaluated. However, in fields with limited ESIs, such as child welfare, we are using the principles of implementation science to understand better how to develop and implement initiatives to improve outcomes in a way that will help to build the evidence base.
Why Is Implementation Science Important?
Historically, child welfare systems have either done a good job of implementing ineffective initiatives or have done a poor job of implementing some potentially effective initiatives. The result of either combination is less than desirable. Combining research-informed and well-operationalized initiatives with an ineffective implementation strategy generally results in no improvement in outcomes. Similarly, outcomes are not likely to improve if an initiative is well-implemented but does not fit the needs of the population or the problem to be solved, or if the initiative is not well-operationalized.
Here are some practical examples of challenges agencies have encountered:
- A county-administered child welfare agency adapted two initiatives that showed promising evidence of improving permanency outcomes for children by improving the protective capacity of the caregiver(s) through a structured capacity assessment and change-focused service provision. As these initiatives were focused on the caregivers, they were not appropriate initiatives for the population of children in long-term foster care that no longer had available caretakers.
- In a second example, a state agency that was developing and implementing a practice model began implementation in some areas of the state with great enthusiasm and positive trends in outcomes. As they continued to implement statewide in some areas that had inadequate numbers of staff to implement effectively, the expected results were not achieved. The state agency also implemented the practice model without a fidelity process, so it was impossible to know across the state if the staff were, if fact, incorporating the concepts of the practice model in their work with children and families.
The effective implementation helps to address these types of challenges. Using an effective implementation approach in child welfare, we can identify research-informed and well-operationalized initiatives that are matched to the population and problems to be addressed. Then using evidence-based strategies for implementation, we can help state and local child welfare systems make needed changes and improve outcomes.
The Public Knowledge® Approach to Effective Implementation
Public Knowledge® builds on existing models of implementation by using data to guide all implementation activities; by using teams to design, develop and implement initiatives, and by developing monitoring processes that allow tracking and adaptation of activities as needed. We use implementation science concepts to plan and implement statewide child welfare practice models, to design and implement continuous quality improvement (CQI) processes, and to support a range of other initiatives that serve children and families.
Our implementation model includes four interconnected phases, beginning with development, then planning and preparation, initial implementation, and, finally, full implementation.
Development Phase
In the development phase, we work with the agency to identify the problems and root causes to be addressed, explore, and identify potential solutions, install teaming structures to guide the work, develop a theory of change from the root causes to the desired outcomes, select a solution (initiative) that matches the problem, and develop fidelity measures for the chosen improvement initiative to determine if the initiative is implemented as designed.
Planning and Preparation Phase
In the planning and preparation phase, we work with the agency to assess the organization’s capacity and readiness to implement change, including:
- The commitment of the agency leadership, demonstrated by support for the initiative.
- Commitment from stakeholders.
- Organizational climate and culture.
- Perception of the child welfare agency within the larger child welfare system.
- Motivation of supervisors and staff to adopt a new practice.
- Compatibility of the initiative with other initiatives occurring within the agency; and
- The human, technical, and fiscal capacity of the agency.
We work with the agency to develop plans and strategies for implementation activities, communication, monitoring the process, and for training and coaching staff on the initiative. In this phase, since communication is not one-directional, we assess how communication needs to occur, when and with whom, the frequency, the method, and the necessary feedback loops. We also assist the agency with the development of training and coaching plans to support the learning needed to build the competence and confidence of the staff responsible for delivering the initiative, including private providers and resource parents.
Initial Implementation Phase
In the initial implementation phase, we assist the agency to begin implementation of the initiative on a limited basis, often in a specific geographic area. Data is used to:
- Test critical elements and make modifications to the initiative and/or the implementation process.
- Establish baselines and begin to track progress; and
- Examine qualitative and quantitative findings.
We work with the agency to test the initiative and develop plans for sustainability of the initiative. Along with the agency, we reassess the functionality and relevance of the teaming structure and adjust as needed. We communicate progress internally and externally to support initial implementation efforts.
Full Implementation
In full, the sustained implementation we work with the agency to bring the initiative to scale in the implementing jurisdiction. This means the initiative becomes fully integrated into the organization’s day-to-day operations by incorporating it into the established policies and practices. We also work with the agency to assess outcomes and implement the plan for sustainability, including the integration of CQI to support sustainability and the establishment of a fully functioning decision support data system that informs decision-making throughout all levels of the initiative.
Conclusion
Public Knowledge® understands that implementation is not always a linear process. While early activities provide initial steps for successful implementation, we work with agencies to understand that we can always re-visit a stage or task and make adaptations as needed.
There are few, if any, shortcuts to effectively implementing initiatives that serve children and families in child welfare. When agencies identify and select research-informed initiatives, operationalize them, and use effective implementation strategies we’re able to lay the foundation for improved outcomes and evidence building.
1 The Development, Implementation, and Assessment Approach can be found at:
http://www.acf.hhs.gov/sites/default/files/cb/develop_implement_assess_approach.pdf
2 Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa,
FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI
Publication #231).
3 Fixsen, D., Blasé, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children. 79, 213 – 230.
NOTE: Public Knowledge® was formerly the Center for the Support of Families.