Evaluation of Simulation Training and Other Experiential Training Methods
A growing movement has developed to provide experiential learning to professionals in child welfare and other human services to provide a more life-like learning experience. CFRC’s evaluation research has played an important role in the development of these methods in child welfare in Illinois.
Work in human services presents both mental and emotional challenges for professionals in the field. Classroom training alone is insufficient to prepare professionals to work with clients under stress who may have suffered significant trauma. A growing movement has developed to provide experiential learning to professionals in child welfare and other human services to provide a more life-like learning experience and enhance what they gain from training. In February 2016, the Illinois Department of Children and Family Services (DCFS) collaborated with a team from the University of Illinois at Springfield (UIS) to implement the Child Protection Training Academy that provided simulation training in which child welfare professionals work with a mock family in a mock residence and mock courtroom, to simulate child protection investigations and interventions. Every new child protection investigator in Illinois participates in a week-long simulation training program enacting the investigation and court involvement of a mock family suspected of child maltreatment. CPTA has also used a method called problem-based learning to provide online experiential learning as well. The UIS team is now expanding experiential learning to a wide array of different child protection and human service professionals.
The Children and Family Research Center has been evaluating the CPTA program since 2016 using an array of quantitative and qualitative methods. The simulation training evaluation team has produced an array of reports, research briefs, presentations and articles about the CPTA experiential learning program. CFRC’s evaluation research has played an important role in the development of experiential methods in child welfare in Illinois. As of this writing (August 2020), CFRC is also preparing to study experiential learning in other programs and sites.
Video: CFRC's Dr. Ted Cross Talks With Dr. Betsy Goulet About the Child Protection Training Academy's Simulation Training
Recorded on October 6th, 2020 via Zoom
FY2021 Program Evaluation of the Child Protection Training Academy for New DCFS Investigators
Ted Cross, Yu-Ling Chiu, Shufen Wang, Laura Lee, Steve Tran, and Kirsten Havig
In FY2021, the Children and Family Research Center’s (CFRC) evaluation team again used multiple sub-studies to examine the implementation and outcomes of simulation training for new child protection investigators in the Illinois Department of Children and Family Service. This is an important time historically to study simulation training because of the effect of COVID-19 on trainees, their work and the training itself. Chapter 1 summarizes CTPA’s implementation in 2021: adapting to the COVID-19 pandemic through virtual methods, training supervisors in problem-based learning, and re-formatting investigator training. Chapter 2 presents results from the Daily Experience of Simulation Training (DEST) measure. The measure was designed to examine change in trainees’ confidence over the course of simulation training. This is an important time to assess DEST results, because of changes in simulation training during FY2021, as discussed in the Introduction. Chapter 3 offers updated results from a post-training satisfaction survey. The chapter reports trainees’ satisfaction ratings for simulation training over this time period. It also provides qualitative results from the analysis of open-ended items in the post-training satisfaction survey. Chapter 4 examines whether simulation training is related to employee turnover. Using two different analytic methods, it asks whether investigators trained using simulation training have stayed in their jobs longer than investigators who were not provided simulation training. Chapter 5 examines the relationship of simulation training to child safety. We compared sim-trained and non-sim-trained investigators on the likelihood that children in their investigations were involved in re-reports to DCFS. The last chapter provides the conclusion of this year’s evaluation and recommendations for improving the program.
FY2020 Program Evaluation of the Child Protection Training Academy for New DCFS Investigators
Yu-Ling Chiu, Laura Lee, and Theodore P. Cross
The FY2020 evaluation report of the Child Protection Training Academy presents an array of data on implementation and impacts. Chapter 1 focuses on the implementation of a second simulation laboratory for new investigators that opened in Chicago in April 2019. The results suggest that the Chicago laboratory is a modest re-invention of the Springfield laboratory, using Rogers’ terminology on diffusion of innovations. The combined work of DCFS, UIUC and UIS aimed at producing programs that were comparable clearly had an effect, and the extra work and travel of UIS trainers to help make this happen are noteworthy. Chapter 2 presents results from the Daily Experience of Simulation Training (DEST) measure. As in previous years, the DEST in FY2020 shows that the confidence that trainees report increased substantially from the beginning to the end of the simulation training week. Chapter 3 offers new quantitative results from a post-training satisfaction survey. Results corroborate the satisfaction that trainees reported on the DEST. Across eight satisfaction items, trainees gave simulation training positive to very positive ratings. Chapter 4 presents a content analysis of responses on the post-training satisfaction survey in which trainees describe their simulation training experience and offer suggestions for program improvement. The survey respondents reported that the training provided realistic simulation and increased their knowledge about what they will face in the field. They described positive emotional effects on increasing confidence and decreasing self-doubt. Chapter 5 discusses the implications for understanding and developing the simulation training program and recommendations for program improvement.
How a Training Team Delivers Simulation Training of Child Protection Investigators
Yu-Ling Chiu and Theodore P. Cross
This article presents results of a qualitative study regarding how a training team delivers simulation training for child protection investigators. Since 2016, a team from the University of Illinois at Springfield (UIS) has collaborated with the Illinois Department of Children and Family Services (DCFS) to implement the Child Protection Training Academy (CPTA) that provides full-scale simulations has been implemented for training all new child protection investigators. Using key informant interviews and focus groups, we explored how the training team, including the simulation trainer, the standardized patients playing the role of the family in a mock family house, and the professionals playing roles in a mock courtroom, collaborate to shape the simulation training. The qualitative data point to the central role of the trainer’s blend of skills, the dedication of the standardized patients staying in character, the interest of role-playing professionals in correcting misconceptions about court, and the teamwork involved in implementing simulation training.
Chiu, Y. & Cross, T. P. (2020). How a training team delivers simulation training of child protection investigators. Children and Youth Services Review, 118. In press. https://doi.org/10.1016/j.childyouth.2020.105390
Moving from Procedure to Practice: A Statewide Child Protection Simulation Training Model
Betsy Goulet , Ted Cross , Yu-Ling Chiu, and Susan Evans
In FY 2015 the Illinois Department of Children and Family Services partnered with the University of Illinois Springfield to develop the Child Protection Training Academy in order to redesign the six-week classroom training for new investigators and create an experiential component. This paper chronicles the goals of the partnership and the planning and implementation of the Academy.
Betsy P. Goulet , Theodore P. Cross , Yu-Ling Chiu & Susan Evans (2020): Moving from procedure to practice: a statewide child protection simulation training model. Journal of Public Child Welfare, DOI: 10.1080/15548732.2020.1777247
FY2019 Program Evaluation of the Child Protection Training Academy for New DCFS Investigators
Yu-Ling Chiu and Theodore P. Cross
The FY2019 evaluation report of CPTA's simulation training included the following sections: 1) description of CPTA's updated training model; 2) daily experience of simulation training (DEST) that measured trainees’ daily changes in confidence on thirteen child protection work skills over the course of the simulation training week; 3) post-training satisfaction survey regarding trainees’ experience of the certificate training ; 4) simulation training and investigator turnover using DCFS employment data. The findings show that the positive results over three years of the program evaluation support the value of CPTA and suggest the potential of its current expansion. It is encouraging that investigators hired since February 2016 are staying on the job longer than investigators hired prior to that date. Data can be used both to advocate for the value of CPTA and to inform efforts at program improvement.
Using Simulation Training to Teach Child Protection Investigators in Illinois-Program Evaluation of the Child Protection Training Academy (CPTA)
Yu-Ling Chiu, Theodore P. Cross, Betsy P. Goulet, Susan Oppegard Evans, and Monico Whittington-Eskridge
The CFRC evaluation team collaborated with the CPTA at the University of Illinois Springfield and the DCFS Office of Learning & Professional Development on a presentation at the 2019 National Child Welfare Evaluation Summit on August 20th. The presentation included discussion of the CPTA simulation training model, the main evaluation findings between 2017 and 2019, and the simulation training expansion at the Chicago site.
Assessing an Innovative Method for Training Child Protection Investigators
Betsy Goulet, Theodore Cross, Yu-Ling Chiu
Presentation at the One Loud Voice Conference in Biloxi Mississippi, April 16, 2019
Assessing an Innovative Method for Training Child Protection Investigators
Yu-Ling Chiu, Theodore Cross, Betsy Goulet, Susan Oppegard Evans, Monico Whittington-Eskridge, and Amy Wheeler
The investigator survey results in the second year evaluation of the Child Protection Training Academy’s Simulation Program were presented on January 20th, 2019 at the Society for Social Work and Research (SSWR) Annual Conference in San Francisco.
FY2018 Program Evaluation of the Child Protection Training Academy for New DCFS Investigators
Ted Cross and Yu-Ling Chiu
The current report presents results from program evaluation activities conducted in FY2018. We used interviews and focus groups to explore the processes through which CPTA has an impact. In addition, we assessed the impact of CPTA on the experience of working DCFS investigators. We surveyed DCFS investigators and compared those investigators who have received simulation training with a group of investigators who had not received simulation training.
Feb 2018 / Research Brief / Simulation Training Evaluation
Assessing an Innovative Method for Training Child Protection Investigators: Program Evaluation of the Child Protection Training Academy’s Simulation Program
Yu-Ling Chiu and Ted Cross
Since 2016, the Child Protection Training Academy (CPTA) at the University of Illinois at Springfield (UIS) has collaborated with the Illinois Department of Children and Family Services (DCFS) to implement the Child Protection Training Academy, which adds an innovative experiential component to the training of new DCFS investigators. This brief reports results of a preliminary program evaluation that the Children and Family Research Center has conducted of the program. We describe what simulation training is, why it could be valuable, what simulation training has been implemented, and what the preliminary data on its implementation and impact suggests about its value for enhancing worker preparation to serve children and families.
Program Evaluation of Child Protection Training Academy for New DCFS Investigators: Initial Report
Ted Cross, Gail Tittle, and Yu-Ling Chiu
Investigating child abuse and neglect is a difficult job and investigators need all the preparation they can get. Given the demands of working with families in child protections, transferring theory to practice is particularly essential. Ideally, the initial training that new child protection workers receive should give them opportunities to practice the skills they need such as engaging families and assessment and critical thinking skills for protecting child safety. The Child Protection Training Academy at the University of Illinois at Springfield (UIS) has collaborated with the Illinois Department of Children and Family Services (DCFS) to add an innovative experiential component to the training of new DCFS investigators. All new investigators come to Child Protection Training Academy at UIS for a week at the end of their initial training to participate in simulations of real life situations that every DCFS investigator encounters. This program evaluation is consistent with the formative state of knowledge in the field and the fact that the Child Protection Training Academy is in a comparatively early stage of development. At this early stage, the program evaluation has focused on gathering data to describe the program’s objectives, methods and training theory, and examining trainees’ and other stakeholders perception of the impact of the simulation training. The goal is to inform program development and improvement, provide evidence of the program’s immediate impact on trainees, and help prepare for more rigorous program evaluation of the CPTA’s simulation training in the future.
Experiential Training for Child Protection Investigators: The Illinois Simulation Training Lab Experience
Ted Cross, Betsy Goulet, Susan Evans, Gail Tittle, and Yu-Ling Chiu
Given the demands of investigating child abuse and neglect, transferring knowledge gained in training into practice to bolster child protection investigators’ skills and confidence is essential. Yet studies of transfer of learning across different domains of employment have shown that only 10 to 15% of training content is transferred to the workplace. Simulation Training Laboratories at the University of Illinois at Springfield is helping to change that with an experiential training program it provides to all new child protection investigators hired by the Illinois Department of Children and Family Services (DCFS). New investigators are trained at a Residential Simulation Laboratory in a mock house designed to simulate a family environment and a Courtroom Simulation Laboratory designed to resemble family court. This Children and Family Research Center is conducting the program evaluation of simulation training. This presentation at an international conference gives a brief overview of the program and presents initial program evaluation results.
Experiential Training for Child Protection Investigators: The UIS Simulation Training Lab Experience
Cross, T., Goulet, B., Evans, S. & Tittle, G.
The Child Protection Training Academy at the University of Illinois at Springfield (UIS) is collaborating with the Illinois Department of Children and Family Services (DCFS) to add an innovative experiential component to the training of new DCFS investigators. The program supplements the classroom-based Foundations Training that every new investigator receives with experiential training days in a Residential Simulation Laboratory and a Courtroom Simulation Laboratory. The Children and Family Research at University of Illinois at Urbana-Champaign is conducting the program evaluation for simulation training. This presentation describes the program, discusses its first year of development, and presents preliminary program evaluation results.