The telesimulation curriculum
Part of the in-person SPs simulation program for medical undergraduate students at the University of Montreal was modified and adapted to meet the logistic and learning requirements for a new telesimulation curriculum. Unlike most of the experiences documented in the literature, which describe the use of mannequin-based telesimulation to train technical skills at a distance, the competencies of the telesimulation program reported in this paper focused on communication skills for which the SP method was adopted . The main objective of the telesimulation curriculum described here was to provide communication and clinical reasoning skills training for first, second, and third-year medical students during the COVID-19 pandemic.
Telesimulation activities were planned and implemented using a combination off-the-shelf telecommunication platform available at our institution such as Zoom™, Teams™, Moodle™, Qualtrics™, and ExamSoft™. The telesimulation curriculum included six communication training activities as well as two formative assessment activities and three certifying assessment activities (see Fig. 1); however, this paper focuses on describing the training activities. The adaptation of the in-person activities to the telesimulation curriculum consisted of a revision of the learning objectives, the scenarios, and the students’ preparatory activities.
The six communication training activities (see Fig. 1) were chosen out of twelve communication simulations by an academic committee chaired by the assistant dean for undergraduate studies and composed of curriculum officers for each year of the program, the directors of the two simulation centers of the two campuses that belong to the University of Montreal, and simulation and pedagogical experts. Although there were no explicit criteria to select the activities, the following elements played a major role in the decisions made by the committee:
Prioritization of activities covering topics that were not taught or discussed in other activities being adapted to an online format such as online problem-based learning (PBL) sessions and online lectures
Prioritization of activities that covered topics or taught skills needed for the objective structured clinical examinations (OSCEs)
Scenarios requiring a level of fidelity attainable with the telesimulation format used in the present study. For instances, scenarios addressing management of aggressive patients and nakedness were not considered due to the level of realism needed.
The number of resources (e.g., instructors, SPs) needed to plan and adapt the activities to the telesimulation format (see Table 1).
While the curriculum presented in this paper was first proposed as a solution to the training needs of the medical students at the University of Montreal during the pandemic, it is now put forward as an innovative way to partly address the training needs of growing student cohorts expected during the next 3 years in the province of Québec .
Process for planning telesimulation activities
The process of planning individual telesimulation activities started a month before the activity was scheduled. Figure 2 shows all the different steps and the timeline considered in the planning and the delivery of the telesimulation activities.
Assemble a team to plan and run the telesimulation activities
The first step was to assemble a team whose mandate was to plan and run the activities (see Fig. 2). This multidisciplinary team consisted of communication and technology experts, simulation coordinators and administrative managers, medical content experts, simulation instructors, pedagogical advisors, and simulation technicians who coordinated the Zoom™ operations. The number of team members needed for the telesimulation activities varied from nine to fourteen according to the complexity and of the technical needs of each activity (e.g., number of scenarios, number of students, and Zoom™ breakout rooms per session). Further information about the team members’ roles and tasks are described in Additional file 1.
Create a common communication channel and prepare for Internet disruptions in advance
A common communication channel was needed to ensure that all team members had the same information at all times but specifically during the planning and the delivery of the telesimulation activity (see Fig. 2). For this purpose, a Teams™ group was set up. This group also served as an alternative communication channel allowing team members to update last-minute changes due to connection issues .
Revise and adapt objectives, content, and the simulation scenarios
Similar to the general rules applied to the translation of face-to-face courses into online courses , the learning outcomes from the in-presence simulation activities to telesimulation did not differ; however, the primary differences and adaptations were made on how the outcomes were achieved. Therefore, the scenarios were reviewed and adapted to the reality of teleconsultations by medical content experts in charge of designing the curriculum for the simulation activities. They were asked to review and adapt the patient’s history and symptoms and add as much detailed information as possible about the pain intensity and the key nonverbal communication behaviors relevant to the case that could be easily conveyed via a video teleconsultation. In some cases, they had been asked that instead of having one long scenario, they develop two short scenarios to allow for rotation of student roles. As the content experts were not necessarily the ones delivering the telesimulations, they were asked to add more detail on the key points that needed to be debriefed. As explained later in this article (see “A test run to test the platform, to clarify roles, to standardize SPs performance, and to prepare for collective debriefings”), content experts also participated in the test-run sessions that were done before each telesimulation, and they supervised the standardization of the SPs.
Represent the Zoom™ virtual environment
The Zoom™ platform was used to create the virtual environment needed for the telesimulation activities. To represent the flow of the Zoom™ virtual environment, each activity was scripted in a flowchart-type diagram (see Fig. 3) that was designed in collaboration between the simulation instructors, the pedagogical advisors, and the Zoom™ operators. These diagrams made it possible to identify the Zoom™ tools necessary for the progression of the activity and the orientation of the students through a path leading them from the waiting room to the debriefing.
Although the flowchart was specific to each activity, they all included four common transitions, namely (a) the virtual waiting room, where the students were first received until all the participants arrived; (b) the main session, where students were briefed about the learning objectives, the different transitions of the activity, the confidentiality policies, the simulation instructions, the division of the groups, and the roles for each participant; (c) then, the flowchart showed the division of groups and the transition to the different scenarios including the timing for each scenario and the timing for debriefing in small groups; and after each scenario, (d) all participants were brought together to a main session where a group debriefing was done sharing the key messages discussed in each group.
A more detailed explanation of the Zoom™ settings needed to conduct telesimulation will be described in “Create and set up the Zoom™ meeting.”
Create a detailed schedule and assign roles to participants
An Excel™ list of all students from both campuses, instructors, and SPs was organized in alphabetical order. An example of the Excel worksheet is shown in Fig. 4. This list was used to create the students’ appointments specifying the day and times of the activity. Each group of participants, including students, instructors, and SPs, were given group appointments and were asked to sign in 15 min prior to the activity. A group of students between 30 and 40 were cited for each time slot. Telesimulations were run simultaneously for a minimum of four and a maximum of eight groups with a ratio of six to eight students, one SP, and one instructor per group.
Specific roles were assigned to learners beforehand by simulation instructors; however, students were only informed of their role during the introduction to the telesimulation activity. Students’ roles included active or primary physician, resident supervisor, and observer. The telesimulation activities often proposed two scenarios in which students had the opportunity to play different roles. Depending on the scenario, one or two students play an active role, and the others act as observers.
Verify resources and prepare for Internet disruptions in advance
The quality of telesimulation activities relies enormously on proper anticipation of technical difficulties [12, 35]. To overcome connection or Internet disruption during the telesimulation activities, a backup plan that considered a relay system of simulation instructors, SPs, and Zoom™ operators was designed for each activity. A minimum of two backup instructors, two backup SPs, and 1 backup Zoom™ operator were scheduled for all the telesimulation activities. Backup resources were trained to rapidly intervene in case that one member had connection difficulties. A detailed list of the backup resources needed per day was done in advance and continuously updated as needed (see Fig. 5). The backup system was tested several times before the telesimulations, and it proved useful while running the activities. During the telesimulation activity, when the backup system needed to be deployed, all team members received alerts and messages via the Teams™ channel to prepare accordingly.
Each telesimulation activity was repeated up to nineteen times depending on the number of students and the number of instructors and SPs available (see Table 1). When possible, the instructors and the SPs were the same for all the iterations.
Create and set up the Zoom™ meeting
Unfamiliar technological platforms and the difficulties associated with them can deviate participants’ attention, increasing the cognitive workload and affecting their capacity to complete a task . We addressed the issue of unfamiliar technology and the cognitive workload associated with it by having dedicated staff to manage the connection and the Zoom™ operations. Zoom™ operators (a main operator and a backup) were in charge of creating and setting up the learning environment, manage the waiting and the breakout rooms, and help participants to solve technical problems.
A set of pre-meeting features were activated a month before each activity by a Zoom™ operator using an educational account (umontreal.ZoomTM.us). Table 2 describes the functions that were activated for each telesimulation activity.
During the telesimulation activities, all participants, including instructors, SPs, and students, were told not to try to enter or exit the discussion or breakout rooms by themselves, and these features were blocked in advance by the Zoom™ operator. This restriction was intended to allow participants to focus on the activity instead of focusing on the logistics, thus minimizing the cognitive workload and the possibilities of error and/or disconnections. The following instructions were given in the Zoom™ connection guidelines (see more detailed information in “Create and share technical and content preparatory documents”):
You will never have to take any action to change rooms. Everything, including moving toward a breakout room, will be done by the ZoomTMoperators. Do not test the ZoomTMoptions during the scenario, just do activate or deactivate your microphone as needed while always keeping your camera on.
Accessing the Zoom™ meeting link
All participants, including the students, instructors, SPs, and Zoom™ operators, had access to the link through the Moodle™ institutional platform, which required a login access. Once participants accessed the meeting, the system recognized them and identified them with the username used to log in to the Moodle™ platform. This allowed us to ensure a secure and a disruption-free connection. In addition, it allowed us to identify participants reentering the meeting after being accidentally disconnected and send them to the right breakout room. The following instruction was given in the Zoom™ connection guidelines received by all the participants before the telesimulation (see more detailed information in “Create and share technical and content preparatory documents”):
If you were inadvertently disconnected from the meeting, reconnect using the link in the Moodle™ platform, and you will be redirected to the room where you were. You do not need to press any buttons to leave the meeting or room.
Generic usernames and passwords were created for the team members, the simulation instructors, and the SPs with the purpose of easily identifying them in the waiting room and giving them priority access to the meeting. After being identified in the waiting room, team members were transferred by the Zoom™ operators to the main room. Team members were asked to connect 20 min before the students.
Create and share technical and content preparatory documents
Technical preparatory material
All the participants, including the students, instructors, and actors, received Zoom™ connection tutorials tailored to each group and their specific needs. These guidelines listed the actions that participants needed to do before connecting, including downloading the Zoom™ application to minimize online interferences and accessing the Zoom™ link via the Moodle™ platform. The guidelines also included a checklist of the actions required once the Zoom™ meeting was opened but before the telesimulation started (see Table 3).
Content preparatory material
To prepare students for the telesimulation activities, we implemented the flipping classroom technique in which students received preparatory material 2 weeks in advance in the form of readings and videos made by the instructors. Videos summarized theory and modelled expert clinicians’ problem-solving and decision-making strategies. They were uploaded to a private YouTube account, and analytics (e.g., most and least watched segments, number and duration of views) was used to monitor how students used the videos to prepare themselves. Although the inclusion of preparatory material was commonly used for our in-person simulations, the use of videos and the analytics were new additions to our pedagogical formula.
Prepare and adapt formative assessment strategies
Formative assessment before and after the telesimulation
After revising the preparatory material, students were asked to complete a questionnaire intended to assess their level of understanding and integration of the concepts behind the telesimulation. The ExamSoft™ platform was used to disseminate these assessments. An evaluation with questions regarding the notions practised during the simulation was made following the telesimulation.
Before each activity, participants (students, SPs, and instructors) were asked to agree to the terms of a telesimulation agreement, which stipulated that active participation and respectful orientation during the activity were mandatory . The Qualtrics XM™ platform was used to distribute and collect the information from this agreement. The details of the telesimulation agreement are described in the Table 4.
A test run to test the platform, to clarify roles, to standardize SPs performance, and to prepare for collective debriefings
A test run was scheduled for each telesimulation 1 or 2 weeks before the activity. It included all participants, except the learners. This practice had different purposes. First, it allowed the team to align the different pieces of information each person held. Second, it served as a rehearsal of the activity with team members playing the role of students. Third, it provided a cross-training practice of the key tasks done by operators and instructors, which allowed a better comprehension of the logistics and facilitated the planning of alternative solutions in case something went wrong. During the test-run sessions, the instructors had the opportunity to interact with the platform as if they were students and revise the flowchart sequence of the telesimulation activity. A discussion about the objectives of the teledebriefing sessions was also included in the test run.
The SPs also participated in a separate test run, which also included the content experts and the SPs coordinator. Its aim was to standardize the nonverbal expressions portrayed in the scenario and some technical aspects such as light, microphone volume, and the video background, which for the most parts was neutral with a white or black wall behind.
During the telesimulation activity
All activities began with a presimulation briefing. Instructors first reminded the students of the technical instructions to facilitate communication and technical logistics involved in each activity. Participants were asked to rename themselves (name and last name) avoiding the use of short names. While instructors briefed the students, the Zoom operator manually assigned the students, the instructors, and the SPs to the different breakout rooms. This task was accomplished using the predetermined schedule of the activity as support. To facilitate this task, participants were asked to write in parentheses at the end of their names the role they had in the telesimulation activity including student, actor, actor backup, instructor, instructor backup, Zoom operator, and Zoom operator backup. Although pre-assignation of the breakout rooms is possible through the Zoom™ web portal, this functionality does not allow rotation of the participants into different breakout rooms, which was needed in our case in order to expose students to different scenarios.
During the briefing, students were also presented with an Excel table where they identified their assigned group and their role during the telesimulation. If someone got disconnected from the meeting, the Zoom operator was able to easily reconnect the participant to the right breakout room when looking at the predetermined schedule of the activity. Once the presimulation briefing was done, the simulation instructors informed the Zoom operator, via the Teams™ chat, that they were ready to initiate the breakout rooms. In the breakout rooms, the instructors further clarified what was expected of each role and briefly presented the general information about the patient (e.g., age, context, and reason for consultation).
Before the students initiated the communication with the SP, students were asked to brainstorm about how they expected the scenario to evolve and which questions to ask to address the doctor-patient encounter. The student playing an active role was informed that his/her colleagues were allowed to hint him/her with some questions to facilitate the encounter via the chat box. After each scenario, instructors did a first debriefing in small groups allowing all students to contribute to the discussion from their different roles. Some of the strategies that the simulation instructors used to facilitate discussion and to help learners to overcome perceptions of relational distance are described in Table 5.
Following the teledebriefing in small groups, the instructors convened for a common plenary session in which the key messages from each breakout room were shared. One student from each breakout room was chosen to convey the key messages.
As connection disruption could alter the timing of the telesimulation activities, the Zoom™ operators also played the role of timekeepers during the activity. In this role, they were responsible for adjusting the schedule of the telesimulation activity in the event of technical or other difficulties delaying operations. When this happened, Zoom™ operators announced the delays and the strategies to overcome them to all team members through the Teams channel.
After the telesimulation
Following each telesimulation activity, team members involved in the telesimulation planning immediately conducted a debriefing session to address the technical and pedagogical aspects that needed improvement. Students, simulation instructors, and SPs were also asked to complete an anonymous survey about the perceived usefulness of the telesimulation activity. The survey consisted of 5-point Likert scale statements (1 = completely disagree, 5 = completely agree) (see Additional file 2). The survey was divided into four categories including preparation and organization of the telesimulation (3 questions), familiarity with the technology used during the telesimulation (10 questions), perceived differences between telesimulation and in-person activities (10 questions), and a global assessment of the experience (5 questions). Data from the usability survey is being currently analyzed.
The telesimulation curriculum presented in this article has been successfully applied to learners (n = 2989) with minimal disruption and/or technical difficulties; however, it is important to highlight the fact that telesimulation in the format presented in this paper is time-consuming and more expensive than conducting in-person simulations. Other studies and reports on telesimulation have underlined its feasibility and proposed it as a low-cost alternative to provide training during the pandemic ; however, our experience suggests that telesimulation could become quite expensive with large student cohorts (more than 250 learners). The requirements to ensure active and engaged participation of the learners during telesimulation, such as small 6–8 student groups (compared to our typical 15–20 student groups during our in-person simulation), as well as the need for personal backup able to react quickly in case of technical or connection difficulties, all contribute to the necessity of securing appropriate resources. In addition, telesimulation activities, in comparison with in-person simulations, require additional standardization processes (test runs, standardization of SPs) that increase the preparation time needed for simulations.
Lessons learned and recommendations
The process of deploying telesimulation for big cohorts with minimal disruptions at the University of Montreal required a well crafted and a structured plan that was revised and adapted several times based on strengths and weaknesses. Table 6 shows the most recurrent problems and its possible solutions.
Based on the expertise and experience acquired by our team in the design and implementation of the telesimulation curriculum for large student cohorts, we propose some recommendations for teams wanting to reproduce our model (see Table 7).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.