Search results

A total of 3851 studies were retrieved from our search, after duplicates were removed. Following review of titles and abstracts by two independent reviewers, a total of 76 studies were selected for full text review (Cohen’s kappa = 0.84). Following independent full text review by two reviewers, a total of 37 studies were selected. These were reviewed by our third reviewer after which 36 articles were included. Reasons for exclusion after full text review included the lack of any video-based interview being implemented (n = 19), letters to the editor (n = 7), did not involve applicants in healthcare fields (n = 3), did not have any measured outcomes (n = 8), or abstracts which full text was already included (n = 3). The search was updated on February 20, 2021, and 5 additional studies met inclusion criteria. Two additional studies were retrieved from review of the references of the included studies. As such, a total of 43 studies were included in our scoping review [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. A PRISMA flow diagram is provided in Fig. 1 to outline the study selection process [62]. A summary of the demographic data and eligibility criteria of the included studies is provided in Table 1. Our outcomes, which were group thematically, are summarized separately in Table 2.

Fig. 1
figure 1

PRISMA
2020 flow diagram for new systematic reviews which included searches of
databases and registers only. From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I,
Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline
for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71. For more information,
visit: http://www.prisma-statement.org/

Table 1 Characteristics of included studies
Table 2 Study outcomes grouped thematically

Study characteristics

Of the included studies, 17 were conference abstracts and 26 were published manuscripts. Forty-two studies were conducted in the USA, while 1 was conducted in Australia.

Twenty-nine studies included medical students applying to residency, 2 studies of students applying to medical/dental school, 2 studies of pharmacy students applying to their respective pharmacy residency, and 10 studies of residents applying to fellowship. Of studies involving medical residency or fellowship applications, disciplines included urology (n = 1), anesthesiology (n = 1), general surgery (n = 1), pediatric surgery (n = 1), gastroenterology (n = 1), internal medicine (n = 1), orthopedic surgery (n = 2), obstetrics and gynecology (n = 2), family medicine (n = 2), complex general surgical oncology (n = 3), female pelvic medicine and reconstructive surgery (n = 1), maternal fetal medicine (n = 1), advanced gastrointestinal minimally invasive surgery (n = 1), and emergency medicine (n = 22).

There was a range of study methodology. There were 36 cross-sectional studies, 4 cohort studies, 2 quasi-experimental studies, and 1 randomized trial. All studies employed some form of quantitative analysis, while one abstract did not outline what analysis they performed. No studies used qualitative methods.

Virtual panel interview

Eighteen studies reported on their use of panel style virtual interviews. Of these, 6 were adjuncts [19,20,21,22,23,24] to the face-to-face interview while 12 were replacements [25,26,27,28,29,30,31,32,33,34,35,36]. Studies used Skype [19, 21, 24,25,26,27,28, 30], FaceTime [19], Zoom [31, 32, 34,35,36], or a combination of two platforms [22, 23, 29]. Eleven studies provided additional information outside the virtual interview itself including a video tour of the facility, video tour of the surrounding communities, Google hangout session with current residents, electronic brochures, and resident contact information to ask additional questions if interested [19, 26, 28,29,30,31,32,33,34,35,36]. The number of interviews varied from a single 15–30 min video-based interview [20, 22,23,24,25,26,27, 30, 32, 33] to multiple interviews with different faculty members [19, 21, 28, 29, 31, 34,35,36]. The number of interviewers also varied from one-on-one interviews [19, 20, 24, 27, 28, 34, 35] to those with up to five interviewers [22, 23, 26, 30,31,32,33]. Three studies described having a dedicated individual (e.g., program coordinator, information technology specialist) who facilitated movement of applicants and interviewers between breakout rooms on the video-based platform [31, 34, 35]. Three studies had an administrative staff responsible for ensuring that applicants had established appropriate audio/visual connections [19, 27, 28].

Nine studies reported favorable perceptions from applicants on video-based panel interviews [20, 21, 24, 26, 31, 32, 34,35,36]. Applicants in these studies felt that they were able to demonstrate their strengths and personality through a video-based format [26, 32, 35, 36], that it was a fair way to present the program to them [21], and that the interviews flowed well [31]. Most applicants found that the video-based interviews met their expectations [20, 26], and they were satisfied with the process [34].

In one study, applicants felt that the video-based interview was less effective in allowing them to represent themselves [28]. An additional study highlighted that video-based interviewing prevented the “gut feelings” about a program that are typically felt at the time of a face-to-face interview [33]. Most applicants in one study also found that the inability to see the city or meet the faculty face-to-face was a drawback to video-based interviews [34]. Other concerns with video-based interviewing included not being able to understand the program’s culture [35] and interact with current residents [29].

Few studies explored the interviewers’ perspective on video-based interviewing and found that interviewers were mostly satisfied with the video-based interview process overall [26, 31, 34]. In one study however, none of the interviewers recommended using video-based interviews as the sole method of interviewing [21].

In the 12 studies that specifically assessed the role of video-based interviewing as a replacement to the current process, two found that it worked well to replace face-to-face interview [31, 34], while 10 studies felt that it was a useful adjunct or screening tool, but was not ready to replace face-to-face interviews entirely [19,20,21,22, 26, 28, 32, 33, 35, 36].

Overall, the panel-based video interview format appeared to be acceptable to applicants and interviewers, although a complete transition away from face-to-face interviews has been met with hesitancy.

Virtual multiple mini-interviews

One study reported on their use of video-based multiple mini-interviews (MMIs) [37]. With their retrospective cohort study comparing applicants to medical and dental school between years that employed face-to-face MMIs with those that employed video-based MMIs, they found that there was no significant difference between interview scores, although there was greater variability in scores in the video-based MMI group.

A total of 76% of applicants and 78% of interviewers were satisfied with the video-based MMI process. Similar to the face-to-face counterpart, the video-based MMIs involved seven questions with 2-min change over time between stations. The interview process was set up and overseen by five administrative staff and two IT staff. This study was the only study that explored the role of a video-based MMI as a replacement to the face-to-face counterpart. This one study demonstrated that the video-based MMI was an acceptable alternative to an in-person process.

One-way video interviewing

One-way video interviewing describes the process by which applicants submit answers to standardized questions in video format to be evaluated as part of the selection process. There were 24 studies that employed this method of video-based interviewing.

Twenty-two of these studies evaluated the standardized video interview (SVI), an online unidirectional interview that was developed by the Association of American Medical Colleges (AAMC) and piloted with the Accreditation Council for Graduate Medical Education (ACGME)-accredited emergency medicine programs. Twelve of these studies were abstracts [38,39,40,41,42,43,44,45,46,47,48,49] and 10 were articles [50,51,52,53,54,55,56,57,58,59]. Applicants submitted an audio/video response to six questions which was subsequently scored from 6 to 30. The goal of the SVI was to provide standardized information about applicant’s interpersonal and communication skills and professionalism. It was introduced in 2016 as a research project and was administered as an operational pilot in the emergency medicine residency selection during the 2018, 2019, and 2020 match cycles. Although the SVI was not continued for the 2021 match cycle, these studies were included in our scoping review to assess the value of unidirectional interviewing.

Overall, there were small correlations between SVI scores and other aspects of the application such as USMLE scores [41], faculty scores of communication and professionalism [51], electronic standardized letter of evaluation (eLOE) [55], patient evaluation of communication skills [46], and traditional interview scores [47, 48]. One study questioned the utility of the SVI as there was a lack of a relationship between SVI scores and applicant ranking [40], while another study found that the SVI score changed the likelihood of a program to invite the applicant for an interview in 7% of cases [56]. Of studies that explored the applicant perspective, two found that applicants did not feel that the SVI should be part of the application process [39, 51].

Apart from the studies that assessed the AAMC’s SVI specifically, two other studies described their use of one-way video interviewing or asynchronous video interviews [60, 61]. Both these studies utilized one-way interviewing as an adjunct to face-to-face interviews, rather than a replacement. Applicants submitted a video response to three standardized questions that were scored to determine which applicants would be subsequently invited for a face-to-face interview.

Higher scores on this one-way interview were correlated with higher in-person interview scores [60, 61]. While one study reported a positive correlation between one-way interview score and rank list placement [60], the other study reported a nonsignificant positive correlation between the two [61].

Overall, results were ambiguous as to what the SVI was measuring and the value that should be attributed to it amongst other aspects of the emergency medicine selection process. Nonetheless, one-way interviews may hold some promise as an initial screen, but there is no evidence we found to demonstrate that it should replace a bidirectional interview.

Technical limitations

Eleven of the included studies discussed technical limitations of video-based interviewing [19,20,21,22,23, 28,29,30, 34, 36, 37]. Three studies discussed issues with connectivity that were resolved by reconnecting or switching to telephone interviews without video [22, 23, 28]. One study discussed the voice delay associated with using the Skype platform for video-based interviews [21]. One study explained that while faculty was nervous about potential technical issues with video-based interviews, only one interviewer had a temporary technical connectivity problem [36].

No studies reported any major concerns from a technical point of view that limited the use of video-based interviewing.

Financial cost

Fourteen studies discussed the financial costs associated with the selection proces s[19,20,21,22,23,24, 28, 29, 31, 33, 34, 36, 37, 59]. Thirteen of these studies reported a reduction in financial costs for either applicants [19, 20, 22, 33, 34, 36], programs/interviewers [23, 24, 29], or both [21, 28, 31, 37]. One study reported that applicants who matched successfully spent significantly more money compared to those that did not match [19]. In another study where applicants were given the option of interview format, 25% of applicants chose a video-based interview due to financial limitations [29]. Additionally, the importance of modifying the surgical fellowship recruitment process given increases in student debt had been alluded to by one study [36].

Overall, video-based interviewing was seen as a way to improve financial costs for all stakeholders, particularly for applicants in whom financial limitations impact their application process.

Opportunity costs

Fifteen studies discussed the opportunity costs associated with the selection process [19,20,21,22,23, 26, 27, 29, 31, 33, 34, 36, 37, 58, 61]. Eleven of these studies reported that video-based interviewing afforded the applicants the ability to expend less time with the interview process [19,20,21,22,23, 26, 28, 33, 34, 36, 37], which would mean less time being taken away from clinical or educational commitments [20, 21, 28, 34].

Two studies discussed that video-based interviewing was particularly beneficial for applicants that could not get time off work to attend face-to-face interviews [29] and for applicants who would not be able to attend the face-to-face interview due to interview scheduling conflicts [20]. One study also highlighted that residency programs themselves would experience fewer disruptions with video-based interviewing since residents could take less time away from clinical duties [36].

One study explained that the introduction of the SVI may have increased the time required for applicants to prepare for the interview itself [28]. From a program’s perspective, one study reported that the utilization of one-way video interviewing prior to face-to-face interviews delayed the face-to-face interviews by 3 weeks and increased the burden of work from the program [61].

While one-way video interviewing may come at the cost of increased work for both applicants and interviewers, bidirectional video-based interviewing allows applicants to take less time away from personal or professional commitments and results in fewer disruptions for the programs themselves.

Environmental cost

None of the 43 studies discussed the environmental impact of interviewing in healthcare as it applies to personnel selection.

Body language

Three studies discussed the role of body language as it pertained to video-based interviewing [21, 23, 33]. One study commented on the lack of physical contact such as a hand shake but explained that this was more of a concern for interviewers [21]. Another study discussed how the loss of video connection in some interviews prevented the assessment of body language which was part of the evaluation [23]. Finally, one study commented on the lack of subjective details from an interaction that are lost in a video-based format [33].

None of the studies discussed the interpretation of body language and how this plays into applicant selection.

Influence on rank list

There were 12 studies that assessed either the perceived or objective impact of video-based interviews on the rank list [19, 26,27,28,29,30, 35, 45, 56, 53, 60, 61]. Two of these studies reported a positive relationship between video-based interview scores and overall rank list [60, 61]. Three studies found that there were no differences in acceptance rate and/or rank list position based on whether applicants had a face-to-face or video-based interview [27,28,29].

Three studies reported from the applicants’ perspective that most applicants were comfortable ranking programs after a video-based interview [26, 30]. One study reported that only 54% of applicants who had a virtual interview felt that the experience was sufficient to make a ranking decision, compared to 92% of their counterparts who had a face-to-face interview experience [35].

From the interviewer perspective of determining the rank list, three studies reported that there was no significant difference of interview type on the ranking of applicants [28, 29].

In terms of the SVI, one study reported the lack of any significant correlation between the SVI scores and rank list position [45], while another study reported that the SVI score changed the likelihood of inviting an applicant for a face-to-face interview in 7% of cases, with lower SVI scores more likely to decrease the chance of an interview invite than higher scores were to increase the chance of an interview invite [56].

Quality assessment

Quality assessment was completed for the 26 peer-reviewed manuscripts included in our scoping review. Using the JBI critical appraisal tools to evaluate each article, 11 studies were felt to have a high risk of bias, 12 with a moderate risk of bias, and only 3 studies with a low risk of bias. Most studies lacked a valid and reliable tool to evaluate applicant/interviewer perspectives on video-based interviews, did not assess or control for potential confounders, or had poor survey response rates. A summary of our quality assessment is provided in Additional file 3.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Disclaimer:

This article is autogenerated using RSS feeds and has not been created or edited by OA JF.

Click here for Source link (https://www.biomedcentral.com/)