Stage 1: development of the SE-C19 questions

In early 2020, there was a limited understanding of the symptoms experienced by outpatients with COVID-19. The SE-C19 questions were developed based on the CDC symptom list [14] and published literature available at the time [15]. Item generation and evaluation of content validity of the SE-C19 were informed by the good measurement principles outlined in the FDA PRO guidance [12] and the Patient-Focused Drug Development guidance [16].

Symptoms identified in the literature and on the CDC website were used to develop 23 questions concerning the following: feeling feverish, chills, sore throat, cough, shortness of breath or difficulty breathing, nausea, vomiting, diarrhea, headache, red or watery eyes, body and muscle aches, loss of taste or smell, fatigue, loss of appetite, confusion, dizziness, pressure or tight chest, chest pain, stomachache, rash, sneezing, sputum/phlegm, and runny nose. Patients were asked to select which of the symptoms they had experienced in the past 24 h in an electronic diary, then rate the severity of each selected symptom at its worst in that 24-h period on a severity scale (mild, moderate, or severe). To aid interpretation of the SE-C19, additional questions were developed to assess patients’ global impressions of overall symptom severity and return to usual health/daily activities, in line with FDA recommendations [6]. The electronic app could be accessed using a smartphone, tablet, or computer.

Stage 2: content validation of the SE-C19

A cross-sectional, applied qualitative research study comprising in-depth, open-ended interviews in patients with a confirmed diagnosis of COVID-19 was conducted. Clinicians provided insights on the clinical experience and evolving symptomatic profile of COVID-19. Interviews were conducted in September and October 2020. This study is reported in accordance with the “Consolidated Criteria for Reporting Qualitative Research” guidelines [17].


We purposively recruited 30 adult (aged ≥ 18 years) outpatients through a healthcare research firm. All patients were recruited in the US and had a confirmed diagnosis of COVID-19, through a positive polymerase chain reaction test, up to 21 days before the interview. Patients self-reported fever (≥ 37.8 °C or feeling feverish or chills), cough, shortness of breath/difficulty breathing, change or loss of taste or smell, vomiting or diarrhea, or body/muscle aches. Patients who had been hospitalized within the past 30 days, were not fluent in English, or who were residing in nursing homes were excluded from the sample. Consistent with recommendations for sample sizes in qualitative healthcare research [18], a target of 30 patients was considered to be sufficient for achieving conceptual saturation, that is, the point at which no new information emerges from additional interviews [19, 20]. Five independent clinicians who regularly treated patients with COVID-19 were also interviewed. These clinicians treated at least five patients per week, both in person and remotely, and were recruited through the same healthcare research firm and Regeneron Pharmaceuticals, Inc.

Interview conduct

Both patient and clinician interviews (~ 60 min) were conducted virtually by three experienced qualitative researchers (NM, KP, AR) with backgrounds in health psychology and clinical research who received study-specific training in conducting the interviews using open-ended questions and the “think aloud” process to meet the study objectives as described further below. The interviewers were unknown to the patients prior to the conduct of the interview and were not involved in any aspect of the patients’ healthcare. The semi-structured patient interview guide was divided into (1) concept elicitation and (2) cognitive debriefing.

The concept elicitation comprised open-ended questions about patients’ COVID-19 symptoms (“Can you tell me about the symptoms you have experienced with COVID-19?”). Interviewers also used structured prompts for all common symptoms of COVID-19 [14] not spontaneously mentioned. The focus of this exercise was to spontaneously elicit proximal and distal concepts associated with the COVID-19 experience, prior to cognitive debriefing of the SE-C19. This was partly in aim to assess conceptual coverage of the SE-C19, see (submitted for publication) for full account of methods and results.

During cognitive debriefing, patients used a “think aloud” method to respond to each of the questions in the SE-C19. Patients were asked to read each question aloud and verbalize their thought process while responding. When necessary, interviewers asked open-ended questions to elicit additional information about the patients’ experience responding to each question (“Tell me what you are thinking about…” or “What are you considering when you respond to this question?”) [21,22,23]. Once patients were given an opportunity to spontaneously provide feedback on the SE-C19, structured probes were used to elicit patients’ understanding of the instructions, clarity, and appropriateness of the instrument. Each question was reviewed to understand its relevance, ease of selecting a response, and whether the 24-h recall period was appropriate. The objective was to ensure the intended meaning of each question of the SE-C19 was consistent with the patient’s interpretation. Patients were also asked about missing symptoms or if any questions should be removed.

Clinicians were presented with a paper copy of the SE-C19 and were asked to provide feedback on its questions (specifically any overlapping symptoms or groups of symptoms), as well as the relevance and appropriateness of the instructions, response options, and recall period.

Data analysis

Interviews were audio-recorded, transcribed, and anonymized. Transcripts were coded by NM, KP, AR in ATLAS.ti software [24] using an open, inductive coding approach [24,25,26,27]. Investigator triangulation [28] was used during analysis; the first transcript was independently coded by three researchers, and minor inconsistencies in codes were discussed and reconciled. Researchers met regularly to discuss and adjust coding guidelines, when necessary (for example, to ensure the codes were capturing the granularity reported by patients). The adequacy of the sample size was assessed by saturation analysis [19, 20] performed on quintiles of transcripts.

The conceptual coverage and comprehensiveness of the SE-C19 was evaluated by mapping its questions to the symptoms elicited from the patient experience. Cognitive debriefing analysis consisted of categorizing issues spontaneously reported by patients or found by probes. Structured, multi-level coding was used to report patient feedback on each question, corresponding response, instructions for completion, issues of clarity/interpretation, and any suggested changes.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit


This article is autogenerated using RSS feeds and has not been created or edited by OA JF.

Click here for Source link (