A total of 40 individuals participated in this study. Most were white (n = 26, 65.0%), female (n = 23, 57.5%), had completed some form of post-high school education (n = 29, 72.5%), and had a chronic health condition (n = 29, 72.5%). Half of the sample was between the ages of 18–49, and the other half was age 50 or older. All participants were in the US including the Northeast (n = 12, 30.0%), West (n = 7, 17.5%), Midwest (n = 5, 12.5%), and South (n = 16, 40.0%). All participants were asked to rate their overall health; of those, fourteen (35.0%) rated their overall health as ‘very good’ or ‘excellent.’ Health satisfaction ratings were also collected and were wide-ranging across a 10-point scale, with an average of 5.8 out of 10 (min = 1, max = 9). (See Table 2).
Form A cognitive debriefing results
All participants who debriefed Form A (n = 20), found it relevant, straightforward, and easy to understand. Participants were able to easily relate the questions to aspects of their daily lives and select an answer accordingly (see Table 3 for additional data).
Instructions and recall
All participants found the instructions for Form A clear and easy to understand. One participant initially missed the instructions but was able to complete the survey with no issues.
Fourteen participants found it easy to recall how they were feeling over the past 4 weeks and to answer each question within that timeframe. Of the 6 who did not, 4 recommended shortening the recall period to 2 weeks and 2 participants felt it was difficult to recall the past 4 weeks due to monotony of the previous months (related to the COVID-19 pandemic) but did not provide an alternative recall period.
Overall, participants found the physical functioning question easy to answer (n = 17). Of those who found it difficult (n = 3), 1 participant felt it was unclear whether the response choices were mutually exclusive (i.e., if they are limited a little in moderate activities, does that mean they cannot do vigorous activities?); another did not engage in vigorous activities and could not answer whether they were limited; and another was unsure how to answer the question because their multiple chronic health conditions limited them in different ways.
Most participants found the role functioning question easy to answer (n = 16). Of the 4 participants who had difficulty answering the question: 1 struggled with recalling regular daily activities over the past 4 weeks, another felt the question was too wordy and suggested changing the wording to “felt or were less productive,” 1 felt their answer would differ depending on whether they focused on work or activities outside of work, and another suggested splitting the question into 2 separate items (1 for physical health and 1 for emotional problems). However, upon further questioning, all participants were able to understand and interpret the question accurately.
Just over half of participants found this question easy to answer (n = 13). The other 7 found it difficult for a variety of reasons. Three struggled to recall their pain over the past 4 weeks—with 2 noting their pain fluctuated requiring them to come up with an average pain level so they could answer the question. While able select a response for this item, 2 participants found it difficult to do so quickly, as they felt the question and response choices were too subjective (i.e., definitions of pain will be different and so answers cannot be accurately compared). Two participants were unsure whether the question was asking about acute or chronic pain and felt their answers would differ depending on the type of pain.
Most participants (n = 19) found the vitality question easy to answer, although 4 took a longer time to select an answer as compared to previous items. The 1 participant who had difficulty answering struggled with recalling times when they felt worn out over the past 4 weeks. Additionally, 2 participants felt the phrase “worn out” was too vague and should specify whether it includes emotional problems or just physical health, however upon further probing each person considered both physical health and emotional problems when answering the question.
Fifteen participants found the social functioning question easy to answer. The 5 participants who found it difficult to answer referred to the COVID-19 social distancing restrictions in place at the time of the interviews. Because social activities were restricted due to local ordinances, these participants experienced interference with social activities in the 4 weeks prior to the interviews. Although the interference was not due to physical health or emotional problems, it made it difficult for them to answer this item, nonetheless.
Overall, participants found the mental health question easy to answer (n = 18). Of those who found it difficult (n = 2), 1 participant felt it was hard to admit, and be vulnerable enough to answer the question, while the other felt the current state of the world (e.g., ongoing COVID-19 pandemic) made it difficult to answer the question.
Form B cognitive debriefing results
Overall, most participants who debriefed Form B (n = 18) were able to easily relate questions to aspects of their daily lives and answer accordingly, and thought it was straightforward and easy to understand, with only 2 participants finding the form confusing or difficult to answer. Of these 2 participants, 1 struggled with whether to consider their health pre-COVID-19, or if they should answer in the present day, while the other was unsure what the survey was asking overall and therefore had a difficult time selecting statements that described them (see Table 3 for additional data).
Instructions and recall
All participants found the instructions for Form B clear and easy to understand. Most participants (n = 18) found it easy to recall how they were feeling over the past 4 weeks and had no difficulty answering each question within that timeframe. Of the 2 who did report issues, 1 recommended shortening the recall period to 2 weeks, while the other suggested it would be easier to remember the past 1–2 weeks, rather than the past 4.
Overall, the physical functioning question was found to be clear and easy to answer (n = 17). Three participants (out of 20) found it difficult to answer, primarily due to general confusion over which statement best described their health and the circumstances limiting their physical functioning.
Role functioning was perceived as easy to answer (n = 17). Participants interpreted “ability to work and do regular daily activities” to mean their general responsibilities as an employee, parent, or member of society, including going to work and completing household chores. Participants who found this item difficult to answer (n = 3) found the double negative statement to be confusing (i.e., you accomplished less than you would like none of the time; n = 1) and had different answers for physical and emotional health and would have preferred to answer each separately (n = 2).
Similar to Form A, just over half of the participants (n = 11) found this item easy to answer. The 9 participants who did not reported this item was difficult to interpret and found it challenging to distinguish between the response choices mild and moderate (given the response choice of very mild), and severe and very severe. Participants also had difficulty averaging their pain over 4 weeks given daily fluctuations. One participant was unsure if the item is referring to chronic or acute pain, which made selecting a statement to describe their pain difficult.
Overall, the item on vitality was easy for participants to answer (n = 17), although 3 reported finding it difficult to select a statement to describe themselves. These participants were confused over what “worn out” was referring to (e.g., does being tired at the end of a busy day qualify?). Participants also questioned the meaning of the heading (“Vitality”) and whether the concept is easily recognizable; ultimately, it was interpreted to mean being worn out mentally, worn out physically, or both.
Similar to the results for Form A, participants found the social functioning item in Form B easy to complete (n = 15), however the ongoing COVID-19 pandemic added difficulty for some individuals (n = 5). Participants who indicated this was difficult to answer noted that all social activities were limited, regardless of their health, making it challenging to decide which statement to select. Participants who found this item easy to complete also brought up COVID-19-related social restrictions, however it did not impede their ability to select a statement, or their understanding of the item.
While 12 participants had no difficulty with the mental health item in Form B, 8 participants found it difficult to select a statement to describe their mental health. Four described their feelings of depression or anxiety as variable and found it difficult to select one statement to best describe them over the past 4 weeks. Some participants (n = 3) also had difficulty selecting a statement if they experienced only depression or only anxiety. The double-barreled nature of the item wording made it difficult to choose the most appropriate statement. Similarly, 2 participants found the word “depressed” to be triggering, articulating there is a difference between being depressed and feeling depressed and it isn’t clear which the item is referring to. Finally, 1 participant found this item difficult to answer in an interview setting with a stranger, while another wasn’t sure how to answer given how the COVID-19 pandemic has influenced all aspects of life.
Comparison of Form A and Form B
There was a general tendency to prefer the last form the respondents had seen. Of the 20 participants who debriefed Form A, 12 preferred Form B after reviewing Form B, while only 7 preferred Form A. One participant had no preference. Of the 20 who debriefed Form B, only 2 preferred Form B after reviewing Form A, while 18 preferred Form A. Taking this recency effect into account, more participants preferred Form A above Form B (See Fig. 1 and Table 4).
Overall, participants found the items in Form A clearer and easier to answer. When comparing Forms A and B, more participants preferred answering questions (Form A) over choosing from a set of statements (Form B). The numbered questions and underlining of key words in Form A fostered quick and easy comprehension and completion of the survey. Participants also felt it looked more professional and was more in line with what they were used to seeing. While participants found some of the titles in Form B to be helpful, overall, they preferred the questions in Form A. Overall preference mostly aligned with participant preferences for individual items within the two forms. Individuals whose overall preference was Form A, also tended to prefer the question/answer items in Form A over the corresponding statement items in Form B and vice-versa. However, this was not always the case.
Despite an overall preference for Form A, almost half of participants (n = 19) preferred the physical functioning question in Form B, finding it clearer and easier to answer. They found it helpful to have the descriptions of vigorous and moderate activity in the response choices (n = 11), and they found the wording easier to understand (n = 9). Eight participants found the response choices in Form A to be challenging when comparing them to Form B.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.