Validation of the English version of the arrhythmia-specific questionnaire in tachycardia and arrhythmia (ASTA): a Rasch evaluation study – Journal of Patient-Reported Outcomes

Aim

The aim of this study was to use Rasch measurement theory to evaluate the English version of the ASTA questionnaire.

Description of the stages in the study process

This psychometric evaluation study was conducted in two stages: I) the earlier performed translation process of the ASTA questionnaire and II) the evaluation of the measurement properties.

Step I: the translation process of the ASTA questionnaire

The arrhythmia-specific questionnaire in tachycardia and arrhythmia (ASTA) questionnaire design

The ASTA questionnaire is aimed to evaluate symptoms and HRQoL in patients with different forms of arrhythmias such as AF, atrial flutter (AFL), Wolff-Parkinson-white syndrome, AV-nodal reentrant tachycardia and those with ventricular arrhythmia and premature ventricular extra beats, and is divided into three parts. Part I evaluates the last episode of arrhythmia, current medication, and the presence of arrhythmia at the time of follow up. Part II measure arrhythmia-specific symptoms, the ASTA nine-item symptom scale. The response format is a four-point Likert type scale; “No” (0); “Yes, to a certain extent” (1); “Yes, quite a lot” (2) and “Yes, a lot” (3). The responses can be summarized and transformed to a scale score ranging between 0 and 100 (raw score; lowest possible score/possible score range × 100), where a higher score implies higher symptom burden due to the arrhythmia. Outside of the symptom scale, there is one item concerning frequency, two about duration, and one about palpitations. The patients are asked if there are factors influencing arrhythmia occurrence and the experience of near syncope and/or syncope in connection with arrhythmia. Part III measures the arrhythmia’s influence on daily life concerns with the ASTA 13-item HRQoL scale, which has the same response alternatives as the ASTA symptom scale. Using the same scoring as for the ASTA symptom scale, a total score can be calculated, where higher scores reflect a worse effect on HRQoL. In additional, two subscales can be used, a physical including seven items, and a mental including six items. (Supplement 1) This study focused on evaluating the ASTA symptom burden and HRQoL scales with Rasch analysis. Therefore, both were treated as unidimensional scales in the present study.

The translation process

The Swedish version has been translated into English some years ago using well-recognized techniques inspired by The Professional Society for Health Economics and Outcomes Research (ISPOR), with translations by native English and Swedish speaking persons [10].

The ASTA questionnaire was initially translated into English by a native Swedish-speaking person who had lived in Great Britain for ten years. The version was discussed in the research team, which included the constructors of the ASTA questionnaire. Thereafter the English version was evaluated by a Swedish electrophysiologist who had worked in Canada and who was skilled in the treatment of patients with arrhythmias. In the next step, the two versions were examined by a translation agency. Finally, the English version was discussed in a focus group with four healthcare professionals. Three of them were native Swedish-speaking physicians, skilled in English and working with patients with arrhythmias. The person who originally translated the ASTA questionnaire confirmed the result, with one correction for wording (one item in the HRQoL scale). The process continued with two native English-speaking persons, not involved in the translation process, filled out the English version of the ASTA questionnaire. As a part of the validation work the patients at the clinic in Adelaide, Australia were asked to consider the wording and to comment on whether or not there were any uncertainties.

Step II: evaluation of the measurement properties

Study population

The study involved patients from the Centre for Heart Rhythm Disorders at the University of Adelaide, Australia. Symptomatic patients who were referred for treatment of AF and/or AFL including ablation and cardioversion during November 2017 until February 2019 were approached to take part in the study by completing the ASTA questionnaire. Patients were included if they were ≥ 18 years, symptomatic, proficient in English and physically and mentally able to complete the questionnaire. In total, 212 patients were invited to participate, of which 205 agreed to complete the ASTA questionnaire. Of the returned questionnaires (n = 205), three were incomplete and were excluded, so the final sample included 202 patients.

Completion of the questionnaire

Patients were asked to complete the questionnaire prior to their appointment with their cardiologist and before treatment. This was done so the patient was not informed of their rhythm prior to completing the form, in order to avoid influencing their responses. The first 26 patients filled out the ASTA questionnaire in its paper version and the others via its web-version.

The questionnaire was made available via a website interface (Fig. 1). Patients would log onto the web-based interface and enter a unique identification number, then proceed through the questionnaire. The interface, provided by Nordsoft AB Sweden, enabled easy use with single click answer selection. On completion, the questionnaire was uploaded to a secure internet-based database for storage and analysis.

ASTA website interface. ASTA=Arrhythmia-Specific questionnaire in Tachycardia and Arrhythmia

As a part of validation of the English version of the ASTA questionnaire the patients were asked to consider the wording and to comment on whether or not there were any uncertainties, but no one pointed anything out concerning this. The most common issue the patients encountered concerned medications. For some, questions regarding frequency and duration of AF episodes rendered in some perplexities.

Statistical analysis

Descriptive statistics were used to present the study population and to evaluate data quality in terms of score distributions for items and scales. Floor and ceiling effects are commonly defined if more than 20% of the respondents achieve the lowest and/or highest scores [11], which was adopted in the present study. These data were analyzed using Stata version 16.1 (StataCorp LP, College Station, TX).

The unidimensional Rasch model for ordered categories (unrestricted polytomous Rasch model) was used to evaluate the ASTA symptom and HRQoL scales [12]. The Rasch analysis was undertaken using RUMM2030 version 5.4 (Rumm Laboratory Pty Ltd, Duncraig, Australia). The analyses were based on five class intervals (i.e., persons with similar levels on the ASTA symptom and HRQoL scales respectively) to ensure a sufficiently large number of persons in each (n ≥ 30). Nine patients had extreme scores (i.e., reported the highest or lowest possible scores on all items) on the ASTA symptom scale and 20 on the ASTA HRQoL scale. In the Rasch model, extreme scores correspond to infinite or indefinite measures on the latent variable and are therefore not estimable. In RUMM, persons with extreme scores are therefore provided with a tentative estimate of their location parameter [12].

The following aspects were evaluated:

Global model fit A perfect global model fit is reflected by mean residual values close to 0 and standard deviations close to 1 for both item and persons. Moreover, the total item trait interaction (chi-square based statistics) should be non-significant [13].

Individual item fit Individual fit of items is reflected by standardized fit residual values within the range ± 2.5 and non-significant Bonferroni corrected p-values [11]. The Bonferroni corrected p-value depends on the number of items and was therefore set at p < 0.006 for the ASTA symptom scale and p < 0.004 for the ASTA HRQoL scale. The individual item fit was also graphically examined using the item characteristic curves. These curves illustrate the probability of a correct response dependent on the person’s ability (level on the latent variable) and the item difficulty [12].

Response categories functioning The ordering of the centralized thresholds for each item was inspected to evaluate the response categories functioning. Thresholds can be defined as the point between two response categories where either response is equally probable. Therefore, disorder thresholds may indicate that the scoring function (i.e., response categories) is not working as intended [11].

Local independency An important assumption of the Rasch model is that items in a test should not be related to each other after the effect of the latent variable is conditioned out. Violations to this assumption, i.e., local dependency, are reflected by high correlations between item residuals. Different critical values have been suggested but correlations greater than 0.2 above the average of all item residual correlations have been suggested as problematic in most situations [14].

Unidimensionality Another important assumption of the Rasch model is that the latent variable is unidimensional, i.e., that all items reflect one underlying construct. This assumption is commonly confirmed by satisfactory model fit statistics and lack of response dependency (i.e., local independency) [15]. A combined principal component analysis (PCA) of residuals and t-test approach was used in the present study. Items with the strongest positive and negative loadings on the first principal component were used to estimate separate person locations and associated standard errors. A series of t-tests was then conducted to compare person locations based on the two different subsets of items. Fewer than 5% of the t-tests are supposed to be significant (p < 0.05), alternatively the lower bound of the Agresi-Coull binominal 95% confidence interval should overlap by 5% to support unidimensionality [16, 17].

Person-item threshold distribution This aspect reflects to what extent the item difficulty represents person ability, i.e., level of symptoms and HRQoL among the respondents. For this purpose, the item thresholds were compared with the person ability level. The mean person location is expected to be around the mean item threshold location, i.e., 0 logits. In addition, the item thresholds are expected to cover about the same range of the logit scale as person locations [11].

Person separation index The person separation index reflects the ability of the measure to discriminate between persons with different levels of the construct. It is also a measure of internal consistency, analogous to Cronbach’s alpha. Thus, the person separation index is expected to exceed 0.7 to support reliability [11]. In the present study, also ordinal alpha and Cronbach’s alpha were calculated to examine internal consistency [18, 19].

Differential item functioning for age and gender Differential item functioning (DIF) implies that different groups have comparable levels of the latent variable but respond differently to individual items. To detect DIF for age and gender, a two-way analysis of variance across these person factors and class intervals was conducted [13] Age was classified as younger (< 65 years) and older (≥ 65 years). The main effect of the person factors was used to detect uniform DIF while the interaction effect between the person factor and class intervals was used to detect non-uniform DIF. Due to the large number of comparisons, Bonferroni corrections were applied: p < 0.002 was used to detect DIF for the ASTA symptom scale and p < 0.001 for the ASTA HRQoL scale.