# Development of an observational – perceptual heat strain risk assessment (OPHSRA) index and its validation – BMC Public Health

Dec 30, 2021

### Questionnaire development

#### Identification and categorization of effective items in producing heat strain

Effective items in producing heat strain were identified by a literature review through a search in known databases and interviews with the experts of occupational health. Then, those were reviewed and the repetitive and irrelative items were omitted. Improper items for designing the qualitative question were also eliminated. In final, 37 items remained in the study. Those were classified into six groups, including personal, environmental, job, administrative, clothing, and lifestyle items based on the balance theory of job design [19]. Based on this theory, a working system is made of five elements of individual, environment, task, tools, and technology, and organization. The balance between these five factors reduces the stress load. In the present study, personal items were considered as the individual factor, environmental items as the environment factor, job items as the task factor, clothing items as the tools and technology factor, administrative items as the organization factor. Additionally, lifestyle was added to these factors [20].

#### Questionnaire design

In this phase, a number of questions were generated for assessing the identified effective items. These questions were divided into three parts of observational, descriptive, and perceptual questions. Moreover, several responses were designed for each question. For quantifying the items, equivalent scores for each response were determined using subject-matter expertise, and later, those were modified based on the opinions of several experts. The draft questionnaire of the observational-perceptual heat strain risk assessment (OPHSRA) index included these questions. To develop the questionnaire, it was administered by the researchers. After developing the questionnaire, it can be administered by other people. The descriptive and perceptual questions are also answered by the workers.

#### Content validity evaluation

For evaluating the content validity, the questions were reviewed by ten experts with a research history on heat stress, including two professors, two associate professors, five assistant professors, and one Ph.D. candidate. They received an electronic mail, including the aims of the study and the draft questionnaire. The reviewers assessed the questions in terms of necessity using a 3-point Likert scale and in terms of relevance, clarify, and simplicity using the 4-point Likert scale. Lawshe and Waltz and Basel methods were applied to estimate the content validity ratio (CVR) and content validity index (CVI), respectively. The values of CVR and CVI greater than 0.79 and 0.62 were accepted, respectively [21, 22]. The questions with CVI values of 0.70 to 0.79 were also revised. Furthermore, averaged CVI and CVR of remained questions were calculated.

#### Reliability evaluation

For evaluating the reliability, the revised draft questionnaire was completed by 200 staff occupied in warm and dry and warm and humid areas, detailed below. Observational questions were also filled out by researchers, as experts. After, coefficients of Cronbach’s alpha (α) and McDonald’s omega were calculated for all questions and each group of questions. Moreover, the item-total correlation (ITC) coefficient of each question was computed, and the questions with ITC less than 0.3 were omitted. The ITC refers to the correlation between the item and the total scale. The minimum acceptable value of α was equal to 0.70 [23].

#### Back-translation evaluation

After preparing the final questionnaire in the English language, a back-translation was performed. For this purpose, a blind translator was asked to translate the questionnaire back into the Persian language. Then, an expert panel, including two English language specialists, two Persian language specialists, and three occupational health specialists with a research history on heat stress, compared the original and translated versions of the questionnaire and examined any discrepancies. If applicable, the panel redrafted the questions and answers until their concept, meaning, and quality became the same.

### Participants

Two hundred Iranian male staff (110 persons from a steel industry in the center of Iran as a hot and dry environment, and 90 persons from a petrochemical industry in the south of Iran as a hot and humidity ambiance) participated in the present study. It was tried that the subjects are selected from different industrial parts with a variety of climatic occupational conditions. For this purpose, the researchers attentively inspected the parts of these industries and elected the duties desired for performing the study. Then, the medical records of individuals working in these duties were investigated and the subjects with inclusion criteria were entered into the study. In the steel industry, these parts included forging, spark, induction melting, steelmaking, isolation, machining, refractory, technical support, engineering post, preventive maintenance, foundry, sandblast, metal waste separation, and administrative. In the petrochemical factory, the parts consisted of the warehouse, cookery, gardening, loading gantry, steel drum production, weighbridge, bitumen production, hydrocarbon, preventive maintenance, research and development, and administration. Inclusion criteria were career length higher than 1 year, no having mental, infectious, pulmonary, cardiovascular, hypertension, renal, hyperthyroidism, digestive, and diabetes diseases, non-use of medications to affect heart rate and blood pressure such as beta-blockers, phenothiazines, diuretics, anticholinergics, antispasmodics, psychotropics, antihistamines, antihypertensives, amphetamine, and decongestants, and non-use of coffee, caffeine, and alcohol from 12 h before the study. Furthermore, their tympanic membrane and auditory canal were medically screened. Exclusion criteria included unwillingness to impressive cooperation and body temperature higher than 39 °C during the activity.

### Sample size calculation

The aim of the present study was the development of an observational-perceptual index. The lowest correlation between the developed index and tympanic temperature was assumed as 0.2. Then, the sample size was computed based on the confidence level of 95% and a test power of 80% (Eq. 1).

$$n=frac{{left({Z}_{1-frac{alpha }{2}}+{mathrm{Z}}_{1-beta}right)}^2}{w^2}+3cong 194$$

(1)

Where ({Z}_{1-frac{alpha }{2}}) is equal to 1.96 for a confidence level of 95%, Z1 − β is equal to 0.84 for a test power of 80%, and W is equal to 0.203 for the lowest correlation coefficient of 0.2. Hence, the minimum sample size was obtained as 194 individuals.

### Measurement instruments

The thermometer of Braun (IRT 6530 model with an accuracy of 0.1 °C) was used to measure the tympanic temperature. The WBGT meter (TES 1369B model with an accuracy of 0.1 °C) was applied for measuring the environmental climatic parameters of dry temperature, wet temperature, globe temperature, and relative humidity. Moreover, the developed questionnaire of OPHSRA was exploited to subjectively evaluate the effective items in producing heat strain through observation, description, and perception.

### Index development

The indirect effect coefficients of the items on thermal strain (variations of tympanic temperature) were calculated by structural equation modeling (SEM). Each of these coefficients was multiplied by the score of the related item, and resultant values were summed together for calculating the total score of the novel index.

### Data analyses

Gathered data were analyzed by the software of statistical package for the social sciences (SPSS) version 18. The normality of variables was examined using skew and kurtosis curves. Based on the results, all items had the normal distribution. Structural equation modeling (SEM) was applied to calculate the effect coefficients of the items. At first, the factor loadings of items of each factor, as direct effect coefficient, were computed using the Varimax method. Then, the regression method was used to calculate the score of each factor, in which factor loadings were multiplied by the data of each item. Later, a theoretical model was drawn using computed scores of the factors in AMOS software. The fitness of the designed model was evaluated using fit indices. Then, the novel index was developed by the indirect effect coefficients of the items in the model. Also, given the relationships between some of the items, it may be redundancy between them. In AMOS, modification Indices identified these redundant items. Co-variation was done between the measurement errors of redundant items based on the suggestion of the software for constraining the redundancy effects and increasing the fitness of the model. Finally, receiver operator curves (ROC) analysis was applied for categorizing the score of the novel index. Boundaries of risk levels included tympanic temperatures of 37.5, 38.0, and 38.5 °C [24]. In ROC curves, nearest points to the ideal state were considered as optimal cut-off points of the developed index. The validity of the index was also apprised using linear and quadratic regression analyses.