The study was carried out in the tri-island nation of Grenada, West Indies. Grenada has been an independent English-speaking nation since 1974. Twenty-three percent of Grenada’s population is under 15 years of age . Throughout the region, approximately half of all households are headed by a single parent . Despite its ranking as an upper middle-income country, approximately a third of its 112,000 residents live in poverty. Unemployment is reported at 20% and the nation’s debt ratio of 110% GDP ranks second worldwide . The gross national income per capita was reported at $11,650 United States Dollars in 2014 . Ethnically, more than 80% of the population identifies as Afro-Caribbean; the remainders are of mixed, or East Indian ancestry .
Child development assessors sample
The characteristics of the seven non-specialist child development assessors are described in Table 1. All of the assessors were female and native Caribbean-English speakers. Five of the assessors were Caribbean nationals and two were American citizens. None of these assessors were psychologists or healthcare professionals, and none had a formal education beyond an undergraduate degree. Their income, representative of their socioeconomic status, was equivalent to Grenada’s 2014 gross national income per capita at $11,750 United States Dollars , adjusted for inflation.
Inter-rater and test-retest reliability sample – child participants
To evaluate inter-rater and test-retest reliability of the adapted INTER-NDA, each of the seven non-specialist child developmental assessors assessed three children, for a total of 21 child participants. Children ranged from 22 to 30 months old and were recruited from the “Saving Brains Grenada” project – a randomized controlled trial study assessing the impact of a Conscious Discipline intervention on reducing corporal punishment rates and improving neurodevelopmental outcomes among Grenadian children . The 21 children included in the assessor training were recruited from four day-care centers around the capital city of St. George’s and were randomly assigned to a child development assessor for evaluation. The majority were female (52.4%), and all the children were Afro-Caribbean and spoke English as their native language (Table 2).
Internal consistency sample – child participants
To evaluate the Internal consistency of the adapted INTER-NDA, n = 145 children between 22 and 30 months old were recruited from the “Saving Brains Grenada” project and assessed on the INTER-NDA. A detailed description of this project and its methodology has been published elsewhere . The children were recruited from communities across Grenada via a community based ECD outreach program called The Roving Caregivers. The sample size (n = 145) is consistent with that recommended by the Toolkit for Measuring Early Child Development in Low- and Middle-Income Countries . The sample consisted of majority female (n = 53.8%), all were Afro-Caribbean, all but one spoke English as a native language, and the average child age was 26 months (SD = 2.21 months) (Table 3).
The INTERGROWTH-21st project neurodevelopment assessment (INTER-NDA)
The INTER-NDA is a comprehensive, rapid assessment of cognition, (fine and gross) motor skills, language and (positive and negative) behavior for children aged 22–30 months (Additional file 1) . Its 37 items are administered using a combination of psychometric techniques (direct administration, concurrent observation and caregiver reports) in approximately 15 min. Children’s performance on the INTER-NDA is scored across a spectrum of abilities, rather than on a predefined checklist and, therefore, affords a wider description of a child’s faculties. It has been reported to have strong agreement with the Bayley Scales of Infant and Toddler Development III edition, (BSID-III) (interclass correlation coefficients 0·75–0·88, p < 0·001 for all domains with little to no bias on Bland Altman analysis); satisfactory internal consistency (Cronbach’s alpha 0.56–0.81) and good unidimensionality across subscales (Comparative Fit Index = 0.90; Tucker-Lewis Index = 0.94) and good levels of inter-rater (k = 0·70; 95% CI 0·47 to 0·88) and test–retest reliability (k = 0·79; 95% CI 0·48 to 0·96) [13, 24].
The INTER-NDA is designed for use across socioeconomic groups and populations. Its operation manual, standardization protocol and forms are freely available at www.intergrowth21.org.uk. The kit consists of common household items encountered across the world. The INTER-NDA was developed in 2014 by the International Fetal and Newborn Growth Consortium for the twenty-first Century (INTERGROWTH-21st) Project, a population-based, longitudinal study in five countries, including Brazil, India, Italy, Kenya, and the United Kingdom . In all INTERGROWTH-21st Project study sites, the INTER-NDA was translated into the local languages of the sites using the WHO Mental Health Initiative translation guidelines, which included processes of cultural customization, translation and back translation . The INTER-NDA’s norms are international standards (rather than population-specific references that are adapted for use in the Caribbean) for child development at 2 years of age, constructed from the INTERGROWTH-21st Project population using, the WHO’s prescriptive methodology. Scaled INTER-NDA domain scores are interpreted against the INTER-NDA’s standards to ascertain a child’s risk of no (>10th centile), mild-to-moderate (3rd to 10th centile) or severe (<3rd centile) neurodevelopmental delay for each of the domains .
Training of child development assessors
Seven non-specialist child development assessors were trained in the INTER-NDA in Grenada over a five-day period by the developer of the INTER-NDA (MF) who is a UK-based paediatrician, and who served as the expert assessor (expert) for the purpose of this study. During the training process, importance was paid to the conceptual basis of each item, the accurate administration of tasks to the child, as well as the accurate and objective reporting of the child’s performance on each item. All sessions were video recorded, and the expert observed the trainee-assessors carrying out three assessments each on 22-30-month-old children, randomly assigned. After these sessions, the expert provided each trainee-assessor with feedback on their administration and scoring of the INTER-NDA and on their interaction with the child and the caregiver.
INTER-NDA adaptation process to the Caribbean context
The process of the linguistic and cultural adaptation of the INTER-NDA was undertaken on day three of the training, after each trainee-assessor had assessed at least one child, and involved all trainee-assessors, the study PIs and the expert assessor. This process adhered to the previously published, recommended guidelines for the adaptation of an ECD instrument . The process consisted of the following steps:
The trainee-assessors and PIs evaluated each of the 37 INTER-NDA items for linguistic and cultural relevance. Items considered to benefit from amendments to fit the local context were identified.
The trainee-assessors proposed alternatives to the phraseology of the items identified above. These options were discussed with the study PIs and expert assessor. Each option was scrutinized for contextual relevance by the trainee and expert assessors; any conflicts identified were discussed among all the assessors and alternative phraseology was proposed.
The eight adapted items were compared to the original INTER-NDA items by the expert assessor to confirm conceptual equivalence in accurately screening for NDIs. Conflicts identified were discussed with the trainee-assessors, alternative phraseology was considered and the process of assessment for contextual and conceptual equivalence repeated until a consensus was achieved.
The final list of adapted items were included into the measure and piloted on children aged 22-30 months.
Evaluation of the INTER-NDA’s administration and scoring by non-specialist child development assessors
Each non-specialist child developmental assessor was evaluated for their ability to (1) administer and (2) score the INTER-NDA in a standardized manner according the INTERGROWTH-21st Project protocols. To assess their ability to administer the INTER-NDA in a standardized manner, each non-specialist child development assessor was rated for protocol adherence on the INTERGROWTH-21st Project’s INTER-NDA protocol adherence checklist (Additional file 2; http://www.medscinet.net/Intergrowth/patientinfodocs/Standardisation%20Protocol.pdf) by the six other non-specialist child developmental assessors and the expert assessor. The agreement between each trainee-assessors’ INTER-NDA scores was compared with the expert’s INTER-NDA scores, across INTER-NDA domains, for the four video recordings of the INTER-NDA described in the reliability experiment above. Protocol adherence scores were summed across all the items to yield an overall INTER-NDA protocol adherence score for each trainee-assessor, from which protocol adherence percentages (range 0-100%) were calculated. The agreement between INTER-NDA domain scores, measured on the adapted INTER-NDA, was compared between trainee-assessors and the expert using kappa coefficients [6, 7].
Statistical analysis – assessment of psychometric properties of the adapted INTER-NDA
All data were analysed in SPSS Statistics v126.96.36.199©IBM Corp. Inter-rater and test-retest reliability between INTER-NDA scores across domains was determined for each trainee-assessor based on their scoring of four videos of the expert assessor administering the INTER-NDA to two-year old children. The second and fourth videos were identical, and trainee-assessors scored these videos separately, without access to the scores of video 1, allowing an assessment of test-retest reliability. Discussion between trainee-assessors and replaying of sections of the videos was not permitted. The video-based approach was selected over a conventional real-time approach to ensure that the child’s scores were not affected by: (i) anxiety about performance in the presence a large number of assessors; and (ii) inability of assessors to hear and see the child’s performance uniformly at all times during the assessment. This approach has been previously used in the INTERGROWTH-21st Project for reliability assessments [14, 24]. Each trainee-assessor administered the INTER-NDA on three children, randomly assigned, Test-retest and inter-rater reliability for the adapted INTER-NDA, across domains, was quantified using kappa coefficients [6, 7]. Kappas of 0.81 and above are considered representative of almost perfect agreement, kappas of 0.61 – 0.80 as substantial agreement, kappas of 0.41 – 0.60 as moderate agreement, kappas of 0.21 – 0.40 as fair agreement, and kappas of 0.20 and below as poor agreement . Internal consistency was determined for each INTER-NDA domain by calculating Cronbach’s alphas on a separate group of n = 145 children between 22 and 30 months of age . Cronbach’s alpha values were considered “good” if they were above a threshold of 0.7 .
The Institutional Review Board at St. George’s University approved the study (IRB #14066). All research personnel involved in the study completed the National Institutes of Health Office of Extramural Research Protecting Human Research Participants online course. Parents/guardians provided informed written consent on behalf of the participating children. Participating child development assessors provided informed written consent.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.