The Artificial Intelligence for Health Imaging (AI4HI) projects is a network of five EU-funded research projects currently working on Artificial Intelligence (AI) solutions based on medical images and related clinical and molecular data, to improve clinical practice. These projects are Primage (GA 826494), Chaimeleon (GA 952172), EuCanImage (GA 952103), Incisive (GA 952179), and Pro-Cancer-I (GA 952159). Although the projects differ in several key aspects, some common strategies and architectures can be foreseen regarding the efforts to construct validated AI tools using medical imaging and combining with relevant related data to estimate clinical events in daily oncologic practice. Basically, data from electronic health records and PACS is selected and extracted based on defined common data elements, de-identified, harmonized to a common framework, and stored in databases and image repositories before the AI models are trained, tuned, and validated to improve a clinical pathway. In this process, researchers should extract and prepare data (data scientists), construct AI models (AI scientist) and design the study to maximize clinical impact (medical scientist).
In medical imaging, AI-related research is largely based upon observational, non-interventional in silico studies performed by computer simulation on routinely collected Real World Data (RWD). As the patient episode is usually closed/completed, the dataset in these observational studies is retrospectively collected and anonymized, and there is no possible link between patients, data collection process, and AI-researchers, with such a post hoc study recruitment policy. The non-interventional nature is guaranteed as researchers only address the design, implementation, and evaluation of the AI algorithms in a computational environment (Fig. 1). The data used in these studies come from Electronic Medical Records (EMRs) from the participant hospitals or research biobanks. AI4HI projects are involved in the construction of research repositories as biobanks for cancer images and related data. The created datasets contain use-cases whose collection is defined by the clinical objective, retrieved data, and clinical endpoints (CEPs) of interest (Fig. 2).
These datasets are used for the training, tuning, and testing of the AI models developed to improve the reproducibility and estimation of CEP events. The training and tuning datasets are used for the construction of the AI solution, while the testing dataset is used for the internal validation analysis (accuracy and repeatability). An external validation set with data from different centers and scanners is constructed and used for a final reproducibility analysis to ensure robustness of the resulting model. The dataset constructed from different centers constitutes the basis for external clinical validations  (Fig. 3).
Indicative examples of AI models and Machine Learning (ML) algorithms currently under research and implementation in the AI4HI projects follow. Researchers are developing AI-based models in an open cloud-based platform to support decision-making in the clinical management of two pediatric cancers (neuroblastoma and diffuse intrinsic pontine glioma). The project utilizes standard-of-care MR and CT images at diagnosis and follow-up time points, together with clinical and molecular data, for the prediction of relevant clinical endpoints such as overall survival, time to progression/relapse, and response to treatment. In addition, special emphasis is given to the automation of the image preparation by building image quality control tools based on unsupervised learning techniques (clusterization), creating ML models from DICOM metadata for the labeling of MR sequences, and training convolutional neural networks (CNNs) for the automatic segmentation of tumor and adjacent organs.
Regarding breast cancer, mammographic images are first passed through a ML-enabled classifier trained with both control and abnormal images and related clinical and pathological data. If classified as abnormal, a second classifier is trained to determine the type of abnormality (lesion, calcifications, or both). Depending on the outcome, different AI-based segmentation models look for the respective region of interest and produce the output masks. Additional classification models will be trained to determine the BIRADS score and breast density, features that are of particular importance in the management of patients. The AI4HI ML solutions for breast cancer also address breast MR images to segment and classify the lesions, combining the outputs with other clinical data for precise disease staging.
Other challenges include the development of AI-powered pipelines for data deidentification, curation, annotation, authenticity protection, and image harmonization. In particular, the development of image harmonization Deep Learning (DL) algorithms is based on either Generative Adversarial Networks (GANs), where images from different manufacturers are converted to a reference, and self-supervised learning techniques, where original images and simple transformations are used as input data to a CNN-based autoencoder, which is then trained to reconstruct an harmonized version of the original image.
The application of validated AI-based solutions is essential for precision medicine to provide physicians with a trustworthy clinical decision support system (CDSS) . Having an impact on a specific clinical pathway is defined by the diagnostic gain in comparison with standard of care and the strong relationship between algorithm event predictions and final ground truth. Ensuring several key aspects, such as clearly defining the technical biases and clinical validation phases, and the evaluation of the impact through the strength of the prediction inference is vital for success. To ensure clinical use, the target population, dataset splitting, validation methodology, reference standards, and clinical performance metrics should be clearly identified . Furthermore, CEPs must be carefully selected, whether these are diagnostic disease behavior, treatment response, or patient prognosis or outcome [4,5,6,7]. In the field of AI-assisted tools as medical devices, their clinical acceptance requires proven capability of extrapolating the computational solutions into multicentric studies and heterogeneous datasets.
Our objective is to present the main steps for AI research that our AI4HI projects share and envision, including additional desirable validation steps such as largescale external validations which will be mandatory before real-life deployment of the research prototypes. Any developed, validated or deployed AI solution aimed toward specific clinical impact in oncologic imaging must be monitored for the following properties: fairness and unbiasedness, universality and standardization, robustness, reliability, explainability and trustworthiness, traceability and monitoring, as well as usability and equity in transferability .
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.