Multimodal AI Systems for Early Cancer Detection Through Integrated Imaging, Genomic Profiles, and Clinical Risk Indicators

Md Abu Bakar  Siddique; Sanjida Alam  Eshra

doi:10.51699/cajmns.v4i2.3375

Authors

Md Abu Bakar Siddique Department of Biological Sciences, St. John's University, New York, USA
Sanjida Alam Eshra Master of Science, Business Analytics, Trine University, USA

DOI:

https://doi.org/10.51699/cajmns.v4i2.3375

Keywords:

Multimodal Artificial Intelligence, Early Cancer Detection, Medical Imaging Analysis, Genomic Data Integration

Abstract

Early cancer detection remains challenging due to disease heterogeneity and limitations of single-modality approaches. Multimodal artificial intelligence (AI) integrates imaging, genomic profiles, and clinical indicators to provide comprehensive insights for improved cancer prediction and personalized oncology. This review evaluates multimodal AI frameworks and their applications in integrating heterogeneous data sources for early cancer detection and precision medicine. A comprehensive review was conducted on recent advances in multimodal AI-based cancer detection systems, focusing on medical imaging, genomic and molecular data, and clinical risk indicators. Relevant literature was analyzed to summarize AI architectures, including machine learning, deep learning, radiomics, transformer-based models, and multi-omics integration strategies. The review also examined current applications, challenges, validation approaches, and future directions for clinical translation of multimodal AI technologies. Multimodal AI approaches demonstrate potential for improving cancer detection by combining complementary information from imaging, genomic, and clinical domains. Imaging-based AI models, including convolutional neural networks and radiomics approaches, effectively extract tumor-related features, while genomic AI models identify molecular signatures associated with cancer progression. Integration of multi-omics and electronic health record data enables more comprehensive patient characterization and risk prediction. However, current evidence highlights challenges related to data heterogeneity, missing information, model interpretability, privacy, bias, and limited external validation. Further development of robust and explainable AI systems is required for reliable clinical implementation. Multimodal AI represents a promising research direction for cancer detection, but standardized validation and responsible implementation are essential before routine clinical adoption.

References

C. Mattiuzzi and G. Lippi, “Current Cancer Epidemiology,” J. Epidemiol. Glob. Health, vol. 9, no. 4, p. 217, 2019, doi: 10.2991/jegh.k.191008.001.

S. V. S. Deo, J. Sharma, and S. Kumar, “GLOBOCAN 2020 Report on Global Cancer Burden: Challenges and Opportunities for Surgical Oncologists,” Ann. Surg. Oncol., vol. 29, no. 11, pp. 6497–6500, Oct. 2022, doi: 10.1245/s10434-022-12151-6.

G. W. Prager et al., “Global cancer control: responding to the growing burden, rising costs and inequalities in access,” ESMO Open, vol. 3, no. 2, p. e000285, 2018, doi: 10.1136/esmoopen-2017-000285.

K. A. Tran, O. Kondrashova, A. Bradley, E. D. Williams, J. V. Pearson, and N. Waddell, “Deep learning in cancer diagnosis, prognosis and treatment selection,” Genome Med., vol. 13, no. 1, p. 152, Dec. 2021, doi: 10.1186/s13073-021-00968-x.

M. J. Iqbal et al., “Clinical applications of artificial intelligence and machine learning in cancer diagnosis: looking into the future,” Cancer Cell Int., vol. 21, no. 1, p. 270, May 2021, doi: 10.1186/s12935-021-01981-1.

P. Kumar et al., “Tinospora cordifolia (Giloy): Phytochemistry, Ethnopharmacology, Clinical Application and Conservation Strategies,” Curr. Pharm. Biotechnol., vol. 21, no. 12, pp. 1165–1175, Oct. 2020, doi: 10.2174/1389201021666200430114547.

J. Lipkova et al., “Artificial intelligence for multimodal data integration in oncology,” Cancer Cell, vol. 40, no. 10, pp. 1095–1110, Oct. 2022, doi: 10.1016/j.ccell.2022.09.012.

K. M. Boehm, P. Khosravi, R. Vanguri, J. Gao, and S. P. Shah, “Harnessing multimodal data integration to advance precision oncology,” Nat. Rev. Cancer, vol. 22, no. 2, pp. 114–126, Feb. 2022, doi: 10.1038/s41568-021-00408-3.

Z. Dlamini, F. Z. Francies, R. Hull, and R. Marima, “Artificial intelligence (AI) and big data in cancer and precision oncology,” Comput. Struct. Biotechnol. J., vol. 18, pp. 2300–2311, Jan. 2020, doi: 10.1016/j.csbj.2020.08.019.

E. Capobianco, “High-dimensional role of AI and machine learning in cancer research,” Br. J. Cancer, vol. 126, no. 4, pp. 523–532, Mar. 2022, doi: 10.1038/s41416-021-01689-z.

L. Schneider et al., “Integration of deep learning-based image analysis and genomic data in cancer pathology: A systematic review,” Eur. J. Cancer, vol. 160, pp. 80–91, Jan. 2022, doi: 10.1016/j.ejca.2021.10.007.

B. Lobato-Delgado, B. Priego-Torres, and D. Sanchez-Morillo, “Combining Molecular, Imaging, and Clinical Data Analysis for Predicting Cancer Prognosis,” Cancers (Basel)., vol. 14, no. 13, p. 3215, Jun. 2022, doi: 10.3390/cancers14133215.

R. S. Vanguri et al., “Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer,” Nat. Cancer, vol. 3, no. 10, pp. 1151–1164, Aug. 2022, doi: 10.1038/s43018-022-00416-8.

R. C. Fitzgerald, A. C. Antoniou, L. Fruk, and N. Rosenfeld, “The future of early cancer detection,” Nat. Med., vol. 28, no. 4, pp. 666–677, Apr. 2022, doi: 10.1038/s41591-022-01746-x.

R. Li, Y. Chen, M. D. Ritchie, and J. H. Moore, “Electronic health records and polygenic risk scores for predicting disease risk,” Nat. Rev. Genet., vol. 21, no. 8, pp. 493–502, Aug. 2020, doi: 10.1038/s41576-020-0224-1.

D. Gu, K. Su, and H. Zhao, “A case-based ensemble learning system for explainable breast cancer recurrence prediction,” Artif. Intell. Med., vol. 107, p. 101858, Jul. 2020, doi: 10.1016/j.artmed.2020.101858.

A. Kline et al., “Multimodal machine learning in precision health: A scoping review,” NPJ Digit. Med., vol. 5, no. 1, p. 171, Nov. 2022, doi: 10.1038/s41746-022-00712-8.

S. K. Patel, B. George, and V. Rai, “Artificial Intelligence to Decode Cancer Mechanism: Beyond Patient Stratification for Precision Oncology,” Front. Pharmacol., vol. 11, Aug. 2020, doi: 10.3389/fphar.2020.01177.

L. Emmett et al., “The Additive Diagnostic Value of Prostate-specific Membrane Antigen Positron Emission Tomography Computed Tomography to Multiparametric Magnetic Resonance Imaging Triage in the Diagnosis of Prostate Cancer (PRIMARY): A Prospective Multicentre Study,” Eur. Urol., vol. 80, no. 6, pp. 682–689, Dec. 2021, doi: 10.1016/j.eururo.2021.08.002.

J. Van Damme et al., “Comparison of 68Ga-Prostate Specific Membrane Antigen (PSMA) Positron Emission Tomography Computed Tomography (PET-CT) and Whole-Body Magnetic Resonance Imaging (WB-MRI) with Diffusion Sequences (DWI) in the Staging of Advanced Prostate Cancer,” Cancers (Basel)., vol. 13, no. 21, p. 5286, Oct. 2021, doi: 10.3390/cancers13215286.

F. S. Furtado et al., “Positron Emission Tomography/Magnetic Resonance Imaging (PET/MRI) Versus the Standard of Care Imaging in the Diagnosis of Peritoneal Carcinomatosis,” Ann. Surg., vol. 277, no. 4, pp. e893–e899, Apr. 2023, doi: 10.1097/SLA.0000000000005418.

S. R. Stahlschmidt, B. Ulfenborg, and J. Synnergren, “Multimodal deep learning for biomedical data fusion: a review,” Brief. Bioinform., vol. 23, no. 2, Mar. 2022, doi: 10.1093/bib/bbab569.

K. M. Boehm et al., “Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer,” Nat. Cancer, vol. 3, no. 6, pp. 723–733, Jun. 2022, doi: 10.1038/s43018-022-00388-9.

R. Schulte-Sasse, S. Budach, D. Hnisz, and A. Marsico, “Integration of multiomics data with graph convolutional networks to identify new cancer genes and their associated molecular mechanisms,” Nat. Mach. Intell., vol. 3, no. 6, pp. 513–526, Apr. 2021, doi: 10.1038/s42256-021-00325-y.

Y. You et al., “Artificial intelligence in cancer target identification and drug discovery,” Signal Transduct. Target. Ther., vol. 7, no. 1, p. 156, May 2022, doi: 10.1038/s41392-022-00994-0.

X.-M. Zhang, L. Liang, L. Liu, and M.-J. Tang, “Graph Neural Networks and Their Current Applications in Bioinformatics,” Front. Genet., vol. 12, Jul. 2021, doi: 10.3389/fgene.2021.690049.

Z. Zuo, P. Wang, X. Chen, L. Tian, H. Ge, and D. Qian, “SWnet: a deep learning model for drug response prediction from cancer genomic signatures and compound chemical structures,” BMC Bioinformatics, vol. 22, no. 1, p. 434, Dec. 2021, doi: 10.1186/s12859-021-04352-9.

G. Muzio, L. O’Bray, and K. Borgwardt, “Biological network analysis with deep learning,” Brief. Bioinform., vol. 22, no. 2, pp. 1515–1530, Mar. 2021, doi: 10.1093/bib/bbaa257.

G. Mirzaei, “GraphChrom: A Novel Graph-Based Framework for Cancer Classification Using Chromosomal Rearrangement Endpoints,” Cancers (Basel)., vol. 14, no. 13, p. 3060, Jun. 2022, doi: 10.3390/cancers14133060.

D. E. Oprea-Lager et al., “Bone Metastases Are Measurable: The Role of Whole-Body MRI and Positron Emission Tomography,” Front. Oncol., vol. 11, Nov. 2021, doi: 10.3389/fonc.2021.772530.

Y. Ming et al., “Progress and Future Trends in PET/CT and PET/MRI Molecular Imaging Approaches for Breast Cancer,” Front. Oncol., vol. 10, Aug. 2020, doi: 10.3389/fonc.2020.01301.

D. R. Sarvamangala and R. V. Kulkarni, “Convolutional neural networks in medical image understanding: a survey,” Evol. Intell., vol. 15, no. 1, pp. 1–22, Mar. 2022, doi: 10.1007/s12065-020-00540-3.

R. Yang and Y. Yu, “Artificial Convolutional Neural Network in Object Detection and Semantic Segmentation for Medical Imaging Analysis,” Front. Oncol., vol. 11, Mar. 2021, doi: 10.3389/fonc.2021.638182.

Md Abu Bakar Siddique and Asim Debnath, “Advancing Medical Science through Nanobiotechnology: Prospects, Applications, and Future Directions,” Journal of Primeasia, vol. 1, no. 1, pp. 1–10, Mar. 2018, doi: 10.25163/primeasia.1110163.

W. Zhu, L. Xie, J. Han, and X. Guo, “The Application of Deep Learning in Cancer Prognosis Prediction,” Cancers (Basel)., vol. 12, no. 3, p. 603, Mar. 2020, doi: 10.3390/cancers12030603.

A. U. Mazlan et al., “A Review on Recent Progress in Machine Learning and Deep Learning Methods for Cancer Classification on Gene Expression Data,” Processes, vol. 9, no. 8, p. 1466, Aug. 2021, doi: 10.3390/pr9081466.

W. S. Alharbi and M. Rashid, “A review of deep learning applications in human genomics using next-generation sequencing data,” Hum. Genomics, vol. 16, no. 1, p. 26, Jul. 2022, doi: 10.1186/s40246-022-00396-x.

V. Romeo et al., “Assessment and Prediction of Response to Neoadjuvant Chemotherapy in Breast Cancer: A Comparison of Imaging Modalities and Future Perspectives,” Cancers (Basel)., vol. 13, no. 14, p. 3521, Jul. 2021, doi: 10.3390/cancers13143521.

M. Madani, M. M. Behzadi, and S. Nabavi, “The Role of Deep Learning in Advancing Breast Cancer Detection Using Different Imaging Modalities: A Systematic Review,” Cancers (Basel)., vol. 14, no. 21, p. 5334, Oct. 2022, doi: 10.3390/cancers14215334.

C. A. Khella, G. A. Mehta, R. N. Mehta, and M. L. Gatza, “Recent Advances in Integrative Multi-Omics Research in Breast and Ovarian Cancer,” J. Pers. Med., vol. 11, no. 2, p. 149, Feb. 2021, doi: 10.3390/jpm11020149.

G. Jung, E. Hernández-Illán, L. Moreira, F. Balaguer, and A. Goel, “Epigenetics of colorectal cancer: biomarker and therapeutic potential,” Nat. Rev. Gastroenterol. Hepatol., vol. 17, no. 2, pp. 111–130, Feb. 2020, doi: 10.1038/s41575-019-0230-y.

MF Akter, MR Islam, AK Manica, MAB Siddique, and Tufael, “Structural and Pharmacological Insights into Withaferin a Binding to Mutant p53 (R248Q): Multi-Faceted Inhibitor in Cancer Treatment,” Integrative Biomedical Research, vol. 6, no. 2, pp. 1–11, 2022.

S. Saxena et al., “Role of Artificial Intelligence in Radiogenomics for Cancers in the Era of Precision Medicine,” Cancers (Basel)., vol. 14, no. 12, p. 2860, Jun. 2022, doi: 10.3390/cancers14122860.

A. Hope et al., “Artificial Intelligence Applications to Improve the Treatment of Locally Advanced Non-Small Cell Lung Cancers,” Cancers (Basel)., vol. 13, no. 10, p. 2382, May 2021, doi: 10.3390/cancers13102382.

H. Arabi and H. Zaidi, “Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy,” Eur. J. Hybrid Imaging, vol. 4, no. 1, p. 17, Dec. 2020, doi: 10.1186/s41824-020-00086-8.

Tufael and Atikur Rahman Sunny, “Artificial Intelligence in Healthcare: A Review of Diagnostic Applications and Impact on Clinical Practice,” Journal of Primeasia, vol. 2, no. 1, pp. 1–5, Jan. 2021, doi: 10.25163/primeasia.219816.

D. B. Seal, V. Das, S. Goswami, and R. K. De, “Estimating gene expression from DNA methylation and copy number variation: A deep learning regression model for multi-omics integration,” Genomics, vol. 112, no. 4, pp. 2833–2841, Jul. 2020, doi: 10.1016/j.ygeno.2020.03.021.

P. Jurmeister et al., “DNA methylation-based classification of sinonasal tumors,” Nat. Commun., vol. 13, no. 1, p. 7148, Nov. 2022, doi: 10.1038/s41467-022-34815-3.

K. Preuss et al., “Using Quantitative Imaging for Personalized Medicine in Pancreatic Cancer: A Review of Radiomics and Deep Learning Applications,” Cancers (Basel)., vol. 14, no. 7, p. 1654, Mar. 2022, doi: 10.3390/cancers14071654.

J. E. van Timmeren, D. Cester, S. Tanadini-Lang, H. Alkadhi, and B. Baessler, “Radiomics in medical imaging—‘how-to’ guide and critical reflection,” Insights Imaging, vol. 11, no. 1, p. 91, Dec. 2020, doi: 10.1186/s13244-020-00887-2.

A. S. Tagliafico, M. Piana, D. Schenone, R. Lai, A. M. Massone, and N. Houssami, “Overview of radiomics in breast cancer diagnosis and prognostication,” The Breast, vol. 49, pp. 74–80, Feb. 2020, doi: 10.1016/j.breast.2019.10.018.

T.-H. Zhang et al., “Transformer for Gene Expression Modeling (T-GEM): An Interpretable Deep Learning Model for Gene Expression-Based Phenotype Predictions,” Cancers (Basel)., vol. 14, no. 19, p. 4763, Sep. 2022, doi: 10.3390/cancers14194763.

N. Aristokli, I. Polycarpou, S. C. Themistocleous, D. Sophocleous, and I. Mamais, “Comparison of the diagnostic performance of Magnetic Resonance Imaging (MRI), ultrasound and mammography for detection of breast cancer based on tumor type, breast density and patient’s history: A review,” Radiography, vol. 28, no. 3, pp. 848–856, Aug. 2022, doi: 10.1016/j.radi.2022.01.006.