i-manager Publications

Unlocking Clinical Insights from Medical Images using Deep Learning

Ushaa Eswaran*, Vishal Eswaran**

* Department of Electronics and Communication Engineering, Indira Institute of Technology and Sciences, Markapur, Andhra Pradesh, India.

** Consumer Value Store Health Centre, Dallas, Texas, United States.

Periodicity:July - December'2023
DOI : https://doi.org/10.26634/jaim.1.2.20044

Abstract

Medical imaging is fundamental to modern precision medicine and the analysis of complex image data requires sophisticated techniques. This review focuses on recent deep-learning techniques for medical image analysis across existing modalities such as X-rays, MRI, CT and ultrasound. The aim of this study was to review the current state-of-the-art methods, highlight proven applications in precision diagnostics and prognosis, analyze key challenges, and identify promising future directions for this rapidly advancing field. Convolutional Neural Networks (CNNs) enable transformative improvements in tasks such as classification, segmentation and detection compared with prior approaches. A wide range of real-world applications across diagnostic, interventional, prognostic and pharmaceutical settings has been presented. The salient challenges concerning model interpretability, multimodal integration, algorithmic robustness, workflow integration, and responsible ethical deployment were discussed. This synergy between medical imaging and artificial intelligence continues to unlock abundant clinically relevant insights latent in images and transforms datadriven precision medicine for patient benefit.

Keywords

Deep Learning, Convolutional Neural Networks (CNNs), Medical Image Analysis, Computer-Aided Diagnosis, Model Interpretation.

How to Cite this Article?

Eswaran, U., and Eswaran, V. (2023). Unlocking Clinical Insights from Medical Images using Deep Learning. i-manager’s Journal on Artificial Intelligence & Machine Learning, 1(2), 37-48. https://doi.org/10.26634/jaim.1.2.20044

References

[1]. AlBadawy, E. A., Saha, A., & Mazurowski, M. A. (2018). Deep learning for segmentation of brain tumors: Impact of cross institutional training and testing. Medical Physics, 45(3), 1150-1158.

[2]. Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M.K. (2018). Medical image analysis using convolutional neural networks: A review. Journal of Medical Systems, 42, 1-13.

[3]. Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher,J. J., Peng, L., ... & Shetty, S. (2019). End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine, 25(6), 954-961.

[4]. Badgeley, M. A., Zech, J. R., Oakden-Rayner, L., Glicksberg, B. S., Liu, M., Gale, W., ... & Dudley, J. T. (2019). Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digital Medicine, 2(1), 31.

[5]. Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495.

[6]. Cabitza, F., Rasoini, R., & Gensini, G. F. (2017). Unintended consequences of machine learning in medicine. JAMA, 318(6), 517-518.

[7]. Chan, H. P., Sahiner, B., Helvie, M. A., Petrick, N., Roubidoux, M. A.,Wilson, T. E., ... & Sanjay-Gopal, S. (1999). Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study. Radiology, 212(3), 817-827.

[8]. Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing machine learning in health careAddressing ethical challenges. The New England Journal of Medicine, 378(11), 981.

[9]. Chartrand, G., Cheng, P. M., Vorontsov, E., Drozdzal, M., Turcotte, S.,Pal, C. J., ... & Tang, A. (2017). Deep learning: a primer for radiologists. Radiographics, 37(7), 2113-2131.

[10]. Chen, D. R., Chang, R. F., Kuo, W. J., Chen, M. C., & Huang, Y. L. (2002). Diagnosis of breast tumors with sonographic texture analysis using wavelet transform and neural networks. Ultrasound in Medicine & Biology, 28 (10), 1301-1310.

[11]. Cheplygina, V., de Bruijne, M., & Pluim, J. P. (2019). Not-so-supervised: a survey of semi-supervised, multiinstance, and transfer learning in medical image analysis. Medical Image Analysis, 54, 280-296.

[12]. Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A.,Do, B. T., Way, G. P., ... & Greene, C. S. (2018). Opportunities and obstacles for deep learning in biology and medicine. Journal of the Royal Society Interface, 15(141), 20170387.

[13]. De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S.,... & Ronneberger, O. (2018). Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine, 24(9), 1342-1350.

[14]. Doi, K. (2005). Current status and future potential of computer-aided diagnosis in medical imaging. The British Journal of Radiology, 78(suppl_1), s3-s19.

[15]. Dreyer, K. J., & Geis, J. R. (2017). When machines think: Radiology's next frontier. Radiology, 285(3), 713- 718.

[16]. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H.M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115-118.

[17]. Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019). Adversarial attacks on medical machine learning. Science, 363(6433), 1287- 1289.

[18]. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018). GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 321, 321-331.

[19]. Giger, M. L. (2018). Machine learning in medical imaging. Journal of the American College of Radiology, 15(3), 512-520.

[20]. Giger, M. L., Chan, H. P., & Boone, J. (2008). Anniversary paper: history and status of CAD and quantitative image analysis: the role of medical physics and AAPM. Medical Physics, 35(12), 5799-5820.

[21]. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A.,...& Webster, D. R. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama, 316(22), 2402-2410.

[22]. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770-778).

[23]. Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017). What do we need to build explainable AI systems for the medical domain?. arXiv preprint arXiv:1712. 09923.

[24]. Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. (2018). Artificial intelligence in radiology. Nature Reviews Cancer, 18(8), 500-510.

[25]. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4700-4708).

[26]. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4700-4708).

[27]. Jha, S., & Topol, E. J. (2016). Adapting to artificial intelligence: Radiologists and pathologists as information specialists. JAMA, 316(22), 2353-2354.

[28]. Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S.,... & Wang, Y. (2017). Artificial intelligence in healthcare: past, present and future. Stroke and Vascular Neurology, 2(4).

[29]. Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17, 1-9.

[30]. Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C., Liang, H.,Baxter, S. L., ... & Zhang, K. (2018). Identifying medical diagnoses and treatable diseases by imagebased deep learning. Cell, 172(5), 1122-1131.

[31]. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.

[32]. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.

[33]. LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, 3361(10), 1995.

[34]. Li, X., Yu, L., Chen, H., Fu, C. W., & Heng, P. A. (2018). Semi- super vised skin lesion segmentation via transformation consistent self-ensembling model. arXiv preprint arXiv:1808.03887.

[35]. Litjens, G., Ciompi, F., Wolterink, J. M., de Vos, B. D., Leiner, T.,Teuwen, J., & Išgum, I. (2019). State-of-the-art deep learning in cardiovascular image analysis. JACC: Cardiovascular Imaging, 12(8 Part 1), 1549-1565.

[36]. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M.,... & Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42, 60-88.

[37]. Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M.,Scholz, P., ... & Kopp-Schneider, A. (2018). Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications, 9(1), 5217.

[38]. Marcus, D. S., Fotenos, A. F., Csernansky, J. G., Morris, J. C., & Buckner, R. L. (2010). Open access series of imaging studies: longitudinal MRI data in nondemented and demented older adults. Journal of Cognitive Neuroscience, 22(12), 2677-2684.

[39]. Milletari, F., Navab, N., & Ahmadi, S. A. (2016, October). V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision (3DV) (pp. 565- 571). IEEE.

[40]. Nagpal, K., Foote, D., Liu, Y., Chen, P. H. C., Wulczyn, E., Tan, F.,... & Stumpe, M. C. (2019). Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digital Medicine, 2(1), 48.

[41]. Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Ré, C. (2020, April). Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In Proceedings of the ACM Conference on Health, Inference, and Learning (pp. 151-159).

[42]. Park, S. H., & Han, K. (2018). Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology, 286(3), 800-809.

[43]. Pesapane, F., Codari, M., & Sardanelli, F. (2018). Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. European Radiology Experimental, 2, 1-10.

[44]. Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T.,... & Ng, A. Y. (2017). Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225.

[45]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5- 9, 2015, Proceedings, Part III 18 (pp. 234-241). Springer International Publishing.

[46]. Sendak, M. P., Balu, S., & Schulman, K. A. (2017). Barriers to achieving economies of scale in analysis of EHR data. Applied Clinical Informatics, 8(03), 826-831.

[47]. Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., ... & Bakas, S. (2020). Federated learning in medicine: Facilitating multi-institutional collaborations without sharing patient data. Scientific Reports, 10(1), 12598.

[48]. Shen, D., Wu, G., & Suk, H. I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221-248.

[49]. Shen, D., Wu, G., & Suk, H. I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221-248.

[50]. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

[51]. Summers, R. M. (2003). Road maps for advancement of radiologic computer-aided detection in the 21st century. Radiology, 229(1), 11-13.

[52]. Suzuki, K. (2017). Overview of deep learning in medical imaging. Radiological Physics and Technology, 10(3), 257-273.

[53]. Suzuki, K., Horiba, I., & Sugie, N. (2003). Neural edge enhancer for supervised edge enhancement from noisy images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1582-1596.

[54]. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D.,...& Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-9).

[55]. Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016). Convolutional neural networks for medical image analysis: Full training or fine tuning?. IEEE Transactions on Medical Imaging, 35(5), 1299-1312.

[56]. Thrall, J. H., Li, X., Li, Q., Cruz, C., Do, S., Dreyer, K., & Brink, J. (2018). Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. Journal of the American College of Radiology, 15(3), 504-508.

[57]. Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine, 25(1), 44-56.

[58]. US Food and Drug Administration. (2020). Artificial Intelligence and Machine Learning in Software as a Medical Device.

[59]. Wang, S., & Summers, R. M. (2012). Machine learning and radiology. Medical Image Analysis, 16(5), 933-951.

[60]. Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F.,... & Goldenberg, A. (2019). Do no harm: a roadmap for responsible machine learning for health care. Nature Medicine, 25(9), 1337-1340.

[61]. Xu, Y., Hosny, A., Zeleznik, R., Parmar, C., Coroller, T., Franco, I., ... & Aerts, H. J. (2019). Deep learning predicts lung cancer treatment response from serial medical imaging. Clinical Cancer Research, 25(11), 3266-3275.

[62]. Yi, X., Walia, E., & Babyn, P. (2019). Generative adversarial network in medical imaging: A review. Medical Image Analysis, 58, 101552.

[63]. Yip, S. S., & Aerts, H. J. (2016). Applications and limitations of radiomics. Physics in Medicine & Biology, 61(13), R150.

[64]. Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018). Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A crosssectional study. Plos Medicine, 15(11), e1002683.

[65]. Zhuang, X., Li, Y., Hu, Y., Ma, K., Yang, Y., & Zheng, Y. (2019). Self-supervised feature learning for 3D medical images by playing a rubik's cube. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, (pp. 420-428). Springer International Publishing.