3D Visualization for Lung Surface Images of Covid-19 Patients based on U-Net CNN Segmentation

  • FX Ferdinandus Department of Electrical Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
  • Esther Irawati Setiawan Department of Informatics, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Indonesia
  • Eko Mulyanto Yuniarno Department of Computer Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
  • Mauridhi Hery Purnomo University Center of Excellence on Artificial Intelligence for Healthcare and Society (UCE AIHeS) Surabaya, Indonesia
DOI: https://doi.org/10.24003/emitter.v10i2.709
Keywords: Covid-19, CT-scan, Deep Learning, Lung segmentation, U-Net

Abstract

The Covid-19 infection challenges medical staff to make rapid diagnoses of patients. In just a few days, the Covid-19 virus infection could affect the performance of the lungs. On the other hand, semantic segmentation using the Convolutional Neural Network (CNN) on Lung CT-scan images had attracted the attention of researchers for several years, even before the Covid-19 pandemic. Ground Glass Opacity (GGO), in the form of white patches caused by Covid-19 infection, is detected inside the patient’s lung area and occasionally at the edge of the lung, but no research has specifically paid attention to the edges of the lungs. This study proposes to display a 3D visualization of the lung surface of Covid-19 patients based on CT-scan image segmentation using U-Net architecture with a training dataset from typical lung images. Then the resulting CNN model is used to segment the lungs of Covid-19 patients. The segmentation results are selected as some slices to be reconstructed into a 3D lung shape and displayed in 3D animation. Visualizing the results of this segmentation can help medical staff diagnose the lungs of Covid-19 patients, especially on the surface of the lungs of patients with GGO at the edges. From the lung segmentation experiment results on ten patients in the Zenodo dataset, we have a Mean-IoU score = of 76.86%, while the visualization results show that 7 out of 10 patients (70%) have eroded lung surfaces. It can be seen clearly through 3D visualization.

Downloads

Download data is not yet available.

References

Nifty format, [Online]. Available: http://nifty.nimh.nih.gov.

W. Zhao, W. Jiang, and X. Qiu, Deep learning for COVID-19 detection based on CT images, Sci Rep, vol. 11, no. 1, Dec. 2021, doi: 10.1038/s41598-021-93832-2. DOI: https://doi.org/10.1038/s41598-021-93832-2

Jeremy Jordan, An Overview of Semantic Image Segmentation, https://www.jeremyjordan.me/semantic-segmentation, May 21, 2018.

X. Liu, Z. Deng, and Y. Yang, Recent progress in semantic image segmentation, Artif Intell Rev, vol. 52, no. 2, pp. 1089–1106, Aug. 2019, doi: 10.1007/s10462-018-9641-3. DOI: https://doi.org/10.1007/s10462-018-9641-3

I. Ulku and E. Akagunduz, A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images, Dec. 2019, doi: 10.1080/08839514.2022.2032924. DOI: https://doi.org/10.1080/08839514.2022.2032924

O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015, vol. 9351, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28. DOI: https://doi.org/10.1007/978-3-319-24574-4_28

F. Ferdinandus, E. Mulyanto Yuniarno, I. Ketut Eddy Purnama, and M. Hery Purnomo, Covid-19 Lung Segmentation using U-Net CNN based on Computed Tomography Image, in 2022 IEEE 9th International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), June 2022. DOI: https://doi.org/10.1109/CIVEMSA53371.2022.9853695

J. F. W. Chan et al., A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster, The Lancet, vol. 395, no. 10223, pp. 514–523, Feb. 2020, doi: 10.1016/S0140-6736(20)30154-9. DOI: https://doi.org/10.1016/S0140-6736(20)30154-9

F. Shi et al., Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19, IEEE Reviews in Biomedical Engineering, vol. 14. Institute of Electrical and Electronics Engineers Inc., pp. 4–15, 2021. doi: 10.1109/RBME.2020.2987975. DOI: https://doi.org/10.1109/RBME.2020.2987975

D. P. Fan et al., Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images, IEEE Trans Med Imaging, vol. 39, no. 8, pp. 2626–2637, Aug. 2020, doi: 10.1109/TMI.2020.2996645. DOI: https://doi.org/10.1109/TMI.2020.2996645

T. Karlita, E. M. Yuniarno, I. K. E. Purnama, and M. H. Purnomo, Detection of COVID-19 on Chest X-Ray Images using Inverted Residuals Structure-Based Convolutional Neural Networks, in 2020 3rd International Conference on Information and Communications Technology, ICOIACT 2020, Nov. 2020, pp. 371–376. doi: 10.1109/ICOIACT50329.2020.9332153. DOI: https://doi.org/10.1109/ICOIACT50329.2020.9332153

A. Kalinovsky and V. Kovalev, Lung Image Segmentation Using Deep Learning Methods and Convolutional Neural Networks, 2016. [Online]. Available: http://imlab.grid.by/

S. G. C, V. S. T, R. v Gowda, P. R. Udupa, and S. Reddy, A Machine learning Classification approach for detection of Covid 19 using CT images, EMITTER International Journal of Engineering Technology, vol. 10, no. 1, pp. 183–194, 2022, doi: 10.24003/emitter.v10i1.672. DOI: https://doi.org/10.24003/emitter.v10i1.672

B. Ait Skourt, A. el Hassani, and A. Majda, Lung CT image segmentation using deep neural networks, in Procedia Computer Science, 2018, vol. 127, pp. 109–113. doi: 10.1016/j.procs.2018.01.104. DOI: https://doi.org/10.1016/j.procs.2018.01.104

T. Tolxdorff, T. M. Deserno Heinz Handels, and A. H. Maier Klaus Maier-Hein Christoph Palm Hrsg, Bildverarbeitung für die Medizin 2020, 2020. [Online]. Available: http://www.springer.com/series/2872 DOI: https://doi.org/10.1007/978-3-658-29267-6

Adhiparasakthi Engineering College. Department of Electronics and Communication Engineering, Institute of Electrical and Electronics Engineers. Madras Section, and Institute of Electrical and Electronics Engineers, Proceedings of the 2018 IEEE International Conference on Communication and Signal Processing (ICCSP) : 3rd - 5th April 2018, Melmaruvathur, India. 2018.

H. Shaziya and K. Shyamala, Pulmonary CT Images Segmentation using CNN and UNet Models of Deep Learning, in 2020 IEEE Pune Section International Conference, PuneCon 2020, Dec. 2020, pp. 195–201. doi: 10.1109/PuneCon50868.2020.9362463. DOI: https://doi.org/10.1109/PuneCon50868.2020.9362463

K. K. Bressem, S. M. Niehues, B. Hamm, M. R. Makowski, J. L. Vahldiek, and L. C. Adams, 3D U-Net for segmentation of COVID-19 associated pulmonary infiltrates using transfer learning: State-of-the-art results on affordable hardware, Jan. 2021, [Online]. Available: http://arxiv.org/abs/2101.09976 DOI: https://doi.org/10.21203/rs.3.rs-259319/v1

E. Martínez Chamorro, A. Díez Tascón, L. Ibáñez Sanz, S. Ossaba Vélez, and S. Borruel Nacenta, Radiologic diagnosis of patients with COVID-19, Radiología (English Edition), vol. 63, no. 1, pp. 56–73, Jan. 2021, doi: 10.1016/j.rxeng.2020.11.001. DOI: https://doi.org/10.1016/j.rxeng.2020.11.001

D. Cozzi et al., Ground-glass opacity (GGO): a review of the differential diagnosis in the era of COVID-19, Japanese Journal of Radiology, vol. 39, no. 8. Springer Japan, pp. 721–732, Aug. 01, 2021. doi: 10.1007/s11604-021-01120-w. DOI: https://doi.org/10.1007/s11604-021-01120-w

E. P. I. K. A. D. N. D. Athanasios Voulodimos, Deep learning models for COVID-19 infected area segmentation in CT images, vol. 6, 2021, doi: 10.1101/2020.05.08.20094664. DOI: https://doi.org/10.1101/2020.05.08.20094664

T. Zhou, S. Canu, and S. Ruan, Automatic COVID-19 CT segmentation using U-Net integrated spatial and channel attention mechanism, Int J Imaging Syst Technol, vol. 31, no. 1, pp. 16–27, Mar. 2021, doi: 10.1002/ima.22527. DOI: https://doi.org/10.1002/ima.22527

K Scott Mader, Kaggle Lung Dataset, 2017.

Ma Jun, et all, Zenodo Lung Covid-19 Dataset, https://zenodo.org/record/3757476, Apr. 20, 2020.

Y. H. Nai et al., Comparison of metrics for the evaluation of medical segmentations using prostate MRI dataset, Comput Biol Med, vol. 134, Jul. 2021, doi: 10.1016/j.compbiomed.2021.104497. DOI: https://doi.org/10.1016/j.compbiomed.2021.104497

Published
2022-12-28
How to Cite
FX Ferdinandus, Esther Irawati Setiawan, Eko Mulyanto Yuniarno, & Hery Purnomo, M. (2022). 3D Visualization for Lung Surface Images of Covid-19 Patients based on U-Net CNN Segmentation. EMITTER International Journal of Engineering Technology, 10(2), 320-337. https://doi.org/10.24003/emitter.v10i2.709
Issue
Vol 10 No 2 (2022)
Section
Articles

Copyright (c) 2022 EMITTER International Journal of Engineering Technology

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .

The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.

Retained Rights/Terms and Conditions

  1. Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
  2. Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
  3. Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
  4. Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.

Plagiarism Check

To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.

Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.