Journal of Bionic Engineering ›› 2022, Vol. 19 ›› Issue (6): 1816-1829.doi: 10.1007/s42235-022-00234-9

• • 上一篇    下一篇

Osteoporotic Vertebral Fracture Classification in X-rays Based on a Multi-modal Semantic Consistency Network

Yuzhao Wang1,2; Tian Bai1,2; Tong Li3; Lan Huang1,2   

  1. 1 College of Computer Science and Technology, Jilin University, Changchun 130012, China  2 Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China  3 Department of Orthopedics, The Second Hospital of Jilin University, Changchun 130012, China
  • 收稿日期:2021-12-28 修回日期:2022-06-15 接受日期:2022-07-03 出版日期:2022-11-10 发布日期:2022-11-10
  • 通讯作者: Yuzhao Wang, Tian Bai, Tong Li & Lan Huang E-mail:huanglan@jlu.edu.cn; yuzhaow20@mails.jlu.edu.cn; baitian@jlu.edu.cn; tli19@mails.jlu.edu.cn
  • 作者简介:Yuzhao Wang1,2; Tian Bai1,2; Tong Li3; Lan Huang1,2

Osteoporotic Vertebral Fracture Classification in X-rays Based on a Multi-modal Semantic Consistency Network

Yuzhao Wang1,2; Tian Bai1,2; Tong Li3; Lan Huang1,2   

  1. 1 College of Computer Science and Technology, Jilin University, Changchun 130012, China  2 Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China  3 Department of Orthopedics, The Second Hospital of Jilin University, Changchun 130012, China
  • Received:2021-12-28 Revised:2022-06-15 Accepted:2022-07-03 Online:2022-11-10 Published:2022-11-10
  • Contact: Yuzhao Wang, Tian Bai, Tong Li & Lan Huang E-mail:huanglan@jlu.edu.cn; yuzhaow20@mails.jlu.edu.cn; baitian@jlu.edu.cn; tli19@mails.jlu.edu.cn
  • About author:Yuzhao Wang1,2; Tian Bai1,2; Tong Li3; Lan Huang1,2

摘要: Osteoporotic Vertebral Fracture (OVFs) is a common lumbar spine disorder that severely affects the health of patients. With a clear bone blocks boundary, CT images have gained obvious advantages in OVFs diagnosis. Compared with CT images, X-rays are faster and more inexpensive but often leads to misdiagnosis and miss-diagnosis because of the overlapping shadows. Considering how to transfer CT imaging advantages to achieve OVFs classification in X-rays is meaningful. For this purpose, we propose a multi-modal semantic consistency network which could do well X-ray OVFs classification by transferring CT semantic consistency features. Different from existing methods, we introduce a feature-level mix-up module to get the domain soft labels which helps the network reduce the domain offsets between CT and X-ray. In the meanwhile, the network uses a self-rotation pretext task on both CT and X-ray domains to enhance learning the high-level semantic invariant features. We employ five evaluation metrics to compare the proposed method with the state-of-the-art methods. The final results show that our method improves the best value of AUC from 86.32 to 92.16%. The results indicate that multi-modal semantic consistency method could use CT imaging features to improve osteoporotic vertebral fracture classification in X-rays effectively.

关键词: Osteoporotic vertebral fracture classification , · Cross-modality , · Unsupervised domain adaptation , · Transfer learning , · Convolutional neural network , · Computer-aided diagnosis

Abstract: Osteoporotic Vertebral Fracture (OVFs) is a common lumbar spine disorder that severely affects the health of patients. With a clear bone blocks boundary, CT images have gained obvious advantages in OVFs diagnosis. Compared with CT images, X-rays are faster and more inexpensive but often leads to misdiagnosis and miss-diagnosis because of the overlapping shadows. Considering how to transfer CT imaging advantages to achieve OVFs classification in X-rays is meaningful. For this purpose, we propose a multi-modal semantic consistency network which could do well X-ray OVFs classification by transferring CT semantic consistency features. Different from existing methods, we introduce a feature-level mix-up module to get the domain soft labels which helps the network reduce the domain offsets between CT and X-ray. In the meanwhile, the network uses a self-rotation pretext task on both CT and X-ray domains to enhance learning the high-level semantic invariant features. We employ five evaluation metrics to compare the proposed method with the state-of-the-art methods. The final results show that our method improves the best value of AUC from 86.32 to 92.16%. The results indicate that multi-modal semantic consistency method could use CT imaging features to improve osteoporotic vertebral fracture classification in X-rays effectively.

Key words: Osteoporotic vertebral fracture classification , · Cross-modality , · Unsupervised domain adaptation , · Transfer learning , · Convolutional neural network , · Computer-aided diagnosis