Journal of Bionic Engineering ›› 2021, Vol. 18 ›› Issue (5): 1045-1058.doi: 10.1007/s42235-021-00074-z

• • 上一篇    下一篇

Hand Gesture-based Artifi cial Neural Network Trained Hybrid Human–machine Interface System to Navigate a Powered Wheelchair

Ashley Stroh 1,Jaydip Desai1   

  1. Department of Biomedical Engineering , Wichita State 
    University , Wichita , KS   67260 , USA
  • 收稿日期:2021-01-22 修回日期:2021-08-16 接受日期:2021-08-26 出版日期:2021-09-10 发布日期:2021-12-03
  • 通讯作者: Jaydip Desai E-mail:jaydip.desai@wichita.edu
  • 作者简介:Ashley Stroh 1,Jaydip Desai1

Hand Gesture-based Artifi cial Neural Network Trained Hybrid Human–machine Interface System to Navigate a Powered Wheelchair

Ashley Stroh 1,Jaydip Desai1   

  1. Department of Biomedical Engineering , Wichita State 
    University , Wichita , KS   67260 , USA
  • Received:2021-01-22 Revised:2021-08-16 Accepted:2021-08-26 Online:2021-09-10 Published:2021-12-03
  • Contact: Jaydip Desai E-mail:jaydip.desai@wichita.edu
  • About author:Ashley Stroh 1,Jaydip Desai1

摘要: Individuals with cerebral palsy and muscular dystrophy often lack fi ne motor control of their fi ngers which makes it diffi cult 
to control traditional powered wheelchairs using a joystick. Studies have shown the use of surface electromyography to steer 
powered wheelchairs or automobiles either through simulations or gaming controllers. However, these studies signifi cantly 
lack issues with real world scenarios such as user’s safety, real-time control, and effi ciency of the controller mechanism. 
The purpose of this study was to design, evaluate, and implement a hybrid human–machine interface system for a powered 
wheelchair that can detect human intent based on artifi cial neural network trained hand gesture recognition and navigate 
a powered wheelchair without colliding with objects around the path. Scaled Conjugate Gradient (SCG), Bayesian Regularization (BR), and Levenberg Marquart (LM) supervised artifi cial neural networks were trained in offl ine testing on eight 
participants without disability followed by online testing using the classifi er with highest accuracy. Bayesian Regularization 
architecture showed highest accuracy at 98.4% across all participants and hidden layers. All participants successfully completed the path in an average of 5 min and 50 s, touching an average of 22.1% of the obstacles. The proposed hybrid system 
can be implemented to assist people with neuromuscular disabilities in near future.


关键词: Electromyography, Artifi cial neural network, Hybrid control, Powered wheelchair;Assistive technology, Hand gesture recognition

Abstract: Individuals with cerebral palsy and muscular dystrophy often lack fi ne motor control of their fi ngers which makes it diffi cult 
to control traditional powered wheelchairs using a joystick. Studies have shown the use of surface electromyography to steer 
powered wheelchairs or automobiles either through simulations or gaming controllers. However, these studies signifi cantly 
lack issues with real world scenarios such as user’s safety, real-time control, and effi ciency of the controller mechanism. 
The purpose of this study was to design, evaluate, and implement a hybrid human–machine interface system for a powered 
wheelchair that can detect human intent based on artifi cial neural network trained hand gesture recognition and navigate 
a powered wheelchair without colliding with objects around the path. Scaled Conjugate Gradient (SCG), Bayesian Regularization (BR), and Levenberg Marquart (LM) supervised artifi cial neural networks were trained in offl ine testing on eight 
participants without disability followed by online testing using the classifi er with highest accuracy. Bayesian Regularization 
architecture showed highest accuracy at 98.4% across all participants and hidden layers. All participants successfully completed the path in an average of 5 min and 50 s, touching an average of 22.1% of the obstacles. The proposed hybrid system 
can be implemented to assist people with neuromuscular disabilities in near future.


Key words: Electromyography, Artifi cial neural network, Hybrid control, Powered wheelchair;Assistive technology, Hand gesture recognition