By DKU Staff
A Duke Kunshan professor-student team has developed technology to detect elevator accidents, which has led to several patent applications and undergraduates authoring research papers that were presented at an international conference.
Titled “Multimodal abnormal event detection system and method in elevators,” the study harnessed artificial intelligence audio and video data analysis technology to provide early warnings about unusual activity inside an elevator, such as the doors being forced or someone falling to the floor. The developers hope it can lead to timely intervention to reduce accidents and detect malfunctions.
“The purpose of the collaborative research was to transform the old approach of identifying causes of failures afterwards to a new method of early intervention based on real-time warnings,” said Ming Li, associate professor of electrical and computer engineering, who led the study.
Li worked on the project in collaboration with Finnish elevator manufacturer Kone, research assistant Ming Cheng, and DKU juniors Xinmeng Chen, Xuchen Gong, Ran Ju, Huangrui Chu and Yechen Wang, with financial support from Kunshan municipal government.
Kone provided field experiment settings for the research, which commenced in July 2020, allowing the team to test their research in its elevators.
Example of applicable scenes
Li and Cheng oversaw the student team, who authored three research papers related to the study that were presented at the 2021 International Conference on Multimodal Interaction.
Kone and the DKU team also submitted three collaborative patent applications, for a system to detect abnormal behavior in elevators based on video surveillance data, a method for detecting calls for help based on audio and video data and a contactless elevator interaction system.
Elevators are normally monitored using manually watched surveillance cameras. The research team’s approach harnessed AI audio and video data analysis technology to take over that job and provide warnings of abnormal events such as fighting, someone forcing the door, or a person falling to the floor.
The prototype system under testing
“The new technology helps monitor the situation inside elevators and provides early warnings in real time, facilitating timely detection of abnormal events and early intervention, thus reducing the occurrence of failures or accidents,” said Li.
As well developing a potential advance in using technology to monitor elevators, the research project also provided the students involved with valuable experience.
“I walked through the whole process of a research project from designing a plan, to solving the research problem and writing up our findings for publication. It was an invaluable experience, which not only helped me to learn research methods and expand my vision in the field, but also reshaped my plans for further learning,” said Gong, who studies data science.
The study enabled students to “participate deeply” in a research project, added Chen, who also studies data science.
The project was a part of Duke Kunshan’s focus on experiential learning, which encourages students to seek creative alignments between their academic study and practical work.
Dean of Graduate Studies and Associate Dean for Research Xin Li said, “Our outstanding researchers are committed to collaborating with leading universities, companies and organizations around the world to solve real-world problems and serve society with knowledge.”
