Intro  Projects  Publication  Posts  News 

Application of Tele-Robotic Technology for Auxiliary Diagnosis and Treatment of COVID−19 in Isolation Ward-(ZJU 2020.02-now)

This project conducts the teleoperation of the dual-arm robot YuMi base the multi-node inertial sensing motion capture technology, which can mapping the diagnosis and treatment operation behavior of the doctors and caregivers. This project aims to minimize the need for medical staff into the isolation ward, and reducing the cross-infection between the COVID-19 patients and the doctors or the caregivers. At present, the hardware construction and hardware debugging of the assembly platform have been completed, and the preliminary clinical verification has been carried out. In view of the related reasons, the related videos and related codes will be released at the right time.

Human-robot Collaborative Assembly Based on Target Recognition Using Kinect-(ZJU 2019.10-now)

In this project, I use the Depth camera kinectV2 to obtain the 2D position coordinate information and rotation information of the target object in the assembly workspace (at present, the rotation information of the object in 2D plane has only one variable - Euler angle around Z axis). Then the dual-arm collaborative robot YuMi grab the object using the position and orientation data after coordinate transformation. After grabbing, YuMi place the object to the appropriate position pre-setted for assembly process, while the human in the same workspace also do other details work such as assembly screws at the same time. The project is aim to improve the working efficiency and give full play to the functional advantages of human and robot. At present, the hardware of the assembly platform has been built and debugged. The following is the video demonstration of this design work.

Design and Security Interaction of the Collaborative Mobile Platform Based on YuMi Robot-(ZJU 2018.02-2018.06)

This project is the graduation design for my bachelor degree. Based on ABB’s dual arm cooperative robot–YuMi, this paper designs a cooperative mobile platform for helping the elderly and disabled people and medical care. This design includes the mechanical structure design, electrical system design and the programming of the software program. The 3D modeling and physical construction building have been completed, and the simulation and debugging of the software program have been carried out. This design is safe, efficient, convenient and practical. Below is the video demo of this project.

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot-(ZJU 2018.05-2018.12)

This project proposed an innovative hand motion trajectory (HMT) gesture recognition system based on background velocity features. Here, a new wearable wrist-worn camera prototype for gesture’s video collection was designed, and a new method for the segmentation of continuous gestures was shown. Meanwhile, a nursing-care assistant robot prototype was designed for assisting the elderly, which is capable of carrying the elderly with omnidirectional motion and grabbing the specified object at home. In order to evaluate the performance of the gesture recognition system, 10 special gestures were defined as the move commands for interaction with the robot. The hardware part of wearable wrist camera is completed independently by Chen Feiyu, and the software program is modified based on Chen Feiyu’s code foundation, and then connects with the interface of nursing-care robot in my graduation design project to realize gesture control. In this project, ten kinds of gesture commands are defined to control the movement of the robot. Below is the video demo of this project.

Application of Face Recognition and Voice Control in Remote Smart Home System-(CUMT 2017.04-2018.03)

This project is my 2nd SRTP project in CUMT. In this work, a smart home system is designed, in wihch the face recognition and vioce control tech. are applied. A patent of this work is granted and the protypo of this work won the 3rd prize in Virtual Instrument Contest of Jiangsu Province. Main contribution description: (1)conducted a face recognition system based on the Face++ & LabVIEW, and implemented this tech into NI myRIO for application；（2）conducted a remote voice control system using Node.js and Raspberry Pi，and realize the voice control using Siri based on Apple’s HomeKit；（3）conducted a tele-monitoring system using MJPG-streamer & LabVIEW，which can realize the realtime monitoring of the home environment；（4）conducted a home environmental parameter acquisition system which can realize the monitoring of the temperature, humidity, strength of illumination and the air quality in home；（5）remote home appliance control system：Through the http transmission protocol, the combination of embedded hardware platform and labview realizes the remote real-time control of home appliances.

R&D of a portable friction coefficient tester-(CUMT 2016.04-2017.04)

This project is my first SRTP project in CUMT. In this work, a prototype was built by the self-designed mechanical structure and data processing program based on LabVIEW, which has been tested successfully. I am in charge of designing the hardware circuit and programming the whole LabVIEW codes. Compared with most of the current brake friction test bench, it has the outstanding advantages and application significance, such as convenient installation, strong portability, greatly reduced cost, small and portable volume, strong field application ability, easy data acquisition.