Parkinson’s Rehabilitation Efficacy Estimation System
Makerizon – Parkinson project portfolio showcase.
Preface#
The full name of this project is an OpenHarmony-based multi-data fusion monitoring system for Parkinson’s disease symptoms. It is one of the projects I participated in and took responsibility for when I was an undergraduate student in Teacher Li Yihao’s Makerizon Studio ↗. Due to confidentiality requirements in the past, the showcase materials for this project could not be released publicly. Now, as the project evolved and the first cohort of older members (including us) gradually left, the project has also slowly come to an end.
I kept thinking about what kind of form I should use to present this project, so that it would neither be showy nor be buried. It carries not only our efforts, but also the friendship among the team members, and our college days that still feel quite beautiful when recalled. My feelings toward it are complicated; perhaps from the very beginning I never treated it as merely a check-in task or just a “job”.
Finally, little by little, I think I found a way to let it be recorded forever without drawing too much attention: documenting it on my blog. Maybe that emotional attachment is just my own projection, because I am someone who likes living in memories, while also having a liberal yet somewhat utilitarian mindset. I hope this project can achieve me once more, as a highlight I can present.
Turning it into a portfolio piece (a showcase post), there has always been a voice reminding me that if I don’t record it, it will slip away. But I still procrastinated until now to truly write about it and explain it. So here we go, with a writing style that may feel clumsy after not reading seriously for a long time, to present this project that I both love and hate.
Project Overview#
This project is an OpenHarmony-based multi-data fusion monitoring system for Parkinson’s disease symptoms that combines big data, the Internet of Things, and modern healthcare on Huawei Cloud. The system helps mild Parkinson’s patients collect data at home, such as hand tremor, finger bending, and EMG signals, to detect worsening trends in time, enabling earlier diagnosis and earlier treatment. It improves diagnostic efficiency, reduces the evaluation workload for medical institutions, and alleviates patients’ financial and psychological burdens.
Based on Huawei Cloud IoT edge-side rule engine technology, the project configures linkage rules via IoTDA and delivers SQL-statement-based rules from the cloud to devices with one click. Through OpenHarmony’s distributed soft bus, devices can perform inter-device linkage directly on the edge side, reducing dependence on network quality and improving overall linkage efficiency. In addition, cloud operators such as feature-extraction and filtering operators are delivered to sensor devices with one click, so that invalid data can be filtered and masked on edge devices. By accessing devices to Huawei Cloud IoT services, the project not only enables a simplified “device model to cloud” workflow, but also gives devices the ability to “think” proactively, greatly reducing the development workload of the Parkinson project.
Background and Pain Points#
China has more than 3 million Parkinson’s patients, and the prevalence among people over 65 is as high as 1.7%. Parkinson’s disease has four core symptoms: tremor, rigidity, slowness, and falls. Resting tremor, muscle rigidity, bradykinesia, and gait disorders bring tremendous suffering to patients and their families. So our team wondered whether we could apply what we had learned to help doctors and patients with assessment.
There are currently very few devices on the market for Parkinson’s detection, and their functions are single-purpose. They cannot integrate with healthcare to enable doctors to monitor patients’ daily motor status. Existing assessment solutions focus on evaluating motor impairment in Parkinson’s disease. Such assessments are based on medical history, patient self-reports, and neurologists’ clinical examinations, and use tools such as the Unified Parkinson’s Disease Rating Scale (UPDRS) and the Modified Bradykinesia Rating Scale. Although these methods are simple to use and can comprehensively reflect changes in condition, they also have shortcomings. First, accuracy is poor, because physician evaluations depend on subjective experience. Second, timeliness is poor: clinical assessments only capture the patient’s current state, and cannot reflect daily states.
There are also computer-vision-based methods, which mainly use cameras and optical motion capture systems to record patients’ movements and compute kinematic parameters. While these can be highly accurate, they are mainly used in clinical trials. Another approach is to collect patients’ motion signals using inertial sensors and then analyze and evaluate symptoms. However, this can suffer from single-source data and can lead to misjudgments.
Parkinson’s disease detection products on the market have the following pain points:
- Complex testing process
Existing Parkinson’s devices on the market suffer from issues such as low sensitivity and complicated testing procedures, and the testing cycle for patients is long.
- Early symptoms are not obvious
In society, many Parkinson’s patients do not show obvious symptoms in the early stage, and there are no corresponding daily testing devices.
- Difficult offline data collection for patients
With the improvement of internet healthcare systems, it is increasingly urgent for doctors to manage and analyze patient data. However, there is a lack of home medical devices that match internet healthcare, and patient data cannot be fed back to doctors in time.
Development Challenges#
Challenges our team encountered during development:
- Private protocol development
We developed a private protocol ourselves, and it needed to be deployed on both the hardware side and the backend. The development cycle reached 3.5 person-months, which is not conducive to rapid product development.
- Large and redundant data volume
In current Parkinson scenarios, each upload can reach 1000 data entries. It is urgent to deliver some filtering and dimensionality-reduction operators to the edge side to reduce redundant and useless data.
- Messy scenario data
For data acquisition in Parkinson scenarios, there was no unified device model; it involved determining more than 30 device attributes, data types, and data quantities, making later porting difficult.
- Multi-device data linkage
For information transmission and linkage among multiple Parkinson assessment devices, there are issues such as complex transmission processes and untimely updates.
Based on the above, our team combined our technical capabilities and used Huawei Cloud IoT edge-side rule engine to improve device linkage efficiency through cloud-side rule delivery and fast edge-side execution. We proposed a wearable Parkinson’s multi-data fusion monitoring system built with OpenHarmony, the Internet of Things, data cross-fusion algorithms, and other technologies. Compared with products on the market, our system can collect multi-source data from Parkinson’s patients: hand tremor acceleration, EMG signals, finger bending signals, and pressure data. With data fusion, it achieves high-accuracy evaluation. In later stages, it also enables on-demand daily testing and evaluation, making it convenient for everyday use.
System Architecture#
The main controller of the Parkinson’s bradykinesia quantitative assessment system is composed of BearPi-HM Nano. It collects hand data via an MPU6050 (BearPi kit E53_SC2 module), bending sensors, EMG sensors, and pressure sensors. It then performs cross-validation analysis using Huawei Cloud IoT and the ModelArts deep learning platform, achieving the best-performing patient model on the prediction dataset. Together with UPDRS and H&Y quantitative assessment standards, it defines a Parkinson’s comprehensive assessment scale to judge severity. Data is transmitted to Huawei Cloud IoT via WiFi using the MQTT protocol; combined with IoT technology and data cross algorithms, data fusion is performed. After data reaches the IoT platform, a Django backend stores it in a database and provides APIs for front-end calls, enabling data to be sent to mobile phones or tablets for intelligent design. Via a HarmonyOS app, patients can record and view daily-state data, analyze the condition in time, and adjust accordingly. The architecture is shown below:
Based on Huawei Cloud IoT edge-side rule engine technology, the project configures linkage rules via IoTDA and delivers SQL-statement-based rules from the cloud to devices with one click. Through OpenHarmony’s distributed soft bus, devices can perform inter-device linkage directly on the edge side, reducing dependence on network quality and improving overall linkage efficiency. In addition, cloud operators such as feature-extraction and filtering operators are delivered to sensor devices with one click, so that invalid data can be filtered and masked on edge devices. By accessing devices to Huawei Cloud IoT services, the project not only enables a simplified “device model to cloud” workflow, but also gives devices the ability to “think” proactively, greatly reducing the overall development workload.
The Parkinson data acquisition module collects Parkinson-related hand data, including resting tremor, finger bending, hand EMG signals, and finger pressure, enabling wireless remote acquisition. In daily life, patients can use the collection device for routine data collection and evaluation. Internally, it consists of an MPU6050 accelerometer (BearPi kit E53_SC2 module), bending sensors, and an EMG sensor, with BearPi-HM Nano as the main controller. Wearable nodes include the wrist, fingers, and arm. Multiple sensors collect all-around data, and with a WiFi module, the hardware features wireless transmission, long working time, light load, and portability.
The data analysis module mainly includes preprocessing and automatic learning analysis on the ModelArts platform. Among the many hand data signals collected, acceleration features are more prominent, so we focus on analyzing acceleration and related signals and performing preprocessing. In processing, we use features such as tremor data and surface EMG signals captured by the accelerometer, EMG sensor, and bending sensor, and apply an IIR filter to remove abnormal data, filter trends, and remove gravitational acceleration. We perform dot-product operations, normalize the data, and finally obtain truly representative data using a distributed fusion data-processing approach. In addition to preprocessing and daily data recording, our team also uses deep algorithms to mine deeper information, including predicting patients’ conditions. In this project, the team used Huawei’s AI platform ModelArts for deep learning training, leveraging its AutoML capabilities for autonomous analysis.
For data presentation, analysis results are processed and displayed in real time via a HarmonyOS app and a web frontend. Data to be pushed to users is analyzed in real time to produce the current condition assessment. Combined with historical data, big-data-driven comprehensive analysis is used to promptly push scientific rehabilitation plans and medical recommendations. In this way, users can understand their condition at home via the HarmonyOS app, obtain rehabilitation plans and medical advice in time, and follow the corresponding training plans to slow disease progression. This addresses issues such as rapid progression caused by failure to detect changes in time, and diagnostic deviations caused by doctors not having a comprehensive view of patients’ daily status.
Design Approach#
Algorithm Design#
On the algorithm side, based on the Huawei Cloud IoT platform, we pushed feature extraction operators and filtering operators down directly to our OpenHarmony devices. Data is filtered and effectively extracted on the edge side, then uploaded to Huawei Cloud IoT and forwarded to an ECS server for recognition. The trained deep learning model is then applied to output Parkinson’s severity grading. For example, for an action such as extending one finger, at least about 100 data samples would be uploaded. With this approach, total data transmission volume is reduced by 50%, improving transmission efficiency and reducing reliance on network quality. The data model is also more complete.
Data Management and Backend#
For Parkinson rehabilitation data transmission and device-model definition, we used HiSilicon Hi3861 V100 as the main control chip. Based on the OpenHarmony operating system and Huawei Cloud IoTDA device access services, data is transmitted via MQTT. Acceleration data, bending sensor data, EMG sensor data, and pressure data are packaged into JSON format and transmitted via WiFi to Huawei’s IoT platform. On the Huawei Cloud platform, we built the device model for the Parkinson rehabilitation glove, enabling secondary development and one-click import. The device model includes acceleration, angular velocity, pressure, bending angle, EMG signal, training duration, and more.
Using Huawei Cloud IoT platform’s data forwarding service, data is forwarded to a Huawei Cloud ECS server. On the server side, a Django-based web application framework is used to implement the full pipeline from data collection to cloud transmission, and then to ECS persistence storage.
Command delivery and algorithm delivery are implemented by calling Huawei Cloud IoT platform APIs from a tablet to issue control commands and deliver operators. This enables controlling paradigm actions such as finger bending, finger pinching, finger tremor, and arm bending, and collecting data via the corresponding sensors. In addition, the filtering algorithm originally deployed in the cloud is delivered to the edge device, enabling remote updates of edge-side algorithms and parameters.
Paradigm Motion Test Method#
When devices collect data, many factors can affect it, such as the acquisition environment and different populations. These factors may lead to low dataset quality and inaccurate analysis results. Therefore, our team collaborated with Henan Province Hospital of Traditional Chinese Medicine. Based on analysis of real Parkinson’s symptom manifestations, we developed a paradigm motion testing method suitable for people over 60, and combined it with our wearable hardware to enable data collection and analysis in a home environment.
The paradigm motion testing method performs tests for each paradigm action in sequence: finger bending test, finger pinching test, hand tremor test, and arm-bending EMG signal test.
Implementation Principles#
The product is based on OpenHarmony and Huawei Cloud IoT edge-side rule engine, enabling cloud-side delivery of linkage rules. By using the edge-side rule engine capability of Huawei Cloud IoT, inter-device linkage rules are delivered to devices from the cloud with one click, and executed directly on the edge side through OpenHarmony’s distributed soft bus. This reduces dependence on network quality and improves overall linkage efficiency.
In addition, the product uses ModelArts AutoML combined with a comprehensive assessment scale to monitor Parkinson’s rehabilitation status, propose medical recommendations, and display results to users in real time. The device also integrates IoT technology and uses WiFi and 4G modules to enable wireless remote data transmission. For data collection, we designed a standardized paradigm motion testing method to ensure high-quality data acquisition.
Using pressure sensors, bending sensors, accelerometers, and EMG sensors, the system collects arm and hand data simultaneously. Through multi-source heterogeneous fusion algorithms, it can also screen for abnormal condition data. For example, when a patient’s tremor frequency is less than 4Hz, it triggers the EMG sensor to perform a secondary identification to determine whether the patient is in a resting tremor state.
Application Scenarios#
At present, Parkinson’s patients are unable to effectively prevent and monitor their conditions on their own. In daily life, they can only go to the hospital for related examinations to understand their conditions. How to quickly and effectively obtain patients’ condition status and provide subsequent preventive control is an urgent problem to solve. In addition, medical staff need to rely on the internet for data collection, transmission, and analysis, providing relatively continuous, non-invasive, and portable monitoring, improving their work efficiency, and saving time and costs. If problems arise, feedback can also be sent directly to doctors to further reduce medical costs and free up traditional medical resources.
Therefore, by using wearable sensor devices to detect patients’ motor status from multiple angles with multiple sensors, the sensor data can be sent to a data processing center for cross-analysis and validation, and combined with quantitative assessment scales to judge disease severity. To some extent, this can predict and monitor the specific progression of Parkinson’s disease. Combined with IoT, data can be pushed to a HarmonyOS app for patients and medical staff to view at any time.
- Home patients
The work focuses on early- to mid-stage Parkinson’s patients aged 65 and above, and is mainly aimed at home-based patients. Patients can purchase the product from medical-assist device stores or online, use the device one to three times per day, and perform brief data collection following our self-designed paradigm measurement method. They can view condition status, analysis, and scientific rehabilitation plans in real time via the HarmonyOS app.
- Hospitals
When doctors face new patients, the product’s severity detection functionality can help doctors assess the current condition more accurately. When doctors face follow-up patients, they can refer to the patient’s daily data recorded by the product, analysis results, and real-time test results to grasp the condition status more accurately, which helps improve diagnostic accuracy. After cooperating with provincial and municipal hospitals such as Henan Province Hospital of Traditional Chinese Medicine, it can also be promoted to lower-level hospitals through their radiating influence.
- Community rehabilitation and elder-care centers
The work can be promoted to community rehabilitation and elder-care centers for regional promotion and sales. Patients can use it under guidance of professional staff, making the usage more professional and scientific.
- Medical research institutions
For universities, research institutes, and similar organizations that need relevant software and hardware as data acquisition terminals, the work can be promoted and sold to research personnel.
Product Value#
This work is mainly used to collect and record patients’ daily motor status data, helping doctors understand patients’ conditions and provide timely rehabilitation suggestions. The primary customer base is middle-aged and elderly Parkinson’s patients with a certain duration of illness. The goal is to enable early detection of prodromal symptoms, achieve early diagnosis and early treatment, and enable communication between doctors and patients beyond time and space constraints, so that Parkinson’s patients can obtain timely rehabilitation advice at home.
Based on the above, our team combined our technical capabilities and used Huawei Cloud IoT edge-side rule engine to deliver linkage rules from the cloud and execute quickly on the edge side, improving device linkage efficiency. We proposed a wearable-sensor-based Parkinson’s bradykinesia quantitative assessment system that combines IoT, data cross-fusion algorithms, and other technologies. Compared with products on the market, our system can collect multi-source data from Parkinson’s patients: hand tremor acceleration, EMG signals, finger bending signals, and pressure data. With data fusion, it achieves high-accuracy evaluation. In later stages, it also enables on-demand daily testing and evaluation, making it convenient for everyday use.