Virtual Reality Helmets: Bridging the Gap between Technology and Healthcare
for Enhanced Patient Experience
Location:
usa
Collaboration:
2019 – present
Industry:
Expertise:
Technologies:
About the Client
Our client is a technological company in the medical field that implements virtual and augmented reality solutions for medical purposes. This allows them to revolutionize the field and provide practical and precise patient treatment methods. The clinic offers its clients virtual reality technologies for the treatment of chronic pain, anxiety, and post-traumatic stress disorders; rehabilitation classes for patients with physical disabilities, etc.
Business Context
Amazinum’s Data Scientists team joined our client’s team to integrate several enhancements into the data processing and analytical system.
At this stage, the client already had algorithms deployed to the analytical server that processed data from VR helmets. Virtual reality tools allowed patients to play medical games, allowing staff to determine the appropriate treatment, and diagnosis of disease or monitor the course of treatment.
Thus, our data scientists had a codebase provided by the client for further work with the 3D data generated by the VR helmets. This data included headset and hands position in a 3D virtual world with information about that world and events that happened inside it.
Challenges for the Amazinum Team
Among the challenges faced by our data scientists was the monolithic architecture of the codebase. It had a number of disadvantages that made working with it inconvenient. Data from the game session originally was processed after the user finished it, and if the user spent a large amount of time inside the game, it could take some time to analyze and process all that information.
From Words to Actions: The Amazinum team's solution
Using RabbitMQ data scientists of Amazinum changed the architecture to the queues-structure. This made it possible to receive data faster right after it has been generated by the VR helmet and process it in chunks on the go. It significantly decreased the game session processing time and improved the performance by a large margin.
The next task of the Amazinum Data Scientists team was to integrate the new technology that collects data about the direction of the pupils.
Due to the direction, accuracy, or smoothness, doctors can identify and monitor both neurological and physical defects or diseases that may occur in people. They respond to three different types of stimuli and make it possible to detect mental effort and the neural pathways that are formed as a result.
Our team’s task was to analyze and reformat the data from the pupils (gaze vector, blink state, pupils’ size, etc) and integrate it into the existing analytical processes. Our data scientists could obtain this data using certain metrics, such as accuracy, tracking, and smoothness. By combining the information about the head and the direction of the eyes, we obtained the final users’ attention vector.
Data analysis technologies were implemented in an analytical system in VR helmets. In development we have used Numpy, Scikit-Learn for data processing; RabbitMQ for query architecture; Matplotlib for visualization.
Data Science in Action: Development of the XGBoost model
Because the production model had problems with accuracy, and the draft model remained difficult in terms of calculations, it was necessary to develop a more universal option. Our Data Scientists settled on the XGBoost model. It could give the same result as the previous ones while spending fewer resources and being more convenient.
To begin with, we needed to pick up data on which the model could work and make predictions. The difficulty was that the data we used was Time Series. Therefore, the Amazinum Data Scientists company came up with a solution to deal with this problem:
- It was necessary to share the data on a window-by-window basis – that is, data for every second; That is, from 5 lines, we derive the average and get a feature line of the data made.
- Amazinum programmers have developed several features that can show historical changes. For example, we received information about major horse races, the duration of these races, and the breaks between them. Functions that a person could adjust independently were also implemented – for example, pressure.
The next stage was the tuning of the model. In this case, we used the same solutions as in the draft model. Our Data Scientists team provided the optimal configuration for a specific data set. This made it possible to build ideal hyperparameters and their settings.
As for overfitting, here the Data Scientists of Amazinum worked according to the same scenario – they used Class_weight, L1, L2 Regularization, and Dropout.
Outcome
The introduction of machine learning and data analysis for virtual and augmented reality solutions in medical applications has led to significant advances in patient care and treatment.
The Amazinum team, implementing Data Analysis technologies into games on medical VR helmets, developed a solution for the client, thanks to which he could receive data about:
- The patient’s state of health in order to use it in the diagnosis and treatment protocol;
- The final coordination of the patient’s attention;
- Movement of the pupils, to establish both physical and psychoneurological defects;
In a global way, the use of Machine Learning and Data Analysis technologies helped:
- Analyze large volumes of data collected during patient interaction with virtual and augmented reality tools;
- Personalize treatment plans and identify patterns
- Measure the effectiveness of virtual and augmented reality treatment;
- Assess the impact of technology on patient treatment and make informed decisions about treatment plans;
- Prescribe more effective treatment and achieve better results for patients.
Machine learning algorithms have a huge potential in the development and evolution of the entire field of Health Care. Amazinum will help develop solutions that will significantly improve patient outcomes, improve medical training, and advance the healthcare industry.
