Learn how GE Healthcare used AWS to create a new AI model that interprets MRI scans

MRI images are understandably complex and contain a lot of data.

Because of this, developers training large language models (LLMs) for MRI analysis have had to split captured images into 2D. But this results in only an approximation of the original image, which limits the model’s ability to analyze intricate anatomical structures. This creates challenges in complex cases involving brain tumors, skeletal disorders or cardiovascular diseases.

But GE healthcare appears to have overcome this huge hurdle by introducing the industry’s first full-body 3D MRI basic research model (FM) at this year’s show. AWS re:Invent. For the first time, models can use full 3D images of the entire body.

GE Healthcare’s FM was built on AWS from scratch (there are very few models designed specifically for medical imaging like MRIs) and is based on over 173,000 images from over 19,000 studies. The developers say they have been able to train the model with five times less computing than was previously needed.

GE Healthcare has not yet marketed the basic model; It is still in an evolutionary research phase. An early evaluator, Mass General Brighamwill start experimenting with it soon.

“Our vision is to put these models in the hands of technical teams working in healthcare systems, giving them powerful tools to develop clinical and research applications faster and also more cost-effectively,” GE’s head of artificial intelligence told VentureBeat. HealthCare, Parry Bhatia.

Enable real-time analysis of complex 3D MRI data

While this is an innovative development, generative AI and LLMs are not new territory for the company. The team has been working with advanced technologies for more than 10 years, Bhatia explained.

One of its star products is AIR Recon DLa deep learning-based reconstruction algorithm that allows radiologists to achieve sharp images faster. The algorithm removes noise from raw images and improves the signal-to-noise ratio, reducing scan times by up to 50%. Since 2020, 34 million patients have been scanned with AIR Recon DL.

GE Healthcare began work on its MRI FM in early 2024. Because the model is multimodal, it can support image-to-text searches, linking images and words, and segmenting and classifying diseases. The goal is to give healthcare professionals more detail than ever in a scan, Bhatia said, which will lead to faster, more accurate diagnosis and treatment.

“The model has significant potential to enable real-time analysis of 3D MRI data, which can improve medical procedures such as biopsies, radiation therapy and robotic surgery,” Dan Sheeran, general manager of healthcare and life sciences, told VentureBeat. from AWS.

It has already outperformed other publicly available research models on tasks including classifying prostate cancer and Alzheimer’s disease. It has shown up to 30% accuracy in matching MRI scans with text descriptions in image retrieval, which may not seem that impressive, but is a big improvement over the 3% ability exhibited by models similar.

“It’s gotten to a stage where it’s delivering really strong results,” Bhatia said. “The implications are enormous.”

Do more with (much less) data

The MRI process requires a few different types of data sets to support various techniques that map the human body, Bhatia explained.

What is known as T1-weighted imaging, for example, highlights fatty tissue and decreases water signal, while T2-weighted images enhance water signals. The two methods are complementary and create a complete image of the brain to help doctors detect abnormalities such as tumors, trauma or cancer.

“MRI images come in different shapes and sizes, similar to how books would come in different formats and sizes, right?” Bhatia said.

To overcome the challenges presented by diverse data sets, the developers introduced a “resize and adapt” strategy so that the model could process and react to different variations. Additionally, data may be missing in some areas (an image may be incomplete, for example), so they taught the model to simply ignore those instances.

“Instead of getting stuck, we taught the model to skip the gaps and focus on what was available,” Bhatia said. “Think of it like solving a puzzle with some pieces missing.”

The developers also employed semi-supervised learning between students and teachers, which is especially useful when there is limited data. With this method, two different neural networks are trained with labeled and unlabeled data, and the teacher creates labels that help the student learn and predict future labels.

“We are now using a lot of these self-supervised technologies, which don’t require large amounts of data or labels to train large models,” Bhatia said. “It reduces dependencies, allowing you to learn more from these raw images than in the past.”

This helps ensure that the model works well in hospitals with fewer resources, older machines, and different types of data sets, Bhatia explained.

He also highlighted the importance of the multimodality of the models. “A lot of technology in the past was unimodal,” Bhatia said. “I would just look at the image, the text. But now they are becoming multimodal, they can go from image to text, text to image, so you can incorporate a lot of things that were done with separate models in the past and really unify the workflow.”

He emphasized that researchers only use data sets to which they have rights; GE Healthcare has partners who license de-identified data sets and are careful to adhere to compliance standards and policies.

Using AWS SageMaker to Address Data and Compute Challenges

There are certainly many challenges in building such sophisticated models, such as limited computational power for gigabyte-sized 3D images.

“It’s a massive volume of 3D data,” Bhatia said. “You need to bring it into the model’s memory, which is a really complex problem.”

To help overcome this, GE Healthcare relied on Amazon SageMakerwhich provides high-speed networking and distributed training capabilities across multiple GPUs, and leveraged Nvidia A100 and Tensor Core GPUs for large-scale training.

“Due to the size of the data and the size of the models, they can’t send it to a single GPU,” Bhatia explained. SageMaker allowed them to customize and scale operations across multiple GPUs that could interact with each other.

The developers also used AmazonFSx in amazon s3 object storage, which enabled faster reading and writing of data sets.

Bhatia noted that another challenge is cost optimization; With Amazon’s Elastic Compute Cloud (EC2), developers were able to move unused or infrequently used data to lower-cost storage tiers.

“Leveraging Sagemaker to train these large models, primarily for efficient, distributed training across multiple high-performance GPU clusters, was one of the critical components that really helped us move faster,” Bhatia said.

He emphasized that all components were built from a compliance and data integrity perspective that took into account HIPAA and other regulations and regulatory frameworks.

Ultimately, “these technologies can really optimize, help us innovate faster, as well as improve overall operational efficiency by reducing administrative burden and ultimately drive better patient care, because more personalized care is now delivered.”

Serving as a basis for other specialized and tuned models.

While the model is specific to the MRI domain for now, the researchers see great opportunities to expand to other areas of medicine.

Sheeran noted that AI in medical imaging has historically been limited by the need to develop custom models for specific conditions in specific organs, requiring expert annotations for each image used in training.

But that approach is “inherently limited” because of the different ways diseases manifest among individuals and introduces generalization challenges.

“What we really need are thousands of such models and the ability to quickly create new ones as we find novel information,” he said. High-quality labeled data sets are also essential for each model.

Now, with generative AI, instead of training discrete models for each disease/organ combination, developers can pre-train a single base model that can serve as the basis for further specialized and refined models.

For example, GE Healthcare’s model could be expanded to areas such as radiation therapy, where radiologists spend a lot of time manually marking organs that might be at risk. It could also help reduce scanning time during X-rays and other procedures that currently require patients to sit in a machine for long periods, Bhatia said.

Sheeran marveled that “we are not only expanding access to medical imaging data through cloud-based tools; “We are changing the way that data can be used to drive AI advances in healthcare.”

Daily insights into business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory changes to practical implementations, so you can share insights for maximum return on investment.

Read our Privacy Policy

Thanks for subscribing. Check out more VB newsletters here.

An error occurred.