MIT Technology Review
Sponsored by Darktrace
The Algorithm
Artificial intelligence, demystified
Shifting sand
Hello Algorithm readers,

In 2017, Google quietly published a blog post about a new approach to machine learning. Unlike the standard way, which requires the data to be centralized in one place, the new method could learn from a series of data sources distributed across multiple devices. The invention allowed Google to train its predictive text model on all of the messages typed by Android users—without actually ever reading or removing them from their phones.

Despite its cleverness, federated learning, as the researchers called it, gained little traction within the AI community at the time. Now, that is poised to change as it finds resonance in a completely new application: its privacy-first approach could very well be the answer to the greatest obstacle facing AI adoption in healthcare today.

“There is a false dichotomy between the privacy of patient data and the utility of their data to society,” says Ramesh Raskar, an MIT associate professor of computer science whose research focuses on AI in health. “People don’t realize the sand is shifting under their feet and that we can now in fact achieve privacy and utility at the same time.”

Over the last decade, the dramatic rise of deep learning has led to stunning transformations across dozens of industries. It has powered our pursuit of self-driving cars, fundamentally changed the way we interact with our devices, and reinvented our approach to cybersecurity. In healthcare, however, despite many studies showing its promising applications in detecting and diagnosing diseases, progress in using deep learning to help real patients has been tantalizingly slow.

Current state-of-the-art algorithms require immense amounts of data to learn. In most cases, more data is also better. In order to meet that threshold, as well as have it represent enough diversity, hospitals and research institutions need to pool their data reserves. But, especially in the US and UK, the idea of centralizing reams of sensitive medical information into the hands of tech companies has repeatedly—and unsurprisingly—proven intensely unpopular.

As a result, research studies on AI’s ability to detect and diagnose conditions have stayed narrow in their scope and applicability. You can’t deploy a breast cancer detection model around the world when it’s only been trained on a few thousand patients from the same hospital.

Continued below


Sponsor Message

19.02.21 - MIT Tech Review png-01[1] Final

Cyber AI that Responds to Cyber-Attacks in Seconds

Powered by AI, Darktrace Antigena fights back against the most advanced cyber-attacks. From rapid ransomware to stealthy insider threats, our world-leading technology responds in seconds – giving you time to catch up.

Download the white paper now

All this could change with federated learning. The technique can train a model using data stored across multiple different hospitals without that data ever leaving their premises or touching a tech company’s servers. It does this by first training separate models at each hospital with the local data available, then sending those models to a central server to be combined together into a master. As each hospital acquires more data over time, it can download the latest master model, update it with new data, and send it back to the central server. Throughout the process, raw data is never exchanged—only the models, which cannot be reverse engineered to reveal that data.

There are some challenges to federated learning. For one, combining separate models risks creating a master model that’s actually worse than each of its parts. Researchers are now working on refining existing techniques to make sure that doesn’t happen, says Raskar. For another, federated learning requires every hospital to have the infrastructure and personnel capabilities for training machine-learning models. There’s also friction in standardizing data collection across all hospitals. But these challenges aren’t insurmountable, says Raskar. “More work needs to be done, but it’s mostly bandaid work.”

In fact, other privacy-first distributed learning techniques have since cropped up in response to these challenges. Raskar, for example, recently co-invented one called split learning with his student Praneeth Vepakomma. Like federated learning, each hospital starts by training separate models, but they only train them halfway. The half-baked models are then sent to the central server, to be combined and finish training. The main benefit would be to alleviate some of the computational burden from the hospitals. The technique is still mainly a proof-of-concept, but in early testing, Raskar and Vepakomma showed that it created a master model with a fidelity close to what it would be if it were trained on a centralized pool of data.

A handful of companies, like IBM Research, are now working on using federated learning to advance real-world AI healthcare applications. OWKIN, a Paris-based startup backed by Google Ventures, is also using it to predict patients’ resistance to different treatments and drugs as well as their survival rates with certain diseases. The company is working with several cancer research centers across the US and Europe to utilize their data for its models. The collaborations have already resulted in a forthcoming research paper, the founders say, on a new model that predicts a patient’s survival rate for a rare form of cancer based on his or her pathology images.

“I’m really excited,” says OWKIN co-founder Thomas Clozel, a clinical research doctor. “The biggest barrier in oncology today is knowledge. It’s really amazing that we now have the power to extract that knowledge and make medical breakthrough discoveries.”

Raskar believes the applications of distributed learning could also extend far beyond healthcare in any industry where people don’t want to share their data. “In distributed, trustless environments, this is going to be very, very powerful in the future,” he says.


For more on distributed learning, try:

  • Google’s original blog post about federated learning

  • A survey of different privacy-focused distributed learning techniques

  • Ramesh Raskar’s paper on split learning and our coverage of that paper

  • His slidedeck on split and federated learning in healthcare

More from TR

Last week, we released our annual print issue on the 10 Breakthrough Technologies of the year, guest-curated by Bill Gates. In addition to two AI and robotics related selections—robot dexterity and smooth-talking AI assistants—the issue includes several articles that dive into AI’s capabilities and limitations. Here’s an excerpt from one of them on why AI has yet to reshape most businesses:

“Despite what you might hear about AI sweeping the world, people in a wide range of industries say the technology is tricky to deploy. It can be costly. And the initial payoff is often modest. It’s one thing to see breakthroughs in artificial intelligence that can outplay grandmasters of Go, or even to have devices that turn on music at your command. It’s another thing to use AI to make more than incremental changes in businesses that aren’t inherently digital.” Read more here

How is emerging technology shaping the future of work?

We’ll answer your most pressing questions at our two-day conference, packed with some of the world’s leading experts and hosted on the MIT campus. To join, purchase your ticket today.

Bits and bytes

DARPA is looking at crazy AI ideas to keep the US ahead of China
This week it showed off several projects, including efforts to give machines common sense, to have them learn faster with less data, and to create chips that reconfigure themselves. (TR)

Google launched a new compact, low-power version of its AI chip
It will let developers and hardware hackers experiment with deep learning on IoT devices and other mobile gadgets. (Google)

Machine-learning algorithms are sniffing out “ethical” cobalt reserves
The crucial ingredient found in most lithium-ion batteries is often mined by children in war-torn environments. (Quartz)

Prisons are using face recognition to crackdown on drug smuggling
A prison in the UK piloted a new surveillance system that checked if visitors were using fake identifies or repeatedly visiting different prisoners. (TR)

Now any business can access the same type of AI that powered AlphaGo
A startup developed a platform that lets companies use reinforcement learning. (TR


Interpreting U.S.-China AI development as an ‘arms race’ or a winner-takes-all competition fundamentally misunderstands the transnational nature of AI development.

—Justin Sherman, cybersecurity policy fellow at think tank New America, on why perpetuating this narrative is dangerous for American policymaking

Karen Hao
Hello! You made it to the bottom. Now that you're here, fancy sending us some feedback? You can also follow me for more AI content and whimsy at @_KarenHao.
Was this forwarded to you, and you’d like to see more?
New Call-to-action
You received this newsletter because you subscribed with the email address:
edit preferences   |   unsubscribe   |   follow us     
Facebook      Twitter      Instagram
MIT Technology Review
One Main Street
Cambridge, MA 02142