MIT Technology Review
The Algorithm
Artificial intelligence, demystified
A moment of silence for our mechanical brethren
12.18.18
Hello Algorithm readers,

What a robot vigil says about our desire to relate with machines. A food delivery robot caught fire at UC Berkeley last Friday, while making its rounds on campus. As newsworthy as that is, what happened next is more interesting: an outpouring of grief from students, who called it a “hero” and a “legend,” and who commemorated its loss with a candlelit memorial. The whole thing is rather ridiculous—and could be written off as a hilarious episode of “things college students do to get out of studying for finals.” But there’s also something kind of noteworthy about the readiness with which people anthropomorphized a machine. It gives us just a little window of insight into how our societal dynamics will change as we grow to accommodate more robots—and human-mimicking AI.

In fact, our complicated relationship with robots has warranted an entire field of study, and it has discovered some pretty fascinating things. Through her research at Georgia Tech, for example, social robotics expert Ayanna Howard found that children naturally interact with humanoids like they would other humans and will work hard to please them if they “show” signs of disappointment. Adults, too, will place significant trust in robots even in high-stress situations, such as during a smoke-filled fire evacuation.

Our tendency to build relationships with non-human entities isn’t new. "We have anthropomorphized things for a very long time," Genevieve Bell, an anthropologist and vice president and senior fellow at Intel, once told me when I asked her what she thought about these kinds of relationships, "whether it was the ways in which we attributed higher level thinking to domestic animals—dogs, cats, horses—or the ways we named our cars and gave them personalities."

But our specific desire to relate with robots and AI feels different from what has come before. Unlike pets, they’re inanimate objects. Unlike cars—or really any other machine—they’re capable of engaging with us at much more intimate and powerful levels. One of the earliest studies conducted on long-term human-robot interaction in 2010 showed that participants developed a far greater emotional attachment to a robotic weight-loss coach than a desktop computer with the same software. Other studies have found that we are reluctant to “hurt” robots and will react to seeing one in “pain” as if it were a human being.

So, in a moment of silliness, students inadvertently struck upon some deeply relevant questions as we march deeper into this century: what it means for us to humanize robots and AI, how we should account for that when we bring them into our lives, and whether all this is okay.


The laborious process of making AI pull a funny. Inspired by research scientist Janelle Shane, author of the delightful blog AI Weirdness, senior AI editor Will Knight and I embarked on a challenge to generate funny Christmas movie synopses with a neural network. Don’t worry, there will be a story of our results (with illustrations!) soon. In the meantime, here’s a peek behind the scenes.

As I mentioned in passing in the last Algorithm, we used a library called textgenrnn, which can generate sentences in the style of the text you train it on. I now empathize with people who say training neural networks is more of an art than a science. To coax good results out of a network, you can either change your dataset or tune the algorithm’s various settings. Both Will and I used the same dataset, a list of synopses from Wikipedia, so we focused on the latter.

Whereas Will cleverly used the defaults and got some pretty decent results, I immediately began changing everything from the number of layers to epochs to the temperature. The layers here refer to the complexity of the neural network: the more layers it has, the more complicated the data it can handle. The number of epochs are the number of times it gets to look at the training data before it spits out its final result. And the temperature is like a creativity setting: the lower the temperature, the more the network will choose common words in the training dataset versus those that rarely appear. (Of course, I didn’t know any of this while doing the exercise. Thank you, Janelle, for explaining this to me later.)

As I blindly tweaked all these knobs, most of the results I got were flat out terrible—sentences starting with three nouns or ending in an article:

  • arthur serial daughter meet greed to as when reunite up and low parents a paws become
  • an dads aunt decides the growing to of cheer to try jingle photograph the holiday mysterious the

In other words, they were incomprehensible, not funny.

Part of the reason, Shane explained, was due to my small training data set and part of it was due to textgenrnn. The algorithm, she said, just isn’t that good at constructing sentences compared to alternatives. But even if I’d used better data and a better algorithm, the challenge I hit on was exceedingly normal. It just takes a lot of manual labor to make a neural network spit out gibberish that humans would consider remotely humorous.

"For some data sets, I'm only showing people maybe one out of a hundred things it generates," Shane admitted. "I'm doing really well if one out of ten is actually funny and worth showing." In many instances, she continued, it takes her more time to curate the results than to train the algorithm. Lesson learned: neural networks aren’t that funny. It’s the humans that are.

More from TR

Martin Tisné, managing director at Luminate, on the need for a bill of data rights: “‘Data ownership’ is a flawed, counterproductive way of thinking about data. It not only does not fix existing problems; it creates new ones. Instead, we need a framework that gives people rights to stipulate how their data is used without requiring them to take ownership of it themselves.” Read more here.

Send me cats!

Thank you to the readers who pointed out the broken link for textgenrnn in the last issue. The working link is here, along with a blog post on how to use it.

In other news, Monica Dinculescu, a developer at Google, released a magical tool for doodling with the help of a neural network. Try it and send me your best feline at algorithm@technologyreview.com.

Are you a thought leader, innovator, and executive looking to make your opinion heard?

We're now accepting applications to the MIT Technology Review Global Panel. Learn how to share your expertise and opinions today.


Research

Making art into science. When training a neural network on a massive amount of data, AI researchers have discovered that the most efficient way to get the best results is to feed the network in stages. Let's say you're training with 10 million images. You want to show the images to the neural net bit by bit, rather than all at once. Otherwise it has trouble learning more nuanced patterns from them.

Instead of showing it a single image at a time, however, which would certainly blow up the amount of training time, AI researchers often use a technique called data-parallelism to feed in multiple images at once. But figuring out the best number of images to include in each batch has always been a mysterious art. Feed in too many images at once and you waste computational power without improving training results; feed in too few and you waste a lot of time.

A new paper from OpenAI now proposes a way for researchers to approximately calculate the ideal size of a batch. They found that a simple statistic for measuring the variation in a given set of data strongly correlates with how much of it you want in each batch. When there is little variation, you want to keep your batch sizes small, and when there is a lot of variation, you want to make them big. Having this approximation handy gives researchers more control over the trade-offs they need to make in training—within the bounds of what’s financially feasible and the bounds of a project timeline.

Bits and Bytes

The man who’s writing the bible on algorithms worries about their prominence
Donald Knuth has been compiling algorithms into a four-volume opus for over 50 years. (NYT)

Voice assistants and chatbots could make websites obsolete
We’ll no longer need to access the internet through visual interfaces once we can do it through speech. (Bloomberg, podcast)

A deeper look at the hidden labor behind AI
In the small fourth and fifth-tier towns of China, people work endlessly to label data for training algorithms. (GQ China, as translated by the ChinAI newsletter)

Facebook has shifted its newsfeed algorithm to favor its own videos
The tech giant has ambitious plans to grow Watch, its competitor to YouTube. (NYT)

Marketers are eager to use AI but don’t know where to start
Investing in sexy tech like voice assistants is tempting but not necessarily the best use of funds. (WSJ)

Quotable

We live in a time when American democratic debate is being influenced by liars spreading memes about our inability to understand the truth.

—Nicholas Thompson and Issie Lapowsky of WIRED on how Russian trolls used memes to divide America

Karen Hao
Hello! You made it to the bottom. Now that you're here, fancy sending us some feedback? You can also follow me for more AI content and whimsy at @_KarenHao.
Was this forwarded to you, and you’d like to see more?
New Call-to-action
You received this newsletter because you subscribed with the email address:
edit preferences   |   unsubscribe   |   follow us     
Facebook      Twitter      Instagram
MIT Technology Review
One Main Street
Cambridge, MA 02142
TR