On YouTube Kids, Startling Videos Slip Past Filters

On YouTube Kids, Startling Videos Slip Past Filters

- in Technology
9
0

But the app contains dark corners, too, as videos that are disturbing for children slip past its filters, either by mistake or because bad actors have found ways to fool the YouTube Kids algorithms.

In recent months, parents like Ms. Burns have complained that their children have been shown videos with well-known characters in violent or lewd situations and other clips with disturbing imagery, sometimes set to nursery rhymes. Many have taken to Facebook to warn others, and share video screenshots showing moments ranging from a Claymation Spider-Man urinating on Elsa of “Frozen” to Nick Jr. characters in a strip club.

This scene from a video on YouTube Kids shows Mickey Mouse in a pool of blood while Minnie Mouse watches, aghast.

Malik Ducard, YouTube’s global head of family and learning content, said that the inappropriate videos were “the extreme needle in the haystack,” but that “making the app family friendly is of the utmost importance to us.”

While the offending videos are a tiny fraction of YouTube Kids’ universe, they are another example of the potential for abuse on digital media platforms that rely on computer algorithms, rather than humans, to police the content that appears in front of people — in this case, very young people.

And they show, at a time when Congress is closely scrutinizing technology giants, how rules that govern at least some of the content on children’s television fail to extend to the digital world.

When videos are uploaded to YouTube, algorithms determine whether or not they are appropriate for YouTube Kids. The videos are continually monitored after that, Mr. Ducard said, a process that is “multilayered and uses a lot of machine learning.” Several parents said they expected the app to be safer because it asked during setup whether their child was in preschool or older.

This video on the app was uploaded to YouTube by a verified account called Freak Family, and has more than 20 million views.

Mr. Ducard said that while YouTube Kids may highlight some content, like Halloween videos in October, “it isn’t a curated experience.” Instead, “parents are in the driver’s seat,” he said, pointing to the ability to block channels, set usage timers and disable search results.

Parents are also encouraged to report inappropriate videos, which someone at YouTube then manually reviews, he said. He noted that in the past 30 days, “less than .005 percent” of the millions of videos viewed in the app were removed for being inappropriate.

“We strive,” he added, “to make that fraction even lower.”

Holly Hart of Gray, Tenn., said she was recently reading while her 3-year-old daughter was in the room when she noticed that Disney Junior characters in the video her daughter was watching started “turning into monsters and trying to feed each other to alligators.” An image previewing a recommended video showed the characters in a provocative pose.

“It was an eye-opener for me,” said Ms. Hart, who had downloaded the app because it was being used at the local elementary school.

Not all of the inappropriate videos feature cartoons. Alisa Clark Wilcken of Vernal, Utah, said her 4-year-old son had recently seen a video of a family playing roughly with a young girl, including a scene in which her forehead is shaved, causing her to wail and appear to bleed.

Most of the videos flagged by parents were uploaded to YouTube in recent months by anonymous users with names like Kids Channel TV and Super Moon TV. The videos’ titles and descriptions feature popular character names and terms like “education” and “learn colors.”

They are independently animated, presumably to avoid copyright violations and detection. Some clips uploaded as recently as August have millions of views on the main YouTube site and run automatically placed ads, suggesting they are financially lucrative for the makers as well as YouTube, which shares in ad revenue. It is not clear how many of those views came on YouTube Kids.

One video on YouTube Kids from the account Subin TV shows the “PAW Patrol” characters in a strip club. One of them then visits a doctor and asks for her cartoon legs to be replaced with long, provocative human legs in stilettos. The account’s description says, “Video created with the purpose of learning and development of children!”

The account that posted the video seen by Ms. Burns’s son is named Super Ares TV and has a Facebook page called PAW Patrol Awesome TV. Questions sent there were mostly ignored, though the account did reply: “That’s a Cute character and video is a funny story, take it easy, that’s it.”