SafeToNet demos anti-sexting child safety tool

By Natasha Lomas
TechCrunch
February 26, 2018

https://finance.yahoo.com/news/safetonet-demos-anti-sexting-child-083046227.html

With rising concern over social media's 'toxic' content problem, and mainstream consumer trust apparently on the slide, there's growing pressure on parents to keep children from being overexposed to the Internet's dark sides.

With rising concern over social media's 'toxic' content problem, and mainstream consumer trust apparently on the slide, there's growing pressure on parents to keep children from being overexposed to the Internet's dark sides. Yet pulling the plug on social media isn't exactly an option.

UK startup SafeToNet reckons it can help, with a forthcoming system of AI-powered cyber safety mobile control tools.

Here at Mobile World Congress it's previewing an anti-sexting feature that will be part of the full subscription service -- launching this April, starting in the UK.

It's been developing its cyber safety system since 2016, and ran beta testing with around 5,000 users last year. The goal is to be "protecting" six million children by the end of this year, says CEO Richard Pursey -- including via pursuing partnerships with carriers (which in turn explains its presence at MWC).

SafeToNet has raised just under £9 million from undisclosed private investors at this point, to fund the development of its behavioral monitoring platform.

From May, the plan is to expand availability to English-speaking nations around the world. They're also working on German, Spanish, Catalan and Danish versions for launch in Q2.

So what's at stake for parents? Pursey points to a recent case in Denmark as illustrative of the risks when teens are left freely using social sharing apps.

In that instance more than 1,000 young adults, many of them teenagers themselves, were charged with distributing child pornography after digitally sharing a video of two 15-year-olds having sex.

The video was shared on Facebook Messenger and the social media giant alerted US authorities -- which in turn contacted police in Denmark. And while the age of consent is 15 in Denmark, distributing images of anyone under 18 is a criminal offense. Ergo sexting can get even consenting teens into legal hot water.

And sexting is just one of the online risks and issues parents now need to consider, argues Pursey, pointing to other concerns such as cyber bullying or violent content. Parents may also worry about their children being targeted by online predators.

"We're a cyber safety company and the reason why we exist is to safeguard children on, in particular, social networking and messaging apps from all those things that you read about every day: Cyber bullying, abuse, aggression, sextortion, grooming," he says.

"We come from the basis that existing parental control systems... simply aren't good enough. They've not kept up to date with the digital world and in particular the world that kids socialize on. So Snapchat, Instagram, less so Facebook, but you get the idea.

"We've tackled this using a whole mixture of deep tech from behavioral analytics, sentiment analysis and so on, all using machine learning, to be able to contextualize messages that children send, share and receive. And then block anything harmful. That's the mission."

Once the SafeToNet app is installed on a child's device, and linked with their parents' SafeToNet account, the software scans for any inappropriate imagery on their device. If it finds anything it will quarantine it and blur the content so it no longer presents a sharing risk, says Pursey.

The software runs continuously in the background on the device so it can also step in in real-time to, for instance, block access to a phone's camera if it believes the child might be about to use it for sexting.

It's able to be so reactive because it's performing ongoing sentiment analysis of everything being typed on the device via its own keyboard app -- and using its visibility into what's being sent and received, how and by whom, to infer a child might be about to send or see something inappropriate.

Pursey says the AI system is designed to learn the child's normal device usage patterns so it can also alert parents to potential behavioral shifts signaled by their online activity -- which in turn might represent a problem or a risk like depression or aggression.

He says SafeToNet's system is drawing on research into social behavioral patterns, including around digital cues like the speed and length of reply, to try to infer psychological impacts.

If that sounds a little Black Mirror/Big Brother, that's kind of intentional. Pursey says it's deliberately utilizing the fact that the children who are its users will know its system is monitoring their device to act as a moderating impulse and rein in risky behaviors.