When Jessie Battaglia started looking for a new babysitter for her one-year-old son, she wanted more information than she could get from a criminal-background check, parent comments and a face-to-face interview.
So she turned to Predictim, an online service that uses "advanced artificial intelligence" to assess a babysitter's personality, and aimed its scanners at one candidate's thousands of Facebook, Twitter and Instagram posts.
The system offered an automated "risk rating" of a 24-year-old candidate, saying she was at a "very low risk" of being a drug abuser. But it gave a slightly higher risk assessment - a two out of five - for bullying, harassment, being "disrespectful" and having a "bad attitude."
The system didn't explain why it had made that decision. But Battaglia, who had believed the sitter was trustworthy, suddenly felt pangs of doubt.
"Social media shows a person's character," said Battaglia, 29, who lives near Los Angeles. "So why did she come in at a two and not a one?"
Predictim is offering parents the same playbook that dozens of other tech firms are selling to employers around the world: artificial-intelligence systems that analyse a person's speech, facial expressions and online history with promises of revealing the hidden aspects of their private lives.
The technology is reshaping how some companies approach recruiting, hiring and reviewing workers, offering employers an unrivalled look at job candidates through a new wave of invasive psychological assessment and surveillance.
The tech firm Fama says it uses AI to police workers' social media for "toxic behaviour" and alert their bosses. And the recruitment-technology firm HireVue, which works with companies such as Geico, Hilton and Unilever, offers a system that automatically analyses applicants' tone, word choice and facial movements during video interviews to predict their skill and demeanour on the job. (Candidates are encouraged to smile for best results.)
But critics say Predictim and similar systems present their own dangers by making automated and possibly life-altering decisions virtually unchecked.
The systems depend on black-box algorithms that give little detail about how they reduced the complexities of a person's inner life into a calculation of virtue or harm. And even as Predictim's technology influences parents' thinking, it remains entirely unproven, largely unexplained and vulnerable to quiet biases over how an appropriate babysitter should share, look and speak.
There's this "mad rush to seize the power of AI to make all kinds of decisions without ensuring it's accountable to human beings," said Jeff Chester, the executive director of the Center for Digital Democracy, a tech advocacy group. "It's like people have drunk the digital Kool-Aid and think this is an appropriate way to govern our lives."
Predictim's scans analyse the entire history of a babysitter's social media, which, for many of the youngest sitters, can cover most of their lives. And the sitters are told they will be at a great disadvantage for the competitive jobs if they refuse.
Predictim's chief and co-founder Sal Parsa said the company, launched last month as part of the University of California at Berkeley's SkyDeck tech incubator, takes ethical questions about its use of the technology seriously. Parents, he said, should see the ratings as a companion that "may or may not reflect the sitter's actual attributes."
But the danger of hiring a problematic or violent babysitter, he added, makes the AI a necessary tool for any parent hoping to keep his or her child safe.
"If you search for abusive babysitters on Google, you'll see hundreds of results right now," he said. "There's people out there who either have mental illness or are just born evil. Our goal is do anything we can to stop them."
A Predictim scan starts at $24.99 and requires a babysitter's name and email address and her consent to share broad access to her social media accounts. The babysitter can decline, but a parent is notified of her refusal, and in an email the babysitter is told "the interested parent will not be able to hire you until you complete this request."
Predictim's executives say they use language-processing algorithms and an image-recognition software known as "computer vision" to assess babysitters' Facebook, Twitter and Instagram posts for clues about their offline life. The parent is provided the report exclusively and does not have to tell the sitter the results.
Parents could, presumably, look at their sitters' public social media accounts themselves. But the computer-generated reports promise an in-depth inspection of years of online activity, boiled down to a single digit: an intoxicatingly simple solution to an impractical task.
The risk ratings are divided into several categories, including explicit content and drug abuse. The start-up has also advertised that its system can evaluate babysitters on other personality traits, such as politeness, ability to work with others and "positivity."
The company hopes to upend the multibillion-dollar "parental outsourcing" industry and has begun advertising through paid sponsorships of parenting and "mummy" blogs. The company's marketing focuses heavily on its ability to expose hidden secrets and prevent "every parent's nightmare," citing criminal cases including that of a Kentucky babysitter charged earlier this year with severely injuring an eight-month-old girl.
"Had the parents of the little girl injured by this babysitter been able to use Predictim as part of their vetting process," a company marketing document says, "they would never have left her alone with their precious child."
But tech experts say the system raises red flags of its own, including worries that it is preying on parents' fears to sell personality scans of untested accuracy.
They also question how the systems are being trained and how vulnerable they might be to misunderstanding the blurred meanings of sitters' social media use. For all but the highest-risk scans, the parents are given only a suggestion of questionable behaviour and no specific phrases, links or details to assess on their own.
When one babysitter's scan was flagged for possible bullying behaviour, the unnerved mother who requested it said she couldn't tell whether the software had spotted an old movie quote, song lyric or other phrase as opposed to actual bullying language.
But Predictim nevertheless says it is gearing up for a nationwide expansion. Executives at Sittercity, an online babysitter marketplace visited by millions of parents, said they are launching a pilot program early next year that will fold in Predictim's automated ratings into the site's current array of sitter screenings and background checks.
"Finding a sitter can come with a lot of uncertainty," said Sandra Dainora, Sittercity's head of product, who believes tools like these could soon become "standard currency" for finding caregivers online. "Parents are always seeking the best solution, the most research, the best facts."
The Washington Post