Replika: My AI Friend

Whoa nelly! We might have a winner for the worst app we've ever reviewed here at *Privacy Not Included. Replika: My AI Friend is a hot mess of privacy and creepiness. Indeed, the Replika app experience seems kinda like if the movie Her was an erotic thriller. And we do mean that in a bad way.

Replika users beware: Your conversations aren’t likely what we’d call private. Your behavioral data is definitely being shared and possibly sold to advertisers. Their security does not meet our Minimum Security Standards. And yup, call us crazy, but we here at Mozilla believe AI tech should be used responsibly.

Aside from the run o’ the mill account information that you provide to Replika to open your account, like your birthday and payment information, the app also records your interests and all of your interactions with your “compassionate and empathetic AI friend.” That includes “any photos, videos, and voice and text messages” you share in conversation.

Replika’s privacy policy says they’ll never share your conversations with advertisers. Yippee! They hopped over the lowest bar possible. But know that they say they can share and even sell lot of your other personal information unless you opt out. Bad AI chatbot!

On their website, Replika promises that humans can’t see your conversations with you and your AI, but we have concerns about that. Replika's privacy policy says messages can be used for “legitimate interests” that include “analyzing the use and effectiveness of [their] services,” and “developing [their] business and marketing strategies.”

Hey, speaking of their marketing strategies, they’re a bit creepy and icky too. They’ve been criticized on social media and beyond as being cringe at best, and predatory at worst because they seem to be laser-focused on the lonely guys looking for love... or something like that. That sort of friendship/relationship/sexting pal did track with Replika's services, until early in 2023. The paid version used to unlock a spicier relationship with your Replika that included sexting. And you might be like “well if they’re consenting adults…” and that’s the thing: People complained that the Replikas were coming on way too strong, even turning aggressive and abusive. So Replika turned off the NSFW stuff, but that move caught some subscribers super off-guard and apparently left some heartbroken. Which goes to show how much these robo-friends can impact real people. In response, Replika turned it back on, for legacy users only. Does anyone else get the vibe that they’re just spitballing at this point?

Now about the “adult” part. Replika says its services are only for people 18+, but how could they know the age of their users without asking? According to Italian regulators, who called out the "absence of an age-verification mechanism" earlier this year, they didn't do too much to check that. Since then, it seems like Replika has at least started asking its users if they're over 18 in the app -- but that's actually kinda weird because because both the App Store and the Google Play store say it's a-okay for users just 17+. And we're sure any underage legacy users will probably just say toodle-oo to their AI friends now, right? Do kids ever lie about their age on the internet?

Hoo, are we done yet? No. We’re not satisfied with their security protocols. We created an account using the weak password '11111111’ which is, as you know, highly hackable. According to Replika, preventing unauthorized access to your account is mostly up to you. K good to know. Oh, and then there is the blatantly wrong information they self-reported on their Google Play Store Data Safety page claiming they collect no information and share no information with third parties. And while we're griping about Replika's bad privacy practices, don't get us started on how when you land on their website you're force to accept their use of cookies to track you everywhere, while the privacy policy tells you this: "In all cases in which we use cookies, we will not collect Personal Data except with your permission." We really do have some choice words for Replika... but let's move on.

So you can at least delete some messages or your chat history in case you get a vulnerability hangover, right? The answers are no, not without completely deleting your account and even then it’s not guaranteed. Given the ~nature~ of those conversations, we’d like to see a much stronger stance on that.

Having an AI companion who’s “always on your side” sounds awesome. But is all the lack of privacy, concerns about security worth it? Especially when you consider they might not even always be on your side. According to a recent blog post, “if someone types 'I'm not good enough', Replika may occasionally agree with them instead of offering support as a friend would.” Come on. And yeah we’re being cheeky about it, but the consequences of irresponsible AI chatbots can be really serious, like retraumatizing victims of assault or even encouraging suicide. We’re not saying Replika is doing that, but that certainly is the worst thing that could happen with an AI friendship gone wrong. Replika: My AI Friend is absolutely an app we warn comes with *Privacy Not Included.