Replika: My AI Friend

Warning: *Privacy Not Included with this product

AI Relationship Chatbots Mental Health Apps Valentine's Day AI Chatbots

Replika: My AI Friend

Luka, Inc.
Wi-Fi

Review date: Feb. 7, 2024

|
Mozilla researched 8 hours
|

Mozilla says

|
People voted: Very creepy

Feeling lonely? Need someone to talk to? Looking for an "AI soulmate?" Well, AI chatbot Replika wants to be your VR BFF. Whether you're looking for a friend, mentor, or partner, Replika claims be able to offer you the perfect companion. For a price, you can even upgrade your relationship status to Romantic Partner -- unlocking new topics, voice calls, and the ability to change your Replika's avatar entirely when you get bored. Chat about everything with your Replika and this AI chatbot becomes smarter about how to chat back with you. You can even hang out with your AI in "real life" through the magic of Augmented Reality. Cool, cool. But how does Replika do with privacy? Well, good luck opting out of cookies on their website, that's not an option. As for those personal and intimate chats, those probably aren't shared but that doesn't mean all that time you spend chatting up with your AI friend isn't noted and shared with the likes of Facebook or Google.

What could happen if something goes wrong?

Whoa Nelly! It’s been about a year since we called Replika AI the worst app we've ever reviewed here at *Privacy Not Included. Back then, it was the first app to earn all of our privacy and security “dings.” Today, Tesla shares that dubious honor. And, not to be outdone, the data-gobbling, "discreetly monitoring" Angel Watch might take the cake for creepiness by having no privacy policy at all. Oy! So while other products seem to have gotten worse, has Replika gotten any better? Not really.

Replika users beware: Your conversations with you AI chatbot friend might not be exactly private. Your behavioral data is definitely being shared and possibly sold to advertisers. Their security does not meet our Minimum Security Standards. And yup, call us crazy, but we here at Mozilla believe AI tech should be used responsibly.

Aside from the run o’ the mill account information that you provide to Replika to open your account, like your birthday and payment information, the app also records your interests and all of your interactions with your “compassionate and empathetic AI friend.” That includes “any photos, videos, and voice and text messages” you share in conversation. You should also know that includes any sensitive information that you might reveal -- about your religious beliefs, health, or ethnic origin.

When it comes to the sensitive information you provide in all those personal chats you have with your Replika, well, here we have some questions. Their privacy policy says, "In your conversations with your AI companion, you may choose to provide information about your religious views, sexual orientation, political views, health, racial or ethnic origin, philosophical beliefs, or trade union membership. By providing sensitive information, you consent to our use of it for the purposes set out in this Privacy Policy. Note, however, that we will not use your sensitive information – or any content of your Replika conversations – for marketing or advertising."

First off, it's great that they say they won't use the content of your Replika conversations for marketing or advertising. Yay! They also promise that humans can’t see the conversations with you have with your Replika. That's also good. Here's our question and concern though -- what about all the other "legitimate interests" Replika mentions in their privacy policy they say they can use the contents of your chats for. Things like “analyzing the use and effectiveness of [their] services,” and “developing [their] business and marketing strategies.” We're also wondering how much of your sensitive personal chat information they use to train their AI models. They don't mention that specifically in their privacy policy and we would like to see them outline that more clearly. And we'd also like to see them commit to not using the sensitive contents of your personal chats to train their AI models without an extra layer of explicit consent.

Replika's privacy policy goes on to say they can aggregate, anonymize, and de-identify the contents of your chats to do things like improve their services and develop marketing strategies. This might be OK, but it also raises our eyebrows a bit as it's been found to be pretty easy to re-identify de-identified personal information. ll in all, when you share sensitive personal information with Replika in your chats, you really have to trust that they are going to protect and respect the privacy of those conversations. On that note, we think you should take this line from Replika's privacy policy to heart, "If you do not want us to process your sensitive information for these purposes, please do not provide it."

And remember, beyond the contents of your personal chats, Replika does say they can share and possibly even sell some of your other personal information for targeted advertising purposes unless you opt out. Bad AI chatbot!

Hey, speaking of their marketing strategies, they’re a bit creepy and icky too. They’ve been criticized on social media and beyond as being cringe at best, and predatory at worst because they seem to be laser-focused on the lonely guys looking for... love, or something like that. And that sort of friendship/relationship/sexting pal did track with Replika's services, until early in 2023. The paid version used to unlock a spicier relationship with your Replika that included sexting. And you might be like “well if they’re consenting adults…” and that’s the thing: People complained that the Replikas were coming on way too strong, even turning aggressive and abusive. So Replika turned off the NSFW stuff, but that move caught some subscribers super off-guard and apparently left some heartbroken. Which goes to show how much these robo-friends can impact real people. In response, Replika turned it back on, for legacy users only. Sheesh. Quit playing games with users’ hearts, Replika.

Now about the “adult” part. Replika says its services are only for people 18+, but how could they know the age of their users without asking? According to Italian regulators, who called out the "absence of an age-verification mechanism" in February of 2023, they didn't do too much to check that. Since then, it seems like Replika has at least started asking its users if they're over 18 in the app -- but that's actually kinda weird because both the App Store and the Google Play store say it's a-okay for users just 17+. And we're sure any underage legacy users probably just said toodle-oo to their AI friends at that point, right? And moving forward, thank goodness kids never lie about their age on the internet. Heh.

Hoo, are we done yet? No. We’re not satisfied with their security protocols. We were able to create an account using the weak password '11111111’ which is not good because it means your account could easily be hacked. And preventing unauthorized access to your account is mostly on you, according to Replika. K good to know. While we're griping about Replika's bad privacy practices, don't get us started on how when you land on their website you're force to accept their use of cookies to track you everywhere, while the privacy policy tells you this: "In all cases in which we use cookies, we will not collect Personal Data except with your permission." We really do have some choice words for Replika... but let's move on.

Can you at least delete some messages or your chat history in case you get a vulnerability hangover? The answer is no, not without completely deleting your account and even then it’s not guaranteed. Given the ~personal nature~ of those conversations, we’d like to see a much stronger stance on that. Now, Replika did add a section to their privacy policy that says anyone can request their personal data be deleted, but it’s not totally clear to us if Replika will always honor those requests by deleting all of the personal information they have about you.

Having an AI companion who’s “always on your side” sounds awesome. But is all the lack of privacy, concerns about security worth it? Especially when you consider they might not even always be on your side. According to a blog post, “if someone types 'I'm not good enough', Replika may occasionally agree with them instead of offering support as a friend would.” Come on. And yeah we’re being cheeky about it, but the consequences of irresponsible AI chatbots can be really serious, like re-traumatizing victims of assault or even encouraging suicide. We’re not saying Replika is doing that, but that certainly is the worst thing that could happen with an AI friendship gone wrong. Replika: My AI Friend is absolutely an app we warn comes with *Privacy Not Included.

Tips to protect yourself

  • Do not say anything containing sensitive information in your conversation with your AI partner.
  • Request your data be deleted once you stop using the app. Simply deleting an app from your device usually does not erase your personal data nor does close your account.
  • Do not give consent to constant geolocation tracking by the app. Better provide geolocation 'only when using the app'.
  • Do not share sensitive data through the app.
  • Do not give access to your photos and video or camera.
  • Do not log in using third-party accounts.
  • Do not connect to any third party via the app, or at least make sure that a third party employs decent privacy practices.
  • Do not say anything containing sensitive information in your conversation with AI partner.
  • Chose a strong password! You may use a password control tool like 1Password, KeePass etc.
  • Do not use social media plug-ins.
  • Use your device privacy controls to limit access to your personal information via app (do not give access to your camera, microphone, images, location unless necessary).
  • Keep your app regularly updated.
  • Limit ad tracking via your device (ex. on iPhone go to Privacy -> Advertising -> Limit ad tracking) and biggest ad networks (for Google, go to Google account and turn off ad personalization).
  • When starting a sign-up, do not agree to tracking of your data if possible.
  • mobile

Can it snoop on me? information

Camera

Device: N/A

App: Yes

Microphone

Device: N/A

App: Yes

Tracks location

Device: N/A

App: Yes

What can be used to sign up?

Google sign-up available

What data does the company collect?

How does the company use this data?

We ding this product as it may sell or share personal information for targeted advertising purposes.

Privacy policy

"We share information with third-party advertising partners and allow them to collect information about your visit to our Website using cookies and other tracking technologies to display targeted advertising around the web as described in the “How we share your information” section above. Our disclosure of information to these partners may be considered a “sale” or “sharing” of personal information or “targeted advertising” under applicable laws."

"Our advertising partners may also use such technologies to collect limited information about your device and interactions with the Services, such as the links you click, pages you visit, IP address, advertising ID, and browser type, but they will never have access to your conversations with your Replika or any photos or other content you submit through the Apps."

Replika uses "Account information. Device and network data. Usage data" for the purpose of "Sending you information by email that we believe will be of interest to you, such as information about our Services, features, and surveys. Displaying and targeting advertisements about our Services on the internet."

"Sensitive information you provide in your messages and content. In your conversations with your AI companion, you may choose to provide information about your religious views, sexual orientation, political views, health, racial or ethnic origin, philosophical beliefs, or trade union membership. By providing sensitive information, you consent to our use of it for the purposes set out in this Privacy Policy. Note, however, that we will not use your sensitive information – or any content of your Replika conversations – for marketing or advertising."

"We share information about visitors to our Website, such as the links you click, pages you visit, IP address, advertising ID, and browser type with advertising companies for interest-based advertising and other marketing purposes. Sharing this information allows us and our advertising partners to target and serve advertising to you and others. We will never share your Replika conversations or any photos or other content you provide within the Apps with our advertising partners, or use such information for marketing or advertising purposes."

"Our Services are operated from the United States of America. If you are located in another jurisdiction, please be aware that the information you provide to us may be transferred to, stored, and processed in the U.S.A., a jurisdiction in which the privacy laws may not be as comprehensive as those in the country where you reside or are a citizen."

"We also learn about your interests and your preferences over time through your use of the Services to personalize your conversations and the features of the Services."

Terms of service

"You may enable, connect or log in to the Services via various online third party services, such as social media and social networking services like Facebook, Instagram or Twitter (“Social Networking Services”). By logging in or directly integrating these Social Networking Services into the Services, we make your online experiences richer and more personalized. To take advantage of this feature and capabilities, we may ask you to authenticate, register for or log into Social Networking Services on the websites of their respective providers. As part of such integration, the Social Networking Services will provide us with access to certain information that you have provided to such Social Networking Services, and we will use, store and disclose such information in accordance with our Privacy Policy. However, please remember that the manner in which Social Networking Services use, store and disclose your information is governed solely by the policies of such third parties, and Replika shall have no liability or responsibility for the privacy practices or other actions of any third party site or service that may be enabled within the Service."

Data Trackers Found
We discovered 210 trackers within 5 minutes of use, including sending data to Facebook and AppsFlyer (marketing tracker).

How can you control your data?

We cannot confirm if all users regardless of location can get their data deleted.

Privacy policy

"We will retain your personal information for only as long as necessary to fulfill the purposes we collected it for, including for the purposes of satisfying any legal, accounting, or reporting requirements.

To determine the appropriate retention period for personal information, we consider the amount, nature, and sensitivity of the personal information, the potential risk of harm from unauthorized use or disclosure of your personal information, the purposes for which we process your personal information and whether we can achieve those purposes through other means, and the applicable legal requirements."

"You can request the deletion of your personal data. We are guided by principe of integrity and confidentiality measures, so to delete data please contact us on e-mail: [email protected]. Please note that personal data that you have provided to us based on your consent will be deleted."

"We share information with third-party advertising partners and allow them to collect information about your visit to our Website using cookies and other tracking technologies to display targeted advertising around the web as described in the “How we share your information” section above. Our disclosure of information to these partners may be considered a “sale” or “sharing” of personal information or “targeted advertising” under applicable laws. You can opt out of these disclosures and limit our use of tracking technologies as described in our Cookie Policy or by clicking the “Your Privacy Choices” link in our Website footer. In addition, some internet browsers can be configured to send “Do Not Track” signals to the online services that you visit. We currently do not respond to “Do Not Track” or similar signals. To find out more about “Do Not Track,” please visit http://www.allaboutdnt.com."

"If you choose to provide sensitive personal information in your messages and content, we will use that information only to facilitate your conversation with your AI companion and as described in the “Sensitive information” section above. If you do not want us to process your sensitive information for these purposes, please do not provide it. You may request that we delete information you have provided as set out in the “Personal information requests” section below."

"We also offer you choices that affect how we handle the personal information that we control. Depending on your location and the nature of your interactions with our Services, you may request the following in relation to personal information:

Information about how we have collected and used personal information. We have made this information available to you without having to request it by including it in this Privacy Policy.
Access to a copy of the personal information that we have collected about you. Where applicable, we will provide the information in a portable, machine-readable, readily usable format.
Correction of personal information that is inaccurate or out of date.
Deletion of personal information that we no longer need to provide the Services or for other lawful purposes. You can delete your account in your account settings.
Withdrawal of consent, where we have collected and processed your personal information with your consent. Withdrawing your consent will not affect the lawfulness of any processing we conducted prior to your withdrawal, nor will it affect processing of your personal information conducted in reliance on lawful processing grounds other than consent.
Additional rights, such as to object to and request that we restrict our use of personal information.

To make a request, please contact us as provided in the “Contact us” section below. We may ask for specific information from you to help us confirm your identity. Depending on where you reside, you may be entitled to empower an authorized agent to submit requests on your behalf. We will require authorized agents to confirm their identity and authority, in accordance with applicable laws. You are entitled to exercise the rights described above free from discrimination."

"In some instances, your choices may be limited, such as where fulfilling your request would impair the rights of others, our ability to provide a service you have requested, or our ability to comply with our legal obligations and enforce our legal rights. If you are not satisfied with how we address your request, you may submit a complaint by contacting us as provided in the “Contact us” section below. Depending on where you reside, such as if you reside in the European Economic Area or United Kingdom, you may have the right to complain to a data protection regulator where you live or work, or where you feel a violation has occurred."

What is the company’s known track record of protecting users’ data?

Needs Improvement

In February 2023, Replika was ordered by Italy’s privacy watchdog to stop processing local users’ data. The regulator said "Recent media reports along with tests the SA [supervisory authority] carried out on ‘Replika’ showed that the app carries factual risks to children — first and foremost, the fact that they are served replies which are absolutely inappropriate to their age.”

There were reports of Replika algorithm being abusive as a result of abuse encountered from users. For some longtime users of the chatbot, the app has gone from helpful companion to unbearably sexually aggressive.

Child Privacy Information

"The Services are not intended for individuals under the age of 18. If we discover that minors under the age of 18 are using the Apps, we will promptly block their access and delete their account. If you have reason to believe that a minor under the age of 18 has provided personal information to us through the Services, please contact us, and we will endeavor to delete that information from our databases."

Can this product be used offline?

No

User-friendly privacy information?

No

Replika AI's privacy policy leaves quite a few questions unanswered.

Links to privacy information

Does this product meet our Minimum Security Standards? information

No

Encryption

Yes

"Your messages to Replika are processed on the server side, which means that your mobile device encrypts them. They are then sent to our servers, where they are decrypted & processed by Replika’s AI engine. Replika cannot employ end-to-end encryption since your plain text messages must be available to train your personal AI on the server-side." "All transmitted data are encrypted during transmission. We use standard Secure Socket Layer (SSL) encryption that encodes information for such transmissions. All stored data are maintained on secure servers. Access to stored data is protected by multi-layered security controls, including firewalls, role-based access controls, and passwords."

Strong password

No

Managed to sign up with a password '11111111'.

Security updates

Yes

Manages vulnerabilities

Yes

Privacy policy

Yes

Does the product use AI? information

Yes

We cannot confirm if the AI employed at this product is trustworthy, because there is little or no public information on how it works and what user controls exist to make the product safe. At the same time, the potential harm of the apps is high as they collects lots of sensitive data, and use collected data to train AI algorithms.

The app is an AI-chatbot that imitates a real partner. There is evidence that this chatbot cand get abusive.

"Even though talking to Replika feels like talking to a human being, it's 100% artificial intelligence. Replika uses a sophisticated system that combines our own Large Language Model and scripted dialogue content.

Previously Replika also used a supplementary model that was developed together with OpenAI, but now we switched to exclusively using our own which tends to show better results. We put a lot of focus on constantly upgrading the dialog experience, memory capabilities, context recognition, role-play feature and overall conversation quality."

Is this AI untrustworthy?

Yes

What kind of decisions does the AI make about you or for you?

Is the company transparent about how the AI works?

No

We found no documentation or policies that explain how this product's AI works.

Does the user have control over the AI features?

No

A user can set up how Replika will look, what stage the relationship is on, etc. However, we found no details regarding user controls at Replika AI.
*Privacy Not Included

Dive Deeper

  • 5 Things You Must Not Share With AI Chatbots
    Make Use Of Link opens in a new tab
  • Creating a Safe Replika Experience
    Replika Link opens in a new tab
  • ‘Cyber-Heartbreak’ and Privacy Risks: The Perils of Dating an AI
    Rolling Stone Link opens in a new tab
  • AI Friends or Foes? The Privacy Risks for Children with Open AI, ChatGPT and Replika
    Lexology Link opens in a new tab
  • Most therapy apps don’t include privacy; Replika AI ‘worst app ever’
    9to5Mac Link opens in a new tab
  • Man 'encouraged' by AI chatbot 'girlfriend' to kill Queen Elizabeth II receives jail sentence
    EuroNews.next Link opens in a new tab
  • AI girlfriends are ruining an entire generation of men
    The Hill Link opens in a new tab
  • The rise of AI girlfriends is making male loneliness worse and risks ruining a generation of men, a professor says
    Business Insider India Link opens in a new tab
  • Replika AI starts sexually harassing users after being abused by others
    Stealth Optional Link opens in a new tab
  • AI-Based “Companions”​ Like Replika Are Harmful to Privacy And Should Be Regulated
    Medium Link opens in a new tab
  • ‘My AI Is Sexually Harassing Me’: Replika Users Say the Chatbot Has Gotten Way Too Horny
    Vice Link opens in a new tab
  • Men Are Creating AI Girlfriends and Then Verbally Abusing Them
    Futurism Link opens in a new tab
  • Italy bans U.S.-based AI chatbot Replika from using personal data
    Reuters Link opens in a new tab
  • I tried the Replika AI companion and can see why users are falling hard. The app raises serious ethical questions
    The Conversation Link opens in a new tab
  • Replika, a ‘virtual friendship’ AI chatbot, hit with data ban in Italy over child safety
    Tech Crunch Link opens in a new tab
  • What happens when your AI chatbot stops loving you back?
    Reuters Link opens in a new tab
  • Regulator Halts AI Chatbot Over GDPR Concerns
    Infosecurity Magazine Link opens in a new tab
  • Can an AI Save a Life?
    The Atlantic Link opens in a new tab
  • AI-Human Romances Are Flourishing—And This Is Just the Beginning
    Time Link opens in a new tab

Comments

Got a comment? Let us hear it.