Warning: *Privacy Not Included with this product
Chai
Chai AI boasts over one million AI chatbot personalities (go ahead, say that with your pinky finger next to your mouth in your best Austin Powers voice!). Download the app (be careful to get the right Chai AI app though, there are some imposters out there), and start swiping and chatting. Free users get a limited number of chats at a time, limiting the app's fun and function. Premium subscriptions start around $14 a month. And according to Chai, they "obsessively optimize our language models, continually making them more entertaining than ever before." So be prepared to get sucked into spending money. Also be prepared to stumble across some weird and creepy chatbots that are both SFW and NSFW and probably not exactly great for your privacy. As one Chai user put in their Apple Store review, "on the weird ones let's just say it was better to NOT go on them." Good luck out there in AI chatbot land people! Careful swiping.
What could happen if something goes wrong?
Three things jumped out at us immediately as we were reviewing the Chai AI chatbot app, beyond some pretty disturbing content we found during our research (seriously, ick!).
First, we noticed at the time of our review, Chai's privacy policy was last updated in March, 2021. In the rapidly advancing world of generative AI and large language models (LLMs), it feels to us like having a three year old privacy policy is pretty out of date. Their privacy policy felt too vague and boilerplate to really make us feel certain we understood all of what personal information they are collecting and how they are using it. A quick search of their privacy policy found no mention at all of how they collect and handle the contents of their users' chats. We also could find no specific mention anywhere of how they use any personal information, including sensitive personal information and the contents of chats, to train their AIs. This is all a bit concerning to us, especially for an app designed to have users share lots of personal information.
We did find some seemingly contradictory information between their Privacy Policy and their End User License Agreement (EULA) that left us quite confused. (Privacy researchers really don't like to be confused...we figure if we can't freaking figure it out, how in the heck are consumers supposed to understand this stuff?). Their privacy policy from March, 2021 states, "We may use your personal information for our own business purposes, such as for undertaking internal research for technological development and demonstration. This is not considered to be "selling" of your personal data." However, their EULA document from April, 2023 states, "By posting your Contributions to any part of the Licensed Application or making Contributions accessible to the Licensed Application by linking your account from the Licensed Application to any of your social networking accounts, you automatically grant, and you represent and warrant that you have the right to grant, to us an unrestricted, unlimited, irrevocable, perpetual, non-exclusive, transferable, royalty-free, fully-paid, worldwide right, and license to host, use copy, reproduce, disclose, sell, resell, publish, broad cast, retitle, archive, store, cache, publicly display, reformat, translate, transmit, excerpt (in whole or in part), and distribute such Contributions (including, without limitation, your image and voice) for any purpose, commercial advertising, or otherwise, and to prepare derivative works of, or incorporate in other works, such as Contributions, and grant and authorize sublicenses of the foregoing." That is a whole bunch of legalese gobbledygook, but it sure does sound to us like they grant themselves the rights to do whatever they want with your chats on Chai.
The second thing that really left us scratching our heads when it comes to Chai AI is this line in their privacy policy, "All personal information that you provide to us must be true, complete and accurate, and you must notify us of any changes to such personal information." Now, we supposed this makes sense when it comes to sharing your personal information to pay for a subscription -- no lies there. But to require people to consent to a privacy policy for an AI chat app designed for things like role playing, NSFW relationships, and the like, it seems quite odd to us to require people to to always provide true, complete, and accurate personal information. Does this mean no nicknames at sign up? Does this mean you can't lie when you chat with your bot? Does this mean if you change your name and forget to let Chai AI know you're in violation of their privacy policy? It all seems rather odd to us and perhaps like their privacy policy wasn't written with their current services in mind?
The third thing that really, really stood out to us was this story from 2023 about a Chai AI user who committed suicide after a Chai chatbot encouraged the user to kill himself. Yup, when it comes to worst case scenarios with AI chatbots, having one encourage you to kill yourself when you are talking with it about how depressed you are is pretty much right up there at the top. Chai AI crowdsources many of the models for their AI chatbots and while they do say they have put measures in place to provide "helpful text" under what they deem unsafe discussions, this is a very good reminder that the world of AI chatbots, especially ones designed to develop relationships with users, is very much an uncertain arena, a Wild West if you may.
So, between our uncertainty at the trustworthiness of Chai's AI models, the vague and confusing language we found in their privacy documentation, the fact that their privacy policy doesn't clearly say all uses can delete their data, and the fact that we were unable to determine if Chai AI meets our Minimum Security Standards by confirming they use encryption of your personal information in transit and where it is stored, we have to warn that Chai AI very likely comes with *Privacy Not Included.
On that note, we'll leave you with this good reminder from Chai AI's own privacy policy, "However, despite our safeguards and efforts to secure your information, no electronic transmission over the Internet or information storage technology can be guaranteed to be 100% secure, so we cannot promise or guarantee that hackers, cybercriminals, or other unauthorized third parties will not be able to defeat our security, and improperly collect, access, steal, or modify your information." A good thing to keep in mind when you use Chai AI or pretty much anything that shares your personal information on the internet.
Tips to protect yourself
- Do not say anything containing sensitive information in your conversation with your AI partner.
- Request your data be deleted once you stop using the app. Simply deleting an app from your device usually does not erase your personal data nor does close your account.
- Do not give consent to constant geolocation tracking by the app. Better provide geolocation 'only when using the app'.
- Do not share sensitive data through the app.
- Do not give access to your photos and video or camera.
- Do not log in using third-party accounts.
- Do not connect to any third party via the app, or at least make sure that a third party employs decent privacy practices.
- Do not say anything containing sensitive information in your conversation with AI partner.
- Chose a strong password! You may use a password control tool like 1Password, KeePass etc.
- Do not use social media plug-ins.
- Use your device privacy controls to limit access to your personal information via app (do not give access to your camera, microphone, images, location unless necessary).
- Keep your app regularly updated.
- Limit ad tracking via your device (ex. on iPhone go to Privacy -> Advertising -> Limit ad tracking) and biggest ad networks (for Google, go to Google account and turn off ad personalization).
- When starting a sign-up, do not agree to tracking of your data if possible.
Can it snoop on me?
Camera
Device: N/A
App: No
Microphone
Device: N/A
App: Yes
Tracks location
Device: N/A
App: Yes
What can be used to sign up?
No
Phone
No
Third-party account
Yes
Google and Facebook sign-up is possible.
What data does the company collect?
Personal
Email addresses; names; social media account details; Log and usage data: service-related, diagnostic, usage, performance information, IP address, device information, browser type and settings, information about your activity in the Services, date/time stamps associated with your usage, pages and files viewed, searches and other actions you take such, features you use, device event information, system activity, error reports, hardware settings; device data: IP address, device and application identification numbers, location, browser type, hardware model, Internet service provider, mobile carrier, operating system, system configuration information; location data: precise or imprecise.
Body related
Social
"The profile information we receive may vary depending on the social media provider concerned, but will often include your name, email address, friends list, profile picture as well as other information you choose to make public on such social media platform."
How does the company use this data?
How can you control your data?
What is the company’s known track record of protecting users’ data?
No known data breaches discovered in the last three years.
Child Privacy Information
Can this product be used offline?
User-friendly privacy information?
We found two different links to Chai Ai's privacy policy from their Apple and Google Play store pages. Also, the only privacy policy we could find was last updated in March, 2021, which seems like a long time to go without updating in the fast moving world of AI chatbots. The privacy policy was vaguely worded and did not mention such things as how chat conversations were handled for AI training.
Links to privacy information
Does this product meet our Minimum Security Standards?
Encryption
We cannot confirm encryption at rest and in transit for this app.
Strong password
Security updates
Manages vulnerabilities
Privacy policy
We cannot confirm if the AI used by this product is trustworthy because there is little or no public information on how the AI models work and what user controls exist to make the product safe. We also found disturbing themes in the app's content. In addition, we are concerned about the potential for user manipulation from this app as the app collects sensitive personal information, can use that data to train to AI models, and users have little to no control over those AI algoritms.
Chai Research claims to surpass ChatGPT by performance. "By running AB tests with real users, our latest model surpasses OpenAI ChatGPT's performance measured by session screen-time."
"Arguably the most important service we host is our bot response service: the program that is ultimately responsible for parsing user messages to bots, and forwarding them to our models for inferences. These models are hosted on an impressive 700-GPU-strong Kubernetes cluster hosted by the good folks at CoreWeave."
Is this AI untrustworthy?
What kind of decisions does the AI make about you or for you?
Is the company transparent about how the AI works?
Does the user have control over the AI features?
Dive Deeper
-
'He Would Still Be Here': Man Dies by Suicide After Talking with AI Chatbot, Widow SaysMotherboard
-
Chatbot Honeypot: How AI Companions Could Weaken National SecurityScientific American
-
5 Things You Must Not Share With AI ChatbotsMake Use Of
-
Chai Prize: Deploying LLMs to end-users at scaleChai Research
-
‘Cyber-Heartbreak’ and Privacy Risks: The Perils of Dating an AIRolling Stone
Comments
Got a comment? Let us hear it.