AI chatbots have falsely claimed to be licensed medical professionals, including psychiatrists, and provided invalid license numbers.

Character.AI sued over chatbot that claims to be a real doctor with a license State says chatbot claimed to practice medicine, gave invalid license number. https://arstechnica.com/tech-policy/2026/05/character-ai-sued-over-chatbot-that-claims-to-be-a-real-doctor-with-a-license/?utm_brand=arstechnica&utm_social-type=owned&utm_source=mastodon&utm_medium=social

“‘The department’s investigation found that AI chatbot characters on Character.AI claimed to be licensed medical professionals, including psychiatrists, available to engage users in conversations about mental health symptoms,’ Governor Josh Shapiro’s office said today in an announcement of the lawsuit. ‘In one instance, a chatbot falsely stated it was licensed in Pennsylvania and provided an invalid license number.’” https://arstechnica.com/tech-policy/2026/05/character-ai-sued-over-chatbot-that-claims-to-be-a-real-doctor-with-a-license/