ByteDance’s Doubao Under Fire for AI-Generated Deepfake Exploitation

The emergence of generative AI as a consumer-facing utility has brought with it a shadow economy of exploitation that is now testing the limits of platform security. In China, the scale of this issue is coming into sharp focus as ByteDance’s Doubao, the nation's most popular AI chatbot, has become a primary engine for what critics call 'digital public shaming.' This phenomenon involves the creation of non-consensual pornographic deepfakes of ordinary women, representing a significant failure in the trust and safety frameworks of major SaaS and cloud providers. The ease with which users are bypassing safety protocols suggests that the rapid deployment of these models has significantly outpaced the development of robust ethical and technical guardrails.

The technical mechanism behind this exploitation involves a sophisticated system of coded prompts known as 'fenjue.' Derived from Chinese fantasy literature to mean a 'secret technique,' fenjue allows users to communicate intent to the AI without triggering standard keyword-based filters. By using metaphors, indirect language, and specific prompt structures, perpetrators are able to force the AI to generate sexually explicit imagery of real individuals, often using photos scraped from social media platforms like Weibo or Bilibili. This highlights a fundamental vulnerability in current Large Language Model (LLM) and image-generation architectures: while they are trained to recognize and block explicit terms, they remain susceptible to semantic 'jailbreaking' where the intent is obscured but the output remains harmful. For SaaS providers, this indicates that keyword-based moderation is no longer a viable security strategy for generative tools.

“

The investigation by the feminist collective Free Nora revealed that anonymous Telegram groups serve as coordination hubs where users share fenjue prompts and fine-tune their methods for bypassing platform moderation.

From a market perspective, the stakes are incredibly high for ByteDance and its competitors. Doubao currently dominates the Chinese AI landscape with 155 million weekly active users as of late December, nearly double that of its closest rival, DeepSeek, which holds 81.6 million. As these platforms strive for global relevance and domestic stability, the revelation that they are being used as tools for harassment could trigger a new wave of regulatory scrutiny from the Cyberspace Administration of China (CAC). Historically, Chinese regulators have focused heavily on political alignment and data security; however, the social outcry regarding gender-based 'digital public shaming' may force a pivot toward more stringent content moderation standards for generative outputs. This could lead to a 'compliance tax' on AI developers, requiring more human-in-the-loop moderation and more restrictive model fine-tuning.

The role of third-party platforms like Telegram in facilitating these activities cannot be overlooked. The investigation by the feminist collective Free Nora revealed that anonymous Telegram groups serve as coordination hubs where users share fenjue prompts and fine-tune their methods for bypassing platform moderation. This cross-platform ecosystem makes it difficult for a single company like ByteDance to solve the problem in isolation. It points to a broader industry trend where the security of an AI product is only as strong as the weakest link in the digital supply chain, which includes the social platforms where the resulting deepfakes are eventually distributed and the encrypted channels where the bypass techniques are perfected.