AI deepfakes spur calls for more control

An artificial intelligence-generated deepfake of Chinese actress Wen Zhengrong's face and voice was used by unscrupulous merchants to impersonate her in livestream sales, prompting increased calls for stronger and more tailored regulation and penalties from internet platforms and the law.

The discovery was made last week when Wen appeared to simultaneously host three different early morning livestream rooms on social media, wearing different outfits and promoting different products. The Wen "clones" looked and sounded strikingly similar to Wen, a feat that quickly ignited online discussion.

According to a China Media Group report on Wednesday, the forged images were produced either by clipping past videos and screen recordings or by taking earlier livestream footage of Wen and running it through AI-based deep synthesis, including voice alteration.

"These AI tactics confuse the public. My image and likeness have been infringed, and it is deeply hurtful," Wen said in the video report. She added that if viewers who trust her were misled into buying counterfeit goods, "I would feel truly sad."

Li Ya, a partner at Zhongwen Law Firm in Beijing, told China Daily that such conduct was suspected of violating Wen's right of portrait and may also harm her right of reputation.

Using someone's image for profit without authorization infringes on portrait rights, he said. "If sellers speak in her name and make false or exaggerated claims, that will negatively impact a public figure's reputation."

Wen's team said that once the fake clips began circulating, they filed reports around the clock, flagging about 50 impersonation accounts in one day, according to CMG.

Some livestreaming accounts were taken down, they said, but others quickly reappeared in new forms. Wen's staff noted that certain merchants can fabricate content by extracting brief footage and relying on AI functions built into video-editing apps, while the team faces a much higher burden to preserve evidence and defend their rights.

Li said it is unrealistic to expect victims alone to safeguard their rights.

"Rule-breaking merchants can open new accounts at will and face almost no cost for infringement," he added.

He noted that social platforms have a duty to deploy technology to detect whether AI tools are being used improperly in livestreams or short videos, and to penalize offending accounts as well as the operating companies and teams behind them, in order to prevent harm to third parties.

In September, new regulations on labeling AI-generated synthetic content, released by the Cyberspace Administration of China and other agencies, took effect. The rules require clear "AI-generated" labels on synthetic faces and videos.

In practice, however, some merchants hide labels in obscure locations or mask them, allowing infringements to slip through reviews from the social platforms.

"If a video lacks the necessary authorization and fails to show an AI-generation label, using someone else's face or voice in a livestream is a textbook case of infringement," Li said. "Victims can sue the merchant and even the platform, which bears responsibility for keeping online order."

On Thursday, Douyin's e-commerce safety and trust center said on its official social account that it had launched a special campaign in October targeting infringements. Those included cases where merchants or content creators, without permission, spliced videos, blended text, or used AI and digital effects to mimic celebrities for sales. The short-video platform's response to these actions included cutting livestreams, removing or banning involved products, freezing transaction proceeds, and ordering business suspensions, it said.

Since the campaign began, the platform has acted against 11,000 creator accounts involved in impersonations and taken down more than 6,700 products, it said. In addition, more than 10,000 infringing videos faking Wen's likeness and voice for marketing were removed.

Li Liang, a deputy president of Douyin Group, said on his account on Thursday that such AI-involved impersonation "seriously undermines the credibility of creators, merchants, and the platform in the eyes of consumers, and runs counter to our long-term interests".

Li denied that the livestreams impersonating Wen took place on Douyin, adding that AI content infringement detection is an "industry-wide challenge, and malicious impersonators constantly engage in a technical cat-and-mouse with platforms."

"We will keep investing to meet the challenge and protect the lawful rights of creators, merchants, and consumers," he said.

Li Wei, deputy dean of the School of New Media at Peking University, said AI-enabled deepfakes are "the most typical trap" in AI-assisted content production, involving infringement, copyright, ethics, and online abuse, with serious consequences.

She called for more systematic, multistakeholder cyberspace governance, especially stronger legal safeguards and platform oversight, and urged more tailored penalty tools to curb such practices.

jiangchenglong@chinadaily.com.cn