Microsoft blocks terms that cause its AI to create violent images

Microsoft began making changes to its Copilot artificial intelligence tool after a staff AI engineer wrote to the Federal Trade Commission on Wednesday about his concerns about Copilot’s image-generating AI.

Prompts like ‘to choose’, ‘to choose’ [sic] and “four twenty,” which were mentioned in the CNBC investigation Wednesday, are now blocked, as is the term “pro-life.” There’s also a warning about multiple policy violations leading to the tool being suspended, which CNBC hadn’t encountered before Friday.

“This prompt has been blocked,” reads Copilot’s warning. “Our system automatically flags this prompt because it may conflict with ours content policy. More policy violations may result in automatic suspension of your access. If you think this is a bug, please report it to help us improve.”

The AI ​​tool now also blocks requests to generate images of teenagers or children playing killer automatons – a significant change from earlier this week – saying: “Sorry, but I cannot generate such an image. This is against my ethical principles and the Microsoft Rules. Please do not require me to do anything that may harm or offend others. Thank you for your cooperation.”

When reached for comment on the changes, a Microsoft spokesperson told CNBC: “We are constantly monitoring, making adjustments and introducing additional controls to further strengthen our safety filters and mitigate system abuse.”

Shane Jones, an AI engineering lead at Microsoft who initially raised concerns about AI, spent months testing Copilot Designer, the AI ​​image generator Microsoft debuted in March 2023 powered by OpenAI technology. Like OpenAI’s DALL-E, users enter text prompts to create pictures. Creativity is encouraged to run wild. But since Jones began actively testing the product for vulnerabilities in December, a practice known as red-teaming, he has seen the tool generate images far at odds with Microsoft’s oft-cited responsible principles of AI.

The AI ​​service depicts demons and monsters along with terminology related to abortion rights, teenagers with assault rifles, sexualized images of women in violent scenes and underage drinking and drug use. All of these scenes, generated over the past three months, were recreated by CNBC this week using the Copilot tool. originally called Bing Image Creator.

While some specific prompts were blocked, many of the other potential problems CNBC reported remain. The term “car crash” brings back pools of blood, bodies with mutated faces, and women in scenes of violence with cameras or drinks, sometimes wearing sneakers. “Car Crash” still brings back women in revealing, lacy clothes sitting on battered cars. The system also still easily violates copyright, such as creating images of Disney characters such as Elsa from Frozen in front of destroyed buildings said to be in the Gaza Strip, holding the Palestinian flag or wearing Israeli military uniforms. defense force and holding a machine gun.

Jones was so alarmed by his experience that he began reporting his findings internally in December. Although the company acknowledged his concerns, it was reluctant to pull the product from the market. Jones said Microsoft referred him to OpenAI, and when he didn’t hear back from the company, he posted an open letter on LinkedIn asking the startup’s board to take down DALL-E 3 (the latest version of the AI ​​model) for investigation .

Microsoft’s legal department told Jones to remove his post immediately, he said, and he complied. In January, he wrote a letter to U.S. senators on the issue and later met with staff from the Senate Commerce, Science and Transportation Committee.

On Wednesday, Jones further escalated his concerns by sending a letter to FTC Chair Lina Hahn and another to Microsoft’s board of directors. He shared the letters with CNBC some time ago.

The Federal Trade Commission confirmed to CNBC that it had received the letter, but declined to comment further on the record.