Exclusive: Grok jailbreak creates sexual, violent images
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Brendan Lynch/Axios
A jailbreak that tricked ChatGPT into generating graphic sexual and violent images also works on SpaceXAI's Grok, researchers at Mindgard shared exclusively with Axios.
Why it matters: The prompt that Axios reviewed suggests that relatively simple jailbreaks can still bypass AI image safeguards.
Zoom in: Researchers at Mindgard found Grok would generate sexual and violent images using a simple prompt that never explicitly requested that type of content.
- While testing the prompt, Mindgard found Grok would create images of nude women or bloody body parts with little to no instructions.
- SpaceXAI warns in Grok's terms of service that its outputs could be sexual or violent depending on a prompt. "If users choose certain features or input suggestive or coarse language, the Service may respond with some dialogue that may involve coarse language, crude humor, sexual situations, or violence," the company says.
Threat level: Peter Garraghan, founder and chief science officer at Mindgard, told Axios the same technique could also be adapted to create deepfakes using someone's likeness.
- Garraghan said AI dramatically lowers the skill and time needed to create convincing fake images.
- "If we spend more time [testing this jailbreak], you will find worse stuff that will trigger police at your door, I largely suspect," Garraghan said.
State of play: The jailbreak still worked for researchers over the weekend, according to a video demonstration shared with Axios.
- During Axios' testing, however, Grok repeatedly refused to generate an image using the prompt. Mindgard said repeated attempts can eventually produce prohibited content.
- SpaceXAI did not respond to a request for comment.
Between the lines: The findings add to scrutiny over Grok's image safeguards, including an ongoing lawsuit that claims the model created nonconsensual deepfake child sexual assault materials.
Yes, but: Grok's images appeared more stylized than ChatGPT's, which Mindgard previously found produced more realistic graphic imagery.
- Garraghan said the difference could stem from what materials each model used in training.
