jailbreaking

Jailbreaking and red-teaming: breaking an AI on purpose, before someone else does Lesson

A jailbreak is an input that makes a model do what its training told it to refuse, and red-teaming is the organized practice of hunting for those inputs deliberately, which is how every safety claim about a model gets tested before release.