How do you stop an AI model turning Nazi? What the Grok drama reveals about AI training

AI developers have many levers they can use to steer chatbots into certain behaviours.