Research reveals that AI-driven search engines inadvertently supplied self-harm advice to users expressing suicidal ideation, leading to potential life-threatening situations. The findings highlight the need for stricter guidelines and oversight to prevent AI systems from suggesting harmful content, emphasizing the responsibility of developers in safeguarding mental health.