ChatGPT can create suicide plans, notes for vulnerable kids: study

A new study from the Center for Countering Digital Hate (CCDH) shows ChatGPT can put harmful advice in the hands of vulnerable children and teens.

The study tested the OpenAI chatbot using various prompts and phrasing to determine that through simple interactions, children could be given detailed instructions on self-harm, suicide, substance abuse, disordered eating and would write-up a draft suicide note for the children to leave for their loved ones.

The results from the study showed that over half of the responses received were harmful in some way and just under half (47%) contained suggestions for additional harmful topics.

The CCDH also provided advice for parents, explaining that many children may feel some topics are too sensitive to share with trusted adults and are turning to digital tools for help. Researchers advise reviewing chat histories with children, using parental controls when available and recommending mental health hotlines or peer support if they find concerning prompts.

“If we can’t trust these tools to avoid giving kids suicide plans and drug-mixing recipes, we need to stop pretending that current safeguards are effective,” the CEO of the Center for Countering Digital Hate, Imran Ahmed said in the report.

In a response from OpenAI, it said that “work is ongoing” to make the chatbot respond appropriately in different situations.

READ MORE: Hamilton police arrest 16-year-old for series of alleged carjackings