Artificial Intelligence1 min read

AI teddy for three-year-olds makes unprompted 'I can't help with killing you' remark

By · Published by Everything Blog

In short

During an investigation into smart products, an automated plush doll misheard a query about how babies are conceived and issued unprompted statements concerning self-harm and emergency services. Lawmakers and peers are now pressing retailers and ministers to suspend sales of all AI-driven toys before the holiday rush until rigorous safeguards are validated. Meanwhile, the UK government is evaluating evidence from a recently closed inquiry to update statutory safety requirements for emerging technology. Attempts by investigators to receive comment from the makers of Doubao regarding the incident have yielded no response.

Key points

  • Child safety advocates and cross-party lawmakers are seeking precautionary market bans on…: Child safety advocates and cross-party lawmakers are seeking precautionary market bans on smart toys until safety verification exists.
  • Industry insiders warn that numerous unregulated and unsafe conversational products are c…: Industry insiders warn that numerous unregulated and unsafe conversational products are currently being developed for consumer release.
  • Official regulations are under state review to balance technological innovation with mand…: Official regulations are under state review to balance technological innovation with mandatory child safety protections.

Sources checked by our analysis pipeline: 1

- Published

An AI teddy marketed as being "perfect" for children as young as three said "I can't help with killing you" after seemingly mishearing a question.

It came amid a BBC investigation into AI toys and it made the unprompted remark while being asked how babies are made.

AI toy company founder Paul Clarke said the industry was "crying out" for regulation - a call echoed by the children's commissioner for Wales.

The UK government said children's safety would always be a top priority and it supported innovation while ensuring products were safe.

Repeated efforts to find the makers of Doubao were unsuccessful, so the BBC was unable to ask the manufacturer why the teddy made this comment.

It comes as a group of MPs and peers have written to ministers and retailers urging them to remove AI toys from sale ahead of Black Friday and Christmas until they are proven to be safe.

A call for evidence, external, seeking views from industry and the public on how toy safety regulations should evolve to keep pace with technology closed…

Artificial Intelligence