Artificial Intelligence1 min read

nemotron-halloween

By · Published by Everything Blog

In short

The interactive garage door uses local hardware to process incoming visitors and dispense treats based on their social interactions. A suite of open models handles the real-time audio pipeline, capturing spoken requests, writing custom responses, and synchronizing facial movements. Visual perception models like YOLO, ByteTrack, and Nemotron 3 Nano Omni track arrivals and analyze costumes. Through this multi-model integration, the system creates an engaging off-grid performance driven by visitor politeness.

Key points

  • The MonsterHouse installation alters its personality from cantankerous to warm when trick…: The MonsterHouse installation alters its personality from cantankerous to warm when trick-or-treaters use words like please and thank you.
  • Dialogue generation relies on NVIDIA Nemotron 3.5 Bolt E4B paired with a custom LoRA adap…: Dialogue generation relies on NVIDIA Nemotron 3.5 Bolt E4B paired with a custom LoRA adapter for pirate-themed responses.
  • NVIDIA Magpie TTS Zeroshot generates the pirate ship persona using voice synthesis derive…: NVIDIA Magpie TTS Zeroshot generates the pirate ship persona using voice synthesis derived from a short reference audio sample.

Sources checked by our analysis pipeline: 1

Building a Haunted House with NVIDIA Nemotron Open Models

Or rather…

Boo! Now Get Off My Lawn!

TL;DR: MonsterHouse is an interactive projection I created for Halloween. He listens to trick-or-treaters, answers in the voice of a grumpy old pirate ship and softens when visitors remember their manners. He is built on the NVIDIA Nemotron Voice Agent with NVIDIA Nemotron ASR, Nemotron 3.5 Bolt E4B 4GB, Magpie TTS Zeroshot and Audio2Face-3D, and he runs on a single workstation.

I wanted the kids visiting my house at Halloween to interact in a fun and unique way. The house was going to be in charge and giving out the treats, but more grumpy than scary. Somehow, my solution was to give the garage door a mouth and teach him to complain.

That became MonsterHouse: a grumpy old building with pirate stories, a fiery grin and a suspicious weakness for “please.” NVIDIA Nemotron Voice Agent supplies the conversation foundation. Nemotron Speech does the listening, Bolt E4B is the brain, NVIDIA Magpie TTS Zeroshot is the live voice, and NVIDIA Audio2Face-3D makes the wooden…

Artificial Intelligence