April 15, 2024

TL;DR: We’ve added on-device wake phrase detection (microWakeWord)! It’s quicker and extra scalable than processing the wake phrase in Dwelling Assistant. We’ll hold supporting wake phrase processing in Dwelling Assistant. Additionally new is extra customization for sentence triggers, further intents for controlling extra units, and higher error messages and debugging instruments.

Watch the total Voice chapter 6 livestream

2023’s Yr of the Voice constructed a stable basis for letting customers management Dwelling Assistant by talking in their very own language.

We proceed with enhancements to Help, together with:

Oh, and “yet another factor”: on-device, open supply wake phrase detection in ESPHome! 🥳🥳🥳

Take a look at this video of the brand new microWakeWord system working on an ESP32-S3-BOX-3 alongside one doing wake phrase detection inside Dwelling Assistant:

On-device vs. streaming wake phrase

microWakeWord

Thanks to the incredible microWakeWord created by Kevin Ahrendt, ESPHome can now carry out wake phrase detection on units just like the ESP32-S3-BOX-3.
You may set up it in your S3-BOX-3 right this moment to strive it out.

Again in Chapter 4, we added wake phrase detection utilizing openWakeWord. Sadly, openWakeWord was too giant to run on low energy units like S3-BOX-3.
So we selected to run wake phrase detection inside Dwelling Assistant as an alternative.

Doing wake phrase detection in HA permits tiny units just like the M5 ATOM Echo Improvement Package to easily stream audio and let the entire processing occur elsewhere. That is nice, because it permits low-powered units utilizing a easy ESP32 chip to be remodeled right into a voice assistant even when they don’t pack the mandatory energy to detect wake phrases.
The draw back is that including extra voice assistants requires extra CPU utilization in HA in addition to extra community site visitors.

Enter microWakeWord. After listening to an interview with Paulus Schoutsen (founding father of Dwelling Assistant) on the Self Hosted podcast, Kevin Ahrendt created a mannequin based mostly on Google’s Inception neural network. As an present contributor to ESPHome, Kevin was in a position to get this new mannequin working on the ESP32-S3 chip contained in the S3-BOX-3! (It additionally works on the, now discontinued, S3-BOX and S3-BOX-Lite)

Kevin has educated three models for the launch of microWakeWord:

  • “okay nabu”
  • “hey jarvis”
  • “alexa”

You may strive these out your self now by following the ESP32-S3-BOX tutorial. Altering the default “okay nabu” wake phrase would require adjusting your ESPHome configuration and recompiling the firmware, which can take a very long time and requires a machine with greater than 2GB of RAM.

We’re grateful to Kevin for growing microWakeWord, and making it part of the open residence!

Sentence set off responses

Including customized sentences to Help is as straightforward as including a sentence trigger to an automation. This allows you to trigger any action in Home Assistant with whatever sentences you want.

Now with the new conversation response action in HA 2024.2, you can also customize the response spoken or printed back to you. Using templating, your response can refer to the current state of your home.

You can also refer to wildcards in your sentence trigger. For example, the sentence trigger:

could have the response:

Playing  trigger.slots.album  by  trigger.slots.artist 

in addition to calling a media service.

You can experiment now with sentence triggers, and custom conversation responses in our automation editor by clicking here:
Open your Home Assistant instance and show your automations.

Improved errors and debugging

Assist users know the phrase “Sorry, I couldn’t understand that” all too well. This generic error message was given for a variety of reasons, such as:

Beginning in HA 2024.2, Help offers totally different error messages for every of those instances.

Screenshot showing the new errors Assist will return in case the intention is understood, but something else is missing.

Now should you encounter errors, you’ll know the place to start out trying! The very first thing to test is that your system is uncovered to Help. Some sorts of units, similar to lights, are uncovered by default. Different, like locks, aren’t and have to be manually uncovered.

As soon as your units are uncovered, ensure you’ve added an acceptable alias so Help will know precisely the way you’ll be referring to them. Units and areas can have a number of aliases, even in a number of languages, so everybody’s desire might be accommodated.

In case you are nonetheless having issues, the Help debug device has additionally been improved. Utilizing the device, you see how Help is decoding a sentence, together with any lacking items.

Open your Home Assistant instance and show your Assist developer tools.

Our group language leaders are exhausting at work translating sentences for Help. If in case you have options for brand spanking new sentences to be added, please create a difficulty on the intents repository or drop us a line at [email protected]

Thank you

Thank you to the Home Assistant community for subscribing to Home Assistant Cloud to help voice and improvement of Dwelling Assistant, ESPHome and different tasks typically.

Due to our language leaders for extending the sentence help to all the assorted languages.

Thank you for supporting the Home Assistant project