Here’s an interesting experiment Google is kicking around on its smart displays: voice-command input without a hotword. A video detailing the feature is up on YouTube from Jan Boromeusz, a Nest Home hacker who has a proven track record of scoring early smart display features before they get announced.

Boromeusz’s Nest Hub Max is somehow in “Dogfood” mode, which means it receives early, non-public builds of the smart display software meant only for internal use at Google. A special menu called “Dogfood features” lists a “Blue Steel” feature that will let the device respond to commands without having to say the “Hey Google” hotword first—you just say a command and it will respond. Boromeusz says the device will listen for commands after “detecting presence,” so if someone is in front of the display, it will just start answering questions.

Today Google’s voice command hardware listens all the time, but only for the “Hey Google” hotword. Once that’s detected, it will start processing additional commands. The more modern implementations also use the hotword as the cutoff point for connecting to the Internet—”Hey Google” detection is processed locally, and anything after that will get uploaded, processed, and stored on Google’s servers. The hotword also acts as a form of consent, not just by having the following words uploaded to the Internet, but also because letting the device listen all the time and respond to every possible thing that could be interpreted as a command would be annoying.

Crazy pills?

It’s not clear how “presence” is being detected for the Blue Steel feature. There is a camera smack on the front of the Home Hub Max, which is used for a “Face Match” feature that can identify a user. if you really want to read too much into the Zoolander-inspired codename, “Blue Steel” is about making a face for the camera. The smaller Nest Hub doesn’t have a front camera though, so this wouldn’t be very scalable across the Nest Hub/Home Hub line. Not to mention that if Google wanted this kind of interaction to be the standard, it would also probably want it to work on smart speakers.

Google also has a more scalable presence-detection feature at its disposal—Ultrasonic Sensing. This is sonar: the speakers pump out inaudible sound and record any bounceback from an object that has moved in front of the device. The sonar-based person detection would probably work on anything with a microphone and speakers, which would be scalable across the entire smart display and smart speaker line.

We should stress that Google is just testing this function for now, and our information is from a leaked, internal build that Google never wanted to show to the public. There are no indications that Blue Steel will start rolling out to consumer devices anytime soon. Getting the balance right on something like this would be absolutely critical and would be the difference between “fast and useful” or “annoying and invasive.”

Listing image by Jan Boromeusz