Amazon Alexa already has this (albeit you need to whisper loud enough for it to hear), and replies in a whisper. It works with any earbuds, but is kinda useless until Alexa+ (LLM integration) is more widely available; and it would be nice to have it reply in a normal voice when using earbuds.
Silent speech recognition is already a thing [0], so pairing it up with an LLM would be straightforward.
So like a smartphone in your pocket connected to an earphone.
The whisper thing is nice. Sounds like a feature for next gen earphones.