My hope was to find a system which does ASR, then LLM processing with MCP use and finally TTS: "Put X on my todo list" / "Mark X as done" -> LLM thinks, reads the todo list, edits the todo list, and tells me "I added X to your todo list", ... "Turn all the lights off" -> llm thinks and uses MCP to turn off the lights -> "Lights have been turned off". "Send me an email at 8pm reminding me to do" .... "Email has been scheduled for 8pm"
That's all I want. It does not have to be fast, but it must be capable of doing all of that.
Oh, and it should be energy efficient. Very important for a 24/7 machine.
You can already do that on most desktop GPU's (even going as far as prev gen Nv 1050/1060/1070 for example).
You'll need a model able to work with tools, like llama 3.2 (https://huggingface.co/meta-llama), serve it, hook up MCPs, include a STT interface, and you're cooking.
Even a bottom of the barrel N95 has audio acceleration features helping with speech to text, but the LLM inference part still will be far from being efficient.
Plus, you need to keep the card at "ready" state, you can't idle/standby it completely.
That's all I want. It does not have to be fast, but it must be capable of doing all of that.
Oh, and it should be energy efficient. Very important for a 24/7 machine.