> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones.

I have been building that for a couple of years now: https://llm.datasette.io