Have you tried Google's Gemma-3n-E4B-IT in their AI Edge Gallery app? It's the first local model that's really blown me away with its power-to-speed ratio on a mobile device.

See: https://github.com/google-ai-edge/gallery/releases/tag/1.0.3

Dozens of locally runnable models can already do that.

Yes, I always evaluate models on my own prompts and use cases. I glance at evaluation postings but I am also only interested in my own use cases.

I heard the OSSmodels are terrible at anything other than math, code etc.