... "seeing as running a heavy language model on local hardware is kinda unlikely for most developers"

for now it is :) but with quantization advances etc. it is not hard to see the trajectory.