Latency is something I notice, but having used Bluetooth since it first came out, 500+ms is either a very noisy radio environment or defective hardware. It’s freakishly outside of normal for the technology - if you were seeing 5ms, maybe 10ms, that’d be within expectations but not even 50ms.
Modern devices should be in the 10-15ms range. If yours are not, return them as defective.
The 5ms I referred to was the difference versus a cable - I’d expect a modern keyboard to be under 10ms and Bluetooth to be under 15ms. Since I’ve measured that the latency for a keystroke to render on screen in VSC at 16-24ms, I’m comfortable saying at least Apple has Bluetooth latency under that level.