Things have gotten a lot better. You were probably using the reference encoder, but there's a newer, much faster encoder: svt-av1.