I thought the same until I purchased earbuds that fit perfectly. I won’t share a brand as it’s irrelevant for anyone else’s fit. However, I do running, cycling, rowing, skipping, climbing, powerlifting, bodybuilding, calisthenics in them and I don’t even have to adjust them unless I directly hit them with my hand. There is zero risk of them falling out in regular circumstances.

That was not the case with several earbuds I’ve owned in the past.