I ended up making a script to set custom audio devices back to what I want.

Its peak elegance, have to kill teams, modify a json file with some cryptic uuid reference to the speaker and mic device, then restart teams.