It seems to me that it would be in YouTube's best interest to provide me with accurate recommendations. The more content they can get me to watch, the more ads I view, which is how they get paid. It would seem that more accurate recommendations would encourage more view time and more add engagement.

Could be; I believe they believe their recommendations are just good enough to get you (or at least their modal user) to watch the ads for which they get paid the most, and if they made more accurate recommendations they'd have to serve ads that didn't pay as well.

Maybe that's too cynical, and accurate recommendations are difficult. I've observed I can do much better by searching for content on my own, so I do.

To put it more baldly:

Your matching problem is: what are the videos I would most like to watch

Youtube's matching problem is: what are the most lucrative videos which the modal user will still watch

Consider the limiting case: if people were willing to watch ads without any content in between, YT would happily serve them exactly that.