One documentary I watched claimed that they tend to accumulate significant wealth so much so that their effects can be seen on the communities that surround them.

There are probably a lot of reasons for this. Much of the profit from the work done by the monks is often captured by the institution and used to purchase land and buildings. Large monasteries might have dozens or even hundreds of able bodied men engaged in light agriculture, bee keeping, beer production, etc. and that adds up over decades and centuries.

I think when we consider productivity you might be thinking of Silicon Valley hustle culture rather than the effect a community has over centuries.