I’ve worked on rails apps for a while now and on previous jobs it wasn’t so bad, but those were smaller apps. At the current company, almost every update manages to slip something through CI that blows up on production. Even with 30,000 rspec tests and tons of cypress tests. The CI is set up to raise exceptions on warnings, but I think it only applies to warnings raised by rails.

I guess rails works alright at the start, but eventually your app gets so big that it becomes a nightmare to verify updates before they roll out.

30,000+ tests and it can't detect breaking changes?

In this case no because the broken code was in another gem (sidekiq) using the redis gem. Our CI does not run the unit tests for every library we import. Only our own.

Do you not have tests for your Jobs?

Yes, we do. In this case the issue was very much inside some internal logic of sidekiq around managing the jobs, not any of our own code.