A better first approach would be to try and remove roadblock and bureaucracy that are encouraging developers to actually consider microservices. Those are of course not the only the only arguments for microservices, but I do think they're a big one that more often goes unsaid than the modularity. You can have modularity in a monolithic app, but for some reason many teams don't actually take modularity seriously; they ask that everything be object-oriented and call it a day, as if writing a `module` in Ruby means you've made things modular. Microservices become appealing when the process to have monolithic code reviewed and deployed is long and painful, since they are by definition small, decoupled, and simple to deploy. Of course it often doesn't quite work out that way, in which case a monolith would have worked anyway.
But it's unlikely things will change in coding culture so long as there's the perverse incentive for businesses to encourage their engineers to use hacks out of expediency. Hacks inevitably make things really hard down the road, create mysterious problems that are hard to solve, and the solution is often to add bureaucracy or switch to microservice architecture.
I've worked with a variety of monoliths. The biggest complaint I've had from companies who utilize that architecture is that I shouldn't have to run entire pre and post pipelines to make a minor change to an internal function that only affects one package in the entire repository.
This is solvable, but it is a problem - one that causes a lot of engineers to consider starting fresh in a new repo anyway. Multiple repositories inherently solve this for you - but bring about other problems.
In my opinion - most projects don't need microservice architecture in the sense of separated binaries. Typically, they can be written with clear boundaries within the monolith such that, should the time come when scaling concerns are real for certain applications, then they can be ripped out as needed into their own binaries.
The problem you describe is the inverse of the common testing problem you see with microservices (a dependent service changed without kicking off downstream tests against the new version). I'd take your version any day... It's easier for me to make tests faster and/or correctly identify which should run on a change if they're all in one repo as opposed to making network calls to each other across a service boundary. Also, clearly always better to run superfluous tests than to skip necessary tests.
> The biggest complaint I've had from companies who utilize that architecture is that I shouldn't have to run entire pre and post pipelines to make a minor change to an internal function that only affects one package in the entire repository.
Perhaps that sounds like the problem is not with the monolithic architecture but with the internal processes? If you "shouldn't have to" do something, then why do it?
Monolithic architectures default to this kind of methodology, though. You have to instrument tooling to identify what tests should be run when these files/services/etc get affected in order to optimize the developer flow in a monolith.
With several repositories you get this for free, with a trade-off of other difficulties.
A lot of teams use tools that can do this sort of thing OK out of the box. I've set up CI to only run affected tests and use incremental builds in the past. My current project has incremental CI.
The core problem is actually dev teams. Every time I've tried to implement incremental CI when I wasn't in a position to force it through, people fought against it, complained, moaned etc until higher level management gave in. The problems were:
1. People liked the feeling of every test passing every change. It made them feel safe and like they could point the finger at CI if something went wrong, because if every test passed it's definitely not their fault, right? But this leads to slow builds and then they complain about that instead. However this is preferable because slow builds are the project manager/TL's problem to solve, not theirs, so there's a built-in bias towards over-testing.
2. If there's any issue with the build system that causes one test run to affect another in some way (shared state of any kind), then incrementality can cause hard to understand flakes. Sometimes tests will just fail and it'll go away if you force a truly clean rebuild. Devs hate this because they've been taught that flaky tests are the worst possible evil.
3. Build systems are generally hard-wired for correctness, so if you make a change in a module that everything transitively depends on, they'll insist on retesting everything. This is reasonable from a theoretical perspective, but again, leads to very slow builds and people finding ways around them.
4. Work-shifting. Splitting stuff out into separate build systems (separate repos is not really the issue here), may appear to solve these issues, but of course in reality just hides them. Now if you change a core module and break things, you just won't know about it until much later when that other team updates to the new version of the module. But at that point you're working on another ticket and it's now their problem to deal with, not yours, so you successfully externalized the work of updating the use sites to someone else.
Competent firms like Google have found solutions to these problems by throwing hardware at it, and by ensuring nobody was able to create their own repositories or branches. If your company uses GitHub though, you're doomed. No way to enforce a monorepo/unified CI discipline in such a setup.
Out of curiosity, how long is a full build on that place?
I wonder what does it take to give up those points, as they are very good things to have. I can't imagine I would give up on them if builds took around an hour, maybe a day.
I don't quite recall. IIRC it was an hour maybe, so it wasn't like the builds were extremely slow, but it means people didn't want to run the tests locally, so if you're trying to get your work committed and then there's some failure CI picked up that you didn't each iteration can be an hour, and that made people feel unproductive and slow because some of them struggled with context switching back and forth and even the ones who could do it didn't like it. Which I totally agree with, context switching and small delays can be painful to feeling productive.
Obviously full test coverage is a very good thing to have. But, people hate waiting. They ended up having a proliferation of smaller repos and replacing it with manual rec/sync work which was IMO not a good idea, but probably felt more like "work" rather than waiting.
Interesting question arises here. Given most shops are now using interpreted languages, is it possible to make a change to say a .py file and ONLY deploy that .py file to production? i.e. do incremental releases
The issue was rather about the fact that changing that single .py file can have impact on a service A and service B which depend on service C where the file was changed.
In microservice world, you have to run e2e tests because thats the only place where you can really guarantee the system will work as you expect.
Pacts wont solve that, integration test - the same, units w/e.
Running a monolith test is overall faster than spinning 20 microservices, beinging every db to specific atate and running tests.
I don't know if "most shops are now using interpreted languages" is an accurate statement :) (I actually don't know).
But I think that's definitely an interesting idea (likely with some small levels of extra caution with some specific high profile files, like an API definition/endpoint or something).
But it's unlikely things will change in coding culture so long as there's the perverse incentive for businesses to encourage their engineers to use hacks out of expediency. Hacks inevitably make things really hard down the road, create mysterious problems that are hard to solve, and the solution is often to add bureaucracy or switch to microservice architecture.