Should I treat them as brain gymnasiums such as Number Theory which few people ever needs for their career, but could improve problem solving ability in general?

From my understanding those brain gymnasiums make your brain work harder and you have to concerntrate for longer period to get them solved. They also open new ways of thinking about the world.