This might be true in the abstract but it's not true of actual compilers dealing with real world calling conventions. Absent inlining or whole program optimization, calling conventions across translation units don't leave much room for flexibility.
The semantics of pass by const reference are also not exactly the same as pass by value in C++. The compiler can't in general assume a const reference doesn't alias other arguments or global variables and so has to be more conservative with certain optimizations than with pass by value.
What's the basis for your claim that it is on an exponential growth trajectory? That's not the way it feels to me as a fairly heavy user, it feels more like an asymptotic approach to expert human level performance where each new model gets a bit closer but is not yet reaching it, at least in areas where I am expert enough to judge. Improvements since the original ChatGPT don't feel exponential to me.
This also tracks with my experience. Of course, technical progress never looks smooth through the steep part of the s-curve, more a sequence of jagged stair-steps (each their own little s-curve in miniature). We might only be at the top of a stair. But my feeling is that we're exhausting the form-factor of LLMs. If something new and impressive comes along it'll be shaped different and fill a different niche.
Big little cores like on mobile or some Intel processors are really not the same thing. The little cores have the same instruction set and address the same memory as the big cores and are pretty transparent to devs apart from some different performance characteristics.
The SPEs were a different instruction set with a different compiler tool chain running separate binaries. You didn't have access to an OS or much of a standard library, you only had 256K of memory shared between code and data. You had to set up DMA transfers to access data from main memory. There was no concept of memory protection so you could easily stomp over code with a write to a bad pointer (addressing wrapped so any pointer value including 0 was valid to write to). Most systems would have to be more or less completely rewritten to take advantage of them.
This looks cool, I find myself wishing for a language and introductory tutorial that isn't so targeted at Python programmes however (though I understand from a commercial point of view why that may make sense).
It seems like this is actually an elegant typed functional language but the Python syntax looks ugly and verbose and like it's trying to hide that compared to something more ML/F# or Haskell inspired.
I'll try and get past that though as it does look like there's something pretty interesting here.
Modules, standardised in C++20, are the official solution both for the standard library and for other code but they are not universally well supported by all major compilers and build systems yet.
Transitioning existing code to use modules is also not entirely straightforward, though probably no more problematic than introducing unity builds.
> though probably no more problematic than introducing unity builds.
A "Unity build" really just means typing #include "foo.cpp" a few times. It's trivial.
Meanwhile, neither Clang nor GCC support standard library modules. They have only partial support for modules themselves. C++ module support is non-existent in almost all build systems. https://en.cppreference.com/w/cpp/compiler_support
The idea of C++ modules is great. It's badly needed. In practice I'm not sure if they're ever going to be genuinely functional and widespread. Which makes me sad. Toy projects don't count.
It's been a lot of chicken-and-egg, but C++ modules finally have momentum. FWIW, Clang does support `import std` with `libc++` now. While I've implemented CMake support for modules, I've worked on laying the groundwork for compilers to be able to provide the required information (P1689) and I've been involved in advocating support from a number of build systems (e.g., Bazel, xmake, Meson, Tup). Some have been more receptive and made progress; others have had less.
The semantics of pass by const reference are also not exactly the same as pass by value in C++. The compiler can't in general assume a const reference doesn't alias other arguments or global variables and so has to be more conservative with certain optimizations than with pass by value.