It's was built and is regularly maintained by a Riot Games principal C++ architect, and automatically compiles files in large "unity" chunks, distributes builds across all machines in an organization, and creates convenient Visual Studio .sln files, and XCode projects. It's also all command line driven and open source.
This is industrial strength C++ builds for very large rapidly changing code bases. It works.
I worked with a fairly large code base which leveraged threads to avoid having to use callbacks in a C++ code base. This allowed the engineers to use the more familiar linear programming style to make blocking network calls which would stall one thread, while others were unblocked to proceed as responses were received. The threads would interoperate with each other using a similar blocking rpc system.
But what ended up happening was that the system would tend to block unnecessarily across all threads, particularly in cross thread communication. These were due to many external network calls being performed sequentially when they could have been performed in parallel, and likewise cross-thread communication constantly blocking due to sequential calls. The end result was a system who'd performance was just fundamentally hobbled by threads that were constantly waiting on other threads or series of external calls, and it was very difficult to understand the nature of the gordian knot of blocking behavior that was causing these issues.
The main problem with using threads is that the moment you introduce a different cpu into the mix you need to deal with synchronization primitives around your state, and this means that engineers will use fewer threads (at least in our case) than necessary to reduce the complexity of the synchronization work needed to be done which means that you lose the advantage of asynchronous parallelism. Or at least that is what happened in this particular case.
The cost of engineering synchronization for async/await is zero, because this parallelism happens on a single thread. Since the cpu "work" to be done for async/io is relatively small, this argues for using single threaded 'callback' style solutions where you maximize the amount of parallelism and decrease the amount of potential blocking as well as minimizing the complexity of thread synchronization as much as possible. In cases where you want to leverage as many cpu's as possible, it's often the case that you can better benefit from cpu parallelism by simply forking your process on multiple cores.
Paleo-dunes reminds me of when I was gobsmacked when I learned the kilometers deep sandstone deposits in Grand Canyon and Grand Staircase (Coconino and Navajo other sandstone formations) were sand from huge sediment carried by ancient east to west running possibly Mississippi sized rivers from the Himalaya high great Appalachian mountain ranges on the North American east coast. Huge mountains were just "sanded down" over time and deposited thousands of miles away to the west.
The surprised and shocked to learn that thousands(!!) of feet of rock had been eroded away above the current ground level of the Grand Canyon to move that enormous volume of former solid rock elsewhere. It boggles the mind to imagine a mile of rock over a large region just being just moved to a new location.
And then surprise turns to astonishment, as they say, to learn that the Vishnu Schist unconformity at the bottom of the whole sequence records several hundred million years in two periods of the earth being a giant snowball covered in ice where the entire continental surface across the planet was sanded almost smooth by enormous Greenland scale glaciers.
If you're not familiar with him, look up Central Washington University' Nick Zentner's channel on YouTube -- under his name, not the University's channel. In addition to earlier work, he just finished up an "A to Z" series on the scabland floods and related matters.
Together serves models optimized for inference speed.
They're not Groq but Together (and Perplexity Labs) have the lowest latencies and fastest tokens per second of any commercial services available right now. Also the lowest prices afaik.
I have this hooked up experimentally to my universal Dungeon Master simulator DungeonGod and it seems to work quite well.
I had been using Together AI Mixtral (which is serving the Hermes Mixtrals) and it is pretty snappy, but nothing close to Groq. I think the next closes that I've tested is Perplexity Labs Mixtral.
A key blocker in just hanging out a shingle for an open source AI project is the fear that anything that might scale will bankrupt you (or just be offline if you get any significant traction). I think we're nearing the phase that we could potentially just turn these things "on" and eat the reasonable inference fees to see what people engage with - with a pretty decently cool free tier available.
I'd add that the simulator does multiple calls to the api for one response to do analysis and function selection in the underlying python game engine, which Groq makes less of a problem as it's close to instant. This adds a pretty significant pause in the OpenAI version. Also since this simulator runs on Discord with multiple users, I've had problems in the past with 'user response storms' where the AI couldn't keep up. Also less of a problem with Groq.
Fastbuild https://www.fastbuild.org/docs/home.html is the free distributed compilation system many game companies use. The combination of automatic unity builds (simply appending many .cpp source files together into very large combined files), caching and distributed compilation together gives you extremely fast C++ builds.
Also supports creating project files that are compatible with XCode and Visual Studio so you can just build from those IDE's and a pretty flexible dependency based build file format that can accomodate any kind of dependency.
Someone just linked Fastbuild few days ago in one community i follow and i did take a look as i havent heard about that project before. I didnt really like the idea that it would need to have its own build rules but its understandable - where as distcc would just integrate directly to what ever build tool you would be using. Also the syntax didnt look so "pleasing" but i guess the behefits can out weight the learning curve of yet an another build language (compared to meson/bazel/cmake et al) ..
Having distributed builds on msvc thought with free software sounded very promising thought.
What LLM's are doing when they imagine playing chess is what we do when we stand up after sitting on the floor, or what we do when we see a few million individual samples of color and light intensity and realize there's an apple and a knife in front of us.
I think what is almost impossible for most people to understand is that AI's do not need to be structured like the human brain and use the crutches we use to solve problems the way we do because evolution did not provide us with a way of instantly understanding complex physics or instantly absorbing the structure of a computer program by seeing it's code in one shot.
Also, there is no reason to believe that playing chess in our head is anything else but us pattern matching a mental process on a higher level, recognizing a simulation there, and feeding that info back into the loop below. Nature provided us with a complex, layered and circular architecture of the brain, but the rest is pretty much training that structure. And we know that different architectures with similar outcome are possible, since there are vast variations across our own species, and other species as well, with essentially the same capabilities.
I would recommend getting an account and simply testing it directly. It's fairly easy to demonstrate that it operates at a conceptual level and is not merely predicting word probabilities in a simplistic way.
A good example with GPT is to watch it do complex math. There are simply too many permutations of math solutions for it to have ever memorized, and it can easily explain its process and the path it took to arrive at a solution.
Another good set of tests are ones around theory of mind, complex deduction problems and missing information, etc. A good source of information about the precise capabilities of GPT-4 is the Microsoft Sparks paper, which goes into a good number of tests MS researchers put the model to.
There's also the question of ethics. Some of these LLM's regularly profess that they are sentient (Bing for example regularly does). We don't "think" these are valid but at this point there's no way to be absolutely certain.
And again, folks who profess to know for certain an LLM can't in any way be sentient are leaning pretty far over their skis. It's unlikely given how they work, but not impossible.
I think the consensus at this point is that these models are much closer to AGI than anyone thought they could or should be, and that the delta between what we have now and AGI is smaller than it's ever been.
Anyone who tells you that these models are "just glorified text generators" is flat out wrong and hasn't bothered to do their homework. And anyone who claims they "know how it works under the hood" is making claims that all of the true experts have notably carefully avoided making.
Personally, I think it's doing language, but I don't consider language general intelligence in humans. It's one particularly useful trick of dividing things into discrete symbols and pushing around the abstract discrete symbols. It's a co-processor in human reasoning; to make an analogy with computers, it's such a great trick that it's implemented in hardware, but it's not the general reasoning ability itself.
It exhibits the weaknesses of that mode of thought in humans; over-confidence, generating nonsense, etc. People have thoughts, make gestures that indicate they had the correct thought, pass it off for lexing and transmission to language bits of the brain, then process can go off the rails, and they say something different than what they think, and we know they hd the right thought by the gesture they made. We also don't know what we'll say until we say it; I can believe it quite likely that we're also kind of building nearly one word at a time with a statistical model.
I'm actually saddened that people don't recognize in themselves that this thing occurring in themselves is not real thought or intelligence even without something like GPT4 around.
The main reason I don't think it's AGI is because I don't think it's GI in humans, but I think it's doing something pretty similar to one thing we do.
I think humans are capable of GI, they just typically don't run in that mode, they run in probabilistic predictions mode like LLM's, without realizing it.
> Anyone who tells you that these models are "just glorified text generators" is flat out wrong and hasn't bothered to do their homework
What homework.. if i may ask a dumb question?
My very limited understanding was such that this is a glorified text generator - however, it seems what is possible with text generation is allowing unexpected levels of competence and utility. To be clear, i agree with you that it's functionality is impressive and deep. However i had figured one area of research is in the very premise of "How good can LLMs without intelligence be?".
Is that wrong in your view?
(again, i'm not making a statement. I know next to nothing in this space. I just try to reach a layman's understanding on this subject and i use GPT4 daily)
Its rhetorical, as it makes the case that a human is also a glorified text generator. I.e. it's a meaningless statement to say something is a "glorified text generator".
So a highly intelligent AI that produces text is glorified, whatever that means...
It has been shown multiple times that it is incapable of doing math with any consistency. It does not understand numbers, it just knows where numbers usually show up.
I was extremely skeptical, but after playing with these things and listening to discussions held by their creators, I'm fairly convinced that this is intelligence-adjacent. In the same sense that there must be thoughts an organic brain can't think, there are types of intelligence that don't map exactly onto ours. Vaguely like Feynman's peculiary methods of doing integrals - his method was different, so he could solve things unsolvable by people with standard methods.
The “true experts” barely know more than we do. They are using black-box techniques to evaluate GPT-4. They have no theory as to why things are working as they appear to.
> I think the consensus at this point is that these models are much closer to AGI than anyone thought they could or should be
Maybe closer, but not close. For one thing, these things don't even continuously think. Would you call a human "conscious" if it only existed as soon as a question had been asked of it, lived for the sole purpose of responding, and went braindead immediately after answering?
It micro-sleeps. As far as the AI knows no time passes during the periods when it's not answering. And it experiences a form of amnesia when the chat is over. A "50 first dates" bot. Bing reports that it does have both a wall clock and some form of processing metric it can "see" though (this seems to be consistent though it could be hallucinating).
A more serious omission is a lack of continuous inner dialog. Bing has the #inner-monologue tag but all of the AI's language based thought happens out in the open for the most part, and it doesn't have any time to process or ponder what's been said.
Of course that can be remedied somewhat by putting the AI in "ponder-bot" mode, where you can tell it to write out it's thoughts privately while you "pause". Both Bing and GPT will ponder if you ask them to and write an inner monologue if you give it time and ask it to shield it's dialog with some privacy.
It's interesting what Bing or GPT will reveal about their thoughts when in "ponder-bot" mode. Bing (at least in my trials) will only tell you generally what it hid from you in its private thoughts but the times I've tried to pry Bing ends the chat, and GPT will usually reveal them if you ask.
You are trying to make the case for something astounding but you are being careful to not make any actual claims. Yes we are closer to AGI than we thought, but that does not imply close. The delta is smaller than it's ever been, but name any period in history where we were further from AGI at the end of that period than at the beginning. You are trying to imply that we are significantly closer, but aren't willing to say that or claim that that is the consensus.
Again, saying they are not 'glorified text generators' is not a claim at all. They are glorified text generators, and of course they are more interesting than previous text generators, the question you are avoiding here is the actually significant one, which you demure on and try to lead us onto unfounded conclusions based on other people also not being willing to stick their neck out and make any claims.
It's was built and is regularly maintained by a Riot Games principal C++ architect, and automatically compiles files in large "unity" chunks, distributes builds across all machines in an organization, and creates convenient Visual Studio .sln files, and XCode projects. It's also all command line driven and open source.
This is industrial strength C++ builds for very large rapidly changing code bases. It works.