Your server/network goes down, but you still want to maintain availability and let your users view and manipulate their data. So now users make edits while offline, and when they come back online you discover they made edits to the same rows in the DB. Now what do you do?
The problem really is about concurrency control - a DB creates a single source of truth so it can be either on or off. But with GoatDB we have multiple sources of truth which are equally valid, and a way to merge their states after the fact.
Think about what Git does for code - if GitHub somehow lost all their data, every dev in the world still has a valid copy and can safely restore GitHub's state. GoatDB does the same but for your app's data rather than source code
So now users make edits while offline, and when they come back online you discover they made edits to the same rows in the DB. Now what do you do?
Store changes in a queue as commands and apply them in between reads if that's what you want. This is really simple stuff. A few hundred thousand items and a few users is not a large scale or a concurrency problem.
Yup. Go ahead and try it, then you'll discover that:
- The queue introduces delays so this doesn't play nice with modern collaborative editing experience (think google docs, slack, etc)
- Let's say change A set a field to 1, and change B set the same field to 2. GoatDB allows you to easily get either 1, 2 or 3 (sum) or apply a custom resolution rule
Your only choices before goat to solve this were: Operational Transformation, raw CRDTs or differential synchronization. GoatDB combines CRDTs with commit graphs so it can do stuff other approaches don't at an unmatched speed
The queue introduces delays so this doesn't play nice with modern collaborative editing experience
Things that can be done millions of times per second per core don't "introduce delays" that a handful of people are going to see.
unmatched speed
Are you seriously trying to say that the database you created in a scripting language that uses linear scanning of arrays is 'unmatched' compared to high performance C++? You may have other features but you have no benchmarks and the scenario you were bragging about is trivial.
Things that can be done millions of times per second per core don't "introduce delays" that a handful of people are going to see.
Oh but they can't. If you tried it, then you surely know that both OT and CRDTs need to consider the entire change history at some key points in order to derive the current value. Diff sync doesn't suffer from the same issue, however the way it keeps track of client shadows introduces writes on the read path, making it horribly expansive to run at scale.
Are you seriously trying to say that the database you created in a scripting language that uses linear scanning of arrays is 'unmatched' compared to high performance C++?
It's not about the language, but about the underlying algorithm. Yes, JS is slower, and surely linear scan is slower than typical DB queries. But what GoatDB does, which is quite unique today, is it's able to resume query execution from the last point a query ran, so you get super efficient incremental updates which are very useful when running on the client side (clients tend to issue the same queries over and over again).
I'm not sure what the point of all this is. Linear scanning arrays does not scale, this is basic computer science. Javascript is going to run at 1/10th the speed of a native language at best. You don't have any benchmarks and are bragging about stuff that was typical 30 years ago. You realize that people have done shared document editing for decades and that every video game keeps a synced state right?
The most important thing here is benchmarks. If you want to claim you have "unmatched" speed, you need benchmarks.
