We think Grafana is still a great tool and many teams are heavily invested in the Grafana ecosystem. We'll continue to invest in Grafana support via ClickHouse's official Grafana plugin and that won't be changing at all with this release.
However, there's a bit of a fundamental difference in the user experience we're targeting. Grafana has really excelled at traditional monitoring dashboards, low cardinality monitoring workflows.
ClickHouse unlocks a newer paradigm of high cardinality, high performance observability. It enables a new set of workflows/UX that allows engineers to query novel problems quickly as opposed to working off of static dashboards. That's really a big focus of ours, so you'll see we do exploration/search/syntax/UI layout is quite different from Grafana due to this.
At this point it isn't even an original realization of ours. Just as an example, Shopify built a complete custom app (only keeping the auth part of Grafana) while migrating to ClickHouse for similar reasons.
The status quo tool Debezium is annoying/heavy because it’s a banana that comes attached to the Java, Kafka Connect, Zookeeper jungle - it’s a massive ecosystem and dependency chain you need to buy into. The Kafka clients outside of Java-land I’ve looked are all sketchy - in Node, KafkaJS went in maintained for years, Confluent recently started maintaining rdkafka-based client that’s somehow slower than the pure JS one and breaks every time I try to upgrade it. The Rust Kafka client has months-old issues in the latest release where half the messages go missing and APIs seem to no-op, and any version will SIGSEGV if you hold it wrong - obviously memory unsafe. The official rdkafka Go client depends on system C library package versions “matching up” meaning you often need a newer librdkafka and libsasl which is annoying; the unofficial pure-go one looks decent though.
Overall the Confluent ecosystem feels targeted at “data engineer” use-cases so if you want to build a reactive product it’s not a great fit. I’m not sure what the performance target is of the Debezium Postgres connector maintainers but I get the sense they’re not ambitious because there’s so little documentation about performance optimization; data ecosystem feels contemporary with “nightly batch job” kind of thing vs product people today who want 0ms latency.
If you look at backend infrastructure there’s a clear trope of “good idea implemented in Java becomes standard, but re-implementing in $AOT_COMPILED_LANGUAGE gives big advantage:
- Cassandra -> ScyllaDB
- Kafka -> RedPanda, …
- Zookeeper -> ClickHouse Keeper, Consul, etcd, …
- Debezium -> All these thingies
There’s also a lot of hype around Postgres right now, so a bit of VC funded Cambrian explosion going on and I think a lot these will die off as a clear winner emerges.
ClickHouse is incredible. It has also replaced a large, expensive and slow Elasticsearch cluster at Contentsquare. We are actually starting an internal team to improve it and upstream patches, email me if interested!
It's not an impression. I'm a dutch native and have used Facebook in the Netherlands since it exists. I don't see how Myanmar has anything to do with what I said. If anything it only demonstrates the point that Facebook is used in wildly different ways.
There are ways to understand why things went wrong and how to prevent them in the future, without assigning blame. Blameless post-mortem.
I'm a reasonable man, I find accountability a positive virtue. I'm also not a foolish one, for I understand trying to assign blame for an act of god is definitely NOT normal. And I understand the "blameless post-mortem" is a tech-industry standard well understood, so I am surprised to find the "blame game" card being played here. Consider:
Every time a hurricane rolls off the coast of west Africa and trashes the Eastern seaboard, you don't see the US blaming west Africa.
You don't see Missouri trying to pin the Joplin tornado onto neighboring Kansas/Oklahoma in order to recoup billions of dollars of damages and loss of human life. You DO get a technical NIST report that is blameless (I have worked with this particular data) [0].
When an earthquake originates in one country but flattens the city in a neighboring country, you don't see one sue the other.
When a typhoon hits SE Asia, they aren't trying to readily assign blame.
What can be assigned blame is a nation's reaction to this force majeure. At that point the people should be holding their own leaders accountable, as the assumption should always be that the neighbors are incompetent, and our own leaders are the best. That is inconvenient for the current President precisely because he politicized the disease. If he had not politicized it, his followers would be more amenable for blameless post-mortems (literal post-mortems, let's remember people are dying). Unfortunately his response was lackluster, and rather than taking accountability (you know, the virtue I agreed w/ y'all on at the beginning), he would rather shift blame. But this implies that he was relying on China to do its part. Which then begs the question: If the US President wants to blame China, why was he sitting back and relying on China on good faith when no other nation was?
To summarize why I don't believe the bullshit that is "assigning blame" for SARS-Cov-2:
- Accountability is a virtue
- Blameless postmortem is a huge cross-industry technical standard, so abandoning that is immediately suspect
- US President politicized the disease; due to this he has political motivations to avoid the virtue of accountability and how he guided the US response (making the act of "blaming" even more suspect as being a political reaction)
- Doublethink of "Did the US President really rely on the Chinese response? Blame them, not him!" (only enabled because of politicization)
There are ways to understand why things went wrong and how to prevent them in the future, without assigning blame. Blameless post-mortem. But that's now been politicized.
my takeaway: the western stereotype of chinese education is that there is herd mentality and groupthink, but the same thing exists on the western side, just with a different herd.
edit: this is highlighted by the conclusion, in which he points out the irony of the lesson from his western education (everything is luck) when the traditional message is "if you work hard you will succeed".
> It is worth stressing that we are dealing here not with individual sadistic psycho-pathology, but with a deep cultural difference. Roman commitment to cruelty presents us with a cultural gap which it is difficult to cross.
I'm not sure that's true. Remember that Germany and Austria were centers of Western civilization, culture, arts and learning before WWII, yet look at the bloodthirsty practices they embraced. Other European countries did similar things in their overseas colonies.
We are not immune; human nature hasn't changed. We are made of the same stuff as Romans, Nazis, ISIL members, Serb ultranationalists and Rwandan Hutu nationalists. Currently it seems there's a public movement to embrace hatred as a norm, and violence, including war internationally and firearms domestically, as an acceptable solution to problems. If those become norms I worry about the next steps.
However, there's a bit of a fundamental difference in the user experience we're targeting. Grafana has really excelled at traditional monitoring dashboards, low cardinality monitoring workflows.
ClickHouse unlocks a newer paradigm of high cardinality, high performance observability. It enables a new set of workflows/UX that allows engineers to query novel problems quickly as opposed to working off of static dashboards. That's really a big focus of ours, so you'll see we do exploration/search/syntax/UI layout is quite different from Grafana due to this.
At this point it isn't even an original realization of ours. Just as an example, Shopify built a complete custom app (only keeping the auth part of Grafana) while migrating to ClickHouse for similar reasons.