+1, I've been using OpenSearch (basically Elasticsearch 7.0), and have been pretty happy with the setup so far.
OpenSearch specifically has an edge over Elasticsearch because it supports vectors up to 10k dimensions, whereas ES maxes out at indexing 1024 dimensions, which isn't enough to support OpenAI's 1536 dimension vectors.
And then there's the benefit of it being well documented / Q&A'd, and able to support regular searching, faceting, etc. as well.
Also if you want to do hybrid retrieval with legacy system in place elasticsearch is a good option. I would like to see some comparison for the hybrid retrieval as well.
You aren't supposed to index vectors larger than ~128 dimensions. Because of concentration of measure which is an aspect of the curse of dimensionality the distances between high-dimensional vectors tend to become identical.
You need to do dimensionality reduction before indexing. Basically it's fine to just pick n first components if you don't want anything fancy.
Maven is building the university of the future - empowering the world’s creators to offer cohort-based courses directly to their audiences. Courses on Maven have community at their core, focus on active learning, and are available anywhere in the world.
Our team has raised $25 million from incredible investors including Andreessen Horowitz, First Round, Naval, and Arlan Hamilton, and we've launched sold-out courses with instructors like Li Jin, Anthony Pompliano, Lenny Rachitsky, and Sahil Lavingia.
We're the leading player in a massive and growing market, but are still early enough for you to make a big impact. Join us!
Maven | Full-stack, Front-end Engineers | Remote (US timezones) | Full time | $120-$170k + equity | https://maven.com
Maven helps anyone build a cohort-based course and deliver an incredible student experience at scale.
Unlike small workshops or on-demand online classes, we integrate both live and asynchronous components to engage students without needing to scale up instructor time or support staff.
Our mission is to democratize education through online cohort-based courses. Right now, many experts and creators don't have the team, budget, or bandwidth to create a high quality cohort-based course. Inspired by the simplicity and power of tools like Shopify and Substack, we are building a platform that enables the best instructors to build the most engaging courses.
Our team of 12 has built and run educational experiences at Udemy, AltMBA, Minerva, Google, General Assembly, Socratic, and Shopify. We're backed by Andreessen Horowitz, First Round Capital, Naval Ravikant, Arlan Hamilton, and many more. We've launched sold-out courses for incredible instructors like Li Jin, Anthony Pompliano, Lenny Rachitsky, Sahil Lavingia, and more.
We're looking for our 2nd designer, and 6th and 7th engineers whose work will span across our system and will have a big impact on our product. We're experienced, ambitious, and supportive. We work with React/Typescript/Next.js, Python/FastAPI, AWS/ECS/RDS.
Maven | Full-stack, Front-end Engineers | Remote (US timezones) | Full time | $120-$160k + equity | https://maven.com
Maven helps anyone build a cohort-based course and deliver an incredible student experience at scale.
Unlike small workshops or on-demand online classes, we integrate both live and asynchronous components to engage students without needing to scale up instructor time or support staff.
Our mission is to democratize education through online cohort-based courses. Right now, many experts and creators don't have the team, budget, or bandwidth to create a high quality cohort-based course. Inspired by the simplicity and power of tools like Shopify and Substack, we are building a platform that enables the best instructors to build the most engaging courses.
Our team of 12 has built and run educational experiences at Udemy, AltMBA, Minerva, Google, General Assembly, Socratic, and Shopify. We're backed by Andreessen Horowitz, First Round Capital, Naval Ravikant, Arlan Hamilton, and many more. We've launched sold-out courses for incredible instructors like Li Jin, Anthony Pompliano, Lenny Rachitsky, Sahil Lavingia, and more.
We're looking for our 6th and 7th engineers whose work will span across our system and will have a big impact on our product. We're experienced, ambitious, and supportive. We work with React/Typescript/Next.js, Python/FastAPI, AWS/ECS/RDS.
Socratic | Generalist Engineer + Android Lead | New York City | Onsite, Full-time | https://socratic.org
Socratic is a top education app where students take a picture of a homework question and we instantly teach them how to answer it.
We're one of the top iOS education apps and our Android app is launching this week! We combine cutting edge machine learning with high quality custom content. Students asked 10 million questions last month, and we're just getting started.
We're looking for an experienced Android lead to own our Android app.
We're also looking for a generalist engineer to work on all aspects of our backend and dabble in mobile.
Socratic | Android Developer | New York | will transfer H1B, full-time, https://socratic.org
Socratic is a education app that combines cutting-edge AI with expert teaching to make learning on your phone easy, free, and accessible to everyone.
We are working to provide high-quality learning that feels and works as well as one-on-one tutoring but at a fraction of the cost. We’ve helped over 30 million students, and have our sights set on the next billion.
We are looking for an experienced Android developer who has built multiple apps from scratch, deployed apps to the Play store, and has worked on a small team. You will be the first Android developer on the team.
yes, would love asciimath support. most of the math that'll exist on the web isn't there yet, and it'll be so much easier to the next billion internet users than learning latex.
katex looks great though. we (socratic.org) might start using it if we can find a good asciimath=>latex converter.
Check out https://github.com/mathjax/asciimathml/blob/master/asciimath.... It uses the AsciiMath parser to generate latex to be fed into mimetex for rendering, but it could probably be daisy-chained with katex. I keep it current with the latest AsciiMath revisions.
I created such a daisy-chained page that uses ASCIIMathML input with KaTeX rendering where possible, and it falls back to MathJax when KaTeX can't handle the expression. See
OpenSearch specifically has an edge over Elasticsearch because it supports vectors up to 10k dimensions, whereas ES maxes out at indexing 1024 dimensions, which isn't enough to support OpenAI's 1536 dimension vectors.
And then there's the benefit of it being well documented / Q&A'd, and able to support regular searching, faceting, etc. as well.