Looks like someone at OpenAI had the bright idea that they could push 'Christmas shopping season' apps 'assisted by ChatGPT' to 'help find the perfect gift' to paying users and everyone (including me) was really disgusted by having that garbage clogging up screen space.
Really just confirmed to me that long term, the best option for inference is just running an open source model on your own hardware, even if that's still expensive and doesn't generate as high quality output.
Long-term memory on top of the base model, but is this idea for local users or for the data-center hosted model used by many different people?
P.S. This quote from the paper sounds just like LLM output:
> "This memory module provides significantly higher expressive power, allowing the model to summarize large volumes of information without losing important context. The model isn't simply taking notes; it's understanding and synthesizing the entire story. Crucially, Titans doesn’t just passively store data. It actively learns how to recognize and retain important relationships and conceptual themes that connect tokens across the entire input."
This is no different from what happens to humans if they're locked into cult programming situations, they'll start believing and regurgitating all kinds of nonsense if their information stream is tightly curated,
Practically, for use with a codebase development effort, if the model remembers the original design decisions, the discussions about costs and benefits, then can remember all that much later in the process, it's going to start getting really good at thinking about what the next step is, or even to make decisions about when a major refactor is neede, etc.
Yeah, I can't imagine not being familiar with every single reference in the bibliography of a technical publication with one's name on it. It's almost as bad as those PIs who rely on lab techs and postdocs to generate research data using equipment that they don't understand the workings of - but then, I've seen that kind of thing repeatedly in research academia, along with actual fabrication of data in the name of getting another paper out the door, another PhD granted, etc.
Unfortunately, a large fraction of academic fraud has historically been detected by sloppy data duplication, and with LLMs and similar image generation tools, data fabrication has never been easier to do or harder to detect.
I really can't imagine that these online degrees have any real value in the modern world of LLM-assited coding - there's no way anyone looking at a resume would think such institutional online degrees still have any value. Perhaps there is some educational value for the student, but even there the only real value is the organizational structure - you might as well form an online study group on discord for free, and get the same learning benefit, just have an LLM write up the syllabus for a course based on a good textbook, no instructor overhead needed.
The OMSCS degree you get is equivalent to the in person one, so there is no way to make the distinction in an interview. I actually don’t see how people see that an experience like this brings no value, given the rigor of the assignments. One certainly would come out with a better knowledge of how things work, develop a better work ethic, and hopefully make some network connections on the way…
The whole point is, if an LLM can easily complete rigoruous assignments and all the student has to do is add a little bit of personalization to the output, then has that student really learned anything? Can they evem come up with a plan to do such tasks without the LLM, even if it takes a lot longer without it?
Educational certifications in the era of LLMs are going to be increasingly meaningless without proof-of-work, and that's going to mean in-class work without access to computational aids, if you really want to evaluate a person's skill level. This of course is the coding interview rationalization - CS students have been gaming auto-graded courses created by CS professors for some decades, and now that's easier than ever.
There is absolutely no way you’re passing OMSCS tests if you’re winging it on the other assignments, and the tests usually account for over 50% of the grade. Certifications you’re right about but there are ways to test knowledge without asking for code snippets.
> there is no way to make the distinction in an interview
Just ask?
Some online degrees state that they're equivalent, but interviewers may still have their own opinions. I would discourage anyone from failing to mention the online nature of a degree in their CV. You're really not doing yourself a favor. A rigorous online degree is something to be proud of. I see people with PhD's proudly announcing their online course certificates on LinkedIn. However, 'discovering' that an education was of a different nature than one had assumed based on the presented materials may raise questions.
This just reeks of you being insecure and thinking online education is of lower quality than in person education. Are you also pining for everyone to go back to the office? The degree GT gives you is literally the same thing as the in person degree. If GT does not make the distinction, why would I???
That says nothing other than that the interviewers have a narrow mind and/or are ignorant. OMSCS is a very well known program, and it's their problem if they don't know it.
This is very debatable. The courses look like they were recorded in the 90s.
The DB course particularly sticks out. My undergrad's DB course was fathoms harder than this. This is what you'd expect a highschooler should be able to learn through a tutorial not a university course.
If it doesn't talk about systems calls like mmap, locking and the design of the buffer pool manager, it's not a university Database course it's a SQL and ER modelling tutorial.
Respectfully, I think you should do more research.
The OMSCS program is well known and well respected in the tech industry. It's a masters degree from the currently 8th ranked computer science school in the U.S.
The university make no distinction between students who take the courses online, vs in person. I.e., the diploma's are identical.
I’ve taken graduate-level courses in databases, including one on DBMS implementations and another on large-scale distributed systems, and I also spent two summers at Google working on Cloud SQL and Spanner. Database research goes further than DBMS implementation research. There is a lot of research on schemas, data representation, logic, type systems, and more. It’s just like how programming language research goes beyond compilers research.
I don't think watching the lectures is the hurdle that anyone at OMSCS is trying to jump. The program has a pretty low graduation rate, and the tests are known to be fairly difficult, which essentially requires the student to do work outside of class or go to the resources available through GT to understand the material. I can look up the highest quality lectures on any subject on YouTube, it doesn't mean I will understand any of it without the proper legwork.
FWIW I meant the diploma is identical, the actual experience will obviously vary. Some people will get better outcomes online, some will get better outcomes in person.
Is this a common thing to have at university? I'm from one of top universities in Poland; our database courses never included anything more than basic SQL where cursors were the absolute end. Even at Masters.
Do not worry, I do not work with databases in professional life as my main aspect. But I was not given a comprehensive education, and not even once there was a focus on anything more in depth. I came out without even knowing how databases work inside.
Naturally, I know what I could do - read a good book or go through open source projects, like Sqlite. But that knowledge was not was my uni gave me...
I am jealous of American/Canadian unis in this aspect.
OMSCS student here. You are absolutely right that the DB course is one of the weaker offerings. There is a newer Database System Implementation course, which is based on Andy Pavlo's excellent undergrad course (which is also available online), but only the first half or so of that course is covered, which is disappointing for a graduate course. In terms of the larger program, however, the two database courses are outliers and most courses are of much higher quality and definitely not undergrad level.
Hey — head TA of DSI here and want to correct some misconceptions.
DSI (6422) is taught by Andy Pavlo’s first PhD student who help to create the CMU course and a rather famous DB person. It is the same contents as the on-campus course (and were actually working to deepen/increase the depth of coverage). It’s designed to bridge between DB Theory and reading Postgres or MySql source code when it comes to DB designs and trade-offs — and covers topics like r-tries which I don’t think is covered elsewhere + a series of 12 seminal DB papers. As in any other grad-level class, you get out as much as you put in — and it’s super rare to have access to a DB researcher like Joy or hear his takes on DB development as a student at scale.
If anything, the feedback we’ve gotten from both on campus undergrad and MS students is that the OMSCS lectures + improvements are making their session more rigorous.
We actually launched a new class (CS 6422) that addresses exactly this and taught by Andy Pavlo’s first PhD student :) OMSCS db classes reviews are outdated IMO
You might as well simply claim "I don't see a CS degree has any value these days". OMSCS is not any less than a "real" graduate school program experience.
I generally agree, but as a caveat sometimes it's cheaper, more robust and more efficient to build an integrated system without having to worry about interoperability. BYD's electric vehicle chasis for example, seems to greatly cut manufacturing costs, even if it makes swap-in repairs harder down the road.
But, I'd guess this accounts for a relatively small fraction of corporate decision on lock-in strategies for rent extraction - advanced users should be able to treat their cell phones OS like laptops, with the same basic concepts, eg just lock down the firmware for the radio output, to keep the carriers happy, and open everything else, maybe with a warranty void if you swap out your OS. Laws are needed for that, certainly.
You can only move fast without crashing all the time if you've already developed expert-level skills, and that takes time. Two examples come to mind: Albert Einstein's work on special vs. general relativity, and Adrej Karpathy's ML tutorials. Now, if you want to explore this in more detail, here are two prompts I wrote myself that you can use to get the full argument:
(1) As an expert in scientific discovery in the 19th and 20th century, let's disassemble a general claim using the specific example of Einstein's work on special relativity and general relativity. First, here is the claim: "If I give you two PhD students, one who completed their thesis in two years and one who took eight years… you can be almost certain that the two-year thesis will be much better." Things to keep in mind: (1) special relativity was baked into Maxwell's electromagnetism and should have been discovered years before Einstein, and (2) general relativity was a novel application of non-Euclidean geometry and mathematics to the gravity problem, that is the acceleration problem, and was quite a unique accomplishment. Discuss the 'amount of research' that went into each development by Einstein and lay out the argument that this disproves our claim, with any caveats you think appropriate.
(2) In general, it seems to take about ten years of diligent focused effort for a person to develop their skill levels to the point where they can make meaningful contributions to any science, engineering, or even artistic field. Einstein seems to follow this trend, if we start counting from his teenage fascination with physics. Another example is the very popular instructional videos on machine learning by Andrej Karpathy, eg "The spelled out intro to neural networks and backpropagation: building micrograd" in which he begins by stating he's been programming neural nets for ten years. Thus, it seems fair to conclude that 'move fast' only makes sense after 'develop the required expertise to know how to move fast'.
Follow-up prompt, as the original author claims to be a computer science professor:
(3) Are there any examples in software development -- I'm thinking about the development of the early yield management applications in the airline industry (e.g. SABRE), for example.
The Economist should not be treated as reliable source of information on medical issues.
[edit] To be more specific, this is a lazy take and is about as insightful as saying 'cancer should not be treated as a single condition' which for HN is about as meaningful as saying 'the CPU and the GPU may both contain chips, but they should not be programmed the same.'
Required reading on Palantir and its cousins, Dataminr etc. : "IBM and the Holocaust, The Strategic Alliance Between Nazi Germany and America's Most Powerful Corporation."
The book is good because of the extensive historical documentation of IBM practices, Nazi procurement orders, and the eagerness that IBM leaders displayed in fulfilling those orders, even though they knew the purpose:
> "The racial portion of the census was designed to pinpoint ancestral Jews as defined by the Nuremberg Laws, ensuring no escape from the Reich's anti-Semitic campaign. In addition to the usual census questions, a special card asked whether any of the individuals grandparents was Jewish."
In a not-so-unique historical inversion, the Israeli government is now using American tech firms like Palantir to assist in their ongoing ethnic cleansing and genocide programs in the West Bank and Gaza, which have certainly not ended, ceasefire or no, as any reading of the statements of Israeli government officials, bloggers, online commentators etc. demonstrates (even though Twitter no longer provides translations of Hebrew to English, it's not hard to decipher the intent).
As far as Palantir and Dataminr's agenda? Same as IBM's - delivering value to their shareholders.
Perhaps the final edit should have included the complaint about 'buggy bloated Javascript' as that's a very substantive issue - and now I don't know if they changed that as 'tone' or because they decided that technical criticism wasn't correct, and there are other issues?
Really just confirmed to me that long term, the best option for inference is just running an open source model on your own hardware, even if that's still expensive and doesn't generate as high quality output.
reply