Do you think that if some CS programs taught parsing, the authors of the bot would parse the HTML to properly extract links, instead of just doing plain text search?
Sure. Ideally CS programs would be more than the trade schools they are today. When I was in school, the "CS Program" was a specialization in the math department. I was already in a Physics program when they created a "Computer Science & Engineering" degree. And I remember both the Math and CSE degrees being pretty decent. But that was back in the day when people took CS related courses because they were interested in computers, not because they wanted to get a super high paying job for a FANG company.
Maybe you're right. You can lead a horse to water, but you can't make it think deep thoughts about parsing.
I doubt it.