Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Many of these paywalls simply have either the full body in the response or will return it if you ask with the right user agent (e.g. the one that google uses). These plugins then either remove some overlays or impersonate the google bot.

Obviously that’s not how it should be implemented and that‘s also not what google suggests you to do as a publisher. Instead you should verify that the bot is actually google https://developers.google.com/search/docs/advanced/crawling/...



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: