Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I mean... knowing how to solve it isn't the trick, it's doing it a million times a minute for your firehose scraper.


Anubis adds a cookie name `within.website-x-cmd-anubis-auth` which can be used by scrapers for not solving it more than once. Just have a fleet of servers whose sole purpose is to extract the cookie after solving the challenges and make sure all of them stay valid. It's not a big deal


Requests are associated with the cookie meaning you can trace and block or rate limit as necessary. The cost of solving the PoW is the cost of establishing a new session. If you get blocked you have to solve again.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: