Pro@programming.devM to Technology@programming.devEnglish · edit-25 days agoCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.imagemessage-square116linkfedilinkarrow-up1707arrow-down12file-textcross-posted to: technology@lemmy.world
arrow-up1705arrow-down1imageCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.Pro@programming.devM to Technology@programming.devEnglish · edit-25 days agomessage-square116linkfedilinkfile-textcross-posted to: technology@lemmy.world
minus-squarewitten@lemmy.worldlinkfedilinkEnglisharrow-up12·1 month agoBelieve me, these AI corporations have way too many IPs to make this feasible. I’ve tried per-IP rate limiting. It doesn’t work on these crawlers.
Believe me, these AI corporations have way too many IPs to make this feasible. I’ve tried per-IP rate limiting. It doesn’t work on these crawlers.