Pro@programming.devM to Technology@programming.devEnglish · edit-216 days agoCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.imagemessage-square115linkfedilinkarrow-up1706arrow-down12file-textcross-posted to: technology@lemmy.world
arrow-up1704arrow-down1imageCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.Pro@programming.devM to Technology@programming.devEnglish · edit-216 days agomessage-square115linkfedilinkfile-textcross-posted to: technology@lemmy.world
minus-squarewho@feddit.orglinkfedilinkEnglisharrow-up2·edit-215 days ago Crawl-delay It’s a nonstandard extension without consistent semantics or wide support, but I suppose it’s good to know about anyway. Thanks for mentioning it.
It’s a nonstandard extension without consistent semantics or wide support, but I suppose it’s good to know about anyway. Thanks for mentioning it.