Europe Pub
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
Pro@programming.devM to Technology@programming.devEnglish ·
edit-2
17 days ago

Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.

message-square
115
link
fedilink
  • cross-posted to:
  • technology@lemmy.world
704

Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.

Pro@programming.devM to Technology@programming.devEnglish ·
edit-2
17 days ago
message-square
115
link
fedilink
  • cross-posted to:
  • technology@lemmy.world

Comments
  • Lemmy;
  • Hackernews.

Source.

  • mormund@feddit.org
    link
    fedilink
    English
    arrow-up
    4    ·
    17 days ago

    I thought Anthropic was also very abusive with their scraping?

    • BodilessGaze@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1      ·
      16 days ago

      Maybe to others, but not to us. Or if they are, they’re very good at masking their traffic.

Technology@programming.dev

Technology@programming.dev

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !Technology@programming.dev

Share interesting Technology news and links.

Rules:

  1. No paywalled sites at all.
  2. News articles has to be recent, not older than 2 weeks (14 days).
  3. No videos.
  4. Post only direct links.

To encourage more original sources and keep this space commercial free as much as I could, the following websites are Blacklisted:

  • Al Jazeera;
  • NBC;
  • CNBC;
  • Substack;
  • Tom’s Hardware;
  • ZDNet;
  • TechSpot;
  • Ars Technica;
  • Vox Media outlets, with exception for Axios;
  • Engadget;
  • TechCrunch;
  • Gizmodo;
  • Futurism;
  • PCWorld;
  • ComputerWorld;
  • Mashable;
  • Hackaday;
  • WCCFTECH;
  • Neowin.

More sites will be added to the blacklist as needed.

Encouraged:

  • Archive links in the body of the post.
  • Linking to the direct source, instead of linking to an article talking about the source.

Misc:

Relevant Communities:

  • Beehaw Technology- Technology Related Discussions.
  • lemmy.zip Technology- Hard Tech news.
Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 402 users / day
  • 1.32K users / week
  • 4.46K users / month
  • 6.61K users / 6 months
  • 4 local subscribers
  • 492 subscribers
  • 1.45K Posts
  • 3.15K Comments
  • Modlog
  • mods:
  • Pro@programming.dev
  • BE: 0.19.12
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code
  • join-lemmy.org