• hperrin@lemmy.ca
    link
    fedilink
    English
    arrow-up
    1
    ·
    9 days ago

    I host tons of open source projects on https://forge.sciactive.com/

    Recently, it was getting absolutely hounded with tons of meaningless requests. I’m assuming they are just results of AI.

    So, I put Anubis in front of it. Now images hosted there won’t load on other sites, people have to wait forever to see the code, and the experience on phones is even worse. Just because AI doesn’t respect anything, and the people who run it are too stupid to realize that.

    This is why we can’t have nice things.

  • Zak@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    9 days ago

    I host a few sites and I haven’t had a problem yet. Sure, logs suggest a majority of traffic is bots, but the cheap VPS I’m hosting on hasn’t melted so I haven’t been motivated to do anything about it.

    • Dave@piefed.nz
      link
      fedilink
      English
      arrow-up
      0
      ·
      9 days ago

      Lemmy is particularly susceptible. Hosting a static site with a limited number of pages is fine. But Lemmy needs to check all posts, find the votes, sort them, and compile the page. It does that for practically every page load, and so loading a page once is quite “expensive”.

      Then add on to that the almost unlimited pages of content for bots to crawl through, bots just get caught endlessly scraping pages. Even powerful servers struggle, which is why most Lemmy instances have implemented some sort of protection against bots.

      • frongt@lemmy.zip
        link
        fedilink
        English
        arrow-up
        0
        arrow-down
        1
        ·
        9 days ago

        It doesn’t need to. Like any similar site, it can cache a lot of that in databases like redis, or cache the rendered page in part or in whole.