cross-posted from: https://lemmy.dbzer0.com/post/77817291

After a lot of memes on Altman saying we don’t deserve RAM if we don’t use it 24/7 @ 100% load, and an earlier occasional interest in older mainframes, I became interested in how a community can share common compute power.

I am not so interested in technicalities, but more in how you see this resource being shared between a group of a thousand people having different needs, from basic accounting and shitposting to rendering 3d and ML jobs.

Everyone has a mid PC, but it’s personal? A couple of workstations for power users? A mainframe server with thin clients for all?

What scheme do you think makes more sense to serve everyone’s need if we deliberately build it from the ground up in a community that had no devices inherited from their past? And how is it applicable to integrate this IRL.

  • Lemvi@piefed.zipEnglish
    5·
    10 days ago

    I guess the most efficient option would be to have a distributed setup where most households have some local compute, which they make available for others to use when they don’t need it themselves. Kinda like with SETI@home or Folding@home.

  • Sims@lemmy.mlEnglish
    31·
    10 days ago

    The old petals framework did that via a torrent like protocol and could even train models over a distributed network, but that was years ago and I have no clue if such a system is viable with all the new features in new models.

    My guess is that we could exploit a distributed kv-cache ala deepseek, but they use their own proprietary algorithm for kv storage/retrieval and a huge datacenter, and with only normal internet connections, I’m not sure its viable fir people ?

    Big models are a problem, but something like aihorde with smaller models combined with a distributed agentic harness could be interesting.