• 1 Post
  • 124 Comments
Joined 1 month ago
cake
Cake day: July 4th, 2026

help-circle




  • why I personally think the current AI “boom” is a bubble around a nascent tech that’s not really ready for lots of the situations it’s been sold for

    Basically this. On top of that, we not far off from the point where you can run something like claude localy. Obviously we don’t have access to claude itself, but we do have access to local big models that come close to it in performance, like kimi-k3 ~500gb on average, deepseek ~1tb or glm5.2 (smallest quant starts from 228gb)

    And it may sound like a lot, but it isn’t in reality — most amd cpus from am4 able to address 128gb of ram, while new am5 can do 256gb. Hell, my gaming PC have 128gb of DDR5. So we already technically can run some of those models on a regular office PC. Obviously, due to ram crisis, it will cost you a fortune, but it’s not like it impossible. Furthermore, demand for gpu with a lot of vram will sooner or later be satisfied, so it just matter of time when average consumer will be able to do it. And well, why would we need dedicated data centers then?


  • I have my eyes on old 32gb amd instinct cards. One of those from ali will cost you ~450-500 euro, which is a lot for old unefficient card, but does provide amazing bandwidth and 32gb of vram. With just one you can run those models almost with full 256k context (or whatever the number there is). Just 4 of those and you can even run models that come close to things like claude and have 1m token context window

    Sadly, I can’t justify it — my rig already have 7900xtx + 4060ti (so 32gb vram in total) and realistically I know that there no reason to buy it right now in memory crisis. In couple of years either Chinese manufacturers will catch up ram wise or someone will come up with dedicated device/accelerator for half the price. Or done other similar breakthrough will happen, maybe 4/8 channels of ram on regular desktop will become a thing, instead of being HEDT/server only














  • (not op, but) Hear me out — what if it’s actually okay for me to use those “free plan” chats? Like, I’m not from US where most of data-centers and companies located, I’m not the one paying with my electricity bill and taxes for those “free” llms — average american Joe is.

    Dunno if this meme popular outside of post-soviet counties, but it’s closest analogy as I see it:

    Meme explanation

    I believe it was made as anti-capitalism propaganda, which is kinda ironic, considering how divided poor worker class and government people in USSR were (google shops “Berezka” that was only available for upper class), but I digress.

    Usually accompanied with slogan (which became name of the meme):

    This gentleman will pay for everything