Hello,

I have been looking into a new laptop, and was coming across ones with these NPUs heavily advertised in them. Doing some reading, they don’t seem extremely functional at this stage.

They are around 45-50 TOPS at the highest it seems. I found some articles and comments suggesting that ‘could’ be useful for locally using smaller models, but also statements conflicting with that. As well, most, if not all, ‘technical use’ of them seem locked into the Windows environment. Even some program from AMD allowing local LLM use requires a Windows Server for it to communicate with, iirc. (AMD, GAIA)

So, is there, currently, any technical use for these, such that it makes much sense to grab a device with one for tinkering?

I’d considered experimenting with smaller models and seeing what comes of those (if small model improvements come through as DeepSeek proponents might suggest).

I’m also just generally new to the technology, but intrigued by the potential to localize usage; not only because of the potential to limit the environmental impact of large data center use.

Any comments, ideas, suggestions, or general pointing in a direction is very appreciated.

Thank you for taking the time. Have a good day!

  • vimmiewimmie@slrpnk.netOPEnglish
    1·
    5 months ago

    Thank you very much for all of this. It makes it seem very possible to at least experiment with these LLMs.

    Unfortunately it’s not likely I’ll be able to take your advice.

    I’ve been looking into new laptops in case I can find a job in tech again, but I don’t have the funds to buy anything. This is the system I have. Though, some of its hardware is failing I think. The battery failed years ago, so it must stay plugged in all the time. Thankfully it turns on even if I remove the battery so it’s not an overcharging concern.

    Sorry I couldn’t try your advice yet.

    I’ll come back to it if I can.

    Thank you again!

    • sga@piefed.socialEnglish
      1·
      5 months ago

      as i said, you do not even need new hardware. for tiny models, your 5 year old (or whatever) cpu is fine.

      I do not mean to press you, just maybe try it once. I for one do not find much use for them unless i have to send something to some superior for some proposal, and i just need grammatical proof read or tonality check.

      I just advocate for local use because the less you rely on big tech, the better. they harvest your data, and also take money from you to use your data. they ask you to pay even more if you do not want them to use it and that just sounds like mafia behaviour to me.