☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 2 months agoGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktopimagemessage-square59linkfedilinkarrow-up1140arrow-down115cross-posted to: Aii@programming.dev
arrow-up1125arrow-down1imageGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktop☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 2 months agomessage-square59linkfedilinkcross-posted to: Aii@programming.dev
minus-squarestuner@lemmy.worldlinkfedilinkarrow-up4·2 months agoI run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
minus-squareCameronDev@programming.devlinkfedilinkarrow-up3·2 months agoI’ll give LM studio a go, thanks.
I run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
I’ll give LM studio a go, thanks.