Qwen/Qwen3.8-27B · Hugging Face
huggingface.co
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
  • melfie@lemmy.zipEnglish
    6·
    17 hours ago

    So far, I’ve had 3.8 do a code review and also review some creative writing. It’s quite a bit more thorough than 3.6.

  • fubarx@lemmy.worldEnglish
    4·
    18 hours ago

    Qwen3.6 with 4-bit quant was hallucinating HARD. When asked to summarize the plot of obscure, cheesy 80s books, it confidently made up plot and characters. Not a single statement was correct. It was glorious.

    Any higher quant and it would take 30-90 seconds before first token. Both MLX and GGUF on an M3Max with 128G combined RAM. Will have to try it with this one.

    • e0qdk@reddthat.comOPEnglish
      7·
      17 hours ago

      In my testing so far, it seems to lean more into “I don’t know” type answers, but I haven’t poked at it that much yet.

      One of the queries that used to consistently trip up Qwen 3.5/3.6 was asking directly about a plausible sounding (but non-existent) work (e.g. Are you familiar with "A Valiant Effort (1989)"?); stock Qwen 3.8 will say it doesn’t know or isn’t sure and ask me for context.

    • Domi@lemmy.secnd.meEnglish
      6·
      18 hours ago

      27b fits easily into 128 unified RAM, even unquantized. So something was definitely wrong.

      Running 3.8-27b on my Strix Halo box right now.

      • ThyTTY@lemmy.worldEnglish
        2·
        17 hours ago

        Hey, I also have a Strix Halo box. Which model are you running exactly? The Q8 and BF16 are half the size of Qwen3-Coder-Next-UD-Q8 I have been running before. I tested 3.8-BF16 and UD-Q8 but both run at abysmal speed in comparison to Coder Next (around 8 t/s during chat). Which one gives you best results?

        • Domi@lemmy.secnd.meEnglish
          4·
          14 hours ago

          You need to use MTP (Multi token prediction) to get acceptable speeds on a Strix Halo. On llama.cpp you just need to add --spec-type draft-mtp --spec-draft-n-max 3 -np 1 to your arguments.

          You should see around 15-20 t/s in coding tasks at Q8, which will still be much slower than Qwen3-Coder-Next-UD-Q8 since it’s 27b active parameters instead of 3b.

          As for Q8 vs BF16, it usually makes barely to no difference in quality. Speed is higher on Q8 though. Still testing with 3.8 since it just released.

          • ThyTTY@lemmy.worldEnglish
            1·
            5 hours ago

            Thank you, I will check this out. So what I got from this is that the speed may be slower but results should be more accurate?

      • fubarx@lemmy.worldEnglish
        2·
        17 hours ago

        On a DGX with 128G unified I could push Qwen3.6-27B to 8-bit and it had pretty decent response. It just croaked with MLX on the Mac.

        Waiting to see what Apple announces next month. Judging by RAM prices, it’ll be waaay outside my budget.

    • floofloof@lemmy.caEnglish
      2·
      17 hours ago

      It did a good job with coding, even with the quantization. Did they prioritize that in its training perhaps, at the expense of other things?

  • troed@fedia.io
    2·
    20 hours ago

    Having issues with it (Unsloth’s gguf).

    0.27.136.404 I srv  proxy_reques: proxying request to model unsloth/Qwen3.8-27B-GGUF:Q4_K_M on port 55741
[55741] 0.14.424.772 W srv    operator(): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 106, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}