I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?
I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?