In my regular research (behind a paywall), I have been saying for a while that I think the future of AI is not large language models (LLM), but small language models (SLM) run on local desktop computers or even mobile phones.
but I suspect they are still going to be bigger than most can comfortably host for time being
ARM and CXMT both have the potential to change this situation rather dramatically.
ARM: The RAM/VRAM divide is kind of a legacy thing at this point, doing more harm than good in the x86 (desktop OS) space, but ARM doesn’t have the same baggage.
CXMT: We know the DRAM cartel have previously engaged in price-fixing, and the current shortage looks suspiciously similar to their old behavior. When confronted by a new challenger, they might be forced to actually compete.
Hence “x86 (desktop OS) space”. It’s not intrinsically part of x86, but it has settled in as a conventional piece of x86 desktop OSes. x86 consoles and ARM desktops don’t assume the same principle, and they can get a lot more mileage out of SoCs as a result.
Yes. The issue is not that ARM systems don’t/can’t communicate with a dGPU over PCIe. It’s that x86 systems (edit: running conventional x86 desktop OSes) have to basically pretend they’re using PCIe to communicate with an iGPU. (Not literally, but a lot of the primitives they use were inherited from AGP/PCI and don’t make a lot of sense for an SoC.)
I wouldn’t say it’s totally legacy. (v)ram bandwidth does matter and while the Apple M-series chips do well with their unified memory architecture don’t forget it’s fixed because it’s part of the CPU chip.
ARM and CXMT both have the potential to change this situation rather dramatically.
ARM: The RAM/VRAM divide is kind of a legacy thing at this point, doing more harm than good in the x86 (desktop OS) space, but ARM doesn’t have the same baggage.
CXMT: We know the DRAM cartel have previously engaged in price-fixing, and the current shortage looks suspiciously similar to their old behavior. When confronted by a new challenger, they might be forced to actually compete.
That has nothing to do with x86 though…
Hence “x86 (desktop OS) space”. It’s not intrinsically part of x86, but it has settled in as a conventional piece of x86 desktop OSes. x86 consoles and ARM desktops don’t assume the same principle, and they can get a lot more mileage out of SoCs as a result.
Aren’t there arm workstations that still take pcie cards and have a similar architecture to x86?
Yes. The issue is not that ARM systems don’t/can’t communicate with a dGPU over PCIe. It’s that x86 systems (edit: running conventional x86 desktop OSes) have to basically pretend they’re using PCIe to communicate with an iGPU. (Not literally, but a lot of the primitives they use were inherited from AGP/PCI and don’t make a lot of sense for an SoC.)
@jumping_redditor @kibiz0r
You really need to read this series:
https://marcin.juszkiewicz.com.pl/2025/07/07/arm-desktop-2025-attempt-part-one/
I wouldn’t say it’s totally legacy. (v)ram bandwidth does matter and while the Apple M-series chips do well with their unified memory architecture don’t forget it’s fixed because it’s part of the CPU chip.