Sun, 26 Apr 2026 18:48:41 +0000

ok. I can say, for being just a Radeon 7600M XT, it can run the Gemma4:26B model at 18 tokens per second. I like that rate. The Gemma4:31B model is genuinely rough though, I can only fit about a 13B quantized model. These new MoE models really interest me y'all. MoE models with ≤4B active params and total Q4 size under ~50GB run at "active-param dense equivalent" speeds on this setup. Effectively I'm running a 3.8B-active model with 26B-quality routing, and the OCuLink bandwidth is just barely fast enough to swap inactive experts without choking the active hot path.
Qwen 3 30B-A3B (3B active, ~17GB) would also work well here. Qwen 3.6 35B-A3B (3B active, ~20GB, multimodal/vision) is another one I'm trying.
But no going real crazy like with Qwen 3 235B-A22B. No way will that ever run.
So, there's a more useful post from me. Yes, I'm exploring the idea: Why not use Ollama over MCP with Claude Opus 4.7 or another flagship model to have it actively ask it questions and then verify final major decisions with me for a more automated set-up? That way if it's a "should we investigate (A) or (B)" type question, it can defer it out through Ollama to a capable second model locally. Why not. Makes my life simpler