@perpetuum_mobile@mastodon.social
2026-04-23 14:08 UTC
@mamba@mstdn.ca Gemma4:26B is sweet spot on my RTX3090:
- 108 token/s - that thing is FAST
- multimodal - you don't need second VLM model
- you can variably turn reasoning on/off using reasoning_effort header in API calls
- performance comparable and sometimes exceeding cloud Gemini-2.5-Flash
New Qwen 3.6 27B is also very strong, the only downside : 3 times slower than Gemma4
Fully local AI on consumer HW finally caught
up with cloud.
Replies (1)
-
@mamba@mstdn.ca 2026-04-23 14:19
@perpetuum_mobile@mastodon.social Since Gemma4 came out, I agree it's been the gold standard for performance vs compute. If SoC is the way forward for local compute (and I think its clear it is) the real jump happens when unified memory architectures can actually handle the token volume an agentic harness needs. Progress on memory overhead for long-context agents, combined with advancements in unified pool architecture, make this a real possibility in the near future.