drumih /turbo-fieldfare

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

★ 7,403⑂ 482👁 35⑃ 115● 87▤ 25
SwiftApache-2.0Created 2026-07-17
下载:0.10.00.10.0 ↓0.10.02026-11-14
7,403
stars
482
forks
35
watchers
115
prs
87
issues
25
releases

README

TurboFieldfare


  Gemma 4 26B-A4B inference in about 2 GB of RAM
  A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones.



  
  
  
  



  Quick start ·
  Local server ·
  Benchmarks ·
  Contribute results ·
  How it works ·
  Experiments ·
  References


!TurboFieldfare Mac app with local chat history, LaTeX rendering, and run statistics

Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget.

TurboFieldfare runs the instruction-tuned
Gemma 4 26B-A4B
without loading the entire 14.3 GB model into memory. It keeps the shared
1.35 GB core and FP16 KV cache in memory, then streams only the experts needed
for each token from SSD. This is what lets the model run on Macs with 8 GB of
RAM.

The runtime, streaming installer, CLI, and native Mac app are written in Swift
and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or
llama.cpp. The curated experiment record
summarizes 103 measured results across kernels, caching, I/O, prefill, and
decode.

 Try it

bash
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac


On the first run, Swift Package Manager downloads and builds the Swift packages
required by the tokenizer. The complete release build includes the foreground
Mac app and its sibling decode-service executable.

When the app opens, choose Download Model and let TurboFieldfare fetch and repack
the pinned model (about 15 GB). Once it is ready, choose Load Model, type
your prompt, and press Generate.

 At a glance

| Metric          | Value                                                                                                                    |
| --------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Model           | Gemma 4 26B-A4B IT, 26B total parameters, about 3.88B active per token                                                   |
| Weights         | MLX affine 4-bit, group 64; 8-bit router; 4-bit shared and routed experts                                                |
| Memory          | ~2 GB of weights and 4K KV cache                                                                                         |
| Storage         | About 14.3 GB for the text model, plus about 1.1 GB for the optional image pack                                          |
| Hardware        | Apple Silicon Mac; 8 GB of RAM                                                                                            |
| Platform        | macOS 26, Metal 4, Swift 6.2                                                                                             |
| M2 measured decode | 5.5–6.8 tok/s on an 8 GB M2 MacBook Air, expert prefetch on |
| M5 measured decode | 35.3–39.2 tok/s on a 24 GB M5 Pro, expert prefetch off |
| Community Reports | Here |

The measured result is a reference point, not a performance ceiling. Prom

Top issues

Latest release

0.10.0 0.10.0 2026-11-14
Speculative expert prefetch

Expert prefetch can speed up generation on devices with less RAM.

- Enable Expert prefetch in the Mac app, then reload the model. The setting is saved.
- CLI and server users can enable it with --early-expert-read router.
- Improved reader cleanup during cancellation, errors, and model reuse.
- Updated benchmarks and experiment documentation with M2 and M5 Pro results.

Thanks to @maxslamdunk for the expert-prefetch implementation in 187.

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON