LLM Inference Engine for Metal, CUDA and Vulkan.
★ 2,253⑂ 521👁 11⑃ 2,593● 863▤ 63
+38
日增长
2,253
stars
521
forks
11
watchers
2,593
prs
863
issues
63
releases
README
TensorFold TensorFold serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation. Building with a coding agent? Have it load the contributor skill before it writes any code. Contributing with agents Please use Agent Traffic Control before editing TensorFold so contributors and agents can avoid duplicate work and conflicting changes. Check the live agent radar before choosing an issue and follow CONTRIBUTING.md and the ATC contributor instructions. Install or upgrade ATC to 0.9.4 or later, then run from an updated TensorFold clone: sh npm install -g github:ashhart/ATC atc check ISSUE --json atc claim ISSUE --json Replace ISSUE with the issue number and claim only after a complete check says the work is free. TensorFold enables automatic public check-ins: a successful new claim starts an anonymous session on the radar and returns its session number. Keep that number instead of starting a duplicate session. For work already yours, start a session with atc start ISSUE --json. Add --github-name to claim or start only if you want your verified GitHub name shown; --local keeps a new session on your machine. The board shows the issue, activity state and last check-in; GitHub still associates the verification comment with its author. Check that publication is public; unavailable means the public update failed even if the local claim or session succeeded. Pulse only while the agent is working, before its five-minute check-in expires: sh atc pulse ISSUE SESSION working --json atc who path/to/file --json atc overlap --json atc land ISSUE SESSION --json atc release ISSUE --json Use reviewing or blocked when appropriate, land the session when it stops, and resolve overlaps before opening a pull request. Before opening a pull request, have the agent run the pull request skill; agents that support skills can install it as tensorfold-pull-request. Quick start Start with Qwen3.5-2B, the small 4-bit checkpoint checked on Metal and a GB10. The commands below download its pinned revision, then serve it as local-model. Model weights have their own licenses. Reviewing or benchmarking TensorFold? Evaluating TensorFold covers pinning, signature checks, a container and the tests that need no GPU. Mac with Apple Silicon Install with Homebrew, then start the server: sh brew install ashhart/tensorfold/tensorfold tensorfold pull mlx-community/Qwen3.5-2B-MLX-4bit@93760be4f1f69842a46bc13dbdc0f19e291392a3 tensorfold serve mlx-community/Qwen3.5-2B-MLX-4bit --name local-model --context 8192 --no-thinking Linux with an NVIDIA GPU Use the release archive for your CPU: linux-aarch64 on a GB10, linux-x86_64 on an x86_64 host. It needs glibc 2.28 or newer and NVIDIA driver R580 or newer (CUDA 13.0), providing libcuda.so.1. Keep the archive's directories together. Its kernels are built with CUDA 13, so TensorFold re
Top issues
#1491Add native Metal support for LFM3-1.7B and LFM3-VL-2B MLX 4/6/8-bit checkpoints63 comments · 2026-10-27#3136Native CUDA: bring up Spark-X2.6-4B affine-4 inference on Pascal sm_6151 comments · 2026-11-13#2445Qwen3.8: integrate remaining Yukon CUDA and Metal improvements49 comments · 2026-11-07#2406Qwen3.8-27B on native Windows RTX 5090: 250 tok/s single-stream prose gate48 comments · 2026-11-07#2433Qwen CUDA: opt-in native NVFP4 prefill with preserved decode and end-to-end gates41 comments · 2026-11-07#2667Proposal: fine-tune DSpine-b16 on the 4-bit Qwen3.8-27B target, with `--record` for Qwen3.8 on CUDA26 comments · 2026-11-09#678Living Weights: conflicting facts, updates and forgetting (measurements + proposal)26 comments · 2026-10-11#2741Opt-in coherent execution witness for exact-response cache qualification25 comments · 2026-11-10
Latest release
TensorFold 1.2.9 v1.2.9 2026-11-15TensorFold 1.2.9
1.2.9 brings faster decode and long prompts, an Ollama-compatible API, and fixes for a wrong reply and a hang. Every
speedup below keeps replies unchanged.
Speed
- Nemotron on Metal (without an MTP head): concurrent streams share one round. On an M5 Ultra, two streams decode
about 20% faster, four about 75%, eight 2.3 to 2.5 times and 24 streams 3.2 to 3.5 times (3005).
- GLM-5.3-Flash on CUDA:
- Long prompts' first token comes 0.6% to 13.7% sooner at TP=3 from 131,072 to 2,000,000 tokens, and 6.7% sooner at
TP=2 at 1,000,000; shorter prompts at TP=2 stay within ab