LOC · category

Local inference

Running big models on small hardware — local, efficient, offline-friendly.

2 tools in this category

AirLLM

Run 70B-parameter LLMs on a single 4GB GPU — AirLLM does layer-by-layer inference so huge models fit on modest hardware, with no quantization, distillation, or pruning required.

building blockApache-2.0pythonllm

view source ↗

LOC·02★ 24k

Colibrì

A pure-C, zero-dependency inference engine that runs frontier Mixture-of-Experts models — like GLM-5.2's 744B — on hardware you already own. It keeps the dense core in RAM and streams experts from disk on demand, treating VRAM, RAM and storage as one tiered memory. It even serves an Anthropic-compatible API, so Claude Code talks to it directly.

building blockApache-2.0inferencepure C

view source ↗