AirLLM

Run 70B-parameter LLMs on a single 4GB GPU — AirLLM does layer-by-layer inference so huge models fit on modest hardware, with no quantization, distillation, or pruning required.