Hardware

Running a language model locally on a Raspberry Pi 5 is practical in 2026 — if you pick the right model. The Pi 5 (8 GB) handles 1–3B parameter models at speeds that work for interactive tasks without a cloud connection or dedicated AI hardware. This is CPU-only inference, which sets a hard ceiling: expect 5–15 tokens per second for 1.5B models, and 2–5 tokens per second for 3B models. This guide covers setup with both Ollama and llama.cpp, real benchmark data, and which models are worth running on Pi hardware. ...

Apple Silicon Macs are the best consumer hardware for running small language models locally in 2026. The reason is architectural: Apple’s unified memory architecture (UMA) lets the GPU and CPU share a single physical memory pool, eliminating the model-size ceiling that limits every Windows laptop and desktop GPU to their VRAM capacity. A Mac with 32 GB of unified memory can load a 32 GB model and run full GPU-accelerated inference on it — no swapping, no offloading. ...

The short answer: for a 3B model at Q4_K_M quantization you need about 4 GB of free RAM. An 8B model needs closer to 7 GB. A 32B model won’t move at all without 22+ GB available. The longer answer depends on which runtime you’re using, how much context you need, and whether you have a GPU. This post breaks down exactly where that memory goes, with verified file size data pulled from HuggingFace, so you can size your hardware against a specific model rather than guess. ...