Shell
How to Run a 27 Billion Parameter Neural Network on a Regular iPhone or Home PC
Bonsai-27B uses 1-bit and ternary quantization to fit a 27B parameter model into just 3.5 GB of RAM, making it runnable on iPhones and laptops without discrete GPUs.