As you already know, most of my local AI tests happen on a 16 GB MacBook Pro. Models in the 4B–9B range are the sweet spot there, and I’ve even run the Ternary Bonsai 2, at 27B (on a heavy quant). This time I wanted to see if you can run a more than one billion parameter model on much smaller hardware: a Raspberry Pi 4 with 4 GB of RAM.

I chose LFM3 1.7B from Liquid Foundation Models. It’s small enough that it should fit, but still large enough to be interesting. It’s the same model I tried in my previous test, as an in-browser model, with custom webGPU kernels. I added a small touchscreen on my Pi and ran everything locally — no SSH, nothing leaving the device.

Optimization Came from Parameter Tuning and a Custom Build of llama.cpp

Setup was straightforward but required a bit of tuning. I installed llama.cpp, played with the parameters a bit, mainly context size and thread count, then compiled a custom binary using ARM-specific flags for the Pi’s processor. The custom build increased generation speed from about 2 tokens per second to roughly 3 tokens per second. That’s a 50% improvement. At 3 tok/s it’s still slow by normal standards, but it’s close enough to reading speed that it almost starts to feel usable for light tasks. Almost.

I ran the same two tests I usually do. The text generation prompt (“what is a dolphin”) produced a surprisingly complete and detailed answer — one of the more thorough responses I’ve gotten from small models in these tests. The coding task was a simple word-count script in Python. The model generated working code and included basic instructions for running it. Both tests stayed entirely on the Pi. To my surprise, the coding tasks was finished quicker than text generation, but I suppose it was because the text about dolphins was pretty long, and just generating the tokens took some time.

A Raspberry Pi 4 with 4 GB is a low-end device. It’s not realistic to use it as a main coding machine. My medium term plan is to treat it as an orchestrator: it can handle medium or low-complexity tasks on its own and forward heavier work to more powerful machines. At 3 tokens per second it’s slow, but for that role it may be enough.

I’ll keep experimenting with further optimizations and different models on this setup. If the Pi can reliably run small agent-style tasks without needing to call out every time, it becomes a useful piece of a larger local stack.

If you like this kind of content, like, share and subscribe to my YouTube channel!