I turned my old phone into a local LLM server, and it handles productivity tasks better than I expected
I turned my old phone into a local LLM server, and it handles productivity tasks better than I expected
摘要: I turned my old phone into a local LLM server, and it handles productivity tasks better than I expected
Ayush Pande
Published
Aug 31, 2026, 8:0
正文
I turned my old phone into a local LLM server, and it handles productivity tasks better than I expected
Ayush Pande
Published
Aug 31, 2026, 8:00 PM EDT
Ayush Pande is a PC hardware and gaming writer. When he's not working on a new article, you can find him with his head stuck inside a PC or tinkering with a server operating system. Besides computing, his interests include spending hours in long RPGs, yelling at his friends in co-op games, and practicing guitar.
Sign in to your
Despite my initial scepticism, locally-hosted LLM models have gotten a lot of oomph to their reasoning capabilities as of late. Newer Mixture-of-Experts models, for example, can run at respectable token rates on my outdated Pascal-era cards, and with a little bit of tinkering, LLMs such as
Qwen3.6-35B-A3B
Gemma-4-26B-A4B
can easily replace their cloud counterparts for my FOSS stack.
However, PCs aren’t the only devices that can drive dedicated AI models for my productivity tasks. Once I realized that my NAS can drive certain models for simple tasks, I began tinkering with different devices in my house. My latest LLM-hosting experiment centered around the Poco M6 Pro – the same cheap and outdated phone I typically use to run simple containerized services on-the-move. As it turns out, with a little bit of llama.cpp magic and certain edge LLMs, it’s possible to use this old phone for simple AI-powered productivity tasks.
I decided to go with good ol’ llama.cpp as the inference engine
Termux made the installation process fairly painless
Before I could try out a local LLM, I needed to install an inference engine on my phone, and for better or worse, I had dozens of options to choose from. While I’ve heard good things about MLC Chat, AnythingLLM Mobile, PocketPal AI, and some other applications, I wanted an inference engine with an extremely tiny footprint on my phone. After all, local AI models can be absolute performance hogs, and since I was planning to