How to Use Ollama on Raspberry Pi 5
Run Phi3, TinyLlama and Llava models
Written By: Cherie Tan
Difficulty
Easy
Steps
3
Ever wanted to test various large language models (LLMs) on Raspberry Pi devices, or simply run them locally? The guide walks you through on installing and running different models like Phi3, TinyLlama, and Llava on the Raspberry Pi 5!
Phi3 is a lightweight language model developed by Microsoft, designed to offer quality results similar to those of much larger models.
Due to its efficiency, Phi3 is an excellent choice for running on a Raspberry Pi. It should provide a usable speed, though not exceptionally fast responses, but the Pi 5 can handle it well.
- To download and start the Phi3 LLM on your Raspberry Pi using Ollama, enter the following command:
ollama run phi3 - Once Phi3 is running on your Raspberry Pi, you can begin interacting with the AI model.
Note: This process may take some time, as Phi3 is still a 2.3GB model.
Be aware that while Phi3 is functional, it may have slower response times for longer answers. If you require a faster LLM for your Raspberry Pi, you might need to consider other models below!
TinyLlama is an extremely lightweight and fast LLM, making it an excellent choice for running on the Raspberry Pi’s processor.
Although it may not produce results as high-quality as larger models like Phi3 or Llama 3, TinyLlama is more than capable of handling basic questions. What it lacks in quality, it compensates for with its impressive speed.
- To run TinyLlama on your Raspberry Pi using Ollama, enter the following command:
ollama run tinyllama - Once the model is running, you can begin interacting with the AI by asking it questions.
Note: This model is quite lightweight, so the download process should be relatively quick.
This is the largest LLM I've tested, boasting 7 billion parameters. Instead of simple queries, I tasked it with describing an image.
In this guide, we are using a Raspberry Pi 5 with 8GB of RAM. However, note that it may not perform as well as smaller language models mentioned earlier.
Nevertheless, let's give it a try! To install llava, use the command:
ollama run llava Note: Downloading this model will take some time, as it's quite large, approximately 3.9 GB.
Likewise, once it's finished installing, you can begin asking it questions!
Likewise, once it's finished installing, you can begin asking it questions!