Raspberry Pi 5 Gets Local AI Upgrade with AI HAT+ 2

Raspberry Pi has introduced the AI HAT+ 2, a new hardware add-on that enables local execution of large language models (LLMs) and vision-language models (VLMs) directly on the Raspberry Pi 5, without relying on cloud infrastructure.

 

Unlike previous AI HAT versions that focused mainly on computer vision tasks, the new AI HAT+ 2 significantly expands capabilities by supporting generative AI workloads, including text generation, code generation, translation, and visual scene understanding.

 

At the core of the upgrade is the Hailo-10H neural accelerator, delivering up to 40 TOPS of INT4 inference performance. The board also includes 8 GB of dedicated onboard memory, allowing models to run independently without consuming system RAM, which is an important improvement for edge AI applications.

 

The AI HAT+ 2 connects to the Raspberry Pi 5 via PCIe, enabling high-bandwidth data transfer between the accelerator and the host system. This design supports low-latency inference and efficient handling of camera input, model outputs, and real-time processing. Due to this architecture, the board is not compatible with Raspberry Pi 4.

 

 

The platform supports models in the 1–1.5 billion parameter range, including LLMs such as Qwen2 and Qwen2.5-Coder. It enables tasks such as local text-based question answering, code generation, language translation, and visual scene description from live camera input.

 

All processing is performed entirely on-device, without cloud dependency, making the solution suitable for privacy-sensitive and offline applications.

 

To adapt models for specific use cases, the AI HAT+ 2 supports fine-tuning through Low-Rank Adaptation (LoRA) and retraining of vision models using Hailo’s toolchain. While the hardware is not designed to compete with cloud-scale AI systems, it enables practical deployment of generative AI within constrained edge environments.

 

Priced at $130, the AI HAT+ 2 is positioned above previous Raspberry Pi vision accelerators. Its main value lies in enabling local generative AI workflows rather than delivering higher raw computer vision performance.

 

Overall, the release demonstrates that running LLMs on Raspberry Pi hardware is now feasible, although limitations in memory capacity and model size still define the boundaries of what can be achieved on-device.

 

Reference: