I Built a CoPilot+ AI PC (without Windows)
TLDRIn this video, the creator unveils a custom Raspberry Pi AI PC, the 'CoPilot,' which challenges Microsoft's Copilot with 13 TOPS of compute, later boosted to 51 TOPS. The $70 Raspberry Pi AI kit, featuring an M.2 HAT and Hailo NPU, is highlighted for its efficiency and potential in machine vision applications. Despite the AI hype, the kit offers practical solutions for real-world problems like traffic monitoring and quality assurance. The video explores the kit's capabilities, compares it with GPUs and other AI devices, and concludes with an ambitious, if not entirely successful, attempt to maximize neural compute power.
Takeaways
- 🤖 The CoPilot PC is a custom Raspberry Pi AI PC, not to be confused with Microsoft's Copilot.
- 🚀 It aims to achieve over 40 TOPS of neural compute, which is more than Apple's M4 or Snapdragon X.
- 💾 The $70 Raspberry Pi AI kit includes an M.2 HAT and a 13 TOPS Hailo NPU for machine learning tasks.
- 🔎 The kit is efficient with 8 TOPS per watt, making it suitable for tasks like object detection and image segmentation.
- 📈 The AI kit is designed for practical applications such as machine vision and real-time video processing.
- 🌟 Raspberry Pi is focusing on real-time video applications like selective recording and traffic monitoring.
- 🔩 The AI kit is a PCI Express device and can be used on other computers besides the Raspberry Pi.
- 🚀 The video demonstrates the potential of combining multiple AI accelerators to achieve over 50 TOPS.
- 🔋 There are limitations, such as power requirements and the complexity of using multiple devices.
- 🛠️ The Raspberry Pi AI kit offers an alternative to the Coral TPU and is expected to evolve with more exposure.
- 👨💻 The presenter, Jeff Geerling, suggests that while the AI kit is niche, it will be loved by those in machine vision and robotics.
Q & A
What is the main difference between the CoPilot PC and Microsoft's Copilot?
-The CoPilot PC is a custom AI PC built using a Raspberry Pi with an AI kit, whereas Microsoft's Copilot is a software development tool. The narrator built the CoPilot PC because they didn't like the way Microsoft marketed their Copilot.
What is the minimum neural compute required for Microsoft's Copilot?
-Microsoft states that you need at least 40 TOPS of neural compute for Copilot.
What is included in the $70 Raspberry Pi AI kit?
-The $70 Raspberry Pi AI kit includes an M.2 HAT and a 13 TOPS Hailo NPU, which is like a machine learning jetpack for the Raspberry Pi.
How does the efficiency of the Hailo-8L NPU compare to the Coral TPU?
-The Hailo-8L NPU runs at 13 TOPS and 8 TOPS per watt, which is more efficient than the Coral TPU that provides 2 TOPS at around 2 TOPS per watt.
What are some practical applications of the Raspberry Pi AI kit?
-The Raspberry Pi AI kit is geared towards machine vision applications such as object detection, pose estimation, image segmentation, and can be used in robotics, safety monitoring, and quality assurance.
Why did the narrator choose not to integrate an NPU into the main Raspberry Pi chip?
-Raspberry Pi doesn't build their own main chips; they rely on Broadcom. The decision to not integrate an NPU into the main chip would be up to Broadcom.
What is the difference between using an NPU and a GPU for AI tasks?
-GPUs are great for training and running models but have higher power requirements. NPUs like the Hailo are better for low-power edge devices where efficiency is crucial.
What limitations does the Raspberry Pi AI kit have in terms of RAM and training models?
-The AI kit doesn't offer a lot of RAM, limiting the size of the models that can be run on it. Training models on the Pi is possible but slow, taking days instead of minutes or hours on a modern GPU.
What is the purpose of the Pineboard's HatBrick! Commander mentioned in the script?
-The Pineboard's HatBrick! Commander is a PCI Express board that adapts one port into two, allowing the connection of multiple AI accelerator boards to the Raspberry Pi.
What is the maximum neural compute the narrator achieved with the Raspberry Pi AI kit and additional PCI Express devices?
-The narrator achieved a maximum of 47 TOPS (actually 51) by connecting multiple AI accelerator boards to the Raspberry Pi, although not all were usable due to power limitations.
What alternative options does the narrator suggest for those who need more neural compute than the AI kit provides?
-For those who need more neural compute, the narrator suggests buying a more powerful NPU, such as Hailo's line of bigger cards with 52 to 208 TOPS of performance, starting at $250.
Outlines
🤖 Custom Raspberry Pi AI PC Introduction
The speaker introduces a custom AI PC built using a Raspberry Pi, which they've dubbed 'CoPilot PC'. Unlike Microsoft's Copilot, this is a DIY project. They critique Microsoft's marketing of Copilot and highlight the Raspberry Pi AI kit's 13 TOPS of neural compute, which is less than Microsoft's requirement but more efficient than Apple's M4 or Snapdragon X. The kit includes an M.2 HAT and a Hailo NPU, offering 8 TOPS per watt. The speaker compares it to the Coral TPU used in their Frigate server, which detects objects at 2 TOPS per watt. The AI kit is geared towards practical applications like machine vision and is expected to be popular with Pi cameras. Raspberry Pi's focus is on real-time video applications, which can solve real-world problems in various sectors.
🚀 Enhancing Raspberry Pi with Multiple AI Accelerators
The speaker demonstrates how to enhance the Raspberry Pi's AI capabilities by adding multiple AI accelerators, including the Hailo NPU and Coral TPUs. They use Pineboard's HatBrick! Commander to expand the number of available PCI Express ports. Despite the complexity of the setup, they manage to get the system to recognize the accelerators, achieving a total of 47 TOPS [51 actually] of neural compute. However, they encounter issues with power supply and initialization, suggesting that the system might be drawing too much power for the Pi's PCI Express port to handle. The speaker concludes that while the AI kit is powerful, it may be more practical to use a more robust NPU for high-performance needs.
🔍 Evaluating the Raspberry Pi AI Kit's Practicality
The speaker reflects on the practicality of the Raspberry Pi AI kit, noting that while it's a niche product, it serves a specific audience well, particularly those interested in machine vision and robotics. They compare it favorably to the Coral as an alternative AI solution. The speaker also speculates on future developments in the field, such as the potential for more integrated AI solutions and the emergence of new products that combine AI with camera modules. They conclude by reiterating their appreciation for the AI kit, despite acknowledging that it may not be suitable for everyone.
Mindmap
Keywords
💡CoPilot
💡TOPS
💡Raspberry Pi AI kit
💡Hailo NPU
💡Coral TPU
💡Machine Vision
💡AI Hype Train
💡Neural Compute
💡HatBrick! Commander
💡Model Zoo
💡Machine Learning
Highlights
Introduction to the custom CoPilot PC, a Raspberry Pi AI PC.
Comparison with Microsoft's Copilot, highlighting differences and personal views.
Microsoft's requirement of at least 40 TOPS of neural compute for Copilot.
The CoPilot PC's capability of reaching 47 TOPS [actually 51], surpassing Apple's M4 and Snapdragon X.
The $70 Raspberry Pi AI kit includes an M.2 HAT and a 13 TOPS Hailo NPU.
Comparison of the Coral TPU and Hailo-8L NPU in terms of TOPS and efficiency.
The focus on machine vision and real-world applications of the AI kit.
Raspberry Pi's preannouncement of an AI camera at Embedded World.
The versatility of the AI kit as a PCI Express device for various computers.
Examples of real-world applications like factory inspections and traffic planning.
The potential of the AI kit for robotics and safety monitoring.
The question of why Raspberry Pi doesn't integrate an NPU into their main chip.
Comparison of different approaches to AI hardware by various companies.
The limitations of the AI kit in terms of RAM and model training capabilities.
Demonstration of the YOLOv5 object identification model running on the Pi 5's CPU and Hailo NPU.
The benefits of using an AI coprocessor for power efficiency and CPU usage.
The potential of the AI kit for pose estimation and gesture-based applications.
The use of image segmentation for applications like iPhone's portrait mode.
The challenge of achieving more than 13 TOPS of neural compute to beat Microsoft's requirement.
The attempt to connect multiple AI accelerators to the Pi and the issues encountered.
The conclusion that a beefier NPU might be a better solution for high TOPS requirements.
The potential future of AI in microcontrollers and camera modules.
Final thoughts on the AI kit's niche appeal and its value for machine vision and robotics.