Overview
AI-enabled Computer Vision is reshaping the retail world, from the typical experiences at a cashier-operated Point of Sale (PoS) to state-of-the-art automated stores. The complexity of the solution, the application of computer vision, and the AI models used depend on the sophistication of the intended shopping experience.
The reference implementations and core services help accelerate the development and deployment of AI-enabled Computer Vision solutions.
You can sign up to know about the upcoming releases.
Reference Implementations
Reference Implementations are open-source sample solutions that provide critical components required to build and deploy the various AI-enabled computer vision use cases. These reference implementations are not meant to be a complete solution. They are an open-source initiative, providing evidence of hardware and software components needed to drive critical components of computer vision workloads.
Automated Self-Checkout
The Automated Self-Checkout reference implementation drives object recognition of product SKUs at the self-checkout station.
Loss Prevention
The Loss Prevention reference implementation recognizes SKUs based on object, text, and barcode, which you can use to validate the product SKU with a higher probability of theft at checkout.
Shopping Cart*
The Shopping Cart reference implementation recognizes SKUs in the shopping cart and detects items placed in the cart and removed from the cart.
Autonomous Stores*
The Autonomous Stores reference implementation maps customers to a virtual basket and recognizes product SKUs being added and replaced from vending machines to fully fledged stores.
*You can sign up to learn about the release of Shopping Cart and Autonomous Stores reference implementations.
Core Services
The core service modules in this framework are the foundation for building AI-enabled computer vision checkout experiences. Built-in core services support specific functional requirements and integrate the necessary frameworks and libraries to reduce the effort in building vertical use case references.
The core services provide:
- Reusable vision pipeline for running any pre-trained deep learning model with encryption, decryption, preprocessing, and post-processing.
- Optimized middleware, frameworks, and domain-specific microservices for faster solutions development.
- Support for Go*, Python*, C* client libraries, and so on to help code easily.
- Use case-specific pre-validated data and the flexibility to run your benchmarks. Combination of hardware and software options to build solutions with scalability and maintainability.
- Choice of computing for the fixed functions to take advantage of diverse workload performance requirements.
Pipeline Management Service
Bringing together a vision-enabled checkout system involves managing a vision pipeline and understanding the computing needs. Retailers will not run a single vision pipeline focused only on checkout workload but will have to run multiple workloads to ensure optimized use of their hardware. Hence, the reference implementations will look at fixed and flexible vision pipelines.
The following figure shows the standard phases of a vision pipeline:
The pipeline includes several microservices that use the Intel hardware-optimized software ingredients to either distribute the workloads or deploy them in a single system. The roadmap showcases the features at a high level.
Performance Evaluation Service
The Performance Evaluation Service is an operating system agnostic tool that can be used for checkout reference use cases and can be tailored for other workloads to extract system metrics.
The Performance Evaluation Service is developed using the following open-source tools to derive the required metrics:
| Metric | Tools |
|---|---|
| CPU Utilization % | System Activity Report (SAR), Sysstat |
| GPU Utilization % | IGT (Integrated and Arc), Intel® XPU Manager (Flex GPU) |
| Compute Utilization % | Intel® XPU Manager (Flex GPU) |
| Disk Read and Disk Read (MBs) | Input/Output top |
| Power Draw (Watts) | Intel® Performance Counter Monitor |
| Memory Utilization % | Free |
| Memory Bandwidth Usage (MBs) | Free |
| Frames per Second | Part of the Pipeline |
| Last Log Update | Custom Pipeline |
| Total Text Count | Custom Pipeline |
| Total Barcode Count | Custom Pipeline |
The code comprises a pipeline profile that you can configure before running the performance test. The profile allows you to define the environment variables, language, and other pertinent parameters. You can use this tool with Docker* containers to quickly deploy and benchmark workloads across Intel’s hardware portfolio.
The performance tests can be run across CPU, iGPU, and dGPU. The tests can be run across all architectures, in combination and individually, to provide flexibility in identifying performance needs.
You can run the script in two ways:
- Defined Pipelines: This allows you to pre-define the number of pipelines and gather the performance metrics.
- Stream Density: This allows you to pre-define the target frames per second (FPS) rate required per pipeline, and then measure metrics.
All the metrics are captured in log files and will be available at the end of the performance run.
Roadmap
Take a look at what will be available in the forthcoming releases.
Features:
- Camera Management Service
- Sensor Fusion Service
- Deployment Service
- Shopping Cart
- Autonomous Stores
Resources:
- Learning videos
To know about the future offerings of the Automated Self-Checkout Retail Reference Implementation, see the complete Roadmap on GitHub.