Machine Learning in Retail: Grocery Product Recognition in Hypermarkets
Location:
Sweden
Collaboration:
2019 – present
Industry:
Technologies:
About the Client
Our client provides quality solutions to improve retail by optimizing business and customer interaction. The software provided by our client creates a unique brand experience, integrating solutions not only in digital commerce but also in the physical store environment.
Tasks for Amazinum Team
Our client asked us to develop the next feature for their platform – this is grocery product detection and recognition using Python, machine learning algorithms, Artificial Intelligence, and Image Recognition. The Amazinum team had to develop a solution that would fit into the software our customers were developing. Its essence was to improve self-service checkouts by automating the system of selecting goods on scales. Using the store’s cameras, the 3 or 5 most likely items would be offered on the screen. This allows supermarket visitors to speed up the search among articles.
The task’s difficulty was that the GCP service data collection platform did not give us the images we needed. Therefore, we had to collect data ourselves and train the model, improving it to the point where it could recognize goods in every aspect, and even in packages.
Our work consisted of:
- data collection;
- model evaluations;
- processing and marking;
- deployment of the model;
- deploy the forecast of the object detection model;
- data and model training.
Working with the Amazinum Data Scientists Team
From the very beginning, the company has chosen GCP cloud storage. Experiments were conducted using its Cloud Vision API, which allows you to use models trained by Google, and AutoML Vision, which creates a unique ML model for object detection and Image Classification.
During testing, we analyzed that the Cloud Vision API has not fully satisfied the needs, and AutoML gives a low performance. Thus, it was decided to stop the two streams of the AutoML Model and Keras Model at the same time.
Since the images provided by Google were not enough, because they did not reflect the real picture of the products that we can find in the supermarket window. So we collected such data ourselves, photographing the products from different angles. This made it possible to balance all classes of products and achieve a more accurate recognition of objects. Using the Vertex AI model, we were able to train AutoML models.
Therefore, the model needed further training.
The following model deployment has been used and tested:
- Local machine prediction of Keras model;
- Web browser prediction via web app by Keras model converted to tensorflow.js;
- Prediction by Keras model deployed on GCP VM via a Web App using;
- Local Machine Prediction of AutoML model;
- Web browser prediction via web app by AutoML model converted to tensorflow.js;
- Prediction by AutoML model deployed on GCP VM via Web App using;
- Prediction by AutoML model deployed on GCP VM via Web API;
- Prediction by AutoML model deployed on GCP Kubernetes via Web API.
So, we have accurately classified objects according to their ranks using retail Image Recognition technology.
The solution worked both for cash registers with staff and for self-service cash registers.
End-user software accurately labeled objects regardless of their condition, angle, position, or wrapping.
You can see several graphs that our clients have received for conducting their analytics:
- The graph of cross purchases for three categories of products. That is, the client received information about the number of receipts in which these categories of goods were purchased together
- The graph of unique visitors by season
- A graph that visualized the store’s revenue trends by week.
Outcome
Image Recognition for retail execution made it possible to provide a high-quality and functional solution in which the final buyer in the store saves time, limits himself from the stress of order fulfillment, and avoids inconvenience. By implementing Image Recognition technologies, you become a leader among competitors, because in the near future, such a service will operate in an even wider range.
By involving Artificial Intelligence in the field of Retail, end-users received the following benefits:
- more involvement of customers in the store;
- retention of regular customers;
- increasing the guarantee of security and protection against fraud;
- facilitating the work of store staff;
- saving the client’s time and resources, as well as increasing the potential for their service.
Write to us and we can tell you how you can improve your retail business now to be a leader in the future.
