AI for Jewelry Recognition
Location:

usa

Collaboration:

2022

Industry:
Expertise:

Technologies:

huggingface icon
meta logo
yolo logo
fastapi logo

About the Client

Our client was an international IT company Yellow Media with which we cooperated under the white-label model. The company provides services to clients from various industries and markets, and develops unique, effective, and innovative software products for leading enterprises. Accordingly, they provided our Data Scientists as outsourced specialists for their client.

Business Context

The Amazinum company has outsourced its specialists to our partners. Thus, the end customer was a jewelry marketplace where users can upload photos of their jewelry for sale. Thus, our data scientists faced two tasks:

  • Classify jewelry according to certain characteristics according to the photo. The aim here was to help users with profile filling, so they spend less time describing item features.
  • Carrying out the identification of the model of the jewelry by its photograph, with the same aim to improve user experience on the website.

Challenges for Amazinum Data Scientists

Among the challenges our Data Scientists faced was a large amount of unorganized data. From the photo database, Amazinum specialists carried out the gathering and filtering of the data.

Amazinum Data Scientists in Action

Task 1

To perform the first task, namely the classification of jewelry according to certain characteristics, the client provided access to a database with products and images. Specialists needed to develop the following classification models – type of jewelry; type and color of the metal; type and shape of the stone.

Accordingly, first, it was necessary to prepare the data. For each type of model, data scientists had to collect as many diverse image samples as possible, which would be balanced across classes. After collecting the dataset, they started building a pipeline to train the model on the dataset. They trained several image classification architectures for neural networks – ResNet, MobileNet, YOLO, and Vision Transformer, hyper-tuned them, and performed evaluations until they stopped at the baseline architecture, which satisfied both the specialists and the client.

During prototyping, the scope has changed to include a process that would perform jewelry localization on the image. It finds jewelry and removes irrelevant details by cropping the image. Such flow improves the accuracy of further classification processes. Data science specialists decided to replace the jeweler-type classification model with a jeweler detection model that performs both localization and classification using the YOLO detection model.

After that, they deployed a solution as a WebAPI server with an endpoint that accepts images and returns results with a description of each characteristic of the received jewelry product.

Task 2

To implement the second part of the project, the client provided a list of jewelry brands and products that the solution has to detect. Data scientists performed the filtering and collection of the data.

After that, they used the Vision Transformer Model, which collected visual embeddings from images. The model transforms the image into a vector of characteristics that describes an image. Initially, the data scientists used a pre-trained Vision Transformer, later they conducted experiments on tuning a model on the jewelry dataset. Such a tuned model is oriented toward extracting visual characteristics specifically for jewelry. In this way, specialists made a solution using the Image Retrieval System. They formed a database of the brands and products that needed to be recognized and created visual embeddings for each image in the database. And deployed it as an endpoint to existing AI WebAPI, which, upon receiving an image, vectorizes it and searches for the most similar images in the database.

Review

YellowMedia logo

AI Development for IT Company

Their easy solutions to complex problems, pleasant communication, and the team members’ responsibility are impressive.

Amazinum impresses the client with their ability to provide easy solutions to complex issues. Their responsible team manages the project efficiently, ensuring everything is at a high level. They also facilitate pleasant communication through online meetings.

Content:

Vitaliy Fedorovych

CEO, Data Scientist at Amazinum

Vitaliy Fedorovych contact us photo

Hello there!

Amazinum Team assists you through all data science development processes:
from data collection to valuable insights generation.
Get in touch with our CEO and Data Scientist to figure out the next move together

Contact Us

Click or drag a file to this area to upload.

This will close in 0 seconds

Book a FREE consultation today and get

a 10% discount on your next project

You will receive:

  • A qualified specialist with experience in your field
  • High-quality and fast solution for your business
  • Convenient models of cooperation from POC to a full-fledged project

Leave your email and we will contact you

No limits to solutions with Amazinum