Smart Eyes: The Integration of AI and ML in Surveillance Cameras for Enhanced Security

Location:

usa

Collaboration:

2019 – present

Technologies:

Tensorflow
Numpy icon
Pandas icon
PyTorch icon

About the Client

Our client is a technological company in the medical field that implements virtual and augmented reality solutions for medical purposes. This allows them to revolutionize the field and provide practical and precise patient treatment methods. The clinic offers its clients virtual reality technologies for the treatment of chronic pain, anxiety, and post-traumatic stress disorders; rehabilitation classes for patients with physical disabilities, etc.

Business Context

Amaizunu’s Outstaff Data Scientist team joined our client’s team to implement the Computer Vision solution. The client provided us with a service based on technologies:

  • Face detection;
  • Face recognition;
  • Person detection;
  • Person re-identification;
  • Object tracking.

Amazinum Data Scientists in Action

Person Re-Identification

The system associates images of the same person taken from different cameras or from the same camera on different occasions in order to not alert people that were marked as trusted.

The main challenge was to find a solution that would provide similar embeddings to the same person regardless of the image source. The system uses a pre-trained OSNet model, finetuned on a custom-labeled data set.

Based on the data provided to us by the client, we researched algorithms that would satisfy our needs. From the available trained models, we chose the one that suited the best and implemented it in the application itself. After a suitable model was selected, Data Scientists at Amazinum evaluated it. Based on the collected results, a team of specialists retrained her according to the existing Torchreid project.

re-identification graph

Challenge

Among the difficulties encountered by the Data Scientists Amazinum was the problem with person re-identification in night vision

Solution

Since the client gave our specialists access to the installed surveillance cameras, we used their data to train the model. At the same time using different approaches. According to one of the methods, we trained the model on the night picture immediately. With the other, they trained her on the day picture, which was then trained on the night picture.

Evaluation of the models made it possible to choose the best approach for its training and implement it in the service.

Face Detection

Our client’s task was to teach the model to work with faces, that is, to recognize or identify them. This would help create a personal experience of a specific surveillance camera and would increase the security of objects. Facial recognition technology can compare a detected face to a pre-existing database of faces, allowing surveillance cameras to identify known individuals and flag potential security threats or security issues.

Solution

In this task, our data scientists followed similar steps as in the previous one.

  • Search for a model
  • Evaluation of models and selection of the best one
  • Implementation of the model at production companies

To work with faces, we chose three models.

RetinaFace – used to search for absolutely all faces in the frame, regardless of quality or distance.

Another model is used to choose the best quality of faces found by RetinaFace. Thus, they can be used in face recognition.

To evaluate the model, the data scientists of Amazinum used certain AP metrics, map@.5-.95, Precision, Recall, F1 score, and confusion matrix.

bid-ask metrics
America, europe, asia metrics

Face Recognition

The face recognition model was chosen like all the previous ones. That is the selection of the model, its testing, training, and implementation.  To train the model, we used the CASIA-WebFace dataset. It has about 453,453 images for 10,575 IDs after face detection.

Challenge

Among the challenges faced by the Data Scientist of Amazinum was that the face detector we used did not provide a sufficient level of model training.

Solution

To solve this problem, we tested the use of other detectors. One of the facial landmark detectors that have proven to perform well in these conditions is the multi-task CNN.

After that, it was possible to proceed to the face recognition model. It consists of several steps:

  • Face recognition. An algorithm that analyzes a photo or video determines the areas that contain faces.
  • Identification of signs. A feature extraction algorithm is used to identify personal facial features that are unique to each person.
  • Comparison of faces. At this stage, the model compares the face with the existing database, guided by certain metrics.
  • Face ID. If the detected face matches a face in an existing database, the model identifies the person

To evaluate the model, the data scientists of Amazinum used certain AP metrics, map@.5-.95, Precision, Recall, F1 score, and confusion matrix.

Person detection

The person detection model is actually the main model after which further work was to be carried out.

To accomplish this task, Amazinum’s Data Scientists researched available solutions and looked for models. After that, the model was evaluated and the best one was chosen. The model was selected based on certain parameters.

  • The performance of the model, that is, how well it searches for a person;
  • the architecture of the model – how many resources it needs, and its speed.

Among the models we selected were YoloV4, YoloV5, and YoloR. However, we continued our work with YoloR, as it was the most optimal for our task.

After that, our specialists began to retrain the model, based on data from surveillance cameras. In this way, we received videos from different locations from different clients, cut them into frames, labeled them, and formed a training data set.

Challenge

Among the challenges faced by the Amazinum Data Scientists team was improving the performance of the pre-trained model.

Solution

To cope with this task, we conducted experiments with model hyperparameters. Cleaning and correcting data was carried out, as well as adding external data to the data set, i.e. photos or videos taken from the Internet.

For this, we used technologies and libraries NumPy, Pandas, TensorFlow, PyTorch, ONNX.

Object tracking

Data Scientists of Amazinum needed to develop a model that could recognize objects. For this, we looked for models, evaluated them, and chose the one that would satisfy all our needs. For the object detector, our specialists used the new version of YoloV7, with the DeepSort sorting algorithm. For comparison already detected people TorchReid.

Since the metrics were lower than we expected, the Data Scientists team decided to create a custom Object tracking logic. The developed technology better suited the client’s requirements regarding Object tracking.

number of tracked keywords metrics

Outcome

The use of computer vision technologies in surveillance cameras helped to make the protection of areas more reliable. End users will be able to get:

  • Improved security. Because facial recognition and object detection can help identify potential security threats in real-time, allowing security personnel to take quick action to prevent crime and ensure public safety.
  • Greater efficiency and effectiveness. Machine learning algorithms can automate the monitoring and analysis of large volumes of data.
  • Savings of funds. You can optimize security costs by automating many processes. At the same time, computer vision technologies will help you increase efficiency.
  • Personalized experience. Computer vision technologies, such as face recognition or face detectors, will help you create supervision that will be customized for you.
  • Statistical information in real-time. You can get up-to-date information about people and their behavior and other key indicators.

For our client, as the owner of the surveillance camera software and service, the implementation of machine learning technologies provided:

  • Increasing the trust of his clients;
  • Optimizing processes and increasing efficiency;
  • Increasing the number of customers
  • Market leadership among competitors;
  • Provision of more effective services
Content:

Vitaliy Fedorovych

CEO, Data Scientist at Amazinum

Vitaliy Fedorovych contact us photo

Hello there!

Amazinum Team assists you through all data science development processes:
from data collection to valuable insights generation.
Get in touch with our CEO and Data Scientist to figure out the next move together

Contact Us

Click or drag a file to this area to upload.

This will close in 0 seconds