Beyond Monitoring: The Power of AI and ML in Construction Site Surveillance




2021 – 2022


Numpy icon

About the Client

Our client is a company that provides services of remote monitoring and innovative use of cameras. The company’s service covers a wide range of equipment that works with images in various industries, including construction, mining, environmental monitoring, and scientific research.

Business Context

Develop an algorithm for detecting and tracking changes at the construction site.The Data Scientists of Amazinum joined our client’s team to introduce a Computer Vision solution to one of the client’s services. The solutions were to improve and develop algorithms that would help monitor safety on construction sites based on OSHA regulations. Thus, tasks among our data scientists were:

  • Improve the accuracy of the existing model;
  • Create an algorithm that could connect equipment classes to subjects;
  • Increase the number of detected classes in the existing model;
  • Create a model for detecting heavy machinery;

Amazinumn Solution in Client’s Project

Improving the Accuracy Of The Client’s Model

At the beginning of the work, the client had a model that detected three construction-related classes – worker, hard hat, and vest. The existing model produced good results, but our task was to improve its accuracy even more to get reliable monitoring of compliance with OSHA regulations.


Among the challenges faced by the company of our Data Scientists was the problem of a large number of False positives. That is, the model detected a worker where none was present.

number of tracked keywords graph

From Tasks to Actions:

Our first step was to increase the dataset itself. We achieved this by using two methods. In the first method, we simply added new images. As for another method, our Data Scientists have integrated the technique of image augmentation. This means the image had to be changed to look different from its original appearance. For example, adding black and white versions of the image or blurring it, etc. This allowed the model to understand more cases and detect images more correctly.

The next step was to change the architecture of the model. The existing version of the model used the YOLOv3 architecture. Our specialists replaced it with YOLOv5. This object detection model produced better accuracy and was more relevant, complex, and optimized.

To deal with false positives, our data scientists decided to add a background image that would not have target classes. In this way, we achieved a result in which the machine did not detect inappropriate subjects as workers.

Creating Complex Classes With Attributes

The next task within the client’s project was to create complex classes that could describe worker better as entities. Amazinum Data scientists created an algorithm using the NumPy library, which attached classes of equipment to subjects (workers), thus allowing them to perform various operations on them.

Working With Classes

One of the tasks of our client was to increase the number of detected classes by model. We have added personal equipment classes (goggles, protective masks, protective gloves) and light auxiliary equipment (scissor lift, lifting platforms, ladder) to the model. Based on classes of light assistive equipment, we created an algorithm that could determine whether a worker interacts with this equipment. For example, if workers are at height. Data Scientists at Amazinum used NumPy algorithms to achieve the desired results.

model accuracy graph

Heavy Machinery Model

In addition, our data specialists created a model of heavy machinery. In this way, the model could distinguish not only workers on the construction site but also machine-type equipment like a static crane, roller, bulldozer, excavator, truck, loader, concrete mixer, etc.

Other Tasks

Amazinum data scientists developed an algorithm for detecting and tracking changes in the state of the site structures using a video stream.


The use of innovative cameras equipped with Computer Vision algorithms on construction sites has shown a positive result. Machine learning algorithms have made it possible to monitor construction sites and prevent potential hazards on sites. Among the advantages that can be reaped by the end user, using our client’s technologies:

  • Reduction of construction accidents, such as injuries and deaths;
  • Control of the construction site;
  • Identification of potential threats and dangers;
  • Avoiding carelessness of workers regarding PPE.

Outsourced workers at Amazinum helped the client company obtain valuable data that can be used to improve safety and identify patterns or trends in workplace accidents. Thanks to this, you can build a strategy for improving your service.

Thanks to the implementation of computer vision, companies or entrepreneurs providing services in the field of construction were able to increase the trust of their customers in them and ensure safety at the workplace. All this, in conclusion, helped to attract new customers and increase their income.

Computer vision technology is confidently gaining its place in numerous industries. Who knows in which industry it will cause a real revolution, but it will definitely help you improve your business.

Stories about your project and Outsourced data science company Amazinum will help you bring any of your ideas to life.


Vitaliy Fedorovych

CEO, Data Scientist at Amazinum

Vitaliy Fedorovych contact us photo

Hello there!

Amazinum Team assists you through all data science development processes:
from data collection to valuable insights generation.
Get in touch with our CEO and Data Scientist to figure out the next move together

Contact Us

Click or drag a file to this area to upload.

This will close in 0 seconds

Book a FREE consultation today and get a 10% discount on your next project

You will receive:

  • A qualified specialist with experience in your field
  • High-quality and fast solution for your business
  • Convenient models of cooperation from POC to a full-fledged project

Leave your email and we will contact you

No limits to solutions with Amazinum