Maintenance and Improvement in a Machine Vision System

By July 7, 2022September 6th, 2022Quality Control Insights
Maintenance of Machine Vision

Machine vision (MV) is a popular tool of the manufacturing industry that is being leveraged in manufacturing and production for defect detection, product quality assurance, inventory management, and more. Coupled with AI, MV uses deep learning-based training algorithms to develop trainable systems that are easy to implement and can identify a wide spectrum of defects by performing an automated visual inspection on products that are manufactured in thousands every day. Since machine vision plays a critical role in the manufacturing process, any sub-optimal performance of the system can have significant adverse effects. In an automated process where the feedback from MV system acts as a controlling input to upstream processes, this can result in unacceptable variation in product quality; in a downstream process, a failed system could result in excessive false positives or false negatives. While the former may lead to a bottleneck and lower the throughput, the latter could result in faulty product reaching the market, and this could be even more disastrous. Regular maintenance activities carried out on a system ensure that all components of the system continue to perform as intended during the installation, and improvement of a system assures a better result than was originally estimated.

In the following discussion, we take a look at environmental factors and model drift, that can affect the Maintenance and Improvement in a Machine Vision System.

Environmental drift

Performance of an MV system may degrade due to ageing of components and effect of harsh operating conditions that usually exist in industrial settings. Some of the factors that contribute to this environmental drift are as follows:

Lighting

The primary goal of machine vision lighting is to maximize the contrast on the features of interest by minimizing the contrast elsewhere, so that it is possible to visualize the possible defects with enough light-dark contrast. An unintended change of illumination on the object under inspection, probably due to altered direction of the sun (seasonal variation), the overhead work area lights or lighting from another inspection station, can cause the image acquisition process to falter. If detected during maintenance, these ambient light effects can be minimized by use of daylight cut filters, high brightness pulsed LED, reduced sensor exposure time and stepping down the aperture. Similarly, performance degradation of lighting equipment with time can be remedied by periodic calibration of LED lighting.

Vibration & shock

Occasionally, systems are installed near heavy equipment, the operation of which may cause vibrations, and in the long term, misalignment of cameras, lenses and other associated hardware. This can lead to blurring of images, and large vibrations can even cause irreversible damage to sensitive hardware. On the same lines, the housing of the computing resources though designed to be robust enough to handle routine operations, their connections, cabling, mountings, etc. may get affected by vibrations and become loose over a period of time resulting in faulty output. Maintenance activity would involve routine checks to be performed on track grade cables and lockable connectors of the MV cameras, and checking all lockable screws that fix the focus lenses on metal mounts and provide protection from vibrations.

Dust/moisture/heat

MV systems are often installed in harsh environments where they are regularly exposed to heat, dust, aerosols, etc. The dust and mist may obscure or deflect the path of the light being reflected from the object resulting in poor quality of image acquisition. Dust, or moisture can adhere to the LED or lens surfaces, reducing the amount of light reaching the sensor. This may lead to a variance in the output. If a system fails to capture the image of an inspected object or does not reject a defective product off the line (false negative), the most likely reason is a failing sensor. Though modern cameras are available in housings which effectively protect against dust, water, and dirt, maintaining a schedule for inspection and cleaning can ensure that the setup remains free of any foreign matter.

Other Factors

Components and parts like belts, gears, and actuators in material handling equipment undergo natural wear and tear.  This can result in a deviation from the calibrated performance. For example, a change in speed of the belt may lead to a mismatch with the shutter speed, resulting in a bad image or no image at all. Stable power supply is an essential requirement for any electronic equipment. A supply beyond the permissible limits over a sustained period can affect their performance adversely. Improper earthing can also result in failure of electronic equipment, especially, in places prone to thunderstorm and lightning conditions. Electromagnetic interference could also lead to faulty signals from the image acquisition system. These erroneous signals may form the basis for the software to decide the pass-fail decision and lead to inaccurate results. The maintenance activity plan should include regular monitoring of these factors before they start affecting the Maintenance and Improvement in a Machine Vision System.

Model drift

In any machine vision based inspection system, the AI model is trained using historical image data available at the time of software development. The project is then commissioned and the real production pipeline begins to use this capability. Initially all may be well, and the model may behave as designed with this training data, but even an optimally trained model may not stay reliable forever. Changes in real world over a time period can cause the production data to deviate from the baseline data used to train the model. This may lead to the model losing its predictive capability and result in underperformance. This decay in model accuracy and performance over time is called model drift. Continuous improvement and regular optimization of the models by ‘retraining’ them is the only way to ensure life-long accuracy of results, post-deployment.

Types of model drift

There are two types of model drift – concept and data. Concept drift occurs when the statistical properties of the dependent variable (target) to be predicted change over time, and the patterns that the model learned are no longer valid. Data drift occurs when the properties of the independent variables change i.e. values of the features used to define them change. For instance, a changed color because of changes in consumer preferences, a new SKU with addition of new products, etc., can lead to data drift. This kind of variation causes the data to drift away from the model.

Retraining of model

In both cases, the model needs to be retrained by either re-annotating the existing images or by training with new set of annotated images. The redundant images can be eliminated from the training dataset, all the while being careful that it continues to handle the earlier data correctly. Retraining of the model can be either time based or continuous, and the choice of approach depends completely on the rate at which the data is changing.

Time based Retraining:

In a time-based approach, the model is retrained at regular intervals. This approach makes sense when frequency of retraining is in sync with the rate of change of data. Retraining a model too often, when there is no drift, is wasteful in terms of both computational resources as well as the efforts of the data science team, and may even render a model in a lesser relevant state than the previous version. Furthermore, retraining at large intervals may miss the point of retraining and leave the model with noisy data. Notwithstanding, it is prudent to retrain the model more often (than necessary), than too little.

Continuous Retraining:

This approach can be performance based or data driven. In the performance based retraining, a maximum threshold for error is predefined and an established set of performance metrics in the inferenced data is regularly monitored based on this threshold value, to assess when a dubious inference i.e. drift occurs. When there are enough such samples (past the pre-decided threshold) where drift has taken place, the model is triggered to be retrained. Data-driven retraining is triggered when a data drift occurs i.e. there is a change in input data and the model needs to be retrained on fresh data.

Retrain or Replace

Sometimes retraining the model is not the solution. When there is a significant drift in performance or a major external change in data, there might be a need to rethink the model. Rapid technology innovation can offer revolutionary improvements, most notably, to the DL model itself (the actual neural network).  In cases where a defect is unprecedented and of an uncommon kind, it could make sense to improve not just by retraining with newer data, but by replacing the older DL model with a newer and better one.

Maintenance and Improvement in a Machine Vision System are ongoing activities that help in reducing environmental and model drift in MV systems. Qualitas Technologies follows proactive maintenance practices with an aim to minimize failures and reduce costs. The industrial machine vision solution offered by Qualitas Technologies, has a vital ‘grading’ phase built-in into the system, where low-confidence decisions made by the inference engine are collected at regular intervals and routed to human operators for verification. The inputs received from operators are included in the next cycle of DL model training to address the low-confidence judgments discovered earlier. This obviates model drift and allows for an objective measurement of the MV system’s accuracy and precision. Regular maintenance and improvement ensure smooth functioning of the machine vision system without compromising its efficiency and effectiveness.

Also read: Advice for maintaining a machine vision system

Leave a Reply

Schedule A Demo
close slider