NEUSS, GERMANY – Artificial Intelligence (AI) is renowned for image recognition and classification skills, suggesting a great fit with the objectives of AOI. How is the technology being applied and how is performance improving?

AI in AOI

Equipment suppliers are racing to integrate AI into their solutions. Equally, manufacturers are racing to integrate the technology into their processes. As surface-mount electronics assembly is already highly automated, AI brings the opportunity to extend machine-like speed and repeatability further into complex tasks that require learning, judgement, and adaptability. Automatic optical inspection (AOI) is a prime example.

AI’s skills in image classification, already widely used in applications such as disease diagnosis, automated driving, content moderation, and others, are a great fit with industrial quality control, and AOI in particular. Conventional approaches to AOI are heavily dependent upon the judgement of experts when setting up the system, introducing new products, and subsequently while production is ongoing to inspect images of suspected defect areas.

AI first became available in commercial AOI software to help with setting up and running the equipment. Automatic component library matching, which uses deep learning to identify component types from images and so enable the optimum library to be selected automatically. In addition, AI is used to assist 3D measurement of components to generate data for parts that are not found in any existing library.

Secondary Judgement

While AI has simplified and accelerated library management, the technology is now ready to offer greater value to manufacturers by applying its learning and classification skills to improve inspection accuracy on the production line. Here, relying on human judgement to classify defects as real defects, false positives, or false negatives can be time consuming and introduces variability into the manufacturing process. While some defects are easy to classify, such as missing or extremely poorly aligned components, or severe soldering problems, others are more difficult to see or to identify clearly as being below acceptable standards.

When judgement is left to human experts, individual operators can apply different criteria based on their experience level and opinions. Inspection can be slow as judgements take time. Some defects may be overlooked while, on the other hand, excessively strict assessments can produce false positives that demand unnecessary rework. Introducing AI to assist these secondary judgements offers the opportunity to relieve dependence on experts and eliminate inaccuracies thereby delivering a boost to productivity.

The AOI system shares images of detected defect areas with human operators as well as with the AI Judgement software application that hosts machine-learning models. The human experts assess the nature of the defects and their judgements are fed back to the AI software. By repeatedly adjusting as the experts judgements are logged, the model quickly acquires the experts’ best judgement skills and eliminate human errors. When trained, the model makes judgments allowing the operators to work confidently and more quickly as well as maintaining a consistently high level of accuracy. Thus, the operators can match the performance of skilled inspectors.

AI-assisted secondary judgement can enhance repeatability to prevent defective units escaping from the factory and can quickly identify false positives to prevent non-defective assemblies being sent for unnecessary rework. This stabilises the factory’s quality performance and increases productivity.

Confidence Index

Yamaha’s AI Judgement software for AOI provides comprehensive information for the operator that explains its own Good/NoGood decisions, including images, tables, and a confidence indicator (figure 2). In the case of soldering defects such as bridges or contamination, this index is shown as a graphical heatmap and calculated anomaly index. Judgements of other defects such as character recognition are expressed with a matching ratio. The software also reports its own performance with calculations of the anomaly detection rate and over-detection suppression. Through repeated data gathering and analysis including noise reduction, and with tools to build and optimise custom AI models, the AI Judgement software lets users control the machine-learning lifecycle and ultimately perform quality control without professional assistance.

From Assistance to Automation

Moving forward from AI-assisted secondary judgement by human operators, the next step is for fully autonomous AOI that performs consistently up to the standard of the best human experts in the company. Yamaha’s AI Judgement software is ready to connect seamlessly with the remote repair station and can share AI- judgement results directly with inline AOI systems, to continuously improve inspection accuracy. Leveraging AI to automate secondary judgement lets inline AOI systems operate continuously without intervention, at a high rate, with minimal false negatives or false positives to prevent defects escaping and avoid unnecessary rework.

Conclusion

Artificial intelligence can enhance multiple aspects of AOI, including accelerating library creation and automatically generating missing component data. With its image classification capabilities, AI is ready for deployment in the production line to assist and eventually automate secondary judgement. Historically, this task has demanded the attention of highly trained inspectors. AI now lets operators quickly reach a comparable level of proficiency. The roadmap is headed towards end-to- end automation from assembly to AOI, with learning, to continue improving quality assurance and raising productivity.