How and When to Apply Deep Learning in Machine Vision

AI is seen as computing-intensive, expensive and slow, so is it ready?

(Image courtesy of Cognex.)

(Image courtesy of Cognex.)

In industrial automation, vision is one technology that can make your robot or machine much more flexible and able to work with disordered or unexpected information. For example, a pick and place robot without vision or other sensing will follow a path from a pick location—typically a fixture—to a place location. With vision, robots and other automated systems can do a whole host of more advanced tasks. In the vision business, these are typically grouped into four categories, abbreviated as GIGI:

  • Guidance
  • Inspection
  • Gauging
  • Identification

An example of a robot using vision for guidance might be an autonomous mobile robot, or a robot using vision to locate and pick a part. For inspection, a vision system may be trained to spot defects in surface finish or burrs on a part. Vision inspection is commonly done in the food and agricultural industry—sorting fruits by color or removing burnt food products from a conveyor belt, for example. Vision systems use calibrated cameras, which can perform accurate measurement and gauging for quality purposes or during assembly. Lastly, identification typically refers to reading or scanning, such as optical character recognition or code scanning. Identification can also refer to identification of features such as edges.

So, why use complex artificial intelligence (AI) in machine vision? There are thousands of robust and reliable vision solutions working today, using rules-based programming to guide, inspect, gauge and identify all manner of parts in all manner of industries. According to Brian Benoit, senior product manager at Cognex, deep learning in machine vision presents tremendous opportunities for machine vision applications, but there’s still an important place for rules-based systems.

Rules-Based vs. Deep Learning in Machine Vision Programming

(Image courtesy of Cognex.)

(Image courtesy of Cognex.)

Compared to machine learning (ML) systems, rules-based systems have a key advantage today: they’re faster. To design a rules-based machine vision system, programmers must choose from a toolbox of things rules-based vision can do reliably and accurately, and chain them together to create the desired result. These tools include counting pixels, defining an edge based on contrast in an image, using a histogram, and locating a pattern. These tools work fast and reliably for guidance, inspection and gauging, but they often struggle when faced with nonoptimal conditions, especially lighting. A rules-based system is greatly challenged by low contrast and variable light. In addition, a pattern-matching or optical character recognition (OCR) program may struggle with: a label presented at an angle to the camera, which changes the aspect ratio of the image; a wrinkled label that partially obscures the characters; or even a glare on part of the label. According to Benoit, this is where deep learning shines.

Challenges for Cognex’s rules-based (not deep learning) OCR code reading solution. (Image courtesy of Cognex.)

Challenges for Cognex’s rules-based (not deep learning) OCR code reading solution. (Image courtesy of Cognex.)

So, what is deep learning technology, anyway? In his presentation at A3 Vision Week, Benoit used the analogy of teaching a child what a house is.

“You wouldn’t say to a child, ‘Okay, a house is a structure with four walls, a peaked roof, a door on the front, and windows,’” said Benoit. “What you would do is show the child examples of houses, and the child would make the connections to categorize a ‘house’ and make those connections naturally. That way, even if the examples were all ranch-style homes, for example, the child could see a flat-roofed house from the American Southwest or a bungalow and still identify it as part of the ‘house’ category.”

While rules-based programming defines specific things to identify in an image, deep learning uses training to allow the system to make connections that will give it flexibility to find a pattern or identify a feature more flexibly. That’s what makes deep learning so useful in the “identify” category of machine vision applications, especially in reading and identifying human-readable text or codes, for example.

However, both Benoit and other experts are clear that deep learning or other AI tools will not replace programmed systems. “Deep learning tools such as neural networks will complement other machine vision techniques. For example, such a neural network could judge the probability that a data matrix code exists within an image,” said Arnaud Lina, director of research and innovation at Matrox Imaging. “But to decode it, traditional barcode algorithms would be used.”

Of course, apart from rules-based machine vision, the other common technology for vision is a highly complex but familiar one: the human eye. How do deep learning systems measure up to manual inspection?

Deep Learning vs. Manual Inspection

(Image courtesy of Cognex.)

(Image courtesy of Cognex.)

The human brain is obviously much more flexible and powerful than any AI system today. We can make advanced inferences from context, solve complex problems, and use our experience interacting with the world to inform our analysis of visual information. That’s why human manual inspection is often used in agriculture, medicine, challenging reading, and challenging cosmetic defects, for example. But human vision also has several drawbacks in industrial applications.

Human inspection is slower than machine vision, and human workers have limited attention spans. Human inspectors also mean higher labor costs than automated systems, and human error or mistakes often mean higher recall costs.

While today’s machine learning systems still can’t compete with human intuition or complex problem-solving, they are more consistent, more reliable and faster. In addition, computer vision can capture and log all the vision data, allowing for analysis, traceability and process improvement.

Currently, deep learning machine vision systems are in use in the automotive industry, for example, reading VINs printed on metal, embossed on molded parts, or dot peened on cast parts. These methods create text that would be hard for a rules-based OCR system to read, but which are identified by deep learning systems fast and reliably.

Is Deep Learning Ready for Prime Time?

According to Benoit, not only is deep learning technology ready for use in manufacturing applications, but it’s also becoming a competitive advantage. “Deep learning is becoming a strategic imperative,” said Benoit. “Over the past 5-6 years, we have seen situations where they’re adding inspections where they didn’t before, and some customers are winning because of it. Deep learning machine vision is a technology that can lower costs and increase yield.”

One important factor that makes deep learning-based machine vision a competitive differentiator is that ability to capture and use the data for process improvement and drive maintenance. When your competitors have access to this data and you don’t, there is a risk that you’ll fall behind, said Benoit.

ROI for Deep Learning-Based Machine Vision

(Image courtesy of Cognex.)

(Image courtesy of Cognex.)

While expensive, computing-intensive deep learning systems aren’t necessarily better than rules-based programming in all vision applications, they do provide direct cost savings compared to manual inspection. A properly calibrated and trained system can provide consistent results across units, lines, shifts and even factories. This can help deliver production improvements such as higher yield and real-time process control feedback.

According to Cognex, documented outcomes of deep learning implementations include:

  • Inspection reports for end customers
  • Traceability for future recalls
  • Data recording for training improvements
  • Opportunity for process analytics

“Customers ask us, ‘Is this technology ready or am I embarking on a science experiment?’ and Cognex says that we have seen a number of customers get to an ROI very quickly with these deep learning solutions,” said Benoit.