Robotics technology is found everywhere from industrial processing to vacuum cleaners, pool cleaners and remote-controlled drones. Many advanced robotics are used in many essential commercial applications like food processing. Industrial control and packaging often in the form of a programmable and automated arm.
Robotic Vision and Machine Vision Systems
One such advanced robotics technique used in modern industry is robotic vision or machine vision systems. Machine vision systems is the technology and methods that are used to provide imaging-based automatic analysis and inspection for process control and robot guidance.
Vision systems are one of the primary tools to be considered by manufacturers looking to automate and enhance their production process. Machine vision systems can be regarded as advanced robots with eyes that identify, inspect and communicate critical information on eliminating costly errors, improve productivity and enhance customer satisfaction through delivery consistency of high quality products. Primarily used for online inspection, machine vision systems can do mundane, complex and repetitive tasks at a high speed with accuracy and consistency. Any deviations and errors in the manufacturing process are quickly detected and relayed to the proper personnel so that controlled modifications can be made to reduce waste or scraps in the manufacturing process and minimise costly downtime. Machine vision can also be applied for non-inspection tasks like guiding robots to pick parts, welding seams, dispensing liquids and placing components for assembly.
Machine vision systems come in different designs, sizes and shapes to suit any application, but possess the same core elements. Every vision system has one or more sensors that take images or capture pictures for analysis. It includes inspection software and processing elements that execute user-defined programs that defines the inspection. It is essential to know that there are significant differences between all vision systems available on the market for any given application. It is also important to choose the optimal lighting and optics for the application because failing this may result in false rejects or even worse, false positives. If this happens product quality will suffer and customer satisfaction will go down.
A Short History Of Machine Vision Systems
Machine vision is a branch of computer science and robotics that has grown in the past 20 years and has become an important feature of the manufacturing industry. At present, machine vision systems provide great flexibility and more automation options to the manufacturer which in turn helps in product sorting, finding defects and completing a number of tasks faster and more efficiently.
How did this technology start?
The 1950s. 2 dimensional imaging for statistical pattern was created. James J. Gibson introduced optical flow and based on his theory, mathematical models for optical flow computation on a pixel-by-pixel basis were created.
The 1960s. Larry Roberts started studying 3D machine vision. Roberts wrote his PhD thesis at MIT about the possibility of extracting 3D geometric information from any 2 dimensional view. This study lead to further research in MITs AI laboratory and other research institutions that looked at computer vision in the context of simple objects and blocks.
The 1970s. MITs AI laboratory opens a machine vision course. The course tackled real world objects and low-level vision tasks. In 1978, David Marr made a breakthrough when he created a bottom up approach in understanding scenes through computer vision. It started with a 2D sketch that was built upon by the computer to get a final 3D image.
The 1980s. Machine vision systems start to go beyond the world of research with new concepts and theories emerging. OCR or optical character recognition systems were initially used in different industry applications to read and verify letters, numbers and symbols. Smart cameras were developed in the late 1980s. These smart cameras lead to more widespread use and more industrial applications.
The 1990s. Machine vision systems are becoming more common in manufacturing and leading to the creation of the machine vision industry. Hundreds of companies began selling machine vision systems. LEDs for vision systems were developed and sensor advancements were made in enhancing sensor functions and control architecture. With these enhancements and developments, the cost of machine vision systems also began to go down.
At present, machine vision systems continue to improve and move forward. 3D machine vision systems that scan products along the conveyor belt at high speeds are becoming more affordable in many different industries. Different systems that do everything from slope measurement to thermal imaging are readily available. The market for machine vision systems is growing continuously because it is driven by:
- Improvements in user-friendly interfaces
- The shift from analog to digital interfaces
- The need of machine vision in other applications
- The rise and popularity of smart cameras
- The gradual decline in machine vision prices
- Readily-connectible equipment
- Increasing measurement speed and throughput
- Significant return of investment for manufacturers
Many present day machine vision systems researchers are promoting a top-down approach because of the difficulties Marr’s framework exhibits. Purposive vision is a new theory that is exploring the idea of not needing a complete 3D object model in order to achieve the many machine vision goals.
Another area where machine vision systems are advancing is with EEG sensors in gesture-based interfaces. Gesture-based interfaces allow users to control computers and machinery with gestures as compared to using keyboards and other input devices.
How A Machine Vision System Works
The basic components of a machine vision system consist of:
- One or more digital or analog camera(s) with optical lenses. Cameras can produce either colored or black and white images.
- A frame grabber (the camera interface used to digitise the image).
- An embedded processor or a PC processor. In most cases, the elements mentioned above are built-in to one device called the smart camera.
- The device I/O or communications link used in reporting system results.
- Lens for focusing and taking close-ups.
- Specialised light sources like halogen lamps, fluorescent lamp or LEDs.
- An image processing algorithm or the software used in imaging and detection of features in common images.
- A sync-sensor used in detecting objects which provides the signal for the sampling and processing of images.
- The required regulations to reject or remove defective products.
The whole process begins by illuminating the product or the object with lighting. The optics then couples the image to the camera sensor. The camera then converts the captured image from the optical to analog or digital form to be processed by the computer. The computer then process the captured image.
There are many known variations of vision systems in the market today but in general, there are 2 categories:
- Machine vision with an embedded single sensor or with a single smart camera.
- Machine vision systems with multiple camera systems.
The use and application will depend on the number of sensors needed, but also on other factors like cost, performance and the environment where the system need to be implemented. Smart cameras are designed to better tolerate harsh and rigid operating environments compared to multi-camera systems. On the other hand, multi camera systems are more affordable and deliver a higher performance for more complex applications.
Another way of knowing the differences between the two types is their processing requirements. For many industrial applications, it is more efficient to have more independent points of inspection along the whole assembly line. Smart cameras are the best choice for such applications because these cameras are self-contained and are easily programmable to perform specific tasks. It can also be modified if needed without affecting the other inspections within the assembly line. Doing this will distribute processing among different cameras. Similarly, other parts of the production line may be more suited for a centralised processing approach. It’s not uncommon for some assembly lines to have up to 32 sensors, so in cases like this, a multi-camera system is well-suited because it is more affordable and the operator will not have a hard time managing them.
Vision Systems Applications
Machine vision systems are efficient enough to do 100% online inspection resulting in enhanced product quality, higher product yields and lower production costs. Consistent product quality and appearance will result in high and consistent customer satisfaction and a large market share.
Here are a few areas where machine vision systems are used:
1. Hazard Analysis Critical Control Point
The most common application of machine vision systems in production and packaging is the monitoring of product quality on the manufacturing or assembly line. Quality assurance is an essential part in the success of a commercial product. One way of improving product quality is the implementation of a “Hazard Analysis Critical Control Point” or HACCP that aims to prevent hazards in food manufacturing. There are 7 steps in creating this plan:
- Conduct a hazard analysis
- Identify the critical control points
- Establish the critical limits of each of the critical control points
- Create the critical control point monitoring requirements
- Establish all the required corrective actions
- Establish the record keeping steps and procedures
- Establish the procedures for ensuring the HACCP system is working as planned
You can monitor the established critical control points by taking advantage of machine vision systems. If machine vision is used, cameras are placed at all the predetermined critical control points. The system can also record the number and percentage of flaws at a specific critical control point that can satisfy a portion of the record keeping procedures. A machine vision system can ensure the quality and the integrity of the package or product when HACCP is properly implemented. But you have to remember that every industry, packaging and product line will have their own critical points.
Some of the more common machine vision systems major applications include inspection, verification, recognition, identification, and location analysis and data collection.
Gauging and flaw detection are the 2 main types of inspection. Gauging inspection collects quantitative correlations to design data as a way of assuring the objects are within the accepted design specifications. Flaw detection or cosmetic inspection checks for unwanted defects.
Verification simply assures that the operation is running as designed and that there are no flaws in the line of production and fabrication.
Recognition uses descriptors known to the machine that are associated with the object. This allows the system to identify that specific object on the line. Identification is almost the same as recognition, but the difference is that identification uses symbols found directly on the object to determine the category it belongs to. Identification and recognition are truly useful in a fast paced packaging environment.
5. Location Analysis
Location analysis has 2 types – position and guidance. Position location analysis simply assesses the position of an object relative to the line. Guidance location analysis assesses the position of any object, but it uses this information to provide feedback in directing activity on the line.
6. Data Collection
Data collection is a machine vision application that is commonly used in quality assurance. Data collection classifies each object that it monitors and logs the data to the system. The system then alerts the user if the distribution is out of the specified range. If the technology is capable, it can also take action to fix the problem.
Common industry applications of machine vision systems include:
- Color matching
- Die attach bond inspection
- Sub-assembly verification
- Location and alignment for pick and place
- Ball grid array inspection
- Measure solder paste levels
- Robotic guidance
- Wafer positioning
- Test tube cap and color inspection
- Package integrity
- Vial reading verification
- Pattern recognition
- Data matrix and 2D barcode reading
- Pixel counting
- Optical character recognition
The Bright Future Of Machine Vision Systems
During the last 15 years, machine vision systems technology has matured substantially and has become an important and indispensable tool for automating manufacturing. Today, machine vision applications are present in many industries including pharmaceutical, electronics, medical, automotive and general consumer goods. The continuous improvement in cost, performance, ease of use and the algorithmic robustness has encouraged vision systems’ use in general manufacturing automation. Further advances in these areas will shape the future of machine vision systems.
The future of machine vision systems will be advanced through flexibility, smart technology and advances in hardware and software. With the current economic climate, research and development has been decreasing in many fields including technology, but advances in machine vision will see even greater technological growth as compared to the average industrial hardware.
As manufacturers begin to install flexible lines that can run multiple products, machine vision will also need to advance to keep up. Advancements in LEDs will also directly affect machine vision systems because these LEDs are the most used lighting sources for the production line.
2. Smart Technology
Advancements in smart technology will include higher quality and more affordable smart cameras. Smart cameras as mentioned are standalone vision systems. The camera itself contains a processor which eliminates the need for a computer and other processing systems. These cameras usually contain linear or matrix image sensors, image memory, image digitisation circuits, program and data memory, communication interface, I/O lines, lenses, LED and a real-time OS. Smart cameras have increased in quality and decreased in price through the years thanks to the expansion of their uses.
3. Hardware and Software Improvements
Advancements in computer software will enhance ease of use by simplifying setups and usage. Different people with different skill levels will be working on such systems so that both the process engineer and the line operator can easily use the system. At present, extensive training is in place when implementing a new machine vision system. Now, operating systems and other supporting software that can be updated to increase usability which may decrease the need for costly and time-consuming training.
Hardware speed has also increased and is continually improving. New vision processing hardware supports image acquisition from non-standard cameras. Experts also believe that faster hardware also results in a faster system. If bottlenecks happen in the production line, faster hardware can eliminate these bottlenecks by allowing the system to process the information faster, thus it can scan more items in a second. Increased speed in hardware can also be applied in the potential image resolution. If higher resolutions are implemented, the system will be able to pick up even the smallest details and flaws. This in turn will make the whole quality assurance systems more precise and accurate.
Trends and Predictions for Machine Vision Systems
1. From CCD to CMOS
It is so far clear that CMOS (complementary metal oxide semiconductor) is the image sensor technology of today and of the future. CMOS has the advantage of higher speeds, higher levels of integration and lower power requirements to enable smaller cameras. The latest CMOS image sensors are outperforming CCD (charge-coupled device) in almost every aspect including dark current, uniformity and noise reading. CMOS image sensors provide less noise and higher dynamic range. On the other hand, CCD is a mature technology that will still be used and applied for certain applications for some time to come.
CMOS sensors are also less expensive as compared to CCD sensors. A CMOS camera has weaker blooming effects if the light sources overload sensor sensitivity, which causes the sensor to bleed the light source onto other pixels.
2. Pixels Will Be Smaller For Industrial Sensors
This trend is pushed by the consumer market where there’s a constant demand for smaller pixels to get higher resolution in small packages. One known benefit of smaller pixels applied in many industrial applications is the ability to have more pixels using the same optical format. This means that you can use the same optics and you can increase the resolution with the same system outline. At present, pixel size is 4.5 to 5.5 um (micrometer). Within 2-3 years, industrial sensors will have a pixel size of 3 to 3.5 um.
3. More Image Processing
With the quality of image sensors increasing, camera makers can spend less time and energy on correcting defects and more time improving image processing and functions. While the camera size can be reduced with the use of CMOS, the camera needs to be large enough to properly address heat issues. This will give camera manufacturers some space to add other processing like color processing, digital zoom, image enhancement, rotation and other features.
4. Lens Enhancements at Lower Costs
CMOS sensors have low noise and high dynamic range. The high dynamic range allows applying more gain, maintaining the current requested camera output resolution in bits. As a result, lens aperture can be selected to balance lens aberrations and lens diffraction. When needed, the degree of freedom can also be extended by using neutral density filters. Also, the recent developments in CMOS image sensor technology will provide more freedom to select the lens parameter, further enhancing performance.
- More affordable lenses
- A more economic lens because of the mean time between failure (MTBF) due to increased reliability because there will be no moving parts
- Lenses with improved image uniformity
- Lenses with enhanced consistent output because all functions are digitally implemented
- Lenses with a more compact outline which is more power efficient and easier to install
5. CoaXPress and USB3 Vision Will Gain Market Share
Many machine vision system designers are moving towards USB3 Vision because it avoids a frame grabber when either the cable length or speed limits are not issues. USB3 Vision will likely continue to gain more market share specifically with planned increases in speed.
Machine visions systems have been an integral part of manufacturing and packaging facilities around the world. The wide variety of applications makes this technology highly flexible and innovative. In the future, we are sure to see many advances in the popularity and quality of these machines.