With passive stereo, there are no limitations in resolution or in the type of illumination.
New technologies in production lines bring new requirements for inspection systems. One example for coming 3-D applications in semiconductor industries is inspection of small solder balls or pins used to connect wafers and dies. These components must be inspected with 3-D methods to measure the precise height of the conducting elements.
In general, the objects to be inspected are getting smaller with new technologies. At this time the typical dimensions of such components are around 5µm. Such sizes require high optical resolution of the inspection system, in the range of at least 5µm. Such small metallic objects often have partially reflective surfaces and come with the demand for high-processing speed.
Well-known 3-D approaches with resolutions in that range are triangulation methods and interferometry. There is a tradeoff between resolution and accuracy, on one hand, and processing speed on the other. Interferometry methods provide high resolution and can handle partly specular reflecting surfaces, but are relatively slow. Triangulation methods are quite fast, but slower with higher resolutions, and work best if surface reflectivity is almost diffuse.
Moving in stereo. Stereo is a common approach and widely used in combination with area cameras, e.g., for high-precision object measurements based on special markers. Stereo in combination with line scan cameras is a novel approach providing high speed and very high optical resolution, which opens new possibilities in 3-D inspection. The basic principle is the same as for area cameras: two line sensors in a stereo configuration combined in one camera.
The stereo concept is to capture two images in a so-called stereo configuration from the same object. Thus, the two stereo images show the same scene from slightly different perspectives and are the basis for triangulation: a triangle can be spanned including the object point and the two corresponding image points. If the positions of the two image points and the camera parameters are known, the distance from the camera to the object point can be calculated from this triangle ().
The preconditions for stereo are:
Active illuminations with fringe patterns or random patterns help identify corresponding image points by bringing known texture on even untextured surfaces and enhance the stability of the algorithm. With passive stereo the structure of the object surface itself is used to find the matching points. The latter approach has the advantage that there are no limitations in resolution or in the type of illumination involved by the active illumination.
The precondition for this approach is structures on the surface of the object can be resolved in the image. Two important parameters influence that: the type of illumination and the optical resolution. The illumination can be chosen in a way that fine structures show up with high contrast. The right optical resolution ensures those structures are resolved in the image.
For stereo, a standard approach to find the corresponding image points is based on pattern matching. In doing so, a small area around each point in one image is used as a pattern to be matched with the corresponding stereo image. The best match of the pattern in the corresponding image is defined as the corresponding image point. Doing this for all points (pixels) in the image results in dense 3-D data.
Unique Features of Stereo with Line Scan
The first stereo line scan cameras had optical resolutions of about 50µm, like other well-known 3-D cameras. The typical dark field line scan illumination has been used to provide high-intensity light for high speed. The advantages of those stereo cameras are speed and getting color images in combination with 3-D. Demand for higher optical resolution came next. It turned out that almost all applications requiring high resolution have partly reflective surfaces as well. To provide the right technology for those new requirements, camera makers developed high-resolution stereo cameras and bright diffuse line illuminations. Combining both – the high-resolution stereo cameras and the diffuse tube light – provides a solution for those applications. Those applications clearly show additional unique advantages of stereo cameras based on line scan:
Typical applications for high-resolution stereo cameras are small objects with metallic surfaces that have to be measured precisely in height.
One application example is to measure the height and shape of small pillars used as connecting elements in semiconductor components (). By using diffuse illumination, specular reflexes are avoided, and the underlying structure on the surface shows up. The images and height map are shown in Figure 3. Here, a 3-D stereo camera with 2.5µm resolution is used.
pseudo-color pseudo 3-D
Another example is small solder joints on PCBs with partly specular reflecting surfaces (). (Images are made with the novel 3-DPIXA stereo camera.)
Inspection of wire bonds requires high resolution as well. Even with the reflective surface on the thin wires, the passive stereo approach shows the relevant details in 3-D. The right curvature is important for the functionality and reliability of the product and can be verified with this approach.
pseudo-color pseudo 3-D
For most applications the stereo algorithms are running on GPUs in the PC. This requires a high-speed data interface to transfer image data from the camera to the PC, making the system quite complex. Running intelligent stereo algorithms fast on embedded hardware will permit calculations to be performed in the camera, which will make the stereo systems smaller, easier to use, and will reduce total system cost. This is an important development step toward wider use of stereo in industrial machine vision.
On the other hand, still-higher optical resolution is necessary as objects get smaller. At higher resolution, the height range becomes limited, since the depth of field is reduced with higher optical magnification. The extension of the height range of line scan stereo with new optical and camera designs will be another challenge for the future.
Klaus Riemer, Ph.D., is 3D product manager at Chromasen’s GmbH (chromasens.de); email@example.com.