UKIVA - Photonex 2015 The technology of light

Hyperspectral imaging in the machine vision world

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

1

Photonex 2015 - Hyperspectral imaging in the machine vision world

Hyperspectral technology from imec

Basis: CMOS sensor CMV2000, 2048 * 1088 pixels

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

2

Photonex 2015 - Hyperspectral imaging in the machine vision world

Motivation - material differentiation White mate paper / white glossy plastics

monochrome picture Visible light (VIS) + Near infrared (NIR)

color picture Visible light (VIS)

 No differences „visible“ Oct. 2015

XIMEA GmbH , Jürgen Hillmann

3

Photonex 2015 - Hyperspectral imaging in the machine vision world

Motivation - material differentiation White mate paper / white glossy plastics reflectance spectra / sun light

paper

400 417 433 450 466 483 499 516 532 548 565 581 597 613 630 646 662 678 694 710 726 742 758 774 789 805 821 837 853 868 884 899 915 931 946 961 977 992

plastics

 spectral signatures can be used to differentiate materials (individual interaction with light) Oct. 2015

XIMEA GmbH , Jürgen Hillmann

4

Photonex 2015 - Hyperspectral imaging in the machine vision world

Motivation - spectral imaging Principles of spectral imaging prism or diffraction grating

image sensor  spectral camera

incident light (line / slit)

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

5

Photonex 2015 - Hyperspectral imaging in the machine vision world

Hyperspectral imaging

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

6

Photonex 2015 - Hyperspectral imaging in the machine vision world

Fabry-Perot spectral filters

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

7

Photonex 2015 - Hyperspectral imaging in the machine vision world

HSI sensor types from imec (used in XIMEA cameras) filter layouts line scan

‘wedge’ design 100 bands ~ 600 - 975 nm

Oct. 2015

Snapshot Mosaic

‘per-pixel’ design 4x4: ~ 470 – 630 nm 5x5: ~ 600 – 975 nm

XIMEA GmbH , Jürgen Hillmann

8

Photonex 2015 - Hyperspectral imaging in the machine vision world

Linescan • High resolution, fast and flexible • 100 spectral bands • 600-975 nm, 4nm incremental steps

• FWHM 15nm • Spatial resolution 2048 x (100 bands x 8 pixels)

• 170 frames/sec

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

9

Photonex 2015 - Hyperspectral imaging in the machine vision world

Snapshot Mosaic • Extremely compact, robust • Spectral resolution: 4x4 mosaic: 16 bands, visible light 470-630nm 5X5 mosaic: 25 bands, near infrared 600-975nm

• Spatial resolution: 4x4 mosaic: 512 x 272 pixels 5X5 mosaic: 409 x 217 pixels

} to 2.2 MPix with interpolation

• 170 data-cubes/sec

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

10

Photonex 2015 - Hyperspectral imaging in the machine vision world

Snapshot Mosaic 4X4 XIMEA camera MQ022HG-IM-SM4X4-VIS

Key specification Spectral resolution: FWHM: Spatial resolution: Speed: Oct. 2015

4x4 mosaic = 16 bands in 470-630nm ~ 15nm from 512x272 up to 170 data-cubes / s (full sensor frame) XIMEA GmbH , Jürgen Hillmann

11

Photonex 2015 - Hyperspectral imaging in the machine vision world

Snapshot Mosaic 5X5 XIMEA camera MQ022HG-IM-SM5X5-NIR

Key specification Spectral resolution: FWHM: Spatial resolution: Speed: Oct. 2015

5x5 mosaic = 25 bands in 600-975nm ~ 16nm from 409x217 up to 170 data-cubes / s (full sensor frame) XIMEA GmbH , Jürgen Hillmann

12

Photonex 2015 - Hyperspectral imaging in the machine vision world

RAW image interpretation (calibration files) 2 leaves on a stone Snapshot mosaic 5X5-NIR, 675-975nm

Single 5X5 pattern, wavelength peaks [nm]

Oct. 2015

900

909

892

882

683

809

821

797

784

693

759

772

746

732

708

943

949

935

927

975

861

873

852

840

955

XIMEA GmbH , Jürgen Hillmann

Spectrum (Raw-values)

675 nm

955 nm

13

Photonex 2015 - Hyperspectral imaging in the machine vision world

Active area

Sensor width (2048 Px)

Sensor height (1088 Px)

Start 1. 5X5 pattern

Active area height (e.g. 1080 Px = 216 px spatial resolution)

RAW image interpretation (calibration files)

Acive area width (e.g. 2045 Px = 409 px spatial resolution)

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

14

Photonex 2015 - Hyperspectral imaging in the machine vision world

RAW image interpretation (calibration files) Simplified, the content of the calibration file is: Sensor size (e.g. 2048 * 1088) Active sensor area (e.g. 5X5 NIR pattern: width: 2045 pixels, height 1080 pixels) Offset from the sensor edges to this active area. (e.g. offset_x=0, offset_y=3) Response measurement results for each band (400-1000 nm, 1nm steps) (virtual bands, correction info) Peak wavelength(s) of each band

The calibration files are stored on the camera.

Oct. 2015

XIMEA GmbH , Jürgen Hillmann

15

Photonex 2015 - Hyperspectral imaging in the machine vision world

Demosaicing / interpolation The spatial resolution in case of a snapshot mosaic sensor is about SM 5X5: