DON26BZ01-NV011 TITLE: Compact Battery Operated Mid-wave Infrared (MWIR) Hyperspectral, High-Definition, Real-Time Video Camera Integrated with Photonic Crystal

COMPONENT TECHNOLOGY PRIORITY AREA(S): Advanced Materials;Microelectronics;Quantum Science

PROJECTED CMMC LEVEL REQUIREMENT: Level 2 (Self)

OBJECTIVE: Develop and demonstrate a compact battery-operated mid-wave infrared (MWIR) hyperspectral imaging (HSI) photonic chip video camera for integration into mobile network enabled small sensor platforms.

DESCRIPTION: Hyperspectral imaging allows quantitative evaluation of material composition and spatial distribution and finds numerous applications in areas such as remote sensing and military reconnaissance. In particular, the operational utility of HSI for detection, recognition and identification of hard-to-detect targets in environments cluttered with background noise is especially critical. Spectral imaging can aid the detection, acquisition and tracking of a potentially camouflaged, low-signature target, with significantly improved accuracy that cannot otherwise be detected using more conventional imaging means.

Conventional HSI systems [Refs 1, 2] tend to use large, bulky optical elements, such as a Michelson interferometer or other tunable optical filter components to spectrally resolve the input optical signals, and therefore usually have the characteristics of significant size, weight, and power (SWaP) consumption, mechanical complexity, as well as non-compliance with military specifications. More importantly, the mechanical mechanism of the conventional tunable filtering system gives rise to extremely slow spectral scanning speed and thus, slow imaging speed at that rate of one hyperspectral image per approximately one to two minutes. Traditionally, hyperspectral algorithms have considered only static images, and existing algorithms process single frames without regard for sequential similarities or correlations. The difficulty in capturing and processing hyperspectral video sequences in real-time is correlated directly to the high dimensionality of the data. As a result, conventional HSI systems cannot be deployed to the more demanding field applications that require images that can be captured and analyzed on a real-time basis at a much higher frame rates due to HSI’s inherent image acquisition speed bottleneck.

A typical hyperspectral image consists of a high-resolution 3-dimensional (3-D) data cube, with two dimensions in space and a third dimension in wavelength. A focal plane array (FPA) can only acquire a 2D data set in one exposure. In the conventional approach, spectral scanning is thus often used to attain the third dimension of wavelength for a 3-D data cube. As stated earlier, this process makes HSI operation very slow because wavelength scanning requires multiple exposures over a specific spectral range. In addition to the very slow scan speed, it also suffers from a low signal-to-noise ratio (SNR) resulted from a high level of noise in infrared detectors and a low light throughput caused by narrow-band filters used in spectral scanning. The use of narrow-band filters also limits the number of spectral bands.

Infrared spectroscopy routinely uses spectral multiplexing to overcome the challenge of detector noise. This is known as the Fellgett’s multiplexing advantage [Ref 3]). The best example is Fourier-transform infrared spectroscopy (FTIR). Instead of spectral scanning, it projects an unknown spectrum onto a serial of sinusoidal functions constructed by a Michelson interferometer and thus greatly improves light throughput. However, it is difficult to integrate FTIR with FPA because of their bulky size and single channel operation. Recently, on-chip multiplexing has emerged as a new approach for hyperspectral sensing. It uses the spectral response of judiciously designed nanostructures to construct the multiplexing basis. Exploiting optical interference and resonance effects at the nanoscale, these nanostructures can provide a highly complex and diverse range of response functions suitable for efficient multiplexing [Ref 2]. They can be directly integrated into FPA to have an ultra-compact form factor. Multiplexing can be performed in both spectral and spatial domain. Advanced data-driven optimization such as machine learning can be used together with compressive sensing to reconstruct 3D data cube in single-shot operation [Ref 4].

It is therefore the objective of this SBIR topic to develop a battery-operated, compact, high-performance MWIR HSI camera system capable of capturing HSI video at real-time or higher frame rates in the room temperature thermal infrared region. One of the key challenges in on-chip photonic multiplexing of a photonic crystal-integrated FPA is the computational design. Constructing the multiplexing basis is a delicate balance between the physical limit of on-chip photonic structures and the imposed requirement from demultiplexing algorithms. The former requires solving multi-scale Maxwell’s equations, and the latter requires large-scale data-driven optimization of demultiplexing algorithms. The coupled design process needs to be iterated efficiently to reach any useful design. It is expected that this challenge can be addressed by using massively parallel simulation of electrodynamics paired with efficient optimization algorithms such as adjoint method.

The project should demonstrate a systematic design method that leverages large-scale simulation, machine learning, and data-driven design to realize real-time hyperspectral video imaging. The final goal of this project is to experimentally demonstrate a battery operated MWIR HSI video camera with the following specifications.

System required parameters include:

1. Wavelength range: 3-5 microns

2. Array size: Threshold -- 1280 x 1024 pixels; Objective -- 2048 x 1536pixels

3. Spectral resolution: below 5 nm

4. Pixel pitch: Threshold – 12 microns; Objective – 8 microns

5. Real-time hyperspectral video imaging Programmable; 0.0015 Hz to 125 Hz frames per second

6. Acquisition time of hyperspectral image with 500 spectral bands: < 40 ms or minimum 25 video data cubes (each with 500 spectral bands) per second.

7. Size and Weight: 7.5 grams and < 4.9 cm³

8. Battery Type: Lithium-ion battery enhanced by using carbon-based nanostructures with a specific energy > 600 Wh/kg at 0.5C discharge rate, and specific capacity of > 600 Ah/kg.

9. Low power consumption, starting at 600 mW

10. Humidity Non-condensing between 5% - 95%

11. Non-Operating Temperature Range -57 °C to +80 °C (-65 °F to +176 °F)

12. Operating Temperature Range -40 °C to +71 °C (-40 °F to +160 °F)

13. Operational Altitude 40,000 ft (~12km)

14. Shock 40g w/ 11ms half-sine pulse, 3-axis

15. Vibration 5.8 grms 3-axis, 1hr each

Responding to the 21.2 AC1 S&T Domain comments: Surface Optics produces multi-spectral camera that can only provide multispectral images with about 10 spectral bands. Also, their multi-spectral camera is in the SWIR band. This current topic is for the first time a topic that can produce MWIR hyperspectral images at better than real-time video frame rate (24 frames per second or higher) with up to 500 (not 10 in the multispectral camera situation) high-resolution hyperspectral images per frame. This current proposed technology can produce up to 50 times more spectral information than the current multispectral camera in the market. Hence, there is zero overlap in terms of technology innovation between what Surface Optics and other commercial concerns market as multi-spectral or hyperspectral cameras and this current topic. In fact, the current proposed topic's performance and SWaP are far superior to any commercially available hyperspectral images by a 10 to 1 to 50 to 1 wide margin.

PHASE I: Demonstrate the feasibility of using massive parallel computing to design hyperspectral photonic chips. The design should shorten the acquisition time by 100X, i.e., reducing the time from seconds in traditional HSI to a few milliseconds. It should improve the spectral resolution by 10 X, i.e., going from tens of nanometers to a few nanometers, which translates into over 500 spectral bands in the MWIR band (3 to 5 um). The spatial resolution should be consistent with today’s FPA resolution to reach 3.1 -mega pixel for high-definition images. Lastly, the design could be realized in ultra-compact form factor, reducing traditional HSI’s size and weight by 10 – 20 times.

The Phase I effort will include prototype plans to be developed under Phase II.

PHASE II: Optimize the unit cell design and lattice lengths to allow for (i) polarization insensitivity, (ii) WFOV performance, and (iii) broadband performance. Demonstrate a fully coupled design process using machine learning and physical simulation. Perform experimental verification of the generated design by demonstrating a real time MWIR hyperspectral imager with 5 nm spectral resolution, 1.3-million-pixel count, at 30 Hz frame rate. Demonstrate tunable focal lengths using lens-embedded photonic crystals.

PHASE III DUAL USE APPLICATIONS: Outputs from Phase II are anticipated to be TRL 7 but may require additional effort to refine to a more manufacturable design.

Concentrate on the manufacturability as well as the fabrication process itself to prepare for commercial offerings of a fully functional product. Demonstrate and market the final product to DoW and the Navy transition partner. Perform additional Phase III planning once a design is established and manufacturing requirements and manufacturability become more apparent.

The commercial potential includes new handheld and portable instruments for chemical, photometric, and biological sensing. Photonic crystal cameras can be integrated into compact form factors that enable in situ measurement for manufacturing process analysis and in-process feedback control. Applications include solid-state lighting characterization and testing, emissions control, portable sensing, and personal health care.

REFERENCES:

1. Blake, T. A.; Kelly, J. F.; Gallagher, N. B.; Gassman, P. L. and Johnson, T. J. "Passive standoff detection of RDX residues on metal surfaces via infrared hyperspectral imaging." Analytical and Bioanalytical Chemistry, 395(2), 2009, pp. 337-348. https://doi.org/10.1007/s00216-009-2907-5
2. Tochon, G.; Chanussot, J.; Dalla Mura, M. and Bertozzi, A. L. "Object tracking by hierarchical decomposition of hyperspectral video sequences: Application to chemical gas plume tracking." IEEE Transactions on Geoscience and Remote Sensing, 55(8), 2017, pp. 4567-4585. https://doi.org/10.1109/TGRS.2017.2694159
3. Fellgett, P. B. "On the ultimate sensitivity and practical performance of radiation detectors." Journal of the Optical Society of America, 39(11), 1949, pp. 970-976. https://doi.org/10.1364/JOSA.39.000970
4. Wang, Z.; Yi, S.; Chen, A.; Zhou, M.; Luk, T. S.; James, A.; Nogan, J.; Ross, W.; Joe, G.; Shahsafi, A.; Wang, K. X.; Kats, M. A. and Yu, Z. "Single-shot on-chip spectral sensors based on photonic crystal slabs." Nature Communications, 10(1), 1020. 2019. https://doi.org/10.1038/s41467-019-08994-5

KEYWORDS: Photonic crystal; Hyperspectral; Camera; Focal plane array; On-chip; Filter

TPOC:
NAVAIR SBIR/STTR POC
navair-sbir@us.navy.mil