Full-Color Metaoptical Imaging in Visible Light

S.C. are part of the startup Tunoptix, which is commercial- izing some of the technologies described in this review paper. aesthetically pleasing images, as reviewed in this article, or capture more information from the scene, such as depth or spectral. We envision that some of them can even perform computation for object detection or scene understanding.

DOI: 10.1002/adpr.202100265 Metaoptics is a fast-growing field with the potential to dramatically miniaturize image sensors. Unlike many other scientific endeavors, metaoptics are poised to make a technological and commercial impact within a few decades of inception. In particular, metaoptics is touted for next-generation, multifunctional optical elements, with the potential to largely replace refractive optics. The performance of metaoptics for full-color imaging, however, remains poor due to strong chromatic aberrations. Arguably, color cameras are the most prevalent in lives, and the poor performance of metaoptics in this application significantly limits their commercial opportunities. Herein, the current landscape of full-color imaging in the visible wavelength range using metaoptics is reviewed. Imaging approaches using a single metaoptic are focused on, where the size and the weight can be reduced by the largest amount. Future research directions to this effect are outlined.
optics suffer from strong chromatic aberration, significantly limiting their usage in full-color imaging systems. With advances in scatterer engineering via computational electromagnetics, inverse design, and computational imaging, several research groups have reported broadband imaging using these diffractive optics. In this article, we review the current state of the field of broadband imaging using metaoptics. We discuss different means for mitigating chromatic aberration and the limitations of these approaches. We primarily focus on full-color imaging in the visible regime, where images are captured using a redÀgreenÀblue (RGB) camera under broadband, incoherent illumination. These criteria are used as this is how most color cameras function. We also outline the outstanding challenges and future research directions on broadband metaoptical imaging. While full-color imaging using a hybrid system of metaoptics combined with refractive optics can potentially be achieved, [5] here, we consider imaging only using a single metaoptic, where the corresponding size and weight reduction benefits are fully exploited. We do not discuss the research efforts on multilevel diffractive optics in this review to keep the focus solely on metaoptics.

Chromatic Aberrations in Metalens
A conventional metaoptical lens, commonly known as a metalens, with a hyperboloidal phase profile, suffers from strong axial chromatic aberration: lights of different wavelengths focus at different distances from the lens. While the phase response of an individual scatterer in a metaoptic does depend on the optical wavelength, the primary reason for chromatic aberration is phase wrapping [6] (Figure 1a). In a metaoptic, like any diffractive optics, the phase is wrapped to ε when the phase reaches 2π þ ε. The spatial locations where the phase value reaches 2π þ ε, however, depend on the design wavelength. In a metalens, phase wrapping occurs at specific spatial locations, making it very difficult to achieve the ideal phase distribution for every wavelength. [6] This can be easily explained for a quadratic lens phase profile ϕðrÞ, which is an approximation of the hyperboloid phase for r << f , f being the focal length and r the radial coordinate) Assuming a nominal wavelength λ o and a focal length f o , the phase wrapping radii will be given by where N is an integer that governs the amount of phase shift. In a metalens, the phase-wrapping points are fixed, which makes the focal length of the metalens f inversely proportional to the optical wavelength λ. Thus, the chromatic aberration in a metalens comes from the overall phase distribution and not just the wavelength dependence of each scatterer. To achieve broadband operation of a metalens, we need to realize different phase distributions for each wavelength based on a fixed distribution of scatterers.
As the text will discuss, it is indeed possible to create such phase profiles for multiple different wavelengths by appropriately engineering the scatterers. Many of these scatterers consist of multiple subwavelength components, often called a metamolecule in analogy of a molecule being created from multiple atoms. By judiciously designing these metamolecules, one can engineer both the group delay and group delay dispersion in a metalens. Thus researchers have demonstrated broadband focusing using metaoptics. [7][8][9][10] The aperture and NA of these demonstrated broadband metalenses remain very small, and thus the cameras are limited in terms of light collection and spatial resolution. Such limitations on aperture and NA of broadband metalenses have been extensively analyzed using the fundamental time-bandwidth product. [11] Specifically, researchers showed that the maximum achievable fractional optical bandwidth for a broadband metalens with scatterers behaving as optical waveguides is governed by the relation Figure 1. Chromatic aberration in a metalens. a) The phase wrapping in a metalens occurs at specific spatial locations, causing chromatic aberration. b) By considering the time taken by the light to reach the focal spot from the center and the edge of the lens, a fundamental limit on the fractional bandwidth of achromatic metaoptics can be derived. a) Reproduced with permission. [6] Copyright 2016, Optical Society of America. b) Reproduced with permission. [11] Copyright 2020, Optical Society of America.
www.advancedsciencenews.com www.adpr-journal.com Δω Here, Δω ω c is the fractional optical bandwidth; t is the thickness of the metaoptic; F is the focal length; Δn is the index contrast between the metalens material and surrounding material; and n b is the background material index. Such limits have also been explored for metaoptics using PanchratnamÀBerry-phase elements. [11,12] These analyses clearly show that, for a metalens with a given thickness, there is a trade-off between aperture, NA, and the fractional bandwidth. We will review different ways to achieve broadband operation, which is bounded by this fundamental limit. We will also explore ways to possibly circumvent this limit using computational imaging.

Multiwavelength Polychromatic Metalens
Early attempts at creating broadband metaoptics resulted in polychromatic, multiwavelength elements. [13,14] These polychromatic metalenses exhibit the same focal lengths for distinct target wavelengths, but the light does not focus on the same plane at intermediate wavelengths. Such optics could potentially provide achromatic operation, if the illumination could always be restricted to certain discrete wavelengths. The basic principle underpinning such polychromatic designs is to engineer metamolecules that simultaneously satisfy the required phase shifts at each spatial point in the metaoptic for the set of design wavelengths. Initial reports for such designs were limited to infrared wavelengths (1300, 1550, and 1800 nm) and cylindrical lensing based on a diatomic grating structure [13] (Figure 2a). A similar concept was explored for creating a 2D unit cell for polarization-insensitive, multiwavelength operation (200 μm aperture and NA of 0.46 Figure 2b). [6] Here, the multiwavelength operation was demonstrated at 915 and 1550 nm. The reason for such different wavelengths was that the unit cell comprised two very different diameters of cylindrical scatterers to minimize mode coupling. Following similar techniques, various spatial multiplexing methods capable of creating polychromatic metalenses have been reported (Figure 2c). [15] While such techniques can be applied to larger apertures, they are not suitable for broadband operations and have not yet been demonstrated in color-imaging applications. Given that the essence of these designs is metamolecule engineering, inverse methods are well Figure 2. Polychromatic metaoptics via unit-cell engineering. a) Cylindrical metalens made of amorphous silicon focuses light at three specific wavelengths at the same plane. The pitch of the metaoptics is S ¼ 1 μm. b) By engineering the unit cell, a metalens operates at two different wavelengths. c) Polychromatic metalens can be realized via spatial multiplexing of the scatterers. a) Reproduced with permission. [13] Copyright 2015, AAAS. b) Reproduced with permission. [6] Copyright 2016, Optical Society of America. c) Reproduced with permission. [15] Copyright 2016, Springer Nature Ltd.
suited, as recently reported [16] in a 1 cm-aperture metaoptic used for projection in a virtual reality system, where the wavelengths of the light source can be easily controlled.
Other than unit cell engineering, researchers have demonstrated multiwavelength operation by vertically stacking metaoptics ( Figure 3a). [17] In one such approach, plasmonic scatterers are used to create three metalenses independently optimized for focusing light at 650, 550, and 450 nm. The aperture of the metalenses used was % 200 μm and NA is 0.1. Similar stacked metaoptics have also been demonstrated for polychromatic imaging at infrared wavelengths. [18] In another approach, by engineering metamolecules near the zone boundaries, that is, where the phase wraps modulo 2π, polychromatic focusing was achieved for 658, 532, and 488 nm using a 2 mm-diameter metalens with an NA of 0.7 ( Figure 3b). [19] This work also demonstrated color projection of an image for applications related to virtual reality. We note that multichrome diffractive optics have been explored using multiorder diffractive optics before. [20][21][22] However, the design and operating principles of multiwavelength metaoptics are different due to lack of any higher orders of diffracted light.

Dispersion Engineering
While there are many impressive demonstrations of polychromatic lenses that do have applications in projection and while there is a large degree of control over the illumination wavelengths, these designs are not quite suitable for broadband imaging. The spectral filters used in color cameras are quite broad, spanning from tens to hundreds of nanometers in bandwidth relative to the assumed discrete, laser-like illumination of polychromatic designs. As such, if polychromatic designs were used with color cameras, the broad bandwidth of the color filters would cause defocused light from other incoming wavelengths to arrive at the sensor, blurring the resultant image. To ensure high-quality broadband imaging, one needs to ensure that the optics focus all colors in the desired wavelength band (e.g., the full visible spectrum) on the same plane. The phase profile ϕðr, ωÞ of a metalens can be expanded using a Taylor series near the design frequency ω o (4) Figure 3. RGB polychromatic metalens. a) By stacking three metasurfaces, red, green, and blue (RGB) colors can be focused on the same plane. b) By engineering the metamolecules at the zone boundary, red, green, and blue light can be focused on the same plane. a) Reproduced with permission. [17] Copyright 2017, Springer Nature Ltd. b) Reproduced with permission. [19] Copyright 2021, AAAS. Broadband imaging requires designing metamolecules to achieve both phase ϕðr, ω o Þ and group delay ∂ϕ ∂ω over a broad range of wavelengths. By exploiting the full electromagnetic interactions between scatterers in a metamolecule, it is indeed possible to achieve desired phase and group delay for different wavelengths. Such a metamolecule design, also known as dispersion engineering for broadband imaging, was first reported under reflection: a bandwidth of 140 nm nm at the central frequency of 1520 nm [7] and a bandwidth of 60 nm at the central frequency of 520 nm [23] were achieved. The reflective geometry was chosen to facilitate the engineering of meta-atoms. A reflective design, however, is far more limited in its applications for consumer cameras.
Progress was made soon after, with dispersion engineering exploited for broadband, transmissive metaoptics. [8,9] By engineering the metamolecules, light over the entire visible spectrum can be focused at the same focal length. In one work, the metalens demonstrated broadband focusing over the 400 À 660 nm range with an aperture of 100 μm and NA of 0.106 [9] ( Figure 4c). In another work, a metalens of aperture 200 μm and NA of 0.02 was used to focus the 470 À 670 nm range ( Figure 4a). Both of these demonstrations depend on the polarization of the incident light. By engineering the meta-atoms, researchers demonstrated an polarization-insensitive achromatic metalens: while each anisotropic metamolecule has some polarization dependence, by restricting the rotation angle of each anisotropic element to either 0 or 90 , polarization-insensitive operation was achieved. [24] Here, the achromatic operation is demonstrated over 460 À 700 nm using a 26.4 μm-aperture metalens of NA of 0.2. While such broadband focusing provides a route to color imaging, the demonstrated apertures and NAs are very small for most practical applications. Similar dispersion engineering methods have also been used for infrared wavelength range ð1200 À 1650 nmÞ, though these demonstrations equally suffer from very small apertures [10] (Figure 4b). By exploiting the height as another parameter, 3D-printed achromatic metalenses have been demonstrated in the 1000 À 1800 nm range, although the aperture of these lenses is less than 100 μm. [25] Using fishnet metaoptics, researchers also demonstrated achromatic focusing in the 640 À 1200 nm range; [26] however, the apertures of these lenses are very small, %50 μm, and no color imaging was reported. As such, dispersion-engineered metalenses suffer from a stringent trade-off between the NA and aperture size versus achievable optical bandwidth. A more detailed review of dispersion-engineered metaoptics can be found in other studies. [27,28]

Computational Imaging
Another approach to achieve full-color imaging is by exploiting computational imaging. Using metaoptics, we can capture an intermediate image that preserves as much information as possible over all the color channels in the sensor. This intermediate image can be subsequently decoded in software to extract a  [8] Copyright 2018, Springer Nature Ltd. b) Reproduced with permission. [9] Copyright 2018, Springer Nature Ltd. c) Reproduced with permission. [10] Copyright 2018, Springer Nature Ltd.
www.advancedsciencenews.com www.adpr-journal.com full-color image. In this way, the modulation transfer function (MTF) at one specific wavelength will always be inferior to that of a metalens specifically designed for that same wavelength; however, when considering the full visible spectrum, we can increase the total integrated MTF relative to a traditional metalens. In this process, we need to ensure the MTF is broad, that is, the area under the 1D MTF curve should be maximized, while also ensuring that the point spread function (PSF) is the same for all the wavelengths in a given color channel. This essentially means that we need to have a large Strehl ratio, defined as the ratio between the volume under the MTF surface of the metaoptics and the volume under the MTF surface for a diffraction-limited lens.

Forward Design
One way to achieve the aforementioned functionality is by exploiting extended depth of focus (EDOF) metalenses. These lenses produce an elongated focal zone or line instead of diffracting light to a focal spot. Due to chromatic dispersion, the centroid of the focal line changes with wavelength; however, if we can extend the depth of focus significantly, we can ensure that light at every wavelength reaches the sensor in a similar manner, mapping into a color-invariant PSF. One way to achieve this is using cubic metaoptics. [29] Extending the depth of focus using a cubic-phase plate was proposed more than 20 years ago and arguably, pioneered the field of computational imaging. [30][31][32][33] While the application of chromatic correction of refractive lenses using such wavefront coding was explored earlier, this was only recently applied for metaoptics (Figure 5a). [29] While an ordinary metalens can provide a broad MTF at one wavelength (here, green), for red and blue light, the MTF degrades significantly, exhibiting zeros in spatial frequency that represents an irrecoverable loss of scene content. Cubic metaoptics, however, not only produce a wavelength-invariant MTF but also capture a large range of spatial frequencies without any zeros, enabling deconvolution for image extraction. Unfortunately cubic EDOF metaoptics also produce an accelerating beam that induces lateral chromatic aberration. This can potentially be circumvented using a rotationally symmetric EDOF lens, such as a log-asphere or shifted-axicon lens. [34] These demonstrated EDOF lenses based on canonical-phase masks unfortunately enable little control over the MTF.

Inverse Design
One way to increase the operating optical bandwidth and the total integrated MTF is to use inverse design to arrive at high-performance solutions with nonintuitive forms. Here, one can define the desired performance of the metaoptics using figure of merit (FOM) and optimize the scatterer distribution to reach the desired FOM. A recent work on an inverse-designed EDOF lens defines the FOM to maximize the intensity along a line in the optical axis. [35] The optical bandwidth can be defined as the wavelength range where the correlation function between the MTFs remains larger than 0.5. The inverse-designed metaoptics thus indeed showed larger optical bandwidth compared to that of a metalens or other canonical EDOF lenses ( Figure 5). In fact, a bandwidth of 290 nm was reported in inverse-designed EDOF metaoptics, at least twice larger than other EDOF metaoptics. The image quality captured via the EDOF metaoptics also outperforms the traditional EDOF metaoptics, measured via the structural similarity (SSIM) factor: for red, green, and blue channel, the calculated SSIM factors are 0.64ð0.31Þ, 0.8ð0.65Þ, and 0.48ð0.47Þ for inverse designed (cubic) EDOF metaoptics. With inverse design, the bandwidth can be further increased, as shown by the correlation coefficient. a) Reproduced with permission. [29] Copyright 2018, AAAS. b) Reproduced with permission. [35] Copyright 2021, De Gruyter.

End-to-End Design
In most demonstrations of metaoptics for full-color computational imaging, the metaoptics and the reconstruction method are separately designed. Moreover, the deconvolution methods typically used are based on linear operators, the performance of which is solely determined by the area under the 1D MTF curve and requires higher signal to noise ratio (SNR) to achieve the same spatial resolution as in a traditional system. Researchers have already argued that joint optimization of optics and software together can reduce form factor and enhance performance. [36] Such codesign techniques, also known as "end-to- Figure 6. Co-optimized hardwareÀsoftware for full-color imaging: a) Design framework to co-optimize the metaoptic and reconstruction algorithm via a differentiable pipeline. b) Images captured using a single 500 μs, f =2 metaoptics. c) Same images captured using a six-element compound optic. Reproduced with permission. [39] Copyright 2021, Springer Nature Ltd.
www.advancedsciencenews.com www.adpr-journal.com end" design, have already found applications in diffractive optics. [37] Recently, such efforts have been applied to metaoptics for imaging and polarimetry, [38] as well as for broadband fullcolor imaging. [39] Here, simple cylindrical scatterers are used as the meta-atoms, and a differentiable proxy function is developed to connect the scatterer geometry to the phase distribution. The image formation via metaoptics and computational reconstruction is modeled in a differentiable pipeline (Figure 6a) that can be optimized using automatic differentiation. To account for geometric aberration, spatially varying PSFs are considered.
Here, the computational reconstruction techniques leverage a combination of classical, linear deconvolution that helps to preserve generality, while also utilizing state-of-the-art featureextraction networks and nonlinearities to maximize the image quality and denoising, while maintaining high spatial resolution. Figure 6b shows the images captured using the end-to-end designed metaoptics and computational reconstruction. The end-to-end optimized metaoptic has an aperture of 500 μm and a focal length of 1 mm and the deconvolution routine only required 58 ms. Figure 6c shows the same images captured using a commercially available six-element compound lens. The image quality obtained via metaoptics is comparable with that of the refractive lens, as quantitatively proven using SSIM and peak signal-to-noise ratio metrics. [39] 6. Outlook We reviewed various approaches to achieve broadband full-color imaging in the visible range. When color information from a scene is of importance, we believe that a color camera is necessary for capturing and distinguishing this information versus a monochrome sensor. While there are several works showing an identical focal length over a broadband range, for practical applications one needs to capture full-color images. Unfortunately, the captured image quality often is poor, even though the focal length remains the same over a large optical bandwidth. Figure 7 shows published full-color images using metaoptics over the years, which, while impressive in their own right, are clearly inferior in quality to those produced by widely available commercial cameras based on refractive lenses. While some recent images are complex and comparable with those of refractive lens-captured images, they require computational reconstruction. The reason for this apparent discrepancy between demonstrated focusing behavior and actual imaging comes from noisy capture, poor MTF, and a limited field of view. Table 1 shows performance of various metaoptics reported so far for full-color imaging. In this section, we outline several challenges the metaoptics community must confront and possible directions for future developments.

Standardizing Benchmarking of Image Quality
We need a standardized metric to assess image quality using metaoptics between different methods going forward. While the information capacity of an imaging system is well captured by the MTF and SNR, these metrics alone are also known to not provide sufficient criteria for consumer photography, where almost always some computational amelioration of the images is performed, as extensively written by researchers from the industry. [40] Unfortunately, these metrics used for consumer photography, such as SSIM or peak signal-to-noise ratios, require imaging with a fixed dataset for fair comparison. This will require determining a fixed set of images to compare the performance for various different approaches in the metaoptics imaging community.

Larger Aperture
Any optical imaging system will need to collect a certain number of photons to be useful, and this requires a large aperture. For example, current cameras in smartphones have an aperture of at least 2 mm. Most achromatic metaoptics reported so far have sub-mm apertures. The increase in aperture from the mm to cm scale has four primary challenges. First, there are some fundamental limits in achieving broadband operation in an ultrathin form factor, as described by others. [11] With a computational back end, there may exist a route to circumvent this, although this has not been explicitly proven. Second, Seidel aberrations of a lens increase with aperture. [41,42] This makes capturing high-quality images at larger aperture difficult, especially while maintaining a large field of view. A computational back end may also be of benefit here, while another option is to use multiple metaoptics as a means to increase the field of view without introducing large aberrations. [43][44][45][46][47] Third, metaoptics presents a challenging multiscale electromagnetics problem: while each scatterer needs to be modeled rigorously using Maxwell's equations to maintain accuracy, a full metaoptic must be described in terms of ray or Fourier optics to be computationally feasible. Several recent Figure 7. Full-color imaging using single metaoptics over the years shows progress, but the image quality is still worse than a simple refractive lens. While some recent results may be comparable with compound refractive lenses, this require a large amount of computation. While the academic community is focusing on many other applications of metaoptics, we believe not being able to capture high-quality color imaging will significantly limit the commercial application space of metaoptics. Hence, it is imperative to solve this problem, and we outline several possible directions toward that end. Images from left to right are reproduced with permission. [17] Copyright 2017, Springer Nature Ltd. The three images from bottom to top: Reproduced with permission. [8] Copyright 2018, Springer Nature Ltd. Reproduced with permission. [9] Copyright 2018, Springer Nature Ltd. Reproduced with permission. [29] Copyright 2018, AAAS. Reproduced with permission. [34] Copyright 2019, Optical Society of America. Reproduced with permission. [35] Copyright 2021, De Gruyter. Reproduced with permission. [39] Copyright 2021, Springer Nature Ltd.
www.advancedsciencenews.com www.adpr-journal.com works based on transfer matrix, [48][49][50] scatterer proxy functions, [39,51] or deep learning [52,53] have shown large acceleration in the forward simulation and could be beneficial for designing achromatic metaoptics. Fourth, the fabrication of large-aperture metaoptics is difficult. Most current visible metaoptics are fabricated using electron beam lithography, which is not conducive to scalable manufacturing. While the dimensions of simple structures, such as cylindrical or square pillars, are within reach of deep-ultraviolet or immersion lithography systems, [54][55][56] complicated metamolecules will be difficult to fabricate. Moreover, the lithography systems are well suited to fabricate apertures at the die size, which is around 2.5 cm. Anything larger than that, which may find applications for space-based optics, will require step-and-stitch approaches combining multiple dies and can be very expensive. Approaches involving synthetic apertures could be beneficial for such large-scale metaoptics. [57] In fact, arrays of achromatic metaoptics have already been reported for lightfield imaging. [58] Another promising direction could be nanoimprint lithography, as recently demonstrated by NIL Technology. However, the index of the resist typically used in nanoimprint is low and may affect the efficiency of the metaoptics. Granted, using the resist as a mask for etching could potentially alleviate the problem, as recently demonstrated by NIL technologies for near-infrared lenses. Finally, metaoptics in visible wavelength requires a thin film of thickness % 500 À 700 nm. Creating high-quality films with such thickness could be a challenge. Current foundry services in semiconductor photonics, which are primarily centered around integrated photonics industries, generally work with % 200 À 400 nm-thick films. As such, highvolume manufacturing of metaoptics will remain an important milestone to achieve for any application and not just for full-color imaging.

Efficiency
Efficiency is an important metric for optical elements. Traditional optical elements are primarily characterized by transmission efficiency and Strehl ratio. In the metaoptical community, researchers also report focusing efficiency, which is defined as the ratio of the power inside a circle around the focal spot with a radius of three times the full-width-half maxima of the spot size and the total power in the focal plane. This metric is connected to the MTF and Strehl ratio. While high transmission efficiency ð> 90%Þ can be achieved using a transparent material in the visible range, such as SiN, GaN, or TiO 2 , most full-color metaoptics suffer from either low focusing efficiency or low Strehl ratio, especially when implementing high NA or fast lenses. Multiwavelength metaoptics in particular, which exploit multiplexed structures in a unit cell, significantly suffer from low efficiency (% 22 À 32% in one wavelength). [6] With dispersion engineering, the efficiency still remains relatively poor (% 40%). [8] Computational imaging-based approaches for full-color imaging seek to ensure all colors and angles reach the sensor in an identical manner, which can result in an optic with a lower Strehl ratio. However, the use of deconvolution and denoising methods in computational imaging can restore high-frequency image components with a well-designed MTF. Such computational approaches still require high-efficiency metaoptics to ensure adequate SNR for processing.

Imaging in the Wild
For a large number of practical applications, the imaging needs to be performed using ambient light. Unfortunately, in most works, lasers or light-emitting diodes, including organic light emitting diode (OLED) displays, were used in a controlled lab environment, often used in conjunction with an optical relay.
In near-and long-wave-infrared wavelength range, some works did perform imaging using just a single metaoptic, [43,59,60] but they used a filter to limit the wavelength range of the incident light. Similar filtered imaging methods using metaoptic integrated cameras have been reported recently for green light. [61] To the best of our knowledge, there has been no demonstration FOV was not reported, so we estimated FOV assuming that aperture diameter equals sensor size; b) FOV is determined from off-axis simulations, which were taken up to AE10°; c) FOV is determined for a Strehl ratio of 0.8; d) In this design, the reported pitch corresponds to the side length in a hexagonal lattice; e) Reported for the largest aperture among the designs proposed in this work.
www.advancedsciencenews.com www.adpr-journal.com of full-color imaging using a metaoptics in the wild. This may need a departure from the "device engineering" mindset of researchers working in the metaoptics field. Without improving just metaoptics, a system-level performance needs to be assessed. Thankfully, there are several startups working on metaoptics, such as Metalenz, Leia, Lumotive, and Tunoptix. As such, we hope that full-color imaging systems involving metaoptics will be commercialized soon.

Video Capture
While full-color imaging is still an important achievement, many applications require video imaging with a frame rate of 33 frames s À1 . High-quality full-color video imaging has not yet been demonstrated using metaoptics. This problem is related to the use of relay optics, requiring longer exposure times. The need for relay optics is intimately related to the aperture as well, especially because most commercial sensors are covered with a glass slip of thickness % 1 mm. Thus to ensure small f-number, the aperture at least needs to be in the mm-scale. Achieving such apertures is difficult using dispersion engineering. While computational reconstruction may allow larger aperture, current computational times may not allow real-time video capture. Pruning of the computationally expensive algorithms or hardware acceleration techniques needs to be applied to solve this problem.

Computational Metaoptics
Finally, the computational framework developed for "end-to-end" design can be used for many other sensing and computational tasks ( Figure 8). We believe such co-optimized hardware and software can find applications in depth sensing, object detection, or for optical information processing. One particularly promising direction will be optical neural networks. [62,63] While all-optical diffractive neural networks have already been reported, [64] a computational back end can potentially allow performing inference using incoherent light. [65,66] Figure 8. Outlook for co-optimized hardwareÀsoftware platform: by optimizing a single metaoptic or a stack of metaoptics, along with a computational backend, a dramatic reduction in size, weight, power, and latency of image sensors can be achieved. Such sensors can either capture aesthetically pleasing images, as reviewed in this article, or capture more information from the scene, such as depth or spectral. We envision that some of them can even perform computation for object detection or scene understanding.