Metamaterials

based thin portable

fundus camera
Brief intro for bridging the
image quality gap between
bulky desktop/handheld cameras
and smartphone snap-ons

Petteri Teikari, PhD

London, United Kingdom
http://petteri-teikari.com/
Artwork by From metamaterials to metadevices
Version “Tue 27 August 2019“
@artofolly: “Cry A Rainbow “ Nikolay I. Zheludev & Yuri S. Kivshar
Nature Materials volume11, pages917–924 (2012)
https://doi.org/10.1038/nmat3431
Metamaterials
What are these?
Metasurfaces Intro
Optical Metasurfaces: Progress
and Applications
Annual Review of Materials Research
Vol. 48:279-302 (Volume publication date July 2018) 
First published as a Review in Advance on March 21, 2018 
https://doi.org/10.1146/annurev-matsci-070616-124220
Shengyuan Chang, Xuexue Guo, and Xingjie Ni
Department of Electrical Engineering, The Pennsylvania State University, University Park

A metasurface is an artificial nanostructured interface that

has subwavelength thickness and that manipulates light by
spatially arranged meta-atoms—fundamental building blocks
of the metasurface. Those meta-atoms, usually consisting of
plasmonic or dielectric nanoantennas, can directly change
light properties such as phase, amplitude, and polarization. As
a derivative of three-dimensional (3D) metamaterials, metasurfaces
have been emerging to tackle some of the critical challenges rooted
in traditional metamaterials, such as high resistive loss from resonant
plasmonic components and fabrication requirements for making 3D
nanostructures. In the past few years, metasurfaces have
achieved groundbreaking progress, providing unparalleled
control of light, including constructing arbitrary wave fronts and
realizing active and nonlinear optical effects. This article provides a
systematic review of the current progress in and applications of
optical metasurfaces, as well as an overview of metasurface
building blocks based on plasmonic resonances, Mie resonance,
and the Pancharatnam-Berry phase.
Metalenses quite mainstream already

https://www.technologyreview.com/s/613918/why-metalenses-are-about-to-revolutionize-chip-making/
arxiv.org/abs/1906.10681 : Metalens with Artificial Focus Pattern

Structure and characterization of metalens with

‘M’shaped focus.(a)SEM picture of the metalens’s overall
structure. (b) Magnified view of the lens which shows the
structureof the cylinder phase shifter,size of scale bar is
https://www.economist.com/science-and-technology/2019/07/25/h 500 nm. Both SEM picture are taken after gold sputter.
ow-to-make-a-flat-lens (c) Characterized focal plane with incidence of 685 nm
red light, two cross-sections are specified. (d) Focus
profile at two cross-sections specified on ‘M’shaped
focus.
Metalenses have been cool for a while #1
18 Jan 2013
http://doi.org/10.1126/science.1230054
By leveraging metamaterials and compressive imaging, a low-profile
aperture capable of microwave imaging without lenses, moving parts, or
phase shifters is demonstrated. This designer aperture allows image
compression to be performed on the physical hardware layer rather
than in the postprocessing stage, thus averting the detector, storage, and
transmission costs associated with full diffraction-limited sampling of a scene. A
guided-wave metamaterial aperture is used to perform compressive image
reconstruction at 10 frames per second of two-dimensional (range and angle)
sparse still and video scenes at K-band (18 to 26 gigahertz) frequencies, using
frequency diversity to avoid mechanical scanning. Image acquisition is
accomplished with a 40:1 compression ratio.
The imaging system we present here combines a computational imaging approach
with custom aperture hardware that allows compression to be performed on the
physical layer that is used to do the illumination and/or recording. The use of
metamaterials is a convenient tool for the creation of such apertures, as
metamaterial techniques offer a well-understood design path. Leveraging the
resonant nature of metamaterial elements also creates frequency diversity of the
measurement modes, giving an all-electrical method of quickly sweeping through a
mode set. This metamaterial approach scales linearly with frequency through
terahertz frequencies with correspondingly higher resolutions. Due to their small
form factor and lack of moving parts, similar systems may extend microwave and
millimeterwave imaging capabilities
Metalenses have been cool for a while #2

5 Oct 2017
Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), Harvard University. Present
address: Magic Leap Inc., Plantation, FL 33322, USA.http://doi.org/10.1126/science.aam8100

Recent progress in metasurface designs fueled by advanced-fabrication

techniques has led to the realization of ultrathin, lightweight, and flat
lenses (metalenses) with unprecedented functionalities. Owing to
straightforward fabrication, generally requiring a single-step lithography, and the
possibility of vertical integration, these planar lenses can potentially
replace or complement their conventional refractive and diffractive
counterparts, leading to further miniaturization of high-performance
optical devices and systems. Here we provide a brief overview of the evolution of
metalenses, with an emphasis on the visible and near-infrared spectrum, and
summarize their important features: diffraction-limited focusing, high-quality
imaging, and multifunctionalities. We discuss impending challenges,
including aberration correction, and also examine current issues and solutions.
We conclude by providing an outlook of this technology platform and identifying
promising directions for future research.
Metalenses for imaging #1
Metasurface optics for full-color computational imaging Science Advances 09 Feb 2018: http://doi.org/10.1126/sciadv.aar2114
Conventional imaging systems comprise large Design, simulation, and fabrication of imaging
and expensive optical components that metasurfaces.
successively mitigate aberrations. Metasurface
optics offers a route to miniaturize imaging (A) The metasurfaces are made up of silicon nitride nanoposts,
where the thickness T, lattice constant p, and diameter d are the
systems by replacing bulky components with design parameters. (B) Schematic of a metasurface comprising
flat and compact implementations. The diffractive an array of nanoposts. (C) Simulation of the nanoposts’
nature of these devices, however, induces severe transmission amplitude and phase via RCWA. Simulated
chromatic aberrations, and current intensity along the optical axis of the singlet metasurface lens (D)
and EDOF metasurface (E), where, going from top to bottom in
multiwavelength and narrowband achromatic each panel, 400, 550, and 700 nm wavelengths are used. The
metasurfaces cannot support full visible dashed lines indicate the desired focal plane where the sensor
spectrum imaging (400 to 700 nm). We will be placed. Optical images of the singlet metasurface lens (F)
and the EDOF device (G). Scale bars, 25 μm.m.
combine principles of both computational
imaging and metasurface optics to build a
system with a single metalens of numerical
Imaging with white light.
aperture ~0.45, which generates in-focus images
under white light illumination. Our metalens
exhibits a spectrally invariant point spread Images were taken under white light illumination of color printed
RGB (A) and ROYGBIV (B) text, a colored rainbow pattern (C),
function that enables computational
and picture of a landscape (D) with a blue sky, green leaves, and
reconstruction of captured images with a single
multicolor flowers. The appropriately cropped original object
digital filter. This work connects computational
imaging and metasurface optics and patterns used for imaging are shown in the left column. Scale
demonstrates the capabilities of combining these bars, 20 μm.m.
disciplines by simultaneously reducing
aberrations and downsizing imaging
systems using simpler optics.
Metalenses for imaging #2
Metalens-Based Miniaturized Optical Systems Micromachines 2019, 10(5), 310; https://doi.org/10.3390/mi10050310

Metasurfaces have been studied and widely applied to optical systems. A metasurface-based flat lens
(metalens) holds promise in wave-front engineering for multiple applications. The metalens has become a
breakthrough technology for miniaturized optical system development, due to its outstanding
characteristics, such as ultrathinness and cost-effectiveness. Compared to conventional macro- or
meso-scale optics manufacturing methods, the micro-machining process for metalenses is relatively
straightforward and more suitable for mass production. Due to their remarkable abilities and superior
optical performance, metalenses in refractive or diffractive mode could potentially replace traditional
optics. In this review, we give a brief overview of the most recent studies on metalenses and their
applications with a specific focus on miniaturized optical imaging and sensing systems. We discuss
approaches for overcoming technical challenges in the bio-optics field, including a large field of view
(FOV), chromatic aberration, and high-resolution imaging.
Metalenses for imaging #3
Imaging with flat optics: metalenses or diffractive lenses? https://arxiv.org/abs/1901.05042 (2019)

Recently, there has been an explosion of

interest in metalenses for imaging. The
interest is primarily based on their sub-
wavelength thicknesses. Diffractive lenses
have been used as thin lenses since the late
19th century. Here, we show that multi-level
diffractive lenses (MDLs), when
designed properly can exceed the
performance of metalenses. Furthermore,
MDLs can be designed and fabricated with
larger constituent features, making them
accessible to low-cost, large area volume
manufacturing, which is generally Thirdly, we point out that the fabrication
challenging for metalenses. The support complexity of metalenses is far higher than
substrate will dominate overall thickness for all those of MDLs. As can be seen in Table 1 (columns
flat optics. Therefore the advantage of a slight 4 and 7), the minimum feature widths required for
Bending of light via (a) refraction and (b) diffraction. metalenses are significantly smaller than those for
decrease in thickness (from ~2λ to ~λ/2)
Schematic of the constituent element of a (c) conventional MDLs. In addition, metalenses generally
afforded by metalenses may not be binary diffractive lens, (d) multi-level diffractive lens (MDL) and require high-index materials, whereas MDLs
useful. We further elaborate on the differences (e) metalens. Photographs of a broadband visible MDL can be fabricated in low-index polymers. It is
between these approaches and clarify that fabricated in a polymer film on a glass substrate are shown in important to appreciate that any transparent
metalenses have unique advantages (f) side-view emphasizing the small thickness, which is material can be used for the MDL. This allows
when manipulating the polarization dominated by the substrate and (g) front view. Recently, MDLs to be mass manufactured at low cost via
states of light. metalenses high-volume imprinting techniques
Metamaterials for Augmented Reality (AR)
Metasurface Optics for Ultra-Compact Augmented Reality
(AR) Visors https://doi.org/10.1364/CLEO_QELS.2019.FTh3M.4
Elyas Bayati, Shane Colburn, and Arka Majumdar
Metasurface eyepiece for augmented reality
Gun-Yeal Lee, Jong-Young Hong, SoonHyoung Hwang, Seokil Moon,
Hyeokjung Kang, Sohee Jeon, Hwi Kim, Jun-Ho Jeong & Byoungho Lee
School of Electrical and Computer Engineering, Seoul National University
https://doi.org/10.1038/s41467-018-07011-5
Photonics World | 27 Apr 2018 | https://optics.org/news/9/4/46
SPIE Photonics Europe: Harvard student tops
AR/VR design challenge Zhujun Shi awarded top
score for her presentation on metamaterial-based
waveguide displays. Shi, a member of Federico Capasso’s
research group at Harvard, claimed the top score in the
competition, for her presentation on the potential for
metamaterials to improve the AR/VR experience.
Entitled “Wide field-of-view waveguide displays
enabled by polarization-dependent metagratings”,
Shi’s presentation showed how components based on the
new class of optical materials can deliver high-quality images
- alongside possibilities for additional degrees of freedom in
polarization manipulation, and a smaller form factor than
traditional liquid crystal lenses.
Metamaterials for Automotive Lidar
https://www.laserfocusworld.com/optics/article/1
4036818/metasurface-beam-steering-enables-s
olidstate-highperformance-lidar
To be useful, automotive lidar requires the repeated capture of an entire 3D scene, rather than just a point or
line; to accomplish this, a lidar system can either be flash-based (relying on a single, spatially diffused laser pulse
and time-of-flight imaging optics) or based on beam-scanning. While beam-scanning greatly simplifies the
receiving optoelectronics, it requires a method of moving a beam in 2D densely, accurately, and reliably.
Beam-scanning lidar systems used a mechanical beam scanner -- which comes with all the potential problems
of mechanical systems: complexity, wear and resulting inaccuracy, and mechanical failure -- while newer
breeds of lidar sensors utilize MEMS mirrors or optical phased arrays. However, both of these recent
approaches lack performance due to the small optical aperture of MEMS mirrors and the low efficiency of
phased arrays.
Mass-manufacturable high-performance lidar
Now, Lumotive (Seattle, WA), a Bill Gates-funded startup developing lidar systems for autonomous vehicles, has
unveiled its own nonmechanical beam-steering technology based on a reflective optical
metasurface combined with a liquid crystal layer. The silicon-based metasurface can be manufactured
using standard CMOS technology, enabling straightforward mass-production of the unit. The Lumotive system is
high-performance: its large (25 x 25 mm) optical aperture delivers long range, a 120-degree field-of-view with
high angular resolution, and fast random-access beamsteering, according to Lumotive.
"Lumotive's solution is ideal for automakers and Tier-1s seeking safer yet more cost-effective perception
solutions for their vehicles," says Lumotive cofounder and CEO William Colleran. "Our lidar sensors benefit
tremendously from the unique attributes of beam-steering LCMs [Liquid Crystal Metasurfaces, the company's
brand name for the system] which simultaneously offer large optical aperture, wide field of view, and fast
scanning while having no moving parts. LCMs deliver the combination of performance and commercial
viability that will finally eliminate barriers to adoption of lidar for both ADAS [Advanced Driver Assistance
Systems] and autonomous vehicles."
Nonplanar wavefront “extension” to Metaoptics
Metasurface optic for Gaussian We have proposed a design method for
beams accepts a nonplanar input metasurfaces that can manipulate
beam Most metasurface optics only nonplanar wavefronts. As an example, we
accept plane-wave beams; a new hybrid have designed a Gaussian reflective
metal-dielectric metastructure focuses a metasurface, i.e., the metasurface equivalent of
diverging Gaussian beam back on itself. a concave spherical mirror. We demonstrated
that such metasurfaces can produce a
reflected cross-polarized beam that perfectly
John Wallace Aug 13th, 2019
https://www.laserfocusworld.com/optics/article/1403800 overlaps with the incident field. Our optimized
1/metasurface-optic-for-gaussian-beams-accepts-a-non designs have power efficiencies over 90%,
planar-input-beam
obtained by using TiO2 nanofins as
A Gaussian reflective metasurface for metasurface building block placed on an Al
back mirror. Such a metasurface building block
advanced wavefront manipulation
Jade Martínez-Llinàs, Clément Henry, Daniel Andrén,
ensures minimal losses, together with a
Ruggero Verre, Mikael Käll, and Philippe Tassin homogeneous and complete phase coverage.
Optics Express Vol. 27, Issue 15, pp. 21069-21082 (2019)
https://doi.org/10.1364/OE.27.021069
We envision several applications for
Concept and design of the reflective metasurface: (a) Spherical concave mirror reflective metasurfaces like the ones
(left) and metasurface with the same behavior as a concave mirror (right) presented in this article. For example, they
reflecting an incoming beam. (b) Gaussian reflective metasurface reflecting a could be used to realize ultracompact
Gaussian beam back onto itself. (c) Top view of the metasurface designed to
reflect a circularly polarized Gaussian beam. All the nanofins have the same reflective mirrors and confocal
dimensions and their angle of rotation changes with the square of their distance microcavities or could be used as
from the center. (d) Unit cell consisting of a nanofin on top of a substrate. The
efficiency of the metasurface can be optimized by adjusting the height (t), length
mirrors in optical tweezers and other
(L), and width (w) of the nanofins and the unit cell size (P). The 2π phase nanophotonic devices.
coverage can be obtained by varying the angle of rotation (θ).).
Stacking metalenses
Overlaid optical metalenses have new properties
Overlaying two film layers patterned with a nanoscale array can manipulate the propagation of
light to create a capable ultrathin lens. John Wallace Aug 1st, 2019
One challenge in the development of multifunctional metalenses is their Multifocal metalens based on multilayer
limited efficiency. One possible way to improve this is to stack the Pancharatnam–Berry phase elements architecture
metalens. By doing this, graduate student Ronghui Lin and his supervisor, Ronghui Lin and Xiaohang Li
Xiaohang Li, discovered, at least in simulations, that new phenomena can be Optics Letters Vol. 44, Issue 11, pp. 2819-2822 (2019)
enabled when one metalens is laid on top of another. https://doi.org/10.1364/OL.44.002819

The team looked at a metalens with a surface covered in an array of fins or

cylinders with an elliptical cross section. By varying the relative orientation of
these fins, the lens can add a geometric phase to incoming circularly
polarized light. Lin and Li used finite-difference time-domain (FDTD)
simulations to model light propagation in a metalens system comprising two
stacked phase elements. Their results showed that by twisting the
relative alignment of the two layers, a phenomenon similar to a Moiré
effect can be observed; the team used this phenomenon to develop a
bifocal metalens with controllable focal length and intensity ratio.
“We believe this multilayer metalens architecture could also apply to other
systems and achieve more complicated functionalities,” says Lin.
Metamaterials for Spectral Light Engines
Metamaterial light mixer Arbitrary spectral matching using multi-LED
lighting systems
generates 11 wavelengths Aleix Llenas; Josep Carreras
simultaneously 03 Jul 2018 Optical Engineering, 58(3), 035105 (2019).
https://doi.org/10.1117/1.OE.58.3.035105
Sandia Lab's source mixes two laser pulses of NIR to
produce range of eleven distinct outputs from NIR, through
visible to UV. https://optics.org/news/9/7/5 A simple yet counterintuitive optical feedback
controller for spectrally tunable lighting systems
https://doi.org/10.1038/s41467-018-04944-9 Aleix Llenas; Josep Carreras
Optical Engineering, 58(7), 075104 (2019).
https://doi.org/10.1117/1.OE.58.7.075104

 SuperK SELECT is a tunable multi-

wavelength filter based on Acousto-Optic
Tunable Filter technology (AOTF)
https://www.nktphotonics.com/lasers-fibers/product/super
k-select-multi-line-tunable-filter/

Supercontinuum White Light laser

 second-hamonic generation (SHG), third-harmonic generation (THG), fourth-harmonic
https://www.nktphotonics.com/lasers-fibers/product/super
generation (FHG), sum-frequency generation (SFG), two-photon absorption-induced k-extreme-fianium-supercontinuum-lasers/
photoluminescence (TPA PL), four-wave mixing (FWM) and six-wave mixing (SWM)
Metamaterials for “Large-scale” Optics
Large area metalenses: design,
characterization, and mass
manufacturing
Alan She, Shuyan Zhang, Samuel Shian, David R. Clarke, and Federico
Capasso John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138
Optics Express Vol. 26, Issue 2, pp. 1573-1585 (2018)
https://doi.org/10.1364/OE.26.001573 - Cited by 32 - Related articles
Optical components, such as lenses, have traditionally been made in the bulk
form by shaping glass or other transparent materials. Recent advances in
metasurfaces provide a new basis for recasting optical components
into thin, planar elements, having similar or better performance using arrays
of subwavelength-spaced optical phase-shifters. The technology required to
mass produce them dates back to the mid-1990s, when the feature sizes of
semiconductor manufacturing became considerably denser than the wavelength
of light, advancing in stride with Moore’s law. This provides the possibility of
unifying two industries: semiconductor manufacturing and lens-
making, whereby the same technology used to make computer chips is used to
make optical components, such as lenses, based on metasurfaces. Using a
scalable metasurface layout compression algorithm that exponentially reduces
design file sizes (by 3 orders of magnitude for a centimeter diameter lens) and
stepper photolithography, we show the design and fabrication of
metasurface lenses (metalenses) with extremely large areas, up to
centimeters in diameter and beyond. Using a single two-centimeter
diameter near-infrared metalens less than a micron thick fabricated in this way, we
experimentally implement the ideal thin lens equation, while demonstrating high-
quality imaging and diffraction-limited focusing.
Flatness requirements. (a) As the metalens size
increases, the flatness over the device surface
becomes increasingly relevant. The flatness
requirements were studied by computer simulations of
metalenses (diameter: 2 cm, focal length: 50 mm, design
wavelength: 1550 nm), in which the surface curvature was Focusing and imaging performance. (a) The thinness of the device
varied (spatial period ranging from 1 to 100 mm, and allows for imaging setups very similar to the ideal thin lens equation, which
perturbation amplitude from 0 to 200 μm.m). The intensity of was used to demonstrate imaging capabilities. (b) Image of focal spot with 7
the optical field is shown. The metalens is situated at the mm gaussian illumination at λ = 1550 nm. = 1550 nm. (c) The measured modulation
bottom of each plot, and the vertical (z) and horizontal (r) transfer function (MTF) from (b) is plotted with the theoretical diffraction-
axes are the propagation direction and radial dimension, limited MTF. Error bars: standard deviation. (d) Chromatic focal shift as a
respectively. Each group of four columns on the left and right function of the wavelength of illumination. The measured deviation of focal
show the optical behavior for even (cosine) and odd (sine) length from that of the design wavelength at 1550 nm (light blue dots, error
bars: standard deviation) is plotted together with the linear fit (blue line). (e)
spatial perturbations, respectively, with respect to the
Hyperspectral image of focal spot in the same configuration as (b) for λ = 1550 nm. =
metalens center. Quality of focus is reduced for shorter 1440-1590 nm in 10 nm increments linearly binned to RGB channels (center
spatial periods and higher perturbation amplitudes. In (b), the
surface profile of the 4-inch wafer we used (including
wavelengths λ = 1550 nm. R = 1590, λ = 1550 nm. G = 1515, and λ = 1550 nm. B = 1480 nm). The spot, which is
largely white, indicates little chromatic aberration, which can be attributed to
metalenses) was measured (using Toho FLX-2320-S) and the low NA (0.07). Horizontal and vertical line cuts at the RGB center
the Fourier transform calculated in (c) to obtain the major wavelengths are also shown. Using the thin lens setup in (a), simple, single-
contributions to spatial frequencies, which mainly occurred at lens imaging was demonstrated at λ = 1550 nm. = 1550 nm for (f) the Harvard university
Λ−1 < 0.02 mm−1 (or Λ > 20 mm). The inner and outer white −1 < 0.02 mm−1 (or Λ−1 < 0.02 mm−1 (or Λ > 20 mm). The inner and outer white > 20 mm). The inner and outer white logo and (g) US Air Force 1951 resolution target, without any additional
circles denote Λ−1 < 0.02 mm−1 (or Λ > 20 mm). The inner and outer white at 50 and 20 mm, respectively. optical components.
Deep Learning for Metamaterial Design
Data-Based Design Method for
Metamaterials Uses Artificial
Intelligence

POHANG, South Korea, July 19, 2019 — The

process for designing metamaterials could be
improved by using data-driven approaches based
on deep learning, according to researchers at
Pohang University of Science and Technology
(POSTECH). 

The group led by professor Junsuk Rho taught an

artificial neural network to recognize the
correlation between the extinction spectra
of electric and magnetic dipoles and core-
shell nanoparticle designs that included
material information and shell thicknesses. The AI
system allowed greater freedom of design by “So, I’d like to take our findings a step further by
categorizing types of materials and adding
material types as a design factor, making it developing a complete design method of metamaterials
possible to design materials to meet
specific optical requirements. An analysis of utilizing AI. Also, I’d like to make innovative and
the metamaterials obtained through this design
method showed that they exhibited optical practical metamaterials by training AI with reviews of the
properties that were identical to the properties
that were input into the artificial neural network. design constructed in consideration of final products.” 
https://www.photonics.com/Articles/Data-Based
_Design_Method_for_Metamaterials_Uses/a6492 ACS Applied Materials & Interfaces https://pubs.acs.org/doi/10.1021/acsami.9b05857
5
Metamaterials for photonic ICs
 Research
Labs
Both in academia
and in industry
Academia | UK
●
https://www.ucl.ac.uk/institute-for
-materials-discovery/research/na
nomaterials/metamaterials
●
https://www.ee.ucl.ac.uk/~npanoiu
/ERC.html
●
https://www.ee.ucl.ac.uk/~olegm/
pon/
●
https://www.london-nano.com/re
search-categories/metamaterials
-plasmonics-non-linear-optics
●
https://www.imperial.ac.uk/natural
-sciences/research/engineering-
novel-solutions/metamaterials/
●
https://www.np.phy.cam.ac.uk/pe
ople/research-themes/metamate
rials
●
https://www.computing.co.uk/ctg/
news/3073842/new-technique-t
o-print-3d-materials-with-optical-
or-microwave-properties-develop
ed
Academia | USA

https://share-ng.sandia.gov/news/resources/news_releases/optical_metamaterials/

https://www.seas.harvard.edu/capasso/research/plasmonics/

Sandia National Laboratories has created the first inverse-design software Mirage
for optical metamaterials — meaning users start by describing the result they want, and
the software fills in the steps to get there. The modern design approach takes guesswork
out of engineering as-yet theoretical technologies like ultracompact, high-performance
cameras and cloaking armor that could make wearers invisible to detection.
Academia | South Korea
29 Nov 2017 https://optics.org/news/8/11/44

Korean-UK group makes “credit card-thick” metasurface lenses from graphene

and gold, to focus terahertz beams. Thick, flat lenses with tunable features based on graphene and
gold have been developed by a partnership of Korean- and UK-based researchers. They say that such optical devices
“could become optical components for advanced applications, such as amplitude-tunable lenses, lasers (so-called
http://photonics.postech.ac.kr/, Pohang vortex phase plates), and dynamic holography.”

Rho's research group is not only developing a new concept of novel optical The scientists work at the Center for Integrated Nanostructure Physics, in the Institute for Basic Science, the 
nanomaterials having extraordinary and unprecedented eletromagnetic properties Korea Advanced Institute of Science and Technology and the University of Birmingham. The work has been published
based on fundamental physics and experimental studies of deep sub-wavelength light- in Advanced Optical Materials.
matter interaction, but also realizing engineering device applications including, but
not limited to, super-resolution imaging, negative index materials, tunable high refractive
index metasurface, highly efficient light trapping device, ultra-sensitive biomedical
sensor, nanoscale laser, and next generation scalable nanomanufacturing
methodologies.

http://oeqelab.snu.ac.kr/
“Using metalenses, you can make microscopes, cameras, and tools used in very sensitive optical
●
Augmented reality near-eye display using Pancharatnam-Berry phase lenses Scientific
reports 9 (1), 6616, 2019 measurements, much more compact,” clarifies Teun-Teun Kim, lead author of the study. The
●
Metasurface Holograms and Metalenses for AR/VR Digital Holography and Three-
metalenses were designed specifically for terahertz radiation. This radiation can pass through some
Dimensional Imaging, Tu3A. 1 2019 materials such as fabrics and plastics, but at a shorter depth than microwave radiation. For this reason it is
●
Metasurface eyepiece for augmented reality Nature communications 9 (1), 4562. 2018 employed for surveillance and security screening. Kim added, “While conventional optical lenses have a
thickness of several centimeters to several millimeters, this metalens is just a few tens of
micrometers thick. The intensity of the focused light can be effectively controlled and it could find useful
applications in ultra-small optical instruments.”
 Academia | China
Metamaterial Lensing Devices Machine-learning reprogrammable metasurface imager
Molecules 2019, 24(13), 2460 https://doi.org/10.1038/s41467-019-09103-2 06 March 2019
College of Information Science and Engineering, Northeastern University, Shenyang 110004, China
College of Information & Control Engineering, Shenyang Jianzhu University, Shenyang ●
State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, 100871, Beijing, China

https://doi.org/10.3390/molecules24132460 ●
Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore, 117583, Singapore
●
State Key Laboratory of Millimeter Waves, Southeast University, 210096, Nanjing, China

Metamaterials and Metasurfaces

Special Issue: Metamaterials and Metasurfaces
State Key Laboratory of Optical Technologies on Nano‐Fabrication and Micro‐Engineering, Fabrication and Micro‐Fabrication and Micro‐Engineering, Engineering,
Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, 610209 China
18 July 2019 https://doi.org/10.1002/adom.201900885

Working principle of the real-time digital-metasurface microwave reprogrammable imager. a The

proposed machine-learning metasurface imager can be optimized for different kinds of scenes. The
optimization is performed by training the manifold representing the metasurface configuration (the surfaces)
to be close to the training data (scatters). b The illustration of training the reprogrammable imager by using the
Nanjing University, China
http://congress2019.metamorphose-vi.org/index.php/component/svisor/?ta PCA method, in which the training person is Hengxin Ruan (coauthor). c The map of coding metasurface and
sk=showScheduleSvisor&Itemid=156 the unit cell. d The illustration of real-time reprogrammable metasurface imager imaging a moving person
behind a wall.
Industry Labs Samsung
https://www.sait.samsung.co.kr/saithome/research/metaphotonics.do
Tools from Industry PlanOpSim
 The Problem
with current fundus
snap-ons

Image quality too

poor with the current
snapons
Short intro Smartphone fundus snap-ons not the most active field

https://dx.doi.org/10.1155/2015/823139

Optical accessory for a mobile device WO EP ES IT PL 

Patent EP3068284B1 Andrea Russo D-EYE S.r.l.
D-Eye/Peek nice size, but crappy images
Enter Metamaterials
Alan She et al. (February 2018)
http://doi.org/10.1126/sciadv.aap9957
https://www.eurekalert.org/pub_releases/2018-02
/hjap-rcm022118.php

Let 1.5 years to pass until the “mainstream tech”

media finds this and never even link the original
article in their “buzz article”

https://www.d-eyecare.com/

Igor Bonifacic, 
@igorbonifacic
21 Aug 2019

https://www.engadget.com/2019/08
/21/harvard-metalens-better-than-hu
man-eye/

https://youtu.be/KsBFhVTZkfc
Remidio has already quite nice offering

quality retinal imaging system which Remidio Innovative Solutions Pvt. Ltd
does not require pupil dilation for the pupil sizes Bangalore, Karnataka, India
above 3.3 mm. https://remidio.com/
The “High-End” could be cheaper and more portable
Smartphone-Based Retinal Camera Outperforms
Tabletop System (TOPCON TRC-50DX) in Detecting Diabetic
Retinopathy At one-fifth the cost, Remidio Fundus-on-Phone
(FOP) promises dramatic public health impact. PR Newswire 21
Nov 2018.
https://www.mpo-mag.com/contents/view_breaking-news/2018-11-21/smartphone-ba
sed-retinal-camera-outperforms-tabletop-system-in-detecting-diabetic-retinopathy

Sensitivity and Specificity of Smartphone-Based

Retinal Imaging for Diabetic Retinopathy
Sabyasachi Sengupta et al. (Feb 2019)
https://doi.org/10.1016/j.oret.2018.09.016

Can the Nexy really track

eye “fast enough” with its
Optomed Aurora traditional motors (VCM?),
novel faster autotracking
https://www.optomed.com/designcases-aurorasalescase Optomed Aurora
introduces a totally new concept in fundus imaging where high quality
imaging meets ease of use. Optomed Aurora is the first and only hand-
held fundus camera in the market with 50 degrees field of view,
featuring a large 4” color screen and a totally new state-of-the-art user
actuators needed
interface with simple menus and icons. Compact size makes
Optomed Aurora an ideal tool for clinics of all sizes for screening of
(MEMS?)
various eye diseases, such as diabetic retinopathy, glaucoma and
AMD.
https://www.nextsight.info/
Optical Designs for fundus cameras, see:

https://www.slideshare.net/PetteriTeikariPhD/practical-consideration
s-in-the-design-of-embedded-ophthalmic-devices-100648266

https://www.slideshare.net/PetteriTeikariPhD/optical-designs-for-fundus-cameras https://www.slideshare.net/PetteriTeikariPhD/hyperspectral-retinal-imaging
 Specs Recap
What would you like
to have at least if you
started designing a
fundus camera for
clinical use
Overview
BASICS
●
Image the retina without mydriasis
●
Image through
opaque ocular media (lens
yellowing, cataracts), as most of
the eye patients are older and
have these issues
●
Should be able to take images
automatically, (non-”20D imaging
”, which probably drives the
operator crazy.
●
Field-of-view at least 45° so you
can sell it as “NHS compliant”
diabetic retinopathy screening
fundus camera.
EXTRAS / Nice-to-Haves
●
Rather wide field of-view, e.g. 90° FOV on mobile phone
●
Separate illumination and imaging paths (like done Wang et al. 2018 in their
trans-pars-planar design?
If adaptive optics lenses (e.g. ICFO’s SmartLens; or Dynamic Optics Zhang et al. 2018) become more
affordable , then they would be easier to integrate into existing devices with the paths
separated.
●
And then you want throw some deep learning to the device itself to have an
image restoration model to correct for the scattering/PSF of the ocular
media.
●
The mechanical enclosure could be as small as possible (think of D-Eye,
Peek Retina), with mechanical attachments to VR-like headset visors for
stability.
●
Can one do fundus images easily freehand on ophthalmologist’s waiting room, rural areas,
etc., and if one wants to do video recording (e.g. when assessing dynamic retinal vessel
response to flicker, or just for multiframe reconstruction), does one get extra stability with
the headset form factor?
Overview Example “Path Split”
In practice you need “pars plana”
real-time tracker? The optical
window for arc-shaped illumination
pattern not the biggest in practice Have your tunable lenses / adaptive lenses on the imaging path
without affecting lighting distribution, intensity, spectrum, etc.

Illumination not
through the pupil

Quite broadband / polychromatic

the illumination used

https://www.dropbox.com/s/k7hso3y5hm7s41d/fundus_opticalDesigns.pdf?dl=0
Don’t “be the engineer” Think of how fundus cameras are actually used

Saccadio Comprehensive Eye Examination

https://www.youtube.com/watch?v=ntEKT-C_km4
http://www.saccadio.com/

Virtual clinics in ophthalmology:

The Glaucoma Monitoring Service at Moorfields Eye Hospital
by Aachal Kotecha PhD
https://www.kingsfund.org.uk/sites/default/files/media/Aachal_Kotecha.pdf
AI Needs to Be Validated Your “committee-of-expert” of ground truths,
and generalizability of your model to out-of-distribution (OOD) samples

A Hierarchical Probabilistic U-Net for Modeling

Multi-Scale Ambiguities Simon A. A. Kohl, 
Bernardino Romera-Paredes, Klaus H. Maier-Hein, Danilo Jimenez Rezende
, S. M. Ali Eslami, Pushmeet Kohli, Andrew Zisserman, Olaf Ronneberger
(Submitted on 30 May 2019) https://arxiv.org/abs/1905.13077

Michael F. Chiang at APTOS 2018 https://2018.asiateleophth.org/

Knowles Professor of Ophthalmology & Medical Informatics and Clinical
Epidemiology, Oregon Health

Dealing with inter-expert variability in retinopathy of prematurity: A

machine learning approach V.Bolón-Canedo, E.Ataer-Cansizoglu, D.Erdogmus,
J. Kalpathy-Cramer, O. Fontenla-Romero, A.Alonso-Betanzosa, M.F.Chiang
https://doi.org/10.1016/j.cmpb.2015.06.004 (2015)
AI Needs to Be Validated Your “committee-of-expert” of ground truths,
and generalizability of your model to out-of-distribution (OOD) samples

https://doi.org/10.1016/j.ophtha.2017.08.046

Researchers developing machine learning algorithms

generally split the available data into separate sets to be
used for training, testing, and validation. However, all data
may be drawn from a single site or a narrow range of sites,
and the results may not generalize to other
populations or to all imaging devices and protocols. We
argue that before clinical deployment is considered,
an independent study, or one with independent oversight http://doi.org/10.1001/jama.2019.10306
(like clinical drug trials), should be required of sufficient size
to be confident about detecting clinically important but less Machine learning can identify patterns in the expanding, heterogeneous data sets to
common events. Also, should learning algorithms be create models that accurately classify a patient’s diagnosis or predict what a patient
allowed to continue to learn after independent approval may experience in the future. However, realizing the potential benefit of machine
when exposed to real-life data, which may alter their learning for patients in the form of better care requires rethinking how model
performance possibly for the better, but in an unchecked performance during machine learning is assessed. A framework for
way? Regulatory authorities will need to adapt to these rigorously evaluating the performance of a model in the context of the subsequent
new challenges. actions it triggers is necessary to identify models that are clinically useful.
Hardware only
can only get you so
far.

Need to integrate
computational
imaging (i.e. deep
learning image
restoration)
Multiframe (video) Reconstruction of the fundus
Imagine the worst case: Trying to get a sharp image with EDVR: Video Restoration
with Enhanced Deformable Convolutional
this 20D handheld
https://www.instructables.com/id/Retinal-Imaging-Device-OphthalmicDocs-Fundus/ Networks Xintao Wang et al. (2019)
http://www.odocs-tech.com/ https://xinntao.github.io/projects/EDVR

Wouldn’t it just be nice to record a video of your struggle to get fundus in

focus, properly illuminated, patient having eye open, etc. and let the
algorithm construct you the best possible image from all of the frames?
Computational Correction for Opaque Medium
On the use of deep learning for computational imaging
George Barbastathis, Aydogan Ozcan, and Guohai Situ (2019)
https://doi.org/10.1364/OPTICA.6.000921
Imaging through diffuse media [280,281] is a classical challenging inverse
problem with significant practical applications ranging from non-invasive
medical imaging through tissue to autonomous navigation of vehicles in foggy
conditions.
Three-Dimensional Cataract Crystalline Lens Imaging With Swept-
Source Optical Coherence Tomography
Alberto de Castro; Antonio Benito; Silvestre Manzanera; Juan Mompeán; Belén
Cañizares; David Martínez; Jose María Marín; Ireneusz Grulkowski; Pablo Artal (2019)
http://doi.org/10.1167/iovs.17-23596
A dedicated swept-source optical coherence tomography (SS-OCT) is a useful tool
to study in vivo the subtle opacities in the cataractous crystalline lens, revealing its
position and size three-dimensionally. The use of these images allows obtaining more
detailed information on the age-related changes leading to cataract.

Learning-based imaging through optically thick diffusers using a fully connected

DNN with MAE TLF (after [299], Fig. 3; reprinted with permission). First row:
speckle patterns input to the DNN; second row: corresponding ground-truth
images selected among a database of English letters; third row: corresponding
reconstructions by the DNN.
Computational Correction for Turbid Medium
https://doi.org/10.1364/OE.26.030162

DNN-based aberration correction in a wavefront sensorless adaptive

Four groups of results in 200 repetitions compensating for random phase-masks. (a)
Four groups of the point spread functions before (left) and after (right) correction at
optics system Qinghua Tian, Chenda Lu, Bo Liu, Lei Zhu, Xiaolong Pan, Qi Zhang, Leijing
the focal plane gained by CMOS camera and the intensity is normalized. (b) The Yang, Feng Tian, and Xiangjun Xin
comparison of Zernike coefficient amplitudes between phase-mask (in blue bars)
and reconstructed phase (in orange bars). (c) is the reconstructed phase pattern Optics Express Vol. 27, Issue 8, pp. 10765-10776 (2019) •https://doi.org/10.1364/OE.27.010765
loaded on SLM for each group. (d) The intensity profile at the center section of Airy
(ideal spot), NO AO (without AO corrected point spread function), T-T corr (tip-tilt
corrected point spread function) and ML-AO (machine learning guided AO corrected
point spread function) of four groups mentioned in (a).
Put all to a “deep fundusCameraISP”
CameraNet: A Two-Stage Framework for Effective Camera image signal processing (ISP) Learning
Zhetong Liang, Jianrui Cai, Zisheng Cao, and Lei Zhang https://arxiv.org/abs/1908.01481 August 2019
Enter “VR Headset” fundus platform
Open Indirect The Team:
Ophthalmoscope (OIO) Dhruv Joshi: Team Coordinator
+ Deep Learning

An open-source, ultra-low cost, portable

Sandeep Vempati: Engineering
screening device for retinal diseases Lead

https://hackaday.io/project/11943-op Tristan Swedish: Original Design

en-indirect-ophthalmoscope + Deep Learning + Optical
Improvements

Diabetic Retinopathy is a complication of Devesh Jain: Original Design

diabetes causing damage to the retina, eventually Dr Nag Konda

leading to blindness. The cost of state of the art
retinal imaging devices required for identifying this Sankalp Modi
disorder lies in the range $10,000 - $25,000. This
Ebin Philip
makes them inaccessible for the population in rural
areas or developing countries. We aim to develop a Ayush Yadav
device under $400 which can provide reasonable
quality retinal images to clinicians. Based around Raspberry Pi 3 + Pi IR Aamir Jowher

camera, and WaveShare 5 inch Nish Mohith Kurukuti

OIO(OWL) is an idea conceived in the Fall 2015 MIT Touchscreen LCD for Raspberry Pi
Media Lab Engineering Health Course. Deepika Dixit
Development of the electronics, enclosure, and optics
have continued since Spring 2016 at the Srujana
Innovation Centre at the LV Prasad Eye Institute,
Hyderabad, India.
A bit longer and more structured review

Embedded deep learning in ophthalmology: making ophthalmic imaging smarter

https://doi.org/10.1177/2515841419827172
https://arxiv.org/abs/1810.05874