J. Semicond. > 2021, Volume 42 > Issue 1 > 013105
|

REVIEWS

Neuromorphic vision sensors: Principle, progress and perspectives

Fuyou Liao1, 2, Feichi Zhou2 and Yang Chai1, 2,

+ Author Affiliations

 Corresponding author: Yang Chai, e-mail: ychai@polyu.edu.hk

DOI: 10.1088/1674-4926/42/1/013105

PDF

Turn off MathJax

Abstract: Conventional frame-based image sensors suffer greatly from high energy consumption and latency. Mimicking neurobiological structures and functionalities of the retina provides a promising way to build a neuromorphic vision sensor with highly efficient image processing. In this review article, we will start with a brief introduction to explain the working mechanism and the challenges of conventional frame-based image sensors, and introduce the structure and functions of biological retina. In the main section, we will overview recent developments in neuromorphic vision sensors, including the silicon retina based on conventional Si CMOS digital technologies, and the neuromorphic vision sensors with the implementation of emerging devices. Finally, we will provide a brief outline of the prospects and outlook for the development of this field.

Key words: image sensors, silicon retina, neuromorphic vision sensors, photonic synapses



[1]
McCulloch W S, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys, 1943, 5(4), 115 doi: 10.1007/BF02478259
[2]
Hebb D O. The organization of behavior: a neuropsychological theory. J Wiley; Chapman & Hall, 1949
[3]
Carver M. Analog VLSI and neural systems. Addison-Wesley, 1989
[4]
Maher M A C, Deweerth S P, Mahowald M A, et al. Implementing neural architectures using analog VLSI circuits. IEEE Trans Circuits Syst, 1989, 36(5), 643 doi: 10.1109/31.31311
[5]
Mead C. Neuromorphic electronic systems. Proc IEEE, 1990, 78(10), 1629 doi: 10.1109/5.58356
[6]
Hodgkin A L, Huxley A F. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol, 1952, 117(4), 500 doi: 10.1113/jphysiol.1952.sp004764
[7]
Merolla P A, Arthur J V, Alvarez-Icaza R, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 2014, 345(6197), 668 doi: 10.1126/science.1254642
[8]
Zidan M A, Strachan J P, Lu W D. The future of electronics based on memristive systems. Nat Electron, 2018, 1(1), 22 doi: 10.1038/s41928-017-0006-8
[9]
Wang C Y, Wang C, Meng F, et al. 2D layered materials for memristive and neuromorphic applications. Adv Electron Mater, 2019, 6(2), 1901107 doi: 10.1002/aelm.201901107
[10]
Tuma T, Pantazi A, Le Gallo M, et al. Stochastic phase-change neurons. Nat Nanotechnol, 2016, 11(8), 693 doi: 10.1038/nnano.2016.70
[11]
Kuzum D, Jeyasingh R G D, Lee B, et al. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett, 2012, 12(5), 2179 doi: 10.1021/nl201040y
[12]
Yang C S, Shang D S, Liu N, et al. A synaptic transistor based on quasi-2D molybdenum oxide. Adv Mater, 2017, 29(27), 1700906 doi: 10.1002/adma.201700906
[13]
Tian H, Guo Q, Xie Y, et al. Anisotropic black phosphorus synaptic device for neuromorphic applications. Adv Mater, 2016, 28(25), 4991 doi: 10.1002/adma.201600166
[14]
Takeo O, Tsuyoshi H, Tohru T, et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat Mater, 2011, 10(8), 591 doi: 10.1038/nmat3054
[15]
Posch C, Serrano-Gotarredona T, Linares-Barranco B, et al. Retinomorphic event-based vision sensors: bioinspired cameras with spiking output. Proc IEEE, 2014, 102(10), 1470 doi: 10.1109/JPROC.2014.2346153
[16]
Steffen L, Reichard D, Weinland J, et al. Neuromorphic stereo vision: A survey of bio-inspired sensors and algorithms. Front Neurorobot, 2019, 13, 28 doi: 10.3389/fnbot.2019.00028
[17]
Zhou F, Zhou Z, Chen J, et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat Nanotechnol, 2019, 14(8), 776 doi: 10.1038/s41565-019-0501-3
[18]
Mennel L, Symonowicz J, Wachter S, et al. Ultrafast machine vision with 2D material neural network image sensors. Nature, 2020, 579(7797), 62 doi: 10.1038/s41586-020-2038-x
[19]
Wang C Y, Liang S J, Wang S, et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci Adv, 2020, 6(26), eaba6173 doi: 10.1126/sciadv.aba6173
[20]
Seo S, Jo S H, Kim S, et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat Commun, 2018, 9(1), 1 doi: 10.1038/s41467-017-02088-w
[21]
Tian H, Wang X, Wu F, et al. High performance 2D perovskite/ graphene optical synapses as artificial eyes. 2018 IEEE International Electron Devices Meeting (IEDM), 2018, 38.6.1
[22]
Choi C, Choi M K, Liu S, et al. Human eye-inspired soft optoelectronic device using high-density MoS2-graphene curved image sensor array. Nat Commun, 2017, 8(1), 1 doi: 10.1038/s41467-016-0009-6
[23]
Gu L, Poddar S, Lin Y, et al. A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature, 2020, 581(7808), 278 doi: 10.1038/s41586-020-2285-x
[24]
Boyle W S, Smith G E. Charge coupled semiconductor devices. Bell Syst Tech J, 1970, 49(4), 587 doi: 10.1002/j.1538-7305.1970.tb01790.x
[25]
Theuwissen A J. Solid-state imaging with charge-coupled devices. Springer Science & Business Media, 2006
[26]
Taylor S A. CCD and CMOS imaging array technologies: technology review. UK: Xerox Research Centre Europe, 1998
[27]
Bigas M, Cabruja E, Forest J, et al. Review of CMOS image sensors. Microelectron J, 2006, 37(5), 433 doi: 10.1016/j.mejo.2005.07.002
[28]
Kim Y, Chortos A, Xu W, et al. A bioinspired flexible organic artificial afferent nerve. Science, 2018, 360(6392), 998 doi: 10.1126/science.aao0098
[29]
Lee G J, Choi C, Kim D H, et al. Bioinspired artificial eyes: Optic components, digital cameras, and visual prostheses. Adv Funct Mater, 2017, 28(24), 1705202 doi: 10.1002/adfm.201705202
[30]
Song Y M, Xie Y, Malyarchuk V, et al. Digital cameras with designs inspired by the arthropod eye. Nature, 2013, 497(7447), 95 doi: 10.1038/nature12083
[31]
Jeong K H, Kim J, Lee L P. Biologically inspired artificial compound eyes. Science, 2006, 312(5773), 557 doi: 10.1126/science.1123053
[32]
Ko H C, Stoykovich M P, Song J, et al. A hemispherical electronic eye camera based on compressible silicon optoelectronics. Nature, 2008, 454(7205), 748 doi: 10.1038/nature07113
[33]
Posch C. Bio-inspired vision. J Instru, 2012, 7(01), C01054 doi: 10.1088/1748-0221/7/01/C01054
[34]
Gollisch T, Meister M. Eye smarter than scientists believed: Neural computations in circuits of the retina. Neuron, 2010, 65(2), 150 doi: 10.1016/j.neuron.2009.12.009
[35]
Masland R H. The fundamental plan of the retina. Nat Neurosci, 2001, 4(9), 877 doi: 10.1038/nn0901-877
[36]
Rodieck R W, Rodieck R W. The first steps in seeing. Sunderland, MA: Sinauer Associates, 1998
[37]
Cho D D, Lee T. A review of bioinspired vision sensors and their applications. Sens Mater, 2015, 27(6), 447 doi: 10.18494/SAM.2015.1083
[38]
Pereda A E. Electrical synapses and their functional interactions with chemical synapses. Nat Rev Neurosci, 2014, 15(4), 250 doi: 10.1038/nrn3708
[39]
Tan Z H, Yang R, Terabe K, et al. Synaptic metaplasticity realized in oxide memristive devices. Adv Mater, 2016, 28(2), 377 doi: 10.1002/adma.201503575
[40]
Yu H, Gong J, Wei H, et al. Mixed-halide perovskite for ultrasensitive two-terminal artificial synaptic devices. Mater Chem Front, 2019, 3(5), 941 doi: 10.1039/C9QM00061E
[41]
Etienne-Cummings R, Van der Spiegel J. Neuromorphic vision sensors. Sens Actuators A, 1996, 56(1/2), 19 doi: 10.1016/0924-4247(96)01277-0
[42]
Indiveri G, Douglas R. Neuromorphic vision sensors. Science, 2000, 288(5469), 1189 doi: 10.1126/science.288.5469.1189
[43]
Moini A. Vision chips. Springer Science & Business Media, 1999
[44]
Lichtsteiner P, Posch C, Delbruck T. A 128 × 128 120 db 30 mW asynchronous vision sensor that responds to relative intensity change. 2006 IEEE International Solid State Circuits Conference – Digest of Technical Papers, 2006, 2060
[45]
Lichtsteiner P, Posch C, Delbruck T. A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J Solid-State Circuits, 2008, 43(2), 566 doi: 10.1109/JSSC.2007.914337
[46]
Leñero-Bardallo J A, Serrano-Gotarredona T, Linares-Barranco B. A 3.6 μs latency asynchronous frame-free event-driven dynamic-vision-sensor. IEEE J Solid-State Circuits, 2011, 46(6), 1443 doi: 10.1109/JSSC.2011.2118490
[47]
Posch C, Matolin D, Wohlgenannt R. A QVGA 143 dB dynamic range frame-free pwm image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J Solid-State Circuits, 2011, 46(1), 259 doi: 10.1109/JSSC.2010.2085952
[48]
Posch C, Matolin D, Wohlgenannt R. A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression. 2010 IEEE International Solid-State Circuits Conference (ISSCC), 2010, 400
[49]
Berner R, Brandli C, Yang M, et al. A 240 × 180 10 mW 12 μs latency sparse-output vision sensor for mobile applications. 2013 Symposium on VLSI Circuits, 2013, C186
[50]
Brandli C, Berner R, Yang M, et al. A 240 × 180 130 dB 3 μs latency global shutter spatiotemporal vision sensor. IEEE J Solid-State Circuits, 2014, 49(10), 2333 doi: 10.1109/JSSC.2014.2342715
[51]
Lichtsteiner P, Delbruck T. A 64 × 64 AER logarithmic temporal derivative silicon retina. Research in Microelectronics and Electronics, 2005 PhD, 2005, 2, 202 doi: 10.1109/RME.2005.1542972
[52]
Zhou F, Liu Y, Shen X, et al. Low-voltage, optoelectronic CH3NH3PbI3– xCl x memory with integrated sensing and logic operations. Adv Funct Mater, 2018, 28(15), 1800080 doi: 10.1002/adfm.201800080
[53]
Lei S, Wen F, Li B, et al. Optoelectronic memory using two-dimensional materials. Nano Lett, 2015, 15(1), 259 doi: 10.1021/nl503505f
[54]
Lee J, Pak S, Lee Y W, et al. Monolayer optical memory cells based on artificial trap-mediated charge storage and release. Nat Commu, 2017, 8(1), 14734 doi: 10.1038/ncomms14734
[55]
Lee D, Hwang E, Lee Y, et al. Multibit MoS2 photoelectronic memory with ultrahigh sensitivity. Adv Mater, 2016, 28(41), 9196 doi: 10.1002/adma.201603571
[56]
Lipatov A, Sharma P, Gruverman A, et al. Optoelectrical molybdenum disulfide (MoS2)-ferroelectric memories. ACS Nano, 2015, 9(8), 8089 doi: 10.1021/acsnano.5b02078
[57]
Roy K, Padmanabhan M, Goswami S, et al. Graphene–MoS2 hybrid structures for multifunctional photoresponsive memory devices. Nat Nanotechnol, 2013, 8(11), 826 doi: 10.1038/nnano.2013.206
[58]
Xiang D, Liu T, Xu J, et al. Two-dimensional multibit optoelectronic memory with broadband spectrum distinction. Nat Commun, 2018, 9(1), 2966 doi: 10.1038/s41467-018-05397-w
[59]
Qin S, Wang F, Liu Y, et al. A light-stimulated synaptic device based on graphene hybrid phototransistor. 2D Mater, 2017, 4(3), 035022 doi: 10.1088/2053-1583/aa805e
[60]
Ni Z, Wang Y, Liu L, et al. Hybrid structure of silicon nanocrystals and 2D WSe2 for broadband optoelectronic synaptic devices. 2018 IEEE International Electron Devices Meeting (IEDM), 2018, 38.5.1
[61]
Wang W, Panin G N, Fu X, et al. MoS2 memristor with photoresistive switching. Sci Rep, 2016, 6(1), 31224 doi: 10.1038/srep31224
[62]
Tran M D, Kim H, Kim J S, et al. Two-terminal multibit optical memory via van der Waals heterostructure. Adv Mater, 2019, 31(7), 1807075 doi: 10.1002/adma.201807075
[63]
Hu L, Yuan J, Ren Y, et al. Phosphorene/ZnO nano-heterojunctions for broadband photonic nonvolatile memory applications. Adv Mater, 2018, 30(30), 1801232 doi: 10.1002/adma.201801232
[64]
Zhou F, Chen J, Tao X, et al. 2D materials based optoelectronic memory: Convergence of electronic memory and optical sensor. Research, 2019, 2019, 9490413 doi: 10.34133/2019/9490413
[65]
Chai Y. In-sensor computing for machine vision. Nature, 2020, 579, 32 doi: 10.1038/d41586-020-00592-6
[66]
Zhang K, Jung Y H, Mikael S, et al. Origami silicon optoelectronics for hemispherical electronic eye systems. Nat Commun, 2017, 8(1), 1782 doi: 10.1038/s41467-017-01926-1
[67]
Laughlin S B, van Steveninck R R R, Anderson J C. The metabolic cost of neural information. Nat Neurosci, 1998, 1(1), 36 doi: 10.1038/236
Fig. 1.  (Color online) Schematic diagram of (a) the composition of human visual system, (b) multilayer structure of human retina, and (c) a biological synapse.

DownLoad: Full-Size Img PowerPoint
Fig. 2.  (Color online) (a) Abstracted pixel schematic of DVS. (b) Principle of operation[45]. (c) The response of a DVS array to a person moving in the field of view of the sensor. (d) A DVS array is observing a 500 Hz spiral on an analog oscilloscope. (e) The DVS output is a continuous sequence of address events (x, y) in time. Red and blue events represent an increase or decrease change of light, respectively[15].

DownLoad: Full-Size Img PowerPoint
Fig. 3.  (Color online) (a) Abstracted pixel schematic of ATIS pixel. (b) The principle of operation of two types of asynchronous AER events. (c) Change detection events recorded (upper) and gray-level updates at the corresponding pixel positions (below)[15, 16].

DownLoad: Full-Size Img PowerPoint
Fig. 4.  (Color online) The circuit and output of DAVIS vision sensor. (a) The pixel circuit of DAVIS pixel combines an APS with a DVS. (b) A snapshot from DAVIS sensors illustrating a captured APS frame in gray scale with the DVS events in color. The football was flying toward the person. Inset: 5 ms of output right after the frame capture of the football. (c) Space-time 3D view of DVS events during 40 ms of a white rectangle spinning on a black disk at 100 Hz. Green events are older and red events are newer[49].

DownLoad: Full-Size Img PowerPoint
Fig. 5.  (Color online) Neuromorphic vision sensors based on ORRAM. (a) I–V characteristics of ORRAM with optical set and electrical reset. Inset, schematic structure of the MoOx ORRAM and its cross-section scanning electron microscopy (SEM) image. Scale bar, 100 nm. (b) Light-tunable synaptic characteristics under light intensity of 0.22, 0.45, 0.65 and 0.88 mW/cm2, respectively, with a pulse width of 200 ms. (c) Illustrations of the image memory function of ORRAM array. The letter F was stimulated with a light intensity of 0.88 mW/cm2. (d) Images before (left columns) and after (right columns) ORRAM image sensor pre-processing. (e) The image recognition rate with and without ORRAM image preprocessing[17].

DownLoad: Full-Size Img PowerPoint
Fig. 6.  (Color online) NN vision sensors. (a) Schematic of the 2D Perovskite/Graphene optical synaptic device[21]. (b) Schematic of an artificial optic-neural synapse device based on h-BN/WSe2 heterostructure[20]. (c) Optical image of WSe2/h-BN/Al2O3 vdW heterostructure based device (left) and its structural diagram (right)[19]. (d) Optical microscope image of the photodiode array consisting of 3 × 3 pixels. The upper right: Schematic of a WSe2 photodiode. The bottom right: SEM image of the pixel. (e) Schematics of the classifier. (f) Schematics of the autoencoder[18].

DownLoad: Full-Size Img PowerPoint
Fig. 7.  (Color online) A hemispherical retina based on perovskite nanowire array and its properties. (a) Side view of a completed EC-EYE. (b) The structure diagram of the EC-EYE. (c) Photocurrent and responsivity depend on light intensity of a perovskite nanowire photoreceptor. (d) I–V characteristics and the response of individual pixels. (e) The comparison of field of view (FOV) of the planar and hemispherical image sensors. (f) The reconstructed letter ‘A’ image of EC-EYE and its projection on a flat plane[23].

DownLoad: Full-Size Img PowerPoint

Table 1.   Comparison of three representative silicon retina.

ParameterDVS[45, 46]ATIS[47, 48]DAVIS[49, 50]
Major functionAsynchronous temporal contrast event detectionDVS + Intensity measurement for each eventDVS + Synchronous imaging
Noise2.1%0.25%0.4% APS, 3.5% DVS
Pixel complexity26 transistors, 3 caps, 1 photodiode77 transistors, 3 caps, 2 photodiodes47 transistors, 3 caps,
1 photodiode
Power consumption (mW)2450–1755–14
Resolution128 × 128304 × 240240 × 180
Pixel size (μm2)40 × 4030 × 3018.5 × 18.5
Latency (μs)1543
Dynamic range120 dB125 dB130 dB DVS, 51 dB APS
Date of publication200820112013
ApplicationDynamic scenesSurveillanceDynamic scenes
DownLoad: CSV

Table 2.   Comparisons of neuromorphic vision sensors based on emerging devices.

Neuromorphic vision sensorsDevice structureTerminal numberLight wavelength (nm)Array sizeFunctionsRef.
ORAM vision sensorsWSe2/BN FETThree405–6383 × 9Multibit optoelectronic memory/broadband spectrum distinction[58]
CuIn7Se11 FETThree5433 pixelsOptoelectronic memory[53]
Pd/MoOx/ITOTwo3658 × 8Contrast enhancement/noise reduction[17]
NN vision sensorsGra./2D perovskite/
gra. FET		Three520–High photo-responsivity/high stability/pattern recognition[21]
WSe2/h-BN FETThree405, 532, 655–Colored and color-mixed pattern recognition/ultra-low power consumption[20]
WSe2 dual-gate FETFour65027 pixelsUltrafast recognition and encoding[18]
WSe2/h-BN/Al2O3 FETThree–8 × 8Pattern recognition/edge enhancement/contrast correction[19]
Hemispherically shaped vision sensorsSilicon photodiodesTwo620–70016 × 16Hemispherical electronic eye cameras/arbitrary curvilinear shapes[32]
Silicon photodiodesTwo5328 × 8hemispherical shapes/FOV(140–180°)[30]
MoS2/graphene heterostructure FETThree515>12 × 12High-density array design/small optical aberration/simplified optics[22]
Silicon-based lateral P–i–N photodiodesTwo543, 594, 633676 pixelsHemisphere-like structures[66]
Ionic liquid/Perovskite nanowire/liquid-metalTwoSunlight10 × 10High responsivity /reasonable response speed/low detection limit/wide FOV[23]
DownLoad: CSV
[1]
McCulloch W S, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys, 1943, 5(4), 115 doi: 10.1007/BF02478259
[2]
Hebb D O. The organization of behavior: a neuropsychological theory. J Wiley; Chapman & Hall, 1949
[3]
Carver M. Analog VLSI and neural systems. Addison-Wesley, 1989
[4]
Maher M A C, Deweerth S P, Mahowald M A, et al. Implementing neural architectures using analog VLSI circuits. IEEE Trans Circuits Syst, 1989, 36(5), 643 doi: 10.1109/31.31311
[5]
Mead C. Neuromorphic electronic systems. Proc IEEE, 1990, 78(10), 1629 doi: 10.1109/5.58356
[6]
Hodgkin A L, Huxley A F. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol, 1952, 117(4), 500 doi: 10.1113/jphysiol.1952.sp004764
[7]
Merolla P A, Arthur J V, Alvarez-Icaza R, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 2014, 345(6197), 668 doi: 10.1126/science.1254642
[8]
Zidan M A, Strachan J P, Lu W D. The future of electronics based on memristive systems. Nat Electron, 2018, 1(1), 22 doi: 10.1038/s41928-017-0006-8
[9]
Wang C Y, Wang C, Meng F, et al. 2D layered materials for memristive and neuromorphic applications. Adv Electron Mater, 2019, 6(2), 1901107 doi: 10.1002/aelm.201901107
[10]
Tuma T, Pantazi A, Le Gallo M, et al. Stochastic phase-change neurons. Nat Nanotechnol, 2016, 11(8), 693 doi: 10.1038/nnano.2016.70
[11]
Kuzum D, Jeyasingh R G D, Lee B, et al. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett, 2012, 12(5), 2179 doi: 10.1021/nl201040y
[12]
Yang C S, Shang D S, Liu N, et al. A synaptic transistor based on quasi-2D molybdenum oxide. Adv Mater, 2017, 29(27), 1700906 doi: 10.1002/adma.201700906
[13]
Tian H, Guo Q, Xie Y, et al. Anisotropic black phosphorus synaptic device for neuromorphic applications. Adv Mater, 2016, 28(25), 4991 doi: 10.1002/adma.201600166
[14]
Takeo O, Tsuyoshi H, Tohru T, et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat Mater, 2011, 10(8), 591 doi: 10.1038/nmat3054
[15]
Posch C, Serrano-Gotarredona T, Linares-Barranco B, et al. Retinomorphic event-based vision sensors: bioinspired cameras with spiking output. Proc IEEE, 2014, 102(10), 1470 doi: 10.1109/JPROC.2014.2346153
[16]
Steffen L, Reichard D, Weinland J, et al. Neuromorphic stereo vision: A survey of bio-inspired sensors and algorithms. Front Neurorobot, 2019, 13, 28 doi: 10.3389/fnbot.2019.00028
[17]
Zhou F, Zhou Z, Chen J, et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat Nanotechnol, 2019, 14(8), 776 doi: 10.1038/s41565-019-0501-3
[18]
Mennel L, Symonowicz J, Wachter S, et al. Ultrafast machine vision with 2D material neural network image sensors. Nature, 2020, 579(7797), 62 doi: 10.1038/s41586-020-2038-x
[19]
Wang C Y, Liang S J, Wang S, et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci Adv, 2020, 6(26), eaba6173 doi: 10.1126/sciadv.aba6173
[20]
Seo S, Jo S H, Kim S, et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat Commun, 2018, 9(1), 1 doi: 10.1038/s41467-017-02088-w
[21]
Tian H, Wang X, Wu F, et al. High performance 2D perovskite/ graphene optical synapses as artificial eyes. 2018 IEEE International Electron Devices Meeting (IEDM), 2018, 38.6.1
[22]
Choi C, Choi M K, Liu S, et al. Human eye-inspired soft optoelectronic device using high-density MoS2-graphene curved image sensor array. Nat Commun, 2017, 8(1), 1 doi: 10.1038/s41467-016-0009-6
[23]
Gu L, Poddar S, Lin Y, et al. A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature, 2020, 581(7808), 278 doi: 10.1038/s41586-020-2285-x
[24]
Boyle W S, Smith G E. Charge coupled semiconductor devices. Bell Syst Tech J, 1970, 49(4), 587 doi: 10.1002/j.1538-7305.1970.tb01790.x
[25]
Theuwissen A J. Solid-state imaging with charge-coupled devices. Springer Science & Business Media, 2006
[26]
Taylor S A. CCD and CMOS imaging array technologies: technology review. UK: Xerox Research Centre Europe, 1998
[27]
Bigas M, Cabruja E, Forest J, et al. Review of CMOS image sensors. Microelectron J, 2006, 37(5), 433 doi: 10.1016/j.mejo.2005.07.002
[28]
Kim Y, Chortos A, Xu W, et al. A bioinspired flexible organic artificial afferent nerve. Science, 2018, 360(6392), 998 doi: 10.1126/science.aao0098
[29]
Lee G J, Choi C, Kim D H, et al. Bioinspired artificial eyes: Optic components, digital cameras, and visual prostheses. Adv Funct Mater, 2017, 28(24), 1705202 doi: 10.1002/adfm.201705202
[30]
Song Y M, Xie Y, Malyarchuk V, et al. Digital cameras with designs inspired by the arthropod eye. Nature, 2013, 497(7447), 95 doi: 10.1038/nature12083
[31]
Jeong K H, Kim J, Lee L P. Biologically inspired artificial compound eyes. Science, 2006, 312(5773), 557 doi: 10.1126/science.1123053
[32]
Ko H C, Stoykovich M P, Song J, et al. A hemispherical electronic eye camera based on compressible silicon optoelectronics. Nature, 2008, 454(7205), 748 doi: 10.1038/nature07113
[33]
Posch C. Bio-inspired vision. J Instru, 2012, 7(01), C01054 doi: 10.1088/1748-0221/7/01/C01054
[34]
Gollisch T, Meister M. Eye smarter than scientists believed: Neural computations in circuits of the retina. Neuron, 2010, 65(2), 150 doi: 10.1016/j.neuron.2009.12.009
[35]
Masland R H. The fundamental plan of the retina. Nat Neurosci, 2001, 4(9), 877 doi: 10.1038/nn0901-877
[36]
Rodieck R W, Rodieck R W. The first steps in seeing. Sunderland, MA: Sinauer Associates, 1998
[37]
Cho D D, Lee T. A review of bioinspired vision sensors and their applications. Sens Mater, 2015, 27(6), 447 doi: 10.18494/SAM.2015.1083
[38]
Pereda A E. Electrical synapses and their functional interactions with chemical synapses. Nat Rev Neurosci, 2014, 15(4), 250 doi: 10.1038/nrn3708
[39]
Tan Z H, Yang R, Terabe K, et al. Synaptic metaplasticity realized in oxide memristive devices. Adv Mater, 2016, 28(2), 377 doi: 10.1002/adma.201503575
[40]
Yu H, Gong J, Wei H, et al. Mixed-halide perovskite for ultrasensitive two-terminal artificial synaptic devices. Mater Chem Front, 2019, 3(5), 941 doi: 10.1039/C9QM00061E
[41]
Etienne-Cummings R, Van der Spiegel J. Neuromorphic vision sensors. Sens Actuators A, 1996, 56(1/2), 19 doi: 10.1016/0924-4247(96)01277-0
[42]
Indiveri G, Douglas R. Neuromorphic vision sensors. Science, 2000, 288(5469), 1189 doi: 10.1126/science.288.5469.1189
[43]
Moini A. Vision chips. Springer Science & Business Media, 1999
[44]
Lichtsteiner P, Posch C, Delbruck T. A 128 × 128 120 db 30 mW asynchronous vision sensor that responds to relative intensity change. 2006 IEEE International Solid State Circuits Conference – Digest of Technical Papers, 2006, 2060
[45]
Lichtsteiner P, Posch C, Delbruck T. A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J Solid-State Circuits, 2008, 43(2), 566 doi: 10.1109/JSSC.2007.914337
[46]
Leñero-Bardallo J A, Serrano-Gotarredona T, Linares-Barranco B. A 3.6 μs latency asynchronous frame-free event-driven dynamic-vision-sensor. IEEE J Solid-State Circuits, 2011, 46(6), 1443 doi: 10.1109/JSSC.2011.2118490
[47]
Posch C, Matolin D, Wohlgenannt R. A QVGA 143 dB dynamic range frame-free pwm image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J Solid-State Circuits, 2011, 46(1), 259 doi: 10.1109/JSSC.2010.2085952
[48]
Posch C, Matolin D, Wohlgenannt R. A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression. 2010 IEEE International Solid-State Circuits Conference (ISSCC), 2010, 400
[49]
Berner R, Brandli C, Yang M, et al. A 240 × 180 10 mW 12 μs latency sparse-output vision sensor for mobile applications. 2013 Symposium on VLSI Circuits, 2013, C186
[50]
Brandli C, Berner R, Yang M, et al. A 240 × 180 130 dB 3 μs latency global shutter spatiotemporal vision sensor. IEEE J Solid-State Circuits, 2014, 49(10), 2333 doi: 10.1109/JSSC.2014.2342715
[51]
Lichtsteiner P, Delbruck T. A 64 × 64 AER logarithmic temporal derivative silicon retina. Research in Microelectronics and Electronics, 2005 PhD, 2005, 2, 202 doi: 10.1109/RME.2005.1542972
[52]
Zhou F, Liu Y, Shen X, et al. Low-voltage, optoelectronic CH3NH3PbI3– xCl x memory with integrated sensing and logic operations. Adv Funct Mater, 2018, 28(15), 1800080 doi: 10.1002/adfm.201800080
[53]
Lei S, Wen F, Li B, et al. Optoelectronic memory using two-dimensional materials. Nano Lett, 2015, 15(1), 259 doi: 10.1021/nl503505f
[54]
Lee J, Pak S, Lee Y W, et al. Monolayer optical memory cells based on artificial trap-mediated charge storage and release. Nat Commu, 2017, 8(1), 14734 doi: 10.1038/ncomms14734
[55]
Lee D, Hwang E, Lee Y, et al. Multibit MoS2 photoelectronic memory with ultrahigh sensitivity. Adv Mater, 2016, 28(41), 9196 doi: 10.1002/adma.201603571
[56]
Lipatov A, Sharma P, Gruverman A, et al. Optoelectrical molybdenum disulfide (MoS2)-ferroelectric memories. ACS Nano, 2015, 9(8), 8089 doi: 10.1021/acsnano.5b02078
[57]
Roy K, Padmanabhan M, Goswami S, et al. Graphene–MoS2 hybrid structures for multifunctional photoresponsive memory devices. Nat Nanotechnol, 2013, 8(11), 826 doi: 10.1038/nnano.2013.206
[58]
Xiang D, Liu T, Xu J, et al. Two-dimensional multibit optoelectronic memory with broadband spectrum distinction. Nat Commun, 2018, 9(1), 2966 doi: 10.1038/s41467-018-05397-w
[59]
Qin S, Wang F, Liu Y, et al. A light-stimulated synaptic device based on graphene hybrid phototransistor. 2D Mater, 2017, 4(3), 035022 doi: 10.1088/2053-1583/aa805e
[60]
Ni Z, Wang Y, Liu L, et al. Hybrid structure of silicon nanocrystals and 2D WSe2 for broadband optoelectronic synaptic devices. 2018 IEEE International Electron Devices Meeting (IEDM), 2018, 38.5.1
[61]
Wang W, Panin G N, Fu X, et al. MoS2 memristor with photoresistive switching. Sci Rep, 2016, 6(1), 31224 doi: 10.1038/srep31224
[62]
Tran M D, Kim H, Kim J S, et al. Two-terminal multibit optical memory via van der Waals heterostructure. Adv Mater, 2019, 31(7), 1807075 doi: 10.1002/adma.201807075
[63]
Hu L, Yuan J, Ren Y, et al. Phosphorene/ZnO nano-heterojunctions for broadband photonic nonvolatile memory applications. Adv Mater, 2018, 30(30), 1801232 doi: 10.1002/adma.201801232
[64]
Zhou F, Chen J, Tao X, et al. 2D materials based optoelectronic memory: Convergence of electronic memory and optical sensor. Research, 2019, 2019, 9490413 doi: 10.34133/2019/9490413
[65]
Chai Y. In-sensor computing for machine vision. Nature, 2020, 579, 32 doi: 10.1038/d41586-020-00592-6
[66]
Zhang K, Jung Y H, Mikael S, et al. Origami silicon optoelectronics for hemispherical electronic eye systems. Nat Commun, 2017, 8(1), 1782 doi: 10.1038/s41467-017-01926-1
[67]
Laughlin S B, van Steveninck R R R, Anderson J C. The metabolic cost of neural information. Nat Neurosci, 1998, 1(1), 36 doi: 10.1038/236

    • Search

    Advanced Search >>

    GET CITATION

    shu

    Export: BibTex EndNote

    Article Metrics

    Article views: 8238 Times PDF downloads: 700 Times Cited by: 0 Times

    History

    Received: 05 June 2020 Revised: 09 July 2020 Online: Accepted Manuscript: 08 September 2020Uncorrected proof: 11 September 2020Published: 09 January 2021

    Catalog

      Email This Article

      User name:
      Email:*请输入正确邮箱
      Code:*验证码错误
      Send
      Article Navigation >  Journal of Semiconductors  > 2021  >  42(1): 013105
      Fuyou Liao, Feichi Zhou, Yang Chai. Neuromorphic vision sensors: Principle, progress and perspectives[J]. Journal of Semiconductors, 2021, 42(1): 013105. doi: 10.1088/1674-4926/42/1/013105 ****F Y Liao, F C Zhou, Y Chai, Neuromorphic vision sensors: Principle, progress and perspectives[J]. J. Semicond., 2021, 42(1): 013105. doi: 10.1088/1674-4926/42/1/013105.
      Citation:
      Fuyou Liao, Feichi Zhou, Yang Chai. Neuromorphic vision sensors: Principle, progress and perspectives[J]. Journal of Semiconductors, 2021, 42(1): 013105. doi: 10.1088/1674-4926/42/1/013105 ****
      F Y Liao, F C Zhou, Y Chai, Neuromorphic vision sensors: Principle, progress and perspectives[J]. J. Semicond., 2021, 42(1): 013105. doi: 10.1088/1674-4926/42/1/013105.
      shu

      Neuromorphic vision sensors: Principle, progress and perspectives

      DOI: 10.1088/1674-4926/42/1/013105
      • 1.

        The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen 518057, China

      • 2.

        Department of Applied Physics, The Hong Kong Polytechnic University, Hong Kong, China

      More Information
      • Fuyou Liao：received his Ph.D. degree in 2020 at Fudan University. Now he is a postdoctoral researcher at The Hong Kong Polytechnic University Shenzhen Research Institute. He has published papers in reputed journals, such as Nano Research, Small, ACS Applied Electronic Materials, etc. His research interest is two-dimensional materials-based electronic and optoelectronic devices
      • Yang Chai：is an Associate Professor at the Hong Kong Polytechnic University, the vice president of Physical Society of Hong Kong, and a member of The Hong Kong Young Academy of Sciences. He has published over 100 papers, including Nature, Nature Nanotechnology, Nature Electronics, etc. His current research interest is low-dimensional material for electron device application
      • Corresponding author: e-mail: ychai@polyu.edu.hk
      • Received Date: 2020-06-05
      • Revised Date: 2020-07-09
      • Published Date: 2021-01-10

      Catalog

        Journal of Semiconductors © 2017 All Rights Reserved   京ICP备05085259号-2

        /

        DownLoad:  Full-Size Img  PowerPoint
        Return
        Return