J. Semicond. > 2024, Volume 45 > Issue 9 > 092401
|

ARTICLES

Multiframe-integrated, in-sensor computing using persistent photoconductivity

Xiaoyong Jiang1, Minrui Ye4, Yunhai Li3, Xiao Fu2, 3, , Tangxin Li2, Qixiao Zhao2, Jinjin Wang2, Tao Zhang4, Jinshui Miao2 and Zengguang Cheng1,

+ Author Affiliations

 Corresponding author: Xiao Fu, xiaofu@mail.sitp.ac.cn; Zengguang Cheng, zgcheng@fudan.edu.cn

DOI: 10.1088/1674-4926/24040002

PDF

Turn off MathJax

Abstract: The utilization of processing capabilities within the detector holds significant promise in addressing energy consumption and latency challenges. Especially in the context of dynamic motion recognition tasks, where substantial data transfers are necessitated by the generation of extensive information and the need for frame-by-frame analysis. Herein, we present a novel approach for dynamic motion recognition, leveraging a spatial-temporal in-sensor computing system rooted in multiframe integration by employing photodetector. Our approach introduced a retinomorphic MoS2 photodetector device for motion detection and analysis. The device enables the generation of informative final states, nonlinearly embedding both past and present frames. Subsequent multiply-accumulate (MAC) calculations are efficiently performed as the classifier. When evaluating our devices for target detection and direction classification, we achieved an impressive recognition accuracy of 93.5%. By eliminating the need for frame-by-frame analysis, our system not only achieves high precision but also facilitates energy-efficient in-sensor computing.

Key words: in-sensor, MoS2, photodetector, persistent photoconductivity, reservoir computing



[1]
Tan H W, van Dijken S. Dynamic machine vision with retinomorphic photomemristor-reservoir computing. Nat Commun, 2023, 14, 2169 doi: 10.1038/s41467-023-37886-y
[2]
Liu Y Q, Liu D, Gao C S, et al. Self-powered high-sensitivity all-in-one vertical tribo-transistor device for multi-sensing-memory-computing. Nat Commun, 2022, 13, 7917 doi: 10.1038/s41467-022-35628-0
[3]
Jiang C P, Xu H H, Yang L, et al. Neuromorphic antennal sensory system. Nat Commun, 2024, 15, 2109 doi: 10.1038/s41467-024-46393-7
[4]
Zhou F C, Chai Y. Near-sensor and in-sensor computing. Nat Electron, 2020, 3, 664 doi: 10.1038/s41928-020-00501-9
[5]
Mennel L, Symonowicz J, Wachter S, et al. Ultrafast machine vision with 2D material neural network image sensors. Nature, 2020, 579, 62 doi: 10.1038/s41586-020-2038-x
[6]
Jang H, Hinton H, Jung W B, et al. In-sensor optoelectronic computing using electrostatically doped silicon. Nat Electron, 2022, 5, 519 doi: 10.1038/s41928-022-00819-6
[7]
Lopez-Sanchez O, Lembke D, Kayci M, et al. Ultrasensitive photodetectors based on monolayer MoS2. Nat Nanotechnol, 2013, 8, 497 doi: 10.1038/nnano.2013.100
[8]
Jayachandran D, Oberoi A, Sebastian A, et al. A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector. Nat Electron, 2020, 3, 646 doi: 10.1038/s41928-020-00466-9
[9]
Zhang Z F, Zhao X L, Zhang X M, et al. In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array. Nat Commun, 2022, 13, 6590 doi: 10.1038/s41467-022-34230-8
[10]
Zhou Y, Fu J W, Chen Z R, et al. Computational event-driven vision sensors for in-sensor spiking neural networks. Nat Electron, 2023, 6, 870 doi: 10.1038/s41928-023-01055-2
[11]
Xiang D, Liu T. Extending in-sensor computing from static images to dynamic motions. Nat Electron, 2023, 6, 801 doi: 10.1038/s41928-023-01070-3
[12]
Appeltant L, Soriano M C, Van der Sande G, et al. Information processing using a single dynamical node as complex system. Nat Commun, 2011, 2, 468 doi: 10.1038/ncomms1476
[13]
Lee H, Lee S, Kim J, et al. Stretchable array electromyography sensor with graph neural network for static and dynamic gestures recognition system. NPJ Flex Electron, 2023, 7, 20 doi: 10.1038/s41528-023-00246-3
[14]
Chen J W, Zhou Z, Kim B J, et al. Optoelectronic graded neurons for bioinspired in-sensor motion perception. Nat Nanotechnol, 2023, 18, 882 doi: 10.1038/s41565-023-01379-2
[15]
Liao F Y, Zhou Z, Kim B J, et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat Electron, 2022, 5, 84 doi: 10.1038/s41928-022-00713-1
[16]
Wu Y C, Liu C H, Chen S Y, et al. Extrinsic origin of persistent photoconductivity in monolayer MoS2 field effect transistors. Sci Rep, 2015, 5, 11472 doi: 10.1038/srep11472
[17]
George A, Fistul M V, Gruenewald M, et al. Giant persistent photoconductivity in monolayer MoS2 field-effect transistors. NPJ 2D Mater Appl, 2021, 5, 15 doi: 10.1038/s41699-020-00182-0
[18]
Van der Sande G, Brunner D, Soriano M C. Advances in photonic reservoir computing. Nanophotonics, 2017, 6, 561 doi: 10.1515/nanoph-2016-0132
[19]
Zhong Y N, Tang J S, Li X Y, et al. Dynamic memristor-based reservoir computing for high-efficiency temporal signal processing. Nat Commun, 2021, 12, 408 doi: 10.1038/s41467-020-20692-1
[20]
Li H, Zhang Q, Yap C C R, et al. From bulk to monolayer MoS2: Evolution of Raman scattering. Adv Funct Materials, 2012, 22, 1385 doi: 10.1002/adfm.201102111
[21]
Choudhary N, Park J, Hwang J Y, et al. Growth of large-scale and thickness-modulated MoS2 nanosheets. ACS Appl Mater Interfaces, 2014, 6, 21215 doi: 10.1021/am506198b
[22]
Chakraborty B, Matte H S S R, Sood A K, et al. Layer-dependent resonant Raman scattering of a few layer MoS2. J Raman Spectrosc, 2013, 44, 92 doi: 10.1002/jrs.4147
[23]
Brown N M D, Cui N Y, McKinley A. An XPS study of the surface modification of natural MoS2 following treatment in an RF-oxygen plasma. Appl Surf Sci, 1998, 134, 11 doi: 10.1016/S0169-4332(98)00252-9
[24]
Jadwiszczak J, O’Callaghan C, Zhou Y B, et al. Oxide-mediated recovery of field-effect mobility in plasma-treated MoS2. Sci Adv, 2018, 4, eaao5031 doi: 10.1126/sciadv.aao5031
[25]
Liu K Q, Zhang T, Dang B J, et al. An optoelectronic synapse based on α-In2Se3 with controllable temporal dynamics for multimode and multiscale reservoir computing. Nat Electron, 2022, 5, 761 doi: 10.1038/s41928-022-00847-2
Fig. 1.  (Color online) (a) Schematic of traditional frame-by-frame detecting system. Detector genetrates output for subsequent computing in every single frame. (b) Schematic of multiframe-integrated, in-sensor computing using persistent photoconductivity effect. Detector continuously detects multiple frames and only generates one final output state for analysis, which already memorizes the information of past and current frames, spatially and temporally. The final state is input to the subsequent linear classifier, serving as the readout layer.

DownLoad: Full-Size Img PowerPoint
Fig. 2.  (Color online) (a) Schematic image of retinomorphic MoS2 photodetector. (b) Scanning electron microscopy (SEM) imaging (left) and wavelength-dispersive X-ray spectroscopy (WDS) imaging (sulfur element and molybdenum element image are present in red and green, respectively) of retinomorphic MoS2 photodetector, Scarbar, 50 μm. (c) Raman spectra of retinomorphic MoS2 photodetector. The inset shows the transmission electron microscopy (TEM) images of cross-sectional MoS2 flake, Scarbar, 10 nm. (d) X-ray photoelectron spectroscopy (XPS) of the retinomorphic MoS2 photodetector. The inset shows the optical image of a 1 cm × 1 cm retinomorphic MoS2 photodetector array.

DownLoad: Full-Size Img PowerPoint
Fig. 3.  (Color online) (a) I−V characterization of retinomorphic MoS2 photodetector in logarithmic scale. The inset shows the I−V curve on a linear scale. (b) The persistent photoconductivity effects observed in retinomorphic MoS2 photodetector illuminated under laser pulses (520 nm, 10 mW). Pink rectangle: light on; blue rectangle: light off. (Insert: Photocurrent of Au/MoS2/Au device were measured under illumination by light pulses with different power (λ = 520 nm, 3, 5, 10, 12, 14 mW laser power)). (c) 3-bit light pulse inputs ranging from "000" to "111", each with a pulse width and interval of 100 and 900 ms were used. (d) The resultant normalized photocurrent characteristics, including input−output feature extraction, were analyzed using a retinomorphic MoS2 photodetector.

DownLoad: Full-Size Img PowerPoint
Fig. 4.  (Color online) (a) Schematic of the mission proposed, target can move in two directions (clockwise/anticlockwise), we use light pulse irradiated onto detector array (8 × 8 pixels in our simulation) to represent the location of target. (b) Schematic of four heatmaps of photocurrent of all detectors after every single frame, the green one refers to clockwise, the blue one refers to anticlockwise, darker a pixel is, later a light pulse appears here. (c) Evolution of the accuracy rates based on multiframe-integrated RC system and traditional FC network within 100 epochs.

DownLoad: Full-Size Img PowerPoint
[1]
Tan H W, van Dijken S. Dynamic machine vision with retinomorphic photomemristor-reservoir computing. Nat Commun, 2023, 14, 2169 doi: 10.1038/s41467-023-37886-y
[2]
Liu Y Q, Liu D, Gao C S, et al. Self-powered high-sensitivity all-in-one vertical tribo-transistor device for multi-sensing-memory-computing. Nat Commun, 2022, 13, 7917 doi: 10.1038/s41467-022-35628-0
[3]
Jiang C P, Xu H H, Yang L, et al. Neuromorphic antennal sensory system. Nat Commun, 2024, 15, 2109 doi: 10.1038/s41467-024-46393-7
[4]
Zhou F C, Chai Y. Near-sensor and in-sensor computing. Nat Electron, 2020, 3, 664 doi: 10.1038/s41928-020-00501-9
[5]
Mennel L, Symonowicz J, Wachter S, et al. Ultrafast machine vision with 2D material neural network image sensors. Nature, 2020, 579, 62 doi: 10.1038/s41586-020-2038-x
[6]
Jang H, Hinton H, Jung W B, et al. In-sensor optoelectronic computing using electrostatically doped silicon. Nat Electron, 2022, 5, 519 doi: 10.1038/s41928-022-00819-6
[7]
Lopez-Sanchez O, Lembke D, Kayci M, et al. Ultrasensitive photodetectors based on monolayer MoS2. Nat Nanotechnol, 2013, 8, 497 doi: 10.1038/nnano.2013.100
[8]
Jayachandran D, Oberoi A, Sebastian A, et al. A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector. Nat Electron, 2020, 3, 646 doi: 10.1038/s41928-020-00466-9
[9]
Zhang Z F, Zhao X L, Zhang X M, et al. In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array. Nat Commun, 2022, 13, 6590 doi: 10.1038/s41467-022-34230-8
[10]
Zhou Y, Fu J W, Chen Z R, et al. Computational event-driven vision sensors for in-sensor spiking neural networks. Nat Electron, 2023, 6, 870 doi: 10.1038/s41928-023-01055-2
[11]
Xiang D, Liu T. Extending in-sensor computing from static images to dynamic motions. Nat Electron, 2023, 6, 801 doi: 10.1038/s41928-023-01070-3
[12]
Appeltant L, Soriano M C, Van der Sande G, et al. Information processing using a single dynamical node as complex system. Nat Commun, 2011, 2, 468 doi: 10.1038/ncomms1476
[13]
Lee H, Lee S, Kim J, et al. Stretchable array electromyography sensor with graph neural network for static and dynamic gestures recognition system. NPJ Flex Electron, 2023, 7, 20 doi: 10.1038/s41528-023-00246-3
[14]
Chen J W, Zhou Z, Kim B J, et al. Optoelectronic graded neurons for bioinspired in-sensor motion perception. Nat Nanotechnol, 2023, 18, 882 doi: 10.1038/s41565-023-01379-2
[15]
Liao F Y, Zhou Z, Kim B J, et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat Electron, 2022, 5, 84 doi: 10.1038/s41928-022-00713-1
[16]
Wu Y C, Liu C H, Chen S Y, et al. Extrinsic origin of persistent photoconductivity in monolayer MoS2 field effect transistors. Sci Rep, 2015, 5, 11472 doi: 10.1038/srep11472
[17]
George A, Fistul M V, Gruenewald M, et al. Giant persistent photoconductivity in monolayer MoS2 field-effect transistors. NPJ 2D Mater Appl, 2021, 5, 15 doi: 10.1038/s41699-020-00182-0
[18]
Van der Sande G, Brunner D, Soriano M C. Advances in photonic reservoir computing. Nanophotonics, 2017, 6, 561 doi: 10.1515/nanoph-2016-0132
[19]
Zhong Y N, Tang J S, Li X Y, et al. Dynamic memristor-based reservoir computing for high-efficiency temporal signal processing. Nat Commun, 2021, 12, 408 doi: 10.1038/s41467-020-20692-1
[20]
Li H, Zhang Q, Yap C C R, et al. From bulk to monolayer MoS2: Evolution of Raman scattering. Adv Funct Materials, 2012, 22, 1385 doi: 10.1002/adfm.201102111
[21]
Choudhary N, Park J, Hwang J Y, et al. Growth of large-scale and thickness-modulated MoS2 nanosheets. ACS Appl Mater Interfaces, 2014, 6, 21215 doi: 10.1021/am506198b
[22]
Chakraborty B, Matte H S S R, Sood A K, et al. Layer-dependent resonant Raman scattering of a few layer MoS2. J Raman Spectrosc, 2013, 44, 92 doi: 10.1002/jrs.4147
[23]
Brown N M D, Cui N Y, McKinley A. An XPS study of the surface modification of natural MoS2 following treatment in an RF-oxygen plasma. Appl Surf Sci, 1998, 134, 11 doi: 10.1016/S0169-4332(98)00252-9
[24]
Jadwiszczak J, O’Callaghan C, Zhou Y B, et al. Oxide-mediated recovery of field-effect mobility in plasma-treated MoS2. Sci Adv, 2018, 4, eaao5031 doi: 10.1126/sciadv.aao5031
[25]
Liu K Q, Zhang T, Dang B J, et al. An optoelectronic synapse based on α-In2Se3 with controllable temporal dynamics for multimode and multiscale reservoir computing. Nat Electron, 2022, 5, 761 doi: 10.1038/s41928-022-00847-2

    • Search

    Advanced Search >>

    GET CITATION

    shu

    Export: BibTex EndNote

    Article Metrics

    Article views: 1191 Times PDF downloads: 121 Times Cited by: 0 Times

    History

    Received: 01 April 2024 Revised: 26 April 2024 Online: Accepted Manuscript: 23 May 2024Uncorrected proof: 20 June 2024Published: 15 September 2024

    Catalog

      Email This Article

      User name:
      Email:*请输入正确邮箱
      Code:*验证码错误
      Send
      Article Navigation >  Journal of Semiconductors  > 2024  >  45(9): 092401
      Xiaoyong Jiang, Minrui Ye, Yunhai Li, Xiao Fu, Tangxin Li, Qixiao Zhao, Jinjin Wang, Tao Zhang, Jinshui Miao, Zengguang Cheng. Multiframe-integrated, in-sensor computing using persistent photoconductivity[J]. Journal of Semiconductors, 2024, 45(9): 092401. doi: 10.1088/1674-4926/24040002 ****X Y Jiang, M R Ye, Y H Li, X Fu, T X Li, Q X Zhao, J J Wang, T Zhang, J S Miao, and Z G Cheng, Multiframe-integrated, in-sensor computing using persistent photoconductivity[J]. J. Semicond., 2024, 45(9), 092401 doi: 10.1088/1674-4926/24040002
      Citation:
      Xiaoyong Jiang, Minrui Ye, Yunhai Li, Xiao Fu, Tangxin Li, Qixiao Zhao, Jinjin Wang, Tao Zhang, Jinshui Miao, Zengguang Cheng. Multiframe-integrated, in-sensor computing using persistent photoconductivity[J]. Journal of Semiconductors, 2024, 45(9): 092401. doi: 10.1088/1674-4926/24040002 ****
      X Y Jiang, M R Ye, Y H Li, X Fu, T X Li, Q X Zhao, J J Wang, T Zhang, J S Miao, and Z G Cheng, Multiframe-integrated, in-sensor computing using persistent photoconductivity[J]. J. Semicond., 2024, 45(9), 092401 doi: 10.1088/1674-4926/24040002
      shu

      Multiframe-integrated, in-sensor computing using persistent photoconductivity

      DOI: 10.1088/1674-4926/24040002
      • 1.

        School of Microelectronics, Fudan University, Shanghai 200433, China

      • 2.

        Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China

      • 3.

        University of Chinese Academy of Sciences, Beijing 100049, China

      • 4.

        School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China

      More Information
      • Xiaoyong Jiang got his bachelor's degree in 2017 from Nanjing University and his master’s degree in 2019 from University of Pennsylvania. Now he is a doctoral student at Fudan University under the supervision of Prof. Zengguang Cheng. His research focuses on AI infrared visual chips based on sensing, memory and computing integration
      • Xiao Fu received his doctoral degree from Dongguk University, Seoul, Korea, in 2020. He is currently an assistant Professor with the Shanghai Institute of Technical Physics of the Chinese Academy of Sciences, Shanghai, China. His current research interests include in-sensor computing devices
      • Zengguang Cheng received his Bachelor's degree from Xidian University in 2009 and his Doctoral degree from the National Center for Nanoscience and Technology in 2015. Following his Ph.D., he worked as a postdoctoral researcher at the University of Oxford until he joined Fudan University in early 2020. Currently, he holds a professorship in the School of Microelectronics at Fudan University. His research interests include the applications of photonic computing and the development of novel non-volatile memory technologies
      • Corresponding author: xiaofu@mail.sitp.ac.cn; zgcheng@fudan.edu.cn
      • Received Date: 2024-04-01
      • Revised Date: 2024-04-26
        • Available Online: 2024-05-23

      Catalog

        Journal of Semiconductors © 2017 All Rights Reserved   京ICP备05085259号-2

        /

        DownLoad:  Full-Size Img  PowerPoint
        Return
        Return