J. Semicond. >  Just Accepted

ARTICLES

An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor

Zhe Wang1, §, Jia-xing Song1, 2, §, Na Tian3, Xing-jia Ni1, Xu Yang1, Run-jiang Dou1, Peng Feng1, 2, Jian Liu1, 2, Nan-jian Wu1, Li-yuan Liu1, 2 and Shuang-ming Yu1, 2,

+ Author Affiliations

 Corresponding author: Shuang-ming Yu, yushuangming@semi.ac.cn

DOI: 10.1088/1674-4926/26030004CSTR: 32376.14.1674-4926.26030004

PDF

Turn off MathJax

Abstract: Single-photon avalanche diode (SPAD) image sensors are widely used in direct time-of-flight (D-TOF) imaging, but their ranging performance is often constrained by limited laser power. This article presents a SPAD-based D-TOF imaging system that combines a reconfigurable macro-pixel sensor architecture with a lightweight depth completion algorithm to achieve long-range depth imaging with enhanced spatial resolution under low optical power. The proposed sensor adopts a back-side illuminated (BSI) 3D-stacked architecture with programmable macro-pixels that enhance detection sensitivity and enable flexible sensitivity–resolution trade-offs. An injection-locked ring-oscillator-based time-to-digital converter (RO-TDC) array achieves a time resolution of 152.5 ps, enabling accurate TOF measurement at an optical power of 10 mW. To compensate for macro-pixel-induced resolution loss, a probabilistic normalized convolutional neural network (pNCNN) is employed for depth completion using sparse depth inputs only. Experimental results demonstrate that up to 30 × effective resolution enhancement of the system can be achieved via the depth completion algorithm without changing the physical resolution of the sensor. Additionally, the proposed system achieves a maximum ranging distance of 90 m and a range-to-power figure-of-merit (FOM) of 9 m/mW, which validates the effectiveness of the system.

Key words: SPAD, reconfigurable macro-pixel, time-to-digital converter (TDC), depth completion algorithm



[1]
Brown M Z, Burschka D, Hager G D. Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell, 2003, 25(8): 993 doi: 10.1109/TPAMI.2003.1217603
[2]
Hennad A, Cockett P, McLauchlan L, et al. Characterization of irregularly-shaped objects using 3D structured light scanning. 2019 International Conference on Computational Science and Computational Intelligence (CSCI), 2020: 600
[3]
Li D, Hu J, Ma R, et al. SPAD-based LiDAR with real-time accuracy calibration and laser power regulation. IEEE Trans Circuits Syst II: Express Briefs, 2023, 70(2): 431
[4]
Zheng L X, Zheng Y N, Wan C G, et al. High-precision GRO-based ROIC adopting sliding scale average method for SPAD array. IEEE Trans Circuits Syst II: Express Briefs, 2024, 71(9): 4151 doi: 10.1109/tcsii.2024.3382317
[5]
Wan C G, Yang W H, Zhang J, et al. A three-segment TDC with a half-cycle single-stage vernier ring for SPAD sensors. IEEE Trans Circuits Syst II: Express Briefs, 2025, 72(10): 1378
[6]
Park B, Park I, Park C, et al. A 64 × 64 SPAD-based indirect time-of-flight image sensor with 2-tap analog pulse counters. IEEE J Solid-State Circuits, 2021, 56(10): 2956 doi: 10.1109/JSSC.2021.3094524
[7]
Zhang C, Lindner S, Antolović I M, et al. A 30-frames/s, 252 × 144 SPAD flash LiDAR with 1728 dual-clock 48.8-ps TDCs, and pixel-wise integrated histogramming. IEEE J Solid-State Circuits, 2019, 54(4): 1137 doi: 10.1109/JSSC.2018.2883720
[8]
Ma Z Q, Wu Z, Xu Y. Compact SPAD pixels with fast and accurate photon counting in the analog domain. J Semicond, 2021, 42(5): 052402 doi: 10.1088/1674-4926/42/5/052402
[9]
El-Desouki M M, Palubiak D, Deen M J, et al. A novel, high-dynamic-range, high-speed, and high-sensitivity CMOS imager using time-domain single-photon counting and avalanche photodiodes. IEEE Sens J, 2011, 11(4): 1078 doi: 10.1109/JSEN.2010.2058846
[10]
Manuzzato E, Tontini A, Seljak A, et al. A 64× 64-pixel flash LiDAR SPAD imager with distributed pixel-to-pixel correlation for background rejection, tunable automatic pixel sensitivity and first-last event detection strategies for space applications. 2022 IEEE International Solid-State Circuits Conference (ISSCC), 2022: 96
[11]
Patanwala S M, Gyongy I, Mai H N, et al. A high-throughput photon processing technique for range extension of SPAD-based LiDAR receivers. IEEE Open J Solid State Circuits Soc, 2022, 2: 12 doi: 10.1109/OJSSCS.2021.3118987
[12]
Malass I, Uhring W, Le Normand J P, et al. SiPM based smart pixel for photon counting integrated streak camera. 2013 Conference on Design and Architectures for Signal and Image Processing, 2013: 135
[13]
Ximenes A R, Padmanabhan P, Charbon E. Mutually coupled time-to-digital converters (TDCs) for direct time-of-flight (dTOF) image sensors ‡. Sensors, 2018, 18(10): 3413 doi: 10.3390/s18103413
[14]
Nissinen I, Mantyniemi A, Kostamovaara J. A CMOS time-to-digital converter based on a ring oscillator for a laser radar. ESSCIRC 2004 - 29th European Solid-State Circuits Conference, 2004: 469
[15]
Eldesokey A, Felsberg M, Holmquist K, et al. Uncertainty-aware CNNs for depth completion: Uncertainty from beginning to end. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, 2020: 12011
[16]
Silberman N, Hoiem D, Kohli P, et al. Indoor segmentation and support inference from RGBD images. Computer Vision – ECCV 2012. Berlin, Heidelberg: Springer, 2012: 746
[17]
Huang Z X, Fan J M, Cheng S G, et al. HMS-net: Hierarchical multi-scale sparsity-invariant network for sparse depth completion. IEEE Trans Image Process, 2020, 29: 3429 doi: 10.1109/TIP.2019.2960589
[18]
Lu K Y, Barnes N, Anwar S, et al. From depth what can you see? Depth completion via auxiliary image reconstruction. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 11303
[19]
Gu J Q, Xiang Z Y, Ye Y W, et al. DenseLiDAR: A real-time pseudo dense depth guided depth completion network. IEEE Robot Autom Lett, 2021, 6(2): 1808 doi: 10.1109/LRA.2021.3060396
[20]
Wu Z Y, Wang H Y, Deng X Y. Multi-scale spatial propagation network for depth completion. 2024 4th International Conference on Computer, Control and Robotics (ICCCR), 2024: 151
[21]
Zhuo S L, Xia T, Zhao L, et al. Solid-state dToF LiDAR system using an eight-channel addressable, 20-W/ch transmitter, and a 128 × 128 SPAD receiver. IEEE J Solid State Circuits, 2023, 58(3): 757 doi: 10.1109/JSSC.2022.3227078
[22]
Wu Y F, Sun M, Zhou S F, et al. An adaptive beam-steering dToF LiDAR system using addressable multi-channel VCSEL transmitter. IEEE Trans Circuits Syst I Regul Pap, 2025, 72(5): 2089
[23]
Hu J, Liu B Z, Ma R, et al. A 32 × 32-pixel flash LiDAR sensor with noise filtering for high-background noise applications. IEEE Trans Circuits Syst I Regul Pap, 2022, 69(2): 645 doi: 10.1109/TCSI.2020.3048367
Fig. 1.  (Color online) Visual flow of the system.

DownLoad: Full-Size Img PowerPoint
Fig. 2.  3D stacked 64 × 64 SPAD image sensor.

DownLoad: Full-Size Img PowerPoint
Fig. 3.  (Color online) (a) 8 × 8 macro-pixel (b) Detailed circuit schematic of single pixel.

DownLoad: Full-Size Img PowerPoint
Fig. 4.  (Color online) Timing diagram for dual mode readout.

DownLoad: Full-Size Img PowerPoint
Fig. 5.  (Color online) (a) RO-TDC circuit array (b) Detailed circuit structure of RO-TDC.

DownLoad: Full-Size Img PowerPoint
Fig. 6.  (Color online) Overall architecture of the proposed pNCNN-based depth completion network.

DownLoad: Full-Size Img PowerPoint
Fig. 7.  (Color online) Photograph of the chip mounted on the test board.

DownLoad: Full-Size Img PowerPoint
Fig. 8.  (Color online) Ranging performance of sub-pixel configured for (a) 1 × 1, (b) 2 × 2, (c) 4 × 4 and (d) 8 × 8.

DownLoad: Full-Size Img PowerPoint
Fig. 9.  (Color online) Linearity test results in RO-TDC array.

DownLoad: Full-Size Img PowerPoint
Fig. 10.  (Color online) 16 × 16 sensor-generated sparse depth image and the completed 480 × 480 dense depth map.

DownLoad: Full-Size Img PowerPoint

Table 1.   Layer-wise configuration of the NCNN

Layer Index Layer Type Kernel Size Stride Padding
1 NConv 5 × 5 1 2
2 NConv 5 × 5 1 2
3 NConv 5 × 5 1 2
4 NConv 3 × 3 1 1
5 NConv 3 × 3 1 1
6 NConv 3 × 3 1 1
7 NConv 1 × 1 1 0
DownLoad: CSV

Table 2.   Algorithm performance comparison

Parameters This work RAL[19] ICCCR[20]
MAE(m) 0.019 0.021 0.038
RMSE(m) 0.020 0.074 0.096
Params(k) 57 18800 16500
DownLoad: CSV

Table 3.   Comparison with previous works.

Parameters Unit This work JSSC'23[21] TCAS-I'25[22] TCAS-I'22[23]
Process Node - 180/130 nm 180 nm BCD 180 nm BCD 180 nm
Array Size - 64 × 64 128 × 128 120 × 80 32 × 32
DCR @Vex Hz 40 @3 V 100 k @1.8 V 10 @3 V 1.2 k @3 V
PDP % 12.9%@850 nm 1%@940 nm 3.6%940 nm N/A
Frame Rate[3D imaging] fps 6(a) 1 N/A 30
Time Res. [1LSB] ps 152.5 100 100 200
Timing Depth bit 12 10 N/A 13
Timing DNL LSB +0.50/−0.25 +0.6/−0.7 +0.6/−0.7 +0.24/−0.29
Timing INL LSB +0.81/−0.59 +1.4/−1.3 +1.4/−1.3 +1.02/−2.53
Measurement Range m 90 15 15 20
Ranging Accuracy(b) cm 6.1 20 5 6
Ranging Precision(c) cm 6.8 N/A N/A 7
FOV degree 20 × 20 36 × 36 N/A 20 × 20
Scanning Mechanism - Flash Solid-State
Scanning + Flash		 Solid-State
Scanning		 Flash
Background Light - Indoor Indoor Outdoor Indoor/70 klux
Power consumption mW 210 37 227 240
Laser Power mW 10 150 60(d) 20
FOM(e) m/mW 9 0.10 0.25 1.00
(a) The frame rate obtained by accumulating 2560 pulses into one frame. (b) Accuracy characterizes the deviation between the measured value and the true value, reflecting the magnitude of the systematic error. (c) Precision characterizes the consistency of results across multiple measurements, reflecting the level of random noise and timing jitter. (d) Estimated based on the total 204mW laser transmitter power consumption. (e) FOM (Range/Laser Power) is defined as the ratio of measurement range to laser power, indicating the ranging efficiency per unit optical power.
DownLoad: CSV
[1]
Brown M Z, Burschka D, Hager G D. Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell, 2003, 25(8): 993 doi: 10.1109/TPAMI.2003.1217603
[2]
Hennad A, Cockett P, McLauchlan L, et al. Characterization of irregularly-shaped objects using 3D structured light scanning. 2019 International Conference on Computational Science and Computational Intelligence (CSCI), 2020: 600
[3]
Li D, Hu J, Ma R, et al. SPAD-based LiDAR with real-time accuracy calibration and laser power regulation. IEEE Trans Circuits Syst II: Express Briefs, 2023, 70(2): 431
[4]
Zheng L X, Zheng Y N, Wan C G, et al. High-precision GRO-based ROIC adopting sliding scale average method for SPAD array. IEEE Trans Circuits Syst II: Express Briefs, 2024, 71(9): 4151 doi: 10.1109/tcsii.2024.3382317
[5]
Wan C G, Yang W H, Zhang J, et al. A three-segment TDC with a half-cycle single-stage vernier ring for SPAD sensors. IEEE Trans Circuits Syst II: Express Briefs, 2025, 72(10): 1378
[6]
Park B, Park I, Park C, et al. A 64 × 64 SPAD-based indirect time-of-flight image sensor with 2-tap analog pulse counters. IEEE J Solid-State Circuits, 2021, 56(10): 2956 doi: 10.1109/JSSC.2021.3094524
[7]
Zhang C, Lindner S, Antolović I M, et al. A 30-frames/s, 252 × 144 SPAD flash LiDAR with 1728 dual-clock 48.8-ps TDCs, and pixel-wise integrated histogramming. IEEE J Solid-State Circuits, 2019, 54(4): 1137 doi: 10.1109/JSSC.2018.2883720
[8]
Ma Z Q, Wu Z, Xu Y. Compact SPAD pixels with fast and accurate photon counting in the analog domain. J Semicond, 2021, 42(5): 052402 doi: 10.1088/1674-4926/42/5/052402
[9]
El-Desouki M M, Palubiak D, Deen M J, et al. A novel, high-dynamic-range, high-speed, and high-sensitivity CMOS imager using time-domain single-photon counting and avalanche photodiodes. IEEE Sens J, 2011, 11(4): 1078 doi: 10.1109/JSEN.2010.2058846
[10]
Manuzzato E, Tontini A, Seljak A, et al. A 64× 64-pixel flash LiDAR SPAD imager with distributed pixel-to-pixel correlation for background rejection, tunable automatic pixel sensitivity and first-last event detection strategies for space applications. 2022 IEEE International Solid-State Circuits Conference (ISSCC), 2022: 96
[11]
Patanwala S M, Gyongy I, Mai H N, et al. A high-throughput photon processing technique for range extension of SPAD-based LiDAR receivers. IEEE Open J Solid State Circuits Soc, 2022, 2: 12 doi: 10.1109/OJSSCS.2021.3118987
[12]
Malass I, Uhring W, Le Normand J P, et al. SiPM based smart pixel for photon counting integrated streak camera. 2013 Conference on Design and Architectures for Signal and Image Processing, 2013: 135
[13]
Ximenes A R, Padmanabhan P, Charbon E. Mutually coupled time-to-digital converters (TDCs) for direct time-of-flight (dTOF) image sensors ‡. Sensors, 2018, 18(10): 3413 doi: 10.3390/s18103413
[14]
Nissinen I, Mantyniemi A, Kostamovaara J. A CMOS time-to-digital converter based on a ring oscillator for a laser radar. ESSCIRC 2004 - 29th European Solid-State Circuits Conference, 2004: 469
[15]
Eldesokey A, Felsberg M, Holmquist K, et al. Uncertainty-aware CNNs for depth completion: Uncertainty from beginning to end. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, 2020: 12011
[16]
Silberman N, Hoiem D, Kohli P, et al. Indoor segmentation and support inference from RGBD images. Computer Vision – ECCV 2012. Berlin, Heidelberg: Springer, 2012: 746
[17]
Huang Z X, Fan J M, Cheng S G, et al. HMS-net: Hierarchical multi-scale sparsity-invariant network for sparse depth completion. IEEE Trans Image Process, 2020, 29: 3429 doi: 10.1109/TIP.2019.2960589
[18]
Lu K Y, Barnes N, Anwar S, et al. From depth what can you see? Depth completion via auxiliary image reconstruction. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 11303
[19]
Gu J Q, Xiang Z Y, Ye Y W, et al. DenseLiDAR: A real-time pseudo dense depth guided depth completion network. IEEE Robot Autom Lett, 2021, 6(2): 1808 doi: 10.1109/LRA.2021.3060396
[20]
Wu Z Y, Wang H Y, Deng X Y. Multi-scale spatial propagation network for depth completion. 2024 4th International Conference on Computer, Control and Robotics (ICCCR), 2024: 151
[21]
Zhuo S L, Xia T, Zhao L, et al. Solid-state dToF LiDAR system using an eight-channel addressable, 20-W/ch transmitter, and a 128 × 128 SPAD receiver. IEEE J Solid State Circuits, 2023, 58(3): 757 doi: 10.1109/JSSC.2022.3227078
[22]
Wu Y F, Sun M, Zhou S F, et al. An adaptive beam-steering dToF LiDAR system using addressable multi-channel VCSEL transmitter. IEEE Trans Circuits Syst I Regul Pap, 2025, 72(5): 2089
[23]
Hu J, Liu B Z, Ma R, et al. A 32 × 32-pixel flash LiDAR sensor with noise filtering for high-background noise applications. IEEE Trans Circuits Syst I Regul Pap, 2022, 69(2): 645 doi: 10.1109/TCSI.2020.3048367

    • Search

    Advanced Search >>

    GET CITATION

    shu

    Export: BibTex EndNote

    Article Metrics

    Article views: 22 Times PDF downloads: 4 Times Cited by: 0 Times

    History

    Received: 02 March 2026 Revised: 20 April 2026 Online: Accepted Manuscript: 14 May 2026

    Catalog

      Email This Article

      User name:
      Email:*请输入正确邮箱
      Code:*验证码错误
      Send
      Article Navigation >  Journal of Semiconductors  > 2026  > Accepted Manuscript
      Zhe Wang, Jia-xing Song, Na Tian, Xing-jia Ni, Xu Yang, Run-jiang Dou, Peng Feng, Jian Liu, Nan-jian Wu, Li-yuan Liu, Shuang-ming Yu. An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor[J]. Journal of Semiconductors, 2026, In Press. doi: 10.1088/1674-4926/26030004 ****Z Wang, J X Song, N Tian, X J Ni, X Yang, R J Dou, P Feng, J Liu, N J Wu, L Y Liu, and S M Yu, An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor[J]. J. Semicond., 2026, 47(7): 072201 doi: 10.1088/1674-4926/26030004
      Citation:
      Zhe Wang, Jia-xing Song, Na Tian, Xing-jia Ni, Xu Yang, Run-jiang Dou, Peng Feng, Jian Liu, Nan-jian Wu, Li-yuan Liu, Shuang-ming Yu. An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor[J]. Journal of Semiconductors, 2026, In Press. doi: 10.1088/1674-4926/26030004 ****
      Z Wang, J X Song, N Tian, X J Ni, X Yang, R J Dou, P Feng, J Liu, N J Wu, L Y Liu, and S M Yu, An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor[J]. J. Semicond., 2026, 47(7): 072201 doi: 10.1088/1674-4926/26030004
      shu

      An algorithm-assisted high-resolution D-TOF imaging system with reconfigurable macropixel-based SPAD image sensor

      DOI: 10.1088/1674-4926/26030004
      CSTR: 32376.14.1674-4926.26030004
      • 1.

        State Key Laboratory of Semiconductor Physics and Chip Technologies, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China

      • 2.

        College of Materials Sciences and Opto-Electronic Technology, University of Chinese Academy of Sciences, Beijing 100049, China

      • 3.

        Southwest Institute of Technical Physics, Chengdu 610041, China

      More Information
      • Zhe Wang engaged in postdoctoral research work at the Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China, in 2021, where he became an Assistant Researcher in 2023. His research interests include analog and mixed-signal integrated circuits, SPAD image sensors, and image processing algorithms
      • Jia-xing Song：Jiaxing Song is pursuing an M.S. degree with the State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China. His current research interests include mixed-signal circuits design and the design of time-of-flight imaging
      • Shuang-ming Yu：Shuangming Yu received the B.S. degree in electronic science and technology from Beijing University of Posts and Telecommunications in 2010, and the Ph.D. degree in the Institute of Semi-conductors, Chinese Academy of Sciences (CAS), Beijing China. In 2015, He joined the State Key Laboratory of Superlattices and Microstructures, Institute of Semi-conductors of CAS, as an Assistant Professor and has been an Associate Professor since 2020. His current research interests include vision chip design and ultralow-power circuit design
      • Corresponding author: yushuangming@semi.ac.cn
      • Received Date: 2026-03-02
      • Revised Date: 2026-04-20
        • Available Online: 2026-05-14

      Catalog

        Journal of Semiconductors © 2017 All Rights Reserved   京ICP备05085259号-2

        /

        DownLoad:  Full-Size Img  PowerPoint
        Return
        Return