A new publication from Opto-Electronic Advances, DOI 10.29026/oea.2023.220049 discusses a high performance “non-local” generic face reconstruction model using the lightweight Speckle-Transformer (SpT) Unet.
Credit: OEA
A new publication from Opto-Electronic Advances, DOI 10.29026/oea.2023.220049 discusses a high performance “non-local” generic face reconstruction model using the lightweight Speckle-Transformer (SpT) Unet.
By exploiting the feature extraction and generalization capabilities of existing advanced computer neural networks, combined with the speed of light, low energy consumption and parallel multi-dimensional optical signal processing capabilities of optical artificial intelligence algorithms, optical artificial intelligence for computational imaging (CI) is designed and developed. Significant progress has been made in CI, in which electrical convolutional neural networks (CNNs) have demonstrated that image reconstruction ranging from non-invasive medical imaging through the tissue to autonomous navigation of vehicles in foggy conditions can be reconstructed. However, due to the limited “local” kernel size of the convolutional operator, the performance of CNNs is inaccurate for spatially dense patterns, such as the generic face images. Therefore, a “non-local” kernel that can extract long-term dependencies of the feature maps is urgently needed. The transformers are modules that rely entirely on the attention mechanism and can be easily parallelized. Moreover, the Transformer assumes minimal prior knowledge about the structure of the problem as compared to their convolutional and recurrent counterparts in deep learning. In vision, transformers have been successfully used for image recognition, object detection, segmentation, image super-resolution, video understanding, image generation, text-image synthesis, and so on. However, based on current knowledge, none of the investigations has explored the performance of the transformers in CI, such as speckle reconstruction.
In this article, a “non-local” model, termed the Speckle-Transformer (SpT) UNet, is implemented for highly accurate, energy-efficient paralleling processing of the speckle reconstructions. The network is a UNet architecture including advanced transformer encoder and decoder blocks. For better feature reservation/extraction, the authors propose and demonstrate three key mechanisms, i.e., pre-batch normalization (pre-BN), position encoding in multi-head attention/multi-head cross-attention (MHA/MHCA), and self-built up/down sampling pipelines. For the “scalable” data acquisition, four different grits of diffusers within the 40 mm detection range are considered. It is worth noting that the SpT UNet is a lightweight network that is less than one order of parameters compared with other state-of-art “non-local” networks, such as ViT, and SWIN Transformer in vision computation.
The authors further quantitatively evaluate the network performance with four scientific indicators: Pearson correlation coefficient (PCC), structural similarity measure (SSIM), Jaccard index (JI), and peak signal-to-noise ratio (PSNR). The lightweight SpT UNet reveals a high efficiency and strong comparative performance with Pearson Correlation Coefficient (PCC), and structural similarity measure (SSIM) exceeding 0.989, and 0.950, respectively. For optical artificial intelligence, as the paralleling processing model, the light-weight SpT UNet can be further implemented as an all-optical neural network with surpassing feature extraction, light speed and passive processing abilities.
Article reference: Wang YYD, Wang H, Gu M. High performance “non-local” generic face reconstruction model using the lightweight Speckle-Transformer (SpT) UNet. Opto-Electron Adv 6, 220049 (2023). doi: 10.29026/oea.2023.220049
Keywords: speckle reconstruction / non-local model / generic face images / lightweight network
# # # # # #
The Centre for Artificial-Intelligence Nanophotonics (CAIN) of USST is a world-class institution for photonics research led by Professor Min Gu, an internationally- renowned expert in Photonics and Foreign Fellow of the Chinese Academy of Engineering. With the support from the School of Optical-electrical and Computer Engineering and its leading discipline Optical Engineering, the centre focuses its research on artificial intelligence nanophotonic chip and related fields. It has also been seeking for an all-round “production-education-research” cooperation with well-known hi-tech enterprises home and abroad to achieve the industrialization of its scientific research achievements. Its ultimate goal is to develop cutting-edge photonics technology, embracing a more intelligent and greener future.
At present, the centre has more than 60 staff members in all. All the teaching and research staff are Ph.D. holders from top domestic or overseas institutions. Their research fields cover nonlinear optics, material sciences, ultra-precision processing, nanophotonics, biomedical science and intelligent computing, etc. In addition to good working conditions, the centre boasts world-class facilities, including optical laboratories, materials laboratories, biomedical laboratories, and clean room for nano material processing, ensuring the implementation of various research work.
Aiming at the leading world-class institutions, the centre will steadily improve its construction and management in the future. It is going to build up ten or so professional research teams composed of chief scientists, teaching and research staff, PhD and MA students, as well as support teams consisting of administrative staff and technical engineers, making concerted efforts in the study of future optics and artificial intelligence nanophotonics.
# # # # # #
Opto-Electronic Advances (OEA) is a high-impact, open access, peer reviewed monthly SCI journal with an impact factor of 8.933 (Journal Citation Reports for IF2021). Since its launch in March 2018, OEA has been indexed in SCI, EI, DOAJ, Scopus, CA and ICI databases over the time and expanded its Editorial Board to 36 members from 17 countries and regions (average h-index 49).
The journal is published by The Institute of Optics and Electronics, Chinese Academy of Sciences, aiming at providing a platform for researchers, academicians, professionals, practitioners, and students to impart and share knowledge in the form of high quality empirical and theoretical research papers covering the topics of optics, photonics and optoelectronics.
# # # # # #
More information: http://www.oejournal.org/oea
Editorial Board: http://www.oejournal.org/oea/editorialboard/list
All issues available in the online archive (http://www.oejournal.org/oea/archive).
Submissions to OEA may be made using ScholarOne (https://mc03.manuscriptcentral.com/oea).
ISSN: 2096-4579
CN: 51-1781/TN
Contact Us: [email protected]
Twitter: @OptoElectronAdv (https://twitter.com/OptoElectronAdv?lang=en)
WeChat: OE_Journal
# # # # # #
Journal
Opto-Electronic Advances
DOI
10.29026/oea.2023.220049
