KnE Energy | VII International Conference on Photonics and Information Optics (PhIO) | pages: 509–522

, , and

1. Introduction

For image registration photo and video cameras with millions of pixels are usually used. Part of information in these images can be considered as superfluous. Development of computational photography new methods [1-2], including compressed sensing (CS, compressive sensing, compressed sampling) allows to obtain full object image with an acceptable quality through registration of several projections of object.

Compressed sensing (compressive sensing, sparse sampling, or) has become to be widely applied about ten years ago [3-6] and has already found a lot of opportunities for application:

  • tomography and magnetic resonance [7-9];

  • encryption [10-12];

  • beamforming [13];

  • face recognition [14];

  • visible and infrared cameras [15-20];

  • digital holography [21-30], and etc.

For example, CS is being more and more widely used for digital holography tasks. Digital holography is a method that allows obtaining information about object's characteristics (amplitude or phase profile, temperature, index of refraction [31], etc.) by registration of interference of object and reference beams. CS was implemented for digital Gabor [21] and Fresnel [22] holograms in the visible range, for registration of amplitude [23] and phase [24] wave parameters by millimeter range holograms, for holographic tomography [7], for depth resolution increasing by inline holography [25], for acceleration of hologram processing by using graphic processors [26], and etc.

In the paper CS methods are considered and experimental results on reconstruction of binary and non-binary images from single-pixel camera registrations are given.

2. Methods

Compressed sensing is based on sparsity of signals. Decomposition of these signals in an appropriate basis contains many coefficients that are close or equal to zero. Since most coefficients are small then the relatively few large coefficients include majority of the object information. Therefore the method exploits the sparsity of a signal to achieve full signal reconstruction with a significantly little quantity of single measurements. Similar ideas underlie in many image, video and audio compression methods (such as JPEG [32], MPEG, and etc.).

Compressed sensing is based on Nyquist-Shannon's theorem: if an analog signal is bandlimited then it can be reconstructed lossless when discrete peaks are taken with the more often period than maximum frequency half-cycle. By nonlinear algorithms of reconstruction the high resolution signals and images can be reconstructed from incomplete data. A few number of selections decomposed in an appropriate basis that differs from the sparse signal basis are received in the beginning of the CS. Since the number of selections is limited the problem of image reconstruction in initial basis requires solution of underdetermined system of linear equations (system has fewer equations than variables). There are a huge number of various image candidates which can be result for this selection. Therefore to select the best candidate additional constraint is necessary to be introduced.

The classical solution of these problems is to minimize system energy (norm minimization). Usually it is mathematical operation including product of the reverse matrixes and basis selection. However this method can give not good results for many practical applications as few number of unknown coefficients (that are absent in selection) have zero energy. The sum in absolute values is subject to minimization [3]. Therefore search algorithm of the candidate with the smallest norm can be written rather as the linear programming task that has already effective methods of solution. It results in comparable results of norm using when many coefficients are equal to zero.

A single-pixel detector [15-20] using is very practical CS technology. To provide a set of random masks, a digital micromirror device (DMD) consisting of N tiny mirrors array is used. The radiation from an object is reflected from a set of DMD mirrors and is registered by a single-pixel detector. The number of registrations with the different masks (positions of DMD mirrors) was equal to hundreds or thousands. That number is significantly less than a number of pixels of nowaadays digital cameras. Firstly a single-pixel detector was used in CS for object information acquisition using standard diffraction registration [15-16]. Further it was applied for digital holographic information: registration of phase objects in inline holography [27-28], two-step inline holography [29], optical scanning holography [30], and etc.

The implementation scheme for numerical model experiments is showed in Fig. 1. It is one of the possible CS scheme implementation on the basis of a single-pixel detector and DMD.

Figure 1

Scheme of intensity registration by single-pixel detector using DMD.

fig-1.jpg

The beam passes through the object and is reflected from DMD that forms random pattern of zero and unit values. Next the radiation is focused by lens on to the single-pixel detector that measures obtained light intensity. The researcher can do any number of registrations. The obtained vector of the registered signals can be presented as a convolution of signal distribution on an object and on DMD:

Ym= DMD ,m×x,

(1) where у m is a single m th registered signal, DMD[•,m] is a m th random matrix transformed to 1D array, x[•] is unknown amplitude distribution of object image transformed to 1D array, [•] is 1D array with N pixels, N is quantity of pixels of the DMD and initial object image, [•, •] is 2D matrix with N × M pixels, M is quantity of registrations. Values of x[•] can be approximately decomposed in some basis (for example, discrete cosine transform basis).

The DMD[•, •] matrix with DMD arrays for all registrations is computational transformed in frequency representation by means of transform to the matrix DMD [•, •]. As a result there is a set of the low and high frequencies. Next m differences of DMD [•, m] × x [•] - y m are minimized, where x [•] is initial image that is represented in frequency domain. The object image reconstructed by selecting of obtained x values and inverse transform calculation.

Quality of image was evaluated by means of the normalized standard deviation (NSTD) between initial and reconstructed object images [33]:

NSTD=1n1=1N1n2=1N2Rn1,n2×On1,n22n1=1N1n2=1N2R2n1,n2×n1=1N1n2=1N2O2n1,n2,

(2)

where R(n 1 ,n 2 ) is the array of signals of the reconstructed image; O(n 1 ,n 2 ) is the array of signals of the initial image; (n 1 ,n 2 ) are discrete coordinates in the object plane; N 1× N 2 is quantity of pixels of the initial and reconstructed object images. NSTD value shows degree of visual resemblance to initial object image, including similarity in reproduction of halftones. If NSTD value is equal to zero, then images are identical and if to unit (1 or 100 %) - they are completely different. Usually, if NSTD value is less than 0.03 (or 3 %), then from the visual point of view the images seem identical. If NSTD value is more than 0.5 (or 50 %), then images seem different.

Dependencies of NSTD value vs normalized quantity of registrations will be constructed. Quantity of registration will be normalized on number of initial object pixels. Calculation time will be estimated on the computer Intel Core i7-2600 CPU 3.40GHz 16GB RAM Geforce GTX 760 4GB. Dependencies of processing time vs normalized quantity of registrations will be constructed. Quantity of registration will be normalized on number of initial object pixels too.

3. Results

Reconstruction of binary objects

Numerical experiments on reconstruction of binary objects by using compressed sensing and l2-norm minimization were performed. Examples of initial binary objects are shown in Fig. 2.: “M” letter (Fig. 2a; 142 × 80 pixels; total 11360 pixels), “Е” letter (Fig. 2b; 70 × 80 pixels; total 5600 pixels), “Ph” letters (Fig. 2c; 142 × 80 pixels; total 11360 pixels), “I” letter (Fig. 2d; 50 × 80 pixels; total 4000 pixels).

Figure 2

Initial binary object images.

fig-2.jpg

The results of image reconstruction are shown in Fig. 3 and Fig. 4. The average number of registrations for each object in Fig. 3 and Fig. 4a-d is equal to 800 that is 10 times less than the average number of pixels in initial objects (about 8000 pixels in each object): 1140 registrations of “M”, 560 registrations of “E”, 1140 registrations of “Ph”, and 390 registrations of “I”. For objects in Fig 4e-h average number of registrations is equal to 565 (7 % of average total pixel quantity for each object): 800 registrations of “M”, 390 registrations of “E”, 800 registrations of “Ph”, and 270 registrations of “I”. For objects in Fig 4j-m the average number of registrations is equal to 242 (3 % of total pixel quantity for each object): 340 registrations of “M”, 170 registrations of “E”, 340 registrations of “Ph”, and 120 registrations of “I”.

Figure 3

Reconstructed binary images by l2-norm minimization using quantity of registrations 10 times less than quantity of total object pixels quantity: 1140 (a, c), 560 (b), and 390 (d) registrations.

fig-3.jpg
Figure 4

Reconstructed binary images by compressed sensing using quantity of registrations 10 (a-d), 14 (e-h), and 33 (j-m) times less than quantity of total object pixels quantity: average 800 (a-d), 565 (e-h), and 242 (j-m) registrations.

fig-4.jpg

Obtained results demonstrated that the quality of the reconstructed images remains acceptable even if losses are more than 90% for the standard l2-norm minimization. Only several percentage of total number of pixels is enough for acceptable recovery of images from visual point of view by the CS. Dependencies of reconstruction quality (NSTD) vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for the CS and l2-norm minimization are shown in Fig. 5-6.

Figure 5

Dependencies of reconstruction quality (NSTD) vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for binary objects using l2-norm minimization.

fig-5.jpg
Figure 6

Dependencies of reconstruction quality (NSTD) vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for binary objects using compressed sensing.

fig-6.jpg

As it can be seen for CS use of 50% registrations relatively total object pixels quantity decreases the error to 0.1 (or 10 %) that corresponds to high reconstruction quality. The l2-norm minimization can give acceptable quality of image reconstruction from the visual point of view. However NSTD value is 5 times higher.

Obtained dependencies of reconstruction time vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for binary objects using compressed sensing is shown in Fig. 7.

Figure 7

Dependencies of reconstruction time vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for binary objects using compressed sensing.

fig-7.jpg

As it can be seen direct object reconstruction implementation lasts from several seconds to several minutes for up to 1200 registrations (15 % relatively average total object pixels quantity). The main contributions to the calculation time are processes of image decomposition on basis, calculating coefficients of decomposition and image reconstruction. The core of compressed sensing technique lasts an order of magnitude less time and equal to from the fraction of second to several seconds only.

Reconstruction of non-binary images

Single-pixel registrations and reconstruction of 2D grayscale and color objects by l2-norm minimization and compressed sensing is modeled. Example of the test object is standard grayscale image “Lenna”. The resolution is 128 × 128 pixels and so it consists of 16384 pixels. The “Lenna” image is shown in Fig. 8.

Figure 8

Initial grayscale image “Lenna”.

fig-8.jpg

The results of numerical reconstruction of this object by using the l2-norm minimization and compressed sensing methods are shown in Fig. 9-10. The average number of signal registrations is equal to:

  • 1000 that is 16.4 times less than total number of object's pixels (Fig. 9a, 10a),

  • 3000 that is 5.5 times less than total number of object's pixels (Fig. 9b, 10b),

  • 5000 that is 3.3 times less than total number of pixels (Fig. 9c, 10c),

  • 7000 that is 2.3 times less than total number of object's pixels (Fig. 9d, 10d).

Figure 9

Reconstructed grayscale image by l2-norm minimization using 1000 (a), 3000 (b), 5000 (c), and 7000 (d) registrations.

fig-9.jpg
Figure 10

Reconstructed grayscale image by compressed sensing using 1000 (a), 3000 (b), 5000 (c), and 7000 (d) registrations.

fig-10.jpg

The object image reconstructed by the compressed sensing can be visually distinguishable at losses about 90% of pixel's information (Fig. 10a). The l2-norm minimization is significantly worse in compare than compressed sensing as doesn't use various optimization features of the CS. The quality of the reconstructed image is acceptable if losses are 60% of information. The image quality can be increased by growth of number of registrations by a single-pixel detector.

Obtained dependencies of reconstruction quality (NSTD) vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for grayscale object using l2-norm minimization and compressed sensing are shown in Fig. 11.

Figure 11

Dependencies of reconstruction quality (NSTD) vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for grayscale object using l2-norm minimization and compressed sensing.

fig-11.jpg

Average NSTD values are better than that ones for the case of binary objects (see Fig. 5-6) due to a large effect of zero signal pixels to the binary objects NSTDs.

Dependency of reconstruction time vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for grayscale object using compressed sensing is shown in Fig. 12.

Figure 12

Dependency of reconstruction time vs normalized quantity of registrations (in percentage relatively total object pixels quantity) for grayscale object using compressed sensing.

fig-12.jpg

As it can be seen direct object reconstruction implementation lasts from several minutes to tens of minutes for up to 10000 registrations (60 % relatively average total object pixels quantity). These values are larger than that ones for the binary object cases because grayscale object is more complicated and more quantity of registrations is needed.

The main contributions to the calculation time are processes of image decomposition on basis, calculating coefficients of decomposition and image reconstruction like in the binary case. The core of compressed sensing technique lasts an order of magnitude less time and equal to from tens of seconds to several minutes only. Acceleration of the processing time can be performed by use of more powerful computers or systems, processors (GPUs, CUDA, DSPs), FPGAs, optimization of program codes, and other similar methods.

4. Conclusion

The analysis of image reconstruction by the compressed sensing and l2-norm minimization methods by using a single-pixel detector and the digital micromirror device was performed.

Numerical experiments on reconstruction of binary, gray-scale and color objects were conducted. Images with up to 256 × 256 pixels were used as objects. Quantity of registrations was normalized to number of initial object pixels and ranged from 0.01 % to 70 %.

Obtained results demonstrated that binary objects can be distinguishable using 97 % (in quantity of registrations relatively total object pixels quantity) losses of information content. The reconstructed images of grayscale objects have acceptable visual quality at 80 % losses.

Obtained values of processing time are equal to several seconds for binary objects and several minutes for grayscale images using direct implementation of reconstruction by compressed sensing. These values can be significantly reduced by use of more powerful computers or systems, processors, FPGAs, optimization of program codes, and other similar methods.

Acknowledgements

This work was partially supported by the Russian Foundation for Basic Research (RFBR) № 17-07-00829.

References

1 

Raskar R., Tumblin J. Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors, A K Peters/CRC Press, 350 p. (2009).

2 

Lukac R. Computational Photography: Methods and Applications, CRC Press, 564 p. (2016).

3 

Donoho D.L. Compressed sensing // IEEE Transactions Information Theory, Vol. 52, Pp. 1289-1306 (2006).

4 

Candes E.J. Compressed sampling // International Congress of Mathematicians, Vol. 3, Pp. 1433-1452 (2006).

5 

Eldar Y.C., Kutyniok G. Compressed Sensing: Theory and Applications, Cambridge University Press, 558 p. (2012).

6 

Foucart S., Rauhut H. A Mathematical Introduction to Compressive Sensing, Birkhäuser Basel, 625 p. (2013).

7 

Hahn J., Lim S., Choi K., Horisaki R., Brady D. Video-rate compressed holographic microscopic tomography // Optics Express, Vol. 19 (8), Pp. 7289-7298 (2011).

8 

Lustig, M., Donoho, D., Pauly, J.M. Sparse MRI: The application of compressed sensing for rapid MR imaging // Magnetic Resonance in Medicine, Vol. 58 (6), Pp. 1182-1195 (2007).

9 

Graff, C.G., Sidky, E.Y. Compressive sensing in medical imaging // Applied Optics, Vol. 54 (8), Pp. c23-c44 (2015).

10 

Rawat, N., Hwang, I.-C., Shi, Y., Lee, B.-G. Optical image encryption via photon-counting imaging and compressive sensing based ptychography // Journal of Optics, Vol. 17 (6), Pp. 065704 (2015).

11 

Zhou, N., Li, H., Wang, D., Pan, S., Zhou, Z. Image compression and encryption scheme based on 2D compressive sensing and fractional Mellin transform // Optics Communications, Vol. 343, Pp. 10-21 (2015).

12 

Zhou, N., Zhang, A., Zheng, F., Gong, L. Novel image compression-encryption hybrid algorithm based on key-controlled measurement matrix in compressive sensing // Optics and Laser Technology, Vol. 62, Pp. 152-160 (2014).

13 

Geoffrey F Edelmann, Charles F Gaumond. Beamforming using compressed sensing // JASA Express Letters, Vol. 130(4), Pp. EL232-EL237 (2011).

14 

Kuang chih Lee, Jeffrey Ho, David Kriegman. Acquiring linear subspaces for face recognition under variable lighting // IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Pp. 684-698 (2005).

15 

Takhar, D., Laska, J.N., Wakin, M.B., Duarte, M.F., Baron, D., Sarvotham, S., Kelly, K.F., Baraniuk, R.G. A new compressive imaging camera architecture using optical-domain compression // Proceedings of SPIE, Vol. 6065, Pp. 606509 (2006).

16 

Duarte M.F., Davenport M.A., Takhar D., Laska J.N., Sun T., Kelly K.F., Baraniuk R.G. Single-pixel imaging via compressed sampling: Building simpler, smaller, and less-expensive digital cameras // IEEE Signal Processing Magazine, Vol. 25, Pp. 83-91 (2008).

17 

Sun, B., Edgar, M.P., Bowman, R., Vittert, L.E., Welsh, S., Bowman, A., Padgett, M.J. 3D computational imaging with single-pixel detectors // Science, Vol. 340, Issue 6134, Pp. 844-847 (2013).

18 

Watts, C.M., Shrekenhamer, D., Montoya, J., Lipworth, G., Hunt, J., Sleasman, T., Krishna, S., Smith, D.R., Padilla, W.J. Terahertz compressive imaging with metamaterial spatial light modulators // Nature Photonics, Vol. 8 (8), Pp. 605-609 (2014).

19 

Edgar, M.P., Gibson, G.M., Bowman, R.W., Sun, B., Radwell, N., Mitchell, K.J., Welsh, S.S., Padgett, M.J. Simultaneous real-time visible and infrared video with single-pixel detectors // Scientific Reports, Vol. 5, Pp. 10669 (2015).

20 

Sun, M.-J., Edgar, M.P., Gibson, G.M., Sun, B., Radwell, N., Lamb, R., Padgett, M.J. Single-pixel three-dimensional imaging with time-based depth resolution // Nature Communications, Vol. 7, Pp. 12010 (2016).

21 

Brady D.J., Choi K., Marks D.L., Horisaki R., Lim S. Compressed holography // Optics Express, Vol. 17, Pp. 13040-13049 (2009).

22 

Rivenson Y., Stern A., Javidi B. Compressed Fresnel holography // Journal of Display Technology, Vol. 6 (10), Pp. 506-509 (2010).

23 

Cull C.F., Wikner, D.A., Mait, J.N., Mattheiss, M., Brady, D.J. Millimeter-wave compressed holography // Applied Optics, Vol. 49 (19), Pp. E67-E82 (2010).

24 

Qiao, L., Wang Y., Shen Z., Zhao Z., Chen Z. Compressed sensing for direct millimeter-wave holographic imaging // Applied Optics, Vol. 54 (11), Pp. 3280-3289 (2015).

25 

Rivenson Y., Stern A., Javidi B. Improved depth resolution by single-exposure in-line compressed holography // Applied Optics. Vol. 52 (1), Pp. A223-A231 (2013).

26 

Endo Y., Shimobaba T., Kakue T., Ito T. GPU-accelerated compressed holography // Optics Express, Vol. 24 (8), Pp. 8437-8445 (2016).

27 

Clemente P., Durán V., Tajahuerce E., Torres-Company V., Lancis J. Single-pixel digital ghost holography // Physical Review A - Atomic, Molecular, and Optical Physics, Vol. 86 (4), Pp. 041803 (2012).

28 

Clemente P., Durán V., Tajahuerce E., Andrés P., Climent V., Lancis J. Compressed holography with a single-pixel detector // Optics Letters, Vol. 38 (14), Pp. 2524-2527 (2013).

29 

Li, J., Li, Y., Wang, Y., Li, K., Li, R., Li, J., Pan, Y. Two-step holographic imaging method based on single-pixel compressed imaging // Journal of the Optical Society of Korea, Vol. 18 (2), Pp. 146-150 (2014).

30 

A-qian S., Ding-fu Z., Sheng Y., You-jun H., Peng Z., Jian-ming Y., Xin Z. Optical scanning holography based on compressed sensing using a digital micro-mirror device // Optics Communications, Vol. 385, Pp. 19-24 (2017).

31 

Schnars U., Jueptner W. Digital Holography: Digital Hologram Recording, Numerical Reconstruction, and Related Techniques. Berlin-Heidelberg: Springer-Verlag, 164 p. (2005).

32 

Wallace, G.K. The JPEG still picture compression standard // IEEE Transactions on Consumer Electronics, Vol. 38 (1), Pp. xviii-xxxiv (1992).

33 

Fienup J.R. Invariant error metrics for image reconstruction // Applied Optics, Vol. 36, Issue 32, Pp. 8352–8357 (1997).

FULL TEXT

Statistics

  • Downloads 3
  • Views 93

Navigation

Refbacks



ISSN: 2413-5453