blind deconvolution and phase retrieval

Slide 1:

Praveen Pankajakshan ARIANA INRIA/CNRS/UNS October 11, 2009 On Blind Deconvolution and PSF Retrieval Sophia Antipolis

Confocal Laser Scanning Microscope (CLSM):

October 11 2009 2 /37 Confocal Laser Scanning Microscope (CLSM) Laser Beam splitter Dichroic mirror Objective In focus plane Pin hole Photomultiplier tube (PMT)

Point-Spread Function :

October 11 2009 3 /37 Point-Spread Function Point-spread function by imaging point source.

Problem Statement:

October 11 2009 4 /37 Problem Statement blurring due to the diffraction-limited nature or aberrations during imaging, Even when pinhole diameter is 1 airy units (AU * ), 30% of light collected is from out-of-focus regions. noise due to the reduced number of photons detected, exact PSF of the system is unknown and varies with experiments. * Images from fluorescence microscopes are affected by

Slide 5:

October 11 2009 5 /37

Subject:

October 11 2009 6 /37 Subject

BD Problem Illustration:

October 11 2009 7 /37 BD Problem Illustration = N(  Observed image Unknown true object Unknown Point Spread Function (PSF) Goal: Given i , recover both o and h . i o h x y x z x y x z x y x z

Experimental PSF Illustration:

October 11 2009 8 /37 Experimental PSF Illustration Experimentally obtained PSF is limited by noise and blur as well. Blur over estimates the size of the true PSF . Z XY Z XY

Imaging model:

October 11 2009 9 /37 Imaging model Image formation statistics is Poisson with where * denotes 3D convolution (LSI), denotes all possible objects on the discrete spatial domain Ω , models the PSF and the low frequency background respectively, 1/γ is the photon conversion factor, and γi( x ) is the observed photon at the detector . Assumption: PMT operates in the photon counting mode with no read out or dark noise

Slide 10:

October 11 2009 10 /37

Estimating the fluorescence distribution :

October 11 2009 11 /37 Estimating the fluorescence distribution Likelihood of the observed data i knowing the specimen o and PSF h is given as [L. Mandel 1979] Find o such that: Approach: Maximum likelihood experctation maximization (MLEM) algorithm [Richardson74] [Lucy72] Assumption: h is known!

Constraints on the object:

October 11 2009 12 /37 Constraints on the object Gibbsian distribution Pr(o) with total variation (TV) functional captures the prior knowledge of the object where

Maximum A Posteriori Approach:

October 11 2009 13 /37 Maximum A Posteriori Approach From the Bayes theorem, the posterior probability is Thus the conditional probability can be written as:

Maximum A Posteriori object estimate:

October 11 2009 14 /37 Maximum A Posteriori object estimate Minimizing the cost function w.r.t o : Richardson-Lucy with TV Regularization [Dey et al. 04]

Toy problem results:

15 /37 Toy problem results original RL RL+TV

Alternate Minimization Algorithm:

October 11 2009 16 /37 Alternate Minimization Algorithm The cost function to be minimized has the form One solution is to alternatively minimize the cost function in o and h [Hebert et al. 89]

Parametric PSF model:

October 11 2009 17 /37 Parametric PSF model Diffraction-limited PSF approximation (in the LSQ sense) [Zhang et al. 06] for D<3AU. D is the diameter of the pinhole for CLSM [Zhang et al. 06] B. Zhang, J. Zerubia and J-C. Olivo-Marin, “ A study of Gaussian approximations of fluorescence microscopy PSF models, ” SPIE Conf. on microbiology, San Jose, Jan. 2006.

Analysis of Cost Function:

October 11 2009 18 /37 Analysis of Cost Function Analysis of the cost function J(·|o,i, ·) variation with lateral PSF parameter σ r . For this experiment, the object is known and the observation is generated using a known 3-D Gaussian model. The axial PSF parameter σ z is varied by a factor ± ε to monitor its effect on the estimated parameter σ r and vice versa.

Analysis of Cost Function:

October 11 2009 19 /37 Analysis of Cost Function Analysis of the cost function J(·|o, i, ·) variation with axial PSF parameter σ z .

Blind Deconvolution Problem Revisited:

October 11 2009 20 /37 Blind Deconvolution Problem Revisited = N(  Observed image Unknown true object Unknown Point Spread Function (PSF) Goal: Given i , recover both o and h . i o h x y x z x y x z x y x z

Blind and non-Blind Deconvolution:

October 11 2009 21 /37 Blind and non-Blind Deconvolution Observation non-Blind Object Blind The PSF is recovered automatically. Recovered images from blind deconvolution is as good as those recovered with the exact PSF for the simulated case . z

Analytical and Estimated PSF Comparison:

October 11 2009 22 /37 Analytical and Estimated PSF Comparison Residue showing the difference between the estimated PSF and the analytical PSF

Preliminary results on real data:

October 11 2009 23 /37 Preliminary results on real data Rendered sub-volume rendered of an (a) Arabidopsis Thaliana immersed in water and (b) restored using the AM algorithm. ©UNS/INRIA

Publications on this subject:

October 11 2009 24 /37 Publications on this subject Journal : P. Pankajakshan, B. Zhang, L. Blanc-F é raud, Z. Kam, J.-C. Olivo-Marin and J. Zerubia, “ On blind deconvolution for thin layered confocal imaging ,” to appear in Journal of Applied Optics, vol. 48, no. 21, July 2009. Conference : P. Pankajakshan, B. Zhang, L. Blanc-F é raud, Z. Kam, J.-C. Olivo-Marin and J. Zerubia, “ Blind decvolution for diffraction-limited fluorescence microscopy, ” Proc. of IEEE International Symposium on Biomedical Imaging (ISBI), pp. 740-743, May, 2008. P. Pankajakshan, B. Zhang, L. Blanc-F é raud, Z. Kam, J.-C. Olivo-Marin and J. Zerubia, “ Parameteric blind deconvolution for Confocal Laser Scanning Microscopy, ” IEEE Conference on Engineering and Medicine in Biology (EMBC), pp. 6531-6534, August 2007.

Slide 25:

October 11 2009 25 /37

Spherically Aberrated PSF Model:

October 11 2009 26 /37 Spherically Aberrated PSF Model For the spherical aberrated case, P a (λ, x ) is the aberrated pupil function for a wavelength λ where ψ is the phase factor due to aberrations . Small defocusing and small angle approximation [P. A. Stokseth 1969] [H. H. Hopkins 1955]

Illumination Pupil function for a WFM:

October 11 2009 27 /37 Illumination Pupil function for a WFM

Aberrated Phase for a WFM:

October 11 2009 28 /37 Aberrated Phase for a WFM

Object and aberrated PSF:

October 11 2009 29 /37 Object and aberrated PSF

Z shift:

October 11 2009 30 /37 Z shift Δ z

ML Phase retrieval:

October 11 2009 31 /37 ML Phase retrieval Phase estimation by MLE: θ are the parameters of the phase to be estimated. Regularization is achieved by a functional form of phase from Geometrical optics: where d is the depth of the specimen, n i is the objective immersion medium and n s is the specimen immersion medium, θ i and θ s are the azimuthal angles.

Axial Profile comparison:

October 11 2009 32 /37 Axial Profile comparison

Object estimation after 30 iterations:

October 11 2009 33 /37 Object estimation after 30 iterations Estimation of the object using (a) aberrated PSF (b) diffraction-limited approximation.

Phase and object retrieval:

October 11 2009 34 /37 Phase and object retrieval

Slide 35:

October 11 2009 35 /37

Conclusions and Perspectives:

October 11 2009 36 /37 Conclusions and Perspectives Current work : Model chosen is for the diffraction-limited PSF , the alternate minimization algorithm jointly estimates a separable 3D Gaussian PSF and the object, developed algorithms for phase and object estimation in spherically aberrated case. Looking ahead : Develop new parametric models to sparsely represent the spherically aberrated PSF, Perform additional experimentation on confocal image data of specimens, perform more tests on PSF estimation by phase parameter estimation, improving the prior representation of the specimen.

Thank You:

October 11 2009 37 /37 Thank You

Thank you!:

October 11 2009 38 /37 Thank you!

Example of a chart:

October 11 2009 39 /37 Example of a chart

Examples of default styles:

October 11 2009 40 /37 Examples of default styles Text and lines are like this Hyperlinks like this Visited hyperlinks like this Table Text box Text box With shadow