Blind Deblurring of Natural Images

Apresentação em tema: "Blind Deblurring of Natural Images"— Transcrição da apresentação:

1 Blind Deblurring of Natural Images
Encontro Nacional de Ciência - Ciência 2010 7 de Julho de 2010 – Lisboa, Portugal Blind Deblurring of Natural Images Mariana S. C. Almeida Luís B. Almeida

2 Encontro Nacional de Ciência - Ciência 2010
Motivation - Photograph enhancement Horizontal motion blur: Out-of-focus blur: Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

3 Encontro Nacional de Ciência - Ciência 2010
Motivation - applications Photography. Video and surveillance. Astronomy. Remote sensing: - Radar imaging. - Tomography and other biomedical imaging. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

4 Encontro Nacional de Ciência - Ciência 2010
Problem formulation Unknowns to be estimated (sharp image and blurring filter) Observation (noisy blurred image) Mathematical convolution (2D) Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

5 Encontro Nacional de Ciência - Ciência 2010
Problem formulation Unknowns to be estimated (sharp image and blurring filter) Observation (noisy blurred image) noise Mathematical convolution (2D) Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

6 Proposed approach - Motivation
Motivation (edge properties): Image information is essentially in the edges. Only a small percentage of pixels have a relevant edge value, being almost vanished (close to zero) in most pixels. (Edges are sparse). Edges of the sharp “Lena”. Blurring degradations smooth the sharp edges, spreading the edges’ intensity around more pixels (edges becoming less sparse). Edges of a blurred “Lena”. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

7 Proposed approach - Steps
Edges extracted from “Lena”. An edge detector was developed. Initial image estimate. A strategy for inducing a solution with sparse edges was developed. Initially, images with very sparse (and sharp) edges are estimated. Sparsity is gradually reduced and the shape of the blurring filter and details of the sharp image are gradually learned. Edges Data connection term which is based on de degradation model and a regularization term (developed for the method) which imposes sparsity on the extracted edges. Through a continuation approach, the regularizing parameter is intially strong and being gradually reduced over the method iterations. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

8 Proposed approach - Algorithm
Image estimate is initialized with the blurred image (and the blurring filter with the identity). The “spartity property” is set to be high. Iterative cycle: Repeat the cycle, until the stopping criterion is met. Compute new image estimate. Compute new filter estimate. Reduce the intensity of the “sparsity property”. Edges Data connection term which is based on de degradation model and a regularization term (developed for the method) which imposes sparsity on the extracted edges. Through a continuation approach, the regularizing parameter is intially strong and being gradually reduced over the method iterations. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

9 Synthetic experiments - tested images
Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

10 Encontro Nacional de Ciência - Ciência 2010
Synthetic experiments - tested filters Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

11 Synthetic Results – no noise
Original image: Blurred image: Results: Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

12 Synthetic Results – 30dB of BSNR
Original image: Degraded image: Results: Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

13 Results - actual photos
Sharp scene: Blurred photo: Results: Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

14 Method comparison (actual photo)
Acquired Blurred photo Result of method 1 Result of method 2 Result of our method Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

15 Encontro Nacional de Ciência - Ciência 2010
Conclusions A new blind deblurring method was developed: Almost no prior knowledge about the blur is required. Can, however, use available information about the filter. Is suitable for a wide set of blurring degradations. Good performance on a wide set blurs and images (synthetic and real-life). Proved to be better than other existing approaches: better results and suitable to a wider range of blurs. Includes a new edge detector, new image prior and a new learning strategy. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

16 Encontro Nacional de Ciência - Ciência 2010
Publications Single-layer deblurring: [1] M. S. C. Almeida and L. B. Almeida, “Blind deblurring of natural images,” in IEEE Int. Conf. Acoustics, Speech, and Signal Processing - ICASSP, 2008. [2] M. S. C. Almeida and L. B. Almeida, “Blind and semi-blind deblurring of natural images,” IEEE Trans. Image Processing, vol. 19, no. 1, pp. 36–52, January 2010. [3] M. S. C. Almeida and L. B. Almeida, “Processo de focagem cega de imagens”, Portuguese Patent, October, 2009. Two-layer deblurring: [4] M. S. C. Almeida and L. B. Almeida, “Blind deblurring of foreground-background images,” in IEEE Inter. Conf. on Image Processing – ICIP, Cairo, Egypt, 2009. [5] M. S. C. Almeida and L. B. Almeida, “Blind deblurring of two-layer images,” in RecPad, Aveiro, Protugal, 2009. Encontro Nacional de Ciência - Ciência 2010 | 7 Julho 2010 Lisboa

17 Encontro Nacional de Ciência - Ciência 2010
| Julho 2010 Lisboa

18 Encontro Nacional de Ciência - Ciência 2010
| Julho 2010 Lisboa

19 Problem formulation Unknowns to be estimated (sharp image and blurring filter) Observation (blurred noisy image) noise Mathematical convolution (2D)

20 Proposed Deblurring Method (Cost function)
- edge extractor - image edges Cost function: - regularizing parameter - regularizing term: favors images with sparse response to the edge extractor

21 Deblurring Method (edge detector)
4 directional filters are combined: - edge image - output of filter with direction - set of directions

22 Image Prior/Regularizer
Sparse prior on image edges: - sparsifying parameter (0<q<1) - scaling parameter - small parameter log-Likelihood function: Regularizing term:

23 Guided Optimization Cost function:
is initially set to a large value and is slowly decreased over iterations: Initially, the main features/details are estimated. Smaller details are progressively taken into account as decreases. can be initialized with a larger value, which is progressively decreased over iterations.

24 Algorithm

25 Color Images Edge detector: Cost function: - sharp color image
- blurred color image channel of

26 Multi-frame Scenario Cost function: indexes channels indexes frames
(Noise of different frames is assumed to be independent, Gaussian and with the same variance)

27 Quantitative method comparison

28 Contributions Proposed solution: Existing methods:
- Assume some information about the blurring filter: The blurring filter is parameterized (segment, circle, etc) Information about the filter is addressed in a soft manner, through regularization schemes. - Consequently have more application restrictions. Almost no prior knowledge about the blur is required. (practically blind) Can, however, use available information about the filter. Is thus suitable for a wider set of blurring degradations. Includes a new edge detector, new image prior and a new optimization strategy.

29 Encontro Nacional de Ciência - Ciência 2010
| Julho 2010 Lisboa

30 Proposed approach - Algorithm
Image estimate is initialized with the blurred image (and the blurring filter with the identity). The “spartity property” is set to be high. Iterative cycle: Repeat the cycle, until the stopping criterion is met. Compute new image estimate. Compute new filter estimate. Reduce the intensity of the “sparsity property”. Edges Data connection term which is based on de degradation model and a regularization term (developed for the method) which imposes sparsity on the extracted edges. Through a continuation approach, the regularizing parameter is intially strong and being gradually reduced over the method iterations. Encontro Nacional de Ciência - Ciência 2010 | Julho 2010 Lisboa

31 Encontro Nacional de Ciência - Ciência 2010
Motivation - Photograph enhancement Horizontal motion blur: Encontro Nacional de Ciência - Ciência 2010 | Julho 2010 Lisboa

32 Encontro Nacional de Ciência - Ciência 2010
Motivation - Photograph enhancement Horizontal motion blur: Out-of-focus blur: Encontro Nacional de Ciência - Ciência 2010 | Julho 2010 Lisboa