Curso de Visão Computacional Marcelo Gattass

Apresentação em tema: "Curso de Visão Computacional Marcelo Gattass"— Transcrição da apresentação:

1 Curso de Visão Computacional Marcelo Gattass 2009.2
Processamento de imagens capturadas para algoritmos de Visão Computacional Curso de Visão Computacional Marcelo Gattass 2009.2

2 Resumo Motivação (aplicações) Requisitos Deteção de pontos e arestas
Analise da variação local Deteção de arestas Deteção de pontos Haris SHIFT Viola Jones

3 object instance recognition (matching)
Motivação object instance recognition (matching) David Lowe

4 Sample results using the Viola-Jones Detector
Motivação Sample results using the Viola-Jones Detector Notice detection at multiple scales

5 Small set of 111 Training Images
Motivação Small set of 111 Training Images

6 How do we build panorama?
Motivação How do we build panorama? We need to match (align) images Darya Frolova, Denis Simakov, The Weizmann Institute of Science, 2004

7 Matching with Features
Motivação Matching with Features Detect feature points in both images Darya Frolova, Denis Simakov, The Weizmann Institute of Science, 2004

8 Matching with Features
Motivação Matching with Features Detect feature points in both images Find corresponding pairs Darya Frolova, Denis Simakov, The Weizmann Institute of Science, 2004

9 Matching with Features
Motivação Matching with Features Detect feature points in both images Find corresponding pairs Use these pairs to align images

10 Matching with Features
Motivação Matching with Features Problem 1: Detect the same point independently in both images no chance to match! We need a repeatable detector Darya Frolova, Denis Simakov, The Weizmann Institute of Science, 2004

11 Matching with Features
Motivação Matching with Features Problem 2: For each point correctly recognize the corresponding one ? We need a reliable and distinctive descriptor Darya Frolova, Denis Simakov, The Weizmann Institute of Science, 2004

12 Example: Build a Panorama
Motivação Example: Build a Panorama M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003

13 Motivação Photosynth

14 Busca de padrões geométricos
Motivação Mauricio Ferreira, Diogo Carneiro e Carlos Eduardo Lara Visão 2005

15 Requisitos Uma caracteristica deve ser robusta o suficiente para continuar se destacando mesmo quando a cena é capturada em diferentes condições

16 Requisitos Tom Duerig Types of invariance Illumination

17 Requisitos Tom Duerig Types of invariance Illumination Scale

18 Tom Duerig Types of invariance Illumination Scale Rotation

19 Types of invariance Illumination Scale Rotation Affine Tom Duerig
Requisitos Tom Duerig Types of invariance Illumination Scale Rotation Affine

20 Types of invariance Illumination Scale Rotation Affine
Requisitos Tom Duerig Types of invariance Illumination Scale Rotation Affine Full Perspective

21 Tipos de características de uma imagem
Globais: histograma, conteúdo de freqüências, etc... Locais: regiões com determinada propriedade, arestas, cantos, curvas, etc... 21

22 Modelo Matemático: Função
Tipos Modelo Matemático: Função 0% 20% 40% 60% 80% 100% Níveis de cinza Posição ao longo da linha x u v L L(u,v) Função

23 Derivadas direcionais
Requisitos Derivadas direcionais f(x,y) y x 23

24 Tipos Norma da derivada

25 Tipos Mínimo de formas quadráticas de matrizes simétricas positivas definidas

26 Autovetores e autovalores de matrizes simétricas positivas definidas
Tipos Autovetores e autovalores de matrizes simétricas positivas definidas

27 Minimização como um problema de autovalores
y p x' y' x mínimo

28 Teorema Espectral (Teorema dos Eixos Principais)
Toda matriz simétrica S (S = ST ) pode ser fatorada em:  - matriz diagonal real Q – matriz ortogonal, formada pelos autovetores de S MMachado

29 Estimando Orientação Local em Imagens
Usando o Teorema Espectral no problema de orientação: Mudança de base por rotação J é máximo se y só tem componente na direção do autovetor de maior autovalor MMachado

30 Identificando Estruturas Lineares com PCA
Problema. Dados os vetores v1,...,vk, em N dimensões, estimar a orientação média quando o sinal de vi é ignorado. Solução. A orientação média é dada pelo eixo principal da matriz MMachado

31 Matriz de Variância-Covariância
N variáveis M observações Variância Covariância Matriz de Variância-Covariância MMachado

32 PCA X2 X1 Maior Componente Principal maximiza a variância
minimiza a variância Menor Componente Principal MarcoMachado X1

33 Interpretação geométrica
(max)-1/2 (min)-1/2 1, 2 – autovalores of S 33

34 PCA Variância total = soma das variâncias Variância total = traço de S
Eixos principais também representam a variância total do conjunto de dados. Primeiro eixo: 1/traço(S) Segundo eixo: 2/traço(S) MarcoMachado

35 Estimando Orientação Local em Imagens
Interpretação dos Autovalores 1=0, 2=0 Intensidade constante, sem estrutura 1>0, 2=0 Estrutura linear (invariante por deslocamento em uma única direção) 1>0, 2>0 A estrutura desvia do modelo de estrutura linear Ruído Curvatura Múltiplas orientações 1=2 Estrutura isotrópica

36 Comportamento local: Classificação
2 “Edge” 2 >> 1 “Corner” 1 e 2 são grandes, 1 ~ 2; aumenta em todas as direções 1 e 2 são pequenos; Quase constante em “Edge” 1 >> 2 “Flat” 1 36

37 Detecção de arestas 1 > 2

38 Operadores clássicos Prewitt’s Diferencia Suavisa 38

39 Openadores clássicos Sobel’s Diferencia Suavisa 39

40 Detector de arestas 40

41 Quality of an Edge Detector
Robustness to Noise Localization Too Many/Too less Responses True Edge Poor robustness to noise Poor localization Too many responses Khurram Hassan-Shafique 41

42 Canny Edge Detector Criterion 1: Good Detection: The optimal detector must minimize the probability of false positives as well as false negatives. Criterion 2: Good Localization: The edges detected must be as close as possible to the true edges. Single Response Constraint: The detector must return one point only for each edge point. Khurram Hassan-Shafique 42

43 Hai Tao 43

44 The result General form of the filter (N.B. the filter is odd so h(x) = -h(-x) the following expression is for x < 0 only) Camillo J. Taylor 44

45 Approximation Canny’s filter can be approximated by the derivative of a Gaussian Canny Derivative of Gaussian Camillo J. Taylor 45

46 Canny Edge Detector Convolution with derivative of Gaussian
Non-maximum Suppression Hysteresis Thresholding Khurram Hassan-Shafique 46

47 Algorithm Canny_Enhancer
Smooth by Gaussian Compute x and y derivatives Compute gradient magnitude and orientation Khurram Hassan-Shafique 47

48 Canny Edge Operator Khurram Hassan-Shafique 48

49 Canny Edge Detector Khurram Hassan-Shafique 49

50 Canny Edge Detector Khurram Hassan-Shafique 50

51 Algorithm Non-Maximum Suppression
We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: at which point is the maximum, and where is the next one? Khurram Hassan-Shafique 51

52 Non-Maximum Suppression
Suppress the pixels in ‘Gradient Magnitude Image’ which are not local maximum Khurram Hassan-Shafique 52

53 Non-Maximum Suppression
Khurram Hassan-Shafique 53

54 Non-Maximum Suppression
Khurram Hassan-Shafique 54

55 Hysteresis Thresholding
Khurram Hassan-Shafique 55

56 Hysteresis Thresholding
If the gradient at a pixel is above ‘High’, declare it an ‘edge pixel’ If the gradient at a pixel is below ‘Low’, declare it a ‘non-edge-pixel’ If the gradient at a pixel is between ‘Low’ and ‘High’ then declare it an ‘edge pixel’ if and only if it is connected to an ‘edge pixel’ directly or via pixels between ‘Low’ and ‘ High’ Khurram Hassan-Shafique 56

57 Hysteresis Thresholding
Khurram Hassan-Shafique 57

58 Resultado de algoritmo de histerese
58

59 Subpixel Localization
One can try to further localize the position of the edge within a pixel by analyzing the response to the edge enhancement filter One common approach is to fit a quadratic polynomial to the filter response in the region of a maxima and compute the true maximum. -1 1 59

60 Detector de cantos

61 Comportamento local: Classificação
2 “Edge” 2 >> 1 “Corner” 1 e 2 são grandes, 1 ~ 2; aumenta em todas as direções 1 e 2 são pequenos; Quase constante em “Edge” 1 >> 2 “Flat” 1 61

62 Harris Detector: Mathematics
Measure of corner response: (k – empirical constant, k = ) 62

63 Harris Detector: Mathematics
1 “Corner” “Edge” “Flat” R > 0 R < 0 |R| small 2 R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge |R| is small for a flat region 63

64 Algoritmo Comparação dos gráficos 2 “Edge” “Corner” R < 0 R > 0
“Flat” “Edge” |R| small R < 0 1 64

65 Algoritmo Comparação dos gráficos 2 “Edge” “Corner” R < 0 R > 0
“Flat” “Edge” |R| small R < 0 1 65

66 Algoritmo Comparação dos gráficos 2 “Edge” “Corner” R < 0 R > 0
“Flat” “Edge” |R| small R < 0 1 66

67 Harris Detector The Algorithm:
Find points with large corner response function R (R > threshold) Take the points of local maxima of R 67

68 Harris Detector: Workflow
68

69 Harris Detector: Workflow
Compute corner response R 69

70 Harris Detector: Workflow
Find points with large corner response: R>threshold 70

71 Harris Detector: Workflow
Take only the points of local maxima of R 71

72 Harris Detector: Workflow
72

73 Example: Gradient Covariances
Corners are where both eigenvalues are big from Forsyth & Ponce Detail of image with gradient covar- iance ellipses for 3 x 3 windows Full image 73

74 Example: Corner Detection (for camera calibration)
courtesy of B. Wilburn 74

75 Example: Corner Detection
courtesy of S. Smith SUSAN corners 75

76 Harris Detector: Summary
Average intensity change in direction [u,v] can be expressed as a bilinear form: Describe a point in terms of eigenvalues of M: measure of corner response A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive 76

77 Harris Detector: Some Properties
Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

78 Harris Detector: Some Properties
Partial invariance to affine intensity change Only derivatives are used => invariance to intensity shift I  I + b Intensity scale: I  a I R x (image coordinate) threshold

79 Harris Detector: Some Properties
But: non-invariant to image scale! All points will be classified as edges Corner !

80 Harris Detector: Some Properties
Quality of Harris detector for different scale changes Repeatability rate: # correspondences # possible correspondences C.Schmid et.al. “Evaluation of Interest Point Detectors”. IJCV 2000

81 SIFT (Scale Invariant Feature Transform)

82 A 500x500 image gives about 2000 features
SIFT stages: Scale-space extrema detection Keypoint localization Orientation assignment Keypoint descriptor detector descriptor matching ( ) local descriptor A 500x500 image gives about 2000 features

83 1. Detection of scale-space extrema
For scale invariance, search for stable features across all possible scales using a continuous function of scale, scale space. SIFT uses DoG filter for scale space because it is efficient and as stable as scale-normalized Laplacian of Gaussian.

84 Scale space  doubles for the next octave K=2(1/s)

85 Detection of scale-space extrema

86 Keypoint localization
X is selected if it is larger or smaller than all 26 neighbors

87 2. Accurate keypoint localization
Reject points with low contrast and poorly localized along an edge Fit a 3D quadratic function for sub-pixel maxima

88 Accurate keypoint localization
Change sample point if offset is larger than 0.5 Throw out low contrast (<0.03)

89 Eliminating edge responses
Let Keep the points with r=10

90 Maxima in D

91 Remove low contrast and edges

92 3. Orientation assignment
By assigning a consistent orientation, the keypoint descriptor can be orientation invariant. For a keypoint, L is the image with the closest scale, orientation histogram

93 Orientation assignment

94 Orientation assignment

95 Orientation assignment

96 Orientation assignment

97 SIFT descriptor

98 4. Local image descriptor
Thresholded image gradients are sampled over 16x16 array of locations in scale space Create array of orientation histograms (w.r.t. key orientation) 8 orientations x 4x4 histogram array = 128 dimensions Normalized, clip values larger than 0.2, renormalize σ=0.5*width

99 SIFT extensions

100 PCA

101 PCA-SIFT Only change step 4
Pre-compute an eigen-space for local gradient patches of size 41x41 2x39x39=3042 elements Only keep 20 components A more compact descriptor

102 GLOH (Gradient location-orientation histogram)
SIFT 17 location bins 16 orientation bins Analyze the 17x16=272-d eigen-space, keep 128 components

103 Multi-Scale Oriented Patches
Simpler than SIFT. Designed for image matching. [Brown, Szeliski, Winder, CVPR’2005] Feature detector Multi-scale Harris corners Orientation from blurred gradient Geometrically invariant to rotation Feature descriptor Bias/gain normalized sampling of local patch (8x8) Photometrically invariant to affine changes in intensity

104 Multi-Scale Harris corner detector
Image stitching is mostly concerned with matching images that have the same scale, so sub-octave pyramid might not be necessary.

105 Multi-Scale Harris corner detector
gradient of smoother version Corner detection function: Pick local maxima of 3x3 and larger than 10

106 Orientation assignment
Orientation = blurred gradient

107 MOPS descriptor vector
Scale-space position (x, y, s) + orientation () 8x8 oriented patch sampled at 5 x scale. See the Technical Report for more details. Bias/gain normalisation: I’ = (I – )/ 40 pixels 8 pixels

108 Detections at multiple scales

109 Feature matching Exhaustive search Hashing Nearest neighbor techniques
for each feature in one image, look at all the other features in the other image(s) Hashing compute a short descriptor from each feature vector, or hash longer descriptors (randomly) Nearest neighbor techniques k-trees and their variants (Best Bin First)

110 Wavelet-based hashing
Compute a short (3-vector) descriptor from an 8x8 patch using a Haar “wavelet” Quantize each value into 10 (overlapping) bins (103 total entries) [Brown, Szeliski, Winder, CVPR’2005]

111 Nearest neighbor techniques
k-D tree and Best Bin First (BBF) Indexing Without Invariants in 3D Object Recognition, Beis and Lowe, PAMI’99

112 Reference Chris Harris, Mike Stephens, A Combined Corner and Edge Detector, 4th Alvey Vision Conference, 1988, pp David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, 60(2), 2004, pp Yan Ke, Rahul Sukthankar, PCA-SIFT: A More Distinctive Representation for Local Image Descriptors, CVPR 2004. Krystian Mikolajczyk, Cordelia Schmid, A performance evaluation of local descriptors, Submitted to PAMI, 2004. SIFT Keypoint Detector, David Lowe. Matlab SIFT Tutorial, University of Toronto. Matthew Brown, Richard Szeliski, Simon Winder, Multi-Scale Oriented Patches, MSR-TR , 2004.

113 FIM

114 Arestas e cantos Locais de mudanças significativas na intensidade da imagem 114

115 Edgedels = edge elements
115