Randomized Algorithms Eduardo Laber Loana T. Nogueira.

Apresentação em tema: "Randomized Algorithms Eduardo Laber Loana T. Nogueira."— Transcrição da apresentação:

1 Randomized Algorithms Eduardo Laber Loana T. Nogueira

2 Quicksort Objective

3 Quicksort Objective – Sort a list of n elements

4 An Idea

5 Imagine if we could find an element y  S such that half the members of S are smaller than y, then we could use the following scheme

6 An Idea Imagine if we could find an element y  S such that half the members of S are smaller than y, then we could use the following scheme  Partition S\{y} into two sets S 1 and S 2

7 An Idea Imagine if we could find an element y  S such that half the members of S are smaller than y, then we could use the following scheme  Partition S\{y} into two sets S 1 and S 2  S 1 : elements of S that are smaller than y  S 2 : elements of S that are greater than y

8 An Idea Imagine if we could find an element y  S such that half the members of S are smaller than y, then we could use the following scheme  Partition S\{y} into two sets S 1 and S 2  S 1 : elements of S that are smaller than y  S 2 : elements of S that are greater than y  Recursively sort S 1 and S 2

9 Suppose we know how to find y

10 Time to find y: cn steps, for some constant c

11 Suppose we know how to find y Time to find y: cn steps, for some constant c we could partition S\{y} into S 1 and S 2 in n-1 additional steps

12 Suppose we know how to find y Time to find y: cn steps, for some constant c we could partition S\{y} into S 1 and S 2 in n-1 additional steps T(n)  2T(n/2) + (c+1)n The total number os steps in out sorting procedure would be given by the recurrence

13 Suppose we know how to find y Time to find y: cn steps, for some constant c we could partition S\{y} into S 1 and S 2 in n-1 additional steps T(n)  2T(n/2) + (c+1)n The total number os steps in out sorting procedure would be given by the recurrence  c’nlogn

14 What´s the problem with the scheme above? Quicksort

15 What´s the problem with the scheme above? Quicksort How to find y?

16 Deterministic Quicksort Let y be the first element of S

17 Deterministic Quicksort Let y be the first element of S Split S into two sets: S

18 Deterministic Quicksort Let y be the first element of S Split S into two sets: S – S < : elements smaller than y – S > : elements greater than y

19 Deterministic Quicksort Let y be the first element of S Split S into two sets: S – S < : elements smaller than y – S > : elements greater than y Qsort ( S )

20 Performance – Worst Case: O( n 2 ) – Avarage Case: O( nlogn )

21 Performance – Worst Case: O( n 2 ) (The set is already sorted) – Avarage Case: O( nlogn )

22 Performance – Worst Case: O( n 2 ) (The set is already sorted) – Avarage Case: O( nlogn )

23 An Randomzied Algorithm

24 An algorithm that makes choice (random) during the algorithm execution

25 Randomized Quicksort (RandQS)

26 Choose an element y uniformly at random of S

27 Randomized Quicksort (RandQS) Choose an element y uniformly at random of S – Every element of S has equal probability fo being chosen

28 Randomized Quicksort (RandQS) Choose an element y uniformly at random of S – Every element of S has equal probability fo being chosen By comparing each element fo S with y, determine S

29 Randomized Quicksort (RandQS) Choose an element y uniformly at random of S – Every element of S has equal probability fo being chosen By comparing each element fo S with y, determine S Recursively sort S

30 Randomized Quicksort (RandQS) Choose an element y uniformly at random of S – Every element of S has equal probability fo being chosen By comparing each element fo S with y, determine S Recursively sort S – OUTPUT: S

31 Intuition For some instance Quicksort works very bad  O( n 2 )

32 Intuition For some instance Quicksort works very bad  O( n 2 ) Randomization produces different executions for the same input. There is no instance for which RandQS works bad in avarage

33 Analysis

34 For sorting algorithms: we measure the running time of RandQS in terms of the number of comparisons it performs

35 Analysis For sorting algorithms: we measure the running time of RandQS in terms of the number of comparisons it performs This is the dominat cost in any reasonable implementation

36 Analysis For sorting algorithms: we measure the running time of RandQS in terms of the number of comparisons it performs This is the dominat cost in any reasonable implementation Our Goal: Analyse the expected number of comparisons in an execution of RandQS

37 Analysis S i : the ith smallest element of S

38 Analysis S i : the ith smallest element of S S 1 is the smallest element of S

39 Analysis S i : the ith smallest element of S S 1 is the smallest element of S S n is the largest element of S

40 Analysis S i : the ith smallest element of S Define the random variable S 1 is the smallest element of S S n is the largest element of S x ik = 1, if S i and S j are compared 0, otherwise

41 Analysis S i : the ith smallest element of S Define the random variable S 1 is the smallest element of S S n is the largest element of S x ik = 1, if S i and S j are compared 0, otherwise Dado um experimento aleatório com espaço amostral S, uma variável aleatória é uma função que associa a cada elemento amostral um número real

42 Analysis X ij is a count of comparisons between S i and S j : The total number of comparisons:   i = 1 n j > i X ij

43 Analysis X ij is a count of comparisons between S i and S j : The total number of comparisons:   i = 1 n j > i X ij We are interested in the expected number of comparisons E[ ]   i = 1 n j > i X ij

44 Analysis X ij is a count of comparisons between S i and S j : The total number of comparisons:   i = 1 n j > i X ij We are interested in the expected number of comparisons E[ ] =   i = 1 n j > i X ij   i = 1 n j > i E[X ij ] By the linearity of E[]

45 Analysis P ij : the probability that S i and S j are compared in an execution

46 Analysis P ij : the probability that S i and S j are compared in an execution Since X ij only assumes the values 0 and 1, ij

47 Analysis – Binary Tree T of RandQS Each node is labeled with a distinct element of S y T

48 Analysis – Binary Tree T of RandQS Each node is labeled with a distinct element of S y T S<S<

49 Analysis – Binary Tree T of RandQS Each node is labeled with a distinct element of S y T S<S< S>S>

50 Analysis – Binary Tree T of RandQS Each node is labeled with a distinct element of S y T S<S< S>S> The root of T is compared to the elements in the two sub-trees, but no comparisons is perfomed between an element of the left and righ sub-trees

51 Analysis – Binary Tree T of RandQS Each node is labeled with a distinct element of S y T S<S< S>S> The root of T is compared to the elements in the two sub-trees, but no comparisons is perfomed between an element of the left and righ sub-trees There is an comparison between S i and S j if and only if one of these elements is an ancestor of the other

52 Analysis – Binary Tree T of RandQS Consider the permutation  obtained by visiting the nodes of T in increasing order of the level numbers, and in a lef-to-rigth order within each level

53 Example: S=(3, 6, 2, 5, 4,1)

54 2

55 2 1 {3, 6, 5, 4} {1}

56 Example: S=(3, 6, 2, 5, 4,1) 2 15 {3, 6, 5, 4} {6}{3, 4} {1}

57 Example: S=(3, 6, 2, 5, 4,1) 2 15 {3, 6, 5, 4} 46 3 {6}{3, 4} {3} {1}

58 Example: S=(3, 6, 2, 5, 4,1) 2 15 {3, 6, 5, 4} 46 3 {6}{3, 4} {3} {1}

59 Example: S=(3, 6, 2, 5, 4,1) 2 15 {3, 6, 5, 4} 46 3 {6}{3, 4} {3} {1}

60 Example: S=(3, 6, 2, 5, 4,1) 2 15 {3, 6, 5, 4} 46 3 {6}{3, 4} {3} {1}  = (2, 1, 5, 4, 6, 3)

61 Back to the Analysis To compute p ij we make two observations: – There is a comparison between S i and S j if and only if S i or S j occurs earlier in the permutation  than any element S l such that i < l < j Any of the elements S i, S i+1,..., S j is likely to be the first of theses elements to be chosen as a partitioning element, and hence to appear first in  The probability that this first element is either S i or S j is exactly 2/(j-i+1)

62 Back to the Analysis To compute p ij we make two observations: – There is a comparison between S i and S j if and only if S i or S j occurs earlier in the permutation  than any element S l such that i < l < j Any of the elements S i, S i+1,..., S j is likely to be the first of theses elements to be chosen as a partitioning element, and hence to appear first in  The probability that this first element is either S i or S j is exactly 2/(j-i+1)

63 Back to the Analysis To compute p ij we make two observations: – There is a comparison between S i and S j if and only if S i or S j occurs earlier in the permutation  than any element S l such that i < l < j Any of the elements S i, S i+1,..., S j is likely to be the first of theses elements to be chosen as a partitioning element, and hence to appear first in  The probability that this first element is either S i or S j is exactly 2/(j-i+1)

64 Back to the Analysis To compute p ij we make two observations: – There is a comparison between S i and S j if and only if S i or S j occurs earlier in the permutation  than any element S l such that i < l < j Any of the elements S i, S i+1,..., S j is likely to be the first of theses elements to be chosen as a partitioning element, and hence to appear first in  The probability that this first element is either S i or S j is exactly 2/(j-i+1)

65 Análise Therefore, P ij = 2/(j-i+1)

66 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =

67 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =   2/(j-i+1) i = 1 n j > i

68 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =   2/(j-i+1) i = 1 n j > i    2/k i = 1 k=1 n n-1+1

69 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =   2/(j-i+1) i = 1 n j > i    2/k i = 1 k=1 n n-1+1  2  i = 1 k=1 n n  1/k

70 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =   2/(j-i+1) i = 1 n j > i    2/k i = 1 k=1 n n-1+1  2  i = 1 k=1 n n  1/k Série Harmônica

71 Análise Therefore, P ij = 2/(j-i+1)   i = 1 n j > i E[X ij ] =   i = 1 n j > i P ij =   2/(j-i+1) i = 1 n j > i    2/k i = 1 k=1 n n-1+1  2  i = 1 k=1 n n  1/k Série Harmônica  2 nln n

72 RandQs x DetQs Expected time of RandQs: O( nlogn ) A certain expected value may not garantee a reasonable probability of success. We could have, for example, the following probabilities of executing O( n 2 ) operations of executing O( nlogn ) operations 1-

73 RandQs x DetQs For n=100 => in 7 % cases, the algorithm would execute in O( n 2 ). Some times we want to garantee that the algorithm performance will not be far from its avarage one

74 RandQs x DetQs For n=100 => in 7 % cases, the algorithm would execute in O( n 2 ). Some times we want to garantee that the algorithm performance will not be far from its avarage one Objective: Prove that with high probability the RandQS algorithm works well

75 High Probability Bound

76 The previous analysis only says that the expected running time is O(nlog n)

77 High Probability Bound The previous analysis only says that the expected running time is O(nlog n) This leaves the possibility of large “deviations” from this expected value

78 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array

79 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array – Splits it into smaller and larger elements

80 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array – Splits it into smaller and larger elements – Recurses on both subarrays

81 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array – Splits it into smaller and larger elements – Recurses on both subarrays Fix an element x in the input

82 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array – Splits it into smaller and larger elements – Recurses on both subarrays Fix an element x in the input x belogs to a sequence of subarrays

83 High Probability Bound RECALL: – Quicksort choses a pivot at random from the input array – Splits it into smaller and larger elements – Recurses on both subarrays Fix an element x in the input x belogs to a sequence of subarrays x´s contribution to the running time is proportional to the number of different subarrays it belongs

84 High Probability Bound Every time x is compared to a pivot, its current subarray is split and x goes to one of the subarrays

85 High Probability Bound Every time x is compared to a pivot, its current subarray is split and x goes to one of the subarrays With high probability, x is compared to O(log n) pivots

86 High Probability Bound Every time x is compared to a pivot, its current subarray is split and x goes to one of the subarrays With high probability, x is compared to O(log n) pivots With probability 1- n -c, for some pos. constant c

87 GOOD and BAD Splits

88 We say that a pivot is good if each of the subarrays has size at most ¾ (equivalently,at least ¼) of the size of the split subarray

89 GOOD and BAD Splits We say that a pivot is good if each of the subarrays has size at most ¾ (equivalently,at least ¼) of the size of the split subarray Otherwise, it is bad

90 GOOD and BAD Splits We say that a pivot is good if each of the subarrays has size at most ¾ (equivalently,at least ¼) of the size of the split subarray Otherwise, it is bad The probability of a good and of a bad split is ½

91 GOOD and BAD Splits We say that a pivot is good if each of the subarrays has size at most ¾ (equivalently,at least ¼) of the size of the split subarray Otherwise, it is bad The probability of a good and of a bad split is ½ X can participate in at most log 4/3 n good splits

92 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits

93 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits Set M so that log 4/3 n  M/4

94 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits Set M so that log 4/3 n  M/4 Let M = 32 ln n:

95 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits Set M so that log 4/3 n  M/4 Let M = 32 ln n log 4/3 n  8 ln n Good choice

96 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits Set M so that log 4/3 n  M/4 Let M = 32 ln n log 4/3 n  8 ln n exp(-M/8)  1/n 4 Good choice

97 High Probability Bound Upper bound the probability of less than M/4 good splits in M splits Set M so that log 4/3 n  M/4 Let M = 32 ln n log 4/3 n  8 ln n exp(-M/8)  1/n 4 The probablility of participating in more than M = 32 ln n splits is less than 1/n 4 Good choice

98 High Probability Bound The probability that any element participates in more than M=32 ln n splits is less than 1/n 3

99 High Probability Bound The probability that any element participates in more than M=32 ln n splits is less than 1/n 3 With probability at least 1-1/n 3, the running time of quicksort is O(n log n)

100 Advantages of Randomized Algorithms For many problems, randomized algorithms run faster than the best know deterministic algorithms Many randomized algorithms are simpler to describe and implement than deterministic algorithms for comparable performance

101 Advantages of Randomized Algorithms For many problems, randomized algorithms run faster than the best know deterministic algorithms Many randomized algorithms are simpler to describe and implement than deterministic algorithms for comparable performance

102 Advantages of Randomized Algorithms For many problems, randomized algorithms run faster than the best know deterministic algorithms Many randomized algorithms are simpler to describe and implement than deterministic algorithms for comparable performance

103 Minimum Cut Problem Entrada: – Grafo G=(V,E) Saída: – Conjunto S  V que minimiza, ou seja, o número de arestas que tem uma extremidade em S e outra em

104 Minimum Cut Problem Notação – d(v) : grau do vértice v no grafo – N(v) : vizinhança de v

105 Minimum Cut Problem - Applications Network Reliability: If a graph has a small min- cut, then it is poorly connected Clustering – Web pages = nodes – Hyperlinks = edges Divide the graph into cluster with little connection between different clusters.

106 Contração de arestas Dado um grafo G=(V,E) e uma aresta e=(u,v), a contração da aresta e produz o grafo G/e=(V’, E’), onde

107 Contração de arestas - Exemplo G G/e a b c d uv f g a b c d uv f g e

108 Lema 1 Lema 1: o tamanho do corte mínimo em G/e é maior ou igual ao tamanho do corte mínimo em G

109 Lema 1 Lema 1: o tamanho do corte mínimo em G/e é maior ou igual ao tamanho do corte mínimo em G Prova: podemos associar cada corte em G/e a um corte em G com mesmo tamanho. Basta substituir em S os nós obtidos por contração pelos nós originais.

110 Lema 2 Lema 2: Se o corte mínimo em um grafo tem tamanho k, então d(v)  k, para todo v  V.

111 Lema 2 Lema 2: Se o corte mínimo em um grafo tem tamanho k, então d(v)  k, para todo v  V. Prova: Caso contrário S={ v } seria um corte de tamanho menor que k.

112 Corolário Corolário 1: Se o corte mínimo em um grafo tem tamanho k, então |E|  k.n/2

113 Corolário Corolário 1: Se o corte mínimo em um grafo tem tamanho k, então |E|  k.n/2 Prova: Segue do lema 2 e de que

114 Randomized MinCut G 0  G Para i=1 até |V|-2 – selecione aleatoriamente e i em G i-1 – faça G i = G i-1 / e i Retorne os vértices de um dos supervértices obtidos

115 Probabilidade Lema 3: Seja t 1, t 2,..., t k uma coleção de eventos. Temos que Prova: Base: Assuma que vale para k, provar para k+1.

116 Teorema Seja C = { e 1, e 2,..., e k } um corte mínimo no grafo G=(V,E). Se nenhuma aresta de C é escolhida pelo RndMinCut, então as arestas que sobram no grafo final são as arestas de C.

117 Teorema - prova Sejam A e B os dois supervértices obtidos e seja A C e B C as componentes conexas obtidas retirando C. Como nenhuma aresta de C foi escolhida:

118 Teorema - prova Logo, A=A C e B=B C De fato, assuma que  a  A C e  b  B C tal que a,b  A. Neste caso, existe um caminho em A entre a e b que utiliza somente arestas escolhidas pelo algoritmo. Como qualquer caminho entre a e b tem que utilizar arestas de C, logo uma aresta de C é escolhida. Contradição!

119 Análise Seja C o conjunto de arestas de um corte mínimo em G. Calculamos a probabilidade do algoritmo não escolher nenhuma aresta de C para contração. Se isto acontece, o algoritmo retorna um corte mínimo.

120 Análise Sorte i : evento indicando que o algoritmo não sorteou uma aresta de C na i-ésima iteração.

121 Análise Temos: Segue da relação entre o tamanho do corte e o número de arestas (corolário 1) que:

122 Análise Na segunda iteração o grafo G 1 tem n-1 vértices e seu corte mínimo tem tamanho |C|. Logo,

123 Análise Em geral, Segue que,

124 Análise Logo, conseguimos um min-cut com probabilidade maior ou igual a n=100 => 0,02 % (RUIM) O que fazer ?

125 Randomized MinCut 2 Repetir o processo RndMinCut várias vezes e devolver o melhor corte encontrado Repetindo K vezes a probabilidade de encontrar o corte mínimo é

126 Análise Repetindo n 2 /2 vezes a probabilidade de sucesso é (e-1)/e  64 % K repetições => O (K.m) tempo

127 Complexidade do Algoritmo mais rápido de Karger Running time: O(n 2 log n) Space: O(n 2 ) Este algoritmo encontra um min-cut com probabilidade  (1/log n) [D. Karger e C. Stein, Stoc 1993] Quem se habilita ?

128 Minimum Cut Problem – Deterministic Algorithm Complexity O(nm log n 2 /m) J. Hao and J. B. Orlin[1994] (baseado em fluxo em redes)

129 Dois tipos de algoritmos randomizados Algoritmos Las Vegas – Sempre produzem a resposta correta – Tempo de execução é uma variável aleatória Exemplo: RandQs – Sempre produz seqüência ordenada – Tempo de término varia de execução para execução em uma dada instância

130 Dois tipos de algoritmos randomizados Algoritmos Monte-Carlo – Podem produzir respostas incorretas – A probabilidade de erro pode ser cotada – Executando o algoritmo diversas vezes podemos tornar a probabilidade de erro tão pequena quanto se queira Exemplo: Min-Cut

131 Lema Se o algoritmo contract termina quando o número de vértices no grafo é exatamente t, então a probabilidade de um dado Min-cut sobreviver é de:

132 Algoritmo Fast Cut Executar o algoritmo até um dado valor de t e rodar um algoritmo determinístico a partir de um dado valor. Problema: Algoritmo determinístico é lento.

133 Teoremas Teorema 10.16 – Fast Cut executa em O (n 2 logn) – Prova: Teorema 10.17 – Algoritmo Fast Cut é bem sucedido com probabilidade

134 MAX SAT Entrada n variáveis booleanas : x 1,... X n m cláusulas : C 1,... C m Pesos w i >= 0 para cada clausula C i Objetivo : Encontrar uma atribuição de verdadeiro/falso para x i que maximize a soma dos pesos das clausulas satisfeitas

135 MAX SAT Algoritmo Aleatorizado Para i=1,...,n Se random(1/2) = 1 x i  true Senão x i  false Com probabilidade ½ dizemos que uma variável é verdadeira ou falsa

136 MAX SAT Teorema : O algoritmo tem aproximação ½ Prova: Considere a variável aleatória x j Logo,

137 MAX SAT E(x j ) = Pr(clausula j ser satisfeita) L j : número de literais na clausula j Obs: A clausula j não é satisfeita somente se todos literais forem 0 C j = (x 1 v x 3 v x 5 v x 6 ) Devemos ter x 1 =0, x 3 = 1, x 5 =0 e x 6 =0 Probabilidade = (1/2) 4 Caso Geral=(1/2) Lj

138 MAX SAT Probabilidade da clausula j ser satisfeita é 1-(1/2) Lj Logo, 0,5-aproximação Obs: é um limite superior

139 MAX SAT O que aconteceria se toda clausula tivesse exatamente 3 literais? 7/8-aproximação Hastad 97) Se MAXE3SAT tem aproximação (7/8 +  para algum  > 0, P = NP

140 MAX SAT: Desaleatorização C 1 = (x 1 v x 2 v  x 3 )w 1 C 2 = (  x 2 v  x 3 )w 2 Cada folha da árvore corresponde a um atribuição: Cada folha esta associada a um peso (soma dos pesos das clausulas satisfeitas pela atribuição correspondente a folha)

141 MAX SAT: Desaleatorização E(c(I)) = 7/8 w 1 + w 2 /2