CS 3353 Lecture 5 Vidroha debroy Material adapted courtesy of Prof. Xiangnan Kong and Prof. Carolina Ruiz at Worcester Polytechnic Institute Searching for a number Find a specific number 91-3 26 54 73 3 68 103 131 88 91 19 1
Searching for a number Find a specific number 91 - what about now? 3 54 87 17 73 61 21 103 35 68 41 94 11 131 49 8 39 66 76 99 73 26 13 71 85 383 102 88 91 97 114 22 57 12 19 63 111 Why Sorting? Practical Application People by last name Countries by population Search engine results by relevance Fundamental to other algorithms Different algorithms have different asymtotic and constantfactor trade-offs no single best sort for all scenarios knowing one way to sort just isn't enough Keeping data in order allows it to be searched more efficiently 2
Sorting Sorting: Rearranging the values in an array or collection into a specific order (usually into their "natural ordering"). one of the fundamental problems in computer science it is estimated that 25~50% of all computing power is used for sorting activities. Popular Sorting algorithms: bubble sort: swap adjacent pairs that are out of order selection sort: look for the smallest element, move to front insertion sort: build an increasingly large sorted front portion merge sort: recursively divide the array in half and sort it heap sort: place the values into a sorted tree structure quick sort: recursively partition array based on a middle value Sorting Example Bubble Sort 3
Problem Definition Sorting takes an unordered collection and makes it an ordered one. 1 2 3 4 5 6 Input: 77 42 35 12 101 5 Output: 1 2 3 4 5 6 5 12 35 42 77 101 One Idea: Bubble Water 3 3 3 2 2 3 1 Air 1 2 1 2 1 4
"Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 77 42 35 12 101 5 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 77 42 42 77 35 12 101 5 5
"Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 42 77 35 35 77 12 101 5 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 42 35 77 12 12 77 101 5 6
"Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 42 35 12 77 101 5 No need to swap "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 42 35 12 77 101 5 101 5 7
"Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end Bubble the largest value to the end using pairwise comparisons and swapping 1 2 3 4 5 6 42 35 12 77 5 101 Largest value correctly placed The Bubble Up Algorithm Bubble-Sort-step1( A ): for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) ( A, x, y ): tmp = A[x] A[x] = A[y] A[y] = tmp 8
Need More Iterations Notice that only the largest value is correctly placed All other values are still out of order So we need to repeat this process 1 2 3 4 5 6 42 35 12 77 5 101 Largest value correctly placed Repeat Bubble Up How Many Times? If we have N elements And if each time we bubble an element, we place it in its correct location Then we repeat the bubble up process N 1 times. This guarantees we ll correctly place all N elements. 9
N - 1 2/1/2016 Bubbling All the Elements 1 2 3 4 5 6 42 35 12 77 5 101 1 2 3 4 5 6 35 12 42 5 77 101 1 2 3 4 5 6 12 35 5 42 77 101 1 2 3 4 5 6 12 5 35 42 77 101 1 2 3 4 5 6 5 12 35 42 77 101 The Bubble Up Algorithm Bubble-Sort( A ): for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) 10
Inner loop Outer loop 2/1/2016 The Bubble Up Algorithm Bubble-Sort( A ): for i = 1 (N 1): for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) The Bubble Up Algorithm Bubble-Sort( A ): for i = 1 (N 1): To do N-1 iterations To bubble a value for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) 11
Reducing the Number of Comparisons 1 2 3 4 5 6 77 42 35 12 101 5 1 2 3 4 5 6 42 35 12 77 5 101 1 2 3 4 5 6 35 12 42 5 77 101 1 2 3 4 5 6 12 35 5 42 77 101 1 2 3 4 5 6 12 5 35 42 77 101 Reducing the Number of Comparisons Assume the array size N On the i th iteration, we only need to do N-i comparisons. For example: N = 6 i = 4 (4 th iteration) Thus, we have 2 comparisons to do 1 2 3 4 5 6 12 35 5 42 77 101 12
The Bubble Up Algorithm Bubble-Sort( A ): for i = 1 (N 1): for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) The Bubble Up Algorithm Bubble-Sort( A ): for i = 1 (N 1): for k = 1 (N i): if A[k] > A[k+1]: ( A, k, k+1 ) 13
Code Demo An Animated Example i 1 k 98 23 45 14 6 67 33 42 14
An Animated Example i 1 k 1 98 23 45 14 6 67 33 42 An Animated Example i 1 k 1 98 23 45 14 6 67 33 42 15
An Animated Example i 1 k 1 23 98 45 14 6 67 33 42 An Animated Example i 1 k 2 23 98 45 14 6 67 33 42 16
An Animated Example i 1 k 2 23 98 45 14 6 67 33 42 An Animated Example i 1 k 2 23 45 98 14 6 67 33 42 17
An Animated Example i 1 k 3 23 45 98 14 6 67 33 42 An Animated Example i 1 k 3 23 45 98 14 6 67 33 42 18
An Animated Example i 1 k 3 23 45 14 98 6 67 33 42 An Animated Example i 1 k 4 23 45 14 98 6 67 33 42 19
An Animated Example i 1 k 4 23 45 14 98 6 67 33 42 An Animated Example i 1 k 4 23 45 14 6 98 67 33 42 20
An Animated Example i 1 k 5 23 45 14 6 98 67 33 42 An Animated Example i 1 k 5 23 45 14 6 98 67 33 42 21
An Animated Example i 1 k 5 23 45 14 6 67 98 33 42 An Animated Example i 1 k 6 23 45 14 6 67 98 33 42 22
An Animated Example i 1 k 6 23 45 14 6 67 98 33 42 An Animated Example i 1 k 6 23 45 14 6 67 33 98 42 23
An Animated Example i 1 k 7 23 45 14 6 67 33 98 42 An Animated Example i 1 k 7 23 45 14 6 67 33 98 42 24
An Animated Example i 1 k 7 23 45 14 6 67 33 42 98 After First Pass of Outer Loop i 1 k 8 Finished first Bubble Up 23 45 14 6 67 33 42 98 25
The Second Bubble Up i 2 k 1 23 45 14 6 67 33 42 98 The Second Bubble Up i 2 k 1 No 23 45 14 6 67 33 42 98 26
The Second Bubble Up i 2 k 2 23 45 14 6 67 33 42 98 The Second Bubble Up i 2 k 2 23 45 14 6 67 33 42 98 27
The Second Bubble Up i 2 k 2 23 14 45 6 67 33 42 98 The Second Bubble Up i 2 k 3 23 14 45 6 67 33 42 98 28
The Second Bubble Up i 2 k 3 23 14 45 6 67 33 42 98 The Second Bubble Up i 2 k 3 23 14 6 45 67 33 42 98 29
The Second Bubble Up i 2 k 4 23 14 6 45 67 33 42 98 The Second Bubble Up i 2 k 4 No 23 14 6 45 67 33 42 98 30
The Second Bubble Up i 2 k 5 23 14 6 45 67 33 42 98 The Second Bubble Up i 2 k 5 23 14 6 45 67 33 42 98 31
The Second Bubble Up i 2 k 5 23 14 6 45 33 67 42 98 The Second Bubble Up i 2 k 6 23 14 6 45 33 67 42 98 32
The Second Bubble Up i 2 k 6 23 14 6 45 33 67 42 98 The Second Bubble Up i 2 k 6 23 14 6 45 33 42 67 98 33
After Second Pass of Outer Loop i 2 k 7 Finished second Bubble Up 23 14 6 45 33 42 67 98 The Third Bubble Up i 3 k 1 23 14 6 45 33 42 67 98 34
The Third Bubble Up i 3 k 1 23 14 6 45 33 42 67 98 The Third Bubble Up i 3 k 1 14 23 6 45 33 42 67 98 35
The Third Bubble Up i 3 k 2 14 23 6 45 33 42 67 98 The Third Bubble Up i 3 k 2 14 23 6 45 33 42 67 98 36
The Third Bubble Up i 3 k 2 14 6 23 45 33 42 67 98 The Third Bubble Up i 3 k 3 14 6 23 45 33 42 67 98 37
The Third Bubble Up i 3 k 3 No 14 6 23 45 33 42 67 98 The Third Bubble Up i 3 k 4 14 6 23 45 33 42 67 98 38
The Third Bubble Up i 3 k 4 14 6 23 45 33 42 67 98 The Third Bubble Up i 3 k 4 14 6 23 33 45 42 67 98 39
The Third Bubble Up i 3 k 5 14 6 23 33 45 42 67 98 The Third Bubble Up i 3 k 5 14 6 23 33 45 42 67 98 40
The Third Bubble Up i 3 k 5 14 6 23 33 42 45 67 98 After Third Pass of Outer Loop i 3 k 6 Finished third Bubble Up 14 6 23 33 42 45 67 98 41
The Fourth Bubble Up i 4 k 1 14 6 23 33 42 45 67 98 The Fourth Bubble Up i 4 k 1 14 6 23 33 42 45 67 98 42
The Fourth Bubble Up i 4 k 1 6 14 23 33 42 45 67 98 The Fourth Bubble Up i 4 k 2 6 14 23 33 42 45 67 98 43
The Fourth Bubble Up i 4 k 2 No 6 14 23 33 42 45 67 98 The Fourth Bubble Up i 4 k 3 6 14 23 33 42 45 67 98 44
The Fourth Bubble Up i 4 k 3 No 6 14 23 33 42 45 67 98 The Fourth Bubble Up i 4 k 4 6 14 23 33 42 45 67 98 45
The Fourth Bubble Up i 4 k 4 No 6 14 23 33 42 45 67 98 After Fourth Pass of Outer Loop i 4 k 5 Finished fourth Bubble Up 6 14 23 33 42 45 67 98 46
The Fifth Bubble Up i 5 k 1 6 14 23 33 42 45 67 98 The Fifth Bubble Up i 5 k 1 No 6 14 23 33 42 45 67 98 47
The Fifth Bubble Up i 5 k 2 6 14 23 33 42 45 67 98 The Fifth Bubble Up i 5 k 2 No 6 14 23 33 42 45 67 98 48
The Fifth Bubble Up i 5 k 3 6 14 23 33 42 45 67 98 The Fifth Bubble Up i 5 k 3 No 6 14 23 33 42 45 67 98 49
After Fifth Pass of Outer Loop i 5 k 4 Finished fifth Bubble Up 6 14 23 33 42 45 67 98 i 5 k 4 6 14 23 33 42 45 67 98 50
Inner loop Outer loop 2/1/2016 Analysis of Bubble Sort and Loop Invariant Bubble Sort Algorithm Bubble-Sort( A ): for i = 1 (N 1): To do N-1 iterations To bubble a value for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) 51
Inner loop Inner loop Outer loop Outer loop 2/1/2016 Loop Invariant It is a condition or property that is guaranteed to be correct with each iteration in the loop Usually used to prove the correctness of the algorithm for i = 1 (N 1): To do N-1 iterations To bubble a value for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) Loop Invariant for Bubble Sort for i = 1 (N 1): To do N-1 iterations To bubble a value for k = 1 (N 1): if A[k] > A[k+1]: ( A, k, k+1 ) By the end of iteration i the right-most i items (largest) are sorted and in place 52
N - 1 2/1/2016 N-1 Iterations 1 2 3 4 5 6 42 35 12 77 5 101 1 2 3 4 5 6 35 12 42 5 77 101 1 2 3 4 5 6 12 35 5 42 77 101 1 2 3 4 5 6 12 5 35 42 77 101 1 2 3 4 5 6 5 12 35 42 77 101 1 st 2 nd 3 rd 4 th 5 th Correctness of Bubble Sort (using Loop Invariant) Bubble sort has N-1 Iterations Invariant: By the end of iteration i the right-most i items (largest) are sorted and in place Then: After the N-1 iterations The rightmost N-1 items are sorted This implies that all the N items are sorted 53
Bubble Sort Time Complexity Best-Case Time Complexity The scenario under which the algorithm will do the least amount of work (finish the fastest) What if we had a pass and nothing was swapped? What does that tell us? Worst-Case Time Complexity The scenario under which the algorithm will do the largest amount of work (finish the slowest) Summary Bubble Up algorithm will move largest value to its correct location (to the right) Repeat Bubble Up until all elements are correctly placed: Maximum of N-1 times Can finish early if no swapping occurs We reduce the number of elements we compare each time one is correctly placed Not the most efficient algorithm around but is easy to understand 54