Heap

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Task management

How to manage the processes running in the operating system.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Queue

• Short task may have to wait a long time.
• High priority task cannot get prioritized.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

ArrayList

Have to sort every time a new task arrives.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap (min, max)

Very fast for getting the lowest/highest priority element.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap (min, max)

• Specialized tree-based data structure.
• If A is a parent node of B then the key of node A is ordered with respect to the key of node B with the same ordering applying across the heap
  • min-heap: root is always smaller than children.
  • max-heap: root is always larger than children.
• A common implementation would be binary heap, which is based on complete binary tree.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap (min, max)

       100

       /    \ 

   19     36

   /  \     /  \

17   3  25 1 

           2

       /       \ 

   19        3

   /  \       /  \

20   31  5   8 

max-heap

min-heap

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap (min, max)

• Min-Heap
  • Root is the smallest node of the whole tree.
• Max-Heap
  • Root is the largest node of the whole tree.
• Complete Binary Tree

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Complete Binary Tree

• Every level, (except the last one), has 2^n nodes
• can be represented by array.
  • 0-base:
    • children: n -> 2n+1, 2n+2
    • parent: n -> (n-1)/2
  • 1-base:
    • children: n -> 2n, 2n+1
    • parent: n -> n/2

               0

        /             \

     1                 2

   /    \            /    \ 

 3      4        5       6

 /\    /  \     /  \     /  \

7 8 9 10 11 12 13 14

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert, (offer)
• delete
  • delete root, (poll)
• initialize

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert

                90

          /             \

       70             50

     /      \         /    \

  65      44     30 20

  /  \     / 

35 21 8 

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
  • At most sift up to the root, operation times would be the depth, log(N)
  • O(logN)

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
• delete

                90

          /             \

       70             50

     /      \         /    \

  65      44     30 20

  /  \     / 

35 21 8 

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
• delete
  • After swapping, at most sift down to the leaf, operation time would be the depth, log(N).
  • O(logN).
  • delete root, O(logN)
    • O(1), get the largest/smallest element.
    • O(logN), to moderate.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
• delete
• initialize

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
• delete
• initialize
  • Insert elements one by one, O(NlogN).

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert
• delete
• initialize
  • sift down level by level, O(N)

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Basic Operations

• insert, O(logN)
• delete, O(logN)
• initialize, O(N)
  • Insert elements one by one, O(NlogN).
  • sift down level by level, O(N)

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap in Java - PriorityQueue

• Operations
  • E peek();
  • E poll();
  • boolean offer(E element);
• Constructor
  • PriorityQueue(int capacity)
  • How does Java know how to compare elements in the heap.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Comparator v.s. Comparable

• Comparator
  • noun
  • Helps other objects to compare
• Comparable
  • adjective
  • If some class is comparable, that means its instances know how to compare to other instances.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Comparator v.s. Comparable

• Ten deaf people needs to stand in height-increasing order.
  • If they can see each other's height, problem solved.
  • What if not? There has to be a guy, who is not blind, to help them.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Comparator v.s. Comparable

Copyright © 直通硅谷

http://www.zhitongguigu.com/

class MyClass implements Comparable<MyClass> {
   @Override
   public int compareTo(final MyClass o) {
       return this.val - o.val; // increasing (minHeap)
   }
}

PriorityQueue<MyClass> heap = new PriorityQueue<>(capacity);

Comparator v.s. Comparable

Copyright © 直通硅谷

http://www.zhitongguigu.com/

class MyComparator implements Comparator<MyClass> {
   @Override
   public int compare(MyClass a, MyClass b) {
       return a.val - b.val; // increasing (minHeap)
   }
}
MyComparator myComparator = new MyComparator();
PriorityQueue<MyClass> heap 
    = new PriorityQueue<>(capacity, myComparator);

Comparator v.s. Comparable

Copyright © 直通硅谷

http://www.zhitongguigu.com/

PriorityQueue<MyClass> heap = 
    new PriorityQueue<MyClass>(cap, new Comparator<MyClass>() {
    @Override
    public int compare(MyClass a, MyClass b) {
         return a.val - b.val;  // increasing (minHeap)
    }
});

Add Elements into Heap

• Default: min-heap (PriorityQueue)
• If need max-heap
  • Comparator & Comparable
  • add (-num) into Heap

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap Summary

• insert, O(logN)
• delete, O(logN)
• initialize, O(N)
• Good at maintaining a dynamic data stream, from which largest/smallest is always needed, while there is no need for sorting.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Merge Sorted Array/List

• Two sorted array/list
  • Two pointer, move the smaller one backward.
• K sorted array/list
  • Impossible to manually maintain k pointers.
  • Data structure to maintain k pointers and capable of returning the largest/smallest element with high efficiency.
  • Heap

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Merge k Sorted List

Merge k sorted linked lists and return it as one sorted list. Analyze and describe its complexity.

public ListNode merge(ListNode[] lists);

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Merge k Sorted List

Merge k sorted linked lists and return it as one sorted list. Analyze and describe its complexity.

Insert all head nodes into minHeap.

while (minHeap is not empty)

     root = minHeap.pop();

     Add root into result

     Insert root.next into minHeap.

return result

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Merge k Sorted List

public ListNode mergeKLists(ListNode[] lists) {
    if (lists == null || lists.length == 0)
        return null;
    PriorityQueue<ListNode> minHeap = new PriorityQueue<ListNode>(
            lists.length,
            new Comparator<ListNode>() {
        @Override
        public int compare(ListNode node1, ListNode node2) {
            return node1.val - node2.val;
        }
    });
    
    for (int i = 0; i < lists.length; i++) {
        if (lists[i] != null) {
            minHeap.add(lists[i]);
        }
    }
    
    ListNode dummy = new ListNode(-1);
    ListNode cur = dummy;
    while (!minHeap.isEmpty()) {
        cur.next = minHeap.poll();
        cur = cur.next;
        if (cur.next != null)
            minHeap.add(cur.next);
    }
    return dummy.next;
}

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Find Maximum Number

• O(N) to find maximum
• What if we need to update the list very often, that said, when the maximum is removed, a new number will come instantly.
  • Need to compare n times again. O(N)
  • Sort? Need to sort every time.
    • O(NlogN) for the first time and O(N) every time.
    • Inserting number is very time consuming.
  • Heap.
    • Good at dealing with data stream.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Sliding Window Maximum

Given an array nums, there is a sliding window of size k which is moving from the very left of the array to the very right. You can only see the k numbers in the window. Each time the sliding window moves right by one position.

For example,

Given nums = [1,3,-1,-3,5,3,6,7], and k = 3.

Return [3,3,5,5,6,7].

Note:

You may assume k is always valid, ie: 1 ≤ k ≤ input array's size for non-empty array.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Sliding Window Maximum

Copyright © 直通硅谷

http://www.zhitongguigu.com/

public int[] maxSlidingWindow(int[] nums, int k) {
    if (nums == null || nums.length == 0) {
        return nums;
    }
    int[] result = new int[nums.length - k + 1];
    PriorityQueue<Integer> maxHeap = new PriorityQueue<>(k, 
        new Comparator<Integer>() {
            public int compare(Integer a, Integer b) {
                return b - a;
            }
        });
    for (int i = 0; i < nums.length; i++) {
        if (i >= k) {
            maxHeap.remove(nums[i-k]);
        }
        maxHeap.offer(nums[i]);
        if (i >= k - 1) {
            result[i - k + 1] = maxHeap.peek();
        }
    }
    return result;
}

Sliding Window Maximum

Copyright © 直通硅谷

http://www.zhitongguigu.com/

public int[] maxSlidingWindow(int[] nums, int k) {
    if (nums == null || nums.length == 0) {
        return nums;
    }
    int[] result = new int[nums.length - k + 1];
    PriorityQueue<Integer> minHeap = new PriorityQueue<>(k);
    for (int i = 0; i < nums.length; i++) {
        if (i >= k) {
            minHeap.remove(-nums[i-k]);
        }
        minHeap.offer(-nums[i]);
        if (i >= k - 1) {
            result[i - k + 1] = -minHeap.peek();
        }
    }
    return result;
}

Sliding Window Maximum

Copyright © 直通硅谷

http://www.zhitongguigu.com/

public int[] maxSlidingWindow(int[] nums, int k) {
    if (nums == null || nums.length == 0) {
        return nums;
    }
    Deque<Integer> deque = new LinkedList<>();
    int[] result = new int[nums.length - k + 1];
    for (int i = 0; i < nums.length; i++) {
        if (!deque.isEmpty() && deque.peekFirst() == i - k) {
            deque.poll();
        }
        while (!deque.isEmpty() && nums[deque.peekLast()] < nums[i]) {
            deque.pollLast();
        }
        deque.offer(i);
        if (i >= k - 1) {
            result[i - k + 1] = nums[deque.peek()];
        }
    }
    return result;
}

Kth Largest Element in an Array

Find the kth largest element in an unsorted array. Note that it is the kth largest element in the sorted order, not the kth distinct element.​​

For example,
Given [3,2,1,5,6,4] and k = 2, return 5.

Note:
You may assume k is always valid, 1 ≤ k ≤ array's length.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Kth Largest Element in an Array

Find the kth largest element in an unsorted array. Note that it is the kth largest element in the sorted order, not the kth distinct element.​​

• Find largest element for k times. O(kn)
• Sort. O(nlogn)
• Min-Heap
  • Build a min heap with capacity of k.
  • Insert all numbers into the heap.
    • If the heap is full, then only insert when number is larger than the root.
  • Root of the heap will be the kth largest element.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Kth Largest Element in an Array

Find the kth largest element in an unsorted array. Note that it is the kth largest element in the sorted order, not the kth distinct element.​

public int findKthLargest(int[] nums, int k) {
    Comparator<Integer> comparator = new Comparator<Integer>() {
        public int compare(Integer a, Integer b) {
            return a - b;
        }
    };
    
    PriorityQueue<Integer> minHeap = new PriorityQueue(k, comparator);
    for (int i = 0; i < k; i++) {
        minHeap.add(nums[i]);
    }
    for (int i = k; i < nums.length; i++) {
        if (nums[i] >= minHeap.peek()) {
            minHeap.poll();
            minHeap.add(nums[i]);
        }
    }
    return minHeap.poll();
}

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Kth Largest Element in an Array

Find the kth largest element in an unsorted array. Note that it is the kth largest element in the sorted order, not the kth distinct element.​​

• Find largest element for k times. O(kn)
• Sort. O(nlogn)
• Min-Heap. O(nlogk)
• Recursion. O(n)
  • Randomly select one number, find its position in array. O(n)
  • If pos > k, then recursion in left half.
  • Otherwise, recursion in right half.
  • Time: n + n/2 + n/4 + .... = 2n = O(n)

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Find Median in Data Stream

Median is the middle value in an ordered integer list. If the size of the list is even, there is no middle value. So the median is the mean of the two middle value.

Examples: 
[2,3,4] , the median is 3

[2,3], the median is (2 + 3) / 2 = 2.5

Design a data structure that supports the following two operations:

- void addNum(int num) - Add a integer number from the data stream to the data structure.
- double findMedian() - Return the median of all elements so far.

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Find Median in Data Stream

Median is the middle value in an ordered integer list. If the size of the list is even, there is no middle value. So the median is the mean of the two middle value.

class MedianFinder {

   public void addNum(int num);

   public double findMedian();

}

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Find Median in Data Stream

Median is the middle value in an ordered integer list. If the size of the list is even, there is no middle value. So the median is the mean of the two middle value.

minHeap

(big nums)

maxHeap

(small nums)

• size of minHeap and maxHeap should be the same (diff <= 1).
• minHeap.peek > maxHeap.peek

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Find Median in Data Stream

Median is the middle value in an ordered integer list. If the size of the list is even, there is no middle value. So the median is the mean of the two middle value.

minHeap

(big nums)

maxHeap

(small nums)

Median

• If num > minHeap.peek
  • Add into minHeap
• Otherwise
  • Add into maxHeap
• Maintain size diff

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Heap Summary

• Data Stream
  • Need to be maintained/update all the time.
• Max or min is needed, but no need for sorting
• Other use cases
  • Heap sort

Copyright © 直通硅谷

http://www.zhitongguigu.com/

Homework 18

• Sort Characters By Frequency
• Top K Frequent Elements
• Design Twitter

Copyright © 直通硅谷

http://www.zhitongguigu.com/