BLOOM FILTERS

• Relax the problem constraints - what if a query can return "no" or "maybe"?
• Hash each value with N hash functions
• For practical purposes, can cheat w/ just 1 underlying hash
• Insert!

bloom filters cont'd

• Tradeoff between false positive rate and size of underlying bit array - 1% error rate requires only around 9.6 bits per element, 0.1% error rate only needs 4.8 more
• add, test both constant-time unlike any other constant-space set data structure 
• Never 'fills up', just reports more false pos

uses

• Cache filtering for content delivery networks - avoid caching one-hit wonders
• Aforementioned browser malicious URL detection - only double-check on a hit
• 'molecular fingerprints' used to search huge databases of chemical structures
• Can be extended to allow for removing items from the filter

Hyper log log

• Try to estimate # of times coin was flipped by length of longest run of heads or tails

• Hash the incoming data, look for runs

• Bucket many estimates, take harmonic mean of the results

• First N bits of hash determine bucket

• i.e. 01001001011000101 -> bucket 5, run of 3 zeroes = if buckets[5] < 4, buckets[5] = 4