Standard natural language
isn't great for searching
But the state of the art
can do amazing things!
With these new techniques,
you can search for a movie that's more
Or find a movie that is less
Map-reduce to preprocess & transform the text into n-grams
word2vec trains a neural network, assigning a vector to every word
NumPy, Cython, Numba & Numexpr
for high-performance vector math
word2vec uses a neural network to assign a vector to every word
The neural network estimates the probability that every word in the vocabulary appears around the training word.
Check the sentence to see if the word appears nearby, then backpropagate.
By Christopher Moody