Transducers

Before We Start

:dependencies [[org.clojure/clojure "1.7.0-alpha1"]
               [org.clojure/core.async "0.1.338.0-5c5012-alpha"]]

Reduce

https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/reduce

Reduce

function

Usage: (reduce f col)

             (reduce f val col)

Overly simple example:

(reduce + [1 2 3 4 5])

Reducing Function

This time less trivially:

(defn word-counts 
  [running-counts next-word]
  (update-in running-counts [next-word] 
    (fnil inc 0)))

user=> (word-counts {"a" 1} "a")
{"a" 2}

user=> (word-counts {"a" 1} "b")
{"b" 1, "a" 1}

user=> (reduce word-counts {} (str/split "that man is not that great , man" #" "))
{"," 1, "great" 1, "not" 1, "is" 1, "man" 2, "that" 2}

A reducing function takes the prior reduction, the next element, and returns a new redution

More Reducing Functions

(defn word-counts 
  [running-counts next-word]
  (update-in running-counts [next-word] 
    (fnil inc 0)))

Rich calls this fn signature
  whatever, input -> whatever

But this doesn't tell the full story, because whatever in the input can be a totally different thing than the whatever in the output.

Reducing FNs Are General

CAN EXPRESS ALL SORTS OF THINGS LIKE MAP AND FILTER

(defn myfilter 
  [predicate] 
  (fn [running next] 
    (if (predicate next) 
      (conj running next) 
      running)))

user=> (reduce (myfilter #(= 2 %)) [] [1 2 3 4 3 2 1])
[2 2]

Write map as an exercise.

But At The End of the Day, We're coupled!

(defn myfilter 
  [predicate] 
  (fn [running next] 
    (if (predicate next) 
      (conj running next) 
      running)))

Do We Have To Be?

(defn myfilter2
  [predicate] 
  (fn [reducing]
    (fn [running next] 
      (if (predicate next) 
        (reducing running next) 
        running))))

Back To Rich's function signatures:

A *transducer* is 
  reducing fn -> reducing fn
Or as he puts it
  (whatever, input -> whatever) -> (whatever, input -> whatever)

Simple Transducer Example

This satisfies the function signature Rich gave.

(defn lower-words 
  [reducing]
  (fn [whatever input]
    (reducing whatever (.toLowerCase input))))

(require '[clojure.string :as str])

user=> (reduce (lower-words word-counts) {} 
         (str/split "That man is not that great , man" #" "))
{"," 1, "great" 1, "not" 1, "is" 1, "man" 2, "that" 2}

Wait If these All Share a signature...

Then yes! We can comp them!

(reduce 
  ((comp
     lower-words (myfilter2 #(= "that" %))) 
    word-counts)
  {} (str/split "That man is not that great , man" #" "))


{"that" 2}

Are We There?

Yes.

Also note that we aren't coupled to a data type, that's in the reducing fn.  So we can generally apply this to all kinds of stuff.

Now With Transduce

(transduce (map #(.toLowerCase %)) 
           word-counts {} 
           (str/split "That man is not that great , man" #" "))

(time 
  (doseq [x (range 1000)] 
    (->> (str/split "That man is not that great , man" #" ") 
         (map #(.toLowerCase %))
         (reduce word-counts {}))))
"Elapsed time: 57.015 msecs"

(time 
  (doseq [x (range 1000)] 
    (transduce (map #(.toLowerCase %)) 
               word-counts {} 
               (str/split "That man is not that great , man" #" "))))

"Elapsed time: 47.94 msecs"

Also a performance gain (maybe)

Also works with core.async

(require '[clojure.core.async :as async])

(let [channel (async/chan 2 (map #(.toLowerCase %)))] 
  (async/go 
    (async/>! channel "Hello World!") 
    (prn (async/<! channel))))

#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@75acd47>
"hello world!"

You TRy One...

Write FizzBuzz with transducers.  This is not exactly an intuitive fit, but you can write a FizzBuzz, a Fizz, and a Buzz, and cond them for this.