Streams à la carte

Extensible Pipelines with Object Algebras

Aggelos Biboudis¹, Nick Palladinos², George Fourtounis¹, Yannis Smaragdakis¹


University of Athens¹

Nessos Information Technologies²

(under review)

Stream Libraries

  • functional-inspired pipelines 
  • lazy
  • fixed behavior and operators e.g.,
    • C# (LINQ), F#(Seq), Scala(Views) implement Pull-streams
    • Java 8 implement Push-streams
    • Java 8 doesn't accept custom operators

Why un-fix the behavior?

  1. operators naturally push or pull ⇒ variable performance
  2. to mix-in behaviors e.g.:
    • log with push
    • fuse with pull
    • blocking or not with push or pull

avoiding other pathological cases

Iterator<Long> iterator = Stream
    .flatMap(x -> Stream.iterate(0L, i -> i + 2).map(y -> x * y))

iterator.hasNext(); // Out-of-memory :-(

Expression problem

Object Algebras: A design pattern to the rescue

An abstract factory

interface ExpFactory {
  Exp lit(int x);
  Exp add(Exp e1, Exp e2);

A generic factory

interface ExpFactory<Exp> {
  Exp lit(int x);
  Exp add(Exp e1, Exp e2);

An expression

<Exp> Exp mkAnExp(ExpFactory<Exp> f) {
    return f.add(f.lit(1), 
                 f.add(f.lit(2), f.lit(3)));

Algebraic Signatures

signature\ Exp
signature Exp
add : Exp \times Exp \rightarrow Exp
lit : Int \rightarrow Exp

in the Object algebras realm

  • interfaces are named algebras
  • implementations are named factories
  • new cases (by extending the algebra) 
  • new functions (by implementing the algebra)

we propose

  • A library
  • Inspired by Object Algebras
  • Provide extensible streams with:
    • Pluggable operators
    • Pluggable behaviors
    • Mixedin behaviors
  • Affect performance (in a good way)

What is the object algebra of Streams?

interface StreamAlg<C<_>> {
    <T>     C<T> source(T[] array);
    <T, R>  C<R> map(Function<T, R> f, C<T> s);
    <T, R>  C<R> flatMap(Function<T, C<R>> f, C<T> s);
    <T>     C<T> filter(Predicate<T> f, C<T> s);

(for intermediate operators)

What is the object algebra of Streams?

interface ExecStreamAlg<E<_>, C<_>> extends StreamAlg<C> {
  <T> E<Long> count(C<T> s);
  <T> E<T>    reduce(T identity, BinaryOperator<T> acc, C<T> s);

(for terminal operators)

How do you extend streams?

Add new operators (by extending the algebra) 

interface TakeStreamAlg<C<_>> extends StreamAlg<C> { 
    <T> C<T> take(int n, C<T> s);

Add new behavior (by implementing the algebra) 

class PushFactory implements StreamAlg<Push>

Let's use a stream

PushFactory alg = new PushFactory();

int sum = alg.sum(
  -> x * x,
            alg.filter(x -> x % 2 == 0,

Streams a la carte

<E, C> E<Long> cart(ExecStreamAlg<E, C> alg) {
    return alg.reduce(0L, Long::sum,
            alg.flatMap(x -> 
    -> x * y, alg.source(v2)),

Declaring streams: reducing a Cartesian product

cart(new ExecPushFactory()).value;
cart(new ExecPullFactory()).value;
cart(new ExecFusedPullFactory()).value;
cart(new LogFactory<>(new ExecPushFactory())).value;
cart(new LogFactory<>(new ExecPushFactory())).value;
cart(new ExecFutureFactory<>(new ExecPushFactory())).get();
cart(new ExecFutureFactory<>(new ExecPullFactory())).get();

Using streams with various factories

object algebras are for construction

an algebra that fuses maps&filters

sometimes we need fully fledged pull

our pathogenic case from earlier with large nested stream

How did we encode higher-kinded types?

How did we encode higher-kinded types?

public class Id<T> implements App<Id.t, T> {
    public final T value;
    public Id(T value) {
        this.value = value;
    public static <A> Id<A> prj(App<Id.t, A> app) {
        return (Id<A>) app;
    public static <A> App<Id.t, A> inj(Id<A> app) {
        return (App<Id.t, A>) app;
    public static class t {}
interface App<C, T> {} 


  • Id (a type level X => X)
  • Push   (T -> Unit)->Unit
  • Pull (extends Iterator)
  • Future (extends FutureTask)

To sum up

  1. A library implementation
  2. Inspired by Object Algebras
  3. Extensible operators
  4. Pluggable behaviors
  5. Mixedin behaviors
  6. Performance is still there


Thank you


By Aggelos Biboudis