Picnic: put containers into a backpack

by Dmitrii Kovanikov

HASKELL.SG: 6 Sep 2018

About me

What I do

Haskell Adept at Holmusk (present)

Contacts

@ChShersh

@ChShersh

Haskell Software Developer at Serokell (2016-2018)

Haskell Lecturer at the ITMO University (2015-2018)

Dmitrii Kovanikov

Cofounder, Mentor, Developer at Kowainik (free time)

@chshersh

What is Backpack?

Backpack — mixin package system.

Mixin libraries can have signatures which permit implementations of values and types to be deferred, while allowing a library with missing implementations to still be type-checked.

New fields in .cabal files: signatures, reexported-modules, mixins

New file extension: .hsig

Supported currently only by cabal-install (stack issue)

New compiler errors!

Interface for containers

Why we need it?

1. Polymorphic functions.

groupBy :: (Foldable f, Ord k) => (a -> k) -> f a -> Map k (NonEmpty a)

ghci> groupBy even [1..10]
fromList [ (False,  9 :| [7,5,3,1])
         , (True,  10 :| [8,6,4,2])
         ]

2. Benchmarks.

3. Property-based tests for laws (and unit tests as well)

* lookup k (insert k v m) ≡ Just v
* insert k b . insert k a ≡ insert k b
* member k (delete k m)   ≡ False

Challenges

1. Data types are different across libraries (like Map and HashMap).

2. Every library has its own constraints for the keys (Map requires Ord constraint and HashMap requires Eq and Hashable constraints).

3. Types might have different kinds (consider Map and IntMap).

4. Some maps don’t have efficient modification operations since they are based on arrays (Map from primitive-containers package).

5. Different libraries implement the same functions with different constraints.

Typeclass-based solution (1 / 2)

class ( Monoid set
      , MonoFoldable set
      , Eq (ContainerKey set)
      , GrowingAppend set
      )
  => SetContainer set where

    type ContainerKey set

    member    :: ContainerKey set -> set -> Bool
    notMember :: ContainerKey set -> set -> Bool

    union        :: set -> set -> set
    difference   :: set -> set -> set
    intersection :: set -> set -> set

    keys :: set -> [ContainerKey set]

mono-traversable

Typeclass-based solution (2 / 2)

class StaticMap t where
    type Key t :: Type
    type Val t :: Type

    size   :: t -> Int
    lookup :: Key t -> t -> Maybe (Val t)
    member :: Key t -> t -> Bool

class StaticMap t => DynamicMap t where
    insert     :: Key t -> Val t -> t -> t
    insertWith :: (Val t -> Val t -> Val t) -> Key t -> Val t -> t -> t

    delete :: Key t -> t -> t
    alter :: (Maybe (Val t) -> Maybe (Val t)) -> Key t -> t -> t

relude

Problems with typeclass-based solutions?

Backpack solution (1 / 4)

signature Map (Map, Key, empty, alter) where

data Map k v
class Key k

instance (Show k, Show v) => Show (Map k v)

empty :: Map k v
alter :: Key k => (Maybe v -> Maybe v) -> k -> Map k v -> Map k v

Map.hsig

cabal-version:       2.0
name:                containers-sig

library
  signatures:          Map
  build-depends:       base

containers-sig.cabal

Backpack solution (2 / 4)

module Map.Contrib.Group (groupBy) where

import Map (Key, Map)

groupBy :: (Foldable f, Key k) => (a -> k) -> f a -> Map k (NonEmpty a)
groupBy = < ... some implementation ... >

Map/Contrib/Group.hs

cabal-version:       2.0
name:                containers-contrib

library
  exposed-modules:     Map.Contrib.Group
  build-depends:       base, containers-sig

containers-contrib.cabal

Backpack solution (3 / 4)

{-# LANGUAGE ConstraintKinds #-}

module Map.Ord (Map, Key, empty, alter) where

import qualified Data.Map.Strict as M

type Map = M.Map
type Key = Ord

empty :: Map k v
empty = M.empty

alter :: Key k => (Maybe v -> Maybe v) -> k -> Map k v -> Map k v
alter = M.alter

Map/Ord.hs

{-# LANGUAGE ConstraintKinds #-}

module Map.Ord (Map, Key, module Map) where

import Data.Map.Strict as Map

type Key = Ord
cabal-version:       2.0
name:                containers-ordered-strict

library
  exposed-modules:     Map.Ord
  reexported-modules:  Map.Ord as Map

  build-depends:       base, containers

containers-ordered-strict.cabal

Backpack solution (4 / 4)

module Main where

import Map.Contrib.Group (groupBy)

main :: IO ()
main = do
    putStrLn "### Map ###"
    print $ groupBy (`mod` 2) ([1..10] :: [Int])

Main.hs

cabal-version:       2.0
name:                containers-example

executable map-exe
  main-is:             Main.hs
  build-depends:       base
                     , containers-ordered-strict
                     , containers-contrib

containers-example.cabal

Backpack HashMap

{-# LANGUAGE ConstraintKinds #-}



module Map.Hash where

import Data.Hashable (Hashable)
import Data.HashMap.Strict as HM

type Map = HM.HashMap

type Key a = (Eq a, Hashable a)
{-# LANGUAGE ConstraintKinds #-}



module Map.Ord where


import Data.Map.Strict as M

type Map = M.Map

type Key = Ord
{-# LANGUAGE FlexibleInstances    #-}
{-# LANGUAGE MonoLocalBinds       #-}
{-# LANGUAGE UndecidableInstances #-}

module Map.Hash where

import Data.Hashable (Hashable)
import Data.HashMap.Strict as HM

type Map = HM.HashMap

class (Eq k, Hashable k) => Key k
instance (Eq k, Hashable k) => Key k
   • Type constructor ‘Key’ has conflicting definitions in the module
      and its hsig file
      Main module: type Key a =
                     (ghc-prim-0.5.2.0:GHC.Classes.Eq a,
                      hashable-1.2.7.0:Data.Hashable.Class.Hashable a)
                     :: Constraint
      Hsig file:  class Key k
      Illegal parameterized type synonym in implementation of abstract data.
      (Try eta reducing your type synonym so that it is nullary.)
    • while checking that containers-unordered-strict-0.0.0:Map.Hash
      implements signature Map in
      containers-contrib-0.0.0[Map=containers-unordered-strict-0.0.0:Map.Hash]

Using mixins

cabal-version:       2.0
name:                containers-example

executable map-exe
  main-is:        Main.hs
  build-depends:  base
                , containers-ordered-strict
                , containers-unordered-strict
                , containers-contrib
  mixins:         containers-contrib (Map.Contrib.Group as Map.Contrib.Group.Ord)
                            requires (Map as Map.Ord)
                , containers-contrib (Map.Contrib.Group as Map.Contrib.Group.Hash)
                            requires (Map as Map.Hash)

containers-example.cabal

import qualified Map.Contrib.Group.Hash as HM (groupBy)
import qualified Map.Contrib.Group.Ord as M (groupBy)

main :: IO ()
main = do
    putStrLn "### Map ###"
    print $ M.groupBy (`mod` 2) ([1..10] :: [Int])
    putStrLn "### HashMap ###"
    print $ HM.groupBy (`mod` 2) ([1..10] :: [Int])

Main.hs

Backpack IntMap

{-# LANGUAGE DerivingStrategies         #-}
{-# LANGUAGE FlexibleInstances          #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE TypeFamilies               #-}

module Map.Int
       ( Map
       , Key
       , empty
       , alter
       ) where

import qualified Data.IntMap.Strict as M

newtype Map k v = IM { unIM :: M.IntMap v }
    deriving newtype (Show)

type Key = (~) Int

empty :: Map k v
empty = IM M.empty

alter :: Key k => (Maybe v -> Maybe v) -> k -> Map k v -> Map k v
alter f k = IM . M.alter f k . unIM

Map/Int.hs

Problems with IntMap

newtype Map k v = IM { unIM :: M.IntMap v }
    deriving newtype (Show)

type Key = (~) Int

-- Doesn't compile without 'Key' constraint!
toList :: Map k v -> [(k, v)]

Map/Int.hs

Map.hsig

signature Map where

...

toList :: Key k => Map k v -> [(k, v)]

Read-only containers

signature ROMap (Map, Key, lookup, ...)

What doesn't work

Maps and Sets from primitive-containers package don't have modification operations. This means we need to split our signatures across two packages.

What works

signature Map (Map, Key, lookup, ...)

data Map k v
class Key k

lookup :: Key k 
       => k 
       -> Map k v 
       -> Maybe v
signature Map (insert, delete, ...)

import ROMap (Map, Key)

containers-sig-readonly:Map.hsig

containers-sig:Map.hsig

signature Map (Map, Key, delete, ...)

data Map k v
class Key k

delete :: Key k 
       => k 
       -> Map k v 
       -> Map k v

Backpack in the wild

* haskell-backpack/backpack-str: String signatures

* unpacked-containers: ordered containers with unpacked keys

* ezyang/backpack-regex-example: Learning examples

* ezyang/reflex-backpack: Reflex specialized to Spider

* kowainik/containers-backpack: Backpack interface for containers

* hasktorch/hasktorch: Haskell Backpack bindings for PyTorch

How to spend free time

Backpack tutorial: Try Backpack, part 1

Backpack tutorial: Try Backpack, part 2

Picnic: put containers into a backpack

ghc-proposal: Backpack

Backpack Thesis

Questions?

Made with Slides.com