Implementing a VMware Client with Extensible Records

by Matt Russell

What Makes up a Virtual Machine?

name
description
number of cpus
memory in megabytes
status (running, off, suspended, etc)
....

Interacting with VMware

<retrievePropertiesRequest>
  <ref>vm-444</ref>
  <properties>
    <property>name</property>
    <property>hardware</property>
  </properties>
</retrievePropertiesRequest>

Request

Response

<retrievePropertiesResponse>
  <ref>vm-444</ref>
  <properties>
    <name>My-VM</name>
    <hardware>
      <numCpus>4</numCpus>
      <memoryMb>4096</memoryMb>
    </hardware>
  </properties>
</retrievePropertiesResponse>

What should our Virtual Machine (VM) API look like?

Express all of the possible VM fields
Query only for the fields we care about
Without Maybe for every field

Building Blocks of the VMware Client

Vinyl: Extensible record library using type level lists
Composite: Group of libraries focusing on making extensible records easy to work with
HaXml: Utilities for parsing, filtering, transforming and generating XML documents

Our Virtual Machine Type

-- From Vinyl
data Rec :: (u -> *) -> [u] -> * where
  RNil :: Rec f '[]
  (:&) :: !(f r) -> !(Rec f rs) -> Rec f (r ': rs)

-- From Composite
newtype (:->) (s :: Symbol) a = Val { getVal :: a }
pattern (:*:) :: a -> Rec Identity rs -> Rec Identity ((s :-> a) ': rs)

let vm1 :: Rec Identity '[Name, Description, Hardware]
    vm1 = "my-vm" :*: "Matt's awesome VM" :*: (Hardware 6 4096) :*: RNil

>>> view name vm1
>>> "my-vm" :: Text

type Name        = "name"        :-> Text
type Description = "description" :-> Text
type Hardware    = "hardware"    :-> Hardware

HaXml Schema Generation

class SchemaType a where
  parseSchemaType :: XMLParser a
  schemaTypeToXML :: a -> XMLContent

data Hardware = {
  numCpus  :: Int,
  memoryMb :: Int
}

instance SchemaType Hardware where
  parseSchemaType = ...
  schemaTypeToXML = ...

Use HaXml to auto generate Haskell types from XSD files

Our Ideal API

let vm1 :: Rec Identity '[Name, Description, Hardware]
    vm1 = retrieveProperties "vm-444" (Proxy @'[Name, Description, Hardware])

    vm2 :: Rec Identity '[VMName, VMHardware]
    vm2 = retrieveProperties "vm-444" (Proxy @'[VMName, VMHardware]

-- description is a lens from the extensible record to the description field
>>> view description vm1
>>> "Matt's awesome VM"

>>> view description vm2 --- type error!

Retrieve properties takes a proxy for the fields we want to request

Implementing Retrieve Properties

-- From Vinyl
class RecApplicative rs where
  rpure :: (forall x. f x) -> Rec f rs

recordToList :: Rec (Const a) rs -> [a]

-- From Composite (slight oversimplification)
class ReifyNames (rs :: [*]) where
  reifyNames :: Rec f rs -> Rec (Const Text) rs

recFromProxy :: forall rs proxy . RecApplicative rs 
             => proxy rs -> Rec (Const ()) rs
recFromProxy _ = rpure $ Const ()

fieldNames :: forall rs proxy . (ReifyNames rs, RecApplicative rs, Functor f) 
           => proxy rs -> [Text]
fieldNames = recordToList . reifyNames . recFromProxy

>>>> fieldNames (Proxy @'[Name, Description, Hardware])
>>>> ["name", "description", "hardware"]

Determining the "names" of the fields

Implementing Retrieve Properties

class RecordParserFromSchema rs where
  recordParserFromSchema :: Rec XmlParser rs

instance forall s a (rs :: [*]) 
         . (KnownSymbol s, SchemaType a, RecordParserFromSchema rs) 
         => RecordParserFromSchema (s :-> a ': rs) where
  recordParserFromSchema = (Val <$> parseSchemaType) :& recordFromSchema

instance RecordParserFromSchema '[] where
  recordParserFromSchema = RNil

-- From Vinyl
rtraverse :: Applicative h 
          => (forall x. f x -> h (g x)) -> Rec f rs -> h (Rec g rs)

propertiesParser :: RecordParserFromSchema rs 
                 => proxy rs -> XmlParser (Rec Identity rs)
propertiesParser _ = rtraverse (Identity <$>) recordParserFromSchema

Determining the parsers of the fields

Implementing Retrieve Properties

propertiesParser :: RecordParserFromSchema rs 
                 => XmlParser (Rec Identity rs)

fieldNames       :: forall rs proxy 
                 . (ReifyNames rs, RecApplicative rs, Functor f) 
                 => proxy rs -> [Text]

retrieveProperties :: (RecordParserFromSchema rs, ReifyNames rs, RecApplicative rs) 
                   => proxy rs -> Text -> IO (Either ParseError (Rec Identity rs))
retrieveProperties p ref = do

  let requestFields  = fieldNames p
      responseParser = propertiesParser p

  resp <- sendRequest ref requestFields
  return (responseParser fieldNames)

Putting it all together

Other cool things in the VMware Client

References to VMs (and other objects) are newtypes with safe constructors
Type safety to ensure we can only request properties for fields on the objects
Using HaXml to autogenerate types and endpoints
Using Classy Lenses to work with VMware's OO API

Thank You!

Let's chat at ICFP!

or connect after:

matt@simspace.com

@mrussell247