AiiDA-GraphQL

Purpose of an API

  • controlled access to AiiDA data
  • based on AiiDA ORM
  • through web-protocols (HTTP, WS)
  • enable development of client applications without ANY server-side changes

What is REST?

REpresentational State Transfer

Operations

  • Server does not track client state
  • CRUD operations mapped to HTTP verbs

Resources

  • Level of detail to include in Collection
  • N+1 problem .. when to include,
    when to link to other node data
  • Filtering? Sorting?
Collection:    /api/v2/nodes

Resource:      /api/v2/nodes/<id>

An example from AiiDA

https://dev-aiida-dev.materialscloud.org/curated-cofs/api/v4/nodes/page/1?perpage=10&attributes=true&attributes_filter=process_label,process_state,exit_status,exit_message&full_type="process.%25|%25"&orderby=-ctime
{
  "data": {
    "nodes": [
      {
        "attributes": {
          "exit_message": null,
          "exit_status": 0,
          "process_label": "get_pe",
          "process_state": "finished"
        },
        "ctime": "Sun, 20 Oct 2019 11:56:24 GMT",
        "full_type": "process.calculation.calcfunction.CalcFunctionNode.|aiida_lsmo.workchains.isotherm_calc_pe.get_pe",
        "id": 70876,
        "label": "get_pe",
        "mtime": "Fri, 08 Nov 2019 15:20:52 GMT",
        "node_type": "process.calculation.calcfunction.CalcFunctionNode.",
        "process_type": "aiida_lsmo.workchains.isotherm_calc_pe.get_pe",
        "user_id": 2,
        "uuid": "3b209edf-022b-4de5-a314-4b01d76fd6cc"
      },
      {
        "attributes": {
          "exit_message": null,
          "exit_status": 0,
          "process_label": "get_isotherm_output",
          "process_state": "finished"
        },
        "ctime": "Sun, 20 Oct 2019 11:56:23 GMT",
        "full_type": "process.calculation.calcfunction.CalcFunctionNode.|aiida_lsmo.workchains.isotherm.get_isotherm_output",
        "id": 70111,
        "label": "get_isotherm_output",
        "mtime": "Sun, 20 Oct 2019 11:56:23 GMT",
        "node_type": "process.calculation.calcfunction.CalcFunctionNode.",
        "process_type": "aiida_lsmo.workchains.isotherm.get_isotherm_output",
        "user_id": 2,
        "uuid": "cf7cc430-6e8c-4c25-b596-5a72dc7266bc"
      }
    ]
  }
}

"breaks" REST

GraphQL

Query language for APIs and a runtime for fullfulling those queries.

Operations

  • Server does not track client state
  • Operations not mapped to HTTP verbs
  • Protocol-agnostic

Resources

Single endpoint: /api/v2/graphql
    {
      nodes {
        uuid
        mtime
        ctime
        user {
          firstName
          lastName
        }
      }
    }

Linked fields
directly included

User-defined
fields

  • Client defines fields to retrieve
  • No N+1: linked entities directly included

Comparison

https://dev-aiida-dev.materialscloud.org/curated-cofs/api/v4/nodes/page/1?perpage=10&attributes=true&attributes_filter=process_label,process_state,exit_status,exit_message&full_type="process.%25|%25"&orderby=-ctime
{
   nodes(skip: 0, limit: 10,
         orderby:"-ctime") {
     
    ctime
    full_type
    id
    label
    mtime
    node_type
    process_type
    user { id }
    uuid
    
    attributes {
      exit_message
      exit_status
      process_label
      process_state
    }
}

Strawberry-GraphQL PoC

Using Python annotations and dataclasses to define interface.

Python 3.7

Python 3.6

graphql-core 3 support

Using Poetry instead of Pip

@strawberry.type
class Root:
    @strawberry.field
    def node(self, info, uuid: strawberry.ID)
    		-> typing.Optional[Node]:
        q = QueryBuilder()
        q.append(orm.Node,
        	filters={"uuid": {"==": uuid}},
            project=["*"])
        entry = q.first()

        if entry:
            return Node.from_orm(entry)

        return None

    @strawberry.field
    def nodes(self, info) -> typing.List[Node]:
        q = QueryBuilder()
        q.append(orm.Node, project=["*"])
        return [Node.from_orm(entry)
                for entry in q.iterall()]
@strawberry.interface
class Node:
    uuid: strawberry.ID
    ctime: str
    mtime: str
    label: str
    user: User

    @staticmethod
    def from_orm(ormobj):
        try:
            return (
              	DC_REGISTRY[ormobj[0].node_type]
                .from_orm(ormobj))
        except KeyError:
            return BareNode.from_orm(ormobj)

@strawberry.type
class BareNode(Node):
    @staticmethod
    def from_orm(ormobj):
        return BareNode(
            uuid=ormobj[0].uuid,
            ctime=ormobj[0].ctime,
            mtime=ormobj[0].mtime,
            label=ormobj[0].label,
            user=User.from_orm(
              ormobj[0].user),
        )

Annotations

Dataclass

access to GraphQL query

Endpoint specification

Mapping AiiDA ORM to GraphQL entities

"Why?"

"N+1"

      {
        "attributes": {
          "exit_message": null,
          "exit_status": 0,
          "process_label": "get_pe",
          "process_state": "finished"
        },
        "ctime": "Sun, 20 Oct 2019 11:56:24 GMT",
        "full_type": "process.calculation.calcfunction.CalcFunctionNode.|aiida_lsmo.workchains.isotherm_calc_pe.get_pe",
        "id": 70876,
        "label": "get_pe",
        "mtime": "Fri, 08 Nov 2019 15:20:52 GMT",
        "node_type": "process.calculation.calcfunction.CalcFunctionNode.",
        "process_type": "aiida_lsmo.workchains.isotherm_calc_pe.get_pe",
        "user_id": 2,
        "uuid": "3b209edf-022b-4de5-a314-4b01d76fd6cc"
      }

Other advantages

  • Server gets to see which fields are being used
  • .. can adapt backend queries.
    GraphQL → QueryBuilder
{
  nodes {
    uuid
    mtime
    ctime
    user {
      firstName
      lastName
    }
  }
}

Disadvantages

  • HTTP-Proxy based caching not possible anymore
Made with Slides.com