MASSIF (UPBL10), high-throughput MX sample characterization

ID30B

ID30A3

ID30A2

ID30A1

MASSIF (UPBL10), high-throughput MX sample characterization

ID30A-1

MASSIF (UPBL10), high-throughput MX sample characterization

ID30A-1

Staubli arm used both as sample changer and goniometer

Oscillation synchronisation done with ESRF MUSST card (Ethernet GPIB)

All axis are Icepap-driven

MASSIF (UPBL10), high-throughput MX sample characterization

ID30A-1

"Big dewar"

3 cells of 8 pucks, 10 SPINE samples each: 240 samples

1 Icepap-driven axis for cell selection

Keyence cameras for detection

Wago modules for operation

MASSIF (UPBL10), high-throughput MX sample characterization

ID30A-3

Dectris Pilatus 3 2M (will be replaced by Eiger 4M, when Eiger will be ready)

KB mirror with NF8753-driven motors

MD2M diffractometer (MD2 little brother with Icepap-driven axes,  Galil DMC213-driven axis for omega)

EMBL SC3 sample changer

Other parts are similar to ID30A-1

MASSIF (UPBL10), high-throughput MX sample characterization

ID30B, collaboration with

Dectris Pilatus 6M 

Tunable wavelength, fluorescence detector

Bended single mirror, IcePAP driven

MD2S diffractometer (latest EMBL development)

EMBL SC3 sample changer + new EMBL sample changer and plate robot

ESRF Big Dewar

MASSIF software components map

 Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

Antoine de Saint-Exupéry

We don't need

MAGIC

Configuring with Beacon

M. Guijarro, S. Petitdemange

Beacon, brilliant Beamline Configuration

Client/server architecture

Objects configuration repository

Settings cache based on

Channels: peer-to-peers data exchange between Beacon clients

Distributed Lock Manager

Drop-in replacement for Tango database (experimental)

Beacon objects configuration repository

A set of .yml files organized in directories

A file can contain multiple objects

Each object must have an unique name

Beacon objects configuration repository

bacon:~/local/beamline_control/configuration % tree
.
├── calibrated_diodes.dat
├── eh
│   ├── bpms.yml
│   ├── detector_cover.yml
│   ├── fshutter.yml
│   ├── i0i1.yml
│   ├── __init__.yml
│   ├── MD2S.yml
│   ├── motors
│   │   ├── DtoX.yml
│   │   ├── ehtable.yml
│   │   ├── fshut.yml
│   │   ├── __init__.yml
│   │   ├── mirror.yml
│   │   ├── slitbox.yml
│   │   └── ss2lits.yml
│   ├── musst.yml
│   ├── tango_shutters.yml
│   ├── transmission.yml
│   └── wagos.yml
├── __init__.yml
├── oh
│   ├── bpms.yml
│   ├── frontend.yml
│   ├── __init__.yml
│   ├── mbv1_diode.yml
│   ├── monocal_diode.yml
│   ├── motors
│   │   ├── energy_wl.yml
│   │   ├── __init__.yml
│   │   ├── mono.yml
│   │   ├── pslits.yml
│   │   ├── ss1lits.yml
│   │   ├── transfocator.yml
│   │   └── undulators.yml
│   ├── transfocator.yml
│   └── wagos.yml
├── percent.arr
└── undulators.dat

Beacon objects configuration repository

Objects are singletons

By default, a dict-like object is returned

The plugin key allow to specify particular functions for object instantiation

        - compatible with any library

Setting keys in __init__.yml assign properties to a whole set of objects in a directory

Beacon settings cache

Beacon client API offers various Settings objects

Settings are a thin helper layer on top of Redis structures

>>> from beacon import settings
>>> d = settings.HashSetting("example")
>>> d
<HashSetting name=example value={}>
>>> d["foo"]="bar"
>>> d
<HashSetting name=example value={'foo': 'bar'}>
>>> 

linguijarro:~ % redis-cli
127.0.0.1:6379> keys *
1) "example"
127.0.0.1:6379> hgetall example
1) "foo"
2) "bar"
127.0.0.1:6379> 

Server-side

Client-side

Beacon channels

Peer-to-peers data exchange between clients connected to the same beacon server

Serialization with standard "pickle" Python module

Designed for small amount of data, ie. states, scalar values, simple strings

Under the hood: nanomsg (SURVEY and BUS scalability protocols)

+ gevent

Beacon channels example

Client A

>>> from beacon import channels
>>> c = channels.Channel("test", "hello, world")
>>> c.value
'hello, world'
>>> 

Client B

>>> from beacon import channels
>>> c = channels.Channel("test")
>>> c.value
'hello, world'
>>> c.value = 42
>>>

>>> c.value
42

Client C

>>> from beacon import channels
>>> c = channels.Channel("test")
>>> c.value
42
>>> c.value = "unlimited power"

>>> c.value
'unlimited power'

Positioning with the ESRF Motion (Emotion) library

C. Guilloud, M. Guijarro, M. Perez

Emotion features

• 4 well-defined base classes: Controller, Axis, Group & Encoder
• powered by gevent
• Virtual (pseudo) axes
• State of the art control: software limits, user positions, backlash handling, smooth stopping, stepper motors under-resolution move compensation
• Events
• Easy to configure, multiple back-ends support
• Mockup object for simulation / testing
• Unit tests suite, 70% code coverage
• Comes with a generic Tango server

Default back-end is XML

Beacon back-end

• integration with full beamline configuration
• Beacon plugin to get Emotion objects from Beacon

Emotion configuration with Beacon

bacon:~/local/beamline_control/configuration % tree
.
├── calibrated_diodes.dat
├── eh
│   ├── bpms.yml
│   ├── detector_cover.yml
│   ├── fshutter.yml
│   ├── i0i1.yml
│   ├── __init__.yml
│   ├── MD2S.yml
│   ├── motors
│   │   ├── DtoX.yml
│   │   ├── ehtable.yml
│   │   ├── fshut.yml
│   │   ├── __init__.yml
│   │   ├── mirror.yml
│   │   ├── slitbox.yml
│   │   └── ss2lits.yml
│   ├── musst.yml
│   ├── tango_shutters.yml
│   ├── transmission.yml
│   └── wagos.yml
├── __init__.yml
├── oh
│   ├── bpms.yml
│   ├── frontend.yml
│   ├── __init__.yml
│   ├── mbv1_diode.yml
│   ├── monocal_diode.yml
│   ├── motors
│   │   ├── energy_wl.yml
│   │   ├── __init__.yml
│   │   ├── mono.yml
│   │   ├── pslits.yml
│   │   ├── ss1lits.yml
│   │   ├── transfocator.yml
│   │   └── undulators.yml
│   ├── transfocator.yml
│   └── wagos.yml
├── percent.arr
└── undulators.dat

Emotion Virtual Axes

Example: 2-leg table, can translate and rotate

controller:
    class: tabsup
    d1: 730
    d2: 1208
    axes:
        -
            name: $tyf
            tags: real front
        -
            name: $tyb
            tags: real back
        -
            name: ttrans
            tags: ttrans
        -
            name: tzrot
            tags: trot

Configuration (Beacon)

$ indicates a reference to another object

Supported motor controllers

Hardware

PI E517, PI E712, PI E753

Galil DMC 213

PMD 206

NewFocus 8753

FlexDC

IcePAP

Virtual

Slits

KB mirror

2-leg table

3-leg table

Energy

Software

Tango Undulator

MD2 Exporter (for MD2 motors)

PiezoJack

Add new controllers easily

# mandatory methods
def initialize_axis(axis)
def start_one(motion)
def stop(axis)
def state(axis)
def read_position(axis)

# optional
def read_velocity(axis)
def set_velocity(axis)
def read_acceleration(axis)
def set_acceleration(axis, new_acc)
def set_position(axis, new_position)

def start_all(motion)
def stop_all()

def set_on(axis)
def set_off(axis)

def initialize_encoder(encoder)
def read_encoder(encoder)
def set_encoder(encoder, new_value)

def home_search(axis)
def home_set_hardware_position(axis, home_pos)
def home_state(axis)

def limit_search(axis, limit)

MASSIF beamlines are the first ESRF beamlines without spec

MXCuBE is the main software for users

But...

What about beamline commissioning ?

Doing calibration and alignment procedures ?

Testing the beamline ?

Introducing

K _ _ _ _ s

A. Beteva,

M. Guijarro,

S. Petitdemange

Khoros, MASSIF sequencer and beamline control software

Beamline components

Scanning

Sequences

• Python functions
• Helpers: cleanup context manager, Task abstraction

• Step by step: timescan, "ascan", "dscan", "a2scan", "d2scan"
• Data Manager, writes different file formats (spec, HDF5, ...)

• High-level objects (actuator, shutter, diffractometer, pixel detector, etc.)
• Configuration with Beacon (Khoros Beacon plugin)
• Counters & measurement groups
• Motors via Emotion

Khoros library

>>> from khoros import setup
>>> setup("/home/guijarro/local/test_setup.py")
Reading setup file '/home/guijarro/local/test_setup.py`
Initializing 'i0`
Initializing 'm0`
Initializing 'm1`
Done.
True
>>> DEFAULT_OFFSET
0.0
>>> add(3,6)
9
>>> m0.move(1)
>>> m0.position()
1.0
>>> ascan(m0,1,2,10,0.1,i0)
>>> last_scan_data()
array([[  1.        ,   0.375     ],
       [  1.11111111,   5.625     ],
       [  1.22222222,  -7.        ],
       [  1.33333333,   8.625     ],
       [  1.44444444, -25.5       ],
       [  1.55555556,   0.125     ],
       [  1.66666667,  -8.125     ],
       [  1.77777778,   2.28571429],
       [  1.88888889,  -3.125     ],
       [  2.        ,  10.625     ]])
>>>

# Khoros setup file
# -----------------
# nothing is imported by default,
# let's import basic scans
from khoros.core.scans import *

# basically any Python code can be
# imported; objects goes to the
# 'setup_globals' namespace.
# When setup is executed from the
# interpreter, objects are exposed in
# the global namespace too

# for demonstration purpose let's
# define some functions
def double(n):
  return 2*n

def add(x,y):
  return x+y

# Beamline components are loaded prior
# to setup file execution, so it's
# possible to use beamline components here:
DEFAULT_OFFSET = m0.position()

test_setup.py

Khoros Beamline Components

from khoros.core.measurement import CounterBase, AverageMeasurement
import PyTango.gevent
import time

class tango_attr_as_counter(CounterBase):
    def __init__(self, name, config):
        CounterBase.__init__(self, name)
        tango_uri = config.get("uri")
        self.__control = PyTango.gevent.DeviceProxy(tango_uri)
        self.attribute = config.get("attr_name")

    def read(self, acq_time=None):
        meas = AverageMeasurement()
        for reading in meas(acq_time):
            reading.value = self.__control.read_attribute(self.attribute).value
        return meas.average

No framework burden: the only requirement is to have 'name' (string) and 'config' (dict) as constructor arguments

Some helper base classes are provided

Khoros Sequences

from khoros import *
from khoros.setup_globals import *
import logging
import sys
import numpy as np

def quick_realign(slits={s1v:0.2}, cnt=i0, mot=thgt, npts=40, start=-0.2, stop=0.2):    
    def restore_slits(motor_pos = [(m, m.position()) for m in slits.iterkeys()):
        for m, pos in motor_pos:
            logging.info("Restoring '%s` to %f", m.name, pos)
            m.move(pos)

    def quick_realign_cleanup(transmission = attenuators.get_transmission()):
        fast_shutter.close()   
        safshut.close()
        attenuators.set_transmission(transmission)
       
    with cleanup(quick_realign_cleanup):
        with cleanup(restore_slits):
            for m, pos in slits.iteritems():
               m.move(pos, wait=False)        
        
            safshut.open()
            fast_shutter.open()
            attenuators.set_transmission(100)
            flux = cnt.read()
            logging.info("Flux before quick realign: %f", flux)
                
            dscan(mot, start, stop, npts, 0.1, cnt)
            data = last_scan_data()
            max_pos = data[np.argmax(data[:,1]),0]

            mot.move(max_pos)
        centrebeam()
        flux = cnt.read()
        logging.info("Flux after quick realign: %f", flux)
        

Khoros Shell

Web application, a la IPython Notebook (Jupyter)

Khoros Shell architecture

bottle + gevent

gipc: gevent-cooperative child processes and inter-process communication

Khoros Shell architecture

Shell is part of the package, but it is an optional component

Nice proof-of-concept, with only 800 Javascript LOC (shell.js + synoptic.js),  700 Python LOC (shell.py + interpreter.py)

Currently in use in production on ID30B

Dependencies list

JavaScript: jQuery 2.1, jQueryUI, jQuery Layout, MouseTrap, Dygraph, CodeMirror

Python: bottle, gevent, gipc

Khoros integration within MXCuBE 2

Straightforward with dedicated Hardware Objects

One "Khoros" Hardware Object is used as a "footbridge" for others to get access to Khoros exported objects

from HardwareRepository.BaseHardwareObjects import HardwareObject
import os
import sys
import khoros

class Khoros(HardwareObject):
  def __init__(self, *args):
    HardwareObject.__init__(self, *args)

  def init(self, *args):
     setup_file = self.getProperty("setup_file")

     khoros.setup(setup_file, self.__dict__)