3rdof December, 2021
BeamLine Instrumentation Support Software
Presentation for ACU
Project history and milestones
at the heart of ESRF data acquisition
TUPV043
baked with love by the ESRF Instrumentation Services & Development Division Software Group
Beamline Control Unit: A. Beteva, G. Berruyer, L. Claustre, S. Debionne, S. Fisher, M. Guijarro, P. Guillou, C. Guilloud, A. Homs, M.C. Lagier, D. Mammeri, A. Mauro, J. Meyer, C. Muzelle, M. Oscarsson, E. Papillon, S. Petitdemange, V. Valls, H. Witsch / Data Analysis Unit: W. De Nolf
Beacon : web application to configure a whole beamline
Flint : data visualisation
based on the ESRF silx toolkit
Modular architecture, Abstraction layers and adapters
Data publishing and archiving enables online data analysis
Services built on top of
BLISS integrated environment covers a wide range of beamline control needs
BLISS shell integrates with Daiquiri UI the ESRF web-based framework for user interfaces
Daiquiri UI
see: THAR02
+
BLISS stands for BeamLine Instrumentation Support Software,
it is a new synchrotron experiments control system written in Python
BLISS controls today 24 beamlines (out of 48), full deployment is aimed in 2023
TUPV043
BLISS Key Principles
Direct hardware control
Persistent
settings cache
Data publishing,
Transient storage
Scan acquisition chain represented as a tree
All-in-one solution
BLISS Beacon server
Web interface for configuration editing
Beacon server
Devices & sequences configuration in YAML format
User sessions to group beamline devices for an experiment + Python setup file
configuration management
&
services
Transient data store
Settings cache
Centralized logging and log viewer web application
class: elmo
udp:
url: 160.103.51.174
axes:
- name: nth
acceleration: 36
steps_per_unit: 30577
velocity: 180.0
home_velocity: 15
velocity_low_limit: .inf
velocity_high_limit: 360.0
low_limit: -.inf
high_limit: .inf
sign: 1
backlash: 0.0
- class: aerotech
tcp:
url: id31aerotech-1
axes:
- name: nth_aero
acceleration: 5
steps_per_unit: 1
database implementation
TUPV043
BLISS Modular Architecture
Beacon server
data
streams
Mini frameworks within BLISS, to ease integration of same kind of hardware
Adaptation layer, to make hardware controllers behaving properly in scans, depending on the scan
TUPV043
BLISS data publishing and archiving
Daiquiri web GUI
see: THAR02
- redis streams are used to push data to redis during acquisition
- only references to heavy data (2D) are published to redis, refs will be resolved at runtime when asked by a listener
- online data analysis scripts can be easily written using BLISS Python API
- BLISS fully supports ESRF data policy (based on PaNdata Europe and FAIR data initiatives), and interoperates with ICAT and ESRF data portal
Redis
1
What is Redis ?
- { key: value, } store
- binary safe string
- max 512 MB for a string value
- key can have a data type: List, Set, Sorted Set, Hash (containing multiple string values)
- in memory
- can also persist on disk, but we do not do that for data
- since Redis 6: streams
1
producer
producer
producer
producer
consumer
consumer
consumer
XADD
XREAD
- data publishing using redis streams
- 1 stream per data channel
Redis streams
1
individual string values
we arbitrary limit streams to 2000 string values max
1 string value can correspond to multiple data events
BLISS client API: knows scans structure, read data
Note: Redis data is only transient !
Consumers need to continuously read from it, to get complete data
(Like Nexus Writer for example)
1
References
- Heavy data (== 2D data, up to now) is not stored in redis streams
- References are stored instead
- When receiving a reference, the BLISS client API resolves the reference
- get data from Tango server memory (if still available)
- fallback: reading from disk
References
2
BLISS development tools
Issues list, sprints reviews, discussions and tasks to do on ESRF-hosted gitlab
Continuous Integration:
gitlab-ci
Deployment:
Documentation:
MkDocs
Code formatting
BLISS development methodology
Prioritized
todo
Analysis phase
Coding
Integration tests
Ready to merge
new features
bug fixes
backlog
(tasks)
Kanban