class Subscriber
Process events of one Redis node
Uses bliss.data.node classes for listening (gevent architecture)
Event listener and consumer can run in different modes:
class Subscriber
API
node: Redis node (thread/greenlet safe)
info: Redis node info
state: INIT, ON, OFF, FAULT
start(wait=False, timeout=None): INIT -> ON
stop(wait=False, timeout=None): ON -> OFF
kill(wait=False, timeout=None): ON -> FAULT
join(timeout=None): wait until finished (OFF or FAULT)
done(): finished?
duration(): listening time
class Subscriber
Synchronization primitives
Launcher: to execute functions in the start thread
gevent.Queue (concurrent): queue in listener and dequeue in consumer (same thread)
Queue (parallel): queue in listener and dequeue in consumer (different threads)
Thread (parallel): to execute functions in the listener and consumer thread
class NexusScanWriter(Subscriber)
Process events of one Scan node
Consumer saves Scan data in Nexus compliant HDF5 format
When running in parallel mode, HDF5 writing (consumer) and Redis event reception (listener) run in parallel (h5py releases the GIL on IO).
class NexusSessionWriter(Subscriber)
Process events of one Session node
Runs in sequential mode (1 greenlet in the start thread)
Consumer starts a NexusScanWriter in parallel mode for every new scan. It uses one Thread for the consumer greenlets of all scans. The listener greenlets run in the start thread.
A NexusWriter Tango device manages one NexusSessionWriter which is started in the MainThread (server runs in green mode).
NexusWriterService servername --log=info --logfile=...
Primitives that can be shared across threads and greenlets:
Lock, RLock, Event and Queue
Execute function in another thread:
HubTrigger (without arguments and result)
Launcher (with arguments and result)
class HubTrigger
Trigger function executing in a Hub from any thread.
No arguments passing or receiving result/exception.
API
__init__(func, blocking=False)
send(): causes func to be executed in the hub of the instantiating thread (spawns a greenlet for execution when blocking==False)
Synchronization primitives
threading.Lock: protect against coalescence of triggers
class Lock
Can be acquired once by only one greenlet (across threads)
API
acquire(blocking=True, timeout=None)
release()
Synchronization primitives
threading.Lock:
Remark: "loop + gevent.sleep" when acquiring (only location where I have to do this)
class RLock
Can be acquired multiple times by only one greenlet (across threads)
API
acquire(blocking=True, timeout=None)
release()
Synchronization primitives
threading.RLock:
ThreadResource(gevent.lock.RLock): one gevent RLock per thread
class ThreadResource
Resource that dies with the thread and can be accessed in other threads
API
__init__(init_resource): function to generate the resource
get(): retrieve or create/cache resource for the current thread
pop(): remove the resource for the current thread (if cached)
__iter__(): loop over all thread resources
Synchronization primitives
WeakKeyDictionary: weakref(thread) -> resource
Lock: protect WeakKeyDictionary
class Event
Shared across threads and greenlets
API
wait(timeout=None)
send()
clear()
Synchronization primitives
Lock: protect send and clear (as multiple events are involved)
ThreadResource((gevent.event.Event, HubTrigger(ev.set)))
class Queue
Shared across threads and greenlets
API
put(item)
get()
Synchronization primitives
Lock: protect the queue+event
Event: notify the getters of a new item
class Thread
Has methods to execute functions in this thread from any thread.
Has a stop mechanism.
Result or exception from the thread's main function can be retrieved.
class Thread
API
start(wait=True, timeout=None)
stop(wait=True, timeout=None)
join(timeout=None):
get(timeout=None): like join but return value or raise exception
spawn(run=func, *args, **kwargs)
apply(func, *args, **kwargs)
apply_async(func, *args, **kwargs): returns AsyncResult
class Thread
Synchronization primitives
Launcher: to queue execution in this thread
Event: for start and stop
class Launcher
Execute a function in the instantiating thread from any thread.
Pass arguments and retrieve return value or raise exception.
API
apply(func, *args, **kwargs)
apply_async(func, *args, **kwargs): returns AsyncResult
Synchronization primitives
Queue: tasks enqueued by apply or apply_sync
HubTrigger: dequeue and execution in the instantiating thread
AsyncResult: result of apply_async
class AsyncResult
Shared across threads and greenlets
API
set(result): can be an exception
get(timeout=None): raise when result is an exception
wait(timeout=None):
ready():
successful():
Synchronization primitives
Event: notify getters