Doocspie

The doocspie subpackage provides modules with the classes and functions for higher level I/O operations with the DOOCS control system. The contents of this subpackage is presented below:

import doocspie


help(doocspie.doocspie)

Help on package doocspie.doocspie in doocspie:

NAME
    doocspie.doocspie - Classes and functions for higher level I/O operations.

PACKAGE CONTENTS
    doocspie_exception
    history
    io
    meta
    readout

FILE
    /home/cbehrens/Home/Repositories/gitlab/doocspie/doocspie/doocspie/__init__.py

In the following, we show how to list available DOOCS members and perform I/O operations via setter/getter functions. The involved data types are introduced and exception handling is described.

I/O

Reading data from servers providing DOOCS properties, or changing its content, can be realized by the functions inside the io submodule. The documentation of these functions is shown here:

help(doocspie.doocspie.io)

Help on module doocspie.doocspie.io in doocspie.doocspie:

NAME
    doocspie.doocspie.io - Module with functions for higher level I/O within DOOCS.

DESCRIPTION
    This module provides the functions for higher level I/O operations with the DOOCS control system.

FUNCTIONS
    get(address, event=None, ignore_event_mismatch=False, timestamp_event=False, meta_event=None, start=None, elements=None)
        Get the readout for a given DOOCS address with optional parameters (see 'Args' below).
        
        Args:
            address (str): The property address of the server to get the readout from.
            event (int, optional): Optional event to get the data from.
            ignore_event_mismatch (bool, optional): Optional state for ignoring event mismatches.
            timestamp_event (bool, optional): Optional state to determine using the timestamp as an alternative event.
            meta_event (str, optional): Optional Meta property to replace the default event with.
            start (int, optional): Optional start position for reading out array-like types.
            elements (int, optional): Optional number of elements to read out for array-like types.
        
        Returns:
            Readout: An instance of the readout object with its properties 'data', 'type', 'timestamp', 'event' and 'meta'.
        
        Raises:
            DoocspieException: Context-dependent exception related to DOOCS, pydoocs or doocspie itself.
    
    get_history(address, begin=None, end=None)
        Get the history of a given DOOCS history address with optional 'begin' and 'end' datetimes.
        
        Args:
            address (str): The history property address of the server to get the history readout from.
            begin (datetime, optional): The optional beginning datetime for the history to get data from.
            end (datetime, optional): The optional ending datetime for the history to get data from.
        
        Returns:
            History: An instance of the history object with properties 'times', 'values' and 'states'.
        
        Raises:
            ValueError: Exception for wrong DOOCS history address.
            TypeError: Exception for wrong 'begin' and 'end' types.
            DoocspieException: Context-dependent exception related to DOOCS, pydoocs or doocspie itself.
    
    ls(address)
        List the members of the given 4-layer address scheme 'facility/device/location/property'.
        
        Args:
            address (str): The address part, optionally including the * wildcard, to query the members from.
        
        Returns:
            tuple: The queried members of the 4-layer DOOCS address scheme.
        
        Raises:
            DoocspieException: DOOCS related exception for non-exiting address parts to query from.
    
    set(address, value, allow_resizing=False)
        Set the content of DOOCS server properties with the given value and with optional resizing.
        
        Args:
            address (str): The property address of the server to set the value to.
            value (property dependent): The value to set the server property to.
            allow_resizing(bool, optional): The optional state for allowing resizing of array-like DOOCS types.
        
        Returns:
            None
        
        Raises:
            DoocspieException: DOOCS or pydoocs related exception for server errors or wrong input.

FILE
    /home/cbehrens/Home/Repositories/gitlab/doocspie/doocspie/doocspie/io.py

I/O operations are everyday business when dealing with the DOOCS control system, hence these functions can also easily be accessed directly from doocspie, instead of importing it from the doocspie.io submodule. The following code examples exclusively make use of this shortcut, and we introduce the main I/O functions of doocspie: ls, get, set and get_history.

ls

The ls function lists the members of the 4-layer address scheme of DOOCS as a Python tuple. The function expects optional parts of a string (Python’s str) based on the specific 4-layer address scheme facility/device/location/property with the additional but optionally included wildcard character *. Depending on the actual supplied address part, it returns then the available facilities, devices, locations and/or properties. The following examples return the number of all facilities, and subsequently, all the facilities ending with the .FEL suffix in its facility name:

all_facilities = doocspie.ls("")

print("number of all facilities:", len(all_facilities))

number of all facilities: 272

doocspie.ls("*.FEL")

('TTF2.FEL',
 'FLASH.FEL',
 'XFEL.FEL',
 'XFEL_SIM.FEL',
 'LAB.FEL',
 'TEST.FEL',
 'FLASH_SIM.FEL')

Here, we print the number of all properties for a given location (the wildcard * is optional):

properties_of_location = doocspie.ls("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/*")

print("number of all properties for a given location:", len(properties_of_location))

number of all properties for a given location: 189

All properties starting with the INTEGER prefix of the given location are returned in the following:

doocspie.ls("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER*")

('TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER',
 'TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER.BUFFER',
 'TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER_ARRAY',
 'TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER_REMOTEERROR')

get

The get function provides one the most important features in doocspie, the possibility for reading out data (and metadata) of different type from any DOOCS server. It requires a mandatory address string (str) of the DOOCS property to get the readout from, and the three optional parameters, event, ignore_event_mismatch and meta_event, allow for synchronization and/or a more specific readout. The get function returns an instance of the Readout type, which contains all the relevant readout data implemented as the following read-only Python properties: data, type, timestamp, event and meta. For every parameter mentioned earlier, we will give code examples in the following, and we start with a simple readout without supplying any optional parameter:

readout = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER")

print("type of readout:", type(readout))

type of readout: <class 'doocspie.doocspie.readout.Readout'>

Printing the Readout instance reveals its contents, consisting of its five property-value pairs:

print(readout)

Readout(data=42, type='INT', timestamp=1711375086.221107, event=0, meta=Meta())

The properties (attributes) of the readout can simply be accessed via Python’s dot notation:

print(readout.data, readout.type, readout.timestamp, readout.event, readout.meta)

42 INT 1711375086.221107 0 Meta()

The contents is dependent on the DOOCS data type, and in the example above, data is a scalar value and meta is empty. Here, we show an example, where data is an array and thus has a length:

array_readout = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER_ARRAY")

print("number of readout items:", len(array_readout))

number of readout items: 10

In some cases, which is also DOOCS server dependent, the event information might alternatively be stored in the meta property. In order to use this particular metadata as an alternative event instead, the metadata property can be supplied as an optional parameter as is presented here:

readout_1 = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM")
print("readout_1 event:", readout_1.event)
print("readout_1 meta:", readout_1.meta)

readout_2 = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM",
                         meta_event="timestamp")  # use timestamp from meta as event number
print("readout_2 event:", readout_2.event)

readout_1 event: 0
readout_1 meta: Meta(comment='', timestamp=1711017122, status=0, start=0.0, incr=0.10000000149011612)
readout_2 event: 1711017122

Alternatively, the timestamp can also be used as an event as is demonstrated in the following:

readout_1 = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM")
print("readout_1 event:", readout_1.event)
print("readout_1 timestamp:", readout_1.timestamp)

readout_2 = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM",
                         timestamp_event=True)  # use timestamp as event number
print("readout_2 event:", readout_2.event)

readout_1 event: 0
readout_1 timestamp: 1711375091.639045
readout_2 event: 1711375091.7

When synchronized readout is desired, and the DOOCS server of interest does internally provide a circular (ring) buffer, an event number can optionally be supplied to the get function. The code example below demonstrates this type of synchronization making use of the event parameter:

import time


reference = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/FLOAT.BUFFER")

time.sleep(1)  # artificially introduce timing delay to make synchronization indispensable

unsynchronized = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/FLOAT.BUFFER")
synchronized = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/FLOAT.BUFFER",
                            event=reference.event)

print("reference == unsynchronized is", reference == unsynchronized)
print("reference == synchronized is", reference == synchronized)

reference == unsynchronized is False
reference == synchronized is True

In case of event mismatches, i.e. a discrepancy between requested and returned event, a custom exception will be raised. If one decides to get the readout in such cases anyway, event mismatches can simply be ignored by means of the ignore_event_mismatch parameter as is shown here:

try:
    doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/FLOAT", event=23)
except doocspie.DoocspieException as exc:
    print("exception message:", exc.message)

readout = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/FLOAT", event=23,
                       ignore_event_mismatch=True)
print("readout:", readout)

exception message: event mismatch: (returned - requested) = -23
readout: Readout(data=23.0, type='FLOAT', timestamp=1711342070.452572, event=0, meta=Meta())

For array-like DOOCS types, e.g. SPECTRUM, the optional parameters start and/or elements can be used to read out only chunks of the entire array data. This also improves the readout times when the network bandwidth is limited for large data. The following code example demonstrates how to read out a limited number of elements of a large SPECTRUM for a given readout start position:

import numpy as np

address = "TEST.DOOCS/SINGENERATOR/SIN/DATA.TD"

start = 10
elements = 20

readout_1 = doocspie.get(address)
readout_2 = doocspie.get(address, start=start, elements=elements)

print("numpy arrays are equal:",
      np.all(readout_1.data[start:start+elements] == readout_2.data))

numpy arrays are equal: True

set

The set function allows to change the content of DOOCS server properties with a given value and with optional resizing. The function expects two parameters, a DOOCS address string (str) and a data value (property dependent type), in order to set the content a corresponding server property. The optional parameter allow_resizing of type bool enables the resizing of array-like DOOCS types. The following example demonstrates how to set an integer property to different values while printing its readout data (using the get command) before and after invoking the set command:

doocs_address = "TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER"

readout_before = doocspie.get(doocs_address)
doocspie.set(doocs_address, 42)  # the actual 'set' command
readout_after = doocspie.get(doocs_address)

print("readout data before 'set' command:", readout_before.data)
print("readout data after 'set' command:", readout_after.data)

readout data before 'set' command: 23
readout data after 'set' command: 42

If a particular DOOCS server facilitates the resizing feature of array-like data types, e.g. A_FLOAT or A_INT, the set function provides the optional parameter allow_resizing to do so. The default value of this parameter is False, and in the case of an array-like DOOCS type that is trying to be changed in size without setting it to True, a DoocspieException will be raised. The following code example presents both, trying to change an array-like DOOCS data type without providing the allow_resizing parameter and actually changing the size of the very same DOOCS property:

import numpy as np


doocs_address = "TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS_ALTERNATIVE/PROP_A_FLOAT"

readout_before = doocspie.get(doocs_address)
print("readout data before resizing:", readout_before.data)

try:
    doocspie.set(doocs_address, np.append(readout_before.data, 42))  # use NumPy
except doocspie.DoocspieException as exc:
    print("without resizing -> exception message:", exc.message)

doocspie.set(doocs_address, list(readout_before.data) + [42],  # use list concatenation
             allow_resizing=True)
readout_after = doocspie.get(doocs_address)

print("readout data after resizing:", readout_after.data)

readout data before resizing: [23.]
without resizing -> exception message: wrong input data size
readout data after resizing: [23. 42.]

get_history

The get_history function renders it possible to get the history of a given DOOCS history address string (str), which possesses a .HIST extension. Optional begin and end parameters of type datetime allow specifying the range of interest, and without providing these parameters, the current buffer of the DOOCS history property gets returned. As opposed to the get function, get_history returns an instance of History, which is also shown in the example below:

history_doocs_address = "TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/HIST_HELPER_INTEGER.HIST"

history = doocspie.get_history(history_doocs_address)

print("type of history:", type(history))

type of history: <class 'doocspie.doocspie.history.History'>

When printing the returned History instance, its content is shown in a preview. The history consists of the Python properties times, values and states. This is presented in the code example below together with the printout of the number of the returned history items:

print(history)

History(times=['2024-03-25 14:44:14.585000' '2024-03-25 14:44:14.686000'
 '2024-03-25 14:44:14.788000' ... '2024-03-25 14:58:11.896000'
 '2024-03-25 14:58:11.997000' '2024-03-25 14:58:12.099000'], values=[96 97 98 ... 84 85 86], states=[0 0 0 ... 0 0 0])

print("number of history items:", len(history))

number of history items: 8192

In order to specify a history range of interest, the optional begin and end parameters of type datetime can be supplied. The datetime class is convenient when dealing with dates and times, and more information can be found here: https://docs.python.org/3/library/datetime.html. Here, we print the length of a limited history range of the very same DOOCS property as before:

import datetime


now = datetime.datetime.now()
past = now - datetime.timedelta(seconds=10)  # 10 seconds in the past

history = doocspie.get_history(history_doocs_address, begin=past, end=now)

print("length of limited history range:", len(history))

length of limited history range: 101

The following plot shows the photon flux history at FLASH for five consecutive days, several years back in time. Here, we specify the datetime parameters by a standard ISO format. In this example, we also demonstrate how to access the times and values properties of the History instance.

import matplotlib.pyplot as plt


history = doocspie.get_history("FLASH.FEL/XGM.PHOTONFLUX/FL1.TUNNEL/PHOTONFLUX.UJ.HIST",
                               begin=datetime.datetime.fromisoformat("2021-07-29 08:00:00"),
                               end=datetime.datetime.fromisoformat("2021-08-03 08:00:00"))

plt.plot(history.times, history.values)
plt.xlabel("Times (YYYY-MM-DD)")
plt.ylabel("Values (arb. units)");

The DOOCS history property address must have the .HIST extension as is revealed here:

try:
    doocspie.get_history("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/HIST_HELPER_INTEGER")
except ValueError as err:
    print(err)

address must have '.HIST' extension

Types

In the I/O section above, we have already seen the different data types that occur when dealing with the doocspie library. These custom types, i.e. Readout, Meta and History, will be covered in the following again, but slightly more detailed in order get a deeper understanding.

Readout

The get function returns an instance of the Readout type, which contains all the relevant readout (meta)-data implemented as the following read-only Python properties: data, type, timestamp, event and meta. The following example presents the type and printout of a Readout instance:

readout = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER")

print("type of readout:", type(readout))
print(readout)

type of readout: <class 'doocspie.doocspie.readout.Readout'>
Readout(data=42, type='INT', timestamp=1711375092.708393, event=0, meta=Meta())

The following code shows how to access the readout properties as well as the type of the meta property, which itself is a custom data type of doocspie. The type of data depends on the actual DOOCS server property, and the type reflects its DOOCS data type. The timestamp and event describe the time of readout, where the latter can be utilized for synchronization purposes.

print(readout.data, readout.type, readout.timestamp, readout.event, readout.meta)
print("type of readout.meta:", type(readout.meta))

42 INT 1711375092.708393 0 Meta()
type of readout.meta: <class 'doocspie.doocspie.meta.Meta'>

As is customary, Python properties (attributes) are read-only, which is demonstrated here:

try:
    readout.data = 23
except AttributeError as exc:
    print(exc)

can't set attribute 'data'

Meta

The Readout type contains the meta property of type Meta, and this property is meant to provide additional metadata. In the previous example, the meta property was empty, but in the following, it consists of metadata describing the DOOCS type SPECTRUM. The length of the Meta instance is server dependent, and the example’s number of properties and its printout is shown here:

spectrum = doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM")

print("number of meta properties:", len(spectrum.meta))
print(spectrum.meta)

number of meta properties: 5
Meta(comment='', timestamp=1711017122, status=0, start=0.0, incr=0.10000000149011612)

As the properties are implemented as Python properties (attributes), they can be accessed in the accustomed manner and also provide the customary read-only trait, which is demonstrated here:

print(spectrum.meta.comment, spectrum.meta.timestamp, spectrum.meta.status,
      spectrum.meta.start, spectrum.meta.incr)

try:
    spectrum.meta.comment = "FOO"
except AttributeError as exc:
    print(exc)

 1711017122 0 0.0 0.10000000149011612
cannot set attribute

History

The get_history function returns an instance of the History type, which contains the past data of a particular DOOCS property. The historic data can be accessed by means of the following Python properties: times, values and status, where times is a NumPy array of type datetime and values is a NumPy array of the underlying DOOCS property type. The status property holds corresponding status flags. Here, we present the type of a history readout and its printout:

history_doocs_address = "TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/HIST_HELPER_INTEGER.HIST"

history = doocspie.get_history(history_doocs_address)

print("type of history:", type(history))
print(history)

type of history: <class 'doocspie.doocspie.history.History'>
History(times=['2024-03-25 14:44:15.495000' '2024-03-25 14:44:15.597000'
 '2024-03-25 14:44:15.699000' ... '2024-03-25 14:58:12.809000'
 '2024-03-25 14:58:12.910000' '2024-03-25 14:58:13.012000'], values=[ 5  6  7 ... 93 94 95], states=[0 0 0 ... 0 0 0])

The following code show how to access the History properties for the example readout above:

print(history.times)
print(history.values)
print(history.states)

[datetime.datetime(2024, 3, 25, 14, 44, 15, 495000)
 datetime.datetime(2024, 3, 25, 14, 44, 15, 597000)
 datetime.datetime(2024, 3, 25, 14, 44, 15, 699000) ...
 datetime.datetime(2024, 3, 25, 14, 58, 12, 809000)
 datetime.datetime(2024, 3, 25, 14, 58, 12, 910000)
 datetime.datetime(2024, 3, 25, 14, 58, 13, 12000)]
[ 5  6  7 ... 93 94 95]
[0 0 0 ... 0 0 0]

Exceptions

Exception handling is an extremely important part of reliable and robust software, and doocspie provides the custom exception DoocspieException. This exception can be associated with DOOCS, the core library pydoocs or doocspie itself, and the following provides an example for each case.

DoocspieException

The custom exception DoocspieException possesses the three Python properties code, message and address, which provide all the relevant information during exception handling, i.e. describing the exception’s reason and origin. The following examples demonstrate the exception handling, where either DOOCS, the core library pydoocs or doocspie is the cause, respectively. In each case, the exception itself and as well as its three properties mentioned earlier are printed out.

Example for the DoocspieException associated with DOOCS:

try:
    doocspie.ls("FACILITY_DOES_NOT_EXIST*")
except doocspie.DoocspieException as exc:
    print("exception:", exc)
    print("error code:", exc.code)
    print("error message:", exc.message)
    print("causing address:", exc.address)

exception: DoocsException with {'code': 119, 'message': 'no ens entry'}
error code: 119
error message: no ens entry
causing address: FACILITY_DOES_NOT_EXIST*

Example for the DoocspieException associated with pydoocs:

try:
    doocspie.set("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER_ARRAY", 23)
except doocspie.DoocspieException as exc:
    print("exception:", exc)
    print("error code:", exc.code)
    print("error message:", exc.message)
    print("causing address:", exc.address)

exception: PyDoocsException with 'wrong input data format'
error code: None
error message: wrong input data format
causing address: TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/INTEGER_ARRAY

Example for the DoocspieException associated with doocspie itself:

try:
    doocspie.get("TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM", meta_event="not existing")
except doocspie.DoocspieException as exc:
    print("exception:", exc)
    print("error code:", exc.code)
    print("error message:", exc.message)
    print("causing address:", exc.address)

exception: 'meta_event' must be in ('comment', 'timestamp', 'status', 'start', 'incr')
error code: None
error message: 'meta_event' must be in ('comment', 'timestamp', 'status', 'start', 'incr')
causing address: TEST.DOOCS/UNIT_TEST_SUPPORT/PY_DOOCS/SPECTRUM