fab.preprocessing

 1""" Collection of useful preprocessing tools. 
 2
 3Preprocessors can be applied to `fab.DataSources` or `fab.Instruments` 
 4as decorators, or they can be configured to be applied automatically
 5as part of the configuration toml. By using the `__preprocess__` keyword.
 6
 7For example, the `integrate` preprocessor, when applied to a `fab.DataSource` or a 
 8`fab.Instrument` will substitute the data by a integrated version of it,
 9summed along the given dimensions or axes. (This decorator is already
10built in into the fab library as `fab.preprocessing.integrate`).
11
12Preprocessors can be applied to sources from the config like so:
13
14``` toml
15[instruments.ursa.uv_diode]
16__type__ = 'fab.datasources.SlicedADC'
17hdf_key = '/zraw/FLASH.LASER/FLASH2CPUULGAN1.ADCSCOPE/CH26.TD/dGroup'
18offset = 2268
19window = 6
20period = 45
21baseline = 5
22dim_names = ['shot_id', 'diode_trace']
23__preprocess__ = [ {__name__='fab.preprocessing.integrate', dims='diode_trace'} ]
24```
25
26or to whole instruments like so:
27
28``` toml
29[instruments.ursa]
30__preprocess__ = [ {__name__='fab.preprocessing.integrate', dims='diode_trace'} ]
31```
32
33A very useful preprocessor is the `fab.preprocessing.mathexpr` preprocessor, which allows you to
34apply any mathematical expression to the data. For example, to calculate the
35inverse of the square root of the data, you can use:
36
37``` toml
38__preprocess__ = [ {__name__='fab.preprocessing.mathexpr', expr='1/np.sqrt(x)'} ]
39```
40
41## Custom Preprocessors
42
43You can also build your own custom preprocessors. The `fab.preprocessing.preprocessor` 
44decorator can be used to turn a callable into a preprocessor, as in the examples below
45(this is how the integrate preprocessors is defined in the fab library):
46
47``` python
48from fab.preprocessing import preprocessor
49
50@preprocessor
51def integrate(data, dims=None, axis=-1):        
52    if dims:
53        return data.sum(dim=dims)
54    return data.sum(axis=axis)
55```
56
57A full example of how to use a custom preprocessor is shown below:
58
59``` python
60from fab.preprocessing import preprocessor
61from fab import fab_setup
62
63@preprocessor
64def make_delay(dataset):
65    dataset['delay'] = dataset.BAM - dataset.delay_enc
66    return dataset
67
68fab_setup("preproc_config.toml", beamtime=11013355,
69            context = [make_delay])
70
71from fab.instruments import ursa
72```
73
74With the `preproc_config.toml` config file:
75
76``` toml
77[instruments.ursa.BAM]
78__type__ = 'fab.datasources.BAM'
79hdf_key = "/zraw/FLASH.SDIAG/BAM.DAQ/FL2.SEED5.ARRIVAL_TIME.ABSOLUTE.SA2.COMP/dGroup"
80
81[instruments.ursa.delay_enc]
82__type__ = 'fab.datasources.HDFSource'
83hdf_key = '/zraw/FLASH.SYNC/LASER.LOCK.EXP/F2.PPL.OSC/FMC0.MD22.0.ENCODER_POSITION.RD/dGroup'
84fillna_method = 'ffill'
85__preprocess__ = [ {__name__='fab.preprocessing.rescale', factor=0.020084354} ]
86
87[instruments.ursa]
88__preprocess__ = [{__name__='make_delay'}]
89```
90
91In  the example the `make_delay` preprocessor (meant to be used on an 
92`fab.Instrument`, rather than a `fab.DataSource`), will add a new 'fused' 
93`delay` variable combining the data in both `BAM` the `delay_enc` sources.
94"""
95
96from .preprocessor import preprocessor
97
98from .filters import *