cddm.core

Core functionality and main computation functions are defined here. There are low level implementations.

along with functions for calculating tau-dependent mean signal and mean square of the signal that are needed for normalization.

High-level functions include (in-memory calculation):

  • ccorr() to calculate cross-correlation/difference functions

  • acorr() to calculate auto-correlation/difference functions

For out-of-memory analysis use:

  • iccorr() to calculate cross-correlation/difference functions

  • iacorr() to calculate auto-correlation/difference functions

Finally, normalization of the results:

  • normalize() to normalize the outputs of the above functions.

Module Contents

cddm.core.cross_correlate_fft(f1, f2, t1=None, t2=None, axis=0, n=None, aout=None)

Calculates cross-correlation function of two equal sized input arrays using FFT.

For large arrays and large n, this is faster than correlate. The output of this function is identical to the output of cross_correlate.

See cross_correlate() for details.

cddm.core.auto_correlate_fft(f, t=None, axis=0, n=None, aout=None)

Calculates auto-correlation function of input array using FFT.

For large arrays and large n, this is faster than correlate. The output of this function is identical to the output of auto_correlate.

See auto_correlate() for details.

cddm.core.thread_frame_shape(shape, thread_divisor=None, force_2d=False)

Computes new frame shape for threaded computaton.

Parameters

  • shape (tuple of ints) – Input frame shape

  • thread_divisor (int) – An integer that divides the flattend frame shape. This number determines number of threads.

  • force_2d (bool) – If 1d data, make it 2d regardless of thread_divisor value.

Returns

shape – A length 2 shape

Return type

tuple

cddm.core.reshape_input(f, axis=0, thread_divisor=None, mask=None)

Reshapes input data, for faster threaded calculation

Parameters

  • f (ndarray) – Input array

  • axis (int) – Axis over which the computation is performed.

  • thread_divisor (int) – An integer that divides the flattend frame shape. This number determines number of threads.

  • mask (ndarray) – A boolean mask array. If provided, input data is masked first, then reshaped. This only works with axis = 0.

Returns

array, old_shape – Reshaped array and old frame shape tuple. Old frame shape is needed dor reshaping of output data with reshape_output()

Return type

ndarray, tuple

cddm.core.reshape_output(data, shape=None, mask=None)

Reshapes output data as returned from ccorr,acorr functions to original frame shape data.

If you used reshape_input() to reshape input data before call to ccorr or acorr functions. You must call this function on the output data to reshape it back to original shape and unmasked input array. Missing data is filled with np.nan.

Parameters

  • data (tuple of ndarrays, or ndarray) – Data as returned by acorr() or ccorr() or a numpy array, or a numpy array, as returned by normalize()

  • shape (tuple of ints) – shape of the input frame data.

  • mask (ndarray, optional) – If provided, reconstruct reshaped data to original shape, prior to masking with mask.

Returns

out – Reshaped data, as if there was no prior call to reshape_input on the input data of acorr() or ccorr() functions.

Return type

ndarray

cddm.core.cross_correlate(f1, f2, t1=None, t2=None, axis=0, n=None, align=False, aout=None)

Calculates cross-correlation function of two equal sized input arrays.

This function performs out[k] = sum_{i,j, where k = abs(t1[i]-t2[j])} (real(f1[i]*conj(f2[j]))

Parameters

  • f1 (array-like) – First input array

  • f2 (array-like) – Second input array

  • t1 (array-like, optional) – First time sequence. If not given, regular-spaced data is assumed.

  • t2 (array-like, optional) – Second time sequence. If not given, t1 time sequence is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Computed cross-correlation.

Return type

ndarray

See also

ccorr, cddm.multitau.ccorr_multi

cddm.core.cross_difference(f1, f2, t1=None, t2=None, axis=0, n=None, align=False, aout=None)

Calculates cross-difference (image structure) function of two equal sized input arrays.

This function performs out[k] = sum_{i,j, where k = abs(t1[i]-t2[j])} (abs(f2[j]-f1[i]))**2

Parameters

  • f1 (array-like) – First input array

  • f2 (array-like) – Second input array

  • t1 (array-like, optional) – First time sequence. If not given, regular-spaced data is assumed.

  • t2 (array-like, optional) – Second time sequence. If not given, t1 time sequence is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Computed cross-difference.

Return type

ndarray

See also

ccorr, cddm.multitau.ccorr_multi

cddm.core.auto_correlate(f, t=None, axis=0, n=None, align=False, aout=None)

Calculates auto-correlation function.

This function performs out[k] = sum_{i,j, where k = j - i >= 0} (real(f[i]*conj(f[j]))

Parameters

  • f (array-like) – Input array

  • t (array-like, optional) – Time sequence. If not given, regular-spaced data is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Computed auto-correlation.

Return type

ndarray

See also

acorr, cddm.multitau.acorr_multi

cddm.core.auto_difference(f, t=None, axis=0, n=None, align=False, aout=None)

Calculates auto-difference function.

This function performs out[k] = sum_{i,j, where k = j - i >= 0} np.abs((f[i] - f[j]))**2

Parameters

  • f (array-like) – Input array

  • t (array-like, optional) – Time sequence. If not given, regular-spaced data is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Computed auto-difference.

Return type

ndarray

See also

acorr, cddm.multitau.acorr_multi

cddm.core.cross_count(t1, t2=None, n=None, aout=None)

Culculate number of occurences of possible time delays in cross analysis for a given set of time arrays.

Parameters

  • t1 (array_like or int) – First time array. If it is a scalar, assume regular spaced data of length specified by t1.

  • t2 (array_like or None) – Second time array. If it is a scalar, assume regular spaced data of length specified by t2. If not given, t1 data is taken.

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added. If defeined, this takes precedence over the ‘n’ parameter.

Examples

>>> cross_count(10,n=5)
array([10, 18, 16, 14, 12])
>>> cross_count([1,3,6],[0,2,6],n=5)
array([1, 3, 0, 2, 1])
cddm.core.auto_count(t, n=None, aout=None)

Culculate number of occurences of possible time delays in auto analysis for a given time array.

Parameters

  • t (array_like or int) – Time array. If it is a scalar, assume regular spaced data of length specified by ‘t’

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added. If defeined, this takes precedence over the ‘n’ parameter.

Examples

>>> auto_count(10)
array([10,  9,  8,  7,  6,  5,  4,  3,  2,  1])
>>> auto_count([0,2,4,5])
array([4, 1, 2, 1, 1, 1])
cddm.core.cross_sum(f, t=None, t_other=None, axis=0, n=None, align=False, aout=None)

Calculates sum of array, useful for normalization of correlation data.

This function performs: out[k] = sum_{i,j, where k = abs(t[i]-t_other[j])} (f[i])

Parameters

  • f (array-like) – Input array

  • t1 (array-like, optional) – Time sequence of iput array. If not given, regular-spaced data is assumed.

  • t2 (array-like, optional) – Time sequence of the other array. If not given, t1 time sequence is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Calculated sum.

Return type

ndarray

cddm.core.cross_sum_fft(f, t, t_other=None, axis=0, n=None, aout=None)

Calculates sum of array, useful for normalization of correlation data.

This function is defined for irregular-spaced data only.

See cross_sum() for details.

cddm.core.auto_sum(f, t=None, axis=0, n=None, align=False, aout=None)

Calculates sum of array, useful for normalization of autocorrelation data.

This function performs: out[k] = sum_{i,j, where k = abs(t[i]-t[j]), j >= i} (f[i]+f[j])/2.

Parameters

  • f (array_like) – Input array

  • t1 (array-like, optional) – Time sequence of iput array. If not given, regular-spaced data is assumed.

  • t2 (array-like, optional) – Time sequence of the other array. If not given, t1 time sequence is assumed.

  • axis (int, optional) – For multi-dimensional arrays this defines computation axis (0 by default)

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’.

  • align (bool, optional) – Specifies whether data is aligned in memory first, before computation takes place. This may speed up computation in some cases (large n). Note that this requires a complete copy of the input arrays.

  • aout (ndarray, optional) – If provided, this must be zero-initiated output array to which data is added.

Returns

out – Computed sum.

Return type

ndarray

cddm.core.auto_sum_fft(f, t, axis=0, n=None, aout=None)

Calculates sum of array, useful for normalization of correlation data.

This function is defined for irregular-spaced data only.

See auto_sum() for details.

cddm.core.subtract_background(data, axis=0, bg=None, return_bg=False, out=None)

Subtracts background frame from a given data array.

This function can be used to subtract user defined background data, or to compute and subtract background data.

cddm.core.stats(f1, f2=None, axis=0)

Computes statistical parameters for normalization of correlation data.

Parameters

  • f1 (ndarray) – Fourier transform of the first video.

  • f2 (ndarray, optional) – Second data set (for dual video)

  • axis (int, optional) – Axis over which to compute the statistics.

Returns

(f1mean, f2mean), (f1var, f2var) – Computed mean and variance data of the input arrays.

Return type

(ndarray, ndarray), (ndarray, ndarray)

cddm.core.acorr(f, t=None, fs=None, n=None, norm=None, method=None, align=False, axis=0, aout=None)

Computes auto-correlation of the input signals of regular or irregular time - spaced data.

If data has ndim > 1, autocorrelation is performed over the axis defined by the axis parameter. If ‘aout’ is specified the arrays must be zero-initiated.

Parameters

  • f (array-like) – A complex ND array..

  • t (array-like, optional) – Array of integers defining frame times of the data. If not provided, regular time-spaced data is assumed.

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’

  • norm (int, optional) – Specifies normalization procedure 0,1,2, or 3. Default to 3, except for ‘diff’ method where it default to 1.

  • method (str, optional) – Either ‘fft’ , ‘corr’ or ‘diff’. If not given it is chosen automatically based on the rest of the input parameters.

  • align (bool, optional) – Whether to align data prior to calculation. Note that a complete copy of the data takes place.

  • axis (int, optional) – Axis over which to calculate.

  • aout (a tuple of ndarrays, optional) – Tuple of output arrays. For method = ‘diff’ : (corr, count, _, _) for ‘corr’ and ‘fft’ : (corr, count, squaresum, sum, _)

Returns

  • (corr, count, squaresum, sum, _) ((ndarray, ndarray, ndarray, ndarray, NoneType)) – Computed correlation data for ‘fft’ and ‘corr’ methods, If norm = 3, these are all defined. For norm < 3, some may be NoneType.

  • (diff, count, _, _) ((ndarray, ndarray, NoneType, NoneType)) – Computed difference data for ‘diff’ method.

cddm.core.ccorr(f1, f2, t1=None, t2=None, n=None, norm=None, method=None, align=False, axis=0, f1s=None, f2s=None, aout=None)

Computes cross-correlation of the input signals of regular or irregular time - spaced data.

If data has ndim > 1, calculation is performed over the axis defined by the axis parameter. If ‘aout’ is specified the arrays must be zero-initiated.

Parameters

  • f1 (array-like) – A complex ND array of the first data.

  • f2 (array-like) – A complex ND array of the second data.

  • t1 (array-like, optional) – Array of integers defining frame times of the first data. If not provided, regular time-spaced data is assumed.

  • t2 (array-like, optional) – Array of integers defining frame times of the second data. If not provided, regular time-spaced data is assumed.

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default). Note that ‘aout’ parameter takes precedence over ‘n’

  • norm (int, optional) – Specifies normalization procedure 0,1,2, or 3 (default).

  • method (str, optional) – Either ‘fft’, ‘corr’ or ‘diff’. If not given it is chosen automatically based on the rest of the input parameters.

  • align (bool, optional) – Whether to align data prior to calculation. Note that a complete copy of the data takes place.

  • axis (int, optional) – Axis over which to calculate.

  • f1s (array-like, optional) – First absolute square of the input data. For norm = NORM_COMPENSATED square of the signal is analysed. If not given it is calculated on the fly.

  • f2s (array-like, optional) – Second absolute square of the input data.

  • aout (a tuple of ndarrays, optional) – Tuple of output arrays. For method = ‘diff’ : (corr, count, sum1, sum2) for ‘corr’ and ‘fft’ : (corr, count, squaresum, sum1, sum2)

Returns

  • (corr, count, squaresum, sum1, sum2) ((ndarray, ndarray, ndarray, ndarray, ndarray)) – Computed correlation data for ‘fft’ and ‘corr’ methods, If norm = 3, these are all defined. For norm < 3, some may be NoneType.

  • (diff, count, sum1, sum2) ((ndarray, ndarray, ndarray, ndarray)) – Computed difference data for ‘diff’ method.

Examples

Say we have two datasets f1 and f2. To compute cross-correlation of both datasets :

>>> f1, f2 = np.random.randn(24,4,6) + 0j, np.random.randn(24,4,6) + 0j

>>> data = ccorr(f1, f2, n = 16)

Now we can set the ‘out’ parameter, and the results of the next dataset are added to results of the first dataset:

>>> data = ccorr(f1, f2,  aout = data)

Note that the parameter ‘n’ = 64 is automatically determined here, based on the provided ‘aout’ arrays.

cddm.core.iccorr(data, t1=None, t2=None, n=None, norm=0, method='corr', count=None, chunk_size=None, thread_divisor=None, auto_background=False, viewer=None, viewer_interval=1, mode='full', mask=None, stats=True)

Iterative version of ccorr().

Parameters

  • data (iterable) – An iterable object, iterating over dual-frame ndarray data.

  • t1 (int or array-like, optional) – Array of integers defining frame times of the first data. If it is a scalar it defines the length of the input data

  • t2 (array-like, optional) – Array of integers defining frame times of the second data. If not provided, regular time-spaced data is assumed.

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default).

  • norm (int, optional) – Specifies normalization procedure 0,1,2, or 3 (default).

  • method (str, optional) – Either ‘fft’, ‘corr’ or ‘diff’. If not given it is chosen automatically based on the rest of the input parameters.

  • count (int, optional) – If given, it defines how many elements of the data to process. If not given, count is set to len(t1) if that is not specified, it is set to len(data).

  • chunk_size (int) – Length of data chunk.

  • thread_divisor (int, optional) – If specified, input frame is reshaped to 2D with first axis of length specified with the argument. It defines how many treads are run. This must be a divisor of the total size of the frame. Using this may speed up computation in some cases because of better memory alignment and cache sizing.

  • auto_background (bool) – Whether to use data from first chunk to calculate and subtract background.

  • viewer (any, optional) – You can use viewer.MultitauViewer to display data.

  • viewer_interval (int, optional) – A positive integer, defines how frequently are plots updated 1 for most frequent, higher numbers for less frequent updates.

  • mode (str) – Either “full” or “partial”. With mode = “full”, output of this function is identical to the output of ccorr_multi(). With mode = “partial”, cross correlation between neigbouring chunks is not computed.

  • mask (ndarray, optional) – If specifed, computation is done only over elements specified by the mask. The rest of elements are not computed, np.nan values are written to output arrays.

  • stats (bool) – Whether to return stats as well.

Returns

  • ccorr_data, bg, var (ccorr_type, ndarray, ndarray) – Ccorr data, background and variance data. See ccorr() for definition of accorr_type

  • ccorr_data (ccorr_type) – If stats == False

cddm.core.iacorr(data, t=None, n=None, norm=0, method='corr', count=None, chunk_size=None, thread_divisor=None, auto_background=False, viewer=None, viewer_interval=1, mode='full', mask=None, stats=True)

Iterative version of ccorr()

Parameters

  • data (iterable) – An iterable object, iterating over single-frame ndarray data.

  • t (int or array-like, optional) – Array of integers defining frame times of the data. If it is a scalar it defines the length of the input data

  • n (int, optional) – Determines the length of the output (max time delay - 1 by default).

  • norm (int, optional) – Specifies normalization procedure 0,1,2, or 3 (default).

  • method (str, optional) – Either ‘fft’, ‘corr’ or ‘diff’. If not given it is chosen automatically based on the rest of the input parameters.

  • chunk_size (int) – Length of data chunk.

  • count (int, optional) – If given, it defines how many elements of the data to process. If not given, count is set to len(t1) if that is not specified, it is set to len(data).

  • thread_divisor (int, optional) – If specified, input frame is reshaped to 2D with first axis of length specified with the argument. It defines how many treads are run. This must be a divisor of the total size of the frame. Using this may speed up computation in some cases because of better memory alignment and cache sizing.

  • auto_background (bool) – Whether to use data from first chunk to calculate and subtract background.

  • viewer (any, optional) – You can use viewer.MultitauViewer to display data.

  • viewer_interval (int, optional) – A positive integer, defines how frequently are plots updated 1 for most frequent, higher numbers for less frequent updates.

  • mode (str) – Either “full” or “partial”. With mode = “full”, output of this function is identical to the output of ccorr_multi(). With mode = “partial”, cross correlation between neigbouring chunks is not computed.

  • mask (ndarray, optional) – If specifed, computation is done only over elements specified by the mask. The rest of elements are not computed, np.nan values are written to output arrays.

  • stats (bool) – Whether to return stats as well.

Returns

  • acorr_data, bg, var (acorr_type, ndarray, ndarray) – Acorr data, background and variance data. See acorr() for definition of acorr_type

  • acorr_data (acorr_type) – If stats == False