xal.extension.wirescan.profile

Class ProfileDataProcessor

java.lang.Object

xal.extension.wirescan.profile.ProfileDataProcessor

public class ProfileDataProcessor
extends java.lang.Object

Class for processing profile data contained in ProfileData objects.

The objective of this class is to interpolate any missing data (represented by a value Double.NaN), remove the noise baseline, decouple the cross-talk between the signals, then gate the noise in the signal. Thus, it is expected that the data is processed by the following sequence of method calls:

interpolateMissingData()
removeBaselineNoise(double)
decoupleSignals(int, double)
thresholdFilter(double)

Once this sequence is completed the processed signals will be approximately zero base-lined, decoupled, and normalized to have unity first integral.

One may apply the above sequence of calls using default values for all the arguments with the single call to ProfileDataProcessor.processData()

There are three numeric tuning parameters that must be provided in the processing.

ProfileDataProcessor.removeBaselineNoise(double dblSigThreshold)
dblSigThreshold = amplitude of indicator function deciding signal presence
This value is in the interval [0,1]. Larger will use more signal samples to determine noise properties, but may also include actual signals. Smaller values are more likely to exclude signals in the noise calculations, but will use few samples to characterize the noise.

ProfileDataProcessor.decoupleSignals(int cntSmpMax, double dblFracSigMin)
cntSmpMax = number of samples to average when computing coefficients
This value must be an integer greater than 1. This value determined the number of samples around the maximum value used when computing coupling coefficients. Small numbers will use samples with the most integrity. Larger numbers will include samples more affected by noise, but may provide a better statistical approximation if the data set contains suspect data.
dblFracSigMin fractional signal strength necessary for inclusion in coupling calculation
This value ranges in the interval [0,1]. Large values ensure that only the best samples are used in computing the coupling coefficients, however, it reduces the available sample size. Small values increase the sample size but may include samples corrupted by noise.

ProfileDataProcessor.thresholdFilter(double dblFracSigMax)
dblFracSigMax = fraction of maximum signal where threshold is applied
The argument is the fraction of the maximum signal strength where the threshold is activated. Thus, this value is in the interval (-1,1) where a value of 1 would cause a filter output of zero for all inputs. Although sign is not enforced per se, we are expecting a positive signal. Thus, a value for dblFracSigMax in the interval (0,1) is appropriate. Then if the projection data were strictly non-negative, a value 0 would indicate no action by the filter.

Default values are suggested as constants with this class.

Field Summary

Fields

Modifier and Type	Field and Description
static double[]	ARR_DBL_HIGHPASS_INP_COEFFS Coefficients of the filter input
static double[]	ARR_DBL_HIGHPASS_OUT_COEFFS Coefficients of the filter output
static double[]	ARR_DBL_LOWPASS_INP_COEFFS Coefficients of the filter input
static double[]	ARR_DBL_LOWPASS_OUT_COEFFS Coefficients of the filter output
static double	DBL_CPL_MIN_FRAC_SIGNAL_AMPL Minimum fractional signal amplitude necessary to include sample for coupling coefficient calc.
static double	DBL_SIG_INDICATOR_THRESHOLD Amplitude of signal indicator used when determining noise floor
static double	DBL_THR_MAX_FRAC_SIGNAL_AMPL Maximum fractional signal amplitude for threshold filtering
static int	INT_CPL_MAX_SAMPLE_COUNT Maximum number of samples to use when computing coupling coefficients
static int	INT_DSP_HIGHPASS_ORDER Order of the filter
static int	INT_DSP_LOWPASS_ORDER Order of the filter

Constructor Summary

Constructors

Constructor and Description
ProfileDataProcessor(ProfileData dataOrg) Create a new ProfileDataProcessor object attached to the given profile data object.

Method Summary

Modifier and Type	Method and Description
void	decoupleSignals(int cntSmpMax, double dblFracSigMin) Decouple the project data signals.
int	getDataSize() Convenience method for returning the (vector) size of the associated ProfileData data object.
ProfileData	getProcessedData() Return the processed profile data object.
double[]	getProcessedData(ProfileData.Angle view) Return the processed data for the given profile view.
ProfileData	getRawData() Return the raw profile data object.
void	highPassFilter()
double[]	indicatorFunction(ProfileData.Angle view) Compute and return the signal presence indicator function for the given projection view.
void	interpolateMissingData() Interpolate between missing data points.
void	lowPassFilter()
void	processData() Perform the default processing on the associated projection data.
void	removeBaselineNoise(double dblSigThreshold) Remove the baseline noise signal from the data set.
void	thresholdFilter(double dblFracSigMax) Amplitude threshold filter for use as a noise gate.
java.lang.String	toString() Write out contents both original data and processed data to a string for inspection.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DBL_SIG_INDICATOR_THRESHOLD

public static final double DBL_SIG_INDICATOR_THRESHOLD

Amplitude of signal indicator used when determining noise floor

See Also:

Constant Field Values

INT_CPL_MAX_SAMPLE_COUNT

public static final int INT_CPL_MAX_SAMPLE_COUNT

Maximum number of samples to use when computing coupling coefficients

See Also:

Constant Field Values

DBL_CPL_MIN_FRAC_SIGNAL_AMPL

public static final double DBL_CPL_MIN_FRAC_SIGNAL_AMPL

Minimum fractional signal amplitude necessary to include sample for coupling coefficient calc.

See Also:

Constant Field Values

DBL_THR_MAX_FRAC_SIGNAL_AMPL

public static final double DBL_THR_MAX_FRAC_SIGNAL_AMPL

Maximum fractional signal amplitude for threshold filtering

See Also:

Constant Field Values

INT_DSP_LOWPASS_ORDER

public static final int INT_DSP_LOWPASS_ORDER

Order of the filter

See Also:

Constant Field Values

ARR_DBL_LOWPASS_INP_COEFFS

public static final double[] ARR_DBL_LOWPASS_INP_COEFFS

Coefficients of the filter input

ARR_DBL_LOWPASS_OUT_COEFFS

public static final double[] ARR_DBL_LOWPASS_OUT_COEFFS

Coefficients of the filter output

INT_DSP_HIGHPASS_ORDER

public static final int INT_DSP_HIGHPASS_ORDER

Order of the filter

See Also:

Constant Field Values

ARR_DBL_HIGHPASS_INP_COEFFS

public static final double[] ARR_DBL_HIGHPASS_INP_COEFFS

Coefficients of the filter input

ARR_DBL_HIGHPASS_OUT_COEFFS

public static final double[] ARR_DBL_HIGHPASS_OUT_COEFFS

Coefficients of the filter output

Constructor Detail

ProfileDataProcessor

public ProfileDataProcessor(ProfileData dataOrg)

Create a new ProfileDataProcessor object attached to the given profile data object.

Parameters:

dataOrg - profile data under process

See Also:

ProfileDataProcessor

Method Detail

getDataSize

public int getDataSize()

Convenience method for returning the (vector) size of the associated ProfileData data object.

Returns:

size of the associated projection data vectors

See Also:

gov.sns.tools.data.profile.ProfileData#getDataSize()

getRawData

public ProfileData getRawData()

Return the raw profile data object.

Returns:

raw profile data object

getProcessedData

public ProfileData getProcessedData()

Return the processed profile data object.

Returns:

processed profile data object

getProcessedData

public double[] getProcessedData(ProfileData.Angle view)

Return the processed data for the given profile view.

Parameters:

view - desired profile view

Returns:

processed profile data

indicatorFunction

public double[] indicatorFunction(ProfileData.Angle view)
                           throws java.lang.IllegalArgumentException

Compute and return the signal presence indicator function for the given projection view. The indicator function is computed by subtracting the (running) average from the signal then computing the total variation.

Parameters:

view - projection data

Returns:

signal presence indicator function

Throws:

java.lang.IllegalArgumentException - this should not occur (serious internal error)

See Also:

DigitalFunctionUtility.totalVariation(double[])

processData

public void processData()
                 throws DataProcessingException

Perform the default processing on the associated projection data. The following operations are performed:

interpolateMissingData()
removeBaselineNoise(double)
decoupleSignals(int, double)
thresholdFilter(double)

Default values for the arguments are taken from the global constants of this class.

Throws:

DataProcessingException - data set exceeds maximum bad data points

interpolateMissingData

public void interpolateMissingData()
                            throws DataProcessingException

Interpolate between missing data points. Missing data is indicated with a value Double.NaN. If we encounter more than two contiguous bad data points the operation punts by throwing an exception.

Throws:

DataProcessingException - data set exceeds maximum bad data points

lowPassFilter

public void lowPassFilter()

highPassFilter

public void highPassFilter()

removeBaselineNoise

public void removeBaselineNoise(double dblSigThreshold)

Remove the baseline noise signal from the data set. We assume that each profile signal f^{^}(n) is given by the following:

f^{^}(n) = f(n) + W(n)

where f(n) is the true signal at index n and W(n) is a noise process with a (possibly) nonzero mean given by E[W(n)] = w (independent of n). The value of w represents the baseline noise signal.

To remove this content we compute the signal indicator function for each signal and find the largest index such that all the indicator functions are less than the given value. All data before this index is assumed to be noise. We use that data to compute w for each signal assuming that the original signal is filtered such that E[(W - w)²] is small. The value w is subtracted from each respective signal.

Note, however, that only the mean component of the noise is removed. If the noise process has a non-zero standard deviation σ_W = E[(W(n) - w)²]^½ then spurious noise is still present. The spurious component can be attenuated by further filtering, but at the cost of degrading the true signal f(n). We originally assumed that this value was small enough to get an accurate determination from the data before signal presence.

Parameters:

dblSigThreshold - amplitude of indicator function deciding signal presence

Throws:

DataProcessingException - currently this should not happen

decoupleSignals

public void decoupleSignals(int cntSmpMax,
                            double dblFracSigMin)

Decouple the project data signals. That is, remove the cross-talk signals from each projection data set.

It is assumed that the profile signals may be represented by the equation

f*(n) = GCf(n)

where f*(n) is the vector-valued function of measured profile data (one element for each projection) at time n, G is the gain matrix, C is the cross-talk matrix, and f(n) is the vector-valued function of profile data. The gain matrix is chosen so that the integral of element in f(n) is unity. That is, ∫f(n)dn = (1,1,…,1).

NOTE:

It is best to use this method after the baseline noise have been removed from the profile data.

Parameters:

cntSmpMax - maximum number of samples to average when computing coefficients

dblFracSigMin - minimum fractional signal strength for use in calculation

thresholdFilter

public void thresholdFilter(double dblFracSigMax)

Amplitude threshold filter for use as a noise gate.

Apply a threshold filter to each projection data signal using the given fractional amplitude. Specifically, the projection data signal strength must be strictly greater than dblFracSigMax of the maximum signal strength to achieve a nonzero value. If the value dblFracSigMax is chosen properly the filter will perform as a noise gate where we assume a positive signal in the projection data.

NOTE:
- This is a zero-order filter (no memory), but it is highly nonlinear. Thus, you need to be careful when using this method, especially for computations with the resulting data.
- The argument is the fraction of the maximum signal strength where the threshold is activated. Thus, this value is in the interval (-1,1) where a value of 1 would cause a filter output of zero for all inputs.
- Although sign is not enforced per se, we are expecting a positive signal. Thus, a value for dblFracSigMax in the interval (0,1) is appropriate. Then if the projection data were strictly non-negative, a value 0 would indicate no action by the filter.

Parameters:

dblFracSigMax - fraction of maximum signal where threshold is applied

toString

public java.lang.String toString()

Write out contents both original data and processed data to a string for inspection. The returned string is a formatted table of values as produced by DigitalFunctionUtility#buildValueTable().

Overrides:

toString in class java.lang.Object

Returns:

string of tabulated projection data

See Also:

Object.toString()