quicknxs.interfaces.data_handling package

Submodules

quicknxs.interfaces.data_handling.data_info module

Meta-data information for MR reduction

class quicknxs.interfaces.data_handling.data_info.DataInfo(ws, cross_section, configuration)

Bases: object

Class to hold the relevant information from a run (scattering or direct beam).

determine_data_type(ws)

Inspect the data and determine peak locations and data type. :param workspace ws: Workspace to inspect

get_tof_range(ws)

Determine TOF range from the data :param workspace ws: workspace to work with

peak_range_offset = 0

process_roi(ws)

Process the ROI information and determine the peak range, the low-resolution range, and the background range.

Starting in June 2018, with the DAS upgrade, the ROIs are specified with a start/width rather than start/stop.

Parameters:

ws (workspace) – workspace to work with

tolerance = 0.02

class quicknxs.interfaces.data_handling.data_info.Fitter2(workspace)

Bases: object

DEAD_PIXELS = 10

fit_2d_peak()

Backward compatibility

fit_beam_width()

Fit the data distribution in y and get its range.

fit_peak()

gaussian_1d(value, *p)

1D Gaussian

peak_derivative(value, *p)

Double Gaussian to fit the first derivative of a plateau/peak.

quicknxs.interfaces.data_handling.data_info.chi2(data, model)

Returns the chi^2 for a data set and model pair

quicknxs.interfaces.data_handling.data_manipulation module

Methods used to process data, usually calling Mantid

exception quicknxs.interfaces.data_handling.data_manipulation.NormalizeToUnityQCutoffError

Bases: Exception

When normalizing to unity fails due to no data below Q cutoff

quicknxs.interfaces.data_handling.data_manipulation.extract_meta_data(file_path=None, cross_section_data=None)

Get mid Q-value from meta data :param str file_path: name of the file to read

quicknxs.interfaces.data_handling.data_manipulation.generate_script(reduction_list, pol_state)

Generate a Mantid script for the reflectivity reduction

Parameters:

• reduction_list (list) – list of NexusData objects

• pol_state (str) – cross-section name

quicknxs.interfaces.data_handling.data_manipulation.generate_short_script(reduction_list)

Generate a simple reduction script for Mantid

quicknxs.interfaces.data_handling.data_manipulation.get_scaled_workspaces(reduction_list, xs)

Return a list of scaled workspaces :param list reduction_list: list of NexusData objects :param str xs: cross-section name

quicknxs.interfaces.data_handling.data_manipulation.merge_reflectivity(reduction_list, xs, q_min=0.001, q_step=-0.01)

Combine the workspaces for a given cross-section into a single workspace.

TODO: trim workspaces

trim_first = [item.cross_sections[pol_state].configuration.cut_first_n_points for item in self.data_manager.reduction_list] trim_last = [item.cross_sections[pol_state].configuration.cut_last_n_points for item in self.data_manager.reduction_list]

quicknxs.interfaces.data_handling.data_manipulation.read_log(ws, name, target_units='', assumed_units='')

Read a log value, taking care of units. If the log entry has no units, the target units are assumed. :param ws: workspace :param str name: name of the property to read :param str target_units: units to convert to :param str assumed_units: units of origin, if not specified in the log itself

quicknxs.interfaces.data_handling.data_manipulation.smart_stitch_reflectivity(reduction_list, xs=None, normalize_to_unity=True, q_cutoff=0.01, global_fit=False, poly_degree=None, poly_points=3)

Stitch and normalize data sets

Parameters:

• reduction_list (list[NexusData]) – list of data sets to stitch

• xs (string) – name of the cross-section to use for the first data set

• normalize_to_unity (bool) – if True, the specular ridge will be normalized to 1

• q_cutoff (float) – used if normalize_to_unity = True, data with q < q_cutoff are part of the specular ridge

• global_fit (bool) – if True, use data from all cross-sections to calculate scaling factors

• poly_degree (int) – if not None, find the scaling factor by simultaneously fitting a polynomial and scaling factor to the two curves

• poly_points (int) – number of additional points on each end of the overlap region to include in the fit

Returns:

tuple (list of scaling factors, list of scaling factor errors)

Raises:

NormalizeToUnityQCutoffError: if normalize_to_unity = True, but there is no data below q_cutoff

quicknxs.interfaces.data_handling.data_manipulation.stitch_reflectivity(reduction_list, xs=None, normalize_to_unity=True, q_cutoff=0.01)

Stitch and normalize data sets

Parameters:

• xs (string) – name of the cross-section to use

• normalize_to_unity (bool) – if True, the specular ridge will be normalized to 1

quicknxs.interfaces.data_handling.data_set module

Loader for event nexus files. Uses Mantid Framework

class quicknxs.interfaces.data_handling.data_set.CrossSectionData(name, configuration, entry_name='entry', workspace=None)

Bases: object

Data object to hold loaded reflectivity data

property active_area_x

property active_area_y

property angle_offset

collect_info(workspace)

Extract meta data from DASLogs.

TODO: get average of values post filtering so that it truly represents the data

dangle = 0

property dangle0

det_size_x = 0.0007

det_size_y = 0.0007

property direct_pixel

dist_mod_det = 0

dist_mod_mon = 0

dist_sam_det = 0

property dpix

property dr

property event_workspace

get_background_vs_TOF()

Returns the background counts vs TOF

get_counts_vs_TOF()

Used for normalization, returns ROI counts vs TOF.

get_reduction_parameters(update_parameters=True)

Determine reduction parameter :param bool update_parameters: if True, we will find peak ranges

get_tof_counts_table()

Get a table of TOF vs counts in the region-of-interest (ROI)

The table columns are: - TOF - wavelength - counts normalized by proton charge - error in counts normalized by proton charge - counts - error in counts - size of the ROI

gisans(direct_beam=None)

Compute GISANS

Parameters:

direct_beam (CrossSectionData) – if given, this data will be used to normalize the output

lambda_center = 0

offspec(direct_beam=None)

Extract off-specular scattering from 4D dataset (x,y,ToF,I). Uses a window in y to filter the 4D data and than sums all I values for each ToF and x channel. Qz,Qx,kiz,kfz is calculated using the x and ToF positions together with the tth-bank and direct pixel values.

Parameters:

direct_beam (CrossSectionData) – if given, this data will be used to normalize the output

prepare_plot_data()

Bin events to be used for plotting and in-app calculations

process_configuration()

Process loaded data :param bool update_parameters: If true, we will determine reduction parameters

property r

property raw_dr

property raw_r

reflectivity(direct_beam=None, configuration=None)

Compute reflectivity

property reflectivity_workspace

property tof

property tofdata

update_calculated_values()

Update parameters that are calculated from the configuration

update_configuration(configuration)

Update configuration

property wavelength

property wavelength_range

Returns the wavelength range

property x

property xdata

property xtof

property xy

property y

property ydata

class quicknxs.interfaces.data_handling.data_set.NexusData(file_path: str, configuration: Configuration)

Bases: object

Read a nexus file with multiple cross-section data.

calculate_gisans(direct_beam, progress=None)

Compute GISANS

calculate_offspec(direct_beam=None)

Loop through the cross-section data sets and update the reflectivity.

calculate_reflectivity(direct_beam=None, configuration=None, ws_suffix: str = '')

Loop through the cross-section data sets and update the reflectivity.

Parameters:

• direct_beam (CrossSectionData | None) – Direct beam data

• configuration (Configuration | None) – The configuration

• ws_suffix (str) – String to add to reflectivity workspace name

Example

ws_suffix is used when reducing multiple ROIs for the same run and cross-section, to differentiate the workspace names in the Mantid data service

peak_index = 2 # update the active reduction list data_manager.set_active_reduction_list_index(peak_index) # get the first data set of the active reduction list nexus_data = data_manager.reduction_list[0] # calculate the reflectivity for this data set nexus_data.calculate_reflectivity(ws_suffix=str(peak_index))

get_highest_cross_section(n_points=10)

Get the cross-section with the largest signal at the lower end of its Q range.

Parameters:

n_points (int) – number of points to average over

get_q_range()

Return the Q range for the cross-sections

get_reflectivity_workspace_group()

is_direct_beam()

Returns True if the main cross-section is a direct beam

is_gisans_available()

Verify whether we have GISANS data calculated for all cross-sections

is_offspec_available()

Verify whether we have off-specular data calculated for all cross-sections

load(update_parameters=True, progress=None)

Load cross-sections from a nexus file. :param function progress: call-back function to track progress :param bool update_parameters: if True, we will find peak ranges

set_parameter(param, value)

Loop through the cross-section data sets and update a parameter.

update_calculated_values()

Loop through the cross-section data sets and update.

update_configuration(configuration)

Loop through the cross-section data sets and update the reflectivity.

class quicknxs.interfaces.data_handling.data_set.NexusMetaData

Bases: object

Class used to hold meta-data read before loading the neutron events

is_direct_beam = False

mid_q = 0

quicknxs.interfaces.data_handling.data_set.getIxyt(nxs_data)

Return [x, y, TOF] array @param nxs_data: Mantid workspace

quicknxs.interfaces.data_handling.filepath module

Classes to handle string representations of sets of run numbers and absolute paths to data files

class quicknxs.interfaces.data_handling.filepath.FilePath(file_path: str | List[str], sort: bool | None = True)

Bases: object

Helper class to deal with string representation of one or more absolute file paths.

Example: file_path = ‘/SNS/REF_M/IPTS-25531/nexus/REF_M_38202.nxs.h5+/SNS/REF_M/IPTS-25531/nexus/REF_M_38201.nxs.h5’

NOTE: Paths are sorted

property basename

property dirname

property first_path

property is_composite

classmethod join(dirname: str, basename: str, sort: bool = True) → str

Create the file path for a single file or a set of files using one directory

Example: u’/SNS/REF_M/IPTS-25531/nexus/REF_M_38198.nxs.h5+/SNS/REF_M/IPTS-25531/nexus/REF_M_38199.nxs.h5’

@param dirname: absolute path to a directory @param basename: name of one or more files. If more than one file, they’re concatenated with the merge symbol ‘+’. Example: u’REF_M_38198.nxs.h5+REF_M_38199.nxs.h5’ @param sort: if True, sort the basenames according to increasing run number when more than one file.

@returns string representing the absolute path to the files.

merge_symbol = '+'

property path

run_numbers(string_representation: str | None = None) → List[int] | str

@brief return the run number(s) associated to this file path @details This function assumes the basename of each single file path has the pattern “REF_M_XXXX.*” where ‘XXXX’ is the run number to extract, and * is some file extension @param string_representation: If None, return the run numbers as a list of integers. If ‘long’, return all the run numbers concatenated by the merge symbol (‘+’). If ‘short’, return a compressed string representation. For instance, return run numbers 1, 2, 3, 5, 7, 8 as 1:3+5+7:8

property single_paths

split()

classmethod unique_dirname(file_path)

For composite file paths, check that the dirname of the paths is the same for all files

class quicknxs.interfaces.data_handling.filepath.RunNumbers(numbers: List[int] | List[str] | int | str)

Bases: object

A helper class to handle string representations of one or more run numbers. It translates from a string representation to a list of run numbers, and viceversa

property long: str

@brief Long string representation of the run numbers @details Example: [1, 2, 3, 6] becomes ‘1+2+3+6’

merge_symbol = '+'

property numbers: List[int]

@brief List of run numbers as a list of integers

range_symbol = ':'

property short: str

@brief Short string representation of the run numbers @details Example: [1, 2, 3, 6] becomes ‘1:3+6’

property statement: str

@brief Human readable string representation. @details Examples: ‘12’, ‘12 and 13’, ‘12, 13, and 14’

quicknxs.interfaces.data_handling.gisans module

Computations for GISANS

class quicknxs.interfaces.data_handling.gisans.GISANS(cross_section_data)

Bases: object

Compute grazing-incident SANS

quicknxs.interfaces.data_handling.gisans.merge(reduction_list, pol_state, wl_min=0, wl_max=100)

Merge the off-specular data from a reduction list. :param list reduction_list: list of NexusData objects :param string pol_state: polarization state to consider

The scaling factors should have been determined at this point. Just use them to merge the different runs in a set.

TODO: This doesn’t deal with the overlap properly. It assumes that the user cut the overlapping points by hand.

quicknxs.interfaces.data_handling.gisans.rebin_extract(reduction_list, pol_state, wl_min, wl_max, qy_npts=50, qz_npts=50, use_pf=False)

quicknxs.interfaces.data_handling.gisans.rebin_parallel(reduction_list, pol_state, wl_min, wl_max, wl_npts=2, qy_npts=50, qz_npts=50, use_pf=False)

Process the wavelength bands in parallel.

quicknxs.interfaces.data_handling.instrument module

This instrument description contains information that is instrument-specific and abstracts out how we obtain information from the data file

class quicknxs.interfaces.data_handling.instrument.Instrument

Bases: object

Instrument class. Holds the data handling that is unique to a specific instrument.

USE_SLOW_FLIPPER_LOG = False

classmethod check_direct_beam(ws)

Determine whether this data is a direct beam

direct_beam_match(scattering, direct_beam, skip_slits=False)

Verify whether two data sets are compatible.

static dummy_filter_cross_sections(ws: EventWorkspace, name_prefix: str | None = None) → WorkspaceGroup

Filter events according to an aggregated state log.

Examples: BL4A:SF:ICP:getDI 015 (0000 1111): SF1=OFF, SF2=OFF, SF1Veto=OFF, SF2Veto=OFF 047 (0010 1111): SF1=ON, SF2=OFF, SF1Veto=OFF, SF2Veto=OFF 031 (0001 1111): SF1=OFF, SF2=ON, SF1Veto=OFF, SF2Veto=OFF 063 (0011 1111): SF1=ON, SF2=ON, SF1Veto=OFF, SF2Veto=OFF

@param ws: workspace containing the unfiltered events @param name_prefix: root name of the output WorkspaceGroup. If None, the run number of the workspace is chosen as the root name.

@return a group workspace for each of the four different filter/analyzer conbinations

file_search_template = '/SNS/REF_M/*/nexus/REF_M_%s'

classmethod get_info(workspace, data_object)

Retrieve information that is specific to this particular instrument

@param workspace: Mantid workspace @param data_object: CrossSectionData object

huber_x_cut = 6.5

instrument_dir = '/SNS/REF_M'

instrument_name = 'REF_M'

integrate_detector(ws, specular=True)

Integrate a workspace along either the main direction (specular=False) or the low-resolution direction (specular=True.

Parameters:

• ws – Mantid workspace

• bool (specular) – if True, the low-resolution direction is integrated over

legacy_search_template = '/SNS/REF_M/*/data/REF_M_%s'

load_data(file_path: str, configuration=None) → List[EventWorkspace]

Load one or more data sets according to the needs of the instrument.

This function assumes that when loading more than one data file, the files are congruent and their events will be added together.

Args:

file_path (str): absolute path to one or more data files. If more than one, paths should be concatenated with the plus symbol ‘+’. configuration (Configuration): reduction configuration parameters

Returns:

A list of EventWorkspaces, one for each cross-section

classmethod mid_q_value(ws)

Get the mid q value, at the requested wl mid-point. This is used when sorting out data sets and doesn’t need any overwrites. :param workspace ws: Mantid workspace

n_x_pixel = 304

n_y_pixel = 256

peak_range_offset = 50

pixel_width = 0.0007

classmethod scattering_angle_from_data(data_object)

Compute the scattering angle from a CrossSectionData object, in degrees. @param data_object: CrossSectionData object

tolerance = 0.05

quicknxs.interfaces.data_handling.instrument.apply_dead_time_correction(ws, configuration, error_ws=None) → EventWorkspace

Apply dead time correction, and ensure that it is done only once per workspace.

Parameters:

• ws – workspace with raw data to compute correction for

• configuration – reduction parameters

• error_ws – workspace with error events

quicknxs.interfaces.data_handling.instrument.get_cross_section_label(ws, entry_name)

Return the proper cross-section label.

quicknxs.interfaces.data_handling.instrument.get_dead_time_correction(ws, configuration, error_ws=None)

Compute dead time correction to be applied to the reflectivity curve.

The method will also try to load the error events from each of the data files to ensure that we properly estimate the dead time correction.

Parameters:

• ws – workspace with raw data to compute correction for

• configuration – reduction parameters

• error_ws – workspace with error events

quicknxs.interfaces.data_handling.instrument.mantid_algorithm_exec(algorithm_class, **kwargs)

Helper function for executing a Mantid-style algorithm

Parameters:

• algorithm_class (PythonAlgorithm) – the algorithm class to execute

• kwargs – keyword arguments

Returns Workspace:

if OutputWorkspace is passed as a keyword argument, the value of the algorithm property OutputWorkspace will be returned

quicknxs.interfaces.data_handling.instrument.remove_low_event_workspaces(ws_list, nbr_events_cutoff)

Removes workspaces with number of events below the cutoff from a list of workspaces

Parameters:

• ws_list (list[EventWorkspace])

• nbr_events_cutoff (int) – Minimum number of events

Returns:

Input list with low event workspaces removes

Return type:

list[EventWorkspace]

quicknxs.interfaces.data_handling.off_specular module

Class to execute and hold the off-specular reflectivity calculation.

class quicknxs.interfaces.data_handling.off_specular.OffSpecular(cross_section_data)

Bases: object

Compute off-specular reflectivity

Qx = None

Qz = None

S = None

dS = None

d_wavelength = 0

kf_z = None

ki_z = None

quicknxs.interfaces.data_handling.off_specular.closest_bin(q, bin_edges)

Find closest bin to a q-value :param float q: q-value :param list bin_edges: list of bin edges

quicknxs.interfaces.data_handling.off_specular.get_slice(qz, data, error, q_min, q_max)

Get a slice for a Qz band :param qz: Qz array :param data: 2D data array :param error: uncertainty on the data array :param q_min: lower Qz bound :param q_max: upper Qz bound

quicknxs.interfaces.data_handling.off_specular.merge(reduction_list, pol_state)

Merge the off-specular data from a reduction list. :param list reduction_list: list of NexusData objects :param string pol_state: polarization state to consider

The scaling factors should have been determined at this point. Just use them to merge the different runs in a set.

TODO: This doesn’t deal with the overlap properly. It assumes that the user cut the overlapping points by hand.

quicknxs.interfaces.data_handling.off_specular.proc(data)

Serializable function to be called by each thread

quicknxs.interfaces.data_handling.off_specular.rebin_extract(reduction_list, pol_state, axes=None, use_weights=True, n_bins_x=350, n_bins_y=350, x_min=-0.015, x_max=0.015, y_min=0, y_max=0.1)

Rebin off-specular data and extract cut at given Qz values. Note: the analysis computers with RHEL7 have Scipy 0.12 installed, which makes this code uglier. Refactor once we get a more recent version.

quicknxs.interfaces.data_handling.off_specular.smooth_data(x, y, I, sigmas=3.0, gridx=150, gridy=50, sigmax=0.0005, sigmay=0.0005, x1=-0.03, x2=0.03, y1=0.0, y2=0.1, axis_sigma_scaling=None, xysigma0=0.06, pool=5)

Execute legacy smoothing process by spreading it to a pool of processes.

quicknxs.interfaces.data_handling.peak_finding module

Bare bones version of the scipy 1.1.0 code to find peaks, which we use on a system that cannot run version 1.1.0 but only runs an old version.

Functions for identifying peaks in signals.

https://github.com/scipy/scipy/blob/master/LICENSE.txt

Copyright (c) 2001, 2002 Enthought, Inc. All rights reserved.

Copyright (c) 2003-2017 SciPy Developers. All rights reserved.

quicknxs.interfaces.data_handling.peak_finding.find_peaks(x, height=None, threshold=None, distance=None, prominence=None, width=None, wlen=None, rel_height=0.5)

Find peaks inside a signal based on peak properties. .. versionadded:: 1.1.0

quicknxs.interfaces.data_handling.peak_finding.peak_prominences(x, peaks, wlen=None)

Calculate the prominence of each peak in a signal.

Added in version 1.1.0.

References

Wikipedia Article for Topographic Prominence: https://en.wikipedia.org/wiki/Topographic_prominence

quicknxs.interfaces.data_handling.peak_finding.peak_widths(x, peaks, rel_height=0.5, prominence_data=None, wlen=None)

Calculate the width of each peak in a signal. .. versionadded:: 1.1.0

quicknxs.interfaces.data_handling.processing_workflow module

Data processing workflow, taking results and writing them to files.

class quicknxs.interfaces.data_handling.processing_workflow.ProcessingWorkflow(data_manager: DataManager, output_options: dict | None = None)

Bases: object

Carry out the reduction process for a set of data runs and manages outputs

execute(progress: ProgressReporter | None = None)

Process data and write output files :param ProgressReporter progress: reporter object

get_file_name(run_list=None, pol_state=None, data_type='dat', process_type='Specular')

Construct a file name according to the measurement type. :param list run_list: list of run numbers :param str pol_state: name for the polarization state :param str data_type: file extension :param str process_type: descriptor for the process type

get_gisans_data(progress=None)

get_gisans_slice_output_data(qy, qz, r, dr, pol_state, label, **slice_data_dict)

Produce a data dictionary with a slice of the data.

get_offspec_data()

Get a data dictionary ready for saving

get_output_data()

The QuickNXS format cannot be written from the merged reflectivity, so we have to treat it differently and give it the workspaces for each angle.

get_rebinned_offspec_data()

Get a data dictionary ready for saving

get_slice_output_data(qx, qz, r, dr, pol_state, label, **slice_data_dict)

Produce a data dictionary with a slice of the data.

gisans(progress=None)

Export GISANS.

offspec(raw=True, binned=False)

Export off-specular reflectivity. :param bool raw: if true, the raw results will be saved :param bool binned: if true, the raw results will be binned and saved

send_email()

Collect all files and send them to the user via smtp mail. #TODO: Put smtp server info in config file.

smooth_offspec(data_dict)

NOTE:

Create a smoothed dataset from the off-specular scattering. :param dict data_dict: the output of get_offspec_data()

Note for my own integrity (MD):

I don’t think one should smooth data distributions and do any quantitative work with it following this process. The way this was implemented in the previouse QuickNXS, replicated here, is equivalent to adding an extra resolution, which then would have to be properly taken into account when fitting. In addition, the process doesn’t produce errors in intensity. It effectively only produces a pretty picture and should only be used as such.

specular_reflectivity()

Retrieve the computed reflectivity and save it to file

write_quicknxs(output_data: Dict[str, ndarray], output_file_base: str, xs: list | None = None)

Write QuickNXS output reflectivity file. :param dict output_data: dictionary of numpy arrays :param str output_file_base: template for output file paths :param list xs: list of cross-sections available in the output_data

quicknxs.interfaces.data_handling.quicknxs_io module

Read and write quicknxs reduced files

quicknxs.interfaces.data_handling.quicknxs_io.determine_which_files_to_sum(run_file, data_file_indices)

quicknxs.interfaces.data_handling.quicknxs_io.read_reduced_file(file_path, configuration=None)

Read in configurations from a reduced data file. :param str file_path: reduced data file

quicknxs.interfaces.data_handling.quicknxs_io.write_reflectivity_data(output_path: str, data: list | ndarray, col_names: List[str], as_5col: bool = True)

Write out reflectivity header in a format readable by QuickNXS :param str output_path: output file path :param ndarray or list data: data to be written :param list col_names: list of column names :param bool as_5col: if True, a 5-column ascii will be written (theta is the last column)

quicknxs.interfaces.data_handling.quicknxs_io.write_reflectivity_header(peak_reduction_lists: Dict[int, List[NexusData]], active_list_index: int, direct_beam_list: List[NexusData], output_path: str, pol_state: str)

Write out reflectivity header in a format readable by QuickNXS

Parameters:

• peak_reduction_lists (dict[int, list[NexusData]]) – All reduction lists to include as additional peaks in the header

• active_list_index (int) – The index of the reduction list that the output reflectivity data is for

• direct_beam_list (list[NexusData]) – Direct beam list

• output_path (str) – Output file path

• pol_state (str) – Descriptor for the polarization state