quicknxs.interfaces.data_handling package
Data handling interfaces.
Submodules
quicknxs.interfaces.data_handling.data_info module
Meta-data information for MR reduction.
- class quicknxs.interfaces.data_handling.data_info.DataInfo(ws, cross_section, configuration)[source]
Bases:
objectClass to hold the relevant information from a run (scattering or direct beam).
- peak_range_offset = 0
- process_roi(ws)[source]
Process the ROI information from a Mantid workspace.
Determines the peak range, low-resolution range, and background range. Starting in June 2018, with the DAS upgrade, the ROIs are specified with a start/width rather than start/stop.
- tolerance = 0.02
quicknxs.interfaces.data_handling.data_manipulation module
Methods used to process data, usually calling Mantid.
- exception quicknxs.interfaces.data_handling.data_manipulation.NormalizeToUnityQCutoffError[source]
Bases:
ExceptionWhen normalizing to unity fails due to no data below Q cutoff.
- quicknxs.interfaces.data_handling.data_manipulation.extract_metadata(file_path: str | None = None, cross_section_data=None)[source]
Get mid Q-value from metadata.
- quicknxs.interfaces.data_handling.data_manipulation.generate_script(reduction_list: List[NexusData], pol_state: str)[source]
Generate a Mantid script for the reflectivity reduction.
- Parameters:
reduction_list – list of NexusData objects
pol_state – cross-section name
- quicknxs.interfaces.data_handling.data_manipulation.generate_short_script(reduction_list)[source]
Generate a simple reduction script for Mantid.
- quicknxs.interfaces.data_handling.data_manipulation.get_scaled_workspaces(reduction_list: List[NexusData], xs: str)[source]
Return a list of scaled workspaces.
- quicknxs.interfaces.data_handling.data_manipulation.merge_reflectivity(reduction_list, xs, q_min=0.001, q_step=-0.01)[source]
Combine the workspaces for a given cross-section into a single workspace.
- TODO: trim workspaces
trim_first = [item.cross_sections[pol_state].configuration.cut_first_n_points for item in self.data_manager.reduction_list] trim_last = [item.cross_sections[pol_state].configuration.cut_last_n_points for item in self.data_manager.reduction_list]
- quicknxs.interfaces.data_handling.data_manipulation.read_log(ws: Workspace, name: str, target_units: str = '', assumed_units: str = '')[source]
Read a log value, taking care of units.
If the log entry has no units, the target units are assumed.
- Parameters:
ws – Mantid workspace
name – Name of the property to read
target_units – Units to convert to
assumed_units – Units of origin, if not specified in the log itself
- quicknxs.interfaces.data_handling.data_manipulation.stitch_reflectivity(reduction_list: List[NexusData], xs: str | None = None, normalize_to_unity: bool = True, q_cutoff: float = 0.01, global_fit: bool = False, poly_degree: int | None = None, poly_points: int = 3)[source]
Stitch and normalize data sets.
- Parameters:
reduction_list – list of data sets to stitch
xs – name of the cross-section to use for the first data set
normalize_to_unity – if True, the specular ridge will be normalized to 1
q_cutoff – used if normalize_to_unity = True, data with q < q_cutoff are part of the specular ridge
global_fit – if True, use data from all cross-sections to calculate scaling factors
poly_degree – if not None, find the scaling factor by simultaneously fitting a polynomial and scaling factor to the two curves
poly_points – number of additional points on each end of the overlap region to include in the fit
- Returns:
(list of scaling factors, list of scaling factor errors)
- Return type:
Tuple
- Raises:
NormalizeToUnityQCutoffError: – if normalize_to_unity = True, but there is no data below q_cutoff
quicknxs.interfaces.data_handling.data_set module
Loader for event nexus files using Mantid Framework.
- class quicknxs.interfaces.data_handling.data_set.CrossSectionData(name, configuration: Configuration, entry_name='entry', workspace=None)[source]
Bases:
objectData object to hold loaded reflectivity data.
- property active_area_x
- property active_area_y
- property angle_offset
- collect_info(workspace)[source]
Extract meta data from DASLogs.
TODO: get average of values post filtering so that it truly represents the data
- dangle = 0.0
- property dangle0
- det_size_x = 0.0007
- det_size_y = 0.0007
- property direct_pixel
- dist_mod_det = 0.0
- dist_mod_mon = 0.0
- dist_sam_det = 0.0
- property dpix
- property dr
- property event_workspace
- get_reduction_parameters(update_parameters=True)[source]
Determine reduction parameters from the Nexus file.
- Parameters:
update_parameters – if True, we will find peak ranges
- get_tof_counts_table()[source]
Get a table of TOF vs counts in the region-of-interest (ROI).
The table columns are: - TOF - wavelength - counts normalized by proton charge - error in counts normalized by proton charge - counts - error in counts - size of the ROI
- gisans(direct_beam: CrossSectionData | None = None)[source]
Compute GISANS.
- Parameters:
direct_beam – if given, this data will be used to normalize the output
- lambda_center = 0.0
- offspec(direct_beam: CrossSectionData | None = None)[source]
Extract off-specular scattering from 4D dataset (x,y,ToF,I).
Uses a window in y to filter the 4D data, then sums all I values for each ToF and x channel. Qz, Qx, kiz, kfz are calculated using the x and ToF positions together with the tth-bank and direct pixel values.
- Parameters:
direct_beam – if given, this data will be used to normalize the output
- property r
- property raw_dr
- property raw_r
- property reflectivity_workspace
- property tof
- property tofdata
- update_configuration(configuration: Configuration)[source]
Update configuration.
- property wavelength
- property wavelength_range
Returns the wavelength range.
- property x
- property xdata
- property xtof
- property xy
- property y
- property ydata
- class quicknxs.interfaces.data_handling.data_set.NexusData(file_path: str, configuration: Configuration)[source]
Bases:
objectRead a nexus file with multiple cross-section data.
Structure to read in one or more Nexus data files.
- Parameters:
file_path (str) – absolute path to one or more files. If more than one, paths are concatenated with the plus symbol ‘+’
configuration (Configuration) – Reduction configuration
- calculate_offspec(direct_beam=None)[source]
Update the off-specular reflectivity for all cross sections.
- calculate_reflectivity(direct_beam: CrossSectionData | None = None, configuration: Configuration | None = None, ws_suffix: str = '')[source]
Loop through the cross-section data sets and update the reflectivity.
- Parameters:
direct_beam (CrossSectionData | None) – Direct beam data
configuration (Configuration | None) – The configuration
ws_suffix (str) – String to add to reflectivity workspace name
Example
ws_suffix is used when reducing multiple ROIs for the same run and cross-section, to differentiate the workspace names in the Mantid data service
peak_index = 2 # update the active reduction list data_manager.set_active_reduction_list_index(peak_index) # get the first data set of the active reduction list nexus_data = data_manager.reduction_list[0] # calculate the reflectivity for this data set nexus_data.calculate_reflectivity(ws_suffix=str(peak_index))
- get_highest_cross_section(n_points=10)[source]
Get the cross-section with the largest signal at the lower end of its Q range.
- Parameters:
n_points (int) – number of points to average over
- get_main_cross_section_data() CrossSectionData[source]
Returns the CrossSectionData object corresponding to the main cross section
- is_gisans_available()[source]
Verify whether we have GISANS data calculated for all cross-sections.
- is_offspec_available()[source]
Verify whether we have off-specular data calculated for all cross-sections.
- load(update_parameters: bool = True, progress: Callable | None = None)[source]
Load cross-sections from a nexus file.
- Parameters:
update_parameters – if True, we will find peak ranges
progress – call-back function to track progress
- set_is_direct_beam(is_direct_beam: bool)[source]
Sets the value of is_direct_beam for all cross-sections.
- class quicknxs.interfaces.data_handling.data_set.NexusMetaData[source]
Bases:
objectClass used to hold meta-data read before loading the neutron events.
- is_direct_beam = False
- mid_q = 0
- quicknxs.interfaces.data_handling.data_set.getIxyt(nxs_data: Workspace2D) tuple[source]
Return [x, y, TOF] array and [x, y, TOF] error array from a Nexus file.
quicknxs.interfaces.data_handling.filepath module
Classes to handle string representations of sets of run numbers and absolute paths to data files.
- class quicknxs.interfaces.data_handling.filepath.FilePath(file_path: str | List[str], sort: bool = True)[source]
Bases:
objectHelper class to deal with string representation of one or more absolute file paths.
Example: file_path = ‘/SNS/REF_M/IPTS-25531/nexus/REF_M_38202.nxs.h5+/SNS/REF_M/IPTS-25531/nexus/REF_M_38201.nxs.h5’
NOTE: Paths are sorted
- property basename
- property dirname
- property first_path
- property is_composite
- classmethod join(dirname: str, basename: str, sort: bool = True) str[source]
Create the file path for a single file or a set of files using one directory.
Example: u’/SNS/REF_M/IPTS-25531/nexus/REF_M_38198.nxs.h5+/SNS/REF_M/IPTS-25531/nexus/REF_M_38199.nxs.h5’
- Parameters:
dirname – absolute path to a directory
basename – name of one or more files. If more than one file, they’re concatenated with the merge symbol ‘+’. Example: u’REF_M_38198.nxs.h5+REF_M_38199.nxs.h5’
sort – if True, sort the basenames according to increasing run number when more than one file.
- Returns:
string representing the absolute path to the files.
- Return type:
- merge_symbol = '+'
- property path
- run_numbers(string_representation: str | None = None) List[int] | str[source]
Return the run number(s) associated to this file path.
This function assumes the basename of each single file path has the pattern “REF_M_XXXX.*” where ‘XXXX’ is the run number to extract, and * is some file extension
- Parameters:
string_representation – One of [None, “long”, “short”, “statement”] to return the run numbers as a list of integers, a long string representation, a short string representation, or a human readable statement, respectively.
- property single_paths
- class quicknxs.interfaces.data_handling.filepath.RunNumbers(numbers: List[int] | List[str] | int | str)[source]
Bases:
objectA helper class to handle string representations of one or more run numbers.
Translates from a string representation to a list of run numbers, and viceversa
Initialize the RunNumbers object.
- Parameters:
numbers – a list of numbers or a string containing one or more numbers. For instance, ‘1:3+5’ translates to [1, 2, 3, 5]
- property long: str
Long string representation of the run numbers.
Example: [1, 2, 3, 6] becomes ‘1+2+3+6’
- merge_symbol = '+'
- range_symbol = ':'
quicknxs.interfaces.data_handling.gisans module
Computations for GISANS.
- class quicknxs.interfaces.data_handling.gisans.GISANS(cross_section_data: CrossSectionData)[source]
Bases:
objectCompute grazing-incident SANS.
Initialize GISANS calculations.
The calculations here are meant to match QuickNXS v1. The following are items to improve on: - Background subtraction - Trim the TOF distribution to the chopper bandwidth
- quicknxs.interfaces.data_handling.gisans.merge(reduction_list: List[NexusData], pol_state: str, wl_min: float = 0, wl_max: float = 100)[source]
Merge the off-specular data from a reduction list.
The scaling factors should have been determined at this point. Just use them to merge the different runs in a set.
TODO: This doesn’t deal with the overlap properly. It assumes that the user has cut the overlapping points by hand.
- Parameters:
reduction_list – List of NexusData objects
pol_state – Polarization state to consider
wl_min – Minimum wavelength to consider
wl_max – Maximum wavelength to consider
quicknxs.interfaces.data_handling.instrument module
Container class for instrument-specific data handling.
Abstracts out how we obtaininformation from the data file
- class quicknxs.interfaces.data_handling.instrument.Instrument[source]
Bases:
objectInstrument class. Holds the data handling that is unique to a specific instrument.
- direct_beam_distance(scattering, direct_beam) float[source]
Return a Euclidean squared-distance between slit widths in scattered, direct beams
- direct_beam_match(scattering, direct_beam, skip_slits=False)[source]
Verify whether two data sets are compatible.
- classmethod get_info(workspace, data_object)[source]
Retrieve information that is specific to this particular instrument.
@param workspace: Mantid workspace @param data_object: CrossSectionData object
- integrate_detector(ws: EventWorkspace, specular: bool = True)[source]
Integrate a workspace along either the main direction or the low-resolution direction.
- ws:
Mantid workspace to integrate
- specular:
If True, the workspace is integrated over the low-resolution direction. If False, the workspace is integrated over the main direction.
- load_data(file_path: str, configuration: Configuration | None = None) List[EventWorkspace][source]
Load one or more data sets according to the needs of the instrument.
This function assumes that when loading more than one data file, the files are congruent and their events will be added together.
- Parameters:
file_path (str) – absolute path to one or more data files. If more than one, paths should be concatenated with the plus symbol ‘+’.
configuration (Configuration) – reduction configuration parameters
- Returns:
A list of EventWorkspaces, one for each cross-section
- Return type:
List[EventWorkspace]
- Raises:
InsufficientEventCountError – If the data file does not contain enough events
- classmethod mid_q_value(ws: EventWorkspace) float[source]
Get the mid q value, at the requested wl mid-point.
This is used when sorting out data sets and doesn’t need any overwrites.
- classmethod scattering_angle_from_data(data_object: CrossSectionData) float[source]
Compute the scattering angle from a CrossSectionData object, in degrees.
- exception quicknxs.interfaces.data_handling.instrument.InsufficientEventCountError[source]
Bases:
ExceptionException raised when the number of events in the workspace is too low
- quicknxs.interfaces.data_handling.instrument.get_cross_section_label(ws, entry_name)[source]
Return the proper cross-section label.
quicknxs.interfaces.data_handling.off_specular module
Class to execute and hold the off-specular reflectivity calculation.
- class quicknxs.interfaces.data_handling.off_specular.OffSpecular(cross_section_data: CrossSectionData)[source]
Bases:
objectCompute off-specular reflectivity.
Initialize the OffSpecular class with processed cross-section data.
- Qx = None
- Qz = None
- S = None
- dS = None
- d_wavelength = 0
- kf_z = None
- ki_z = None
- quicknxs.interfaces.data_handling.off_specular.closest_bin(q: float, bin_edges: list) int | None[source]
Find index of closest bin to a q-value.
- quicknxs.interfaces.data_handling.off_specular.get_slice(qz, data, error, q_min, q_max)[source]
Get a slice for a Qz band.
- Parameters:
qz – Qz array
data – 2d data array
error – Uncertainty on the data array
q_min – Lower Qz bound
q_max – Upper Qz bound
- quicknxs.interfaces.data_handling.off_specular.merge(reduction_list: List[NexusData], pol_state: str) Tuple[ndarray, ...][source]
Merge the off-specular data from a reduction list.
The scaling factors should have been determined at this point. Just use them to merge the different runs in a set.
TODO: This doesn’t deal with the overlap properly. It assumes that the user cut the overlapping points by hand.
- quicknxs.interfaces.data_handling.off_specular.proc(data: dict) Tuple[ndarray, ndarray, ndarray][source]
Serializable function to be called by each thread.
- quicknxs.interfaces.data_handling.off_specular.rebin_extract(reduction_list, pol_state, axes=None, use_weights=True, n_bins_x=350, n_bins_y=350, x_min=-0.015, x_max=0.015, y_min=0, y_max=0.1)[source]
Rebin off-specular data and extract cut at given Qz values.
TODO: the analysis computers with RHEL7 have Scipy 0.12 installed, which makes this code uglier. Refactor once we get a more recent version.
- quicknxs.interfaces.data_handling.off_specular.smooth_data(x: ndarray, y: ndarray, I: ndarray, sigmas: float = 3.0, gridx=150, gridy=50, sigmax=0.0005, sigmay=0.0005, x1=-0.03, x2=0.03, y1=0.0, y2=0.1, axis_sigma_scaling: int | None = None, xysigma0: float = 0.06, indices: list | None = None, pool=5)[source]
Execute legacy smoothing process by spreading it to a pool of processes.
- Parameters:
x – x-values of the original data
y – y-values of the original data
I – Intensity values of the original data
sigmas – Range in units of sigma to search around a grid point
gridx – Number of grid points in x direction
gridy – Number of grid points in y direction
sigmax – Sigma in x direction
sigmay – Sigma in y direction
x1 – Lower x bound of the grid
x2 – Upper x bound of the grid
y1 – Lower y bound of the grid
y2 – Upper y bound of the grid
axis_sigma_scaling – Defines how the sigmas change with the x/y value
xysigma0 – x/y value where the given sigmas are used
indices – List of indices to run over
pool – Number of processes to use for the smoothing
quicknxs.interfaces.data_handling.peak_finding module
Bare bones version of the scipy 1.1.0 code to find peaks.
Required as we use a system that cannot run version 1.1.0 but only runs an old version.
Functions for identifying peaks in signals.
https://github.com/scipy/scipy/blob/master/LICENSE.txt
Copyright (c) 2001, 2002 Enthought, Inc. All rights reserved.
Copyright (c) 2003-2017 SciPy Developers. All rights reserved.
- quicknxs.interfaces.data_handling.peak_finding.find_peaks(x, height=None, threshold=None, distance=None, prominence=None, width=None, wlen=None, rel_height=0.5)[source]
Find peaks inside a signal based on peak properties.
Added in version 1.1.0.
- quicknxs.interfaces.data_handling.peak_finding.peak_prominences(x, peaks, wlen=None)[source]
Calculate the prominence of each peak in a signal.
Added in version 1.1.0.
References
Wikipedia Article for Topographic Prominence: https://en.wikipedia.org/wiki/Topographic_prominence
quicknxs.interfaces.data_handling.processing_workflow module
Data processing workflow, taking results and writing them to files.
- class quicknxs.interfaces.data_handling.processing_workflow.ProcessingWorkflow(data_manager: DataManager, output_options: dict | None = None)[source]
Bases:
objectCarry out the reduction process for a set of data runs and manages outputs.
All the reduced data shall come from data manager.
- execute(progress: ProgressReporter | None = None)[source]
Process data and write output files.
- get_file_name(run_list: list | None = None, pol_state: str | None = None, data_type: str = 'dat', process_type: str = 'Specular')[source]
Construct a file name according to the measurement type.
- Parameters:
run_list – List of run numbers
pol_state – Name for the polarization state
data_type – File extension
process_type – Descriptor for the process type
- get_gisans_slice_output_data(qy, qz, r, dr, pol_state, label, **slice_data_dict)[source]
Produce a data dictionary with a slice of the data.
- get_output_data()[source]
Re-format the reflectivity data for output.
The QuickNXS format cannot be written from the merged reflectivity, so we have to treat it differently and give it the workspaces for each angle.
- get_slice_output_data(qx, qz, r, dr, pol_state, label, **slice_data_dict)[source]
Produce a data dictionary with a slice of the data.
- offspec(raw: bool = True, binned: bool = False)[source]
Export off-specular reflectivity.
- Parameters:
raw – If true, the raw results will be saved
binned – If true, the raw results will be binned and saved
- smooth_offspec(data_dict: dict) Tuple[dict, dict][source]
Create a smoothed dataset from the off-specular scattering.
- Note for my own integrity (MD):
I don’t think one should smooth data distributions and do any quantitative work with it following this process. The way this was implemented in the previous QuickNXS, replicated here, is equivalent to adding an extra resolution, which then would have to be properly taken into account when fitting. In addition, the process doesn’t produce errors in intensity. It effectively only produces a pretty picture and should only be used as such.
- write_orso(output_data: Dict[str, ndarray])[source]
Save individual and combined reflectivity curves to ORSO format.
This function collects reflectivity data and metadata and saves one individual ORSO ASCII file per run and peak and one combined (stitched) ORSO ASCII file per peak.
- Parameters:
output_data (Dict[str, np.ndarray]) – Dictionary of scaled and stitched data where the key is the cross-section name and the value is an array of reflectivity data with shape (number of Q points, 5), where the 5 columns are: - Qz: normal momentum transfer - R: reflectivity - dR: reflectivity error - dQz: normal momentum transfer error - theta: scattering angle
- write_quicknxs(output_data: Dict[str, ndarray], output_file_base: str, xs: list | None = None, include_gisans: bool = False, include_offspec: bool = False)[source]
Write QuickNXS output reflectivity file.
- Parameters:
output_data – Dictionary of numpy arrays
output_file_base – Template for output file paths
xs – List of cross-sections available in the output_data
include_gisans – Include GISANS specifc variables in the reduction output file
include_offspec – Include Offspec specific variables in the reduction output file
quicknxs.interfaces.data_handling.quicknxs_io module
Read and write quicknxs reduced files.
- quicknxs.interfaces.data_handling.quicknxs_io.determine_which_files_to_sum(run_file, data_file_indices, run_number_str=None)[source]
Determine which files are summed when reading a saved reduction file.
The saved file has the correct run numbers (numors) in the line that starts with: # Input file indices, however the file does not contain the correct paths the way the file is read ignores any files that were summed in the processing from which the saved file was created.
- quicknxs.interfaces.data_handling.quicknxs_io.read_reduced_file(file_path: str, configuration=None)[source]
Read in configurations from a reduced data file.
- quicknxs.interfaces.data_handling.quicknxs_io.write_reflectivity_data(output_path: str, data: list | ndarray, col_names: List[str], as_5col: bool = True)[source]
Write out reflectivity header in a format readable by QuickNXS.
If as_5col is False, only the first four columns passed will be written.
- quicknxs.interfaces.data_handling.quicknxs_io.write_reflectivity_header(peak_reduction_lists: Dict[int, List[NexusData]], active_list_index: int, direct_beam_list: List[NexusData], output_path: str, pol_state: str, include_gisans: bool = False, include_offspec: bool = False)[source]
Write out reflectivity header in a format readable by QuickNXS.
- Parameters:
peak_reduction_lists – All reduction lists to include as additional peaks in the header
active_list_index – The index of the reduction list that the output reflectivity data is for
direct_beam_list – Direct beam list
output_path – Output file path
pol_state – Descriptor for the polarization state
include_gisans – Include GISANS specific variables
include_offspec – Include Offspec specific variables