pypeit.calibrations module

Class for guiding calibration object generation in PypeIt.

class pypeit.calibrations.Calibrations(fitstbl, par, spectrograph, caldir, calib_ID, frame, det, qadir=None, reuse_calibs=False, show=False, user_slits=None, chk_version=True, state=None)[source]

Bases: object

Class designed to guide the generation of calibration images and objects in PypeIt.

Parameters:

fitstbl (PypeItMetaData) – The class holding the metadata for all the frames in this PypeIt run. If None, we are using this class as a glorified dict to hold the objects.
par (CalibrationsPar) – Parameter set defining optional parameters of PypeIt’s algorithms for Calibrations
spectrograph (Spectrograph) – Spectrograph object
caldir (str, Path) – Path for the processed calibration files.
qadir (str, optional) – Path for quality assessment output. If not provided, no QA plots are saved.
reuse_calibs (bool, optional) – Instead of reprocessing them, load existing calibration files from disk if they exist.
show (bool, optional) – Show plots of PypeIt’s results as the code progresses. Requires interaction from the user.
user_slits (dict, optional) – A limited set of slits selected by the user for analysis. See user_mask().
chk_version (bool, optional) – When reading in existing files written by PypeIt, perform strict version checking to ensure a valid file. If False, the code will try to keep going, but this may lead to faults and quiet failures. User beware!

Variables:

fitstbl (PypeItMetaData) – See instantiation arguments.
par (CalibrationsPar) – See instantiation arguments.
spectrograph (Spectrograph) – See instantiation arguments.
calib_dir (Path) – Path for the processed calibration files.
qa_path (Path) – Path for the QA diagnostics.
reuse_calibs (bool) – See instantiation arguments.
show (bool) – See instantiation arguments.
user_slits (dict) – See instantiation arguments.
det (int, tuple) – The single detector or set of detectors in a mosaic to process.
frame (int) – The index of a raw file in fitstbl used to set the calibration group.
calib_ID (int) – The calibration group associated with frame.
msarc (ArcImage) – Arc calibration frame
mstilt (TiltImage) – Tilt calibration frame
alignments (Alignments) – Alignment calibration frame
msbias (BiasImage) – Bias calibration frame
msdark (DarkImage) – Dark calibration frame
msbpm (numpy.ndarray) – Boolean array with the bad-pixel mask (pixels that should masked are set to True).
wv_calib (WaveCalib) – Wavelength calibration frame
slits (SlitTraceSet) – Slit tracing calibration frame
wavetilts (WaveTilts) – Tilts calibration frame
flatimages (FlatImages) – Flat-field calibration frame
steps (list) – A list of strings setting the set of processing steps to be completed (not necessarily those that were successful). See the default_steps functions of each subclass.
success (bool) – Flag that the calibrations were all generated successfully.
failed_step (str) – If the calibrations were unsuccessful, this is the step that led to the fault.

_chk_objs(items)[source]

Check that the input items exist internally as attributes

Parameters:: items (list) – List of required items for the calibration step
Returns:: True if all exist or if all were successfully loaded, False
Return type:: bool

_chk_set(items)[source]

Check whether a needed attribute has previously been set

Parameters:: items (list) – Attributes to check

_flat_correction_images()[source]

Reconstruct the applied flat-field correction images present in the current FlatImages product.

The set mirrors what FlatImages.show() displays: pixel-to-pixel response, slit (spatial) illumination, and spectral illumination. Each image hovers about 1.0, so its mean/scatter characterize the correction.

Generated by JXP and Claude.

Returns:: Mapping of correction name ('pixelflat', 'spat_illum', 'spec_illum') to its full-detector image (numpy.ndarray). Empty if there are no flats or no slits.
Return type:: dict

_flat_qa_files()[source]

Find the flat-field QA PNGs associated with this calibration.

Generated by JXP and Claude.

Returns:: Sorted list of QA PNG paths (as strings) for this flat’s calibration key. Empty if QA is disabled or none found.
Return type:: list

_slits_qa_files()[source]

Find the slit/order QA PNGs for this calibration — chiefly the echelle order-prediction figure (Edges_*_orders_qa).

Generated by JXP and Claude.

Returns:: Sorted list of QA PNG paths (as strings) starting with Edges and carrying this calibration’s key; empty if QA is disabled or none found.
Return type:: list

_wv_calib_qa_files()[source]

Find the wavelength-calibration QA PNGs for this calibration.

Generated by JXP and Claude.

Returns:: Sorted list of QA PNG paths (as strings) — the per-slit Arc_1dfit_* / Arc_FWHMfit_* figures carrying this calibration’s key. Empty if QA is disabled or none found.
Return type:: list

arc_state()[source]

static association_summary(ofile, fitstbl, spectrograph, caldir, subset=None, det=None, overwrite=False)[source]

Write a file listing the associations between the processed calibration frames and their source raw files for every setup and every calibration group.

Parameters:

ofile (str, Path) – Full path to the output file.
fitstbl (PypeItMetaData) – The class holding the metadata for all the frames to process.
spectrograph (pypeit.spectrographs.spectrograph.Spectrograph) – Spectrograph object
caldir (str, Path) – Path for the processed calibration frames.
subset (numpy.ndarray, optional) – A boolean array selecting a subset of rows from fitstbl for output.
det (int, tuple, optional) – The specific detector (or mosaic) to use when constructing the output processed calibration group file names. If None, a placeholder is used.
overwrite (bool, optional) – Overwrite any existing file of the same name.

base_state(step, object)[source]

bias_state()[source]

bpm_state()[source]

check_calibrations(file_list, check_lamps=True)[source]

Check if the input calibration files are consistent with each other. This step is usually needed when combining calibration frames of a given type. This routine currently only prints out warning messages if the calibration files are not consistent.

Note: The exposure times are currently checked in the combine step, so they are not checked here.

Parameters:

file_list (list) – List of calibration files to check
check_lamps (bool, optional) – Check if the lamp status is the same for all the files. Default is True.

dark_state()[source]

static default_steps()[source]

This defines the steps for calibrations and their order: Note that the order matters!

Returns:: Calibration steps, in order of execution
Return type:: list

find_calibrations(frametype, frameclass)[source]

Find calibration files and identifiers.

Parameters:

frametype (str) – Calibration frame type. Must be a valid frame type; see valid_frametype().
frameclass (CalibFrame) – The subclass used to store the processed calibration data.

Returns:

raw_files (list) – The list of raw files in fitstbl with the provided frametype.
cal_file (Path) – The path with/for the processed calibration frame
calib_key (str) – The calibration identifier
setup (str) – The setup/configuration identifier
calib_id (list) – The calibration groups
detname (str) – The detector/mosaic identifier

flats_state()[source]

Record the flat-field state: status, output/input files, pixel-flat provenance, which corrections are present, the flat QA files, and a per-slit status with per-correction mean/RMS metrics.

Generated by JXP and Claude.

get_align(force=None)[source]

Load or generate the alignment calibration frame.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed alignment image.
Return type:: Alignments

get_arc(force=None)[source]

Load or generate the arc calibration frame.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image.
Return type:: ArcImage

static get_association(fitstbl, spectrograph, caldir, setup, calib_ID, det, must_exist=True, subset=None, include_science=False, proc_only=False)[source]

Construct a dictionary with the association between raw files and processed calibration frames.

Parameters:

fitstbl (PypeItMetaData) – The class holding the metadata for all the frames to process.
spectrograph (pypeit.spectrographs.spectrograph.Spectrograph) – Spectrograph object
caldir (str, Path) – Path for the processed calibration frames.
setup (str) – The setup/configuration of the association.
calib_ID (str, int) – The single calibration group of the association.
det (int, tuple) – The detector/mosaic of the association.
must_exist (bool, optional) – If True, only existing calibration frames in the association are included. If False, the nominal set of processed calibration frame file names are returned, regardless of whether or not they exist.
subset (numpy.ndarray, optional) – A boolean array selecting a subset of rows from fitstbl for output.
include_science (bool, optional) – Include science and standard frames in the association. This parameter is mutually exclusive with proc_only; if both are true, proc_only takes precedence.
proc_only (bool, optional) – If True, only return a dictionary with the names of the processed calibration frames. The dictionary sets the calibration directory to DIR, and the other keys are the capitalized versions of the calibration type keywords; e.g., asn['ARC'] is the processed arc frame. This parameter is mutually exclusive with include_science; if both are true, proc_only takes precedence.

Returns:

The set of raw and processed calibration frames associated with the selected calibration group. This only includes the processed frames if proc_only is True, and it includes the science/standard frames if include_science is True.

Return type:

dict

get_bias(force=None)[source]

Load or generate the bias calibration frame.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image.
Return type:: BiasImage

get_bpm(frame=None, force=None)[source]

Load or generate the bad pixel mask.

This is primarily a wrapper for bpm().

Parameters:

force (str, optional) – Currently ignored
frame (int, optional) – The row index in fitstbl

Returns:

The bad pixel mask, which should match the shape and orientation of a trimmed and PypeIt-oriented science image!

Return type:

numpy.ndarray

get_dark(force=None)[source]

Load or generate the dark calibration frame.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image.
Return type:: DarkImage

get_flats(force=None)[source]

Load or generate the flat-field calibration images.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image.
Return type:: FlatImages

static get_instance(fitstbl, par, spectrograph, caldir, calib_ID, frame, det, **kwargs)[source]

Get the instance of the appropriate subclass of Calibrations to use for reducing data from the provided spectrograph. For argument descriptions, see Calibrations.

get_scattlight(force=None)[source]

Load or generate the scattered light model.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image including the model.
Return type:: ScatteredLight

get_slits(force=None)[source]

Load or generate the definition of the slit boundaries.

Returns:: Traces of the slit edges; also kept internally as slits.
Return type:: SlitTraceSet

get_tiltimg(force=None)[source]

Load or generate the tilt calibration frame.

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: The processed calibration image.
Return type:: TiltImage

get_tilts(force=None)[source]

Load or generate the wavelength tilts calibration frame

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: Object containing the wavelength tilt calibration.
Return type:: WaveTilts

get_wv_calib(force=None)[source]

Load or generate the 1D wavelength calibrations

Parameters:: force (str, optional) – ‘remake’ – Force the frame to be remade. ‘reload’ – Reload the frame if it exists. None – Load the existing frame if it exists and reuse_calibs=True
Returns:: Object containing wavelength calibrations and the updated slit mask array.
Return type:: WaveCalib

process_load_selection(frame, cal_file, force)[source]

Process how pypeit should use any pre-existing calibration files.

If loading is requested but the calibration file (cal_file) does not exist, self.success is set to False, and None is returned.

Parameters:

frame (dict) – A dictionary with two elements: type is the string defining the frame type and class is the pypeit class used to load the pre-existing calibration file.
cal_file (str, Path) – Path to the calibration file.
force (str) –
Defines how to treat a pre-existing calibration file. Must be one of the following options:
- 'remake': Force the calibration be remade.
- 'reload': Reload the frame if it exists.
- None: Load the existing frame if it exists and self.reuse_calibs=True.

Returns:

Either the loaded calibration object or None.

Return type:

object

run_the_steps(stop_at_step=None, reload_only=False, status_only=False)[source]

Run full the full recipe of calibration steps.

scattlight_state()[source]

slits_state()[source]

step_frame_map = {'align': ('align', <class 'pypeit.alignframe.Alignments'>), 'arc': ('arc', <class 'pypeit.images.buildimage.ArcImage'>), 'bias': ('bias', <class 'pypeit.images.buildimage.BiasImage'>), 'bpm': None, 'dark': ('dark', <class 'pypeit.images.buildimage.DarkImage'>), 'flats': ('illumflat', <class 'pypeit.flatfield.FlatImages'>), 'scattlight': ('scattlight', <class 'pypeit.scattlight.ScatteredLight'>), 'slits': ('trace', <class 'pypeit.slittrace.SlitTraceSet'>), 'tiltimg': ('tilt', <class 'pypeit.images.buildimage.TiltImage'>), 'tilts': ('tilt', <class 'pypeit.wavetilts.WaveTilts'>), 'wv_calib': ('arc', <class 'pypeit.wavecalib.WaveCalib'>)}

tiltimg_state()[source]

tilts_state()[source]

wv_calib_state()[source]

class pypeit.calibrations.IFUCalibrations(fitstbl, par, spectrograph, caldir, calib_ID, frame, det, qadir=None, reuse_calibs=False, show=False, user_slits=None, chk_version=True, state=None)[source]

Bases: Calibrations

Child of Calibrations class for performing IFU calibrations. See Calibrations for arguments.

static default_steps()[source]

This defines the steps for calibrations and their order

Returns:: Calibration steps, in order of execution
Return type:: list

class pypeit.calibrations.MultiSlitCalibrations(fitstbl, par, spectrograph, caldir, calib_ID, frame, det, qadir=None, reuse_calibs=False, show=False, user_slits=None, chk_version=True, state=None)[source]

Bases: Calibrations

Calibration class for performing multi-slit calibrations (and also long-slit and echelle). See Calibrations for arguments.

Note

Calibrations are not sufficiently different yet to warrant a different class for echelle reductions. This may change if a different order is eventually required for the set of processing steps (see default_steps()).

static default_steps()[source]

This defines the calibration steps and their order.

Returns:: Calibration steps, in order of execution.
Return type:: list

pypeit.calibrations.check_for_calibs(par, fitstbl, raise_error=True, cut_cfg=None)[source]

Perform a somewhat quick and dirty check to see if the user has provided all of the calibration frametype’s to reduce the science frames

Parameters:

par (PypeItPar)
fitstbl (PypeItMetaData, None) – The class holding the metadata for all the frames in this PypeIt run.
raise_error (bool, optional) – If True, crash out
cut_cfg (numpy.ndarray, optional) – Also cut on this restricted configuration (mainly for chk_calibs)

Returns:

True if we passed all checks

Return type:

bool

pypeit.calibrations.required_calibs(par, fitstbl, spectrograph, run_state)[source]

Determine which calibration steps are required for the given PypeIt file and update the state object accordingly.

This is based on the logic in check_for_calibs() but instead of raising errors for missing frames, it sets the required flag on each calibration step entry in the state.

The mapping from frametypes to calibration steps is:

arc frames → arc and wv_calib steps

tilt frames → tiltimg and tilts steps

trace frames → slits step

bias frames → bias step (if use_biasimage is True)

dark frames → dark step (if use_darkimage is True)

pixelflat/illumflat frames → flats step (if use_pixelflat or use_illumflat is True)

scattlight frames → scattlight step (if subtract_scattlight is True)

align frames → align step (IFU only)

Parameters:

par (PypeItPar) – The full parameter set for the reduction.
fitstbl (PypeItMetaData) – The class holding the metadata for all the frames in this PypeIt run.
spectrograph (Spectrograph) – The spectrograph object for the instrument being reduced.
run_state (RunPypeItState) – The state object to update. Each calibration step entry will have its required field set to True or False.

Returns:

The updated state object.

Return type:

RunPypeItState