pypeit.images.pypeitimage module

Module for the PypeItImage include its Mask

class pypeit.images.pypeitimage.PypeItCalibrationImage(image, ivar=None, nimg=None, amp_img=None, det_img=None, rn2img=None, base_var=None, img_scale=None, fullmask=None, detector=None, spat_flexure=None, PYP_SPEC=None, units=None, exptime=None, noise_floor=None, shot_noise=None, bpm=None, crmask=None, usermask=None, clean_mask=False)

Bases: PypeItImage, CalibFrame

Abstract base class used with PypeIt calibration images.

This class inherits the core datamodel attributes and functionality from PypeItImage, including the version number, but uses the naming conventions driven by CalibFrame. Some more-specific inheritance notes:

• Inheritance order matters! The order must be PypeItImage, then CalibFrame to ensure the correct method resolution order.

• The datamodel version is inherited from PypeItImage.

• There is no need to combine the datamodels because PYP_SPEC is already part of PypeItImage.

• The __init__ function is inherited from PypeItImage.

• The _bundle method is inherited from PypeItImage; the to_file method is inherited from CalibFrame; from_hdu is specific to this class because we need to combine the PypeItImage and CalibFrame functionality; all other IO methods are inherited directly from DataContainer.

_base_header(hdr=None)

Override the base class functions to combine the operations of both base classes.

classmethod from_hdu(hdu, chk_version=True, hdu_prefix=None, **kwargs)

Instantiate the object from an HDU extension.

This uses the PypeItImage.from_hdu() method, and then parses the necessary CalibFrame-specific attributes from one of the relevant headers.

Parameters:

• hdu (astropy.io.fits.HDUList, astropy.io.fits.ImageHDU, astropy.io.fits.BinTableHDU) – The HDU(s) with the data to use for instantiation.

• chk_version (bool, optional) – If True, raise an error if the datamodel version or type check failed. If False, throw a warning only.

• hdu_prefix (str, optional) – Only parse HDUs with extension names matched to this prefix. If None, hdu_prefix is used. If the latter is also None, no prefix is expected.

• **kwargs – Passed directly to _parse().

classmethod from_pypeitimage(pypeitImage, calib_dir=None, setup=None, calib_id=None, detname=None)

Instantiate the object from an existing PypeItImage.

This allows for the general construction of a PypeItImage that can then be subsequently decorated by an appropriate type with default attributes. See, e.g., ArcImage.

Note

• This is not a deep copy.

• To appropriately set the relevant CalibFrame internals, all of the optional arguments must be provided.

Parameters:

• pypeitImage (PypeItImage) – The input image to convert into the appropriate type.

• calib_dir (str, Path, optional) – Output directory for the processed calibration frames

• setup (str, optional) – The string identifier for the instrument setup/configuration; see unique_configurations().

• calib_id (str, list, int) – Identifiers for one or more calibration groups for this calibration frame. See ingest_calib_id().

• detname (str) – The identifier used for the detector or detector mosaic for the relevant instrument; see get_det_name().

Returns:

Image with the appropriate type and CalibFrame internals.

Return type:

PypeItImage

internals = ['process_steps', 'files', 'rawheadlist', 'calib_id', 'calib_key', 'calib_dir']

Combines internals from both base classes.

sub(other)

Over-ride subtraction function from PypeItImage to ensure that the internals are propagated.

class pypeit.images.pypeitimage.PypeItImage(image, ivar=None, nimg=None, amp_img=None, det_img=None, rn2img=None, base_var=None, img_scale=None, fullmask=None, detector=None, spat_flexure=None, PYP_SPEC=None, units=None, exptime=None, noise_floor=None, shot_noise=None, bpm=None, crmask=None, usermask=None, clean_mask=False)

Bases: DataContainer

Container class for processed PypeIt images and associated data.

Image data can be either from a single detector, multiple detectors arranged in a 3D array, or multiple detectors resampled to a single mosaic image. The orientation of the images is always spectral pixels along the first axis and spatial pixels along the second axis; i.e., the shape is \((N_{\rm spec},N_{\rm spat})\).

Class instantiation only requires the image data; all other elements of the datamodel are completely optional. If not defined, the datamodel elements are set to None, with the exception of the image mask (see below). Any code that uses a PypeItImage and accesses the datamodel components must be able to handle when any of the values are None.

The datamodel components are:

Version: 1.3.0

Attribute	Type	Array Type	Description
PYP_SPEC	str		PypeIt spectrograph name
amp_img	numpy.ndarray	numpy.integer	Provides the amplifier that contributed to each pixel. If this is a detector mosaic, this must be used in combination with det_img to select pixels for a given detector amplifier.
base_var	numpy.ndarray	numpy.floating	Base-level image variance, excluding count shot-noise
det_img	numpy.ndarray	numpy.integer	If a detector mosaic, this image provides the detector that contributed to each pixel.
detector	DetectorContainer, Mosaic		The detector (see DetectorContainer) or mosaic (see Mosaic) parameters
exptime	int, float		Effective exposure time (s)
fullmask	ImageBitMaskArray		Image mask
image	numpy.ndarray	numpy.floating	Primary image data
img_scale	numpy.ndarray	numpy.floating	Image count scaling applied (e.g., 1/flat-field)
ivar	numpy.ndarray	numpy.floating	Inverse variance image
nimg	numpy.ndarray	numpy.integer	If a combination of multiple images, this is the number of images that contributed to each pixel
noise_floor	float		Noise floor included in variance
rn2img	numpy.ndarray	numpy.floating	Read noise squared image
shot_noise	bool		Shot-noise included in variance
spat_flexure	float		Shift, in spatial pixels, between this image and SlitTrace
units	str		(Unscaled) Pixel units (e- or ADU)

Regardless of whether or not it is provided directly (see fullmask), the image mask is always instantiated. If fullmask is not provided, this instantiation begins with all values being unmasked. Although the class allows direct access/manipulation of fullmask (see ImageBitMaskArray), convenience functions are provided to interface with the underlying object; see update_mask() and select_flag(), as well as ImageBitMask for the valid flag names.

Additionally, pixels can be flagged on instantiation as being part of a bad-pixel mask (bpm), a cosmic ray hit (crmask), or a user-level bad-pixel mask (usermask). Importantly, if both fullmask and one of these masks are provided, the default behavior is to add, e.g., the crmask to the list of already flagged CRs provided by fullmask (if there are any). To remove any of the relevant flags before including the new flags in fullmask use clean_mask. See the class parameters below.

The following lists only those parameters that are not part of the class datamodel (see above).

Parameters:

• bpm (numpy.ndarray, optional) – The image bad-pixel mask, which typically selects image values that contain detector artifacts. This is a boolean array that must have the same shape as image.

• crmask (numpy.ndarray, optional) – A boolean array selecting pixels that contain cosmic-ray hits. Shape must match image.

• usermask (numpy.ndarray, optional) – A boolean array selecting pixels that the user wishes to ignore. Shape must match image.

• clean_mask (bool, optional) – If true, first remove any existing BPM, CR, or USER mask from fullmask before including the masks provided by bpm, crmask, or usermask.

files

List of source files

Type:

list

rawheadlst

List containing headers of the raw image file. If the source data is a stack of multiple files, this is the set of headers for the last file in the list.

Type:

list

process_steps

List of steps executed during processing of the image data; see process.

Type:

list

_base_header(hdr=None)

Build the generic header written to all PypeIt images.

Parameters:

hdr (astropy.io.fits.Header, optional) – Header object to update. The object is modified in-place and also returned. If None, an empty header is instantiated, edited, and returned.

Returns:

The initialized (or edited) fits header.

Return type:

astropy.io.fits.Header

_bundle()

Package the datamodel for writing.

Returns:

A list of dictionaries, each list element is written to its own fits extension. See DataContainer.

Return type:

list

build_crmask(par, subtract_img=None)

Identify and flag cosmic rays in the image.

This is mainly a wrapper to pypeit.core.procimg.lacosmic(), with the boolean cosmic-ray mask is saved to crmask.

Parameters:

• par (ProcessImagesPar) – Parameters that dictate the processing of the images. See ProcessImagesPar for the defaults.

• subtract_img (numpy.ndarray, optional) – An image to subtract from the primary image before executing the cosmic-ray detection algorithm. If provided, it must have the correct shape (see shape()).

Returns:

Boolean array with flagging pixels with identified cosmic rays; True mean a CR was flagged.

Return type:

numpy.ndarray

build_mask(saturation=None, mincounts=None, slitmask=None, from_scratch=True)

Construct the bit value mask used during extraction.

Warning

By default, this erases any existing mask and starts from scratch! See from_scratch.

The mask bit keys are defined by ImageBitMask. Assuming an instance of PypeItImage called img, any pixel with img.fullmask.mask == 0 is valid, otherwise the pixel has been masked. To determine why a given pixel has been masked (see also select_flag()):

reasons = img.fullmask.flagged_bits([0,0])

To get all the pixel masked for a specific set of reasons, e.g.:

has_cr = img.select_flag(flag='CR')
is_saturated = img.select_flag(flag='SATURATION')

Parameters:

• saturation (float, str, numpy.ndarray, optional) – Saturation limit (i.e., maximum allowed value). Any value above this is flagged with the SATURATION bit. Units must match the image. Can be a single float to use for all pixels or a numpy.ndarray for a pixel-dependent value (e.g., for an image mosaic); if the latter, the shape must match shape. Can also be a string, but, if so, the string must be 'default', and that means the saturation limit is set by detector. The tabulated saturation limit is assumed to be in ADU/DN, which will be converted to counts if units is ‘e-’, and the value used for the saturation includes the detector non-linearity threshold; see nonlinear_counts(). If None, pixels are not flagged for saturation.

• mincounts (float, str, numpy.ndarray, optional) – The minimum valid value; units must match the image. Any value below this is flagged with the MINCOUNTS bit. Can be a single float to use for all pixels or a numpy.ndarray for a pixel-dependent value (e.g., for an image mosaic); if the latter, the shape must match shape. Can also be a string, but, if so, the string must be 'default', and that means the minimum count threshold is set by detector. The tabulated minimum counts is assumed to be in e-, which will be converted to ADU/DN if units is ‘ADU’. If None, pixels are not flagged for a minimum value.

• slitmask (numpy.ndarray, optional) – Slit mask image. Pixels not in a slit are flagged with the OFFSLITS bit; see update_mask_slitmask().

• from_scratch (bool, optional) – Build the mask from scratch. That is, if fullmask already exists, all bits are turned off except for BPM and CR.

datamodel = {'PYP_SPEC': {'descr': 'PypeIt spectrograph name', 'otype': <class 'str'>}, 'amp_img': {'atype': <class 'numpy.integer'>, 'descr': 'Provides the amplifier that contributed to each pixel.  If this is a detector mosaic, this must be used in combination with ``det_img`` to select pixels for a given detector amplifier.', 'otype': <class 'numpy.ndarray'>}, 'base_var': {'atype': <class 'numpy.floating'>, 'descr': 'Base-level image variance, excluding count shot-noise', 'otype': <class 'numpy.ndarray'>}, 'det_img': {'atype': <class 'numpy.integer'>, 'descr': 'If a detector mosaic, this image provides the detector that contributed to each pixel.', 'otype': <class 'numpy.ndarray'>}, 'detector': {'descr': 'The detector (see :class:`~pypeit.images.detector_container.DetectorContainer`) or mosaic (see :class:`~pypeit.images.mosaic.Mosaic`) parameters', 'otype': (<class 'pypeit.images.detector_container.DetectorContainer'>, <class 'pypeit.images.mosaic.Mosaic'>)}, 'exptime': {'descr': 'Effective exposure time (s)', 'otype': (<class 'int'>, <class 'float'>)}, 'fullmask': {'descr': 'Image mask', 'otype': <class 'pypeit.images.imagebitmask.ImageBitMaskArray'>}, 'image': {'atype': <class 'numpy.floating'>, 'descr': 'Primary image data', 'otype': <class 'numpy.ndarray'>}, 'img_scale': {'atype': <class 'numpy.floating'>, 'descr': 'Image count scaling applied (e.g., 1/flat-field)', 'otype': <class 'numpy.ndarray'>}, 'ivar': {'atype': <class 'numpy.floating'>, 'descr': 'Inverse variance image', 'otype': <class 'numpy.ndarray'>}, 'nimg': {'atype': <class 'numpy.integer'>, 'descr': 'If a combination of multiple images, this is the number of images that contributed to each pixel', 'otype': <class 'numpy.ndarray'>}, 'noise_floor': {'descr': 'Noise floor included in variance', 'otype': <class 'float'>}, 'rn2img': {'atype': <class 'numpy.floating'>, 'descr': 'Read noise squared image', 'otype': <class 'numpy.ndarray'>}, 'shot_noise': {'descr': 'Shot-noise included in variance', 'otype': <class 'bool'>}, 'spat_flexure': {'descr': 'Shift, in spatial pixels, between this image and SlitTrace', 'otype': <class 'float'>}, 'units': {'descr': '(Unscaled) Pixel units (e- or ADU)', 'otype': <class 'str'>}}

Data model components.

classmethod from_hdu(hdu, chk_version=True, hdu_prefix=None)

Instantiate the object from an HDU extension.

This overrides the base-class method. Overriding this method is preferrable to overriding the _parse method because it makes it easier to deal with the multiple DataContainer objects contained by PypeItImage.

Parameters:

• hdu (astropy.io.fits.HDUList, astropy.io.fits.ImageHDU, astropy.io.fits.BinTableHDU) – The HDU(s) with the data to use for instantiation.

• chk_version (bool, optional) – If True, raise an error if the datamodel version or type check failed. If False, throw a warning only.

• hdu_prefix (str, optional) – Only parse HDUs with extension names matched to this prefix. If None, hdu_prefix is used. If the latter is also None, no prefix is expected.

classmethod from_pypeitimage(pypeitImage)

Instantiate the object from an existing PypeItImage.

This allows for the general construction of a PypeItImage that can then be subsequently decorated by an appropriate type with default attributes. See, e.g., ArcImage.

Note that this is not a deep copy.

Parameters:

pypeitImage (PypeItImage) – The input image to convert into the appropriate type.

internals = ['process_steps', 'files', 'rawheadlist']

A list of strings identifying a set of internal attributes that are not part of the datamodel.

property is_mosaic

Flag if the image is a mosaic of multiple detectors.

property is_multidetector

Flag if the “image” has more than one detector image that has not been resampled into a mosaic image.

map_detector_value(attr)

Provided a detector-specific value, remap it as necessary for numpy operations with image.

Parameters:

attr (str) – Attribute of DetectorContainer to map to an image, vector, or scalar.

Returns:

An array that can be mapped for use with image containing the detector values.

Return type:

scalar, numpy.ndarray

reinit_mask()

Reset the mask; fullmask is reset to be 0 everywhere.

select_flag(flag=None, invert=False)

Return a boolean array that selects pixels masked with the specified bits in fullmask.

For example, to create a bad-pixel mask based on which pixels have cosmic-ray detections, run:

cr_bpm = self.select_flag(flag='CR')

Or, to create a good-pixel mask for all pixels that are not flagged for any reason, run:

gpm = self.select_flag(invert=True)

Parameters:

• flag (str, array-like, optional) – One or more flags to select when returning the boolean mask. If None, pixels flagged for any reason are returned as True.

• invert (bool, optional) – If False, the return mask is True for masked pixels, False for good pixels (i.e., a bad-pixel mask). If True, invert the sense of the mask (i.e., create a good-pixel mask, True for good pixels, False for bad pixels).

Returns:

Boolean array where pixels with the selected bits flagged are returned as True (if invert is False); i.e., this is a boolean bad-pixel mask (or a good-pixel mask when invert is True). If flag is not provided, pixels flagged for any reason are returned as True.

Return type:

numpy.ndarray

property shape

Return the primary image shape.

show()

Show the image in a ginga viewer.

sub(other)

Subtract this PypeItImage from another.

The following operations are performed:

• the image data is subtracted (images must have the same shape)

• the inverse variance (ivar) is propagated

• the number of images is combined (nimg)

• the RN2 (rn2img) is propagated

• the base variance (base_var) is propagated

• the image scaling (img_scale) is averaged

• the bit mask (fullmask) is joined (using an or operation)

• if it’s the same for both images, the spectrograph name (PYP_SPEC) is propagated

• if it’s the same for both images, the images units (units) is propagated

• if both images provide source file names, the file lists are concatenated

• the detector from the first image (self) is used for the returned image and the detector for the other image is ignored

• if the spatial flexure is defined for the first image, it is propagated regardless of the value for the 2nd image. If it is also defined for the 2nd image and the flexure is different from the first image, a warning is issued. If the flexure is only defined for the 2nd image, it is ignored.

Parameters:

other (PypeItImage) – The image to subtract

Returns:

The result of subtracting other from this image.

Return type:

PypeItImage

update_mask(flag, indx=None, action='turn_on')

Update fullmask by operating on the bits for the provided (list of) flags.

This method alters fullmask in-place.

Parameters:

• flag (str, array-like) – One or more flags to turn on for the selected pixels.

• indx (tuple, slice, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to toggle the provided bit flags. I.e., for the internal fullmask, fullmask.mask[indx] must be a valid (fancy indexing) operation. If None, the action is performed for the full mask!

• action (str, optional) – The action to perform. Must be 'turn_on' or 'turn_off'.

update_mask_cr(crmask_new)

Update the mask bits for cosmic rays

The original are turned off and the new ones are turned on.

Parameters:

crmask_new (numpy.ndarray) – New CR mask

update_mask_slitmask(slitmask)

Update a mask using the slitmask

Parameters:

slitmask (numpy.ndarray) – Slitmask with -1 values pixels not in a slit

version = '1.3.0'

Datamodel version number