pypeit.images.bitmaskarray module

General utility for bit-based masking.

Class usage examples

BitMaskArray objects combine the numpy.ndarray that holds the bit values and the BitMask object used to interpret them into a new subclass of DataContainer. The following builds on the example uses of BitMask objects (see Bitmasks).

Defining a new subclass

Given the definition of ImageBitMask, we can implement a relevant BitMaskArray subclass as follows:

from pypeit.images.bitmaskarray import BitMaskArray

class ImageBitMaskArray(BitMaskArray):
    version = '1.0'
    bitmask = ImageBitMask()

The remaining functionality below is all handled by the base class.

To instantiate a new 2D mask array that is 5 pixels on a side:

shape = (5,5)
mask = ImageBitMaskArray(shape)

You can access the bit flag names using:

>>> mask.bit_keys()
['BPM', 'COSMIC', 'SATURATED']

Bit access

You can flag bits using turn_on(). For example, the following code flags the center column of the image as being part of the detector bad-pixel mask:

import numpy as np
mask.turn_on('BPM', select=np.s_[:,2])

The select argument to turn_on() can be anything that is appropriately interpreted as slicing a numpy.ndarray. That is, arr[select] must be valid, where arr is the internal array held by mask.

Similarly, you can flag a pixel with a cosmic ray:

mask.turn_on('COSMIC', select=(0,0))

or multiple pixels as being saturated:

mask.turn_on('SATURATED', select=([0,1,-1,-1],[0,0,-1,-2]))

and you can simultaneously flag pixels for multiple reasons:

mask.turn_on(['COSMIC', 'SATURATED'], select=([-1,-1],[0,1]))

The mask values themselves are accessed using the mask attribute:

>>> mask.mask
array([[6, 0, 1, 0, 0],
       [4, 0, 1, 0, 0],
       [0, 0, 1, 0, 0],
       [0, 0, 1, 0, 0],
       [6, 6, 1, 4, 4]], dtype=int16)

However, more usefully, you can obtain boolean arrays that select pixels flagged by one or more flags:

>>> mask.flagged(flag='SATURATED')
array([[ True, False, False, False, False],
       [ True, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [ True,  True, False,  True,  True]])

>>> mask.flagged(flag=['BPM', 'SATURATED'])
array([[ True, False,  True, False, False],
       [ True, False,  True, False, False],
       [False, False,  True, False, False],
       [False, False,  True, False, False],
       [ True,  True,  True,  True,  True]])

If you want to select all pixels that are not flagged by a given flag, you can use the invert option in flagged():

>>> gpm = mask.flagged(flag='BPM', invert=True)
>>> gpm
array([[ True,  True, False,  True,  True],
       [ True,  True, False,  True,  True],
       [ True,  True, False,  True,  True],
       [ True,  True, False,  True,  True],
       [ True,  True, False,  True,  True]])

For individual flags, there is also convenience functionality that allows you to access a boolean array as if it were an attribute of the object:

>>> mask.bpm
array([[False, False,  True, False, False],
       [False, False,  True, False, False],
       [False, False,  True, False, False],
       [False, False,  True, False, False],
       [False, False,  True, False, False]])
>>> mask.saturated
array([[ True, False, False, False, False],
       [ True, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [ True,  True, False,  True,  True]])

This convenience operation is identical to calling flagged() for the indicated bit. However bpm is not an array that can be used to change the value of the bits themselves:

>>> indx = np.zeros(shape, dtype=bool)
>>> indx[2,3] = True
>>> mask.bpm = indx # Throws an AttributeError

Instead, you must use the bit toggling functions provided by the class: turn_on(), turn_off(), or toggle().

Tip

Every time flagged() is called, a new array is created. If you need to access to the result of the function multiple times without changing the flags, you’re better of assigning the result to a new array and then using that array so that you’re not continually allocating and deallocating memory (even within the context of how this is done within python).

Input/Output

As a subclass of DataContainer, you can save and read the bitmask data to/from files:

>>> mask.to_file('mask.fits')
>>> _mask = ImageBitMaskArray.from_file('mask.fits')
>>> np.array_equal(mask.mask, _mask.mask)
True

In addition to the mask data, the bit flags and values are also written to the header; see the BIT* entries in the header below:

>>> from astropy.io import fits
>>> hdu = fits.open('mask.fits')
>>> hdu.info()
Filename: mask.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      13   ()
  1  MASK          1 ImageHDU        22   (5, 5)   int16
>>> hdu['MASK'].header
XTENSION= 'IMAGE   '           / Image extension
BITPIX  =                   16 / array data type
NAXIS   =                    2 / number of array dimensions
NAXIS1  =                    5
NAXIS2  =                    5
PCOUNT  =                    0 / number of parameters
GCOUNT  =                    1 / number of groups
VERSPYT = '3.9.13  '           / Python version
VERSNPY = '1.22.3  '           / Numpy version
VERSSCI = '1.8.0   '           / Scipy version
VERSAST = '5.0.4   '           / Astropy version
VERSSKL = '1.0.2   '           / Scikit-learn version
VERSPYP = '1.10.1.dev260+g32de3d6d4' / PypeIt version
DATE    = '2022-11-10'         / UTC date created
DMODCLS = 'ImageBitMaskArray'  / Datamodel class
DMODVER = '1.0     '           / Datamodel version
BIT0    = 'BPM     '
BIT1    = 'COSMIC  '
BIT2    = 'SATURATED'
EXTNAME = 'MASK    '           / extension name
CHECKSUM= 'APGODMFOAMFOAMFO'   / HDU checksum updated 2022-11-10T13:10:27
DATASUM = '1245200 '           / data unit checksum updated 2022-11-10T13:10:27

Note

Currently, when loading a mask, the bit names in the header of the output file are not checked against the bitmask definition in the code itself. This kind of version control should be handled using the version attribute of the class. I.e., anytime the flags in the bitmask are changed, the developers should bump the class version.

class pypeit.images.bitmaskarray.BitMaskArray(shape, asuint=False)[source]

Bases: DataContainer

Object that holds both the mask data and the mask interpretation object.

This is an abstract class that should not be directly instantiated.

Parameters:

shape (tuple) – Shape of the mask to create.
asuint (bool, optional) – When setting the data-type for the mask array (see minimum_dtype()), use an unsigned integer instead of a signed integer (e.g., uint16 instead of int16).

__and__(other)[source]: Override and operation for mask.

__getattr__(item)[source]

Override the attribute access to allow for immediate construction and access to boolean arrays that select array elements with the provided flag.

For example, if 'BPM' is a flag in the bitmask, and mask is an instance of the the relevant subclass, mask.bpm is identical to calling mask.flagged(flag='BPM').

Warning

Using this functionality creates a new numpy array every time it is called. This can be slow for large arrays. This means that, if you need to access the boolean array for a given flag multiple times, you should set it to a new object (e.g., maskbpm = mask.bpm)!

If the attribute (item) requested is not one of the bitmask flags, the DataContainer base class function is called.

Parameters:: item (object) – The attribute being accessed.

__getitem__(item)[source]: Allow direct access to the mask.

__or__(other)[source]: Override or operation for mask.

_bundle()[source]

Override the base-class bundle method so that the bitmask keys can be added to the header.

Returns:: List of dictionaries indicating what should be written to a file.
Return type:: list

_set_keys()[source]: Set lower_keys, which are needed for the bit access convenience method.

bit_keys()[source]: Return a list of the bit keywords.

bitmask = None: BitMask object used to interpret the bit array. Must be defined by the subclass. When defining subclasses, note that the bitmask flags must be case-insensitive strings.

property bits: Return the bit dictionary.

consolidate(flag_set, consolidated_flag)[source]

Consolidate a set of flags into a single flag.

That is, any bit flagged with any of the flags provided by flag_set will also be flagged by consolidate_flag after executing this function.

Parameters:

flag_set (str, array-like) – List of flags that are consolidated into a single flag.
consolidated_flag (str) – Consolidated flag name.

copy()[source]: Create a deep copy.

datamodel = {'mask': {'atype': <class 'numpy.integer'>, 'descr': 'Bitmask values', 'otype': <class 'numpy.ndarray'>}}: Datamodel is simple, containing only the mask array.

flagged(flag=None, invert=False)[source]

Determine if a bit is on in the provided bitmask value. The function can be used to determine if any individual bit is on or any one of many bits is on.

Parameters:

flag (str, array-like, optional) – One or more bit names to check. If None, then it checks if any bit is on.
invert (bool, optional) – Invert the boolean such that unflagged pixels are True and flagged pixels are False.

Returns:

Boolean array indicating where the internal bitmask is flagged by the selected bits. Flagged values are True, unflagged values are False. If invert is True, this is reversed.

Return type:

numpy.ndarray

flagged_bits(index)[source]

Return the list of flagged bit names for a single bit value.

Parameters:: index (tuple) – Tuple with the indices in the array.
Returns:: List of flagged bit value keywords.
Return type:: list

classmethod from_array(arr)[source]

classmethod from_hdu(hdu, chk_version=True, **kwargs)[source]

Instantiate the object from an HDU extension.

This overrides the base-class method, only to add checks (or not) for the bitmask.

Parameters:

hdu (astropy.io.fits.HDUList, astropy.io.fits.ImageHDU, astropy.io.fits.BinTableHDU) – The HDU(s) with the data to use for instantiation.
chk_version (bool, optional) – If True, raise an error if the datamodel version or type check failed. If False, throw a warning only.
**kwargs – Passed directly to _parse().

info()[source]: Print the list of bits and, if available, their descriptions.

internals = ['lower_keys']: A list of strings identifying a set of internal attributes that are not part of the datamodel.

property shape: Return the shape of the internal array.

toggle(flag, select=None)[source]

Toggle bits for selected array elements.

Parameters:

flag (str, array-like) – Bit name(s) to toggle.
select (tuple, slice, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to toggle the provided bit flags. I.e., for the internal mask, mask[select] must be a valid (fancy indexing) operation. If None, the bit is toggled for the full mask!

turn_off(flag, select=None)[source]

Ensure that a bit is turned off in the provided bitmask value.

Parameters:

flag (str, array-like) – Bit name(s) to turn off.
select (tuple, slice, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to turn off the provided bit flags. I.e., for the internal mask, mask[select] must be a valid (fancy indexing) operation. If None, the bit is turned off for the full mask!

turn_on(flag, select=None)[source]

Ensure that a bit is turned on for the selected elements.

Parameters:

flag (str, array-like) – Bit name(s) to turn on.
select (tuple, slice, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to turn on the provided bit flags. I.e., for the internal mask, mask[select] must be a valid (fancy indexing) operation. If None, the bit is turned on for the full mask!

version = None: DataContainer version. Must be defined by the subclass.