pypeit.images.bitmaskarray module
General utility for bit-based masking.
Class usage examples
BitMaskArray
objects combine the numpy.ndarray
that holds the bit values and the BitMask
object used
to interpret them into a new subclass of
DataContainer
. The following builds on the example
uses of BitMask
objects (see Bitmasks).
Defining a new subclass
Given the definition of ImageBitMask
, we can implement a relevant
BitMaskArray
subclass as follows:
from pypeit.images.bitmaskarray import BitMaskArray
class ImageBitMaskArray(BitMaskArray):
version = '1.0'
bitmask = ImageBitMask()
The remaining functionality below is all handled by the base class.
To instantiate a new 2D mask array that is 5 pixels on a side:
shape = (5,5)
mask = ImageBitMaskArray(shape)
You can access the bit flag names using:
>>> mask.bit_keys()
['BPM', 'COSMIC', 'SATURATED']
Bit access
You can flag bits using turn_on()
. For
example, the following code flags the center column of the image as being
part of the detector bad-pixel mask:
import numpy as np
mask.turn_on('BPM', select=np.s_[:,2])
The select
argument to
turn_on()
can be anything that
is appropriately interpreted as slicing a numpy.ndarray. That is,
arr[select]
must be valid, where arr
is the internal array held by
mask
.
Similarly, you can flag a pixel with a cosmic ray:
mask.turn_on('COSMIC', select=(0,0))
or multiple pixels as being saturated:
mask.turn_on('SATURATED', select=([0,1,-1,-1],[0,0,-1,-2]))
and you can simultaneously flag pixels for multiple reasons:
mask.turn_on(['COSMIC', 'SATURATED'], select=([-1,-1],[0,1]))
The mask values themselves are accessed using the mask
attribute:
>>> mask.mask
array([[6, 0, 1, 0, 0],
[4, 0, 1, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 1, 0, 0],
[6, 6, 1, 4, 4]], dtype=int16)
However, more usefully, you can obtain boolean arrays that select pixels flagged by one or more flags:
>>> mask.flagged(flag='SATURATED')
array([[ True, False, False, False, False],
[ True, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False],
[ True, True, False, True, True]])
>>> mask.flagged(flag=['BPM', 'SATURATED'])
array([[ True, False, True, False, False],
[ True, False, True, False, False],
[False, False, True, False, False],
[False, False, True, False, False],
[ True, True, True, True, True]])
If you want to select all pixels that are not flagged by a given flag, you
can use the invert
option in
flagged()
:
>>> gpm = mask.flagged(flag='BPM', invert=True)
>>> gpm
array([[ True, True, False, True, True],
[ True, True, False, True, True],
[ True, True, False, True, True],
[ True, True, False, True, True],
[ True, True, False, True, True]])
For individual flags, there is also convenience functionality that allows you to access a boolean array as if it were an attribute of the object:
>>> mask.bpm
array([[False, False, True, False, False],
[False, False, True, False, False],
[False, False, True, False, False],
[False, False, True, False, False],
[False, False, True, False, False]])
>>> mask.saturated
array([[ True, False, False, False, False],
[ True, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False],
[ True, True, False, True, True]])
This convenience operation is identical to calling
flagged()
for the indicated bit.
However bpm
is not an array that can be used to change the value of the
bits themselves:
>>> indx = np.zeros(shape, dtype=bool)
>>> indx[2,3] = True
>>> mask.bpm = indx # Throws an AttributeError
Instead, you must use the bit toggling functions provided by the class:
turn_on()
,
turn_off()
, or
toggle()
.
Tip
Every time flagged()
is
called, a new array is created. If you need to access to the result of the
function multiple times without changing the flags, you’re better of
assigning the result to a new array and then using that array so that you’re
not continually allocating and deallocating memory (even within the context
of how this is done within python).
Input/Output
As a subclass of DataContainer
, you can save and read
the bitmask data to/from files:
>>> mask.to_file('mask.fits')
>>> _mask = ImageBitMaskArray.from_file('mask.fits')
>>> np.array_equal(mask.mask, _mask.mask)
True
In addition to the mask data, the bit flags and values are also written to the
header; see the BIT*
entries in the header below:
>>> from astropy.io import fits
>>> hdu = fits.open('mask.fits')
>>> hdu.info()
Filename: mask.fits
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 13 ()
1 MASK 1 ImageHDU 22 (5, 5) int16
>>> hdu['MASK'].header
XTENSION= 'IMAGE ' / Image extension
BITPIX = 16 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 5
NAXIS2 = 5
PCOUNT = 0 / number of parameters
GCOUNT = 1 / number of groups
VERSPYT = '3.9.13 ' / Python version
VERSNPY = '1.22.3 ' / Numpy version
VERSSCI = '1.8.0 ' / Scipy version
VERSAST = '5.0.4 ' / Astropy version
VERSSKL = '1.0.2 ' / Scikit-learn version
VERSPYP = '1.10.1.dev260+g32de3d6d4' / PypeIt version
DATE = '2022-11-10' / UTC date created
DMODCLS = 'ImageBitMaskArray' / Datamodel class
DMODVER = '1.0 ' / Datamodel version
BIT0 = 'BPM '
BIT1 = 'COSMIC '
BIT2 = 'SATURATED'
EXTNAME = 'MASK ' / extension name
CHECKSUM= 'APGODMFOAMFOAMFO' / HDU checksum updated 2022-11-10T13:10:27
DATASUM = '1245200 ' / data unit checksum updated 2022-11-10T13:10:27
Note
Currently, when loading a mask, the bit names in the header of the output
file are not checked against the bitmask definition in the code itself.
This kind of version control should be handled using the version
attribute of the class. I.e., anytime the flags in the bitmask are changed,
the developers should bump the class version.
- class pypeit.images.bitmaskarray.BitMaskArray(shape, asuint=False)[source]
Bases:
DataContainer
Object that holds both the mask data and the mask interpretation object.
This is an abstract class that should not be directly instantiated.
- Parameters:
shape (
tuple
) – Shape of the mask to create.asuint (
bool
, optional) – When setting the data-type for the mask array (seeminimum_dtype()
), use an unsigned integer instead of a signed integer (e.g.,uint16
instead ofint16
).
- __getattr__(item)[source]
Override the attribute access to allow for immediate construction and access to boolean arrays that select array elements with the provided flag.
For example, if
'BPM'
is a flag in thebitmask
, andmask
is an instance of the the relevant subclass,mask.bpm
is identical to callingmask.flagged(flag='BPM')
.Warning
Using this functionality creates a new numpy array every time it is called. This can be slow for large arrays. This means that, if you need to access the boolean array for a given flag multiple times, you should set it to a new object (e.g.,
maskbpm = mask.bpm
)!If the attribute (
item
) requested is not one of the bitmask flags, theDataContainer
base class function is called.- Parameters:
item (object) – The attribute being accessed.
- _bundle()[source]
Override the base-class bundle method so that the bitmask keys can be added to the header.
- Returns:
List of dictionaries indicating what should be written to a file.
- Return type:
- bitmask = None
BitMask
object used to interpret the bit array. Must be defined by the subclass. When defining subclasses, note that the bitmask flags must be case-insensitive strings.
- property bits
Return the bit dictionary.
- consolidate(flag_set, consolidated_flag)[source]
Consolidate a set of flags into a single flag.
That is, any bit flagged with any of the flags provided by
flag_set
will also be flagged byconsolidate_flag
after executing this function.
- datamodel = {'mask': {'atype': <class 'numpy.integer'>, 'descr': 'Bitmask values', 'otype': <class 'numpy.ndarray'>}}
Datamodel is simple, containing only the mask array.
- flagged(flag=None, invert=False)[source]
Determine if a bit is on in the provided bitmask value. The function can be used to determine if any individual bit is on or any one of many bits is on.
- Parameters:
- Returns:
Boolean array indicating where the internal bitmask is flagged by the selected bits. Flagged values are True, unflagged values are False. If
invert
is True, this is reversed.- Return type:
- classmethod from_hdu(hdu, chk_version=True, **kwargs)[source]
Instantiate the object from an HDU extension.
This overrides the base-class method, only to add checks (or not) for the bitmask.
- Parameters:
hdu (astropy.io.fits.HDUList, astropy.io.fits.ImageHDU, astropy.io.fits.BinTableHDU) – The HDU(s) with the data to use for instantiation.
chk_version (
bool
, optional) – If True, raise an error if the datamodel version or type check failed. If False, throw a warning only.**kwargs – Passed directly to
_parse()
.
- internals = ['lower_keys']
A list of strings identifying a set of internal attributes that are not part of the datamodel.
- property shape
Return the shape of the internal array.
- toggle(flag, select=None)[source]
Toggle bits for selected array elements.
- Parameters:
flag (
str
, array-like) – Bit name(s) to toggle.select (
tuple
,slice
, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to toggle the provided bit flags. I.e., for the internalmask
,mask[select]
must be a valid (fancy indexing) operation. If None, the bit is toggled for the full mask!
- turn_off(flag, select=None)[source]
Ensure that a bit is turned off in the provided bitmask value.
- Parameters:
flag (
str
, array-like) – Bit name(s) to turn off.select (
tuple
,slice
, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to turn off the provided bit flags. I.e., for the internalmask
,mask[select]
must be a valid (fancy indexing) operation. If None, the bit is turned off for the full mask!
- turn_on(flag, select=None)[source]
Ensure that a bit is turned on for the selected elements.
- Parameters:
flag (
str
, array-like) – Bit name(s) to turn on.select (
tuple
,slice
, numpy.ndarray, optional) – Object used to select elements of the mask array to at which to turn on the provided bit flags. I.e., for the internalmask
,mask[select]
must be a valid (fancy indexing) operation. If None, the bit is turned on for the full mask!
- version = None
DataContainer version. Must be defined by the subclass.