pypeit.bitmask module
Base class for handling bit masks.
Class usage examples
Bitmasks allow you to define a set of bit values signified by strings, and then toggle and interpret bits held by a numpy.ndarray. For example, say you’re processing an image and you want to setup a set of bits that indicate that the pixel is part of a bad-pixel mask, has a cosmic ray, or is saturated. You can define the following:
from pypeit.bitmask import BitMask
bits = {'BPM':'Pixel is part of a bad-pixel mask',
'COSMIC':'Pixel is contaminated by a cosmic ray',
'SATURATED':'Pixel is saturated.'}
image_bm = BitMask(list(bits.keys()), descr=list(bits.values()))
Note
Consistency in the order of the dictionary keywords is critical
to the repeatability of the BitMask
instances. The above is possible because dict
objects
automatically maintain the order of the provided keywords since
Python 3.7.
Or, better yet, define a derived class:
from pypeit.bitmask import BitMask
class ImageBitMask(BitMask):
def __init__(self):
bits = {'BPM':'Pixel is part of a bad-pixel mask',
'COSMIC':'Pixel is contaminated by a cosmic ray',
'SATURATED':'Pixel is saturated.'}
super(ImageBitMask, self).__init__(list(bits.keys()), descr=list(bits.values()))
image_bm = ImageBitMask()
In either case, you can see the list of bits and their bit numbers by running:
>>> image_bm.info()
Bit: BPM = 0
Description: Pixel is part of a bad-pixel mask
Bit: COSMIC = 1
Description: Pixel is contaminated by a cosmic ray
Bit: SATURATED = 2
Description: Pixel is saturated.
>>> image_bm.bits
{'BPM': 0, 'COSMIC': 1, 'SATURATED': 2}
>>> image_bm.keys()
['BPM', 'COSMIC', 'SATURATED']
Now you can define a numpy.ndarray to hold the mask value for each
image pixel; the minimum_dtype()
returns the smallest data type required to represent the list of
defined bits. The maximum number of bits that can be defined is 64.
Assuming you have an image img
:
import numpy
mask = numpy.zeros(img.shape, dtype=image_bm.minimum_dtype())
Assuming you have boolean or integer arrays that identify pixels to mask, you can turn on the mask bits as follows:
mask[cosmics_indx] = image_bm.turn_on(mask[cosmics_indx], 'COSMIC')
mask[saturated_indx] = image_bm.turn_on(mask[saturated_indx], 'SATURATED')
or make sure certain bits are off:
mask[not_a_cosmic] = image_bm.turn_off(mask[not_a_cosmic], 'COSMIC')
The form of these methods is such that the array passed to the method
are not altered. Instead the altered bits are returned, which is why
the lines above have the form m = bm.turn_on(m, flag)
.
Some other short usage examples:
To find which flags are set for a single value:
image_bm.flagged_bits(mask[0,10])To find the list of unique flags set for any pixel:
unique_flags = numpy.sort(numpy.unique(numpy.concatenate( [image_bm.flagged_bits(b) for b in numpy.unique(mask)]))).tolist()To get a boolean array that selects pixels with one or more mask bits:
cosmics_indx = image_bm.flagged(mask, flag='COSMIC') all_but_bpm_indx = image_bm.flagged(mask, flag=['COSMIC', 'SATURATED']) any_flagged = image_bm.flagged(mask)To construct masked arrays, following from the examples above:
masked_img = numpy.ma.MaskedArray(img, mask=image_bm.flagged(mask))
BitMask
objects can be defined
programmatically, as shown above for the ImageBitMask
derived class,
but they can also be defined by reading formatted files. The current
options are:
Fits headers: There are both reading and writing methods for bitmask I/O using astropy.io.fits.Header objects. Using the
ImageBitMask
class as an example:>>> from astropy.io import fits >>> hdr = fits.Header() >>> image_bm = ImageBitMask() >>> image_bm.to_header(hdr) >>> hdr BIT0 = 'BPM ' / Pixel is part of a bad-pixel mask BIT1 = 'COSMIC ' / Pixel is contaminated by a cosmic ray BIT2 = 'SATURATED' / Pixel is saturated. >>> copy_bm = BitMask.from_header(hdr)
- class pypeit.bitmask.BitMask(keys, descr=None)[source]
Bases:
object
Generic class to handle and manipulate bitmasks. The input list of bit names (keys) must be unique, except that values of ‘NULL’ are ignored. The index in the input keys determines the bit value; ‘NULL’ keys are included in the count. For example:
>>> from pypeit.bitmask import BitMask >>> keys = [ 'key1', 'key2', 'NULL', 'NULL', 'key3' ] >>> bm = BitMask(keys) >>> bm.info() Bit: key1 = 0 Bit: key2 = 1 Bit: key3 = 4
Todo
Have the class keep the mask values internally instead of having it only operate on the mask array…
- Parameters
- Raises
ValueError – Raised if more than 64 bits are provided.
TypeError – Raised if the provided keys do not have the correct type.
- descr
List of bit descriptions
- Type
- static _fill_sequence(keys, vals, descr=None)[source]
Fill bit sequence with NULL keys if bit values are not sequential.
The instantiation of
BitMask
does not include the value of the bit, it just assumes that the bits are in sequence such that the first key has a value of 0, and the last key has a value of N-1. This is a convenience function that fills the list of keys with ‘NULL’ for bit values that are non-sequential. This is used primarily for instantiation the BitMask from bits written to a file where the NULL bits have been skipped.- Parameters
- Returns
Three 1D arrays with the filled keys, values, and descriptions.
- Return type
- Raises
ValueError – Raised if a bit value is less than 0.
- consolidate(value, flag_set, consolidated_flag)[source]
Consolidate a set of flags into a single flag.
- flagged(value, flag=None)[source]
Determine if a bit is on in the provided bitmask value. The function can be used to determine if any individual bit is on or any one of many bits is on.
- Parameters
- Returns
Boolean flags that the provided flags (or any flag) is on for the provided bitmask value. Shape is the same as value.
- Return type
- Raises
- flagged_bits(value)[source]
Return the list of flagged bit names for a single bit value.
- classmethod from_header(hdr, prefix=None)[source]
Instantiate the BitMask using data parsed from a fits header.
Todo
This is very similar to the function in ParSet. Abstract to a general routine?
If comments are truncated by the comment line length, they’ll be different than a direct instantiation.
- Parameters
hdr (astropy.io.fits.Header) – Header object with the bits.
prefix (
str
, optional) – Prefix of the relevant header keywords, which overwrites the string defined for the class. If None, uses the default for the class.
- keys()[source]
Return a list of the bit keywords.
Keywords are sorted by their bit value and ‘NULL’ keywords are ignored.
- Returns
List of bit keywords.
- Return type
- minimum_dtype(asuint=False)[source]
Return the smallest int datatype that is needed to contain all the bits in the mask. Output as an unsigned int if requested.
- Parameters
asuint (
bool
, optional) – Return an unsigned integer type. Signed types are returned by default.
Warning
uses int16 if the number of bits is less than 8 and asuint=False because of issue astropy.io.fits has writing int8 values.
- static parse_bits_from_hdr(hdr, prefix)[source]
Parse bit names, values, and descriptions from a fits header.
Todo
This is very similar to the function in ParSet. Abstract to a general routine?
- Parameters
hdr (astropy.io.fits.Header) – Header object with the bits.
prefix (
str
) – The prefix used for the header keywords.
- Returns
Three lists are returned providing the bit names, values, and descriptions.
- prefix = 'BIT'
- to_dict(prefix=None)[source]
Write the bits to a dictionary.
The keys of the dictionary are identical to those use to write the bitmask to a FITS header.
- to_header(hdr, prefix=None)[source]
Write the bits to a fits header.
The header is edited in place!
Todo
This is very similar to the function in ParSet. Abstract to a general routine?
The comment might have a limited length and be truncated.
- Parameters
hdr (astropy.io.fits.Header) – Header object for the parameters. Modified in-place.
prefix (
str
, optional) – Prefix to use for the header keywords, which overwrites the string defined for the class. If None, uses the default for the class.
- toggle(value, flag)[source]
Toggle a bit in the provided bitmask value.
- Parameters
- Returns
New bitmask value after toggling the selected bit.
- Return type
array-like
- Raises
ValueError – Raised if the provided flag is None.
- turn_off(value, flag)[source]
Ensure that a bit is turned off in the provided bitmask value.
- Parameters
- Returns
New bitmask value after turning off the selected bit.
- Return type
- Raises
ValueError – Raised by the dict data type if the input
flag
is not one of the valid bitmask names or if it is None.
- turn_on(value, flag)[source]
Ensure that a bit is turned on in the provided bitmask value.
- Parameters
value (
int
, numpy.ndarray) – Bitmask value. It should be less than or equal tomax_value
; however, that is not checked.
- Returns
New bitmask value after turning on the selected bit.
- Return type
- Raises
ValueError – Raised by the dict data type if the input
flag
is not one of the valid bitmask names or if it is None.
- unpack(value, flag=None)[source]
Construct boolean arrays with the selected bits flagged.
- Parameters
value (numpy.ndarray) – The bitmask values to unpack.
flag (
str
,list
, optional) – The specific bits to unpack. If None, all values are unpacked.
- Returns
A tuple of boolean numpy.ndarrays flagged according to each bit.
- Return type
- version = None