Output Bitmasks

PypeIt uses bitmasks to flag data for various reasons. For a primer on the general use of bitmasks, see the SDSS bitmasks primer.

For the underlying bitmask utility class used by PypeIt, see Bitmasks.

---

Spec2D Bits

In terms of the output data, the main bitmask you will encounter is in the Spec2D Output. The BPMMASK extension in these files contain the bad-pixel bitmask values accrued during the processing and reduction of the data. The main object that manages the bitmasks is ImageBitMask, which defines the following bits:

Bit Name	Bit Number	Decimal Value	Description
BPM	0	1	Component of the instrument-specific bad pixel mask
CR	1	2	Cosmic ray detected
SATURATION	2	4	Saturated pixel
MINCOUNTS	3	8	Pixel below the instrument-specific minimum counts
OFFSLITS	4	16	Pixel does not belong to any slit
IS_NAN	5	32	Pixel value is undefined
IVAR0	6	64	Inverse variance is undefined
IVAR_NAN	7	128	Inverse variance is NaN
EXTRACT	8	256	Pixel masked during local skysub and extraction
BADSCALE	9	512	Bad image rescaling operation (e.g., flat-field value <= 0)
STCKMASK	10	1024	All pixels masked in image stack
USER	11	2048	Pixel masked by user

Viewing and interpreting the bit mask image (python)

To access the bitmask array, we recommend using Spec2DObj directly:

from pathlib import Path
from pypeit.spec2dobj import Spec2DObj

f = Path('spec2d_b27-J1217p3905_KASTb_20150520T045733.560.fits').resolve()
detname = 'DET01'
spec2d = Spec2DObj.from_file(f, detname)
mask = spec2d.bpmmask

The bitmask array is held by the spec2d.bpmmask attribute.

But you can read it directly from the spec2d file:

from astropy.io import fits
from pypeit.images.imagebitmask import ImageBitMaskArray

f = Path('spec2d_b27-J1217p3905_KASTb_20150520T045733.560.fits').resolve()
detname = 'DET01'
hdu = fits.open(f)
mask = ImageBitMaskArray.from_array(hdu[f'{detname}-BPMMASK'].data)

To show all the bit values directly:

from matplotlib import pyplot as plt

plt.imshow(mask, origin='lower', interpolation='nearest')
plt.show()

However, this isn’t necessarily as useful as creating boolean arrays that identify which pixels are flagged due to one or more reasons.

E.g., to show all pixels flagged for having cosmic rays:

plt.imshow(mask.flagged(flag='CR').astype(int), origin='lower', interpolation='nearest')
plt.show()

or as being part of the instrument-specific bad-pixel mask or not part of any slit:

plt.imshow(mask.flagged(flag=['BPM', 'OFFSLITS']).astype(int),
           origin='lower', interpolation='nearest')
plt.show()

You can also use the pypeit_show_2dspec script to include an image that shows the full mask or an image that selects specific flags.

To print the human-readable reason(s) any given value is flagged:

coo = (0,0)   # Tuple with the 2D coordinate of the pixel
print(mask.flagged_bits(coo))

pypeit_print_bpm

This simple executable allows you to effectively do the above via a command-line script. The script usage can be displayed by calling the script with the -h option:

$ pypeit_print_bpm -h
usage: pypeit_print_bpm [-h] [--file FILE] [--det DET] bit

Print out an informative description of a bad pixel masked value. Usually, you
should run pypeit_show_2dspec --showmask first to see the bad pixel mask values.
Then, call this script with the BPM value that you want to findmore information
about.

positional arguments:
  bit          Bad pixel mask value to describe in plain text

options:
  -h, --help   show this help message and exit
  --file FILE  PypeIt spec2d file to use for the description(optional). If
               provided, the bitmask contained in the spec2d file will be used
               to describe the bad pixel mask value. If not provided, the
               default pypeit bad pixel mask will be used. (default: None)
  --det DET    Detector name or number. If a number, the name is constructed
               assuming the reduction is for a single detector. If a string, it
               must match the name of the detector object (e.g., DET01 for a
               detector, MSC01 for a mosaic). This is not required, and the
               value is acceptable. Default is 1. (default: 1)

A typical call and its output looks like this:

% pypeit_print_bpm 23
[INFO]    :: Using the default PypeIt bad pixel mask.
[INFO]    :: The bad pixel mask value (23) corresponds to the following:

             * BPM        : Component of the instrument-specific bad pixel mask
             * CR         : Cosmic ray detected
             * SATURATION : Saturated pixel
             * OFFSLITS   : Pixel does not belong to any slit

[INFO]    :: Please see the following website for more information:
             https://pypeit.readthedocs.io/en/release/out_masks.html