Output Bitmasks

PypeIt uses bitmasks to flag data for various reasons. For a primer on the general use of bitmasks, see the SDSS bitmasks primer.

For the underlying bitmask utility class used by PypeIt, see Bitmasks.

---

Spec2D Bits

In terms of the output data, the main bitmask you will encounter is in the Spec2D Output. The BPMMASK extension in these files contain the bad-pixel bitmask values accrued during the processing and reduction of the data. The main object that manages the bitmasks is ImageBitMask, which defines the following bits:

Bit Name	Bit Number	Decimal Value	Description
BPM	0	1	Component of the instrument-specific bad pixel mask
CR	1	2	Cosmic ray detected
SATURATION	2	4	Saturated pixel
MINCOUNTS	3	8	Pixel below the instrument-specific minimum counts
OFFSLITS	4	16	Pixel does not belong to any slit
IS_NAN	5	32	Pixel value is undefined
IVAR0	6	64	Inverse variance is undefined
IVAR_NAN	7	128	Inverse variance is NaN
EXTRACT	8	256	Pixel masked during local skysub and extraction
BADSCALE	9	512	Bad image rescaling operation (e.g., flat-field value <= 0)
STCKMASK	10	1024	All pixels masked in image stack
USER	11	2048	Pixel masked by user

Viewing and interpreting the bit mask image (python)

To access the bitmask array, we recommend using Spec2DObj directly:

from pathlib import Path
from pypeit.spec2dobj import Spec2DObj

f = Path('spec2d_b27-J1217p3905_KASTb_20150520T045733.560.fits').resolve()
detname = 'DET01'
spec2d = Spec2DObj.from_file(f, detname)
mask = spec2d.bpmmask

The bitmask array is held by the spec2d.bpmmask attribute.

But you can read it directly from the spec2d file:

from astropy.io import fits
from pypeit.images.imagebitmask import ImageBitMaskArray

f = Path('spec2d_b27-J1217p3905_KASTb_20150520T045733.560.fits').resolve()
detname = 'DET01'
hdu = fits.open(f)
mask = ImageBitMaskArray.from_array(hdu[f'{detname}-BPMMASK'].data)

To show all the bit values directly:

from matplotlib import pyplot as plt

plt.imshow(mask, origin='lower', interpolation='nearest')
plt.show()

However, this isn’t necessarily as useful as creating boolean arrays that identify which pixels are flagged due to one or more reasons.

E.g., to show all pixels flagged for having cosmic rays:

plt.imshow(mask.flagged(flag='CR').astype(int), origin='lower', interpolation='nearest')
plt.show()

or as being part of the instrument-specific bad-pixel mask or not part of any slit:

plt.imshow(mask.flagged(flag=['BPM', 'OFFSLITS']).astype(int),
           origin='lower', interpolation='nearest')
plt.show()

You can also use the pypeit_show_2dspec script to include an image that shows the full mask or an image that selects specific flags.

To print the human-readable reason(s) any given value is flagged:

coo = (0,0)   # Tuple with the 2D coordinate of the pixel
print(mask.flagged_bits(coo))

pypeit_print_bpm

This simple executable allows you to effectively do the above via a command-line script. The script usage can be displayed by calling the script with the -h option:

$ pypeit_print_bpm -h
usage: pypeit_print_bpm [-h] [-v VERBOSITY] [--log_file LOG_FILE]
                        [--log_level LOG_LEVEL] [--file FILE] [--det DET]
                        bit

Print out an informative description of a bad pixel masked value. Usually, you
should run pypeit_show_2dspec --showmask first to see the bad pixel mask values.
Then, call this script with the BPM value that you want to findmore information
about.

positional arguments:
  bit                   Bad pixel mask value to describe in plain text

options:
  -h, --help            show this help message and exit
  -v, --verbosity VERBOSITY
                        Verbosity level, which must be 0, 1, or 2. Level 0
                        includes warning and error messages, level 1 adds
                        informational messages, and level 2 adds debugging
                        messages and the calling sequence. (default: 2)
  --log_file LOG_FILE   Name for the log file. If set to "default", a default
                        name is used. If None, a log file is not produced.
                        (default: None)
  --log_level LOG_LEVEL
                        Verbosity level for the log file. If a log file is
                        produce and this is None, the file log will match the
                        console stream log. (default: None)
  --file FILE           PypeIt spec2d file to use for the description(optional).
                        If provided, the bitmask contained in the spec2d file
                        will be used to describe the bad pixel mask value. If
                        not provided, the default pypeit bad pixel mask will be
                        used. (default: None)
  --det DET             Detector name or number. If a number, the name is
                        constructed assuming the reduction is for a single
                        detector. If a string, it must match the name of the
                        detector object (e.g., DET01 for a detector, MSC01 for a
                        mosaic). This is not required, and the value is
                        acceptable. Default is 1. (default: 1)

A typical call and its output looks like this:

% pypeit_print_bpm 23
[INFO]    :: Using the default PypeIt bad pixel mask.
[INFO]    :: The bad pixel mask value (23) corresponds to the following:

             * BPM        : Component of the instrument-specific bad pixel mask
             * CR         : Cosmic ray detected
             * SATURATION : Saturated pixel
             * OFFSLITS   : Pixel does not belong to any slit

[INFO]    :: Please see the following website for more information:
             https://pypeit.readthedocs.io/en/release/out_masks.html