Automated sorting of HIRES frames by instrument configuration

Version History

Version

Author

Date

PypeIt

1.0

Debora Pelliccia

10 Aug 2024

1.16.1.dev

Basics

To prepare for the data reduction, PypeIt, first, automatically associates fits files to specific Frame Types (see Automated typing of HIRES frames) and, then, collects groups of frames in unique instrument configurations (see below). This is performed by the pypeit_setup script, which sorts the frames and writes a PypeIt Reduction File for each unique configuration. See Setup.

HIRES configuration identification

The HIRES instrument configurations are determined by the function pypeit.metadata.PypeItMetaData.unique_configurations(), which finds unique combinations of the following keywords:

fitstbl key

Header Key

dispname

XDISPERS

decker

DECKNAME

binning

BINNING

filter1

FIL1NAME

echangle

ECHANGL

xdangle

XDANGL

The unique configurations are determined by collating the relevant metadata from the headers of all frames found by a run of pypeit_setup, except those that are designated as bias and slitless_pixflat frames. Bias and slitless_pixflat frames can have header data (e.g., filter1) that do not match the instrument configuration that an observer intended for their use. Therefore, PypeIt uses the dispname and binning keys to match the bias and slitless_pixflat frames to the configurations with frames taken with the same cross-disperser and same binning. Note that when using the echangle and xdangle keys to identify configurations, PypeIt uses a relative tolerance of 1e-3 and absolute tolerance of 1e-2 for echangle, and a relative tolerance of 1e-2 for xdangle, to account for small differences in the values of these angles.

After that, pypeit.metadata.PypeItMetaData.set_configurations() associates each frame to the relevant unique configuration (“setup”), by assigning a setup identifier (e.g., A,B,C,D…) to every frames for which the values of the above keywords match the values of the specific unique configuration.

HIRES calibration groups

PypeIt uses the concept of a “calibration group” to define a complete set of calibration frames (e.g., arcs, flats) and the science frames to which these calibration frames should be applied.

By default, pypeit_setup uses the setup identifier to assign frames to a single calibration group. Frames that are in the same calibration group will have the same PypeIt keyword calib. No automated procedure exists to do anything except this. However, the user can edit the PypeIt Reduction File to, within a given configuration, assign specific calibration frames to specific science frames using the data in the calib column of the Data Block.

Testing

To test that PypeIt can successfully identify multiple configurations among a set of files, we have added the test_setup_keck_hires_multiconfig() test to ${PYPEIT_DEV}/unit_tests/test_setups.py.

Here is an example of how to run the test:

cd ${PYPEIT_DEV}/unit_tests
pytest test_setup.py::test_setup_keck_hires_multiconfig -W ignore

The tests require that you have downloaded the PypeIt PypeIt Development Suite and defined the PYPEIT_DEV environmental variable that points to the relevant directory.

The algorithm for this test is as follows:

  1. Collect the names of all files in selected HIRES directories.

  2. Use PypeItSetup to automatically identify the configurations for these files.

  3. Check that the code found two configurations and wrote the pypeit files for each.

  4. For each configuration:

    1. Read the pypeit file

    2. Check that the name for the setup is correct (‘A’ or ‘B’)

    3. Check that the calibration group is the same for all frames (‘0’ or ‘1’)

Because these tests are now included in the PypeIt Unit Tests (GitHub CI), these configuration checks are performed by the developers for every new version of the code.