Setup

Overview

The PypeIt Reduction File is the critical component to any successful run of PypeIt, and pypeit_setup provides an automated means of generating this file based on a set of fits files.

Below, we describe how PypeIt defines unique instrument configurations (“setups”) and sorts data in preparation for the data reduction. Then we describe our recommended method for generating the PypeIt Reduction File, which includes multiple executions of pypeit_setup. We also describe another preparatory script, pypeit_obslog, which provides a simple listing of the available data files; however, use of this script is optional in terms of setup for reducing your data.

Use of Metadata to Identify Instrument Configurations

PypeIt distinguishes between instrument configurations using metadata pulled from the fits file headers, as defined specifically for each spectrograph. To list all of the metadata pulled from each data file and how it maps to the keywords in the data file headers, run, e.g.:

pypeit_obslog keck_deimos -k

(see pypeit_obslog and meta_key_map()). For Keck/DEIMOS, this prints:

Metadata Key   Header Card
------------   -----------
          ra            RA
         dec           DEC
      target      TARGNAME
      decker      SLMSKNAM
     binning          None
         mjd          None
     exptime      ELAPTIME
     airmass       AIRMASS
    dispname      GRATENAM
       hatch      HATCHPOS
   dispangle          None
      idname       OBSTYPE
  lampstat01         LAMPS
     dateobs      DATE-OBS
         utc           UTC
        mode       MOSMODE
         amp       AMPMODE
      object        OBJECT
     filter1      DWFILNAM
     frameno       FRAMENO
  instrument      INSTRUME

where the left column provides the PypeIt-specific metadata keyword and the right column is the associated instrument-specific header card. Any metadata element with a header card set to None means that the metadata keyword or value is conditioned on multiple header values. In this example, the header card that provides the central wavelength given the grating angle (dispangle) for DEIMOS is dependent on the grating used (dispname); see compound_meta().

A subset of this metadata is used to determine the unique instrument configurations used and associate all of the data files to the relevant configuration. Generally speaking, most instrument configurations are set by the name (dispname) and central wavelength (dispangle) of the dispersing element, the name of the decker or slit mask (decker), the name of the beam-splitting dichroic (dichroic), and/or the on-chip binning (binning). The specific set of metadata keys used for each instrument is set by the configuration_keys method; for DEIMOS, see configuration_keys().

The following table provides the list of metadata used to define the instrument configuration for any supported spectrograph (as of PypeIt version 1.10.1dev):

Metadata Key

Type

Example

Description

amp

str

SINGLE:B

Name of the amplifier used to read the detector

arm

str

VIS

Name of the spectrograph arm used to collect the data

binning

str

1,1

On-chip binning

cenwave

str

7500.0

Central wavelength of the disperser

datasec

str

[1:256,1:512]

The science region of the detector

decker

str

long_1.0

Name of the decker or slit mask

decker_secondary

str

long_1.0

Partial Slitmask/decker name; see Automated sorting of Keck/MOSFIRE frames by instrument configuration

detector

str

CHIP1

Name of the detector

dichroic

str

560

Name of the dichroic

dispangle

float

7500.0

Central wavelength for the dispersing element at the observed angle

dispname

str

830G

Name of the dispersing element

filter1

str

J

Name of the order-sorting filter

slitlength

str

Slit length; see Automated sorting of Keck/MOSFIRE frames by instrument configuration

slitwid

str

Slit width; see Automated sorting of Keck/MOSFIRE frames by instrument configuration

Every unique combination of the relevant metadata found in any of the fits files to be reduced represents a unique instrument configuration for PypeIt to consider; these configurations can be determined automatically by pypeit_setup and will be identified by a capital letter, e.g., A. When executing run_pypeit, however, the instrument configuration is set by the Setup Block in the PypeIt Reduction File and not redetermined by the fits files. The latter allows the user flexibility to override PypeIt’s automated configuration settings. Currently, each PypeIt Reduction File should only provide data from one instrument configuration.

If you tend to observe with one instrument configuration and with a simple set of calibrations, then the setup to run PypeIt should be straightforward. However, if you use multiple configurations (e.g. gratings, grating tilts, slitmasks), then you must pay more careful attention to the setups.

Below, we describe how PypeIt automatically determines instrument configurations for a set of files and constructs auto-generated pypeit files.


pypeit_obslog

The pypeit_obslog script allows you to see a simple listing of the data files in a given directory (or directories) and the metadata that PypeIt will pull from their headers. As always, the script usage can be displayed by calling the script with the -h option:

$ pypeit_obslog -h
usage: pypeit_obslog [-h] [-r ROOT] [-k] [-c COLUMNS] [-b] [-t BAD_TYPES] [-g]
                     [-i] [-s SORT] [-e EXTENSION] [-d OUTPUT_PATH] [-o]
                     [-f FILE] [-G]
                     spec

Construct an observing log for a set of files from the provided spectrograph
using PypeItMetaData.

positional arguments:
  spec                  A valid spectrograph identifier: bok_bc,
                        gemini_flamingos1, gemini_flamingos2,
                        gemini_gmos_north_e2v, gemini_gmos_north_ham,
                        gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
                        gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
                        gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
                        keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
                        keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
                        keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
                        keck_nires, keck_nirspec_low, lbt_luci1, lbt_luci2,
                        lbt_mods1b, lbt_mods1r, lbt_mods2b, lbt_mods2r,
                        ldt_deveny, magellan_fire, magellan_fire_long,
                        magellan_mage, mdm_modspec, mdm_osmos_mdm4k,
                        mdm_osmos_r4k, mmt_binospec, mmt_bluechannel, mmt_mmirs,
                        not_alfosc, not_alfosc_vert, ntt_efosc2, p200_dbsp_blue,
                        p200_dbsp_red, p200_tspec, shane_kast_blue,
                        shane_kast_red, shane_kast_red_ret, soar_goodman_blue,
                        soar_goodman_red, tng_dolores, vlt_fors2, vlt_sinfoni,
                        vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
                        wht_isis_blue, wht_isis_red

options:
  -h, --help            show this help message and exit
  -r ROOT, --root ROOT  Root to search for data files. You can provide the top-
                        level directory (e.g., /data/Kast) or the search string
                        up through the wildcard (.e.g, /data/Kast/b). Use the
                        --extension option to set the types of files to search
                        for. (default: current working directory)
  -k, --keys            Do not produce the log; simply list the pypeit-specific
                        metadata keys available for this spectrograph and their
                        associated header cards. Metadata keys with header cards
                        that are None have no simple mapping between keyword and
                        header card. (default: False)
  -c COLUMNS, --columns COLUMNS
                        A comma-separated list of columns to include in the
                        output table. Each column must be a valid pypeit
                        metadata keyword specific to this spectrograph (run
                        pypeit_obslog with the -k argument to see the valid
                        list). Additional valid keywords are directory,
                        filename, frametype, framebit, setup, calib, and
                        calibbit. If 'all', all columns collected for the pypeit
                        metadata table are included. If 'pypeit', the columns
                        are the same as those included in the pypeit file.
                        (default: pypeit)
  -b, --bad_frames      Clean the output of bad frames that cannot be reduced by
                        pypeit. (default: False)
  -t BAD_TYPES, --bad_types BAD_TYPES
                        Dictates how frames that could not be given a valid type
                        should be treated. Options are: "keep" to include them
                        in the output, "rm" to remove them from the output,
                        "only" to only include the frames with unknown types in
                        the output (i.e, the frames with determined types are
                        excluded). (default: keep)
  -g, --groupings       Use this option to only determine the frame type. By
                        default, the script groups frames into expected
                        configuration and calibration groups, and it adds the
                        default combination groups. (default: True)
  -i, --interact        Once the metadata table is created, start an embedded
                        IPython session that you can use to interact with the
                        table (an Astropy.Table called fitstbl) directly.
                        (default: False)
  -s SORT, --sort SORT  Metadata keyword (pypeit-specific) to use to sort the
                        output table. (default: mjd)
  -e EXTENSION, --extension EXTENSION
                        File extension; compression indicators (e.g. .gz) not
                        required. (default: .fits)
  -d OUTPUT_PATH, --output_path OUTPUT_PATH
                        Path to top-level output directory. (default: current
                        working directory)
  -o, --overwrite       Overwrite any existing files/directories (default:
                        False)
  -f FILE, --file FILE  Name for the ascii output file. Any leading directory
                        path is stripped; use -d to set the output directory. If
                        None, the table is just printed to stdout. If set to
                        'default', the file is set to [spectrograph].obslog.
                        Note the file will *not* be written if you also include
                        the -i option to embed and interact with the table (you
                        can write the table using the astropy.table.Table.write
                        method in the embedded IPython session). The table is
                        always written in ascii format using
                        format=ascii.fixed_with for the call to
                        Astropy.table.Table.write . (default: None)
  -G, --gui             View the obs log in a GUI (default: False)

For example, if you’ve been observing with Keck DEIMOS, you can go into the directory with the raw data and execute:

pypeit_obslog keck_deimos

Or you can point to the directory that you want to list:

pypeit_obslog keck_deimos -r /path/to/raw/data

The listing is printed to stdout, but you can also specify a file for the output. You can also select the metadata by which to sort the output, change the columns that are printed, etc.

By default, the columns provided in the output table are identical to the columns in the Data Block of a PypeIt Reduction File; to print columns with all the metadata collected, add -c all to the command line.


pypeit_setup

pypeit_setup is the script one executes to prepare for the data reduction by automatically associating fits files to specific Frame Types and collecting groups of frames taken using unique instrument configurations. The script usage can be displayed by calling the script with the -h option:

$ pypeit_setup -h
usage: pypeit_setup [-h] [-s SPECTROGRAPH] [-r ROOT [ROOT ...]] [-e EXTENSION]
                    [-d OUTPUT_PATH] [-o] [-c CFG_SPLIT] [-b] [-m]
                    [-v VERBOSITY] [-k] [-G]

Parse data files to construct a pypeit file in preparation for reduction using
'run_pypeit'

options:
  -h, --help            show this help message and exit
  -s SPECTROGRAPH, --spectrograph SPECTROGRAPH
                        A valid spectrograph identifier: bok_bc,
                        gemini_flamingos1, gemini_flamingos2,
                        gemini_gmos_north_e2v, gemini_gmos_north_ham,
                        gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
                        gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
                        gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
                        keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
                        keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
                        keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
                        keck_nires, keck_nirspec_low, lbt_luci1, lbt_luci2,
                        lbt_mods1b, lbt_mods1r, lbt_mods2b, lbt_mods2r,
                        ldt_deveny, magellan_fire, magellan_fire_long,
                        magellan_mage, mdm_modspec, mdm_osmos_mdm4k,
                        mdm_osmos_r4k, mmt_binospec, mmt_bluechannel, mmt_mmirs,
                        not_alfosc, not_alfosc_vert, ntt_efosc2, p200_dbsp_blue,
                        p200_dbsp_red, p200_tspec, shane_kast_blue,
                        shane_kast_red, shane_kast_red_ret, soar_goodman_blue,
                        soar_goodman_red, tng_dolores, vlt_fors2, vlt_sinfoni,
                        vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
                        wht_isis_blue, wht_isis_red (default: None)
  -r ROOT [ROOT ...], --root ROOT [ROOT ...]
                        Root to search for data files. You can provide the top-
                        level directory (e.g., /data/Kast) or the search string
                        up through the wildcard (.e.g, /data/Kast/b). Use the
                        --extension option to set the types of files to search
                        for. (default: current working directory)
  -e EXTENSION, --extension EXTENSION
                        File extension; compression indicators (e.g. .gz) not
                        required. (default: .fits)
  -d OUTPUT_PATH, --output_path OUTPUT_PATH
                        Path to top-level output directory. (default: current
                        working directory)
  -o, --overwrite       Overwrite any existing files/directories (default:
                        False)
  -c CFG_SPLIT, --cfg_split CFG_SPLIT
                        Generate the PypeIt files and folders by input
                        configuration. To write all unique configurations
                        identifed, use 'all', otherwise provide the list of
                        configuration letters; e.g., 'A,B' or 'B,D,E' or 'E'.
                        (default: None)
  -b, --background      Include the background-pair columns for the user to edit
                        (default: False)
  -m, --manual_extraction
                        Include the manual extraction column for the user to
                        edit (default: False)
  -v VERBOSITY, --verbosity VERBOSITY
                        Verbosity level between 0 [none] and 2 [all]. Default:
                        1. Level 2 writes a log with filename setup_YYYYMMDD-
                        HHMM.log (default: 1)
  -k, --keep_bad_frames
                        Keep all frames, even if they are identified as having
                        bad/unrecognized configurations that cannot be reduced
                        by pypeit. (This is the opposite of the --bad_frames
                        option in pypeit_obslog; i.e., you have to tell
                        pypeit_setup to keep these frames, whereas you have to
                        tell pypeit_obslog to remove them. (default: False)
  -G, --gui             Run setup in a GUI (default: False)

The following is a step-by-step procedure for preparing for the data reduction.

0. Prepare

Change your working directory to the one where you wish the reduced data products to appear. Any directory will do; however, you likely don’t want this to be the same directory that holds the raw data.

1. First Execution

We recommend you first execute pypeit_setup like this:

pypeit_setup -r path_to_your_raw_data/LB -s keck_lris_blue

where the two command-line options are required and provide:

  • -s: Sets the spectrograph to be reduce. This is camera specific (e.g., you need to reduce LRIS red data separately from LRIS blue data).

  • -r: Sets the full path to the raw data and the root prefix of the FITS files

Specifically note that this first run does not include the -c argument.

This execution of pypeit_setup searches for all *.fits and *.fits.gz files with the provided root directory. Generally, the provided path should not contain a wild-card and it is best if you provide the full path; however, you can search through multiple directories as follows:

pypeit_setup -r "/Users/xavier/Keck/LRIS/data/2016apr06/Raw/*/LB" -s keck_lris_blue

2. Inspect the outputs

The call above creates two files in the setup_files/ folder, created where you executed pypeit_setup:

  • {spectrograph}.sorted: This shows the unique configurations and the list of frames associated with each configuration as determine by the automated procedures in PypeIt. Each unique configuration is given a capital letter identifier (e.g., A,B,C,D…).

  • {spectrograph}.obslog: This provides the default log file, identical to the result of running pypeit_obslog.

Here is an example .sorted file for some example Keck DEIMOS data:

##########################################################
Setup A
     dispname: 830G
       decker: LongMirr
      binning: 1,1
    dispangle: 8099.98291016
          amp: SINGLE:B
      filter1: OG550
#---------------------------------------------------------
              filename |                 frametype |                 ra |                dec |     target | dispname |   decker | binning |          mjd |    airmass | exptime |     dispangle |      amp | filter1 |  lampstat01 |    dateobs |         utc | frameno | calib
DE.20170527.06713.fits |                  arc,tilt |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.077631 | 1.41291034 |     1.0 | 8099.98291016 | SINGLE:B |   OG550 | Kr Xe Ar Ne | 2017-05-27 | 01:51:53.87 |      30 |     0
    d0527_0030.fits.gz |                  arc,tilt |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.077631 | 1.41291034 |     1.0 | 8099.98291016 | SINGLE:B |   OG550 | Kr Xe Ar Ne | 2017-05-27 | 01:51:53.87 |      30 |     0
    d0527_0031.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 |  57900.07851 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:53:10.93 |      31 |     0
DE.20170527.06790.fits | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 |  57900.07851 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:53:10.93 |      31 |     0
    d0527_0032.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.079356 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:54:24.03 |      32 |     0
DE.20170527.06864.fits | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.079356 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:54:24.03 |      32 |     0
DE.20170527.06936.fits | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.080211 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:55:36.93 |      33 |     0
    d0527_0033.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |               45.0 | DOME PHLAT |     830G | LongMirr |     1,1 | 57900.080211 | 1.41291034 |     4.0 | 8099.98291016 | SINGLE:B |   OG550 |          Qz | 2017-05-27 | 01:55:36.93 |      33 |     0
DE.20170527.37601.fits |                   science |  261.0363749999999 | 19.028166666666667 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.435131 | 1.03078874 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 10:26:41.61 |      80 |     0
DE.20170527.38872.fits |                   science |  261.0363749999999 | 19.028166666666667 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.449842 | 1.01267696 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 10:47:52.92 |      81 |     0
    d0527_0081.fits.gz |                   science |  261.0363749999999 | 19.028166666666667 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.449842 | 1.01267696 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 10:47:52.92 |      81 |     0
DE.20170527.41775.fits |                   science |  261.0362916666666 | 19.028888888888886 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.483427 | 1.00093023 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 11:36:15.35 |      83 |     0
    d0527_0083.fits.gz |                   science |  261.0362916666666 | 19.028888888888886 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.483427 | 1.00093023 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 11:36:15.35 |      83 |     0
DE.20170527.43045.fits |                   science |  261.0362916666666 | 19.028888888888886 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.498135 | 1.00838805 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 11:57:25.35 |      84 |     0
DE.20170527.44316.fits |                   science |  261.0362916666666 | 19.028888888888886 |   P261_OFF |     830G | LongMirr |     1,1 | 57900.512854 | 1.02377681 |  1200.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 12:18:36.71 |      85 |     0
DE.20170527.53184.fits |                   science | 349.99316666666664 |           -5.16575 |  Feige 110 |     830G | LongMirr |     1,1 | 57900.615484 | 1.42505162 |    45.0 | 8099.98291016 | SINGLE:B |   OG550 |         Off | 2017-05-27 | 14:46:24.88 |      93 |     0
##########################################################
Setup B
     dispname: 830G
       decker: LongMirr
      binning: 1,1
    dispangle: 8399.93554688
          amp: SINGLE:B
      filter1: OG550
#---------------------------------------------------------
          filename |                 frametype |                 ra |               dec |         target | dispname |   decker | binning |          mjd |    airmass | exptime |     dispangle |      amp | filter1 |  lampstat01 |    dateobs |         utc | frameno | calib
d0914_0002.fits.gz |                      bias | 299.99999999999994 |              80.0 |        unknown |     830G |     None |     1,1 |  58010.07499 |  1.0153979 |     1.0 | 7499.97998047 | SINGLE:B |       R |         Off | 2017-09-14 | 01:48:05.53 |       2 |     1
d0914_0011.fits.gz |                  arc,tilt |  57.99999999999999 |              45.0 |        unknown |     830G | LongMirr |     1,1 | 58010.135443 | 1.41291034 |     1.0 | 8399.93554688 | SINGLE:B |   OG550 | Kr Xe Ar Ne | 2017-09-14 | 03:15:07.98 |      11 |     1
d0914_0013.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |              45.0 |        unknown |     830G | LongMirr |     1,1 | 58010.137123 | 1.41291034 |     6.0 | 8399.93554688 | SINGLE:B |   OG550 |          Qz | 2017-09-14 | 03:17:33.43 |      13 |     1
d0914_0014.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |              45.0 |        unknown |     830G | LongMirr |     1,1 | 58010.138113 | 1.41291034 |     6.0 | 8399.93554688 | SINGLE:B |   OG550 |          Qz | 2017-09-14 | 03:18:59.03 |      14 |     1
d0914_0015.fits.gz | pixelflat,illumflat,trace |  57.99999999999999 |              45.0 |        unknown |     830G | LongMirr |     1,1 | 58010.138969 | 1.41291034 |     6.0 | 8399.93554688 | SINGLE:B |   OG550 |          Qz | 2017-09-14 | 03:20:13.93 |      15 |     1
d0914_0036.fits.gz |                   science | 11.389749999999998 | 9.032555555555556 | PSOJ011p09_OFF |     830G | LongMirr |     1,1 |  58010.48501 | 1.01790951 |  1200.0 | 8399.93554688 | SINGLE:B |   OG550 |         Off | 2017-09-14 | 11:38:32.60 |      36 |     1
d0914_0037.fits.gz |                   science | 11.390166666666666 | 9.033277777777778 | PSOJ011p09_OFF |     830G | LongMirr |     1,1 | 58010.499726 | 1.02369591 |  1200.0 | 8399.93554688 | SINGLE:B |   OG550 |         Off | 2017-09-14 | 11:59:43.61 |      37 |     1
d0914_0038.fits.gz |                   science | 11.389333333333331 | 9.031833333333335 | PSOJ011p09_OFF |     830G | LongMirr |     1,1 | 58010.514673 |  1.0383931 |  1200.0 | 8399.93554688 | SINGLE:B |   OG550 |         Off | 2017-09-14 | 12:21:14.62 |      38 |     1
d0914_0047.fits.gz |                   science |  76.37762499999998 | 52.83063888888889 |        G191B2B |     830G | LongMirr |     1,1 | 58010.641261 | 1.19898553 |    60.0 | 8399.93554688 | SINGLE:B |   OG550 |         Off | 2017-09-14 | 15:23:31.06 |      47 |     1
##end

This sorted file contains two configurations, A and B. The “setup block” (the information between the ###... and #--... lines) describes the instrument configuration specific to this setup as determined using the relevant metadata. Each “setup block” is followed by a table listing the files and relevant metadata for all files matched to that instrument configuration. The data provided is specific to each instrument, as defined by, e.g., pypeit_file_keys().

The sorted file is only provided as a means of assessing the automated setup identification and file sorting, and we encourage you to briefly review the results. You may recognize that you are missing calibrations or you may be surprised to see more configurations than you were expecting. In the latter case, you can help us improve the automated procedures in PypeIt by submitting an issue on GitHub (Submit an issue) and including the sorted file in the issue description.

Importantly, you should use the sorted file to decide which configuration (as selected by its letter) you wish to reduce. Also note that PypeIt cannot interpret any edits you make to this file; all user-level edits to the frame-typing, association of frames with given configurations, etc., must be done via the PypeIt Reduction File.

3. Second execution: Write the pypeit file for one or more setups

Provided you are happy with the sorted file, you should execute pypeit_setup a second time with the –cfg_split (shortcut -c) option. This will generate one or more sub-folders and populate each with a PypeIt Reduction File. Some example uses of the -c option are:

  • -c A: This will generate one folder+file for the chosen configuration

  • -c A,C: This will generate one folder+file for each input configuration

  • -c all: This will generate folders+files for all configurations

An example execution that only produces the PypeIt Reduction File for the A configuration is:

pypeit_setup -r path_to_your_raw_data/LB -s keck_lris_blue -c A

This example will generate a new folder named keck_lris_blue_A and within it will be a file named keck_lris_blue_A.pypeit.

4. Edit the pypeit file

You will likely need to edit your PypeIt Reduction File. This includes adding any desired changes to the Parameter Block, adding/removing files from the Data Block, correcting any erroneous frametypes (see Frame Types), specifying any desired association between science and calibration frames (see Calibration Groups), which science frames to combine (see Combining Science Exposures), and which frames to use as backgrounds and/or part of an AB offset sequence (see A-B image differencing).

Warning

Any execution of run_pypeit will crash if your PypeIt Reduction File includes entries with None frame types defined. You must either remove or edit those entries in the pypeit file by-hand after running pypeit_setup.


Options

Consider your need for the following additional options:

-b option

You should consider using the -b option if you need to specify:

  • the calibration frames that should be used with each science frame (the same can be achieved by dividing your dataset into multiple .pypeit files that all use the same calibrations),

  • groups of science frames that should be combined, and/or

  • groups of frames that should be treated as background frames in an on-off (e.g., ABBA) observation sequence.

This simply adds the relevant columns to the PypeIt Reduction File that you must edit by hand: calib, comb_id, and bkg_id. These columns are added by default for most near-IR spectrographs. See Calibrations and A-B image differencing for the syntax used for the data in these columns and how PypeIt uses them; see also a worked example in the Gemini-GNIRS HOWTO.

-m option

If you wish to include a column where you can include input for Manual Extraction, use the -m option.