Quick-Look Reductions


PypeIt provides a quick-look (QL) script pypeit_ql that executes PypeIt in a mode that should produce results more quickly but with potentially lower fidelity. These results should be used for quick inspection only; e.g., for real-time decision-making while at the telescope.

The primary way this is currently achieved is by:

  1. Enabling re-use of existing calibrations and/or facilitating the creation of calibrations that can be repeatedly applied to the data without reprocessing.

  2. Setting parameters in the reduction that enable a more quick-and-dirty reduction; e.g., only boxcar extraction is performed.

  3. Imposing a more-limited set of use cases, but automating effectively all of the “management” procedures (setting up directories, writing pypeit files, optimizing parameters) that are usually performed by the user when reducing data in those cases.

Particularly because of the latter, the quick-look script follows a specific Directory Structure and makes assumptions about which calibrations can be used that are more lenient than recommended for a robust reduction.

Importantly, pypeit_ql can only be used to reduce data for a single science target in a single instrument configuration. All the science frames provided will be combined. Standard star frames can be included and, as long as they are automatically identified as standards, they will be reduced separately from the science target. For instruments with dither patterns that PypeIt can parse, images will be grouped by dither offset position. These dithers can then be optionally combined using PypeIt’s 2D coadding script.

Here, we describe the algorithm and provide specific usage tutorials. The script usage can be displayed by calling the script with the -h option:

$ pypeit_ql -h
usage: pypeit_ql [-h] [--raw_files RAW_FILES [RAW_FILES ...]]
                 [--raw_path RAW_PATH] [--sci_files SCI_FILES [SCI_FILES ...]]
                 [--redux_path REDUX_PATH] [--parent_calib_dir PARENT_CALIB_DIR]
                 [--setup_calib_dir SETUP_CALIB_DIR] [--clear_science]
                 [--calibs_only] [--overwrite_calibs] [--det DET [DET ...]]
                 [--slitspatnum SLITSPATNUM] [--maskID MASKID]
                 [--boxcar_radius BOXCAR_RADIUS] [--snr_thresh SNR_THRESH]
                 [--ignore_std] [--skip_display] [--coadd2d]
                 [--only_slits ONLY_SLITS [ONLY_SLITS ...]] [--offsets OFFSETS]
                 [--weights WEIGHTS] [--spec_samp_fact SPEC_SAMP_FACT]
                 [--spat_samp_fact SPAT_SAMP_FACT] [--try_old]

Script to produce quick-look PypeIt reductions

positional arguments:
  spectrograph          A valid spectrograph identifier: bok_bc,
                        gemini_flamingos1, gemini_flamingos2,
                        gemini_gmos_north_e2v, gemini_gmos_north_ham,
                        gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
                        gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
                        gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
                        keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
                        keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
                        keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
                        keck_nires, keck_nirspec_high, keck_nirspec_high_old,
                        keck_nirspec_low, lbt_luci1, lbt_luci2, lbt_mods1b,
                        lbt_mods1r, lbt_mods2b, lbt_mods2r, ldt_deveny,
                        magellan_fire, magellan_fire_long, magellan_mage,
                        mdm_modspec, mdm_osmos_mdm4k, mdm_osmos_r4k,
                        mmt_binospec, mmt_bluechannel, mmt_mmirs, not_alfosc,
                        not_alfosc_vert, ntt_efosc2, p200_dbsp_blue,
                        p200_dbsp_red, p200_tspec, shane_kast_blue,
                        shane_kast_red, shane_kast_red_ret, soar_goodman_blue,
                        soar_goodman_red, tng_dolores, vlt_fors2, vlt_sinfoni,
                        vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
                        wht_isis_blue, wht_isis_red

  -h, --help            show this help message and exit
  --raw_files RAW_FILES [RAW_FILES ...]
                        Either a PypeIt-formatted input file with the list of
                        raw images to process and the relevant path, or a space-
                        separated list of the filenames (e.g., "img1.fits
                        img2.fits"). For the latter entry mode, the path
                        containing the files is set using --raw_path. (default:
  --raw_path RAW_PATH   Directory with the raw files to process. Ignored if a
                        PypeIt-formatted file is provided using the --rawfiles
                        option. (default: current working directory)
  --sci_files SCI_FILES [SCI_FILES ...]
                        A space-separated list of raw file names that are
                        science exposures. These files must *also* be in the
                        list of raw files. Use of this option overrides the
                        automated PypeIt frame typing. Should only be used of
                        automatic frame typing fails or is undesirable.
                        (default: None)
  --redux_path REDUX_PATH
                        Path for the QL reduction outputs. (default: current
                        working directory)
  --parent_calib_dir PARENT_CALIB_DIR
                        Directory with/for calibrations for *all* instrument
                        configurations/setups. If provided, the data for your
                        instrument configuration will be placed or pulled from a
                        relevant sub-directory. If None, the redux_path is used.
                        (default: None)
  --setup_calib_dir SETUP_CALIB_DIR
                        Directory with/for calibrations specific to your
                        instrument configuration/setup. Use of this option
                        circumvents the automated naming system for the
                        configuration/setup sub-directories. If None, the code
                        will try to find relevant calibrations in the
                        parent_calib_dir. If no calibrations exist in that
                        directory that match the instrument setup/configuration
                        of the provided data, the code will construct new
                        calibrations (assuming relevant raw files are provided).
                        (default: None)
  --clear_science       Remove the existing output science directories to force
                        a fresh reduction. If False, any existing directory
                        structure will remain, and any alterations to existing
                        science files will follow the normal behavior of
                        run_pypeit. (default: False)
  --calibs_only         Reduce only the calibrations? (default: False)
  --overwrite_calibs    Re-process and overwrite any existing calibration files.
                        (default: False)
  --det DET [DET ...]   A space-separated set of detectors or detector mosaics
                        to reduce. By default, *all* detectors or default
                        mosaics for this instrument will be reduced. Detectors
                        in a mosaic must be a mosaic "allowed" by PypeIt and
                        should be provided as comma-separated integers (with no
                        spaces). For example, to separately reduce detectors 1
                        and 5 for Keck/DEIMOS, you would use --det 1 5; to
                        reduce mosaics made up of detectors 1,5 and 3,7, you
                        would use --det 1,5 3,7 (default: None)
  --slitspatnum SLITSPATNUM
                        Reduce the slit(s) as specified by the slitspatnum
                        value(s) (default: None)
  --maskID MASKID       Reduce the slit(s) as specified by the maskID value(s)
                        (default: None)
  --boxcar_radius BOXCAR_RADIUS
                        Set the radius for the boxcar extraction in arcseconds
                        (default: None)
  --snr_thresh SNR_THRESH
                        Change the default S/N threshold used during source
                        detection (default: None)
  --ignore_std          If standard star observations are automatically
                        detected, ignore those frames. Otherwise, they are
                        included with the reduction of the science frames.
                        (default: False)
  --skip_display        Run the quicklook without displaying any results.
                        (default: True)
  --coadd2d             Perform default 2D coadding. (default: False)
  --only_slits ONLY_SLITS [ONLY_SLITS ...]
                        If coadding, only coadd this space-separated set of
                        slits. If not provided, all slits are coadded. (default:
  --offsets OFFSETS     If coadding, spatial offsets to apply to each image; see
                        the [coadd2d][offsets] parameter. Options are restricted
                        here to either maskdef_offsets or auto. If not
                        specified, the (spectrograph-specific) default is used.
                        (default: None)
  --weights WEIGHTS     If coadding, weights used to coadd images; see the
                        [coadd2d][weights] parameter. Options are restricted
                        here to either uniform or auto. If not specified, the
                        (spectrograph-specific) default is used. (default: None)
  --spec_samp_fact SPEC_SAMP_FACT
                        If coadding, adjust the wavelength grid sampling by this
                        factor. For a finer grid, set value to <1.0; for coarser
                        sampling, set value to >1.0). (default: 1.0)
  --spat_samp_fact SPAT_SAMP_FACT
                        If coadding, adjust the spatial grid sampling by this
                        factor. For a finer grid, set value to <1.0; for coarser
                        sampling, set value to >1.0). (default: 1.0)
  --try_old             Attempt to load old datamodel versions. A crash may
                        ensue.. (default: False)

At present, only a few spectrographs have been extensively tested: keck_deimos, keck_lris_red, keck_nires, shane_kast_blue, and shane_kast_red.


To run pypeit_ql, you need to provide a minimal set of calibrations, either as new raw frames taken during your run or via an applicable set of calibrations taken from previous runs (e.g., for long-slit and/or fixed-format echelle observations).

If you provide only a set of calibrations, pypeit_ql will process them. If you also provide science exposures, they will be reduced using the provided or linked calibrations.

The script effectively requires that PypeIt is able to correctly determine the type of each input frame, without input from the user. If this fails, so too will the script. The only exception to this is that you can specify which frames are science frames, using the --sci_files argument. Importantly, however, files listed using the --sci_files option must also be listed among the raw files (see below).

Specifying the input raw files

The script provides a few ways that you can specify the files to reduce:

  1. Provide a file with a specific format that lists the files to be reduced. The format must follow the standard PypeIt file Data Block; however, only the filename column is required/used.

  2. Provide the directory and list of files directly on the command line.

  3. Provide only the raw directory, which will be used to search for any *.fits* files and reduce all files found.

An example file named input.rawfiles used in the first approach could look like this:

# Data block
raw read
    path /path/to/files
raw end

and you would pass it to the QL script using the --raw_files command-line argument:

pypeit_ql shane_kast_blue --raw_files input.rawfiles

You will get identical behavior if you instead used

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz --raw_path /path/to/files

Finally, if those three files are the only files with the relevant extension in /path/to/files, the third entry option would look like this:

pypeit_ql shane_kast_blue --raw_path /path/to/files

In this example (see more below), the three files are an arc-lamp exposure (b1.fits.gz), a dome-flat exposure (b10.fits.gz), and an on-sky science exposure (b27.fits.gz). PypeIt is generally able to identify science frames from the Shane/Kast spectrograph; however, you could specify the science frame in the above example like so:

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz --raw_path /path/to/files --sci_files b27.fits.gz

Directory Structure

As with typical executions of run_pypeit, pypeit_ql yields directories with calibrations, quality-assessment plots, and science products. The difference is that pypeit_ql keeps the calibrations and science products separate.

For example, executing:

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz --raw_path /path/to/files

will yield two directories where you executed the call: b27/ and shane_kast_blue_A/. Both directories will look very similar to a normal execution of run_pypeit (see Directory Structure), except the latter will only contain calibrations (and related QA plots) and the former will only contain the science results (and related QA plots). The name of the directory with the reduction for the science frames is based on the name of the frame, b27 in this example. The name of directory with the calibrations is always the combination of the instrument name and setup/configuration identifier (e.g. shane_kast_blue_A), just as produced by pypeit_setup. A symlink to the shane_kast_blue_A/Calibrations/ directory is included in b27/ such that the directory mimics the normal output from run_pypeit.

If multiple science frames are provided, the name of the output directory combines the names of the first and last science frames in the stack. For example:

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz b28.fits.gz b29.fits.gz --raw_path /path/to/files

would produce a b27-b29 directory (assuming b27.fits.gz, b28.fits.gz, and b29.fits.gz are all science frames) with the results produced by stacking all 3 science frames. For Shane/Kast, images are stacked via simple coaddition of the frames, not a “2D coadd” that includes spatial and spectral registration of the slit images.

Use of existing calibrations

None of the examples above have provided a path with/for the processed calibration files. This means the code uses the current working directory as the “parent” calibration directory.

The “parent” calibration directory potentially contains calibrations for many different instrument setups/configurations, each in their own directory that follows the PypeIt naming scheme; e.g., shane_kast_blue_A/, shane_kast_blue_B/, etc. After pypeit_ql parses the input files, the script compares the setup/configuration of the science frames to the available calibrations. If any exist, they will be used and any provided calibration frames will be ignored unless you set the --overwrite_calibs option.

In terms of a workflow, this means that you might first run:

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz --raw_path /path/to/files

and each subsequent call can omit the calibration files, like so:

pypeit_ql shane_kast_blue --raw_files b28.fits.gz --raw_path /path/to/files
pypeit_ql shane_kast_blue --raw_files b27.fits.gz b28.fits.gz --raw_path /path/to/files
pypeit_ql shane_kast_blue --raw_files b27.fits.gz b28.fits.gz b29.fits.gz --raw_path /path/to/files

assuming the instrument setup/configuration has not changed. This forces pypeit_ql to match the instrument setup to the available calibrations; however, you can force the code to use a specific set of calibrations by specifying it directly, like so:

pypeit_ql shane_kast_blue --raw_files b28.fits.gz --raw_path /path/to/files --setup_calib_dir ./shane_kast_blue_A/Calibrations

Alternatively, calibrations that maintain long-term stability (or at least stable enough for quick-look) can be stored in a parent directory and you can use them with pypeit_ql without needed to provide any raw calibrations frames. To do so, call:

pypeit_ql shane_kast_blue --raw_files b27.fits.gz --raw_path /path/to/files --parent_calib_dir /path/to/calibration/archive

If no appropriate calibrations are found, the code will fault.


The quick-look script proceeds as follows:

  • Parse the list of files to be processed

  • Extract metadata from the file headers, just as done by pypeit_setup. If any frames provided cannot be typed, the code will fault during this “setup phase.”

  • Isolate all of the science frames. Standard frames will also be isolated, unless the user selects to ignore standards using --ignore_std.

    • All science+standard frames must come from the same instrument setup/configuration.

    • All science images (and separately all standard frames) either set to the same combination group (meaning they will be stacked) or, for some instruments, parsed into dither sequences. Separate dither sequences are combined; i.e., all “A” frames with a given offset are combined.

    • All science+standard images are forced into the same calibration group, if not automatically done.

    • If a “parent” calibration directory is provided (using the --parent_calib_dir option), the script matches the instrument setup/configuration to the available calibrations. If none are found, the code will fault; otherwise, this matching is used to set the calibrations directory (i.e., the same thing that can be done manually using the --setup_calib_dir option).

  • If the calibration directory is not yet defined (either because it wasn’t set directly or no parent directory was set), the code will first try to find any matching calibrations in the current working directory.

    • If matches are found, the calibrations will either be used as they are or, if the user sets the --overwrite_calibs flag, reprocessed using the provided files.

    • If matches are not found, new calibrations are processed using the provided files, and the setup/configuration identifier is forced to be consistent with any existing calibrations in the directory. For example, if shane_kast_blue_A exists in your current working directory but the new files you are trying to process do not match the setup for those calibrations, a new directory called shane_kast_blue_B/ will be created with the newly processed calibrations.

  • Any new calibrations are processed using the pypeit file written to the new calibrations directory (e.g., shane_kast_blue_B). Do not delete these pypeit files! They are what’s used to match the instrument setup/configuration to new frames to be processed. Note that adjustments can be made directly to that file and the calibrations can be re-processed using a normal execution of run_pypeit.

  • The quick-look script can be run by only providing calibrations frames. The code will end here if that is the case.

  • If science (and/or standard) frames are provided, the code creates the output directory and writes a pypeit file used to process the data. Similar to the calibrations, this file can be edited directly and the data can be reprocessed using a normal execution of run_pypeit.

  • If the user selects to coadd 2D frames (e.g., coadding different dither positions) using the --coadd tag, the code will execute pypeit_setup_coadd2d using the written pypeit file and then execute the 2D coadd.

Processing Details

The details of the pypeit_ql script given above is essentially just management of a provided set of files to construct a directory structure and a set of pypeit files. Beyond that, it executes run_pypeit exactly as one would from the command line. The only difference is that it sets specific parameters that execute PypeIt in a mode that should proceed more quickly.

For example, cosmic rays are not identified/rejected in the science image and only boxcar extraction is performed.


The default quicklook mode for longslit reductions is to use the boxcar extraction method, skip bias image subtraction (maintain overscan), and skip CR rejection.

Standard call

Here is a sample call for a standard longslit run on files from the shane_kast_blue Development suite. In the example, the files b1.fits.gz, b10.fits.gz, and b27.fits.gz are an arc, flat, and science frame. But their ordering is not important.

pypeit_ql shane_kast_blue --raw_files b1.fits.gz b10.fits.gz b27.fits.gz --raw_path /path/to/files

This call first generates a shane_kast_blue_A folder with the processed calibrations (assuming there are no other calibrations in the current working directory) and associated QA Outputs. It then generates a separate folder named b27, which holds the Science folder with the processed 2D spectral image and the extracted spectra.

Other Options

Here are a few more options


Over-ride the default boxcar extraction radius with --box_radius. The value is given in arcseconds. This will over-ride the defaut as described in ExtractionPar Keywords.


It will greatly speed things up to isolate the detector(s) of interest. Use --det with the same syntax as the parameter detnum, as described in ReduxPar Keywords.

Here is an example with the (old) keck_lris_red detector:

pypeit_ql keck_lris_red --raw_path /path/to/files --raw_files LR.20160216.40478.fits.gz  --setup_calib_dir /path/to/parent/calib/dir/keck_lris_red/long_600_7500_d560/Calibrations --det 2

This will only process the second detector.


Here are some options specific to multi-slit observations.

Isolating a slit

In general, reducing all of the slits from a multi-slit observation will not be quick. Therefore, you may wish to isolate a single slit.

This can be done in two ways.


Specify the detector and spatial position of the slit you wish to reduce.

Here is an example with keck_deimos:

pypeit_ql keck_deimos --raw_path /path/to/files --raw_files d1010_0056.fits.gz --setup_calib_dir /path/to/parent/calib/dir/keck_deimos/600ZD_M_6500/Calibrations --slitspatnum MSC02:452

Here we have specified --slitspatnum as MSC02:452, which means use the 2nd mosaic and the slit closest to position 452.

This requires that the detector(s) with this slit have been calibrated (or will be calibrated, e.g. by specfiying --det).


Specify the user defined maskID value for the slit of interest. This will be an integer, e.g. 958454.

Here is an example with keck_deimos:

pypeit_ql keck_deimos --raw_path /path/to/files --raw_files d1010_0056.fits.gz --setup_calib_dir /path/to/parent/calib/dir/keck_deimos/600ZD_M_6500/Calibrations --maskID 958454

This requires that the detector(s) with this slit have been calibrated (or will be calibrated, e.g. by specifying --det).