This doc goes through a full run of PypeIt on one of our Keck LRIS multi-object datasets, specifically the keck_lris_red_mark4/multi_600_10000_slitmask dataset. These are Keck LRIS RED multi-slit observations taken with the new Mark IV detector and the 600/10000 grating. See here to find the example dataset.


This doc shows the reduction steps for a multi-slit dataset. For long-slit datasets, the reduction steps are the same, but the setup is considerably simpler. In the text below, we will point out the differences.

If you’re having trouble reducing your data, we encourage you to try going through this tutorial using this example dataset first. Please join our PypeIt Users Slack (using this invitation link) to ask for help, and/or Submit an issue to Github if you find a bug!

The following was performed on a Macbook Pro with 16 GB RAM and took approximately 30 minutes.


Organize data

Identify the folder where the raw data are stored and make sure you have all the calibration files you need, in addition to the science ones. In this example, the raw data are stored in the folder /PypeIt-development-suite/RAW_DATA/keck_lris_red_mark4/multi_600_10000_slitmask.

The files within this folder are:

$ ls -1

This folder can include data from different datasets (e.g., more than one slitmask or observations in various gratings). The script pypeit_setup (see next step) will help to parse the desired dataset.


The following is only for multi-slit datasets. For long-slit observations, the user can skip this step.

Pypeit is able to incorporate slitmask information in the reduction routine, (similarly to how it is done for Keck/DEIMOS observations). However, unlike DEIMOS, LRIS slitmask information is not included in the raw data, and the user needs to add it to the raw data before running PypeIt. The software TILSOTUA can help with that, but the user needs first to obtain the mask design files. The files needed are two output files produced by autoslit when creating the slitmask for the observations, and The ASCII object list file fed as input to autoslit. Those files are processed by TILSOTUA and converted into binary tables, containing the slitmask design information, the object catalog and the mapping between the two. The user, subsequently, needs to append the binary tables to the raw flat field image file. See Slit-masks for more details. In this example, the raw file r230417_01072_with_maskinfo.fits already includes the slitmask information.

If the slitmask information is not found in the raw data file, Pypeit will still be able to reduce the data but will not be able to use the slitmask information.

Run pypeit_setup

The first script to run with PypeIt is pypeit_setup, which examines the raw files and generates a sorted list and (when instructed) one PypeIt Reduction File per instrument configuration.

See complete instructions provided in Setup.


Know the spectrograph you need to use! LRIS went through different upgrades (FITS file format and detectors, see Keck LRIS) and PypeIt uses different spectrograph names to differentiate between them. In this example, we are using observations taken with the new Mark IV detector in the red arm, so we use the spectrograph name keck_lris_red_mark4. PypeIt will throw an error if you use the wrong LRIS spectrograph name.

For this example, we move to the folder where we want to perform the reduction and save the associated outputs and we run:

cd folder_for_reducing
pypeit_setup -s keck_lris_red_mark4 -r /PypeIt-development-suite/RAW_DATA/keck_lris_red_mark4/multi_600_10000_slitmask

This creates, in a folder called setup_files/, a .sorted file that shows the raw files organized by datasets. We inspect the .sorted file and identify the dataset that we want to reduced (in this case it is indicated with the letter A ) and re-run pypeit_setup as:

pypeit_setup -s keck_lris_red_mark4 -r /PypeIt-development-suite/RAW_DATA/keck_lris_red_mark4/multi_600_10000_slitmask -c A

This creates a PypeIt Reduction File called keck_lris_red_mark4_A.pypeit inside a folder called keck_lris_red_mark4_A/, and it looks like this:

# Auto-generated PypeIt input file using PypeIt version: 1.14.1.dev432+gf9a4a24da
# UTC 2023-12-08T19:35:05.109

# User-defined execution parameters
    spectrograph = keck_lris_red_mark4

# Setup
setup read
Setup A:
  amp: '4'
  binning: 2,2
  cenwave: 8081.849304
  decker: frb22022
  dichroic: '680'
  dispangle: 29.418736
  dispname: 600/10000
setup end

# Data block
data read
 path /PypeIt-development-suite/RAW_DATA/keck_lris_red_mark4/multi_600_10000_slitmask
                        filename |                 frametype |                 ra |                 dec |     target |  dispname |   decker | binning |          mjd | airmass |       exptime | dichroic | amp | dispangle |     cenwave |  hatch |                  lampstat01 |    dateobs | frameno | calib
              r230417_01064.fits |                  arc,tilt | 229.99999999999997 |                45.0 | DOME FLATS | 600/10000 | frb22022 |     2,2 | 60051.725892 |    1.41 |   1.000125952 |      680 |   4 | 29.417606 | 8081.269988 | closed | HgI NeI ArI CdI ZnI KrI XeI | 2023-04-17 |    1064 |     0
r230417_01072_with_maskinfo.fits | pixelflat,illumflat,trace | 229.99999999999997 |                45.0 | DOME FLATS | 600/10000 | frb22022 |     2,2 | 60051.729634 |    1.41 |  10.000135168 |      680 |   4 | 29.417606 | 8081.269988 |   open |                          on | 2023-04-17 |    1072 |     0
              r230417_01033.fits |                   science |          203.90275 | -28.032416666666666 |   frb22022 | 600/10000 | frb22022 |     2,2 | 60051.349105 |    1.71 | 500.000625152 |      680 |   4 | 29.418736 | 8081.849304 |   open |                         off | 2023-04-17 |    1033 |     0
data end

Inspecting this file, we want to make sure that all the frame types were accurately assigned in the Data Block. If not, these can be fixed by editing the PypeIt Reduction File directly; see instructions here. We can also remove any bad (or undesired) calibration or science frames from the list, by either deleting them altogether or commenting them out with a #.

In this example, all the frametypes were accurately assigned. We only make edits to the Parameter Block. The edited version looks like this (pulled directly from the PypeIt Development Suite):

# Auto-generated PypeIt input file using PypeIt version: 1.12.3.dev7+ga9075153d
# UTC 2023-05-08T19:54:58.446

# User-defined execution parameters
    spectrograph = keck_lris_red_mark4
    use_biasimage = False
   use_maskdesign = True
    use_alignbox = False
    assign_obj = True
    extract_missing_objs = True
    snr_thresh = 3.0
    use_2dmodel_mask = False

# Setup
setup read
Setup A:
  amp: '4'
  binning: 2,2
  cenwave: 8081.849304
  decker: frb22022
  dichroic: '680'
  dispangle: 29.418736
  dispname: 600/10000
setup end

# Data block
data read
# path PypeIt-development-suite/RAW_DATA/keck_lris_red_mark4/multi_600_10000_slitmask
 path multi_600_10000_slitmask
                        filename |                 frametype |                 ra |                 dec |     target |  dispname |   decker | binning |          mjd | airmass |       exptime | dichroic | amp | dispangle |     cenwave |  hatch |                  lampstat01 |    dateobs | frameno | calib
              r230417_01064.fits |                  arc,tilt | 229.99999999999997 |                45.0 | DOME FLATS | 600/10000 | frb22022 |     2,2 | 60051.725892 |    1.41 |   1.000125952 |      680 |   4 | 29.417606 | 8081.269988 | closed | HgI NeI ArI CdI ZnI KrI XeI | 2023-04-17 |    1064 |     0
r230417_01072_with_maskinfo.fits | pixelflat,illumflat,trace | 229.99999999999997 |                45.0 | DOME FLATS | 600/10000 | frb22022 |     2,2 | 60051.729634 |    1.41 |  10.000135168 |      680 |   4 | 29.417606 | 8081.269988 |   open |                          on | 2023-04-17 |    1072 |     0
              r230417_01033.fits |                   science |          203.90275 | -28.032416666666666 |   frb22022 | 600/10000 | frb22022 |     2,2 | 60051.349105 |    1.71 | 500.000625152 |      680 |   4 | 29.418736 | 8081.849304 |   open |                         off | 2023-04-17 |    1033 |     0
data end

The parameters use_maskdesign, use_alignbox, assign_obj, and extract_missing_objs are all parameters that are specific to the slitmask design ingestion (see Slitmask ID assignment and missing slits, RA, Dec and object name assignment to 1D extracted spectra, and Object position on the slit from slitmask design). These are not used for long-slit observations, therefore the user can ignore them.

The other parameters are set to improve the object finding and extraction, and to instruct PypeIt not to use bias frames for the reduction (given that no bias frames are provided). See User-level Parameters for more details.


PypeIt has a long list of parameters that can be set by the user to customize the reduction. This makes PypeIt very flexible and able to reduce a wide range of data from many instruments. Most parameters are set by default for the specific instrument, see KECK LRISr (keck_lris_red_mark4). Moreover, there are some parameters that are set by default for a specific configuration within the same instrument. For example, in many cases, PypeIt uses by default different wavelength templates for different gratings (see Wavelength Solution). The default parameters are not shown in the PypeIt Reduction File, therefore it may be sometime difficult to know which parameters to set and which ones to leave as default. To help with this, the user can inspect the .par file, which is generated at the very beginning of the main run (see below). This file contains every single available parameter with the assigned value, giving the user an idea of what are the values of the default parameters.

Main Run

Once the PypeIt Reduction File is ready, the main call is simply:

cd keck_lris_red_mark4_A
run_pypeit keck_lris_red_mark4_A.pypeit -o

The -o flag indicates that any existing output files should be overwritten. As there are none, it is superfluous but we recommend (almost) always using it.

The code will run uninterrupted until the basic data-reduction procedures (wavelength calibration, field flattening, object finding, sky subtraction, and spectral extraction) are complete; see PypeIt’s Core Data Reduction Executable and Workflow.

As the code processes the data, it will produce a number of files and QA plots that can be inspected. A number of Inspection Scripts are available to help with this process. We present some of these below.


Slit Edges

The code first uses the trace frames to find the slit edges on the detector. When available, the slitmask design information is used to help find the slit edges (not for long-slit observations).

To check that PypeIt correctly identified every slits, we can run the pypeit_chk_edges script, with this explicit call:

pypeit_chk_edges Calibrations/Edges_A_0_DET01.fits.gz

which opens the ginga image viewer. Here is a zoom-in screenshot from the first tab in the ginga window:


The data shown is the Trace Image, i.e., the flat image. The green/magenta lines indicate the left/right slit edges. The aquamarine labels starting with an S are the internal slit identifiers of PypeIt, while the cyan numbers are the slit ID values from the slitmask design, which within the Pypeit framework are called maskdef_id. The maskdef_id values are never shown for long-slit observations.

See Edges for further details.

We can also inspect the Slits file, which contains the main information on the traced slit edges, organized into left-right slit pairs. This is a FITS file containing one or two extensions. The second extension, called MASKDEF_DESIGNTAB, which is available only for multi-slit observations and if the slitmask design information is provided, contains all the relevant slitmask design information. In this example, the first few rows of MASKDEF_DESIGNTAB looks like this:

from import fits
from astropy.table import Table'Calibrations/Slits_A_0_DET01.fits.gz')

 int64    int64        float64            float64        int64    int64      float64      float64         float64             float64            float64            float64            float64       int16 int64      float64             float64        str32   float64    str32         float64            float64
------- --------- ------------------ ------------------ ------- ---------- ------------ ------------ ------------------ ------------------- ------------------ ------------------ ------------------ ----- ----- ------------------ ------------------- -------- ------- ----------- ------------------ ------------------
      0      1032 193.60062170028687 231.29763960838318     212         11        211.0        247.0 203.89075232578443 -28.092169267295024  9.822391510009766 0.9919275641441345 15.412363052368164     0    11 203.89072727254896  -28.09230361665466   gal172  22.353           r  4.424071886416087  5.398871433622549
      1      1032 241.12293437753794 288.52713346481323     265          8        257.0        306.0  203.8415716281879  -28.07607209753345 13.274372100830078 0.9922505021095276 15.440143585205078     0     8 203.84147689509808 -28.076401475507694   gal227  22.567           r  5.414064692172961  7.860407169533709
      2      1032   302.332216322422 367.47612684965134     335          7        318.0        382.0  203.9166660875971 -28.089147654373047 17.321680068969727 0.9887152314186096 15.405740737915039     0     7 203.91661609856362 -28.089363864800017   gal158  21.796           r  7.868456854365888   9.45355797935212
      3      1032 375.55078342556953  391.8777657151222     384         21        392.0        407.0  203.8961977293406 -28.080303275622498  3.998490333557129 3.9599761962890625 15.393735885620117     1    21 203.89621037750157 -28.080305533949872    star6  15.653           r 2.0029598959835964  1.996361115945478
      4      1032  465.5582481622696 481.88689267635345     474         20        480.0        495.0 203.92635312081273 -28.080791243205006 3.9996068477630615 3.9622230529785156 15.388344764709473     1    20 203.92636894459946 -28.080795440193242   star10  15.728           r  1.999165077712833  2.001801640220989

See Slits for a description of all the columns of this table.


You can view the Arc image used for the wavelength calibration of the science frame with ginga:

ginga Calibrations/Arc_A_0_DET01.fits

As typical of most arc images, we can see a series of arc lines, here oriented approximately horizontally. It is important to inspect the arc image to make sure it looks good, ensuring to get a good wavelength calibration.

See Arc for further details.


It is, also, very important to inspect the PypeIt QA for the wavelength calibration. These are PNG files in the QA/PNG/ folder.


Here is an example of the 1D fits for one of the slit on the detector, written to the QA/PNGs/Arc_1dfit_A_0_DET01_S0212.png file:


The left panel shows the arc spectrum extracted down the center of the slit, with the lines used by the wavelength calibration marked in green. Gray lines mark detected features that were not included in the wavelength solution. The top-right panel shows the fit (red) to the observed trend in wavelength as a function of spectral pixel (blue crosses); gray circles are features that were rejected by the wavelength solution. The bottom-right panel shows the fit residuals (i.e., data - model).

What you hope to see in this QA is:

  • On the left, many of the arc lines marked with green IDs

  • In the upper right, an RMS < 0.1 pixels

  • In the lower right, a random scatter about 0 residuals

In addition, the script pypeit_chk_wavecalib provides a summary of the wavelength calibration for all the spectra. We can run it with this simple call:

pypeit_chk_wavecalib Calibrations/WaveCalib_A_0_DET01.fits

and it prints on screen the following (here truncated) table:

 N. SpatID minWave Wave_cen maxWave dWave Nlin     IDs_Wave_range    IDs_Wave_cov(%) measured_fwhm  RMS
--- ------ ------- -------- ------- ----- ---- --------------------- --------------- ------------- -----
  0    212  6835.4   8469.9 10121.3 1.610   43  6931.379 -  9925.919            91.1           2.7 0.027
  1    265  6038.3   7674.1  9327.6 1.612   45  6336.179 -  9227.030            87.9           2.7 0.025
  2    335  7169.5   8831.4 10485.5 1.611   39  7427.339 - 10472.923            91.8           2.7 0.348
  3    384     0.0      0.0     0.0 0.000    0     0.000 -     0.000             0.0           0.0 0.000
  4    474     0.0      0.0     0.0 0.000    0     0.000 -     0.000             0.0           0.0 0.000

See pypeit_chk_wavecalib for a detailed description of all the columns. Note that the slits with SpatID 384 and 474 have all the values set to 0.0. This is because these slits are actually alignment boxes and they are not reduced by the pipeline.


Wavelength tilts are measured performing a 2D fit to the traced arc lines. There are several QA files written for the 2D fits. One example is QA/PNGs/Arc_tilts_2d_A_0_DET01_S0212.png:


Each horizontal line of black open circles identifies a traced arc line. Red circles shows the points that were rejected in the 2D fitting, and the black points show the model predicted position. The text across the top of the figure gives the RMS of the 2D wavelength solution, which should be less than 0.1 pixels.

The 2D fit for the wavelength tilts can also be inspected using the script pypeit_chk_tilts, which shows a Tiltimg image in a ginga or matplotlib window with the traced and 2D fitted tilts over-plotted. Here is an example:

pypeit_chk_tilts Calibrations/Tilts_A_0_DET01.fits

This shows a zoom-in of a Tiltimg image in a ginga window with overlaid the 2D fitted tilts (blue), the masked pixels (red), and the pixels rejected in the 2D fitting (yellow). The lines that are not overlaid with any colored tilts are the ones that were not detected for the purpose of the performing the 2D fitting.

See Tilts for further details.


PypeIt computes a number of multiplicative corrections to correct the 2D spectral response for pixel-to-pixel detector throughput variations and lower-order spatial and spectral illumination and throughput corrections. We collectively refer to these as flat-field corrections; see here and here. To inspect the Flat images we can use the script pypeit_chk_flats, with this explicit call:

pypeit_chk_flats Calibrations/Flat_A_0_DET01.fits

Here is a zoom-in screenshot from the first tab in the ginga window (pixflat_norm):


This shows the normalized flat field image. The green/magenta lines are the slit edges, which are tweaked using the illumination flat field.

See Flat and Flat Fielding for further details.

Object finding

After the above calibrations are complete, PypeIt will iteratively identify sources, perform global and local sky subtraction, and perform 1D spectral extractions. This process is fully described here: Object Finding.

PypeIt produces QA files that allow you to assess the detection of the objects. One example is QA\PNGs\r230417_01033-frb22022_LRISr_20230417T082242.672_DET01_S0212_obj_prof.png:


This shows the spatial profile of the object’s S/N collapsed along the spectral direction. The dashed red line is the S/N threshold set by the FindObjPar Keywords, and the green circle marks the spatial position of the detected object. This plot is useful to assess if the object was correctly detected and if the S/N threshold (snr_thresh) set is appropriate for the observation. See Object Finding for further details.


PypeIt also measures a spectral flexure correction, by performing a cross-correlation between the sky spectrum extracted in each slit and an archived sky spectrum. The relative shift between the two spectra is then used to correct the wavelength solution for each slit (see Spectral Flexure). Two flexure corrections are computed, one (called global) using the sky lines extracted at the center of the slit, and one (called local) using the sky lines extracted at the location of the science object.

There are two QA files (two for the global and two for the local correction) that can be used to assess the flexure correction. Here is an example of two QA files for the global correction, called QA/PNGs/r230417_01033-frb22022_LRISr_20230417T082242.672_global_DET01_S0212_spec_flex_corr.png and QA/PNGs/r230417_01033-frb22022_LRISr_20230417T082242.672_global_DET01_S0212_spec_flex_sky.png:

../_images/lris_flexure_qa2.png ../_images/lris_flexure_qa.png

The first plot shows the polynomial fit (black line) between the top seven highest cross-correlation values (y-axis) and the corresponding shift in pixels (x-axis). The value of the shift with the highest cross-correlation is the flexure correction applied to the wavelength solution (also printed in the plot). The user should inspect this plot to make sure that the fit is good and that the value of the flexure shift is comparable to what expected for the observations. The second plot shows sky spectrum cutouts around some of the sky lines used for the flexure correction. The black line is the sky spectrum extracted at the center of the slit shifted by the computed flexure correction, while the red line is the archived sky spectrum. This is another way to assess that the computed flexure correction is good. The user should hope to see a good match between the two spectra.


The primary science output from run_pypeit are 2D spectral images and 1D spectral extractions, located in the Science/ folder.; see Outputs.


Slit inspection

It is frequently useful to view a summary of the slits successfully reduced by PypeIt, by running the script pypeit_parse_slits. In this example, we can inspect the reduced 2D spectrum with this explicit call:

pypeit_parse_slits Science/spec2d_r230417_01033-frb22022_LRISr_20230417T082242.672.fits

which print the following (here truncated) table in the terminal:

============================== DET01 ==============================
SpatID   MaskID   MaskOFF (pix)  Flags
0212     0011     0.42           None
0265     0008     0.42           None
0335     0007     0.42           None
0384     0021     0.42           BOXSLIT
0474     0020     0.42           BOXSLIT

The four columns printed to screen are SpatID (the internal PypeIt ID), MaskID, (the slit ID from the slitmask design), MaskOFF (the dither offset) and Flags for each slit. Slits with Flags set to BOXSLIT are alignment boxes and are not reduced by PypeIt. If the calibration failed for some slits, the Flags column will show the reason for the failure. Those slits with None in the Flags column have been successfully reduced. Note that the MaskID and MaskOFF columns are not printed for long-slit observations.

Visual inspection

The 2D spectrum can be visually inspected using the script pypeit_show_2dspec. In this example, we can visualize the 2D spectrum with this explicit call:

pypeit_show_2dspec Science/spec2d_r230417_01033-frb22022_LRISr_20230417T082242.672.fits --removetrace

The --removetrace only shows the object trace in the first channel (the channel showing the calibrated science image), but does not include it in the remaining channels. It is helpful to to use the --removetrace option to better visualize the object traces (especially for faint objects).

We show here a zoom-in screenshot from three (sciimg-DET01, sky_resid-DET01, resid-DET01) of the four tabs in the ginga window:

../_images/lris_sciimg.png ../_images/lris_skyres.png ../_images/lris_res.png

This shows on the top the calibrated science image, in the middle the sky residual image (sky-subtracted calibrated image divided by the uncertainties), and on the right the residual image. The green/magenta lines are the slit edges. The orange lines (shown only in the first channel) are the object traces and the orange text is the PypeIt assigned name (starting with SPAT). Only for multi-slit observations, and if the slitmask design information is provided, The orange text also includes the object name from the slitmask design information (starting with OBJNAME). Yellow lines, if present, indicate sources that, although not detected, have been extracted at the expected location using the slitmask design information (still only for multi-slit observations). See Spec2D Output for further details.

The main assessments to perform are to make sure that the object is well traced, that there are little to no strong sky residuals in the sky_resid channel, and that the data in the resid channel looks like pure noise (see also pypeit_chk_noise_2dspec).


You can see a summary of all the extracted sources in the spec1d*.txt files saved in the Science/ folder. For this example, here are the first few lines of the file Science/spec1d_r230417_01033-frb22022_LRISr_20230417T082242.672.txt:

| slit |                    name | maskdef_id |  objname |     objra |    objdec | spat_pixpos | spat_fracpos | box_width | opt_fwhm |   s2n | maskdef_extract | wv_rms |
|  212 | SPAT0216-SLIT0212-DET01 |         11 |   gal172 | 203.89073 | -28.09230 |       216.3 |        0.454 |      3.00 |    1.184 |  2.21 |           False |  0.027 |
|  265 | SPAT0264-SLIT0265-DET01 |          8 |   gal227 | 203.84148 | -28.07640 |       263.5 |        0.384 |      3.00 |    1.240 |  2.19 |           False |  0.025 |
|  335 | SPAT0338-SLIT0335-DET01 |          7 |   gal158 | 203.91662 | -28.08936 |       338.1 |        0.458 |      3.00 |    0.829 |  4.22 |           False |  0.348 |

It shows a table with the PypeIt names of the extracted spectra in each slit and all the associated information about the extraction and the object. See Extraction Information for a detailed description of this file. Note that the maskdef_id, objname, objra, objdec and maskdef_extract columns are not printed for long-slit observations.

To inspect the 1D spectrum, we can use the script pypeit_show_1dspec, with a call like this:

pypeit_show_1dspec Science/spec1d_r230417_01033-frb22022_LRISr_20230417T082242.672.fits --exten 3

Since spec1d*.fits is a multi-extension FITS file that contains all the 1D extracted spectra (one per each extension), we need to specify which 1D spectrum (i.e., extension) we want to inspect by passing --exten 3 to the call. To help decide which 1D spectrum to visualize, we can run beforehand the following:

pypeit_show_1dspec Science/spec1d_r230417_01033-frb22022_LRISr_20230417T082242.672.fits --list

which lists all the extensions with the associated 1D spectrum PypeIt name and a few other information.

This is a screenshot from the GUI showing the 1D spectrum:


This uses the XSpecGUI from the linetools package. The black line is the flux and the red line is the estimated error. In the window, the user can press ? to open a webpage with the XSpecGUI keystrokes that help you navigate through the spectrum.

See Spec1D Output for further details.