Setup
Overview
The PypeIt Reduction File is the critical component to any successful run of PypeIt, and pypeit_setup provides an automated means of generating this file based on a set of fits files.
Below, we describe how PypeIt defines unique instrument configurations (“setups”) and sorts data in preparation for the data reduction. Then we describe our recommended method for generating the PypeIt Reduction File, which includes multiple executions of pypeit_setup. We also describe another preparatory script, pypeit_obslog, which provides a simple listing of the available data files; however, use of this script is optional in terms of setup for reducing your data.
Raw File Searches
PypeIt scripts that search for raw files in a given directory base the search on
a list of file extensions. These are generally .fits and .fits.gz, but
some spectrographs specify a different set.
Some scripts allow you to specify the extension to use for the search, which
must be one of the allowed extensions for that spectrograph. E.g.,
for a spectrograph that allows .fits and .fits.gz extension, you can
specify to only look for the .fits files, but you cannot have it look for
.fits.bz2 files. If your raw files have extensions that are currently not
allowed by the code, please Submit an issue.
If you have both compressed and uncompressed files in your directory, the search function will generally find both. You are strongly encouraged to only include one version (compressed or uncompressed) of each file in the directory with your raw data.
Use of Metadata to Identify Instrument Configurations
PypeIt distinguishes between instrument configurations using metadata pulled from the fits file headers, as defined specifically for each spectrograph. To list all of the metadata pulled from each data file and how it maps to the keywords in the data file headers, run, e.g.:
pypeit_obslog keck_deimos -k
(see pypeit_obslog and
meta_key_map()). For Keck/DEIMOS, this
prints:
Metadata Key Header Card
------------ -----------
ra RA
dec DEC
target TARGNAME
decker SLMSKNAM
binning None
mjd None
exptime ELAPTIME
airmass AIRMASS
dispname GRATENAM
hatch HATCHPOS
dispangle None
idname OBSTYPE
lampstat01 LAMPS
dateobs DATE-OBS
utc UTC
mode MOSMODE
amp AMPMODE
object OBJECT
filter1 DWFILNAM
frameno FRAMENO
instrument INSTRUME
where the left column provides the PypeIt-specific metadata
keyword and the right column is the associated instrument-specific
header card. Any metadata element with a header card set to None
means that the metadata keyword or value is conditioned on multiple
header values. In this example, the header card that provides the
central wavelength given the grating angle (dispangle) for DEIMOS
is dependent on the grating used (dispname); see
compound_meta().
A subset of this metadata is used to determine the unique instrument
configurations used and associate all of the data files to the relevant
configuration. Generally speaking, most instrument configurations are set by
the name (dispname) and central wavelength (dispangle) of the dispersing
element, the name of the decker or slit mask (decker), the name of the
beam-splitting dichroic (dichroic), and/or the on-chip binning
(binning). The specific set of metadata keys used for each instrument is
set by the configuration_keys method; for DEIMOS, see
configuration_keys().
The following table provides the list of metadata used to define the instrument
configuration for any supported spectrograph (as of PypeIt version
1.10.1dev):
Metadata Key |
Type |
Example |
Description |
|---|---|---|---|
|
str |
SINGLE:B |
Name of the amplifier used to read the detector |
|
str |
VIS |
Name of the spectrograph arm used to collect the data |
|
str |
1,1 |
On-chip binning |
|
str |
7500.0 |
Central wavelength of the disperser |
|
str |
[1:256,1:512] |
The science region of the detector |
|
str |
long_1.0 |
Name of the decker or slit mask |
|
str |
long_1.0 |
Partial Slitmask/decker name; see Automated sorting of Keck/MOSFIRE frames by instrument configuration |
|
str |
CHIP1 |
Name of the detector |
|
str |
560 |
Name of the dichroic |
|
float |
7500.0 |
Central wavelength for the dispersing element at the observed angle |
|
str |
830G |
Name of the dispersing element |
|
str |
J |
Name of the order-sorting filter |
|
str |
Slit length; see Automated sorting of Keck/MOSFIRE frames by instrument configuration |
|
|
str |
Slit width; see Automated sorting of Keck/MOSFIRE frames by instrument configuration |
Every unique combination of the relevant metadata found in any of the fits files to be reduced represents a unique instrument configuration for PypeIt to consider; these configurations can be determined automatically by pypeit_setup and will be identified by a capital letter, e.g., A. When executing run_pypeit, however, the instrument configuration is set by the Setup Block in the PypeIt Reduction File and not redetermined by the fits files. The latter allows the user flexibility to override PypeIt’s automated configuration settings. Currently, each PypeIt Reduction File should only provide data from one instrument configuration.
If you tend to observe with one instrument configuration and with a simple set of calibrations, then the setup to run PypeIt should be straightforward. However, if you use multiple configurations (e.g. gratings, grating tilts, slitmasks), then you must pay more careful attention to the setups.
Below, we describe how PypeIt automatically determines instrument configurations for a set of files and constructs auto-generated pypeit files.
pypeit_obslog
The pypeit_obslog script allows you to see a simple listing of the data
files in a given directory (or directories) and the metadata that PypeIt
will pull from their headers. As always, the script usage can be displayed by
calling the script with the -h option:
$ pypeit_obslog -h
usage: pypeit_obslog [-h] [-r ROOT] [-k] [-c COLUMNS] [-b] [-t BAD_TYPES] [-g]
[-i] [-s SORT] [-e EXTENSION] [-d OUTPUT_PATH] [-o]
[-f FILE] [-G]
spec
Construct an observing log for a set of files from the provided spectrograph
using PypeItMetaData.
positional arguments:
spec A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
keck_nires, keck_nirspec_high, keck_nirspec_high_old,
keck_nirspec_low, lbt_luci1, lbt_luci2, lbt_mods1b,
lbt_mods1r, lbt_mods2b, lbt_mods2r, ldt_deveny,
magellan_fire, magellan_fire_long, magellan_mage,
mdm_modspec, mdm_osmos_mdm4k, mdm_osmos_r4k,
mmt_binospec, mmt_bluechannel, mmt_mmirs, not_alfosc,
not_alfosc_vert, ntt_efosc2, p200_dbsp_blue,
p200_dbsp_red, p200_tspec, shane_kast_blue,
shane_kast_red, shane_kast_red_ret, soar_goodman_blue,
soar_goodman_red, tng_dolores, vlt_fors2, vlt_sinfoni,
vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
wht_isis_blue, wht_isis_red
options:
-h, --help show this help message and exit
-r ROOT, --root ROOT Root to search for data files. You can provide the top-
level directory (e.g., /data/Kast) or the search string
up through the wildcard (.e.g, /data/Kast/b). Use the
--extension option to set the types of files to search
for. (default: current working directory)
-k, --keys Do not produce the log; simply list the pypeit-specific
metadata keys available for this spectrograph and their
associated header cards. Metadata keys with header cards
that are None have no simple mapping between keyword and
header card. (default: False)
-c COLUMNS, --columns COLUMNS
A comma-separated list of columns to include in the
output table. Each column must be a valid pypeit
metadata keyword specific to this spectrograph (run
pypeit_obslog with the -k argument to see the valid
list). Additional valid keywords are directory,
filename, frametype, framebit, setup, calib, and
calibbit. If 'all', all columns collected for the pypeit
metadata table are included. If 'pypeit', the columns
are the same as those included in the pypeit file.
(default: pypeit)
-b, --bad_frames Clean the output of bad frames that cannot be reduced by
pypeit. (default: False)
-t BAD_TYPES, --bad_types BAD_TYPES
Dictates how frames that could not be given a valid type
should be treated. Options are: "keep" to include them
in the output, "rm" to remove them from the output,
"only" to only include the frames with unknown types in
the output (i.e, the frames with determined types are
excluded). (default: keep)
-g, --groupings Use this option to only determine the frame type. By
default, the script groups frames into expected
configuration and calibration groups, and it adds the
default combination groups. (default: True)
-i, --interact Once the metadata table is created, start an embedded
IPython session that you can use to interact with the
table (an Astropy.Table called fitstbl) directly.
(default: False)
-s SORT, --sort SORT Metadata keyword (pypeit-specific) to use to sort the
output table. (default: mjd)
-e EXTENSION, --extension EXTENSION
File extension to use. Must include the period (e.g.,
".fits") and it must be one of the allowed extensions
for this spectrograph. If None, root directory will be
searched for all files with any of the allowed
extensions. (default: None)
-d OUTPUT_PATH, --output_path OUTPUT_PATH
Path to top-level output directory. (default: current
working directory)
-o, --overwrite Overwrite any existing files/directories (default:
False)
-f FILE, --file FILE Name for the ascii output file. Any leading directory
path is stripped; use -d to set the output directory. If
None, the table is just printed to stdout. If set to
'default', the file is set to [spectrograph].obslog.
Note the file will *not* be written if you also include
the -i option to embed and interact with the table (you
can write the table using the astropy.table.Table.write
method in the embedded IPython session). The table is
always written in ascii format using
format=ascii.fixed_with for the call to
Astropy.table.Table.write . (default: None)
-G, --gui View the obs log in a GUI (default: False)
For example, if you’ve been observing with Keck DEIMOS, you can go into the directory with the raw data and execute:
pypeit_obslog keck_deimos
Or you can point to the directory that you want to list:
pypeit_obslog keck_deimos -r /path/to/raw/data
The listing is printed to stdout, but you can also specify a file for the
output. You can also select the metadata by which to sort the output, change
the columns that are printed, etc.
By default, the columns provided in the output table are identical to the
columns in the Data Block of a PypeIt Reduction File; to print columns with
all the metadata collected, add -c all to the command line.
pypeit_setup
pypeit_setup is the script one executes to prepare for the data reduction by
automatically associating fits files to specific Frame Types and
collecting groups of frames taken using unique instrument configurations. The
script usage can be displayed by calling the script with the -h option:
$ pypeit_setup -h
usage: pypeit_setup [-h] [-s SPECTROGRAPH] [-r ROOT [ROOT ...]] [-e EXTENSION]
[-d OUTPUT_PATH] [-o] [-c CFG_SPLIT] [-b] [-f] [-m]
[-v VERBOSITY] [-k] [-G]
Parse data files to construct a pypeit file in preparation for reduction using
'run_pypeit'
options:
-h, --help show this help message and exit
-s SPECTROGRAPH, --spectrograph SPECTROGRAPH
A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
keck_nires, keck_nirspec_high, keck_nirspec_high_old,
keck_nirspec_low, lbt_luci1, lbt_luci2, lbt_mods1b,
lbt_mods1r, lbt_mods2b, lbt_mods2r, ldt_deveny,
magellan_fire, magellan_fire_long, magellan_mage,
mdm_modspec, mdm_osmos_mdm4k, mdm_osmos_r4k,
mmt_binospec, mmt_bluechannel, mmt_mmirs, not_alfosc,
not_alfosc_vert, ntt_efosc2, p200_dbsp_blue,
p200_dbsp_red, p200_tspec, shane_kast_blue,
shane_kast_red, shane_kast_red_ret, soar_goodman_blue,
soar_goodman_red, tng_dolores, vlt_fors2, vlt_sinfoni,
vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
wht_isis_blue, wht_isis_red (default: None)
-r ROOT [ROOT ...], --root ROOT [ROOT ...]
Root to search for data files. You can provide the top-
level directory (e.g., /data/Kast) or the search string
up through the wildcard (.e.g, /data/Kast/b). Use the
--extension option to set the types of files to search
for. (default: current working directory)
-e EXTENSION, --extension EXTENSION
File extension to use. Must include the period (e.g.,
".fits") and it must be one of the allowed extensions
for this spectrograph. If None, root directory will be
searched for all files with any of the allowed
extensions. (default: None)
-d OUTPUT_PATH, --output_path OUTPUT_PATH
Path to top-level output directory. (default: current
working directory)
-o, --overwrite Overwrite any existing files/directories (default:
False)
-c CFG_SPLIT, --cfg_split CFG_SPLIT
Generate the PypeIt files and folders by input
configuration. To write all unique configurations
identifed, use 'all', otherwise provide the list of
configuration letters; e.g., 'A,B' or 'B,D,E' or 'E'.
(default: None)
-b, --background Include the background-pair columns for the user to edit
(default: False)
-f, --flexure Include the manual spatial shift (flexure) column for
the user to edit (default: False)
-m, --manual_extraction
Include the manual extraction column for the user to
edit (default: False)
-v VERBOSITY, --verbosity VERBOSITY
Verbosity level between 0 [none] and 2 [all]. Default:
1. Level 2 writes a log with filename setup_YYYYMMDD-
HHMM.log (default: 1)
-k, --keep_bad_frames
Keep all frames, even if they are identified as having
bad/unrecognized configurations that cannot be reduced
by pypeit. (This is the opposite of the --bad_frames
option in pypeit_obslog; i.e., you have to tell
pypeit_setup to keep these frames, whereas you have to
tell pypeit_obslog to remove them. (default: False)
-G, --gui Run setup in a GUI (default: False)
The following is a step-by-step procedure for preparing for the data reduction.
0. Prepare
Change your working directory to the one where you wish the reduced data products to appear. Any directory will do; however, you likely don’t want this to be the same directory that holds the raw data.
1. First Execution
We recommend you first execute pypeit_setup like this:
pypeit_setup -r path_to_your_raw_data/LB -s keck_lris_blue
where the two command-line options are required and provide:
-s: Sets the spectrograph to be reduce. This is camera specific (e.g., you need to reduce LRIS red data separately from LRIS blue data).
-r: Sets the full path to the raw data and the root prefix of the FITS files
Specifically note that this first run does not include the -c
argument.
This execution of pypeit_setup searches for all *.fits and
*.fits.gz files with the provided root directory. Generally, the
provided path should not contain a wild-card and it is best if
you provide the full path; however, you can search through multiple
directories as follows:
pypeit_setup -r "/Users/xavier/Keck/LRIS/data/2016apr06/Raw/*/LB" -s keck_lris_blue
2. Inspect the outputs
The call above creates two files in the setup_files/ folder, created where
you executed pypeit_setup:
{spectrograph}.sorted: This shows the unique configurations and the list of frames associated with each configuration as determine by the automated procedures in PypeIt. Each unique configuration is given a capital letter identifier (e.g., A,B,C,D…).
{spectrograph}.obslog: This provides the default log file, identical to the result of running pypeit_obslog.
Here is an example .sorted file for some example Keck DEIMOS data:
##########################################################
Setup A
dispname: 830G
decker: LongMirr
binning: 1,1
dispangle: 8099.98291016
amp: SINGLE:B
filter1: OG550
#---------------------------------------------------------
filename | frametype | ra | dec | target | dispname | decker | binning | mjd | airmass | exptime | dispangle | amp | filter1 | lampstat01 | dateobs | utc | frameno | calib
DE.20170527.06713.fits | arc,tilt | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.077631 | 1.41291034 | 1.0 | 8099.98291016 | SINGLE:B | OG550 | Kr Xe Ar Ne | 2017-05-27 | 01:51:53.87 | 30 | 0
d0527_0030.fits.gz | arc,tilt | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.077631 | 1.41291034 | 1.0 | 8099.98291016 | SINGLE:B | OG550 | Kr Xe Ar Ne | 2017-05-27 | 01:51:53.87 | 30 | 0
d0527_0031.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.07851 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:53:10.93 | 31 | 0
DE.20170527.06790.fits | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.07851 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:53:10.93 | 31 | 0
DE.20170527.06864.fits | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.079356 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:54:24.03 | 32 | 0
d0527_0032.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.079356 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:54:24.03 | 32 | 0
d0527_0033.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.080211 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:55:36.93 | 33 | 0
DE.20170527.06936.fits | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | DOME PHLAT | 830G | LongMirr | 1,1 | 57900.080211 | 1.41291034 | 4.0 | 8099.98291016 | SINGLE:B | OG550 | Qz | 2017-05-27 | 01:55:36.93 | 33 | 0
DE.20170527.37601.fits | science | 261.0363749999999 | 19.028166666666667 | P261_OFF | 830G | LongMirr | 1,1 | 57900.435131 | 1.03078874 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 10:26:41.61 | 80 | 0
d0527_0081.fits.gz | science | 261.0363749999999 | 19.028166666666667 | P261_OFF | 830G | LongMirr | 1,1 | 57900.449842 | 1.01267696 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 10:47:52.92 | 81 | 0
DE.20170527.38872.fits | science | 261.0363749999999 | 19.028166666666667 | P261_OFF | 830G | LongMirr | 1,1 | 57900.449842 | 1.01267696 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 10:47:52.92 | 81 | 0
DE.20170527.41775.fits | science | 261.0362916666666 | 19.028888888888886 | P261_OFF | 830G | LongMirr | 1,1 | 57900.483427 | 1.00093023 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 11:36:15.35 | 83 | 0
d0527_0083.fits.gz | science | 261.0362916666666 | 19.028888888888886 | P261_OFF | 830G | LongMirr | 1,1 | 57900.483427 | 1.00093023 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 11:36:15.35 | 83 | 0
DE.20170527.43045.fits | science | 261.0362916666666 | 19.028888888888886 | P261_OFF | 830G | LongMirr | 1,1 | 57900.498135 | 1.00838805 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 11:57:25.35 | 84 | 0
DE.20170527.44316.fits | science | 261.0362916666666 | 19.028888888888886 | P261_OFF | 830G | LongMirr | 1,1 | 57900.512854 | 1.02377681 | 1200.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 12:18:36.71 | 85 | 0
DE.20170527.53184.fits | science | 349.99316666666664 | -5.16575 | Feige 110 | 830G | LongMirr | 1,1 | 57900.615484 | 1.42505162 | 45.0 | 8099.98291016 | SINGLE:B | OG550 | Off | 2017-05-27 | 14:46:24.88 | 93 | 0
##########################################################
Setup B
dispname: 830G
decker: LongMirr
binning: 1,1
dispangle: 8399.93554688
amp: SINGLE:B
filter1: OG550
#---------------------------------------------------------
filename | frametype | ra | dec | target | dispname | decker | binning | mjd | airmass | exptime | dispangle | amp | filter1 | lampstat01 | dateobs | utc | frameno | calib
d0914_0002.fits.gz | bias | 299.99999999999994 | 80.0 | unknown | 830G | None | 1,1 | 58010.07499 | 1.0153979 | 1.0 | 7499.97998047 | SINGLE:B | R | Off | 2017-09-14 | 01:48:05.53 | 2 | 1
d0914_0011.fits.gz | arc,tilt | 57.99999999999999 | 45.0 | unknown | 830G | LongMirr | 1,1 | 58010.135443 | 1.41291034 | 1.0 | 8399.93554688 | SINGLE:B | OG550 | Kr Xe Ar Ne | 2017-09-14 | 03:15:07.98 | 11 | 1
d0914_0013.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | unknown | 830G | LongMirr | 1,1 | 58010.137123 | 1.41291034 | 6.0 | 8399.93554688 | SINGLE:B | OG550 | Qz | 2017-09-14 | 03:17:33.43 | 13 | 1
d0914_0014.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | unknown | 830G | LongMirr | 1,1 | 58010.138113 | 1.41291034 | 6.0 | 8399.93554688 | SINGLE:B | OG550 | Qz | 2017-09-14 | 03:18:59.03 | 14 | 1
d0914_0015.fits.gz | pixelflat,illumflat,trace | 57.99999999999999 | 45.0 | unknown | 830G | LongMirr | 1,1 | 58010.138969 | 1.41291034 | 6.0 | 8399.93554688 | SINGLE:B | OG550 | Qz | 2017-09-14 | 03:20:13.93 | 15 | 1
d0914_0036.fits.gz | science | 11.389749999999998 | 9.032555555555556 | PSOJ011p09_OFF | 830G | LongMirr | 1,1 | 58010.48501 | 1.01790951 | 1200.0 | 8399.93554688 | SINGLE:B | OG550 | Off | 2017-09-14 | 11:38:32.60 | 36 | 1
d0914_0037.fits.gz | science | 11.390166666666666 | 9.033277777777778 | PSOJ011p09_OFF | 830G | LongMirr | 1,1 | 58010.499726 | 1.02369591 | 1200.0 | 8399.93554688 | SINGLE:B | OG550 | Off | 2017-09-14 | 11:59:43.61 | 37 | 1
d0914_0038.fits.gz | science | 11.389333333333331 | 9.031833333333335 | PSOJ011p09_OFF | 830G | LongMirr | 1,1 | 58010.514673 | 1.0383931 | 1200.0 | 8399.93554688 | SINGLE:B | OG550 | Off | 2017-09-14 | 12:21:14.62 | 38 | 1
d0914_0047.fits.gz | science | 76.37762499999998 | 52.83063888888889 | G191B2B | 830G | LongMirr | 1,1 | 58010.641261 | 1.19898553 | 60.0 | 8399.93554688 | SINGLE:B | OG550 | Off | 2017-09-14 | 15:23:31.06 | 47 | 1
##end
This sorted file contains two configurations, A and B.
The “setup block” (the information between the ###... and
#--... lines) describes the instrument configuration specific to
this setup as determined using the relevant metadata. Each “setup
block” is followed by a table listing the files and relevant metadata
for all files matched to that instrument configuration. The data
provided is specific to each instrument, as defined by, e.g.,
pypeit_file_keys().
The sorted file is only provided as a means of assessing the
automated setup identification and file sorting, and we encourage you
to briefly review the results. You may recognize that you are missing
calibrations or you may be surprised to see more configurations than
you were expecting. In the latter case, you can help us improve the
automated procedures in PypeIt by submitting an issue on GitHub
(Submit an issue) and including the sorted file in the issue
description.
Importantly, you should use the sorted file to decide which
configuration (as selected by its letter) you wish to reduce. Also
note that PypeIt cannot interpret any edits you make to this
file; all user-level edits to the frame-typing, association of frames
with given configurations, etc., must be done via the PypeIt Reduction File.
3. Second execution: Write the pypeit file for one or more setups
Provided you are happy with the sorted file, you should execute
pypeit_setup a second time with the –cfg_split (shortcut
-c) option. This will generate one or more sub-folders and populate
each with a PypeIt Reduction File. Some example uses of the -c
option are:
-c A: This will generate one folder+file for the chosen configuration
-c A,C: This will generate one folder+file for each input configuration
-c all: This will generate folders+files for all configurations
An example execution that only produces the PypeIt Reduction File for the A configuration is:
pypeit_setup -r path_to_your_raw_data/LB -s keck_lris_blue -c A
This example will generate a new folder named keck_lris_blue_A
and within it will be a file named keck_lris_blue_A.pypeit.
4. Edit the pypeit file
You will likely need to edit your PypeIt Reduction File. This includes adding any desired changes to the Parameter Block, adding/removing files from the Data Block, correcting any erroneous frametypes (see Frame Types), specifying any desired association between science and calibration frames (see Calibration Groups), which science frames to combine (see Combining Science Exposures), and which frames to use as backgrounds and/or part of an AB offset sequence (see A-B image differencing).
Warning
Any execution of run_pypeit will crash if your PypeIt Reduction File
includes entries with None frame types defined. You must either remove
or edit those entries in the pypeit file by-hand after running
pypeit_setup.
Options
Consider your need for the following additional options:
-b option
You should consider using the -b option if you need to specify:
the calibration frames that should be used with each science frame (the same can be achieved by dividing your dataset into multiple
.pypeitfiles that all use the same calibrations),groups of science frames that should be combined, and/or
groups of frames that should be treated as background frames in an on-off (e.g., ABBA) observation sequence.
This simply adds the relevant columns to the PypeIt Reduction File that you must
edit by hand: calib, comb_id, and bkg_id. These columns are added
by default for most near-IR spectrographs. See Calibrations and
A-B image differencing for the syntax used for the data in these columns and
how PypeIt uses them; see also a worked example in the Gemini-GNIRS HOWTO.
-m option
If you wish to include a column where you can include
input for Manual Extraction, use the -m option.