Coadd 2D Spectra
Overview
This document describes how to combine the 2D spectra from multiple exposures. For a worked examples, see Coadd2D HOWTO.
Coadding must be done outside of the data reduction pipeline (run_pypeit); i.e., PypeIt will not coadd your spectra as part of the data reduction process, although it can combine (without weighting) multiple exposures during reductions (see Combining Science Exposures).
Note
Because the flux of the single reduced science frames is expressed in counts,
coadding frames with different exposure times is not recommended. If the user still
wishes to do so, the fluxes of the individual frames are rescaled by the median
exposure time. For example, if we have four frames with exposure times of
1800, 1800, 1800, and 1200 seconds, the exposure
time of the coadded frame will be:
coadd_exptime = np.percentile([1800,1800,1800,1200],50, method='higher')
and the flux of the individual frames will be rescaled by:
rescale_factor = coadd_exptime / exptime
where exptime is the exposure time of the individual frames. coadd_exptime is saved
in the header of the coadded frame as ALLSPEC2D_EFFECTIVE_EXPTIME, so that the user can
easily convert the flux of the coadded frame from counts to counts/s.
Note, also, that the combination (without weighting) of multiple exposures during main reduction (i.e, Combining Science Exposures) does not perform this rescaling.
coadd2d file
The pypeit_coadd_2dspec script requires an input file to guide the process. The format of this type of Input File Format includes a Parameter Block (required) and a Data Block (required).
Here is an example for keck_lris_blue:
# User-defined execution parameters
[rdx]
spectrograph = keck_lris_blue
detnum = 2
[reduce]
[[findobj]]
snr_thresh=5.0
# Data block
spec2d read
path /path/to/your/Science/folder
filename
spec2d_b170320_2083-c17_60L._LRISb_2017Mar20T055336.211.fits
spec2d_b170320_2090-c17_60L._LRISb_2017Mar20T082144.525.fits
spec2d_b170320_2084-c17_60L._LRISb_2017Mar20T062414.630.fits
spec2d_b170320_2091-c17_60L._LRISb_2017Mar20T085223.894.fits
spec2d end
The opening Parameter Block sets information
and parameters for the reduction steps.
The spectrograph name is required.
See Coadd2DPar Keywords for a list of the options specific to 2D coadds, including Manual Extraction.
The Data Block always begins and ends with spec2d read and spec2d end, respectively.
It (optionally) provides the path
to the Spec2D Output files.
It then includes a (one column) table which
is a simple list of the Spec2D Output files.
Note
The inclusion of the line filename in the example above is required.
This is the line providing the columns names for the data to follow and must
be present, even if there is only one column.
You may also include file-specific options in the Data Block as additional columns. See Data Block for its formatting.
Setup script
Similar to pypeit_setup, we provide a script that helps you construct the
required input file. In this case, pypeit_setup_coadd2d helps you build the
required .coadd2d file.
The script usage can be displayed by calling the script with the
-h option:
$ pypeit_setup_coadd2d -h
usage: pypeit_setup_coadd2d [-h] [-v VERBOSITY] [--log_file LOG_FILE]
[--log_level LOG_LEVEL] (-f PYPEIT_FILE |
-d SCIENCE_DIR [SCIENCE_DIR ...]) [--keep_par]
[--obj OBJ [OBJ ...]] [--det DET [DET ...]]
[--only_slits ONLY_SLITS [ONLY_SLITS ...]]
[--exclude_slits EXCLUDE_SLITS [EXCLUDE_SLITS ...]]
[--spat_toler SPAT_TOLER] [--offsets OFFSETS]
[--weights WEIGHTS]
[--spec_samp_fact SPEC_SAMP_FACT]
[--spat_samp_fact SPAT_SAMP_FACT]
Prepare a configuration file for performing 2D coadds
options:
-h, --help show this help message and exit
-v, --verbosity VERBOSITY
Verbosity level, which must be 0, 1, or 2. Level 0
includes warning and error messages, level 1 adds
informational messages, and level 2 adds debugging
messages and the calling sequence. (default: 2)
--log_file LOG_FILE Name for the log file. If set to "default", a default
name is used. If None, a log file is not produced.
(default: None)
--log_level LOG_LEVEL
Verbosity level for the log file. If a log file is
produce and this is None, the file log will match the
console stream log. (default: None)
-f, --pypeit_file PYPEIT_FILE
PypeIt reduction file (default: None)
-d, --science_dir SCIENCE_DIR [SCIENCE_DIR ...]
One or more directories with spec2d files to stack (use
wildcard to specify multiple directories). (default:
None)
--keep_par Propagate all parameters from the pypeit file to the
coadd2d file(s). If not set, only the required
parameters and their default values are included in the
output file(s). (default: True)
--obj OBJ [OBJ ...] Limit the coadd2d files created to observations of the
specified target. If not provided, a coadd2D file is
written for each target found in the science directory.
The target names are included in the PypeIt spec2d file
names.For example, the target for spec2d file "spec2d_cN
20170331S0216-pisco_GNIRS_20170331T085412.181.fits" is
"pisco". (default: None)
--det DET [DET ...] A space-separated set of detectors or detector mosaics
to coadd. By default, *all* detectors or default mosaics
for this instrument will be coadded. Detectors in a
mosaic must be a mosaic "allowed" by PypeIt and should
be provided as comma-separated integers (with no
spaces). For example, to separately coadd detectors 1
and 5 for Keck/DEIMOS, you would use --det 1 5; to coadd
mosaics made up of detectors 1,5 and 3,7, you would use
--det 1,5 3,7 (default: None)
--only_slits ONLY_SLITS [ONLY_SLITS ...]
A space-separated set of slits to coadd. Example syntax
for argument is DET01:175,DET02:205 or MSC02:2234. If
not provided, all slits are coadded. If both --det and
--only_slits are provided, --det will be ignored. This
and --exclude_slits are mutually exclusive. If both are
provided, --only_slits takes precedence. (default: None)
--exclude_slits EXCLUDE_SLITS [EXCLUDE_SLITS ...]
A space-separated set of slits to exclude in the
coaddition. This and --only_slits are mutually
exclusive. If both are provided, --only_slits takes
precedence. (default: None)
--spat_toler SPAT_TOLER
Desired tolerance in spatial pixel used to identify
slits in different exposures. If not provided, the
default value for the specific instrument/configuration
is used. (default: None)
--offsets OFFSETS Spatial offsets to apply to each image; see the
[coadd2d][offsets] parameter. Options are restricted
here to either maskdef_offsets or auto. If not
specified, the (spectrograph-specific) default is used.
Other options exist but must be entered by directly
editing the coadd2d file. (default: None)
--weights WEIGHTS Weights used to coadd images; see the [coadd2d][weights]
parameter. Options are restricted here to either uniform
or auto. If not specified, the (spectrograph-specific)
default is used. Other options exist but must be entered
by directly editing the coadd2d file. (default: None)
--spec_samp_fact SPEC_SAMP_FACT
Make the wavelength grid finer (spec_samp_fact < 1.0) or
coarser (spec_samp_fact > 1.0) by this sampling factor,
i.e. units of spec_samp_fact are pixels. (default: 1.0)
--spat_samp_fact SPAT_SAMP_FACT
Make the spatial grid finer (spat_samp_fact < 1.0) or
coarser (spat_samp_fact > 1.0) by this sampling factor,
i.e. units of spat_samp_fact are pixels. (default: 1.0)
For example, assuming you have already executed run_pypeit on your data using a pypeit file, you can construct the default coadd2d file(s) using:
pypeit_setup_coadd2d -f keck_lris_A.pypeit
This will produce one .coadd2d file per unique target in the pypeit file
with associated spec2d files in the output science directory. The script
provides additional options that allow you to select specific objects/targets,
specify the method used to set the offsets and/or weights, specify the detectors
to coadd, and/or specify the slits that should be coadded.
pypeit_coadd_2dspec
Once you have prepared a .coadd2d file, the primary script to execute is
pypeit_coadd_2dspec.
The script usage can be displayed by calling the script with the
-h option:
$ pypeit_coadd_2dspec -h
usage: pypeit_coadd_2dspec [-h] [-v VERBOSITY] [--log_file LOG_FILE]
[--log_level LOG_LEVEL] [--show] [--debug_offsets]
[--peaks] [--basename BASENAME] [--debug]
coadd2d_file
Coadd 2D spectra produced by PypeIt
positional arguments:
coadd2d_file File to guide 2d coadds
options:
-h, --help show this help message and exit
-v, --verbosity VERBOSITY
Verbosity level, which must be 0, 1, or 2. Level 0
includes warning and error messages, level 1 adds
informational messages, and level 2 adds debugging
messages and the calling sequence. (default: 2)
--log_file LOG_FILE Name for the log file. If set to "default", a default
name is used. If None, a log file is not produced.
(default: default)
--log_level LOG_LEVEL
Verbosity level for the log file. If a log file is
produce and this is None, the file log will match the
console stream log. (default: None)
--show Show the reduction steps. Equivalent to the -s option
when running pypeit. (default: False)
--debug_offsets Show QA plots useful for debugging automatic offset
determination (default: False)
--peaks Show the peaks found by the object finding algorithm.
(default: False)
--basename BASENAME Basename of files to save the parameters, spec1d, and
spec2d (default: None)
--debug show debug plots? (default: False)
options
Here are commonly used options:
–show
Show a series of matplotlib plots to the screen.
–basename
Provides the basename for the spec1d and spec2d files. If not provided, defaults to a portion of the input spec2d filenames.
–debug
Provides additional debugging diagnostic plots compared to using --show.
run
Then run the script:
pypeit_coadd_2dspec FRB190711_XS.coadd2d --show
The parameters that guide the coadd process are also written to disk for your
records. The default location is *_coadd2d.par, where the wildcard should
contain the frames included and the target/object name. You can choose another
location by modifying –basename.
For worked examples, see Coadd2D HOWTO.
Known Issues and Workarounds
Input frames with variable object spatial position and signal-to-noise
In the case where an object drifts spatially along the slit between exposures
(or purposfully moves through dither patterns), pypeit_coadd_2dspec
computes the offset between the object trace in each of the input images with
respect to the first image. This is controlled through the parameters:
[coadd2d]
offsets = auto
spat_toler = 5
Because pypeit_coadd_2dspec rebins the input input images onto a common
grid for the coaddition, the default execution is to refind all objects in each
of the input frames and target the brightest one as the object of interest.
There are two primary failure modes for (typically longslit) data sets in this
category when offsets = auto:
The presence of intermittent clouds during the observation sequence causes one or more of the input frames to have no object over the
snr_threshdetection threshold. Even if the user perfomed a Manual Extraction, the code will attempt to refind objects on the slit de nouveau. If none can be found, the code will crash with an error.The object of interest is not the brightest object along the slit in every (or any) frame. This may occur when tracking solar system objects and various field stars cross the slit, or if the target of interest is in a crowded field and multiple objects appear on the slit. This failure mode, more common with longslit than multislit observations, will not cause the script to crash, but will not result in the desired coaddition of the desired object.
It should be noted that if offsets are computed by hand and included in the
.coadd2d file parameter block or are zero (e.g., because of stable
guiding), then these failure modes are not present.
The workaround for automatic offset computation is to include the SPAT
identifiction code of the desired object for each of the input frames in the
.coadd2d file parameter block. For instance, if you were to query the
input frame spec1d_*.txt files like this:
$ cat Science/spec1d_20251027.0{210..212}*txt 10:57:07
| slit | name | obj_id | spat_pixpos | spat_fracpos | box_width | opt_fwhm | s2n | manual_extract | wv_rms |
| 241 | SPAT0105-SLIT0241-DET01 | 105 | 105.3 | 0.214 | 3.80 | 1.863 | 2.64 | False | 0.120 |
| 241 | SPAT0171-SLIT0241-DET01 | 171 | 171.2 | 0.353 | 3.80 | 1.555 | 3.90 | False | 0.120 |
| 241 | SPAT0244-SLIT0241-DET01 | 244 | 244.0 | 0.506 | 3.80 | 1.081 | 2.20 | True | 0.120 |
| 241 | SPAT0276-SLIT0241-DET01 | 276 | 276.4 | 0.576 | 3.80 | 2.010 | 8.85 | False | 0.120 |
| 241 | SPAT0307-SLIT0241-DET01 | 307 | 306.9 | 0.641 | 3.80 | 1.465 | 4.51 | False | 0.120 |
| 241 | SPAT0375-SLIT0241-DET01 | 375 | 375.4 | 0.785 | 3.80 | 1.777 | 6.19 | False | 0.120 |
| slit | name | obj_id | spat_pixpos | spat_fracpos | box_width | opt_fwhm | s2n | wv_rms |
| 241 | SPAT0243-SLIT0241-DET01 | 243 | 243.2 | 0.506 | 3.80 | 1.196 | 3.66 | 0.120 |
| 241 | SPAT0316-SLIT0241-DET01 | 316 | 315.5 | 0.658 | 3.80 | 1.568 | 2.18 | 0.120 |
| 241 | SPAT0404-SLIT0241-DET01 | 404 | 404.3 | 0.846 | 3.80 | 1.653 | 13.00 | 0.120 |
| slit | name | obj_id | spat_pixpos | spat_fracpos | box_width | opt_fwhm | s2n | wv_rms |
| 241 | SPAT0008-SLIT0241-DET01 | 8 | 8.1 | 0.009 | 3.80 | 1.660 | 1.98 | 0.120 |
| 241 | SPAT0100-SLIT0241-DET01 | 100 | 100.4 | 0.203 | 3.80 | 1.307 | 1.81 | 0.120 |
| 241 | SPAT0170-SLIT0241-DET01 | 170 | 169.9 | 0.351 | 3.80 | 1.636 | 49.83 | 0.120 |
| 241 | SPAT0241-SLIT0241-DET01 | 241 | 241.0 | 0.501 | 3.80 | 1.883 | 12.04 | 0.120 |
| 241 | SPAT0385-SLIT0241-DET01 | 385 | 384.7 | 0.805 | 3.80 | 1.559 | 2.32 | 0.120 |
and you knew you were looking for an object near the center of the slit
(SLIT0241), you would specify the user_obj_ids for this object in this
fashion:
[coadd2d]
offsets = auto
spat_toler = 5
user_obj_ids = 244, 243, 241
The offset-computation routine will now compute the transformed location of each trace in the rebinned output image, and will perform a manual extraction in order to determine the proper offsets for the coaddition. Please note that the object finding and extraction of the object in the coadded frame is unaffected by this workaround – it is entirely for the benefit of computing spatial offsets between the individual input frames for the purpose of alignment.
You will notice in the example spec1d*.txt files above, the object of
interest was the not the brightest object in any of the input frames.
Furthermore, one of the frames was affected by clouds and the object was
manually extracted prior to the coaddition attempt.
Current Coadd2D Data Model
The outputs produced by pypeit_coadd_2dspec are identical to a standard
run of run_pypeit, except that the results are places in *_coadd
directories. See Spec1D Output and Spec2D Output.