Spec1D Output

Overview 

A primary data product for PypeIt are 1D, calibrated spectra for extracted sources. The most fundamental spectrum may be described by two arrays: flux, wavelength (in vacuum). These together with an error array are the minimal output for even Quick-Look Reductions. There are, however, several methods of extraction, calibration, etc. that yield various data products. Additionally, a .txt file with Extraction Information for each extracted 1D spectrum is also produced.

Naming 

File

The 1D spectra files have names like:

spec1d_b27-J1217p3905_KASTb_2015May20T045733.560.fits

See here for a description of the naming convention.

Objects

Each object is named by its:

spatial position (pixel number) on the reduced image [SPAT]

the slit position (pixel number) on the reduced image [SLIT]

the detector number [DET]

For example:

SPAT0176-SLIT0185-DET01

Extraction 

Because there are several modes of extraction in PypeIt, there may be multiple outputs of the spectral arrays. These are then prefixed by the extraction mode.

Extraction Mode	Description
BOX	Top-hat extraction around the trace. The precise window used is defined by the BOXCAR_APERTURE, in pixels.
OPT	Standard Horne algorithm for extraction using the fitted spatial profile. An estimate of this profile is given by OBJ_FWHM

Therefore, the integrated counts for a boxcar extraction are given by the BOX_COUNTS array with inverse variance BOX_COUNTS_IVAR.

pypeit_show_1dspec 

The spectra may be viewed with the pypeit_show_1dspec script. This script can be used to view both 1D extractions from a basic run of the reductions or 1D coadds.

The script usage can be displayed by calling the script with the -h option:

$ pypeit_show_1dspec -h
usage: pypeit_show_1dspec [-h] [-v VERBOSITY] [--log_file LOG_FILE]
                          [--log_level LOG_LEVEL] [--list] [--exten EXTEN |
                          --obj OBJ] [--extract {BOX,OPT}] [--flux] [-m]
                          file

Show a 1D spectrum

positional arguments:
  file                  PypeIt spec1d file (this script does not work with
                        coadd_1dspec output spectra).

options:
  -h, --help            show this help message and exit
  -v, --verbosity VERBOSITY
                        Verbosity level, which must be 0, 1, or 2. Level 0
                        includes warning and error messages, level 1 adds
                        informational messages, and level 2 adds debugging
                        messages and the calling sequence. (default: 2)
  --log_file LOG_FILE   Name for the log file. If set to "default", a default
                        name is used. If None, a log file is not produced.
                        (default: None)
  --log_level LOG_LEVEL
                        Verbosity level for the log file. If a log file is
                        produce and this is None, the file log will match the
                        console stream log. (default: None)
  --list                Instead of plotting any spectra, simply list the
                        extensions with spectra (default: False)
  --exten EXTEN         Number of the extension to plot (default: 1)
  --obj OBJ             Extension (object) name to plot, e.g.
                        SPAT0424-SLIT0000-DET01 (default: None)
  --extract {BOX,OPT}   Method used to extract the spectrum (default: OPT)
  --flux                Show the flux-calibrated spectrum (if available)
                        (default: False)
  -m, --unmasked        Only show the unmasked data. (default: True)

Here is a typical call:

pypeit_show_1dspec Science/spec1d_b27-J1217p3905_KASTb_2015May20T045733.560.fits --exten 1

Warning

If you get an obscure error when executing the above command, it may be that you’re trying to view a file created by a previous version of PypeIt. As the code develops, we sometimes change the datamodel of different output files, which are often not backward compatible. If you run into this error, try reverting to the relevant PypeIt version (the version used to create the file is typically written to the VERSPYP header keyword) or re-reduce the data with the new PypeIt version.

Interacting with the display

The spec1d mode is a mode enabled by ginga to interact with the spectrum plot. To get a listing of the key-bindings in the ginga window, type ‘h’. If that doesn’t work, or if the key bindings themselves don’t seem to work, you may have exited the mode. To restart the mode, hit the space-bar and then the 1 key. If you’re still having trouble, please submit an issue via GitHub or post a question to the Users Slack.

The key-bindings below are copied from the relevant docstring; if anything is different from the dialog you get when pressing ‘h’ in ginga, you should defer to what is shown there. The key-bindigs are as follows:

Enter the mode

Space, then “1”

Exit the mode

Esc

Mouse/trackpad bindings in mode

Shift + left click : set pan position
middle click : set pan position
scroll : zoom in/out
ctrl + scroll : zoom in/out X axis only
shift + scroll : zoom in/out Y axis only (On MacOS, trackpad only)
alt + scroll(mouse) : zoom in/out Y axis only (Option key on Macs)
alt + scroll : zoom in/out at cursor

Zooming

equals : zoom in one zoom level
ctrl + equals : zoom in X axis one zoom level
plus (shift + equals) : zoom in Y axis one zoom level
minus : zoom out one zoom level
ctrl + minus : zoom out X axis one zoom level
underscore (shift + minus): zoom out Y axis one zoom level
9 : zoom out maintaining cursor position
ctrl + 9 : zoom out X axis maintaining cursor position
left paren (shift + 9): zoom out Y axis maintaining cursor position
0 : zoom in maintaining cursor position
ctrl + 0 : zoom in X axis maintaining cursor position
right paren (shift + 0): zoom in Y axis maintaining cursor position
backquote : zoom X and Y axes to fit window
1 : zoom X axis only to fit window
2 : zoom Y axis only to fit window
k : set lower X range to X value at cursor
l : set upper X range to X value at cursor
K : set lower Y range to Y value at cursor
L : set upper Y range to Y value at cursor

Panning

left arrow : pan left
right arrow : pan right
up arrow : pan up
down arrow : pan down

Regions

[ : mark lower X boundary of region
] : mark upper X boundary of region
backslash : clear region
singlequote : (zoom) set X range to region
f : fit a Gaussian to the region and show result
c : clear fit result

Extraction Information 

A .txt file with the same name as the 1D spectra File is also produced by PypeIt. This file lists the main extraction information for each 1D spectrum. It looks like:

| slit |                    name | spat_pixpos | spat_fracpos | box_width | opt_fwhm |   s2n | wv_rms |
|   69 | SPAT0071-SLIT0069-DET02 |        70.9 |        0.504 |      3.00 |    1.911 |  7.06 |  0.025 |
|  178 | SPAT0186-SLIT0178-DET02 |       186.1 |        0.570 |      3.00 |    1.264 |  2.60 |  0.020 |
|  275 | SPAT0271-SLIT0275-DET02 |       270.5 |        0.434 |      3.00 |    1.317 |  1.89 |  0.021 |
|  371 | SPAT0383-SLIT0371-DET02 |       383.3 |        0.578 |      3.00 |    0.425 |  2.98 |  0.022 |
|  469 | SPAT0461-SLIT0469-DET02 |       461.0 |        0.392 |      3.00 |    0.873 |  0.55 |  0.026 |

where:

slit is the slit position in pixels on the reduced image;
name is the object name (see Objects);
spat_pixpos is the object spatial position in pixels on the reduced image;
spat_fracpos is the fractional location of the object on the slit;
box_width is the width in arcsec of the boxcar;
opt_fwhm is the spatial FWHM in arcsec of the optimally extracted object;
s2n is the Signal-to-Noise ratio (SNR) of the optimally extracted object. If optimal extraction is not performed, the reported SNR is for the boxcar extracted object;
wv_rms is the RMS in pixels of the wavelength solution.

In addition, if reducing Keck DEIMOS or Keck MOSFIRE data and slit-mask design matching is performed, maskdef_id, objname, objra, objdec, and maskdef_extract are also provided for each spectrum (see RA, Dec and object name assignment to 1D extracted spectra).

specutils Interface 

We provide an interface to the specutils package to facilitate use of PypeIt output spectra with code that uses, e.g., specutils.Spectrum. Use of this interface requires you to install the specutils package. This can be done using PypeIt’s Optional Dependencies, or simply by directly installing specutils within the same python environment as you’ve installed PypeIt.

Usage

To read a PypeIt spec1d file using specutils:
```
from pypeit.specutils import SpectrumList
spec = SpectrumList.read(spec1d_file)
```
where spec1d_file is the relative or absolute path to a PypeIt spec1d file. You must always use a SpectrumList to read a spec1d file, even if the file only contains one spectrum. The spec1d loader provides PypeIt-specific options that enable you to specify the type of extraction used and whether or not to use the flux-calibrated spectrum; see pypeit.specutils.pypeit_loaders.pypeit_spec1d_loader(). By default, optimal extraction takes precedence over boxcar extraction, and flux-calibrated data take precedence over uncalibrated counts.
To read a PypeIt pypeit.onespec.OneSpec file:
```
from pypeit.specutils import Spectrum
spec = Spectrum.read(onespec_file)
```
where onespec_file is the relative or absolute path to a PypeIt pypeit.onespec.OneSpec file. For these files, you can use either Spectrum or SpectrumList to read the file, but (by definition) the result of using SpectrumList will just be a list of one Spectrum object. The pypeit.onespec.OneSpec loader provides a PypeIt-specific option that enables you to select the uniform grid wavelength vector, instead of the contribution-weighted wavelengths; see pypeit.specutils.pypeit_loaders.pypeit_onespec_loader().

Note

Importing Spectrum and SpectrumList are shown as coming from the pypeit.specutils module, but the objects themselves are identical to the specutils objects. The reason they are imported from within PypeIt is that, under the hood, the import also “registers” the PypeIt-specific loaders with the relevant specutils module. This circumvents the need to place any pypeit specific code in a ~/.specutils directory (as recommended here) and keeps the import statement to one line. That is,

from pypeit.specutils import Spectrum

is really just shorthand for

from specutils import Spectrum
from pypeit.specutils import pypeit_loaders

Examples

In addition to the pypeit_show_1dspec GUI, these specutils loaders allow you to interact with your spectra using jdaviz. jdaviz is intended to be run within a jupyter notebook, or directly from the command line as a stand-alone application. In order to use jdaviz with PypeIt 1D output files, jdaviz must currently be run from within a jupyter notebook, so that we can properly register the PypeIt-specific loaders with specutils.

Here is an example of how to display a PypeIt spec1d file in the jdaviz GUI within a jupyter notebook:

from pypeit.specutils import SpectrumList
from jdaviz import Specviz

file = 'my_spec1d_file.fits'
spec = SpectrumList.read(file)

specviz = Specviz()
specviz.load_data(spec)
specviz.show()

Current Data Model 

Internally, the spectrum for a single object is held by the SpecObj class. Here is its datamodel, which is written as an astropy.io.fits.BinTableHDU in the spec1d* fits file with this Naming. In addition, one DetectorContainer is written to a fits extension — named, e.g., DET01-DETECTOR — for each detector with at least one spectrum extracted.

All wavelengths are in vacuum.

Multiple SpecObj objects are held internally by a SpecObjs object.

Version: 1.1.14

Obj Key	Obj Type	Array Type	Description
`BOX_CHI2`	ndarray	float	Reduced chi2 of the model fit for this spectral pixel
`BOX_COUNTS`	ndarray	float	Boxcar flux (counts)
`BOX_COUNTS_IVAR`	ndarray	float	Inverse variance of optimally extracted flux using modelivar image (counts^2)
`BOX_COUNTS_NIVAR`	ndarray	float	Boxcar extracted noise variance, sky+read noise only (counts^2)
`BOX_COUNTS_SIG`	ndarray	float	Boxcar extracted noise from IVAR (counts)
`BOX_COUNTS_SIG_DET`	ndarray	float	Boxcar extracted detector noise (counts)
`BOX_COUNTS_SKY`	ndarray	float	Boxcar extracted sky (counts)
`BOX_FLAM`	ndarray	float	Boxcar flux (erg/s/cm^2/Ang)
`BOX_FLAM_IVAR`	ndarray	float	Boxcar flux inverse variance (1e-17 erg/s/cm^2/Ang)^-2
`BOX_FLAM_SIG`	ndarray	float	Boxcar flux uncertainty (1e-17 erg/s/cm^2/Ang)
`BOX_FLAT`	ndarray	float	Boxcar extracted flatfield spectrum, normalized to the peak value.
`BOX_FRAC_USE`	ndarray	float	Fraction of pixels in the object profile subimage used for this extraction
`BOX_FWHM`	ndarray	float	Spectral FWHM (in Angstroms) at every pixel of the boxcar extracted flux.
`BOX_MASK`	ndarray	bool	Mask for boxcar extracted flux. True=good
`BOX_NPIX`	ndarray	float	Number of pixels used for the boxcar extraction; can be fractional
`BOX_R_ASEC`	float		Size of boxcar radius (arcsec)
`BOX_R_PIX`	float		Size of boxcar radius (pixels)
`BOX_WAVE`	ndarray	float	Boxcar Wavelengths in vacuum (Angstroms)
`DEC`	float		Declination (J2000) decimal degree
`DET`	str		A string identifier for the reduced detector or mosaic.
`DETECTOR`	DetectorContainer, Mosaic		Object with the detector or mosaic metadata
`ECH_FRACPOS`	float, floating		Synced echelle fractional location of the object on the slit
`ECH_FRACPOS_ID`	int, integer		Echelle fractional location of the object on the slit multiplied by 1000 used as a unique identifier for the naming model
`ECH_NAME`	str		Name of the object for echelle data. Same as NAME above but order numbers are omitted giving a unique name per object.
`ECH_OBJID`	int, integer		Object ID for echelle data. Each object is given an index in the order it appears increasing from from left to right. These are one based.
`ECH_ORDER`	int, integer		Physical echelle order
`ECH_ORDERINDX`	int, integer		Order indx, analogous to SLITID for echelle. Zero based.
`FLEX_SHIFT_GLOBAL`	float		Global shift of the spectrum to correct for spectralflexure (pixels). This is based on the sky spectrum atthe center of the slit
`FLEX_SHIFT_LOCAL`	float		Local shift of the spectrum to correct for spectralflexure (pixels). This should be a small correction tothe global value, and is based on the sky spectrumextracted near the object
`FLEX_SHIFT_TOTAL`	float		Total shift of the spectrum to correct for spectralflexure (pixels). This is the sum of the global andlocal FLEX_SHIFT
`FWHM`	float		Spatial FWHM of the object (pixels)
`FWHMFIT`	ndarray	float	Spatial FWHM across the detector (pixels)
`MASKDEF_EXTRACT`	bool		Boolean indicating if this is a forced extraction at the expected location from slitmask design.
`MASKDEF_ID`	int, integer		Slitmask definition ID
`MASKDEF_OBJMAG`	float		Magnitude of the object from the slitmask definition
`MASKDEF_OBJMAG_BAND`	str		Magnitude band of the object from the slitmask definition
`MASKDEF_OBJNAME`	str		Name of the object from the slitmask definition
`NAME`	str		Name of the object following the naming model
`OBJID`	int, integer		Object ID for multislit data. Each object is given an index for the slit it appears increasing from from left to right. These are one based.
`OBJTYPE`	str		Object type (e.g., standard, science)
`OPT_CHI2`	ndarray	float	Reduced chi2 of the model fit for this spectral pixel
`OPT_COUNTS`	ndarray	float	Optimal flux (counts)
`OPT_COUNTS_IVAR`	ndarray	float	Inverse variance of optimally extracted flux using modelivar image (counts^2)
`OPT_COUNTS_NIVAR`	ndarray	float	Optimally extracted noise variance, sky+read noise only (counts^2)
`OPT_COUNTS_SIG`	ndarray	float	Optimally extracted noise from IVAR (counts)
`OPT_COUNTS_SIG_DET`	ndarray	float	Optimally extracted detector noise (counts)
`OPT_COUNTS_SKY`	ndarray	float	Optimally extracted sky (counts)
`OPT_FLAM`	ndarray	float	Optimal flux (1e-17 erg/s/cm^2/Ang)
`OPT_FLAM_IVAR`	ndarray	float	Optimal flux inverse variance (1e-17 erg/s/cm^2/Ang)^-2
`OPT_FLAM_SIG`	ndarray	float	Optimal flux uncertainty (1e-17 erg/s/cm^2/Ang)
`OPT_FLAT`	ndarray	float	Optimally extracted flatfield spectrum, normalised to the peak value.
`OPT_FRAC_USE`	ndarray	float	Fraction of pixels in the object profile subimage used for this extraction
`OPT_FWHM`	ndarray	float	Spectral FWHM (in Angstroms) at every pixel of the optimally extracted flux.
`OPT_MASK`	ndarray	bool	Mask for optimally extracted flux. True=good
`OPT_WAVE`	ndarray	float	Optimal Wavelengths in vacuum (Angstroms)
`PYPELINE`	str		Name of the PypeIt pipeline mode
`PYP_SPEC`	str		PypeIt spectrograph name
`RA`	float		Right Ascension (J2000) decimal degree
`S2N`	float		Median signal to noise ratio of the extracted spectrum(OPT if available, otherwise BOX)
`SLITID`	int, integer		PypeIt slit ID (aka SPAT_ID).
`SPAT_FRACPOS`	float, floating		Fractional location of the object on the slit
`SPAT_FWHM`	float		Spatial FWHM of the object (arcsec)
`SPAT_PIXPOS`	float, floating		Spatial location of the trace on detector (pixel) at half-way
`SPAT_PIXPOS_ID`	int, integer		Nearest integer spatial location of the trace on detector (pixel) at half-way used as a unique identifier for the naming model
`SPEC_DET`	ndarray	integer	Array of detector indices for each pixel in the spectral direction. This is only available for mosaic reductions.
`TRACE_SPAT`	ndarray	float	Object trace along the spec (spatial pixel)
`VEL_CORR`	float		Relativistic velocity correction for wavelengths
`VEL_TYPE`	str		Type of heliocentric correction (if any)
`WAVE_RMS`	float, floating		RMS (pix) for the wavelength solution for this slit.
`ech_snr`	float, floating		Median S/N of the echelle of the spectrum
`hand_extract_flag`	bool		Boolean indicating if this is a forced extraction at the location provided by the user.
`maskwidth`	float, floating		Size (in units of spatial fwhm) of the region used for local sky subtraction
`sign`	float		Sign of the object profile (+1 or -1). + is a positive profile above the sky background.
`smash_peakflux`	float		Peak value of the spectral direction collapsed spatial profile
`smash_snr`	float		Peak S/N ratio of the spectral direction collapsed patial profile
`trace_spec`	ndarray	int,numpy.integer	Array of pixels along the spectral direction