Spec1D Output

Overview 

A primary data product for PypeIt are 1D, calibrated spectra for extracted sources. The most fundamental spectrum may be described by two arrays: flux, wavelength (in vacuum). These together with an error array are the minimal output for even Quick-Look Reductions. There are, however, several methods of extraction, calibration, etc. that yield various data products. Additionally, a .txt file with Extraction Information for each extracted 1D spectrum is also produced.

Naming 

File

The 1D spectra files have names like:

spec1d_b27-J1217p3905_KASTb_2015May20T045733.560.fits

See here for a description of the naming convention.

Objects

Each object is named by its:

spatial position (pixel number) on the reduced image [SPAT]

the slit position (pixel number) on the reduced image [SLIT]

the detector number [DET]

For example:

SPAT0176-SLIT0185-DET01

Extraction 

Because there are several modes of extraction in PypeIt, there may be multiple outputs of the spectral arrays. These are then prefixed by the extraction mode.

Extraction Mode	Description
BOX	Top-hat extraction around the trace. The precise window used is defined by the BOXCAR_APERTURE, in pixels.
OPT	Standard Horne algorithm for extraction using the fitted spatial profile. An estimate of this profile is given by OBJ_FWHM

Therefore, the integrated counts for a boxcar extraction are given by the BOX_COUNTS array with inverse variance BOX_COUNTS_IVAR.

pypeit_show_1dspec 

The spectra may be viewed with the pypeit_show_1dspec script, which loads the data and launches a GUI from the linetools package.

The script usage can be displayed by calling the script with the -h option:

$ pypeit_show_1dspec -h
usage: pypeit_show_1dspec [-h] [--list] [--exten EXTEN] [--obj OBJ]
                          [--extract EXTRACT] [--flux] [-m]
                          file

Show a 1D spectrum

positional arguments:
  file               Spectral file

options:
  -h, --help         show this help message and exit
  --list             List the extensions only? (default: False)
  --exten EXTEN      FITS extension (default: 1)
  --obj OBJ          Object name in lieu of extension, e.g.
                     SPAT0424-SLIT0000-DET01 (default: None)
  --extract EXTRACT  Extraction method. Default is OPT. ['BOX', 'OPT'] (default:
                     OPT)
  --flux             Show fluxed spectrum? (default: False)
  -m, --unmasked     Only show unmasked data. (default: True)

Here is a typical call:

pypeit_show_1dspec Science/spec1d_b27-J1217p3905_KASTb_2015May20T045733.560.fits --exten 1

This should launch an XSpecGUI.

Warning

If you get an obscure error when executing the above command, it may be that you’re trying to view a file created by a previous version of PypeIt. As the code develops, we sometimes change the datamodel of different output files, which are often not backward compatible. If you run into this error, try reverting to the relevant PypeIt version (the version used to create the file is typically written to the VERSPYP header keyword) or re-reduce the data with the new PypeIt version.

Options

Here are the typical options you will use:

–list

This prints a list to the screen of all the objects extracted. An example:

EXT0000001 = SPAT0351-SLIT0000-DET01
EXT0000002 = SPAT0392-SLIT0001-DET01
EXT0000003 = SPAT0463-SLIT0003-DET01
EXT0000004 = SPAT0556-SLIT0004-DET01
EXT0000005 = SPAT0621-SLIT0005-DET01
EXT0000006 = SPAT0731-SLIT0006-DET01
EXT0000007 = SPAT0824-SLIT0007-DET01
EXT0000008 = SPAT0865-SLIT0007-DET01
EXT0000009 = SPAT0910-SLIT0008-DET01
EXT0000010 = SPAT0962-SLIT0009-DET01
EXT0000011 = SPAT0073-SLIT0000-DET02
EXT0000012 = SPAT0093-SLIT0000-DET02
EXT0000013 = SPAT0130-SLIT0001-DET02

This indicates the extension of the object with this Naming.

–exten

This is a short-cut of sorts to pull the object you want without typing in its name.

–obj

Plot this object.

–extract

Choice of Extraction method.

–flux

Show the fluxed spectrum (only if it has been fluxed!)

Extraction Information 

A .txt file with the same name as the 1D spectra File is also produced by PypeIt. This file lists the main extraction information for each 1D spectrum. It looks like:

| slit |                    name | spat_pixpos | spat_fracpos | box_width | opt_fwhm |   s2n | wv_rms |
|   69 | SPAT0071-SLIT0069-DET02 |        70.9 |        0.504 |      3.00 |    1.911 |  7.06 |  0.025 |
|  178 | SPAT0186-SLIT0178-DET02 |       186.1 |        0.570 |      3.00 |    1.264 |  2.60 |  0.020 |
|  275 | SPAT0271-SLIT0275-DET02 |       270.5 |        0.434 |      3.00 |    1.317 |  1.89 |  0.021 |
|  371 | SPAT0383-SLIT0371-DET02 |       383.3 |        0.578 |      3.00 |    0.425 |  2.98 |  0.022 |
|  469 | SPAT0461-SLIT0469-DET02 |       461.0 |        0.392 |      3.00 |    0.873 |  0.55 |  0.026 |

where:

slit is the slit position in pixels on the reduced image;
name is the object name (see Objects);
spat_pixpos is the object spatial position in pixels on the reduced image;
spat_fracpos is the fractional location of the object on the slit;
box_width is the width in arcsec of the boxcar;
opt_fwhm is the spatial FWHM in arcsec of the optimally extracted object;
s2n is the Signal-to-Noise ratio (SNR) of the optimally extracted object. If optimal extraction is not performed, the reported SNR is for the boxcar extracted object;
wv_rms is the RMS in pixels of the wavelength solution.

In addition, if reducing Keck DEIMOS or Keck MOSFIRE data and slit-mask design matching is performed, maskdef_id, objname, objra, objdec, and maskdef_extract are also provided for each spectrum (see RA, Dec and object name assignment to 1D extracted spectra).

specutils Interface 

We provide an interface to the specutils package to facilitate use of PypeIt output spectra with code that uses, e.g., specutils.Spectrum1D. Use of this interface requires you to install the specutils package. This can be done using PypeIt’s Optional Dependencies, or simply by directly installing specutils within the same python environment as you’ve installed PypeIt.

Usage

To read a PypeIt spec1d file using specutils:
```
from pypeit.specutils import SpectrumList
spec = SpectrumList.read(spec1d_file)
```
where spec1d_file is the relative or absolute path to a PypeIt spec1d file. You must always use a SpectrumList to read a spec1d file, even if the file only contains one spectrum. The spec1d loader provides PypeIt-specific options that enable you to specify the type of extraction used and whether or not to use the flux-calibrated spectrum; see pypeit.specutils.pypeit_loaders.pypeit_spec1d_loader(). By default, optimal extraction takes precedence over boxcar extraction, and flux-calibrated data take precedence over uncalibrated counts.
To read a PypeIt pypeit.onespec.OneSpec file:
```
from pypeit.specutils import Spectrum1D
spec = Spectrum1D.read(onespec_file)
```
where onespec_file is the relative or absolute path to a PypeIt pypeit.onespec.OneSpec file. For these files, you can use either Spectrum1D or SpectrumList to read the file, but (by definition) the result of using SpectrumList will just be a list of one Spectrum1D object. The pypeit.onespec.OneSpec loader provides a PypeIt-specific option that enables you to select the uniform grid wavelength vector, instead of the contribution-weighted wavelengths; see pypeit.specutils.pypeit_loaders.pypeit_onespec_loader().

Note

Importing Spectrum1D and SpectrumList are shown as coming from the pypeit.specutils module, but the objects themselves are identical to the specutils objects. The reason they are imported from within PypeIt is that, under the hood, the import also “registers” the PypeIt-specific loaders with the relevant specutils module. This circumvents the need to place any pypeit specific code in a ~/.specutils directory (as recommended here) and keeps the import statement to one line. That is,

from pypeit.specutils import Spectrum1D

is really just shorthand for

from specutils import Spectrum1D
from pypeit.specutils import pypeit_loaders

Examples

In addition to the pypeit_show_1dspec GUI, these specutils loaders allow you to interact with your spectra using jdaviz. To do so, use the following lines in a jupyter notebook (you currently must do this from within a notebook):

from pypeit.specutils import SpectrumList
from jdaviz import Specviz

file = 'my_spec1d_file.fits'
spec = SpectrumList.read(file)

specviz = Specviz()
specviz.load_spectrum(spec)
specviz.show()

Current Data Model 

Internally, the spectrum for a single object is held by the SpecObj class. Here is its datamodel, which is written as an astropy.io.fits.BinTableHDU in the spec1d* fits file with this Naming. In addition, one DetectorContainer is written to a fits extension — named, e.g., DET01-DETECTOR — for each detector with at least one spectrum extracted.

All wavelengths are in vacuum.

Multiple SpecObj objects are held internally by a SpecObjs object.

Version: 1.1.10

Obj Key	Obj Type	Array Type	Description
`BOX_CHI2`	ndarray	float	Reduced chi2 of the model fit for this spectral pixel
`BOX_COUNTS`	ndarray	float	Boxcar flux (counts)
`BOX_COUNTS_IVAR`	ndarray	float	Inverse variance of optimally extracted flux using modelivar image (counts^2)
`BOX_COUNTS_NIVAR`	ndarray	float	Boxcar extracted noise variance, sky+read noise only (counts^2)
`BOX_COUNTS_SIG`	ndarray	float	Boxcar extracted noise from IVAR (counts)
`BOX_COUNTS_SIG_DET`	ndarray	float	Boxcar extracted detector noise (counts)
`BOX_COUNTS_SKY`	ndarray	float	Boxcar extracted sky (counts)
`BOX_FLAM`	ndarray	float	Boxcar flux (erg/s/cm^2/Ang)
`BOX_FLAM_IVAR`	ndarray	float	Boxcar flux inverse variance (1e-17 erg/s/cm^2/Ang)^-2
`BOX_FLAM_SIG`	ndarray	float	Boxcar flux uncertainty (1e-17 erg/s/cm^2/Ang)
`BOX_FRAC_USE`	ndarray	float	Fraction of pixels in the object profile subimage used for this extraction
`BOX_FWHM`	ndarray	float	Spectral FWHM (in Angstroms) at every pixel of the boxcar extracted flux.
`BOX_MASK`	ndarray	bool	Mask for boxcar extracted flux. True=good
`BOX_NPIX`	ndarray	float	Number of pixels used for the boxcar extraction; can be fractional
`BOX_RADIUS`	float		Size of boxcar radius (pixels)
`BOX_WAVE`	ndarray	float	Boxcar Wavelengths in vacuum (Angstroms)
`DEC`	float		Declination (J2000) decimal degree
`DET`	str		A string identifier for the reduced detector or mosaic.
`DETECTOR`	DetectorContainer, Mosaic		Object with the detector or mosaic metadata
`ECH_FRACPOS`	float, floating		Synced echelle fractional location of the object on the slit
`ECH_NAME`	str		Name of the object for echelle data. Same as NAME above but order numbers are omitted giving a unique name per object.
`ECH_OBJID`	int, integer		Object ID for echelle data. Each object is given an index in the order it appears increasing from from left to right. These are one based.
`ECH_ORDER`	int, integer		Physical echelle order
`ECH_ORDERINDX`	int, integer		Order indx, analogous to SLITID for echelle. Zero based.
`FLEX_SHIFT_GLOBAL`	float		Global shift of the spectrum to correct for spectralflexure (pixels). This is based on the sky spectrum atthe center of the slit
`FLEX_SHIFT_LOCAL`	float		Local shift of the spectrum to correct for spectralflexure (pixels). This should be a small correction tothe global value, and is based on the sky spectrumextracted near the object
`FLEX_SHIFT_TOTAL`	float		Total shift of the spectrum to correct for spectralflexure (pixels). This is the sum of the global andlocal FLEX_SHIFT
`FWHM`	float		Spatial FWHM of the object (pixels)
`FWHMFIT`	ndarray	float	Spatial FWHM across the detector (pixels)
`MASKDEF_EXTRACT`	bool		Boolean indicating if this is a forced extraction at the expected location from slitmask design.
`MASKDEF_ID`	int, integer		Slitmask definition ID
`MASKDEF_OBJMAG`	float		Magnitude of the object from the slitmask definition
`MASKDEF_OBJMAG_BAND`	str		Magnitude band of the object from the slitmask definition
`MASKDEF_OBJNAME`	str		Name of the object from the slitmask definition
`NAME`	str		Name of the object following the naming model
`OBJID`	int, integer		Object ID for multislit data. Each object is given an index for the slit it appears increasing from from left to right. These are one based.
`OBJTYPE`	str		Object type (e.g., standard, science)
`OPT_CHI2`	ndarray	float	Reduced chi2 of the model fit for this spectral pixel
`OPT_COUNTS`	ndarray	float	Optimal flux (counts)
`OPT_COUNTS_IVAR`	ndarray	float	Inverse variance of optimally extracted flux using modelivar image (counts^2)
`OPT_COUNTS_NIVAR`	ndarray	float	Optimally extracted noise variance, sky+read noise only (counts^2)
`OPT_COUNTS_SIG`	ndarray	float	Optimally extracted noise from IVAR (counts)
`OPT_COUNTS_SIG_DET`	ndarray	float	Optimally extracted detector noise (counts)
`OPT_COUNTS_SKY`	ndarray	float	Optimally extracted sky (counts)
`OPT_FLAM`	ndarray	float	Optimal flux (1e-17 erg/s/cm^2/Ang)
`OPT_FLAM_IVAR`	ndarray	float	Optimal flux inverse variance (1e-17 erg/s/cm^2/Ang)^-2
`OPT_FLAM_SIG`	ndarray	float	Optimal flux uncertainty (1e-17 erg/s/cm^2/Ang)
`OPT_FRAC_USE`	ndarray	float	Fraction of pixels in the object profile subimage used for this extraction
`OPT_FWHM`	ndarray	float	Spectral FWHM (in Angstroms) at every pixel of the optimally extracted flux.
`OPT_MASK`	ndarray	bool	Mask for optimally extracted flux. True=good
`OPT_WAVE`	ndarray	float	Optimal Wavelengths in vacuum (Angstroms)
`PYPELINE`	str		Name of the PypeIt pipeline mode
`RA`	float		Right Ascension (J2000) decimal degree
`S2N`	float		Median signal to noise ratio of the extracted spectrum(OPT if available, otherwise BOX)
`SLITID`	int, integer		PypeIt slit ID (aka SPAT_ID).
`SPAT_FRACPOS`	float, floating		Fractional location of the object on the slit
`SPAT_FWHM`	float		Spatial FWHM of the object (arcsec)
`SPAT_PIXPOS`	float, floating		Spatial location of the trace on detector (pixel) at half-way
`TRACE_SPAT`	ndarray	float	Object trace along the spec (spatial pixel)
`VEL_CORR`	float		Relativistic velocity correction for wavelengths
`VEL_TYPE`	str		Type of heliocentric correction (if any)
`WAVE_RMS`	float, floating		RMS (pix) for the wavelength solution for this slit.
`hand_extract_flag`	bool		Boolean indicating if this is a forced extraction at the location provided by the user.
`maskwidth`	float, floating		Size (in units of spatial fwhm) of the region used for local sky subtraction
`smash_peakflux`	float		Peak value of the spectral direction collapsed spatial profile
`smash_snr`	float		Peak S/N ratio of the spectral direction collapsed patial profile
`trace_spec`	ndarray	int,numpy.integer	Array of pixels along the spectral direction