# Fluxing

## Overview

Fluxing is done after the main run of PypeIt, and requires the extracted 1D spectra.

It is a two step process of generating a Sensitivity Function and then Applying the Sensitivity Function. We describe each in turn. However, we first discuss the basic fluxing concepts/approach.

## Theoretical Underpinnings of Flux Calibration

### Sensitivity Function Units and Definitions

The sensitivity function in PypeIt is defined to be the function $$S_\lambda$$ satisfying

$S_\lambda = \frac{F_\lambda}{N_\lambda}$

with units of $$[{\rm erg/cm^2/photons}]$$, where $$F_\lambda$$ is the specific energy flux in units of $$[{\rm erg/s/cm^2/\mathrm{\mathring{A}}}]$$, and $$N_\lambda$$ is the specific photon flux with units $$[{\rm photons/s/\mathrm{\mathring{A}}}]$$.

PypeIt spec1d files contain $$N_{\rm pix}$$ with units $$[{\rm photons/pixel}]$$. To generate flux-calibrated spectra $$F_\lambda$$, $$S_\lambda$$ must be computed from a spectrophotometric standard-star observations.

$$N_{\lambda}$$ must be determined from $$N_{\rm pix}$$ via

$N_\lambda = \frac{N_{\rm pix}}{\frac{d\lambda}{d{\rm pix}}\ \Delta t},$

where $$\Delta t$$ is the exposure time and $$\frac{d\lambda}{d{\rm pix}}$$ is the wavelength spacing per pixel, which is in general not a constant since PypeIt spec1d spectra (i.e. $$N_{\rm pix}$$) are extracted on an irregularly spaced wavelength grid.

After flux calibration, flux calibrated spectra (i.e. $$F_\lambda$$) are reported in the spec1d files as OPT_FLAM and BOX_FLAM for optimal and boxcar extractions, respectively, in units of $$[10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}]$$.

### Spectroscopic Zeropoints

Flux calibration of PypeIt spectra is expressed via the “spectroscopic zeropoint”, which, by analogy with the imaging zeropoint, is defined to be:

${\rm Zeropoint} \equiv -2.5 \log_{10}{\left[\frac{\frac{\lambda^2}{c}S_\lambda}{\left(\frac{3631 {\rm Jy}}{{\rm photons}/ {\rm s} / \mathrm{\mathring{A}}}\right)}\right]}.$

With this definition we see that an astronomical source with a flat spectrum in frequency $$\nu$$, i.e. $$F_\nu = {\rm const}$$ and AB magnitude equal to the Zeropoint will produce $$N_\lambda = 1\ {\rm photon/s/\mathrm{\mathring{A}}}$$ on the detector, that is the sum of the $$N_{\rm pix}$$ photons per pixel over all pixels corresponding to a $$\Delta \lambda = 1 \mathrm{\mathring{A}}$$ interval will be equal to unity.

From the definition of the spectroscopic zeropoint above, it follows that

$\left(\frac{F_\lambda}{10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}}\right) = 10^{-0.4({\rm Zeropoint - ZPCONST})} \left(\frac{N_\lambda}{\rm photons/s/\mathrm{\mathring{A}}}\right)\left(\frac{\mathrm{\mathring{A}}}{\lambda}\right)^2,$

where $$ZPCONST = 40.09$$ is a dimensionless number defined by

${\rm ZPCONST}\equiv -2.5 \log_{10}{\left[\frac{\frac{\mathrm{\mathring{A}}^2}{c}\times 10^{-17}{\rm erg/s/cm^2/\mathrm{\mathring{A}}}}{3631 {\rm Jy}}\right]}.$

In practice, PypeIt fits and stores the spectroscopic zerpoints and uses the equation above to compute $$F_\lambda$$ from $$N_\lambda$$ and vice-versa.

The sensitivity function script, pypeit_sensfunc, produces a QA plot showing the the zeropoint fit, as shown below. For echelle observations this zeropoint QA is shown for each order.

### Spectroscopic Throughput

The zeropoint is closely related to the spectroscopic throughput. The number of counts per pixel in a spectrum of an object with flux $$F_\lambda$$

$N_{\rm pix} = A\ T(\lambda)\ {\rm Atm}(\lambda)\ \frac{d\lambda}{d_{\rm pix}}\frac{F_\lambda}{h\nu}\Delta t,$

where $$A$$ is the effective aperture of the telescope, $$T(\lambda)$$ is the spectroscopic throughput, $${\rm Atm(\lambda)}$$ is the attenuation caused by the Earth’s atmosphere, $$\frac{d\lambda}{d_{\rm pix}}$$ is the number of $$\mathrm{\mathring{A}}$$ per pixel defined above, $$h\nu$$ is the photon energy, and $$\Delta t$$ is the exposure time.

Based on this equation and the definitions above it follows that the spectroscopic throughput can be written $$T(\lambda) = \frac{h\nu}{A S_\lambda}$$.

Note $$T(\lambda)$$ is clearly dimensionless given the units of $$S_\lambda$$: $$[{\rm erg/cm^2/photons}]$$. As $$S_\lambda$$ is specified by the zeropoint, throughput curves can be computed once the zeropoints given the effective aperture of the telescope.

In addition to the zeropoint QA shown above, the sensitivity function script pypeit_sensfunc also produces a QA plot showing throughput curve(s), computed directly from the spectroscopic zeropoints.

Note that we have defined the spectroscopic throughput above to be that of the telescope + instrument system, but it does not include the attenuation caused by the Earth’s atmosphere (see below). Also, the zeropoints, sensitivity functions, and PypeIt flux calibration algorithms in general do not attempt to remove the impact of slit losses. In the limit where your standard star observations and science observations have exactly the same seeing, the flux calibration will be perfect. In the more realistic scenario where they differ, this will manifest as a wavelength-dependent systematic error in the flux calibration, with the direction of the error depending on the relative seeing between the standard-star and science observations. In future versions, we hope to implement a better treatment of slit losses. For the time being, we recommend that users that require very accurate flux calibration force PypeIt flux-calibrated spectra to agree with photometry. This can be done using the filter parameter option for 1D coadding (see Coadd1DPar Keywords), which can be set in the .coadd1d file, which is used to guide 1D coaddition with the pypeit_coadd1d script (see Coadd 1D Spectra).

### Extinction Correction

Note

This discussion only applies to data at λ < 7000Å using the UVIS sensitivity function algorithm (see pypeit_sensfunc). In this regime, there is no detailed modeling of telluric absorption, but atmospheric extinction is corrected for. See Telluric correction for discussion of how to handle atmospheric effects for IR data.

Attenuation caused by the Earth’s atmosphere is called atmospheric extinction. Extinction is a function of wavelength and airmass, and needs to be corrected for in science spectra for proper flux calibration. It is generally measured by the observatory in a dedicated campaign (sometimes decades ago), and published for use. PypeIt includes measured extinction files for many of the observatories whose spectrographs are supported by the pipeline. The list of current extinction files is in pypeit/data/extinction/README, and shown below:

# Airmass Extinction files
# NOTE: Longitudes are measured increasing to the east, so west longitudes are
#       negative.  Generally grabbed from the Internet.  Buyer beware.
File             Lon         Lat        Elevation
ctioextinct.dat     -70.8150    -30.1653    2215.0
kpnoextinct.dat    -111.600      31.9633    2120.0
lapalmaextinct.dat  -17.8947     28.7636    2396.0
mkoextinct.dat     -155.47833    19.82833   4160.0
mthamextinct.dat   -121.6428     37.34139   1290.0
paranalextinct.dat  -70.4048305 -24.627167  2635.43
palomarextinct.dat -116.86489    33.35631   1713.
lasillaextinct.dat  -70.73      -29.2567    2400.


The extinction correction is applied twice in the flux calibration process:

1. During the creation of the sensitivity function, the spectrophotometric standard star is corrected to what its flux would have been above the atmosphere.

2. While applying the sensitivity function to science spectra, those spectra are also corrected because it is highly unlikely that both the standard and science objects were observed at the same airmass.

PypeIt selects the observatory’s extinction file whose geographic coordinates are closest to those of the telescope associated with the spectrograph being used for the reduction. For most users, this is an automatic selection and nothing further need be done. If, however, you are working with a telescope situated more than 5 deg (geographic coordinates) from one of the listed observatories, the code will crash unless you specify a particular extinction file to use or install a custom extinction file in the PypeIt cache.

#### Specifying an extinction file to use

To specify a particular extinction file, it must be specified in both steps of the flux calibration process:

1. In the Sensitivity Input File, add the following parameter block to the top of the file:

[sensfunc]
[[UVIS]]
extinct_file = <name_of_file>

2. In the Flux File, add the following parameter block to the top of the file:

[fluxcalib]
extinct_file = <name_of_file>


These steps may be used to either specify one of the PypeIt-included extinction files listed above or to employ a user-supplied extinction file (possibly for an observatory not on the list). The latter may be installed in the user’s PypeIt cache using the script pypeit_install_extinctfile, whose usage is described in Installation Scripts.

The above steps are meant as a scaffold during development of new instruments in PypeIt, and users are encouraged to submit a pull request to the main repository with new extinction files for other observatories.

## Creating a PypeIt Sensitivity Function

The sensitivity function is generated from the Spec1D Output file of a processed standard star.

PypeIt uses an archived fluxed spectrum from either the CALSPEC calibration database or one of the files we have grabbed from ESO. If you observed something else, see Adding a Standard Star.

The sensitivity function is generated by dividing the standard star’s flux in units of $$10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}$$ by the standard star’s counts per second per Angstrom [photons/s/$$\mathrm{\mathring{A}}$$]. The sensitivity function is written to disk as a FITS file. It has units of $$[10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}]/[{\rm e-/s/\mathrm{\mathring{A}}}]$$. For more information see Flux Calibration.

To flux calibrate spectra, the spectrum in counts from the spec1D files is used to compute counts per second per Angstrom, by which the sensitivity function is multiplied to yield a fluxed science spectrum in units of $$F_\lambda\ [10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}]$$.

The sensitivity function is written to disk as a FITS file. It has units of $$[10^{-17} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}]/[{\rm photons/s/\mathrm{\mathring{A}}}]$$ or $$[10^{-17} {\rm erg/cm^2/photons}]$$.

If you are using an IR instrument or choose the IR mode (see below) then you will need to have grabbed the telluric files. See Atmospheric Model Grids.

### pypeit_sensfunc

The process is mediated by the pypeit_sensfunc script.

The script usage can be displayed by calling the script with the -h option:

$pypeit_sensfunc -h usage: pypeit_sensfunc [-h] [--algorithm {UVIS,IR}] [--multi MULTI] [-o OUTFILE] [-s SENS_FILE] [-f FLATFILE] [--debug] [--par_outfile PAR_OUTFILE] [-v VERBOSITY] spec1dfile Compute a sensitivity function positional arguments: spec1dfile spec1d file for the standard that will be used to compute the sensitivity function options: -h, --help show this help message and exit --algorithm {UVIS,IR} Override the default algorithm for computing the sensitivity function. Note that it is not possible to set --algorithm and simultaneously use a .sens file with the --sens_file option. If you are using a .sens file, set the algorithm there via: [sensfunc] algorithm = IR The algorithm options are: UVIS = Should be used for data with lambda < 7000A. No detailed model of telluric absorption but corrects for atmospheric extinction. IR = Should be used for data with lambbda > 7000A. Performs joint fit for sensitivity function and telluric absorption using HITRAN models. --multi MULTI List of detector numbers to splice together for instruments with multiple detectors arranged in the spectral direction, e.g. --multi = '3,7'. Note that it is not possible to set --multi and simultaneously use a .sens file with the --sens_file option. If you are using a .sens file, set the multi_spec_det param there via: [sensfunc] multi_spec_det = 3,7 -o OUTFILE, --outfile OUTFILE Output file for sensitivity function. If not specified, the sensitivity function will be written out to a standard filename in the current working directory, i.e. if the standard spec1d file is named spec1d_b24-Feige66_KASTb_foo.fits the sensfunc will be written to sens_b24-Feige66_KASTb_foo.fits. A QA file will also be written as sens_spec1d_b24-Feige66_KASTb_foo_QA.pdf and a file showing throughput plots to sens_spec1d_b24-Feige66_KASTb_foo_throughput.pdf. The same extensions for QA and throughput will be used if outfile is provided but with .fits trimmed off if it is in the filename. -s SENS_FILE, --sens_file SENS_FILE Configuration file with sensitivity function parameters -f FLATFILE, --flatfile FLATFILE Use the flat file for computing the sensitivity function. Note that it is not possible to set --flatfile and simultaneously use a .sens file with the --sens_file option. If you are using a .sens file, set the flatfile there via e.g.: [sensfunc] flatfile = Calibrations/Flat_A_0_DET01.fits Where Flat_A_0_DET01.fits is the flat file in your Calibrations directory --debug show debug plots? --par_outfile PAR_OUTFILE Name of output file to save the parameters used by the fit -v VERBOSITY, --verbosity VERBOSITY Verbosity level between 0 [none] and 2 [all]. Default: 1. Level 2 writes a log with filename sensfunc_YYYYMMDD- HHMM.log  Here is a typical call: pypeit_sensfunc spec1dfile -o Keck_LRISr_600_7500_sens.fits  This analyzes the standard star spectrum in spec1dfile and writes the sensitivity file to Keck_LRISr_600_7500_sens.fits. Note that the spec1dfile to be used in the above argument is the spec1d file associated with the reduced standard star exposure (not the spec1d file of the reduced science frame). Here are the common options used: #### –multi For some instruments (e.g. keck_deimos, gemini_gmos), the spectrum spans across multiple detectors. You can have the sensitivity function handle this by using the --multi option, e.g.: pypeit_sensfunc --multi 3,7  #### –debug Throws a number of plots to the screen #### –algorithm The algorithm options are: • UVIS = Should be used for data with $$\lambda < 7000 \mathrm{\mathring{A}}$$. No detailed model of telluric absorption but corrects for atmospheric extinction. • IR = Should be used for data with $$\lambda > 7000 \mathrm{\mathring{A}}$$. Performs joint fit for sensitivity function and telluric absorption using HITRAN models. #### –sens Provide a Sensitivity Input File to guide the process. Do this if your changes to the defaults are not accommodated by the script inputs. ### Sensitivity Input File This type of Input File Format contains only a Parameter Block where you specify the sensfunc parameters. For example, if you wish to use the MaunaKea telluric grid with your data, you would create a sens file containing: # User-defined execution parameters [sensfunc] algorithm = IR [[IR]] telgridfile = TelFit_MaunaKea_3100_26100_R20000.fits  #### IR without a Standard If you wish to generate a sensitivity function on a standard star that is not part of the PypeIt database and are working in the IR, you can feed the stellar parameters. Here is an example of the lines for a Sensitivity Input File. [sensfunc] algorithm = IR star_mag = 12.1 star_type = A0  Then run on the spec1d file as you would otherwise. For an A0 star, we use the Vega spectrum. Otherwise, we use the Kurucz93 stellar SED. Alternatively, see Adding a Standard Star. ### Sensitivity Output File The name of the file containing the sensitivity function can be directly provided (using the -o command-line option). Otherwise, the file name will default to one that is identical to the provided spec1d file, but with spec1d replaced by sens. The sensitivity data is built and written by SensFunc, which subclasses from DataContainer. Note that the telluric data are only included when using the IR algorithm. Here is its datamodel: Version 1.0.1 HDU Name HDU Type Data Type Description PRIMARY astropy.io.fits.PrimaryHDU Empty data HDU. Contains basic header information. TELLURIC astropy.io.fits.BinTableHDU Telluric model; see Telluric. This is identical to the HDU extension produced in the file produced by pypeit_tellfit. Only provided when using the IR algorithm. SENS astropy.io.fits.BinTableHDU Table with the sensitivity function WAVE astropy.io.fits.ImageHDU Wavelength vectors. May be combined from many detectors; see the multi_spec_det parameter in SensFuncPar Keywords. ZEROPOINT astropy.io.fits.ImageHDU Sensitivity function zeropoints. May be combined from many detectors; see the multi_spec_det parameter in SensFuncPar Keywords. THROUGHPUT astropy.io.fits.ImageHDU Spectrograph throughput measurements. May be combined from many detectors; see the multi_spec_det parameter in SensFuncPar Keywords. TELLURIC table (if present) Column Data Type Description WAVE float64 Wavelength vector TELLURIC float64 Best-fitting telluric model spectrum OBJ_MODEL float64 Best-fitting object model spectrum TELL_THETA float64 Best-fitting telluric model parameters TELL_PRESS float64 Best-fitting telluric model pressure TELL_TEMP float64 Best-fitting telluric model temperature TELL_H2O float64 Best-fitting telluric model humidity TELL_AIRMASS float64 Best-fitting telluric model airmass TELL_RESLN float64 Best-fitting telluric model spectral resolution TELL_SHIFT float64 Best-fitting shift applied to the telluric model spectrum TELL_STRETCH float64 Best-fitting stretch applied to the telluric model spectrum OBJ_THETA float64 Best-fitting object model parameters CHI2 float64 Chi-square of the best-fit model SUCCESS bool Flag that fit was successful NITER int64 Number of fit iterations ECH_ORDERS int64 Echelle order for this specrum (echelle data only) POLYORDER_VEC int64 Polynomial order for each slit/echelle (if applicable) IND_LOWER int64 Lowest index of a spectral pixel included in the fit IND_UPPER int64 Highest index of a spectral pixel included in the fit WAVE_MIN float64 Minimum wavelength included in the fit WAVE_MAX float64 Maximum wavelength included in the fit SENS table Column Data Type Description SENS_WAVE float64 Wavelength vector SENS_COUNTS_PER_ANG float64 Flux in counts per angstrom SENS_LOG10_BLAZE_FUNCTION float64 Log10 of the blaze function for each slit/order SENS_ZEROPOINT float64 Measured sensitivity zero-point data SENS_ZEROPOINT_GPM bool Good-pixel mask for the measured zero points SENS_ZEROPOINT_FIT float64 Best-fit smooth model to the zero points SENS_ZEROPOINT_FIT_GPM bool Good-pixel mask for the model zero points SENS_COEFF float64 Coefficients of smooth model fit to zero points ECH_ORDERS int64 Echelle order for this specrum (echelle data only) POLYORDER_VEC int64 Polynomial order for each slit/echelle (if applicable) WAVE_MIN float64 Minimum wavelength included in the fit WAVE_MAX float64 Maximum wavelength included in the fit SENS_FLUXED_STD_WAVE float64 The wavelength array for the fluxed standard star spectrum SENS_FLUXED_STD_FLAM float64 The F_lambda for the fluxed standard star spectrum SENS_FLUXED_STD_FLAM_IVAR float64 The inverse variance of F_lambda for the fluxed standard star spectrum SENS_FLUXED_STD_MASK bool The good pixel mask for the fluxed standard star spectrum ## Applying the PypeIt Sensitivity Function Once you have generated a Sensitivity Function, you may apply it to one or more Spec1D Output files. The files are modified in place, filling the OPT_FLAM, BOX_FLAM, etc. entries, as described in Spec1D Output. ### Flux File To flux one or more spec1d files, one provides a Flux File with the following format with a Parameter Block (optional) and a Data Block (required). If one wishes to modify the FluxCalibratePar Keywords, add a Parameter Block at the top of the file, e.g.: [fluxcalib] extrap_sens = True  There are several ways to provide the Data Block, which always begins and ends with flux read and flux end, respectively. First, with one senstivity file and a list of spec1dfiles to be fluxed: flux read filename | sensfile spec1dfile1 | sensfile1 spec1dfile2 | ... | ... | flux end  Second, with a (presumably unique) sensitivity file for each spec1dfile: flux read filename | sensfile spec1dfile1 | sensfile1 spec1dfile2 | sensfile2 spec1dfile3 | sensfile3 ... | ... flux end  Third, if the spectrograph has an archived sensitivity function (only DEIMOS to date) then a list of spec1d files: flux read filename spec1dfile1 spec1dfile2 ... flux end  Here is an actual example: flux read filename | sensfile spec1d_UnknownFRBHostY_vlt_fors2_2018Dec05T020241.687.fits | VLT_FORS2_sens.fits spec1d_UnknownFRBHostY_vlt_fors2_2018Dec05T021815.356.fits spec1d_UnknownFRBHostY_vlt_fors2_2018Dec05T023349.816.fits flux end  To aid generating this file, we provide the pypeit_flux_setup script. The script usage can be displayed by calling the script with the -h option: $ pypeit_flux_setup -h
usage: pypeit_flux_setup [-h] [--name NAME] [--objmodel {qso,star,poly}]
paths [paths ...]

Setup configuration files to perform flux calibration, 1D coadding, and telluric
correction.

positional arguments:
paths                 One or more paths for Science folders or sensitivity
to be detected.

options:
-h, --help            show this help message and exit
--name NAME           The base name to use for the output files. Defaults to
the instrument name is used.
--objmodel {qso,star,poly}
science object model used in the telluric fitting. The
options are:

qso = For quasars. You might need to set redshift,
bal_wv_min_max in the tell file.

star = For stars. You need to set star_type, star_ra,
star_dec, and star_mag in the tell_file.

poly = For other type object, You might need to set
fit_wv_min_max, and norder in the tell_file.


### pypeit_flux_calib

Fluxing is performed with the pypeit_flux_calib script.

The script usage can be displayed by calling the script with the -h option:

\$ pypeit_flux_calib -h
usage: pypeit_flux_calib [-h] [--par_outfile] [-v VERBOSITY] flux_file

Flux calibrate 1D spectra produced by PypeIt

positional arguments:
flux_file             File to guide fluxing process.  This file must have the
following format:

filename | sensfile
spec1dfile1 | sensfile1
spec1dfile2 |
...
flux end

OR

filename | sensfile
spec1dfile1 | sensfile1
spec1dfile2 | sensfile2
spec1dfile3 | sensfile3
...
flux end

OR

[fluxcalib]
use_archived_sens = True
filename
spec1dfile1
spec1dfile2
spec1dfile3
...
flux end

That is, you must specify either a sensfile for all
spec1dfiles on the first line, specify one sensfile for
each spec1dfile, or specify no sensfiles and use an
archived one.
Archived sensfiles are available for the following
spectrographs: keck_deimos

options:
-h, --help            show this help message and exit
--par_outfile         Output to save the parameters
-v VERBOSITY, --verbosity VERBOSITY
Verbosity level between 0 [none] and 2 [all]. Default:
1. Level 2 writes a log with filename
flux_calib_YYYYMMDD-HHMM.log


Here is a typical call:

pypeit_flux_calib flux_file.txt


Again, the Spec1D Output files are modified in place; see there for the related datamodel changes. Also see pypeit_show_1dspec for details on how to view them.

### Archival Sensitivity Functions

PypeIt supports using archived sensitivity functions for flux calibration. Currently only experimental keck_deimos sensitivity files are available. They can be applied by adding use_archived_sens = True to the flux file passed to pypeit_flux_calib. For example:

[fluxcalib]
use_archived_sens = True

flux end


## Troubleshooting

### Problem with Empty filename

If you encounter this error when doing flux calibration with the IR algorithm, please do the following steps:

• Make sure you have installed the relevant atmosphere telluric models. See the instructions for installing this Additional Data.

• Write the filename of the corresponding file for your observatory in the parameter telgridfile (i.e. keck_lris_sens.txt), e.g.:

[sensfunc]
algorithm = IR
polyorder = 8
[[IR]]
telgridfile = TelFit_MaunaKea_3100_26100_R20000-006.fits

• Run pypeit_sensfunc with the –sens_file option, e.g.:

pypeit_sensfunc your_spec1dfile -o your_output.fits --sens_file keck_lris_sens.txt


If your star is not in the repository you can add in a new solution if it is in the ESO database.

You will need to place their .dat file in pypeit/data/standards/esofil/ and then edit the esofil_info.txt file accordingly. Make sure the flux column is in flux units rather than magnitudes (i.e. those files starting with f in the ESO database), and these fluxes are in units of $$[10^{-16} {\rm erg/s/cm^2/\mathrm{\mathring{A}}}]$$.

Extra kudos if you submit this as a PR for others benefit.

The guts of the flux algorithms are guided by the apply_flux_calib() class.