pypeit.core.arc module

Module containing the core methods for arc-lamp fitting and basic analysis. Note that there are additional modules in pypeit.core.wavecal related to wavelength calibration.

pypeit.core.arc.detect_lines(censpec, sigdetect=5.0, fwhm=4.0, fit_frac_fwhm=1.25, input_thresh=None, cont_subtract=True, cont_frac_fwhm=1.0, max_frac_fwhm=3.0, min_pkdist_frac_fwhm=0.75, cont_samp=30, nonlinear_counts=10000000000.0, niter_cont=3, nfind=None, bpm=None, verbose=False, debug=False, debug_peak_find=False, debug_cont=False)[source]

Identify peaks in an input arc spectrum that satisfy a series of criteria:

  • Sufficient signal (set by sigdetect)

  • Peak amplitude < nonlinear_counts

  • Peak amplitude > input_thresh (optional)

  • Measured FWHM < fwhm * max_frac_fwhm

By default, the input spectrum has a continuum fitted to it and then subtracted prior to peak finding.

Parameters:

  • censpec (numpy.ndarray) – A 1D spectrum to be searched for significant detections, shape = (nspec,)

  • sigdetect (float, default=5., optional) – Sigma threshold above fluctuations for arc-line detection. Arcs are continuum subtracted and the fluctuations are computed after continuum subtraction.

  • input_thresh (float, str, optional) – Optionally the user can specify the threhsold that peaks must be above to be kept. In this case the sigdetect parameter will be ignored. This is most useful for example for cases where cont_subtract =False, and the user prefers to determine the significance threhsold outside of this routine, rather than using this routines defaults to determine the continuum level and standard deviation of the continuum subtracted spetrum. If a string input of ‘None’ is set then the code will simply return all peaks irrespective of any threshold. This is equivalent to setting the mph parameter to None in the detect_peaks code.

  • fwhm (float, default = 4.0, optional) – Number of pixels per fwhm resolution element.

  • fit_frac_fwhm (float, default 1.25, optional) – Number of pixels that are used in the fits for Gaussian arc line centroiding expressed as a fraction of the fwhm parameter

  • max_frac_fwhm (float, default = 3.0, optional) – maximum width allowed for usable arc lines expressed relative to the fwhm.

  • min_pkdist_frac_fwhm (float, default = 0.75, optional) – minimum allowed separation between peaks expressed relative to the fwhm.

  • cont_frac_fwhm (float, default = 1.0, optional) – width used for masking peaks in the spectrum when the continuum is being defined. Expressed as a fraction of the fwhm parameter

  • cont_subtract (bool, default = True, optional) – If true, the code will continuum subtract the input array by iteratively determining the continuum

  • cont_samp (float, default = 30.0, optional) – The number of samples across the spectrum used for continuum subtraction. Continuum subtraction is done via median filtering, with a width of ngood/cont_samp, where ngood is the number of good pixels for estimating the continuum (i.e. that don’t have peaks).

  • niter_cont (int, default = 3, optional) – Number of iterations of peak finding, masking, and continuum fitting used to define the continuum.

  • nonlinear_counts (float, default = 1e10, optional) – Value above which to mask saturated arc lines. This should be nonlinear_counts= nonlinear*saturation according to pypeit parsets. Default is 1e10 which is to not mask.

  • nfind (int, default = None, optional) – Return only the nfind highest significance lines. The default is None, which means the code will return all the lines above the significance threshold.

  • bpm (numpy.ndarray, optional) – Bad-pixel mask for input spectrum. If None, all pixels considered good. If passed in shape must match that of censpec

  • verbose (bool, default = False) – Output more stuff to the screen.

  • debug (bool, default = False) – Make plots showing results of peak finding and final arc lines that are used.

  • debug_cont (bool, default = False) – Make plots showing results of continuum fitting

Returns:

  • tampl (numpy.ndarray) – The amplitudes of the line detections in the true arc

  • tampl_cont (numpy.ndarray) – The amplitudes of the line detections in the continuum subtracted arc

  • tcent (numpy.ndarray) – The centroids of the line detections

  • twid (numpy.ndarray) – The 1sigma Gaussian widths of the line detections

  • center (numpy.ndarray) – The variance on tcent

  • w (numpy.ndarray) – An index array indicating which detections are the most reliable.

  • arc (numpy.ndarray) – The continuum sutracted arc used to find detections.

  • nsig (numpy.ndarray) – The significance of each line detected relative to the 1sigma variation in the continuum subtracted arc in the line free region. Bad lines are assigned a significance of -1, since they don’t have an amplitude fit

pypeit.core.arc.detect_peaks(x, mph=None, mpd=1, threshold=0, edge='rising', kpsh=False, valley=False, show=False, ax=None)[source]

Detect peaks in data based on their amplitude and other features. This is generally, but not exclusively, used for arc line detecting.

This code was taken from https://github.com/demotu/BMC

Parameters:

  • x (array-like) – 1D vector with data

  • mph ({None, number}, optional (default = None)) – detect peaks that are greater than minimum peak height (if parameter valley is False) or peaks that are smaller than maximum peak height (if parameter valley is True).

  • mpd (positive integer, optional (default = 1)) – detect peaks that are at least separated by minimum peak distance (in number of data).

  • threshold (positive number, optional (default = 0)) – detect peaks (valleys) that are greater (smaller) than threshold in relation to their immediate neighbors.

  • edge ({None, 'rising', 'falling', 'both'}, optional (default = 'rising')) – for a flat peak, keep only the rising edge (‘rising’), only the falling edge (‘falling’), both edges (‘both’), or don’t detect a flat peak (None).

  • kpsh (bool, optional (default = False)) – keep peaks with same height even if they are closer than mpd.

  • valley (bool, optional (default = False)) – if True (1), detect valleys (local minima) instead of peaks.

  • show (bool, optional (default = False)) – if True (1), plot data in matplotlib figure.

  • ax (matplotlib.axes.Axes, optional) – matplotlib.axes.Axes instance to use when plotting. If None and show is True, a new instance is constructed.

Returns:

ind – 1D vector with element indices containing the peaks in x

Return type:

array-like

Notes

The detection of valleys instead of peaks is performed internally by simply negating the data:

ind_valleys = detect_peaks(-x)

The function can handle NaN’s

See this IPython Notebook [1].

__author__ = "Marcos Duarte, https://github.com/demotu/BMC"
__version__ = "1.0.5"
__license__ = "MIT"

Version history:

  • ‘1.0.5’: The sign of mph is inverted if parameter valley is True

References

Examples

>>> from pypeit.core.arc import detect_peaks
>>> x = np.random.randn(100)
>>> x[60:81] = np.nan
>>> # detect all peaks and plot data
>>> ind = detect_peaks(x, show=True)
>>> print(ind)

>>> x = np.sin(2*np.pi*5*np.linspace(0, 1, 200)) + np.random.randn(200)/5
>>> # set minimum peak height = 0 and minimum peak distance = 20
>>> detect_peaks(x, mph=0, mpd=20, show=True)

>>> x = [0, 1, 0, 2, 0, 3, 0, 2, 0, 1, 0]
>>> # set minimum peak distance = 2
>>> detect_peaks(x, mpd=2, show=True)

>>> x = np.sin(2*np.pi*5*np.linspace(0, 1, 200)) + np.random.randn(200)/5
>>> # detection of valleys instead of peaks
>>> detect_peaks(x, mph=-1.2, mpd=20, valley=True, show=True)

>>> x = [0, 1, 1, 0, 1, 1, 0]
>>> # detect both edges
>>> detect_peaks(x, edge='both', show=True)

>>> x = [-2, 1, -2, 2, 1, 1, 3, 0]
>>> # set threshold = 2
>>> detect_peaks(x, threshold = 2, show=True)
pypeit.core.arc.find_lines_qa(spec, cen, amp, good, bpm=None, thresh=None, nonlinear=None)[source]

Show a QA plot for the line detection. Only used for debugging

Detected lines are marked and then color coded by a variety of criteria.

Parameters:

  • spec (numpy.ndarray) – Spectrum used to detect lines

  • cen (numpy.ndarray) – Identified line peaks

  • amp (numpy.ndarray) – Amplitude of the identified lines.

  • good (numpy.ndarray) – Boolean array selecting the good line detections.

  • bpm (numpy.ndarray, optional) – The bad-pixel mask for the spectrum. If None, all pixels are assumed to be valid.

  • thresh (float, optional) – Threshold value for line detection

  • nonlinear (float, optional) – Threshold for nonlinear detector response.

pypeit.core.arc.fit2darc(all_wv, all_pix, all_orders, nspec, nspec_coeff=4, norder_coeff=4, sigrej=3.0, func2d='legendre2d', debug=False)[source]

Routine to obtain the 2D wavelength solution for an echelle spectrograph. This is calculated from the spec direction pixel-centroid and the order number of identified arc lines. The fit is a simple least-squares with rejections.

This is a port of the XIDL code: x_fit2darc.pro

Parameters:

  • all_wv (numpy.ndarray) – wavelengths of the identified lines

  • all_pix (numpy.ndarray) – Spectral direction centroid positions of the identified lines

  • all_orders (numpy.ndarray) – Echelle order number for each of the identified lines

  • nspec (int) – size of the image in the spectral direction

  • nspec_coeff (int, optional) – order of the fitting along the spectral (pixel) direction for each order

  • norder_coeff (int, optional) – order of the fitting in the order direction

  • sigrej (float, optional) – sigma level for the rejection

  • debug (bool, optional) – If True, show extra plots to check the status of the procedure

Returns:

pypeitFit – 2D wavelength solution fit

Return type:

pypeit.core.fitting.PypeItFit

pypeit.core.arc.fit2darc_global_qa(pypeitFit, nspec, outfile=None)[source]

Generate a QA plot for the 2D fit of the wavelength solution generated by pypeit.core.arc.fit2darc

Parameters:

  • pypeitFit (pypeit.core.fitting.PypeItFit) – Fit object for the 2D arc solution

  • nspec (int) – Size of the image in the spectral direction

  • outfile (str, optional) – Name of the outfile to write to disk. If not provided, show to screen.

pypeit.core.arc.fit2darc_orders_qa(pypeitFit, nspec, outfile=None)[source]

QA on 2D fit of the wavelength solution of an Echelle spectrograph.

Each sub-panel contains a single order with the global fit and the residuals.

Parameters:

  • pypeitFit (pypeit.core.fitting.PypeItFit) – Fit object for the 2D arc solution

  • nspec (int) – size of the image in the spectral direction

  • outfile (str, optional) – Write to this file. If not provided, show to screen

pypeit.core.arc.fit_arcspec(xarray, yarray, pixt, fitp)[source]

Fit a series of pre-identified arc spectrum lines.

The implementation is a simple 3-parameter Gaussian (amplitude, centroid, width)

Parameters:

  • xarray (numpy.ndarray) – x-values of the input spectrum. Either pixels or normalized pixel space (0-1)

  • yarray (numpy.ndarray) – Arc spectrum in counts

  • pixt (numpy.ndarray) – Initial guess for the center of the lines

  • fitp (int) – Number of pixels to fit with

Returns:

  • ampl (numpy.ndarray) – amplitudes of the fitted lines

  • cent (numpy.ndarray) – centroids of the fitted lines -999. are bad fits

  • widt (numpy.ndarray) – widths (sigma) of the fitted lines

  • centerr (numpy.ndarray) – error in the centroids of the fitted lines

pypeit.core.arc.get_censpec(slit_cen, slitmask, arcimg, gpm=None, box_rad=3.0, nonlinear_counts=10000000000.0, slit_bpm=None, slitIDs=None, verbose=True)[source]

Extract a boxcar spectrum with radius box_rad (pixels) from the input image using the input trace. By default, outliers within the box are clipped with 3.0 sigma rejection using astropy.stats.sigma_clipped_stats.

Parameters:

  • slit_cen (numpy.ndarray) – Trace down the center of the slit. Shape (nspec, nslits)

  • slitmask (numpy.ndarray) – Image where pixel values identify its parent slit, starting with 0. Pixels with -1 are not part of any slit. Shape must match arcimg.

  • arcimg (numpy.ndarray) – Image to extract the arc from. This should be an arcimage or perhaps a frame with night sky lines. Shape (nspec, nspat)

  • gpm (numpy.ndarray, optional) – Input mask image with same shape as arcimg. Convention True = good and False = bad. If None, all pixels are considered good. Shape must match arcimg.

  • box_rad (float, optional) – Half-width of the boxcar (floating-point pixels) in the spatial direction used to extract the arc.

  • nonlinear_counts (float, optional) – Values exceeding this input value are masked as bad.

  • slitIDs (list, optional) – A list of the slit IDs to extract (if None, all slits will be extracted)

  • slit_bpm (numpy.ndarray, bool, optional) – Bad pixel mask for the slits. True = bad. Shape must be (nslits,). Arc spectra are filled with np.nan for masked slits.

  • verbose (bool, optional) – Print out verbose information?

Returns:

  • arc_spec (numpy.ndarray) – Array containing the extracted arc spectrum for each slit. Shape is (nspec, nslits)

  • arc_spec_bpm (numpy.ndarray) – Bad-pixel mask for the spectra. Shape is (nspec, nslits).

  • bpm_mask (numpy.ndarray) – Bad-slit mask, True means the entire spectrum is bad. Shape is (nslits,).

pypeit.core.arc.iter_continuum(spec, gpm=None, fwhm=4.0, sigthresh=2.0, sigrej=3.0, niter_cont=3, cont_samp=30, cont_frac_fwhm=1.0, cont_mask_neg=False, qa_title='', npoly=None, debug_peak_find=False, debug=False)[source]

Routine to determine the continuum and continuum pixels in spectra with peaks.

The general procedure is to:

  • Detect positive “peaks” using pypeit.core.arc.detect_peaks

  • Optionally, detect negative “peaks” using pypeit.core.arc.detect_peaks

  • Mask these peaks

  • Generate a running median with sampling width set by cont_samp

  • Fit this median continuum with a polynomial of order npoly (if set)

  • Evaluate the fit (npoly is set) or interpolate the median at all locations

Note: This was developed for arc line spectra and may not function well in other contexts.

Parameters:

  • spec (numpy.ndarray of floats) – 1D spectrum with shape (nspec,) for which the continuum is to be characterized

  • gpm (numpy.ndarray, bool) – A mask with shape (nspec,) indicating which pixels are good. True = Good, False=Bad

  • niter_cont (int, default = 3, optional) – Number of iterations of peak finding, masking, and continuum fitting used to define the continuum.

  • npoly (int, default = None, optional) – If set the code will perform a polynomimal fit to the interpolate a running median filter of the continuum points instead of the default behavior which is to just return the interpolated running median filter

  • sigthresh (float, default = 2.0, optional) – Signifiance threshold for peak finding

  • sigrej (float, default = 3.0, optional) – Sigma clipping rejection threshold for threshold determination

  • fwhm (float, default = 4.0, optional) – Number of pixels per fwhm resolution element.

  • cont_samp (float, default = 30.0, optional) – The number of samples across the spectrum used for continuum subtraction. Continuum subtraction is done via median filtering, with a width of ngood/cont_samp, where ngood is the number of good pixels for estimating the continuum (i.e. that don’t have peaks).

  • cont_frac_fwhm (float, default = 1.0, optional) – Width used for masking peaks in the spectrum when the continuum is being defined. Expressed as a fraction of the fwhm parameter

  • cont_mask_neg (bool, default = False, optional) – If True, the code will also search for negative peaks when iteratively determining the continuum. This option is used for object finding in the near-IR where there will also be negative peaks.

  • cont_samp – The number of samples across the spectrum used for continuum subtraction. Continuum subtraction is done via median filtering, with a width of ngood/cont_samp, where ngood is the number of good pixels for estimating the continuum

  • debug (bool, default = False, optional) – Show plots for debugging

Returns:

  • cont (numpy.ndarray of floats) – The continuum determined with shape (nspec,)

  • cont_mask (numpy.ndarray of bool) – A mask indicating which pixels were used for continuum determination with shape (nspec,)

pypeit.core.arc.plot_detect_peaks(x, mph, mpd, threshold, edge, valley, ax, ind)[source]

Plot results of the pypeit.core.arc.detect_peaks function, see its help for a description of the variables.

Only used for debugging

pypeit.core.arc.resize_mask2arc(shape_arc, slitmask_orig)[source]

Rebin the input slitmask to a new shape. Generally used for cases where the arc image has a different binning than the science image.

Parameters:

  • shape_arc (tuple) – Shape of the arc

  • slitmask_orig (numpy.ndarray of floats) – original slitmask

Returns:

slitmask – Slitmask with shape corresponding to that of the arc

Return type:

numpy.ndarray of floats

pypeit.core.arc.resize_slits2arc(shape_arc, shape_orig, trace_orig)[source]

Resizes the trace (spat positions from where the arc was extracted) created with some original binning to be relevant to an arc with a different binning.

Parameters:

  • shape_arc (tuple) – shape of the arc

  • shape_orig (tuple) – original shape of the images used to create the trace

  • trace_orig (numpy.ndarray of floats) – trace that you want to resize

Returns:

trace corresponding to the binning of the arc

Return type:

numpy.ndarray

pypeit.core.arc.resize_spec(spec_from, nspec_to)[source]

Resize the input spectrum (usually an arc spectrum) to a new size using linear interpolation scipy.interpolate.interp1d

Parameters:

  • spec_from (numpy.ndarray) – one or more spectra to resize via interpolation. Shape must be (nspec, nslits) or (nspec,).

  • nspec_to (int) – size of spectrum you wish to resize to.

Returns:

New spectra or spectrum with shape (nspec_to, nslits) or (nspec_to,).

Return type:

numpy.ndarray