pypeit.core.wavecal.autoid module

Module for finding patterns in arc line spectra

class pypeit.core.wavecal.autoid.ArchiveReid(spec, lamps, par, ech_fixed_format=False, ok_mask=None, measured_fwhms=None, use_unknowns=True, debug_all=False, debug_peaks=False, debug_xcorr=False, debug_reid=False, debug_fits=False, orders=None, nonlinear_counts=10000000000.0)[source]

Bases: object

Algorithm to wavelength calibrate spectroscopic data based on an archive of wavelength solutions.

Parameters

  • spec (float ndarray shape of (nspec, nslits) or (nspec)) – Array of arc spectra for which wavelength solutions are desired.

  • lamps (list) – List of arc lamps to be used for wavelength calibration. E.g., [‘ArI’,’NeI’,’KrI’,’XeI’]

  • par (WaveSolutionPar) – Key parameters that drive the behavior of the wavelength-solution algorithms.

  • ech_fixed_format (bool) – Set to True if this is a fixed format echelle spectrograp. The code will then align the archive_arc and the extracted arc for each order for the reidentification.

  • ok_mask (numpy.ndarray, optional) – Integer array with the list of valid spectra spec to use. If None, all spectra are used.

  • measured_fwhms (ndarray, optional) – Array of FWHM (in pixels) measured from the arc lines. Shape (nslit,)

  • use_unknowns (bool, default = True, optional) – If True, arc lines that are known to be present in the spectra, but have not been attributed to an element+ion, will be included in the fit.

  • debug_all (bool, optional) – Convenience parameter that turns on all debugging. Setting debug_all to True is equivalent to setting debug_peaks, debug_xcorr, debug_reid, and debug_fits to True.

  • debug_peaks (bool, optional) – Debug the line identification in the arcline spectra. See debug parameter in func:pypeit.core.wavecal.wvutils.arc_lines_from_spec.

  • debug_xcorr (bool, default = False, optional) – Show plots useful for debugging the cross-correlation used for shift/stretch computation.

  • debug_reid (bool, default = False, optional) – Show plots useful for debugging the line reidentification

  • debug_fits (bool, optional) – Show the arc-line fit debugging plot. See arc_fit_qa().

  • orders (numpy.ndarray, optional) – Order numbers for the provided spectra. Used to match against the relevant archived spectrum for echelle spectrographs. Shape must be \((N_{\rm spec},)\) and these must be provided if ech_fixed_format is True.

  • nonlinear_counts (float, default = 1e10) – For arc line detection: Arc lines above this saturation threshold are not used in wavelength solution fits because they cannot be accurately centroided

debug_peaks

Debug the peak finding.

Type

bool

.. todo::

  • Fill in the rest of the attributes.

get_results()[source]
class pypeit.core.wavecal.autoid.HolyGrail(spec, lamps, par=None, ok_mask=None, islinelist=False, outroot=None, debug=False, verbose=False, binw=None, bind=None, nstore=1, use_unknowns=True, nonlinear_counts=None, spectrograph=None)[source]

Bases: object

General algorithm to wavelength calibrate spectroscopic data

Parameters

  • spec (ndarray) – 2D array of arcline spectra (nspec,nslit)

  • lamps (list) – List of arc lamps to be used for wavelength calibration. E.g., [‘ArI’,’NeI’,’KrI’,’XeI’]

  • par (ParSet or dict, default = default parset, optional) – This is the parset par[‘calibrations’][‘wavelengths’]. A dictionary with the corresponding parameter names also works.

  • ok_mask (ndarray, optional) – Array of good slits

  • islinelist (bool, optional) – Is lines a linelist (True), or a list of ions (False)

  • outroot (str, optional) – Name of output file

  • debug (bool, optional) – Used to debug the algorithm

  • verbose (bool, optional) – If True, the final fit will print out more detail as the RMS is refined, and lines are rejected. This is mostly helpful for developing the algorithm.

  • binw (ndarray, optional) – Set the wavelength grid when identifying the best solution

  • bind (ndarray, optional) – Set the dispersion grid when identifying the best solution

  • nstore (int, optional) – The number of “best” initial solutions to consider

  • use_unknowns (bool, optional) – If True, arc lines that are known to be present in the spectra, but have not been attributed to an element+ion, will be included in the fit.

  • spectrograph (str, optional) – Spectrograph name

Returns

  • all_patt_dict (list of dicts) – A list of dictionaries, which contain the results from the preliminary pattern matching algorithm providing the first guess at the ID lines

  • all_final_fit (list of dicts) – A list of dictionaries, which contain the full fitting results and final best guess of the line IDs

cross_match(good_fit, detections)[source]

Cross-correlate the spectra across all slits to ID all of the lines.

Parameters

good_fit (ndarray (bool)) – Indicates which slits are deemed to be a good fit (although, sometimes a bad fit can be labelled as a good fit). To remedy this, the true good fits are determined in this routine.

cross_match_order(good_fit)[source]

Using the solutions of all orders, identify the good solutions, and refit the bad ones!

TODO: This function needs work… The first few lines of code successfully pick up the good orders, but we need a new routine that (based on an estimated central wavelength and dispersion) can successfully ID all of the lines.

finalize_fit(detections)[source]

Once the best IDs have been found for each slit, perform a final fit to all slits and save the results

get_results()[source]
get_use_tcent(corr, tcent_ecent)[source]

Grab the lines to use

Parameters

  • corr – int Set if pixels correlate with wavelength (corr==1) or anticorrelate (corr=-1)

  • tcent_ecent – A list [tcent, ecent] indicating which detection list should be used. Note that if arr_err is set then the weak keyword is ignored.

Returns

arr, err

Return type

tuple

get_use_tcent_old(corr, cut=True, arr_err=None, weak=False)[source]

Grab the lines to use

Parameters

  • corr – int Set if pixels correlate with wavelength (corr==1) or anticorrelate (corr=-1)

  • arr_err – A list [tcent, ecent] indicating which detection list should be used. Note that if arr_err is set then the weak keyword is ignored.

  • weak – bool, optional If True, return the weak lines

  • cut – bool, optional Cut on the lines according to significance

Returns

arr, err

Return type

tuple

report_final()[source]

Print out the final report of the wavelength calibration

report_prelim(slit, best_patt_dict, best_final_fit)[source]
results_brute(tcent_ecent, poly=3, pix_tol=0.5, detsrch=5, lstsrch=5, wavedata=None)[source]

Need some docs here. I think this routine generates the patterns, either triangles are quadrangles.

Parameters

  • tcent_ecent (list of ndarrays, [tcent, ecent]) –

  • poly – algorithms to use for pattern matching. Only triangles (3) and quadrangles (4) are supported

  • optional – algorithms to use for pattern matching. Only triangles (3) and quadrangles (4) are supported

  • pix_tol – tolerance that is used to determine if a pattern match is successful (in units of pixels)

  • optional – tolerance that is used to determine if a pattern match is successful (in units of pixels)

  • detsrch – Number of lines to search over for the detected lines

  • optional – Number of lines to search over for the detected lines

  • lstsrch – Number of lines to search over for the detected lines

  • optional – Number of lines to search over for the detected lines

  • wavedata –

  • optional –

  • arrerr –

  • optional –

results_kdtree(use_tcent, res, dindex, lindex, ordfit=2)[source]
run_brute(min_nlines=10)[source]

Run through the parameter space and determine the best solution

run_brute_loop(slit, tcent_ecent, wavedata=None)[source]
run_kdtree(polygon=4, detsrch=7, lstsrch=10, pixtol=5)[source]

KD Tree algorithm to wavelength calibrate spectroscopic data. Currently, this is only designed for ThAr lamp spectra. See the ‘run_brute’ function if you want to calibrate longslit spectra.

Parameters

  • polygon (int) –

    Number of sides to the polygon used in pattern matching. For example:

    • polygon=3 –> trigon (two anchor lines and one floating line)

    • polygon=4 –> tetragon (two anchor lines and two floating lines)

    • polygon=5 –> pentagon (two anchor lines and three floating lines)

  • detsrch (int) –

    Number of consecutive detected lines used to generate a pattern. For example, if detsrch is 4, then for a trigon, the following patterns will be generated (assuming line #1 is the left anchor):

    • 1 2 3: (in this case line #3 is the right anchor)

    • 1 2 4: (in this case line #4 is the right anchor)

    • 1 3 4: (in this case line #4 is the right anchor)

  • lstsrch (int) – Number of consecutive lines in the linelist used to generate a pattern. See example above for detsrch

  • pixtol (float) – Tolerance used to find good patterns. An acceptable match if the closest distance to a pattern is < pixtol/npix, where npix is the number of pixels in the spectral direction. Ideally, this should depend on the pattern…

set_grids(ngridw=300, ngridd=3000)[source]
solve_patterns(slit, bestlist, tcent_ecent)[source]
Parameters

  • slit (int) – The ID of the slit

  • bestlist (list or ndarray) –

    A 5 element list, each containing a numpy.ndarray, with the following values required for each index:

    0: central wavelength of the pattern 1: central dispersion of pattern 2: sign of the pattern (note, sign = 1 [-1] if pixels correlate [anticorrelate] with wavelength 3: index of the full list of patterns that were created from the detected arc lines 4: index of the full list of patterns that were created from the line list.

  • tcent_ecent (list) – A list [tcent, ecent] indicating which detection list should be used. Note that if arr_err is set then the weak keyword is ignored.

Returns

Dictionary containing information about the best patterns.

Return type

patt_dict (dict)

solve_slit(slit, psols, msols, tcent_ecent, nstore=1, nselw=3, nseld=3)[source]

Need some docs here. I think this routine creates a 2d histogram of the patterns and searches for the most represented wave_cen and log10(disp). Then it attempts to fit each value determined (default of 1) to try to figure out if it is a reasonable fit.

Parameters

  • slit –

  • psols –

  • msols –

  • tcent_ecent – list, [tcent, ecent]

  • nstore – Number of pattern matches to store and fit

  • nselw – All solutions around the best central wavelength solution within +- nselw are selected to be fit

  • nseld – All solutions around the best log10(dispersion) solution within +- nseld are selected to be fit

Returns

patt_dict, final_dict

Return type

tuple

pypeit.core.wavecal.autoid.arc_fit_qa(waveFit, outfile=None, ids_only=False, title=None, log=True)[source]

QA for Arc spectrum

Parameters

Returns:

pypeit.core.wavecal.autoid.echelle_wvcalib(spec, orders, spec_arxiv, wave_arxiv, lamps, par, ok_mask=None, use_unknowns=True, debug_all=False, debug_peaks=False, debug_xcorr=False, debug_reid=False, debug_fits=False, nonlinear_counts=10000000000.0)[source]

Algorithm to wavelength calibrate echelle data based on a predicted or archived wavelength solution

Parameters

  • spec (numpy.ndarray_, shape=(nspec, norders)) – Array of arc spectra for each order for which wavelength solutions are desired.

  • orders (numpy.ndarray_) – Order numbers for the provided spectra. Used to match against the relevant archived spectrum for echelle spectrographs. Shape must be \((N_{\rm orders},)\)

  • spec_arxiv (numpy.ndarray_, shape=(nspec, narxiv) or (nspec)) – Collection of archival arc spectra for which wavelength solution and line identifications are known

  • wave_arxiv (float ndarray shape (nspec, narxiv) or (nspec)) – Wavelength solutions for the archival arc spectra spec_arxiv

  • lamps (list) – List of arc lamps to be used for wavelength calibration. E.g., [‘ArI’,’NeI’,’KrI’,’XeI’]

  • par (WaveSolutionPar) – Key parameters that drive the behavior of the wavelength-solution algorithms.

  • ok_mask (numpy.ndarray, optional) – Integer array with the list of valid spectra spec to use. If None, all spectra are used.

  • use_unknowns (bool, default = True, optional) – If True, arc lines that are known to be present in the spectra, but have not been attributed to an element+ion, will be included in the fit.

  • debug_all (bool, optional) – Convenience parameter that turns on all debugging. Setting debug_all to True is equivalent to setting debug_peaks, debug_xcorr, debug_reid, and debug_fits to True.

  • debug_peaks (bool, optional) – Debug the line identification in the arcline spectra. See debug parameter in func:pypeit.core.wavecal.wvutils.arc_lines_from_spec.

  • debug_xcorr (bool, default = False, optional) – Show plots useful for debugging the cross-correlation used for shift/stretch computation.

  • debug_reid (bool, default = False, optional) – Show plots useful for debugging the line reidentification

  • debug_fits (bool, optional) – Show the arc-line fit debugging plot. See arc_fit_qa().

  • nonlinear_counts (float, default = 1e10) – For arc line detection: Arc lines above this saturation threshold are not used in wavelength solution fits because they cannot be accurately centroided

Returns

  • all_patt_dict (dict) – Arc lines pattern dictionary with some information about the IDs as well as the cross-correlation values

  • wv_calib (dict) – Dictionary containing the wavelength solution for each order

pypeit.core.wavecal.autoid.full_template(spec, lamps, par, ok_mask, det, binspectral, nsnippet=2, measured_fwhms=None, debug_xcorr=False, debug_reid=False, x_percentile=50.0, template_dict=None, debug=False, nonlinear_counts=10000000000.0)[source]

Method of wavelength calibration using a single, comprehensive template spectrum

The steps are:

  1. Load the template and rebin, as necessary

  2. Cross-correlate input spectrum and template to find the shift between the two

  3. Loop on snippets of the input spectrum to ID lines using reidentify()

  4. Fit with fitting.iterative_fitting()

Parameters

  • spec – ndarray (nspec, nslit) Spectra to be calibrated

  • lamps – list List of arc lamps to be used for wavelength calibration. E.g., [‘ArI’,’NeI’,’KrI’,’XeI’]

  • par – WavelengthSolutionPar ParSet Calibration parameters

  • ok_mask – ndarray, bool Mask of indices of good slits

  • det – int Detector index

  • binspectral – int Binning of the input arc in the spectral dimension

  • nsnippet – int, optional Number of snippets to chop the input spectrum into when ID’ing lines This deals with differences due to non-linearity between the template and input spectrum.

  • measured_fwhms – ndarray, optional Array of FWHM (in pixels) measured from the arc lines. Shape (nslit,)

  • x_percentile – float, optional Passed to reidentify to reduce the dynamic range of arc line amplitudes

  • template_dict (dict, optional) – Dict containing tempmlate items, largely for development

Returns

dict

Dict of wavelength calibration solutions

Return type

wvcalib

pypeit.core.wavecal.autoid.match_qa(arc_spec, tcent, line_list, IDs, scores, outfile=None, title=None, path=None)[source]
Parameters

  • arc_spec –

  • tcent –

  • line_list –

  • IDs –

  • scores –

  • outfile –

  • title –

  • path –

pypeit.core.wavecal.autoid.measure_fwhm(spec, sigdetect=10.0, fwhm=5.0)[source]

Measure the arc lines FWHM, i.e, approximate spectral resolution

Parameters
Returns

Measured arc lines FWHM in binned pixels of the input arc image

Return type

float

pypeit.core.wavecal.autoid.reidentify(spec, spec_arxiv_in, wave_soln_arxiv_in, line_list, nreid_min, det_arxiv=None, detections=None, cc_thresh=0.8, cc_local_thresh=0.8, match_toler=2.0, nlocal_cc=11, nonlinear_counts=10000000000.0, sigdetect=5.0, fwhm=4.0, debug_xcorr=False, debug_reid=False, debug_peaks=False)[source]

Determine a wavelength solution for a set of spectra based on archival wavelength solutions

Parameters

  • spec (float ndarray shape (nspec)) – Arc spectrum for which wavelength identifications are desired.

  • spec_arxiv (float ndarray shape (nspec, narxiv) or (nspec)) – Collection of archival arc spectra for which wavelength solution and line identifications are known

  • wave_soln_arxiv (float ndarray shape (nspec, narxiv) or (nspec)) – Wavelength solutions for the archival arc spectra spec_arxiv

  • line_list (astropy table) – The arc line list used for thew wavelength solution in pypeit format.

  • nreid_min (int) – Minimum number of times that a given candidate reidentified line must be properly matched with a line in the arxiv to be considered a good reidentification. If there is a lot of duplication in the arxiv of the spectra in question (i.e. multislit) set this to a number like 2-4. For echelle this depends on the number of solutions in the arxiv. For fixed format echelle (ESI, X-SHOOTER, NIRES) set this 1. For an echelle with a tiltable grating, it will depend on the number of solutions in the arxiv.

  • Parameters (Optional) –

  • ------------------- –

  • (optional) (det_arxiv) –

    an ndarray of size determined by the number of lines that were detected.

    Arc line pixel locations in the spec_arxiv spectra that were used in combination with line identifications from the line list to determine the wavelength solution wave_soln_arxiv.

  • detections (float ndarray, default = None) – An array containing the pixel centroids of the lines in the arc as computed by the pypeit.core.arc.detect_lines code. If this is set to None, the line detection will be run inside the code.

  • cc_thresh (float, default = 0.8) – Threshold for the global cross-correlation coefficient between an input spectrum and member of the archive required to attempt reidentification. Spectra from the archive with a lower cross-correlation are not used for reidentification

  • cc_local_thresh (float, default = 0.8) – Threshold for the local cross-correlation coefficient, evaluated at each reidentified line, between an input spectrum and the shifted and stretched archive spectrum above which a line must be to be considered a good line for reidentification. The local cross-correlation is evaluated at each candidate reidentified line (using a window of nlocal_cc), and is then used to score the the reidentified lines to arrive at the final set of good reidentifications

  • match_toler (float, default = 2.0) – Matching tolerance in pixels for a line reidentification. A good line match must match within this tolerance to the the shifted and stretched archive spectrum, and the archive wavelength solution at this match must be within match_toler dispersion elements from the line in line list.

  • n_local_cc (int, defualt = 11) – Size of pixel window used for local cross-correlation computation for each arc line. If not an odd number one will be added to it to make it odd.

debug_xcorr: bool, default = False

Show plots useful for debugging the cross-correlation used for shift/stretch computation

debug_reid: bool, default = False

Show plots useful for debugging the line reidentification

Returns

  • (detections, spec_cont_sub, patt_dict)

  • detections (ndarray,) – Pixel locations of arc lines detected.

  • spec_cont_sub (ndarray) – Array of continuum subtracted arc spectra

  • patt_dict (dict) – Arc lines pattern dictionary with some information about the IDs as well as the cross-correlation values

  • Revision History

  • —————-

  • November 2018 by J.F. Hennawi. Built from an initial version of cross_match code written by Ryan Cooke.

pypeit.core.wavecal.autoid.report_final(nslits, all_patt_dict, detections, wv_calib, ok_mask, bad_slits)[source]

Print out the final report for wavelength calibration

Parameters

  • nslits (int) – Number of slits or ders

  • all_patt_dict (dict) – Dictionary containing reidentification information.

  • detections (dict) – Dictionary containing the lines that were detected.

  • wv_calib (dict) – Dictionary holding the wavelength solutions for each slit/orders

  • ok_mask (ndarray, bool) – Mask of indices of good slits

  • bad_slits (ndarray, bool) – List of slits that are bad

pypeit.core.wavecal.autoid.results_kdtree_nb(use_tcent, wvdata, res, residx, dindex, lindex, nindx, npix, ordfit=1)[source]

A numba speedup of the results_kdtree function in the General class (see above). For all of the acceptable pattern matches, estimate the central wavelength and dispersion, and record the index in the linelist and the corresponding indices of the detected lines.

Parameters

  • use_tcent (ndarray) – detected lines

  • wvdata (ndarray) – the linelist

  • res (list) – A flattened list of the results of the ball tree query from the KDTree (this contains all acceptable matches) This needs to be a flattened list for numba

  • residx (list) – This contains the original indices of the unflattened (i.e. nested) ‘res’ list

  • dindex (ndarray) – Indices of the lines in the detected lines for all patterns

  • lindex (ndarray) – Indices of the lines in the linelist for all patterns

  • nindx (int) – Number of acceptable pattens

  • npix (int) – Number of pixels in the spectral direction

  • ordfit (int) – Order of the polynomial used to fit the pixel/wavelength IDs

Returns

  • dind (ndarray) – Indices of the lines in the detected lines that were used for each acceptable pattern

  • lind (linelist index of patterns) – Indices of the lines in the linelist that were used for each corresponding pattern

  • wvcent (ndarray) – Central wavelength of each pattern

  • wvdisp (ndarray) – Central dispersion of each pattern

pypeit.core.wavecal.autoid.set_fwhm(par, measured_fwhm=None)[source]

Set the value of the arc lines FWHM by choosing between the provided parset and the measured_fwhm

Parameters

  • par (WaveSolutionPar) – Key parameters that drive the behavior of the wavelength-solution algorithms.

  • measured_fwhm (float) – Measured arc lines FWHM in binned pixels of the input arc image

Returns

Chosen arc lines FWHM in binned pixels of the input arc image

Return type

float