pypeit.core.flat module

Core module for methods related to flat fielding.

pypeit.core.flat.construct_illum_profile(norm_spec, spat_coo, slitwidth, spat_gpm=None, spat_samp=5, illum_iter=0, illum_rej=None, debug=False)[source]

Construct the slit illumination profile.

Provided an image with the spectral response normalized out, this iteratively filters and rejects the flat-field data to construct the empirical slit illumination profile. The data are collapsed spectrally using the provided coordinate array to construct a 1D profile. Nominally, the provided spatial coordinates and good-pixel mask should be for a single slit.

The iterations involve constructing the illumination profile using illum_filter() and then rejecting deviant residuals. Each rejection iteration recomputes the standard deviation and pixels to reject from the full input set (i.e., rejected pixels are not kept between iterations). Rejection iterations are only performed if illum_iter > 0 and illum_rej is not None.

Parameters:

norm_spec (numpy.ndarray) – Flat-field image (2D array) with the spectral response normalized out.
spat_coo (numpy.ndarray) – An image with the slit spatial coordinates, expected to be with respect to a single slit and span the full image region selected by the good-pixel mask (spat_gpm). Shape must match norm_spec.
slitwidth (float) – Fiducial slit width used to set the median-filter window size.
spat_gpm (numpy.ndarray, optional) – The good-pixel mask that selects the pixels to include in the slit illumination profile calculation. If None, all pixels in the provided images are used. For virtually all practical purposes, this array should be provided.
spat_samp (int, float, optional) – Spatial sampling for slit illumination function. This is the width of the median filter in detector pixels used to determine the slit illumination function, and thus sets the minimum scale on which the illumination function will have features.
illum_iter (int, optional) – Iteratively construct the slit illumination profile and reject outliers. To include rejection iterations, this must be larger than 0, and you have to provide the sigma threshold (illum_rej); otherwise, no iterations are performed.
illum_rej (float, optional) – Sigma rejection threshold for iterations. If None, no rejection iterations will be performed, regardless of the value of illum_iter.
debug (bool, optional) – Construct plots output to the screen that show the result of each iteration. Regardless of this flag, no plots are shown if there are no iterations.

Returns:

Five numpy.ndarray objects are returned:

A boolean array with the pixels used in the construction of the illumination profile. Shape is identical to norm_spec.

A vector with the length of the number of good pixels (sum of the first returned object) with the indices that sorts the flattened list of good coordinate values.

A vector with the sorted coordinate data.

A vector with the data sorted by the respective coordinates.

A vector with the slit illumination profile.

To construct the empirical 2D illumination profile:

illum = np.zeros(norm_spec.shape, dtype=float)
illum[_spat_gpm] = profile[np.argsort(srt)]

where norm_spec is the input array, _spat_gpm is the first returned object, srt is the second returned object, and profile is the last returned object.

Return type:

tuple

pypeit.core.flat.flatfield(sciframe, flatframe, varframe=None)[source]

Field flatten the input image.

This is a simple scaling of the provided science frame by the inverse of the flat frame.

Parameters:

sciframe (numpy.ndarray) – The science frame to flat-field correct
flatframe (numpy.ndarray) – The flat-field image to use for the correction. Shape must match sciframe.
varframe (numpy.ndarray, optional) – The variance in the science frame (sciframe). If provided, the flat-fielding operation is propagated to the variance, and the result is returned. Shape must match sciframe.

Returns:

A tuple of two or three numpy.ndarray objects. The first two are the rescaled science frame and a boolean bad-pixel mask indicating where the flat frame was not positive. If a variance image is provided, the third object is the propagated variance in the rescaled science frame.

Return type:

tuple

pypeit.core.flat.illum_filter(spat_flat_data_raw, med_width)[source]

Filter the flat data to produce the empirical illumination profile.

This is primarily a convenience method for construct_illum_profile(). The method first median filters with a window set by med_width and then Gaussian-filters the result with a kernel sigma set to be the maximum of 0.5 or med_width/20.

Parameters:

spat_flat_data_raw (numpy.ndarray) – Raw flat data collapsed along the spectral direction.
med_width (int) – Width of the median filter window.

Returns:

Returns the filtered spatial profile of the flat data.

Return type:

numpy.ndarray

pypeit.core.flat.illum_profile_spectral_poly(rawimg, waveimg, slitmask, slitmask_trim, model, slit_illum_ref_idx=0, gpmask=None, thismask=None, nbins=20, debug=False)[source]

Use a polynomial fit to control points along the spectral direction to determine the relative spectral illumination of all slits. Currently, this routine is only used for image slicer IFUs.

Parameters:

rawimg (numpy.ndarray) – Image data that will be used to estimate the spectral relative sensitivity
waveimg (numpy.ndarray) – Wavelength image
slitmask (numpy.ndarray) – A 2D int mask, the same shape as rawimg, indicating which pixels are on a slit. A -1 value indicates not on a slit, while any pixels on a slit should have the value of the slit spatial ID number.
slitmask_trim (numpy.ndarray) – Same as slitmask, but the slit edges are trimmed.
model (numpy.ndarray) – A model of the rawimg data.
slit_illum_ref_idx (int) – Index of slit that is used as the reference.
gpmask (numpy.ndarray, optional) – Boolean good pixel mask (True = Good)
thismask (numpy.ndarray, optional) – A boolean mask (True = good) that indicates all pixels where the scaleImg should be constructed. If None, the slitmask that is generated by this routine will be used.
nbins (int) – Number of bins in the spectral direction to sample the relative spectral sensitivity
debug (bool) – If True, some plots will be output to test if the fitting is working correctly.

Returns:

scale_model – An image containing the appropriate scaling

Return type:

numpy.ndarray

pypeit.core.flat.linear_interpolate(x1, y1, x2, y2, x)[source]

Interplate or extrapolate between two points.

Given a line defined two points, \((x_1,y_1)\) and \((x_2,y_2)\), return the \(y\) value of a new point on the line at coordinate \(x\).

This function is meant for speed. No type checking is performed and the only check is that the two provided ordinate coordinates are not numerically identical. By definition, the function will extrapolate without any warning.

Parameters:

x1 (float) – First abscissa position
y1 (float) – First ordinate position
x2 (float) – Second abscissa position
y3 (float) – Second ordinate position
x (float) – Abcissa for new value

Returns:

Interpolated/extrapolated value of ordinate at \(x\).

Return type:

float

pypeit.core.flat.poly_map(rawimg, rawivar, waveimg, slitmask, slitmask_trim, modelimg, deg=3, slit_illum_ref_idx=0, gpmask=None, thismask=None, debug=False)[source]

Use a polynomial fit to control points along the spectral direction to construct a map between modelimg and rawimg. Currently, this routine is only used for image slicer IFUs.

This problem needs to be recast into a chi-squared problem, that can take advantage of polynomial fitting. Refer to the following for variable assignments: https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html where: y = science/model w = model/science_error resid = w*(y-f(x)) Then, iterate over this. The reason to iterate is that there should be a mapping between science and science_error. Once we have an estimate of f(x), we can do the following: spl = spline(science, science_error) new_science_error = spl(model*f(x)) Now iterate a few times until new_science_error (and the fit) is converged.

Parameters:

rawimg (numpy.ndarray) – Image data that will be used to estimate the spectral relative sensitivity
rawivar (numpy.ndarray) – Inverse variance image of rawimg
waveimg (numpy.ndarray) – Wavelength image
slitmask (numpy.ndarray) – A 2D int mask, the same shape as rawimg, indicating which pixels are on a slit. A -1 value indicates not on a slit, while any pixels on a slit should have the value of the slit spatial ID number.
slitmask_trim (numpy.ndarray) – Same as slitmask, but the slit edges are trimmed.
model (numpy.ndarray) – A model of the rawimg data.
slit_illum_ref_idx (int) – Index of slit that is used as the reference.
gpmask (numpy.ndarray, optional) – Boolean good pixel mask (True = Good)
thismask (numpy.ndarray, optional) – A boolean mask (True = good) that indicates all pixels where the scaleImg should be constructed. If None, the slitmask that is generated by this routine will be used.
debug (bool) – If True, some plots will be output to test if the fitting is working correctly.

Returns:

modelmap (numpy.ndarray) – A 2D image with the same shape as rawimg, that contains the modelimg mapped to the rawimg
relscale (numpy.ndarray) – A 2D image with the same shape as rawimg, that contains the relative spectral sensitivity

pypeit.core.flat.smooth_scale(arr, wave_ref=None, polydeg=None, sn_smooth_npix=None)[source]

Smooth the relative sensitivity array using a polynomial fit or a boxcar filter.

Parameters:

arr (numpy.ndarray) – Array containing the relative sensitivity
wave_ref (numpy.ndarray, optional) – Wavelength array corresponding to the relative sensitivity array. Only used if polydeg is not None.
polydeg (int, optional) – Degree of the polynomial fit to the relative sensitivity array. If None, a boxcar filter will be used.
sn_smooth_npix (int, optional) – Number of pixels to use for the boxcar filter. Only used if polydeg is None.

Returns:

arr_smooth – Smoothed relative sensitivity array

Return type:

numpy.ndarray

pypeit.core.flat.sorted_flat_data(data, coo, gpm=None)[source]

Sort a set of data by the provided coordinates.

Parameters:

data (numpy.ndarray) – Data array with arbirary shape and data type.
coo (numpy.ndarray) – Relevant coordinate array. Shape must match data.
gpm (numpy.ndarray, optional) – Good-pixel mask for array. Used to select data (where gpm is True) to sort and return. Shape must match data. If None, all data is used.

Returns:

Four numpy.ndarray objects are returned:

A boolean array with the pixels used in the sorting. Shape is identical to data. If gpm is provided, this is identicall (i.e., not a copy) of the input array; otherwise, it is np.ones(data.shape, dtype=bool).

A vector with the length of numpy.sum(gpm) with the indices that sorts the flattened list of good coordinate values.

A vector with the sorted coordinate data.

A vector with the data sorted by the respective coordinates.

To reconstruct the input data array for the good pixels:

_data = np.zeros(data.shape, dtype=data.dtype)
_data[gpm] = srt_data[np.argsort(srt)]

where data is the input array, gpm is the first returned object, srt is the second returned object, and srt_data is the last returned object of this method.

Return type:

tuple

pypeit.core.flat.tweak_slit_edges(left, right, spat_coo, norm_flat, thresh=0.93, maxfrac=0.1, debug=False)[source]

Adjust slit edges based on the normalized slit illumination profile.

Parameters:

left (numpy.ndarray) – Array with the left slit edge for a single slit. Shape is \((N_{\rm spec},)\).
right (numpy.ndarray) – Array with the right slit edge for a single slit. Shape is \((N_{\rm spec},)\).
spat_coo (numpy.ndarray) – Spatial pixel coordinates in fractions of the slit width at each spectral row for the provided normalized flat data. Coordinates are relative to the left edge (with the left edge at 0.). Shape is \((N_{\rm flat},)\). Function assumes the coordinate array is sorted.
norm_flat (numpy.ndarray) – Normalized flat data that provide the slit illumination profile. Shape is \((N_{\rm flat},)\).
thresh (float, optional) – Threshold of the normalized flat profile at which to place the two slit edges.
maxfrac (float, optional) – The maximum fraction of the slit width that the slit edge can be adjusted by this algorithm. If maxfrac = 0.1, this means the maximum change in the slit width (either narrowing or broadening) is 20% (i.e., 10% for either edge).
debug (bool, optional) – Show flow interrupting plots that show illumination profile in the case of a failure and the placement of the tweaked edge for each side of the slit regardless.

Returns:

Returns six objects:

The threshold used to set the left edge

The fraction of the slit that the left edge is shifted to the right

The adjusted left edge

The threshold used to set the right edge

The fraction of the slit that the right edge is shifted to the left

The adjusted right edge

Return type:

tuple