pypeit.core.ramp module

Instrument-independent up-the-ramp fitting.

Detectors read out non-destructively “up the ramp” record a series of reads of the accumulating charge rather than a single integration. Fitting the per-pixel count rate to those reads (with cosmic-ray/jump detection) recovers the flux while rejecting jumps and beating down read noise. This module holds the parts of that machinery that do not depend on any particular instrument: the count-rate fit, the single-read-noise calibration, the effective-read-noise formula, the worker-thread dispatch, and the on-disk sidecar convention for caching a fitted 2D count-rate image next to a reduction.

The numerical fitting is delegated to the fitramp algorithm of Brandt (2024, https://arxiv.org/abs/2404.01326; reference implementation: https://github.com/t-brandt/fitramp), vendored in pypeit.ext.fitramp.

Instruments plug in through the ramp hooks on Spectrograph (_load_ramp and _count_reads), which read an instrument’s raw cube into the reads, covariance, and header this module operates on; MMT/MMIRS (pypeit.spectrographs.mmt_mmirs) is the first such instrument.

pypeit.core.ramp._calibrate_sigma_block(rng, sub, ndiffs, nx, covar, sig_guess, chisq)

Fit one block of subsampled rows without jump detection and store the per-pixel chi-squared.

A module-level worker for calibrate_sigma(), bound to its arrays with functools.partial() and dispatched over row blocks by _fit_rows(); it writes into the preallocated chisq array.

Parameters:

• rng (tuple) – (row_start, row_stop) block of sub to fit.

• sub (numpy.ndarray) – Subsampled differences, shape (ndiffs, nrows, nx).

• ndiffs (int) – Number of resultant differences.

• nx (int) – Detector width.

• covar (Covar) – Covariance object matching sub.

• sig_guess (float) – Initial single-read-noise guess in electrons.

• chisq (numpy.ndarray) – Preallocated (nrows, nx) output written in place.

pypeit.core.ramp._fit_ramp_block(rng, diffs, ndiffs, nx, covar, sig, countrate)

Fit one block of detector rows with jump detection and store the fitted count rates.

A module-level worker for fit_ramp(), bound to its arrays with functools.partial() and dispatched over row blocks by _fit_rows(); it writes into the preallocated countrate array.

Parameters:

• rng (tuple) – (row_start, row_stop) block of diffs to fit.

• diffs (numpy.ndarray) – Scaled resultant differences, shape (ndiffs, ny, nx).

• ndiffs (int) – Number of resultant differences.

• nx (int) – Detector width.

• covar (Covar) – Covariance object matching diffs.

• sig (float) – Single-read noise in electrons.

• countrate (numpy.ndarray) – Preallocated (ny, nx) output written in place.

pypeit.core.ramp._fit_rows(diffs, nb, workers, worker)

Dispatch worker((row_start, row_stop)) over blocks of detector rows.

The per-pixel ramp fit is independent, so the ny rows of diffs are split into contiguous blocks of nb rows and each block is handed to worker, which is expected to write its results into a preallocated output array. With workers > 1 the blocks are fit concurrently in a thread pool; NumPy releases the GIL during the element-wise arithmetic that dominates fit_ramps(), so threads scale until the memory bus saturates (empirically ~3x at 6 threads).

Parameters:

• diffs (numpy.ndarray) – Scaled resultant differences, shape (ndiffs, ny, nx).

• nb (int) – Number of rows per block.

• workers (int) – Number of worker threads (already resolved; 1 runs serially).

• worker (callable) – Called with a (row_start, row_stop) tuple for each block.

pypeit.core.ramp._resolve_workers(workers)

Resolve the requested number of ramp-fit worker threads.

None selects min(6, os.cpu_count()); any explicit value is passed through (clamped to at least 1). The per-pixel fit is memory-bandwidth bound, so more than ~6 threads does not help and can hurt.

pypeit.core.ramp.calibrate_sigma(diffs, covar, sig_guess=9.0, nrows=200, workers=None, nb=16, return_err=False, n_boot=200, seed=1234)

Calibrate the single-read noise from ramp differences.

Fits a subsample of rows without jump detection (two-pass, with the count-rate guess clamped to non-negative values to debias the second pass) and rescales sig_guess so that the median chi-squared matches the expected degrees of freedom (ngroups - 2).

Parameters:

• diffs (numpy.ndarray) – Scaled resultant differences, shape (ndiffs, ny, nx), electrons.

• covar (Covar) – Covariance object matching diffs.

• sig_guess (float, optional) – Initial guess for the single-read noise in electrons.

• nrows (int, optional) – Number of evenly spaced rows (from the central 80% of the detector) to include in the calibration.

• workers (int, optional) – Number of worker threads; None selects min(6, os.cpu_count()) and 1 disables threading.

• nb (int, optional) – Number of subsampled rows fit per fit_ramps() call.

• return_err (bool, optional) – If True, also return a bootstrap estimate of the uncertainty on the calibrated noise (see n_boot), for inverse-variance weighting when combining multiple darks.

• n_boot (int, optional) – Number of bootstrap resamples of the per-pixel chi-squared ensemble used to estimate the uncertainty when return_err is True.

• seed (int, optional) – Seed for the bootstrap resampling, so the uncertainty is deterministic.

Returns:

Calibrated single-read noise in electrons (unclamped). If return_err is True, a (sigma, sigma_err) tuple is returned instead, with sigma_err the bootstrap standard deviation.

Return type:

float or tuple

pypeit.core.ramp.effective_ronoise(sig, ngroups)

Effective read noise of an up-the-ramp-fitted image.

For N uniformly spaced reads with single-read noise sig, the read-noise contribution to the total-count uncertainty of the fitted slope is sig * sqrt(12 (N-1) / (N (N+1))) (Brandt 2024a).

Parameters:

• sig (float) – Single-read noise in electrons.

• ngroups (int) – Number of reads in the ramp.

Returns:

Effective read noise in electrons.

Return type:

float

pypeit.core.ramp.fit_ramp(diffs, covar, sig, workers=None, nb=16)

Fit all pixels of a ramp, in blocks of rows, with jump detection.

The per-pixel fit is independent, so the detector is fit in blocks of nb rows (rows folded into the pixel axis of fit_ramps()) and, for workers > 1, the blocks are fit concurrently. Results are numerically identical to a row-by-row fit.

Parameters:

• diffs (numpy.ndarray) – Scaled resultant differences, shape (ndiffs, ny, nx), electrons.

• covar (Covar) – Covariance object matching diffs.

• sig (float) – Single-read noise in electrons.

• workers (int, optional) – Number of worker threads; None selects min(6, os.cpu_count()) and 1 disables threading.

• nb (int, optional) – Number of rows fit per fit_ramps() call.

Returns:

countrate – Fitted count rates in e-/s, shape (ny, nx).

Return type:

numpy.ndarray

pypeit.core.ramp.ramp_diffs(reads, covar)

Compute scaled resultant differences from ramp reads.

Parameters:

• reads (numpy.ndarray) – Reads in time order, shape (ngroups, ny, nx), in electrons.

• covar (Covar) – Covariance object providing the time intervals delta_t.

Returns:

diffs – (reads[i+1] - reads[i]) / covar.delta_t[i], shape (ngroups-1, ny, nx), in e-/s.

Return type:

numpy.ndarray

pypeit.core.ramp.rampfit_fresh(rampfit_file, raw_file)

Check whether a preprocessed image exists and is up to date.

A preprocessed image is fresh when its RAWMTIME header card matches the raw cube’s current modification time to within 1 second. Missing or unreadable files (or files without the card) are not fresh.

Parameters:

• rampfit_file (str, Path) – Path to the candidate preprocessed image.

• raw_file (str, Path) – Path to the source raw cube.

Returns:

True if the preprocessed image can be used in place of the cube.

Return type:

bool

pypeit.core.ramp.rampfit_path(raw_file, redux_path, rampfit_dir='RampFit')

Return the preprocessed-image path for a raw up-the-ramp cube.

Preprocessed 2D count-rate images live in the ramp-fit directory inside the reduction directory (alongside Calibrations, Science, etc.), named after the raw cube with a _rampfit marker inserted before the extension so the processed product is distinct from the raw file. The directory name is set by the [rdx] rampfit_dir parameter.

Parameters:

• raw_file (str, Path) – Path to the raw cube.

• redux_path (str, Path) – Path to the reduction directory.

• rampfit_dir (str, optional) – Name of the ramp-fit subdirectory, relative to redux_path.

Returns:

<redux_path>/<rampfit_dir>/<raw stem>_rampfit<ext>

Return type:

Path

pypeit.core.ramp.write_rampfit(rampfit_file, rate, hdu, sig, eff_ronoise, ngroups, raw_mtime, raw_file=None)

Write a preprocessed 2D count-rate image.

The output carries a copy of the raw primary header (hdu[0]) plus the cards RAMPFIT (marker), RAMPSIG, RAMPRON, NGROUPS, RAWMTIME, and RAWPATH (the resolved path of the source raw cube), and a single image extension holding the fitted count rate in e-/s (float32) under a copy of the raw first-extension header (hdu[1]), so all metadata used by pypeit_setup is preserved. RAWPATH lets the freshness check reject a sidecar that a same-named raw cube from a different directory would otherwise map onto (some instruments’ raw file names are only unique within a program, not globally).

The file is written atomically: the FITS data are first written to a temporary file in the same directory, which is then renamed onto the final path. This ensures that a crash or full disk mid-write can never leave a truncated sidecar whose header (and hence its RAWMTIME freshness check) is already flushed, which would otherwise be treated as fresh forever while being unreadable.

Parameters:

• rampfit_file (str, Path) – Output path; its parent directory is created if needed.

• rate (numpy.ndarray) – Fitted count rate in e-/s, trimmed to the data section.

• hdu (astropy.io.fits.HDUList) – Opened source raw cube (headers are copied from hdu[0] and hdu[1]).

• sig (float) – Single-read noise used in the fit (electrons).

• eff_ronoise (float) – Effective read noise of the fitted image (electrons).

• ngroups (int) – Number of reads in the source ramp (recorded in the NGROUPS card).

• raw_mtime (float) – Modification time of the source raw cube.

• raw_file (str, Path, optional) – Source raw cube; when given, its resolved path is recorded in the RAWPATH card for the freshness check.

Raises:

OSError – If the output directory cannot be created or the file cannot be written.