pypeit.state.science_status module

Record and derive science-frame reduction state for RunPypeItState.

This module translates the in-memory products of the science reduction (SpecObjs, AllSpec2DObj, SlitTraceSet) — and, when no live state file is available, the on-disk products — into the science portion of the reduction state. It is kept separate from pypeit.state.run_state (which stays free of product imports) and is used both by the live run (exposure.reduce_exposure) and by the no-state-file derive path (the Dashboard and pypeit_status).

The science reduction has four macro-steps: process -> findobj -> skysub -> extract (matching pypeit_reduce_by_step). findobj and skysub are produced together by findobj_on_exposure and are recorded together.

Priority for the derive path: the final Science products (spec2d/spec1d) are authoritative; Intermediate/ files (written only by pypeit_reduce_by_step) are used only to fill steps the Science products do not yet cover.

pypeit.state.science_status._basename_from_product(path, prefix)[source]

Extract <basename> from a <prefix>_<basename>.fits product name.

Parameters:

  • path (str, pathlib.Path) – The product file path (only its name is used).

  • prefix (str) – The product prefix, e.g. 'spec2d' or 'spec1d'.

Returns:

The exposure basename, or None if the name does not match <prefix>_*.fits.

Return type:

str or None

pypeit.state.science_status._basename_lookup(fitstbl)[source]

Build {basename: (objtype, calib_id, comb_id)} for the science and standard frames in fitstbl.

Parameters:

fitstbl (PypeItMetaData or None) – Metadata.

Returns:

Mapping; empty if fitstbl is None.

Return type:

dict

pypeit.state.science_status._derive_from_intermediate(state, inter_dir, lookup, chk_version=False)[source]

Fill not-yet-covered steps from Intermediate/ files written by pypeit_reduce_by_step: sciImg -> process, spec1d_*_all -> findobj, Sky -> skysub. Science products always win (only steps still pending/running are filled).

Parameters:

  • state (RunPypeItState) – State to update.

  • inter_dir (pathlib.Path) – The Intermediate/ directory.

  • lookup (dict) – {basename: (objtype, calib_id, comb_id)}.

  • chk_version (bool, optional) – Strict version check.

pypeit.state.science_status._det_from_name(detname)[source]

Parse a PypeIt single-detector name (e.g. 'DET01') into its integer.

Parameters:

detname (str) – Detector name.

Returns:

The 1-indexed detector, or None for mosaics ('MSC..') or unparseable names.

Return type:

int or None

pypeit.state.science_status._exposure_entries(state, spectrograph, fitstbl, frames, detectors, calib_ID=-1)[source]

Get-or-create the science state entry for each detector of one exposure.

This is the boilerplate shared by the record_* functions: for each detector it resolves the exposure basename, object type, and detector name, reads the (comb_id, bkg_id) group IDs, and gets or creates the matching ScienceFrameState entry.

Parameters:

  • state (RunPypeItState) – State to update.

  • spectrograph (Spectrograph) – The spectrograph object.

  • fitstbl (PypeItMetaData) – Metadata table.

  • frames (list) – Frame indices combined into this exposure.

  • detectors (list) – Detectors/mosaics being reduced.

  • calib_ID (int, optional) – Calibration group ID (only used if the entry must be created).

Yields:

  • det (int or tuple) – The detector or detector mosaic.

  • detname (str) – The PypeIt detector name (e.g. 'DET01').

  • entry (ScienceFrameState) – The (created or existing) state entry for (basename, det).

pypeit.state.science_status._exposure_ids(fitstbl, frames)[source]

Return the (comb_id, bkg_id) group IDs for an exposure.

Parameters:

  • fitstbl (PypeItMetaData) – The metadata table (indexed by frame row).

  • frames (list) – Frame indices combined into this exposure; the first is used.

Returns:

(comb_id, bkg_id) as ints, with a negative (unset) value mapped to None; (None, None) if either column is absent or unreadable.

Return type:

tuple

pypeit.state.science_status._obj_from_specobj(sobj, extracted)[source]

Build a ScienceObj from a SpecObj.

Parameters:

  • sobj (SpecObj) – A single detected/extracted object.

  • extracted (bool) – Whether this object has a 1D extraction (sets extracted and reads the extraction S/N).

Returns:

ScienceObj.

pypeit.state.science_status._raw_files_by_basename(fitstbl)[source]

Map each science/standard exposure basename to its contributing raw frame filename(s) from the metadata table.

Parameters:

fitstbl (PypeItMetaData or None) – Metadata.

Returns:

{basename: [filename, ...]}; empty if fitstbl is None.

Return type:

dict

pypeit.state.science_status._safe_float(value)[source]

Cast a value to float, never raising.

Parameters:

value – The value to cast (any type, or None).

Returns:

float(value), or None if value is None or cannot be cast.

Return type:

float or None

pypeit.state.science_status._slit_status_map(slits, flag)[source]

Map each slit spat_id to 'fail' (flag set) or 'success'.

Parameters:

  • slits (SlitTraceSet) – The slit set; may be None.

  • flag (str) – Bitmask flag to test ('BADSKYSUB' / 'BADEXTRACT').

Returns:

{spat_id (int): status (str)}; empty if slits is None.

Return type:

dict

pypeit.state.science_status.derive_science_from_disk(state, redux_dir, fitstbl=None, chk_version=False)[source]

Populate state.science from the on-disk products, preferring the final Science products and falling back to Intermediate/ files.

Science spec2d/spec1d are authoritative (a present product implies its step, and the preceding steps, succeeded). Intermediate/ files (written only by pypeit_reduce_by_step) fill steps the Science products do not yet cover. process is inferred success whenever a later step’s product exists.

Note

Mosaic (MSC..) detectors are not yet handled by the derive path (single DET.. detectors only); the live run records mosaics fully.

Parameters:

  • state (RunPypeItState) – State to populate (its existing science entries are updated/extended in place).

  • redux_dir (str) – The reduction directory (containing Science/ and optionally Intermediate/).

  • fitstbl (PypeItMetaData, optional) – Used to tag each exposure’s objtype/calib_id/comb_id.

  • chk_version (bool, optional) – Strict datamodel version check when reading products.

Returns:

The updated state (modified in place).

Return type:

RunPypeItState

pypeit.state.science_status.planned_science_from_fitstbl(fitstbl)[source]

Return the list of planned science/standard frames from the .pypeit metadata — one record per exposure, {frame, objtype, calib_id, comb_id, raw_files} — for the Dashboard to seed before any reduction.

Separated from the seeding (seed_planned_science_entries()) so the (expensive) metadata read can be done once and cached, then re-seeded cheaply on later state loads.

Parameters:

fitstbl (PypeItMetaData or None) – Metadata.

Returns:

One dict per science/standard exposure (empty if fitstbl is None).

Return type:

list

pypeit.state.science_status.record_extract(state, spectrograph, fitstbl, frames, detectors, all_spec2d, all_specobjs_extract, spec1d_file=None, spec2d_file=None)[source]

Record the extract step for each detector: status, per-object extraction metrics, per-slit extraction status, and product paths.

Parameters:

  • state (RunPypeItState) – State to update.

  • spectrograph (Spectrograph) – The spectrograph object.

  • fitstbl (PypeItMetaData) – Metadata table.

  • frames (list) – Frame indices of this exposure.

  • detectors (list) – Detectors/mosaics being reduced.

  • all_spec2d (AllSpec2DObj) – 2D products (keyed by detector name).

  • all_specobjs_extract (SpecObjs) – Extracted objects across all detectors.

  • spec1d_file (str, optional) – Path to the final spec1d file.

  • spec2d_file (str, optional) – Path to the final spec2d file.

pypeit.state.science_status.record_findobj(state, spectrograph, fitstbl, frames, detectors, calib_ID, all_specobjs_find, all_slits)[source]

Record the findobj and skysub steps for each detector: status, object count, per-object find metrics, and per-slit sky status.

Parameters:

  • state (RunPypeItState) – State to update.

  • spectrograph (Spectrograph) – The spectrograph object.

  • fitstbl (PypeItMetaData) – Metadata table.

  • frames (list) – Frame indices of this exposure.

  • detectors (list) – Detectors/mosaics being reduced.

  • calib_ID (int) – Calibration group ID.

  • all_specobjs_find (SpecObjs) – Objects found across all detectors (selected per detector via DET).

  • all_slits (list) – Per-detector SlitTraceSet (same order as detectors).

pypeit.state.science_status.record_process(state, spectrograph, fitstbl, frames, detectors, calib_ID)[source]

Mark the process step success for each detector of an exposure.

Parameters:

  • state (RunPypeItState) – State to update.

  • spectrograph (Spectrograph) – The spectrograph object.

  • fitstbl (PypeItMetaData) – Metadata table.

  • frames (list) – Frame indices combined into this exposure.

  • detectors (list) – Detectors/mosaics being reduced.

  • calib_ID (int) – Calibration group ID.

pypeit.state.science_status.seed_planned_science_entries(state, planned, group_dets)[source]

Seed planned science/standard frame entries into state from a precomputed planned-frame list (see planned_science_from_fitstbl()), so the Dashboard’s Science view lists upcoming frames before they are reduced and keeps showing them after a calibration (re)build replaces the state file.

One entry is created per (frame, detector), with all four macro-steps left pending and the contributing raw frame(s) recorded (so the science (Re)Build can launch pypeit_reduce_by_step). Existing entries (e.g. already reduced, or derived from products) are left untouched apart from backfilling raw_files. Mosaic detectors are skipped (the derive path handles single DET.. detectors only).

Parameters:

  • state (RunPypeItState) – State to populate.

  • planned (list) – Planned-frame dicts (frame, objtype, calib_id, comb_id, raw_files); build one from a metadata table with planned_science_from_fitstbl().

  • group_dets (dict) – {calib_group: [detector, ...]} — the detectors to plan per calibration group (typically the calibration (group, det) pairs the dashboard already derived).

Returns:

The updated state (modified in place).

Return type:

RunPypeItState