pypeit.dashboard.model module

Headless (Qt-free) data/logic layer for the PypeIt Dashboard.

This module holds the dashboard’s PypeIt-facing logic so it can be unit tested with plain pytest (no display required), keeping the Qt views thin. It provides the cheap reduction metadata used by the header banner and the full reduction-state layer (loading or deriving a RunPypeItState).

class pypeit.dashboard.model.DashboardModel(pypeit_file, redux_path=None, derive=True)

Bases: object

Headless (Qt-free) model for one PypeIt reduction.

It acquires a RunPypeItState by source priority (load <root>_state.json if present; otherwise derive it from the on-disk calibration products, the same way pypeit_status does), and exposes a clean, normalized API the Qt views consume — a status table, the (calib_id, det) pairs, the pipeline-aware step order, and graceful edge states. It never raises for a missing/empty/malformed state: such cases are reported via load_status.

Parameters:

• pypeit_file (str, pathlib.Path) – Path to the .pypeit reduction file.

• redux_path (str, optional) – Reduction directory (where the state file lives). Defaults to the directory containing pypeit_file.

• derive (bool, optional) – If True (default) and no state file is present, derive the state via PypeIt (requires the raw data). Set False to skip deriving (e.g. CI tests without RAW_DATA): the model then reports not_started when no state file exists.

Variables:

• pypeit_file (pathlib.Path) – The .pypeit path.

• redux_dir (pathlib.Path) – The reduction directory.

• state_path (pathlib.Path) – The expected <root>_state.json path.

• header_info (HeaderInfo) – Header metadata, or None.

• run_state (RunPypeItState) – The state, or None when unavailable.

• load_status (str) – One of the LOAD_* class attributes.

• error (str) – An error message when relevant, else None.

LOAD_DERIVED = 'derived'

The state was derived from the on-disk products (no state file).

LOAD_ERROR = 'error'

Deriving the state raised an error.

LOAD_FILE_NOT_FOUND = 'file_not_found'

The named .pypeit file is missing.

LOAD_MALFORMED = 'malformed'

A state file is present but could not be read/validated.

LOAD_NOT_STARTED = 'not_started'

No state file exists and the state was not derived.

LOAD_STATE_FILE = 'state_file'

The state was loaded from an existing <root>_state.json file.

SCIENCE_COLUMNS = ['frame', 'detector', 'calib', 'objtype', 'process', 'findobj', 'skysub', 'extract', 'nobj', 'spec2d', 'spec1d']

The per-frame science-status columns consumed by the Science view (mirrors get_science_status()).

STATUS_COLUMNS = ['calibration_group', 'detector', 'step', 'in_pipeline', 'required', 'status', 'output_file']

The normalized status-table columns the views consume.

_ABSENT = 'absent'

Status value used for an entry that is not present in the state (get_status() reports these with a "--" sentinel).

_CALIB_CLASS_BY_PYPELINE = {'Echelle': <class 'pypeit.calibrations.MultiSlitCalibrations'>, 'MultiSlit': <class 'pypeit.calibrations.MultiSlitCalibrations'>, 'SlicerIFU': <class 'pypeit.calibrations.IFUCalibrations'>}

The Calibrations subclass that defines the calibration-step order, keyed on the spectrograph pypeline value. New pipeline types should be added here explicitly; an unknown pypeline falls back to the full set of known steps (see default_steps()).

_PLANNED_SCIENCE_CACHE = {}

Per-.pypeit cache (class-level, so it spans model reloads within a session) of the planned science/standard frames, keyed by the .pypeit path. Computing the planned frames requires building the reduction metadata, which is expensive; caching it means the build happens at most once per session, so re-seeding planned frames on every state reload (live monitoring, post-(re)build) is cheap.

_acquire_state(derive)

Acquire the reduction state by source priority: an existing state file first, otherwise (optionally) derive it from the on-disk products. Sets run_state and load_status.

Parameters:

derive (bool) – Whether to derive the state when no state file is present.

_derive_state()

Derive the reduction state from the on-disk calibration and science products (used when no state file exists). Failures are reported via load_status, never raised.

_load_state_file(derive)

Load and validate the <root>_state.json state file, reporting a malformed/unreadable file via load_status instead of raising.

Parameters:

derive (bool) – Whether a one-time metadata build is permitted when seeding the planned science frames (see _seed_planned_science()).

_planned_science_frames(allow_build=False)

Return the cached list of planned science/standard frames for this .pypeit file. On a cache miss, build the metadata to compute it only when allow_build is True (so live-monitoring reloads and CI loads never trigger a heavy, possibly-failing build).

Parameters:

allow_build (bool, optional) – Permit a one-time metadata build on a cache miss (True at launch; False on refresh/CI).

Returns:

Planned-frame dicts (see planned_science_from_fitstbl()); empty when uncached and a build is not permitted/possible.

Return type:

list

_seed_planned_science(allow_build=False)

Seed the planned science/standard frames into run_state (from the cache, or a one-time build when allow_build), keyed to the calibration (group, det) pairs in the current state. Best-effort: never raises into the load/derive path.

Parameters:

allow_build (bool, optional) – Permit a one-time metadata build on a cache miss.

_set_malformed(error)

Record a malformed/partial/schema-mismatched state file.

Parameters:

error (Exception) – The error raised while reading/validating the state file.

calib_det_pairs()

Enumerate the (calibration_group, detector) pairs present in the state, in sorted order, for the scope drop-downs/selectors.

Returns:

(calib_id, det) tuples; det is left in its raw form (an int or a mosaic tuple) for the view to format.

Return type:

list

property calib_dir

The reduction’s Calibrations/ directory (where output files live).

Returns:

<redux_dir>/Calibrations.

Return type:

pathlib.Path

calib_file_path(prefix, group, det, ext='fits')

Construct the path to a calibration file by naming convention (<prefix>_<setup>_<group>_<detname>.<ext>) — used when the viewer needs a different file than a step’s output_file (e.g. the Edges_* file for the slits step, or the Slits_* file required by pypeit_chk_scattlight).

Parameters:

• prefix (str) – File-type prefix (e.g. Edges, Slits).

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

• ext (str, optional) – Extension (fits or fits.gz).

Returns:

The constructed path (None if the setup is unknown).

Return type:

pathlib.Path or None

calibrations_ready(group, det)

Whether the required calibrations for one (group, det) are all built successfully — the precondition for a science (Re)Build: until the calibrations a science frame depends on exist, its (Re)Build is disabled.

Parameters:

• group (int) – The calibration group ID of the science frame.

• det (int, tuple, list) – The detector (int) or mosaic the science frame is on.

Returns:

True only if at least one required, in-pipeline calibration step exists for (group, det) and all such steps have a success/complete status; False otherwise (including when no matching calibration entries are known).

Return type:

bool

default_steps()

Return the active spectrograph’s calibration steps, in pipeline order (specific to the pypeline; see _CALIB_CLASS_BY_PYPELINE), including bpm.

A pypeline without an entry in _CALIB_CLASS_BY_PYPELINE (e.g. a newly added pipeline type) falls back to the full set of known calibration steps, so the dashboard remains usable until the new pipeline’s order is registered.

Returns:

The ordered step names, or [] if the pipeline is unknown (e.g. the .pypeit file was not found).

Return type:

list

det_name(det)

Return a human-readable name for a detector or mosaic, for display in the views (drop-downs, navigator).

Parameters:

det (int, tuple, list) – The detector (int) or mosaic (tuple/list of ints), as stored in the state.

Returns:

The PypeIt detector name (e.g. DET01, MSC01) when it can be resolved, else a plain readable fallback (Det 1 / Mosaic (1, 5)).

Return type:

str

has_science()

Whether any science-frame state is available.

Returns:

True if the state has at least one science entry.

Return type:

bool

is_stale()

Whether the loaded *_state.json looks out of date relative to the .pypeit file or the calibration outputs.

Returns:

True if the state file’s mtime is older than the .pypeit file or the newest file in Calibrations/; False if there is no state file or nothing newer.

Return type:

bool

is_started()

Whether the reduction has any calibration-state entries.

Returns:

True if a state with at least one entry is available.

Return type:

bool

property log_path

The reduction .log file that run_pypeit / pypeit_run_to_calibstep write to (used by the single-run lock to detect an active run via the log’s modification time).

Returns:

<pypeit_root>.log (next to the .pypeit file, matching pypeit_run_to_calibstep’s default log path).

Return type:

pathlib.Path

output_path(step, group, det)

Full path to a step’s processed output file, or None.

Parameters:

• step (str) – Calibration step name.

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

Calibrations/<output_file>.

Return type:

pathlib.Path or None

science_frame_entry(frame, det)

Return the raw ScienceFrameState for one (frame, det), or None.

Parameters:

• frame (str) – Exposure basename.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

The matching pydantic science entry, or None.

Return type:

object

science_object_qa_files(frame, det, slitid)

Return all per-object QA PNGs for one object’s slit, so the per-object table can open them. The science QA PNGs encode the slit as S{spat_id:04d}, matching ScienceObj.slitid.

Discovery is purely by the QA file-naming convention (no fixed list of QA types), so newly added per-object QA figures are picked up with no dashboard change.

Parameters:

• frame (str) – Exposure basename.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

• slitid (int) – The object’s slit (SPAT_ID).

Returns:

Sorted pathlib.Path of the object’s QA PNGs (empty if none, or if slitid is None).

Return type:

list

science_object_table(frame, det)

Return the per-object science rows for one frame (ScienceObj).

Parameters:

• frame (str) – Exposure basename.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

One dict per detected object, keys objid, slitid, spat_pixpos, fwhm, snr_find, s2n, sign, extracted. Empty if none.

Return type:

list

science_qa_files(frame, det)

Return all on-disk QA PNGs for one science (frame, det): the per-object obj_prof/obj_trace figures and the frame-level spec_flex_* flexure figures.

Derived at view time by globbing <redux>/QA/PNGs/ for files whose name starts with the frame basename and contains the detector name — so no science-state change is needed and it works for both live and derived state.

Parameters:

• frame (str) – Exposure basename (the science-table key).

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

Sorted pathlib.Path of the matching QA PNGs (empty if the QA/PNGs directory or matches are absent).

Return type:

list

science_slit_table(frame, det)

Return the per-slit science rows for one frame (ScienceSlit).

Parameters:

• frame (str) – Exposure basename.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

One dict per slit (sorted), keys slit, status, nobj. Empty if none.

Return type:

list

science_table()

Return the normalized per-frame science-status table the Science view renders (one row per (frame, detector)).

Returns:

Columns SCIENCE_COLUMNS; empty when there are no science entries.

Return type:

astropy.table.Table

slit_table(step, group, det)

Return the per-slit/order rows for a step that tracks them (slits / wv_calib / tilts / flats).

Parameters:

• step (str) – Calibration step name.

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

One dict per slit (sorted by slit id), with keys slit, status, and step-specific metrics: center/slitord_id (slits), rms (wv_calib/tilts), or corrections (flats — a dict name → {'mean':…, 'rms':…}). Empty if the step has no per-slit data.

Return type:

list

status_table()

Return the normalized calibration-status table the views consume.

Built on get_status() but normalized: required becomes a real bool (or None), the "--" sentinels become None/"absent", an in_pipeline column is added (membership in default_steps()), and an empty/unavailable state yields an empty table.

Returns:

Columns STATUS_COLUMNS; empty when the reduction has not started or the state is unavailable.

Return type:

astropy.table.Table

step_entry(step, group, det)

Return the raw RunPypeItState entry for one (step, group, det), or None if absent.

Parameters:

• step (str) – Calibration step name (e.g. wv_calib).

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

The matching pydantic step entry, or None.

Return type:

object

step_metrics(step, group, det)

Return the entry-level metrics for a step, as a plain dict for the detail panel (per-slit metrics come from slit_table()).

Parameters:

• step (str) – Calibration step name.

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

Metric name → value (empty if the entry is absent or has no entry-level metrics). bias → mean/std; slits → nslits; flats → corrections/pixelflat_source.

Return type:

dict

step_order(include_bpm=False)

Return the pipeline-aware step order for the calibration button row.

Parameters:

include_bpm (bool, optional) – Include the bpm step. Default False — bpm runs internally but has no standalone output/QA, so it is omitted from the button row.

Returns:

The ordered step names.

Return type:

list

step_output_files(step, group, det)

The existing Calibrations/ file(s) a (re)build of this step would overwrite — named explicitly in the clobber confirmation dialog.

Most steps write a single output_file; slits writes both Slits_* (its output_file) and the companion Edges_*. Only files that currently exist on disk are returned (a step with no output yet is a fresh build, nothing to overwrite).

Parameters:

• step (str) – Calibration step name.

• group (int) – Calibration group ID.

• det (int, tuple, list) – Detector (int) or mosaic (tuple/list).

Returns:

Existing pathlib.Path outputs (possibly empty).

Return type:

list

class pypeit.dashboard.model.HeaderInfo(pypeit_file, spectrograph, setup, path, redux_dir)

Bases: object

Cheap, no-reduction metadata describing a reduction, used to populate the dashboard’s shared header banner.

Variables:

• pypeit_file (str) – Name of the .pypeit file (basename, for display).

• spectrograph (str) – The spectrograph name (PYP_SPEC), e.g. shane_kast_blue.

• setup (str) – The setup/configuration identifier, e.g. A. May be None if the .pypeit file does not define one.

• path (str) – The reduction path: the spectrograph pypeline value, shown verbatim (e.g. MultiSlit, Echelle, SlicerIFU).

• redux_dir (str) – The reduction directory (absolute path).

path: str = <dataclasses._MISSING_TYPE object>

pypeit_file: str = <dataclasses._MISSING_TYPE object>

redux_dir: str = <dataclasses._MISSING_TYPE object>

setup: str = <dataclasses._MISSING_TYPE object>

spectrograph: str = <dataclasses._MISSING_TYPE object>

pypeit.dashboard.model._parse_pypeit(pypeit_file, redux_path=None)

Parse a .pypeit file once, returning both the header metadata and the instantiated spectrograph (the latter is reused for, e.g., detector-name formatting).

Parameters:

• pypeit_file (str, pathlib.Path) – Path to the .pypeit reduction file.

• redux_path (str, optional) – Reduction directory. If None, the directory containing pypeit_file is used.

Returns:

• header_info (HeaderInfo) – The parsed header metadata.

• spec (Spectrograph) – The instantiated spectrograph.

Raises:

FileNotFoundError – If pypeit_file does not exist.

pypeit.dashboard.model.read_header_info(pypeit_file, redux_path=None)

Parse a .pypeit file and return the cheap metadata needed by the header banner, performing no reduction.

Parameters:

• pypeit_file (str, pathlib.Path) – Path to the .pypeit reduction file.

• redux_path (str, optional) – Reduction directory. If None, the directory containing pypeit_file is used.

Returns:

The parsed header metadata.

Return type:

HeaderInfo

Raises:

FileNotFoundError – If pypeit_file does not exist.