pypeit.collate module

This module contains code for collating multiple 1d spectra by source object.

class pypeit.collate.SourceObject(spec1d_obj, spec1d_header, spec1d_file, spectrograph, match_type)

Bases: object

A group of reduced spectra from the same source object. This contains the information needed to coadd the spectra and archive the metadata.

An instance is initiated with the first spectra of the group. Additional spectra can be compared with this object to see if it matches using the match method, and are added to it if they do.

Parameters:

• spec1d_obj (pypeit.specobj.SpecObj) – The initial spectra of the group as a SpecObj.

• spec1d_header (astropy.io.fits.Header) – The header for the first spec1d file in the group.

• spec1d_file (str) – Filename of the first spec1d file in the group.

• spectrograph (pypeit.spectrographs.spectrograph.Spectrograph) – The spectrograph that was used to take the data.

• match_type (str) – How spectra should be compared. ‘ra/dec’ means the spectra should be compared using the sky coordinates in RA and DEC. ‘pixel’ means the spectra should be compared by the spatial pixel coordinates in the image.

Variables:

• spec_obj_list (list of pypeit.spectrographs.spectrograph.Spectrograph) – The list of spectra in the group as SpecObj objects.

• spec1d_file_list (list of str) – The pathnames of the spec1d files in the group.

• spec1d_header_list – (list of astropy.io.fits.Header): The headers of the spec1d files in the group

_config_key_match(header)

Check to see if the configuration keys from a spec1d file match the ones for this SourceObject.

Parameters:

header (astropy.io.fits.Header) – Header from a spec1d file.

Returns:

True if the configuration keys match,

false if they do not.

Return type:

bool

classmethod build_source_objects(specobjs_list, spec1d_files, match_type)

Build a list of SourceObjects from a list of spec1d files. There will be one SourceObject per SpecObj in the resulting list (i.e. no combining or collating is done by this method).

Parameters:

• specobjs_list (list of pypeit.specobjs.SpecObjs) – List of SpecObjs objects to build from.

• spec1d_files (list of str) – List of spec1d filenames corresponding to each SpecObjs object.

• match_type (str) – What type of matching the SourceObjects will be configured for. Must be either ‘ra/dec’ or ‘pixel’

Returns:

A list of uncollated SourceObjects with one SpecObj per SourceObject.

Return type:

list of SourceObject

combine(other_source_object)

Combine this SourceObject with another. The two objects must be from the same spectrograph and use the same match type.

Parameters:

other_source_object (SourceObject) – The other object to combine with.

Returns:

This SourceObject, now combined with other_source_object.

Return type:

(SourceObject)

match(spec_obj, spec1d_header, tolerance, unit=Unit('arcsec'))

Determine if a SpecObj matches this group within the given tolerance. This will also compare the configuration keys to make sure the SpecObj is compatible with the ones in this SourceObject.

Parameters:

• spec_obj (pypeit.specobj.SpecObj) – The SpecObj to compare with this SourceObject.

• spec1d_header (astropy.io.fits.Header) – The header from the spec1d that dontains the SpecObj.

• tolerance (float) – Maximum distance that two spectra can be from each other to be considered to be from the same source. Measured in floating point pixels or as an angular distance (see unit1 argument).

• unit (astropy.units.Unit) – Units of tolerance argument if match_type is ‘ra/dec’. Defaults to arcseconds. Igored if match_type is ‘pixel’.

Returns:

True if the SpecObj matches this group, False otherwise.

Return type:

bool

pypeit.collate.build_coadd_file_name(source_object)

Build the output file name for coadding. The filename convention is J<hmsdms+dms>_<instrument name>_<YYYYMMDD>.fits when matching by RA/DEC and SPAT_<spatial position>_<instrument name>_<YYYYMMDD>.fits when matching by pixel position. The date portion may be <YYYYMMDD-YYYMMDD> if the files coadded span more than one date.

Currently instrument_name is taken from spectrograph.camera

Returns:

The name of the coadd output file.

Return type:

str

pypeit.collate.coadd(par, coaddfile, source)

coadd the spectra for a given source.

Parameters:

• par (Collate1DPar) – Parameters for the coadding

• source (SourceObject) – The SourceObject with information on which files and spectra to coadd.

pypeit.collate.collate_1d(par, spectrograph, tolerance, spec1d_files)

pypeit.collate.collate_spectra_by_source(source_list, tolerance, unit=Unit('arcsec'))

Given a list of spec1d files from PypeIt, group the spectra within the files by their source object. The grouping is done by comparing the position of each spectra (using either pixel or RA/DEC) using a given tolerance.

Parameters:

• source_list (list of SourceObject) – A list of source objects, one SpecObj per object, ready for collation.

• tolerance (float) – Maximum distance that two spectra can be from each other to be considered to be from the same source. Measured in floating point pixels or as an angular distance (see unit argument).

• unit (astropy.units.Unit) – Units of tolerance argument if match_type is ‘ra/dec’. Defaults to arcseconds. Ignored if match_type is ‘pixel’.

Returns:

The collated spectra as SourceObjects.

Return type:

list

pypeit.collate.copy_spec1d_to_outdir(spec1d_files, outdir)

Copy the spec1d files to the requested outdir, preserving the originals

Parameters:

• spec1d_files (list of str) – List of spec1d files generated by PypeIt.

• outdir (str) – Directory to copy the spec1d files.

Returns:

The pathnames of the newly copied files.

Return type:

list of str

pypeit.collate.create_report_archive(par)

Create an report archive with the desired metadata information.

Metadata is written to three files in the ipac format:

• collate_report.dat contains metadata to report on the coadded output files from the collate process. Like coadded_files.dat it may have more than one row per output file. This file is always written to the current directory.

Returns:

Object for archiving files and/or metadata.

Return type:

ArchiveDir

pypeit.collate.exclude_source_objects(source_objects, exclude_map, par)

Exclude SourceObject objects based on a slit exclude map and the user’s parameters.

Parameters:

• source_objects (list) – List of uncollated SourceObject objects to filter. There should only be one SpecObj per SourceObject.

• exclude_map (dict) – Mapping of excluded slit ids to the reasons they should be excluded.

• par (PypeItPar) – Configuration parameters from the command line or a configuration file.

Returns:

Tuple containing two lists:

• filtered_objects (list): A list of SourceObject with any excluded ones removed.

• excluded_messages (list): A list of messages explaining why some source objects were excluded.

Return type:

tuple

pypeit.collate.find_slits_to_exclude(spec2d_files, par)

Find slits that should be excluded according to the input parameters.

The slit mask ids are returned in a map alongside the text labels for the flags that caused the slit to be excluded.

Parameters:

• spec2d_files (list) – List of spec2d files to build the map from.

• par (PypeItPar) – Parameters from a .collate1d file

Returns:

Mapping of slit mask ids to the flags that caused the slit to be excluded.

Return type:

dict

pypeit.collate.find_spec2d_from_spec1d(spec1d_files)

Find the spec2d files corresponding to the given list of spec1d files. This looks for the spec2d files in the same directory as the spec1d files. It will exit with an error if a spec2d file cannot be found.

Parameters:

spec1d_files (list of str) – List of spec1d files generated by PypeIt.

Returns:

List of the matching spec2d files.

Return type:

list of str

pypeit.collate.flux(par, spectrograph, spec1d_files, failed_fluxing_log)

Flux calibrate spec1d files using archived sens func files.

Parameters:

• par (PypeItPar) – Parameters for collating, fluxing, and coadding.

• spectrograph (Spectrograph) – Spectrograph for the files to flux.

• spec1d_files (list of str) – List of spec1d files to flux calibrate.

• failed_fluxing_log (list of str) – Return parameter describing any failures that occurred when fluxing.

Returns:

The spec1d files that were successfully flux calibrated.

Return type:

list of str

pypeit.collate.get_report_metadata(object_header_keys, spec_obj_keys, file_info)

Gets the metadata from a SourceObject instance used building a report on the results of collation. It is intended to be wrapped in by functools partial object that passes in object_header_keys and spec_obj_keys. file_info is then passed as in by the ArchiveMetadata object. Unlike the other get_*_metadata functions, this is not used for archiving; it is used for reporting on the results of collating.

If another type of file is added to the ArchiveMetadata object, the file_info argument will not be a SourceObject, In this case, a list of None values are returned.

Parameters:

• object_header_keys (list of str) – The keys to read fom the spec1d headers from the SourceObject.

• spec_obj_keys (list of str) – The keys to read from the (SpecObj) objects in the SourceObject.

• file_info (SourceObject) – The source object containing the headers, filenames and SpecObj information for a coadd output file.

Returns:

A tuple of two lists:.

• data_rows (list of list): The metadata rows built from the source object.

• files_to_copy (iterable): An list of tuples of files to copy. Because this function is not used for archving data, this is always None.

Return type:

tuple

pypeit.collate.read_spec1d_files(par, spec1d_files, failure_log)

Read spec1d files.

Parameters:

• (obj (par) – pypeit.par.pypeitpar.PypeItPar): Parameters for collating, fluxing, and coadding.

• spec1d_files (list of str) – List of spec1d files to read.

• failure_log (list of str) – Return parameter describing any failures that occurred when reading.

Returns:

The SpecObjs objects that were successfully read. list of str: The spec1d files that were successfully read.

Return type:

list of str

pypeit.collate.refframe_correction(par, spectrograph, spec1d_files, spec1d_failure_log)

pypeit.collate.write_warnings(par, excluded_obj_log, failed_source_log, spec1d_failure_log)

Write gathered warning messages to a collate_warnings.txt file.

Parameters:

• excluded_obj_log (list of str) – Messages about which objects were excluded from collating and why.

• failed_source_log (list of str) – Messages about which objects failed coadding and why.

• spec1d_failure_log (list of str) – Messages about failures with spec1d files and why.