opticam_new.reduction.catalog ============================= .. py:module:: opticam_new.reduction.catalog Classes ------- .. autoapisummary:: opticam_new.reduction.catalog.Catalog Functions --------- .. autoapisummary:: opticam_new.reduction.catalog.save_catalog opticam_new.reduction.catalog.save_stacked_images opticam_new.reduction.catalog.save_backgrounds opticam_new.reduction.catalog.get_batches opticam_new.reduction.catalog.get_batch_size Module Contents --------------- .. py:class:: Catalog(out_directory, data_directory = None, c1_directory = None, c2_directory = None, c3_directory = None, rebin_factor = 1, flat_corrector = None, background = None, finder = None, threshold = 5, aperture_selector = np.median, remove_cosmic_rays = True, number_of_processors = cpu_count() // 2, show_plots = True, verbose = True) Create a catalog of sources from OPTICAM data. .. py:attribute:: verbose :value: True .. py:attribute:: out_directory .. py:attribute:: logger .. py:attribute:: data_directory :value: None .. py:attribute:: c1_directory :value: None .. py:attribute:: c2_directory :value: None .. py:attribute:: c3_directory :value: None .. py:attribute:: rebin_factor :value: 1 .. py:attribute:: flat_corrector :value: None .. py:attribute:: aperture_selector .. py:attribute:: threshold :value: 5 .. py:attribute:: remove_cosmic_rays :value: True .. py:attribute:: number_of_processors .. py:attribute:: show_plots :value: True .. py:attribute:: file_paths :value: [] .. py:attribute:: ignored_files :value: [] .. py:attribute:: colours :value: ['tab:blue', 'tab:orange', 'tab:green', 'tab:red', 'tab:purple', 'tab:brown', 'tab:pink',... .. py:attribute:: transforms .. py:attribute:: unaligned_files :value: [] .. py:attribute:: catalogs .. py:attribute:: psf_params .. py:method:: _scan_data_directory() Scan the data directory for files and extract the MJD, filter, binning, and gain from each file header. Raises ------ ValueError If more than 3 filters are found. ValueError If the binning is not consistent. .. py:method:: _get_header_info(file) Get the MJD, filter, binning, and gain from a file header. Parameters ---------- file : str The file path. Returns ------- Tuple[float, str, str, float] The BMJD, filter, binning, and gain dictionaries. Raises ------ KeyError If the file header does not contain the required keys. .. py:method:: _parse_header_results(results) Parse the results returned by self._get_header_info(). Parameters ---------- results : Tuple The results. Returns ------- Tuple[str, str] The filter dictionary (file : filter). Raises ------ ValueError If more than 3 filters are found. ValueError If the binning is not consistent. .. py:method:: _log_parameters() Log any and all object parameters to a JSON file. .. py:method:: _plot_time_between_files() Plot the times between each file for each camera. Parameters ---------- show : bool Whether to display the plot. .. py:method:: _set_psf_params(fltr) Set the PSF parameters for a given filter based on the catalog data. Parameters ---------- fltr : str The filter for which to set the PSF parameters. .. py:method:: _get_data(file, return_error = False) Get data from a file. Parameters ---------- file : str Directory path to file. return_error : bool, optional Whether to return the error array, by default False. Returns ------- NDArray | Tuple[NDArray, NDArray] The data array or the data and error arrays. .. py:method:: _get_source_coords_from_image(image, bkg = None, away_from_edge = False, n_sources = None) Get an array of source coordinates from an image in descending order of source brightness. Parameters ---------- image : NDArray The **non-background-subtracted** image from which to extract source coordinates. bkg : Background2D, optional The background of the image, by default None. If None, the background is estimated from the image. away_from_edge : bool, optional Whether to exclude sources near the edge of the image, by default False. n_sources : int, optional The number of source coordinates to return, by default `None` (all sources will be returned). Returns ------- NDArray The source coordinates in descending order of brightness. .. py:method:: create_catalogs(max_catalog_sources = 50, n_alignment_sources = 3, transform_type = 'translation', rotation_limit = None, translation_limit = None, scale_limit = None, overwrite = False, show_diagnostic_plots = False) Initialise the source catalogs for each camera. Some aspects of this method are parallelised for speed. Parameters ---------- max_catalog_sources : int, optional The maximum number of sources above the specified threshold that will be included in the catalog, by default 50. Only the brightest max_catalog_sources sources are included in the catalog. n_alignment_sources : int, optional The (maximum) number of sources to use for image alignment, by default 3. If `transform_type='translation'`, `n_alignment_sources` must be >= 1, and the brightest `n_alignment_sources` sources are used for image alignment. If `transform_type='affine'`, `n_alignment_sources` must be >= 3 and represents that *maximum* number of sources that *may* be used for image alignment. transform_type : Literal['affine', 'translation'], optional The type of transform to use for image alignment, by default 'translation'. 'translation' performs simple x, y translations, while 'affine' uses `astroalign.find_transform()`. rotation_limit : float, optional The maximum rotation limit (in degrees) for affine transformations, by default `None` (no limit). scale_limit : float, optional The maximum scale limit for affine transformations, by default `None` (no limit). translation_limit : float | Iterable[float], optional The maximum translation limit for transformations, by default `None` (no limit). Can be a scalar value that applies to both x- and y-translations, or an iterable where the first value defines the x-translation limit and the second value defines the y-translation limit. overwrite : bool, optional Whether to overwrite existing catalogs, by default False. show_diagnostic_plots : bool, optional Whether to show diagnostic plots, by default False. Diagnostic plots are saved to out_directory, so this parameter only affects whether the plots are displayed in the console. .. py:method:: _align_images(batch, reference_image_shape, reference_coords, transform_type, rotation_limit, scale_limit, translation_limit, n_alignment_sources) Align an image based on some reference coordinates. Parameters ---------- file: str The file path. reference_image : NDArray The reference image. reference_coords : NDArray The source coordinates in the reference image. transform_type : Literal['affine', 'translation'] The type of transform to use for image alignment. rotation_limit : float | None The maximum rotation limit (in degrees) for image alignment. scale_limit : float | None The maximum scaling limit for image alignment. translation_limit : Iterable[float] | None The maximum translation limit for image alignment. n_alignment_sources : int The (maximum) number of sources to use for image alignment. Returns ------- Tuple[List[float], float, float] The transform parameters, background median, and background RMS. .. py:method:: _valid_transform(file, transform, rotation_limit, scale_limit, translation_limit) Find whether a transform is valid given some transform limits. Parameters ---------- file : str The file being transformed. transform : SimilarityTransform The transform. rotation_limit : float | None The rotation limit. scale_limit : float | None The scale limit. translation_limit : Iterable[float] | None The translation limit. Returns ------- bool Whether the transform is valid. .. py:method:: _parse_alignment_results(results, fltr) Parse the results of image alignment. Parameters ---------- results : Tuple The results. fltr : str The filter. Returns ------- Tuple[NDArray, Dict[str, float], Dict[str, float]]: The stacked image, background medians, and background RMSs. .. py:method:: _plot_background_meshes(stacked_images, show) Plot the background meshes on top of the catalog images. Parameters ---------- stacked_images : Dict[str, NDArray] The stacked images for each camera. show : bool Whether to display the plot. .. py:method:: _visualise_psfs(image, fltr, show) Generate PSF plots for each source in an image. Parameters ---------- image : NDArray The image (not background subtracted). fltr : str The image filter. show: bool Whether to display the plot. .. py:method:: create_gifs(keep_frames = True, overwrite = False) Create alignment gifs for each camera. Some aspects of this method are parallelised for speed. The frames are saved in out_directory/diag/*-band_gif_frames and the GIFs are saved in out_directory/cat. Parameters ---------- keep_frames : bool, optional Whether to save the GIF frames in out_directory/diag, by default True. If False, the frames will be deleted after the GIF is saved. overwrite : bool, optional Whether to overwrite existing GIFs, by default False. .. py:method:: _create_gif_frames(file, fltr) Create a gif frames from a batch of images and save it to the out_directory. Parameters ---------- file : str The list of file names in the batch. fltr : str The filter. .. py:method:: _compile_gif(fltr, keep_frames) Create a gif from the frames saved in out_directory. Parameters ---------- fltr : str The filter. keep_frames : bool Whether to keep the frames after the gif is saved. .. py:method:: photometry(photometer) Perform photometry on the catalogs using the provided photometer. Parameters ---------- photometer : BasePhotometer The photometer. Should be a subclass of `BasePhotometer`, or implement a `compute` method that follows the `BasePhotometer` interface. .. py:method:: _photometry(photometer, source_coords, fltr, file) .. py:method:: identify_gaps(files) Identify gaps in the observation sequence and logs them to log_dir/diag/gaps.txt. Parameters ---------- files : List[str] The list of files for a single filter. .. py:function:: save_catalog(filters, stacked_images, catalogs, out_directory, show) .. py:function:: save_stacked_images(stacked_images, out_directory, overwrite) Save the stacked images to a compressed FITS file. Parameters ---------- stacked_images : Dict[str, NDArray] The stacked images (filter: stacked image). .. py:function:: save_backgrounds(camera_files, background_median, background_rms, bmjds, t_ref, out_directory, show) Plot the time-varying background for each camera. Parameters ---------- camera_files : Dict[str, str] The files for each camera {fltr: file}. background_median : Dict[str, List] The median background for each camera. background_rms : Dict[str, List] The background RMS for each camera. bmjds : Dict[str, float] The Barycentric MJD dates for each image {file: BMJD}. t_ref : float The reference BMJD. out_directory : str The directory to which the resulting files will be saved. show: bool Whether to display the plot. .. py:function:: get_batches(input) .. py:function:: get_batch_size(L) Compute the batch size for a given input length. Parameters ---------- L : int The length of the input. Returns ------- int The batch size.