Visualisation Routines

This notebook demonstrates the visualisation routines available in opticam. These routines can ne helpful for sanity checking data, and informing reduction parameters (e.g., aperture sizes).

Generating Data

First, let’s generate some data:

from pathlib import Path

import opticam


out_dir = Path('out')

opticam.generate_observations(
    out_directory=out_dir / 'data',  # path to the directory where the generated data will be saved
    circular_aperture=False,  # disable circular aperture shadow
    n_images=5,
    )

[OPTICAM] variable source is at (131, 115)
[OPTICAM] variability RMS: 0.02 %
[OPTICAM] variability frequency: 0.135 Hz
[OPTICAM] variability phase lags:
    [OPTICAM] g-band: 0.000 radians
    [OPTICAM] r-band: 1.571 radians
    [OPTICAM] i-band: 3.142 radians

Generating observations: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:01<00:00]

Now let’s define a Reducer instance:

reducer = opticam.Reducer(
    out_directory=out_dir / 'reduced' / 'default_background',
    data_directory=out_dir / 'data',
    )

[OPTICAM] out/reduced/default_background not found, attempting to create ...
[OPTICAM] out/reduced/default_background created.

[OPTICAM] Scanning data directory: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:00<00:00]

[OPTICAM] Checking instrument OPTICAM_MX.
[OPTICAM] OPTICAM_MX sucessfully passed all checks.

[OPTICAM] Parsing file headers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:02<00:00]

[OPTICAM] Binning: 4x4
[OPTICAM] Filters: 1:g, 2:r, 3:i
[OPTICAM] 5 1:g images.
[OPTICAM] 5 2:r images.
[OPTICAM] 5 3:i images.

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/diag/header_times.pdf.

Background Meshes

When constructing a background image, it’s important to verify that the mesh size is suitable. Generally, the mesh boxes should be small so that variations in the background across the image can be captured. However, the mesh boxes shouldn’t be smaller than the typical source size. By default, opticam defines the background’s box_size parameter such that images are divided into a 32x32 mesh. Let’s check this default mesh size:

reducer.plot_background_meshes()

We can see that the default mesh size looks pretty good in this case. Note that each time plot_background_meshes() is called, a random image is shown for each filter and the chosen file’s name is shown in the title of each panel. If we call plot_background_meshes() again, we’ll get different files:

reducer.plot_background_meshes()

Let’s define a new reducer instance that uses a background estimator with a larger box_size value:

custom_background = opticam.DefaultBackground(box_size=32)

custom_reducer = opticam.Reducer(
    out_directory=out_dir / 'reduced' / 'custom_background',
    data_directory=out_dir / 'data',
    background=custom_background,
    )

[OPTICAM] out/reduced/custom_background not found, attempting to create ...
[OPTICAM] out/reduced/custom_background created.

[OPTICAM] Scanning data directory: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:00<00:00]

[OPTICAM] Checking instrument OPTICAM_MX.
[OPTICAM] OPTICAM_MX sucessfully passed all checks.

[OPTICAM] Parsing file headers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:02<00:00]

[OPTICAM] Binning: 4x4
[OPTICAM] Filters: 1:g, 2:r, 3:i
[OPTICAM] 5 1:g images.
[OPTICAM] 5 2:r images.
[OPTICAM] 5 3:i images.

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/custom_background/diag/header_times.pdf.

Let’s see how this custom background mesh compares to the default one:

custom_reducer.plot_background_meshes()

We can see that the custom mesh is much larger - probably too large. Let’s stick with the default one for now.

As mentioned above, the images shown when plot_background_meshes() is called are random. However, this is only true if sources have not been cataloged. Once source catalogs have been created, plot_background_meshes() will use the stacked catalog images. Let’s therefore catalog the sources:

reducer.create_catalogs()

[OPTICAM] Creating source catalogs.

[OPTICAM] Aligning 1:g images: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:01<00:00]

[OPTICAM] [OPTICAM] Done.
[OPTICAM] [OPTICAM] 5 image(s) aligned.
[OPTICAM] [OPTICAM] 0 image(s) could not be aligned.

[OPTICAM] Aligning 2:r images: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:01<00:00]

[OPTICAM] [OPTICAM] Done.
[OPTICAM] [OPTICAM] 5 image(s) aligned.
[OPTICAM] [OPTICAM] 0 image(s) could not be aligned.

[OPTICAM] Aligning 3:i images: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:01<00:00]

[OPTICAM] [OPTICAM] Done.
[OPTICAM] [OPTICAM] 5 image(s) aligned.
[OPTICAM] [OPTICAM] 0 image(s) could not be aligned.

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/cat/catalogs.pdf.

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/diag/background.pdf.

Now when we call plot_background_meshes(), each panel will have the corresponding filter as the title instead of a random file name:

reducer.plot_background_meshes()

We can also save the image as a PDF file by passing save=True. This will save the mesh plot to out_directory/diag/background_meshes.pdf. Let’s check if such a file exists in this case:

import os

print(os.path.isfile('out/reduced/default_background/diag/background_meshes.pdf'))

False

We can see that the file does not yet exist, so let’s create it:

reducer.plot_background_meshes(save=True)

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/diag/background_meshes.pdf.

print(os.path.isfile('out/reduced/default_background/diag/background_meshes.pdf'))

True

Alignment GIFs

Before performing photometry, it’s probably a good idea to check that there are no issues in the image-to-image alignments. This can be done in opticam by calling the create_gifs() method of Reducer, which generates GIF files showing the image-to-image alignments for each filter:

reducer.create_gifs(overwrite=True)

[OPTICAM] Creating 1:g GIF frames: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:02<00:00]
[OPTICAM] Creating 2:r GIF frames: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:02<00:00]
[OPTICAM] Creating 3:i GIF frames: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|[00:02<00:00]

The resulting GIF files are stored in the out_directory/cat directory. Let’s take a look at an example:

import base64
from IPython.display import HTML

gif = os.path.join(reducer.out_directory, 'cat/1:g_images.gif')

with open(gif, "rb") as f:
    data = f.read()

b64 = base64.b64encode(data).decode("utf-8")
HTML(f'<img src="data:image/gif;base64,{b64}">')

As we can see, the image-to-image alignments look pretty good! The individual frames are also saved to the out_directory/diag directory by default, but if you only want to save the gif you can pass keep_frames=False.

Growth Curves

Growth curves can be useful for quantifying the extent of extended sources and/or determining suitable aperture sizes for performing photometry. To generate growth curves in opticam, we can call the plot_growth_curves() method of a Reducer instance. This method requires sources be cataloged before it can be called:

reducer.plot_growth_curves()

[OPTICAM] Generating growth curves for targets: {'1:g': [1, 2, 3, 4, 5, 6], '2:r': [1, 2, 3, 4, 5, 6], '3:i': [1, 2, 3, 4, 5, 6]}

[OPTICAM] Growth curves generated.

By default, plot_growth_curves() will generate growth curves for all cataloged sources. This may or may not be desirable. To generate growth curves for only a subset of sources, a target dictionary can be passed. The dictionary keys should list the filters, and the values should give the corresponding target ID(s). A single target ID can be passed as an integer, or a list of IDs can be passed:

targets = {
    '1:g': 1,
    '2:r': [1, 2, 3],
    '3:i': [1, 3, 4, 6],
}

reducer.plot_growth_curves(targets=targets)

[OPTICAM] Generating growth curves for targets: {'1:g': 1, '2:r': [1, 2, 3], '3:i': [1, 3, 4, 6]}

[OPTICAM] Growth curves generated.

Growth curves can also be saved by passing save=True. This will save the growth curves to the out_directory/diag/growth_curves directory as separate PDF files for each filter.

Apertures

When performing photometry, it is a good idea to check how the size of an aperture compares to the source sizes. In opticam, aperture plots can be generated by defining an AperturePhotometer instance and passing it to the plot_apertures() method of Reducer. By default, this will produce a plot for each source in each filter’s catalog:

phot = opticam.AperturePhotometer()

reducer.plot_apertures(photometer=phot)

Alternatively, we can pass a target dictionary to only create plots for specific targets, as in the case of the plot_growth_curves() method:

targets = {
    '1:g': 1,
    '2:r': 2,
    '3:i': [3, 6],
}

reducer.plot_apertures(
    photometer=phot,
    targets=targets,
    )

If the AperturePhotometer instance uses a local background estimator, we can also use plot_apertures() to inspect the annuli used to estimate the local backgrounds. This is especially useful for checking which pixels in an annulus have been sigma-clipped:

annulus_phot = opticam.AperturePhotometer(
    local_background_estimator=opticam.DefaultLocalBackground(),
    )

reducer.plot_apertures(
    photometer=annulus_phot,
    targets=targets,
    )

Aperture plots can also be saved by passing save=True. This will save the aperture plots to the out_directory/diag/apertures directory.

PSFs

Another useful diagnostic, particularly when performing aperture photometry, is to visualise each camera’s PSF. In opticam, this can be done by calling the plot_psfs() method of Reducer:

reducer.plot_psfs()

[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_1.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_2.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_3.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_4.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_5.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/1:g_source_6.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_1.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_2.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_3.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_4.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_5.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/2:r_source_6.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_1.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_2.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_3.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_4.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_5.pdf.
[OPTICAM] Plot saved to /tmp/tmpm36wz6u7/out/reduced/default_background/psfs/3:i_source_6.pdf.

plot_psfs() requires cataloging sources, and so cannot be called before create_catalogs(). Unlike the above methods, plot_psfs() does not display any plots, and instead saves them to a out_directory/psfs directory. This is because individual plots are created for each source in each filter. Since these plots are saved as PDF files, it is somewhat clunky to display them here:

from matplotlib import pyplot as plt
from pdf2image import convert_from_path

dir_path = reducer.out_directory / 'psfs'
psf_plot_files = sorted(os.listdir(dir_path))

# convert PDF pages into a list of PIL.Image.Image instances so they can be plotted with matplotlib
img = convert_from_path(os.path.join(dir_path, psf_plot_files[0]))[0]

fig, ax = plt.subplots(
    figsize=(6, 6)
)

ax.imshow(img)

ax.tick_params(left=False, bottom=False)  # remove ticks
ax.xaxis.set_ticklabels('')  # remove x-axis tick labels
ax.yaxis.set_ticklabels('')  # remove y-axis tick labels
ax.axis('off')  # remove spines

plt.show()

The bottom left panel shows brightness contours (using the stacked catalog image), while the bottom right and top left panels show the brightness profiles along the x- and y-centroids of the source, respectively. The FWHM of the PSF is shown by the red ellipse in the bottom left panel, and the red Gaussians in the top left and bottom right panels show the x- and y-projections of the PSF, respectively. In this case, we can see that this source’s PSF can be described as Gaussian to a very good approximation.

Let’s take a look at another source:

img = convert_from_path(dir_path / psf_plot_files[5])[0]

fig, ax = plt.subplots(
    figsize=(6, 6)
)

ax.imshow(img)

ax.tick_params(left=False, bottom=False)  # remove ticks
ax.xaxis.set_ticklabels('')  # remove x-axis tick labels
ax.yaxis.set_ticklabels('')  # remove y-axis tick labels
ax.axis('off')  # remove spines

plt.show()

Again, we can see that this source’s PSF is approximately Gaussian.

Signal-to-noise Ratios

Another useful diagnostic is to check the signal-to-noise ratio for each source. In opticam, this can be done by calling the plot_snrs() method of Reducer, which computes the signal-to-noise ratio for each source in each filter using the alignment reference images:

reducer.plot_snrs()

In this case, we can see that all six sources have high signal-to-noise ratios. The resulting plot can also be saved by passing save=True, which saves the plot to out_directory/diag/snrs.pdf.

Since signal-to-noise ratios are generated using the alignment reference images, they can give unreliable results for highly variable sources. Fortunately, the signal-to-noise ratio as a function of time is automatically saved when performing aperture photometry, see the Reduction Tutorial, and so it is straightforward to plot the signal-to-noise ratio as a function of time in such cases.

Noise Characterisation

In can be useful to characterise the noise contributions during an observation. In opticam, this can be done by calling the plot_noise() method of Reducer:

reducer.plot_noise()

In this case, we can see that the noise is largely in the transition from sky-dominated to shot-dominated. We can also see that the read nosie is negligible, and there is no contribution from dark noise (since the simulated dark current in zero).

That concludes the current visualisation routines provided in opticam! The routines provided here are intended to make a user’s life easier. If you think there are additional visualisation routines that could be added, feel free to open an issue or pull request on our GitHub.