Pipeline

This section covers the CLI-to-output-file path: a flowchart of the major functions and a walk-through of each.

Flowchart

The diagram below shows the path from a CLI invocation to a written output file. Cross-references below link each box to its API entry; the diagram itself uses bare names so it stays legible. The diagram is rendered as inline SVG by Mermaid (client-side); use the browser’s zoom controls to read the labels at any size.

        flowchart TD
    A[picmaker CLI<br/>argv] --> B[cli.main]
    B --> C[_build_parser<br/>argparse]
    B --> D[_separate_files_and_dirs]
    B --> CO[_collect_option_dicts]
    CO --> E[_normalize_and_validate<br/>per --versions line]
    E --> F[PicmakerOptions.validate]
    B --> PD[_process_directory<br/>per dirpath, recursive or not]
    F --> G[process_images<br/>per directory]
    PD --> G
    G -->|movie=True| H[images_to_pics<br/>pass 1: collect limits]
    H --> I[images_to_pics<br/>pass 2: shared stretch]
    G -->|movie=False| J[images_to_pics<br/>per file]
    I --> K[_process_one_image]
    J --> K
    K --> L[get_outfile]
    L -->|skip if replace='none'| M[Done]
    L --> PR[_pds3_resolve_pointer<br/>only for .LBL inputs]
    PR --> N[read_image_array]
    L --> N
    N --> O[read_one_image_array<br/>format cascade]
    O --> P[pickle / numpy / VICAR / FITS / PIL / PDS3]
    P --> Q[ReadResult<br/>array3d, default_is_up, filter_info]
    Q --> R{hst=True?}
    R -->|yes| S[_hst_mosaic_rgb<br/>WFPC2 quad or ACS panel mosaic]
    R -->|no| T[slice_array]
    T --> U[fill_zebra_stripes<br/>optional]
    U --> V[get_limits]
    V --> W[apply_colormap]
    W --> X[rotate_array_rgb]
    X --> Y[apply_gamma]
    Y --> Z[get_size + array_to_pil]
    S --> Z
    Z --> AA[filter_image]
    AA --> BB[resize_image]
    BB --> CC{sections > 1?}
    CC -->|yes| DD[wrap_image]
    CC -->|no| EE[skip wrap]
    DD --> FF{pad?}
    EE --> FF
    FF -->|yes| GG[pad_image]
    FF -->|no| HH[skip pad]
    GG --> II[write_pil]
    HH --> II
    II --> M

Two short observations on the diagram:

  • The movie=True branch runs images_to_pics() twice. The first pass computes the per-frame limits, the second pass uses the median of those limits so every frame shares one stretch.

  • The HST mosaic branch (hst=True) handles WFPC2 quad-panel and ACS/WFC two-panel composites; it is the only branch that bypasses the single-band slice_array() → apply_colormap() flow.

Major functions

This subsection walks through the major library functions in pipeline order. Cross-references resolve to the picmaker Module reference; each public symbol is a clickable link to its full signature, docstring, and source code (via sphinx.ext.viewcode).

CLI entry point

picmaker.cli.main() is the function bound to the picmaker console script. It builds the argparse parser, splits args.files into files and directories with picmaker.cli._separate_files_and_dirs(), and delegates the two remaining phases to two private helpers: picmaker.cli._collect_option_dicts() (the --versions FILE re-parse loop, returning one normalized option_dict per line) and picmaker.cli._process_directory() (the per-directory walk in recursive or non-recursive mode). Each helper is unit-tested directly in tests/test_cli_helpers.py.

The library equivalent of “run the CLI from Python” is to import picmaker.pipeline.images_to_pics() directly; the kwarg names match the CLI flags one-to-one. The CLI does no I/O of its own — every file operation flows through picmaker.io.

Option validation

picmaker.options.PicmakerOptions is a frozen-by-convention dataclass that holds the ~45 post-normalization knobs that drive the pipeline. Its validate() method runs every mutex / value-validity check that does not depend on raw argparse fields:

  • hst + bands is rejected (HST mosaic mode consumes every detector).

  • frame + size is rejected (both specify output dimensions).

  • frame + wrap_ratio is rejected (incompatible layout decisions).

  • display_upward + display_downward is rejected.

  • twobytes requires a TIFF extension and rejects any filter_name other than 'NONE'.

  • pds3_label_method must be one of the values in PDS3_LABEL_METHODS ('strict', 'loose', 'compound', 'fast'); the value is forwarded as pdsparser.PdsLabel’s method= argument when a PDS3 .LBL is parsed.

The CLI’s picmaker.cli._normalize_and_validate() does a few more checks that are CLI-specific (band/bands mismatch, --scale + --wscale, --overlap + --overlaps, --movie + --versions) because those operate on raw flags that get collapsed before the dataclass is built. Adding a new mutex rule that applies to both surfaces should go in validate().

The reader cascade

picmaker.io.read_one_image_array() is the single-file reader. It tries every supported format in turn (pickle → numpy → VICAR → FITS → PIL → PDS3) and returns a ReadResult triple. Each branch catches its specific exception types so an unrecognized file falls through to the next; the cascade-end OSError is chained from a ExceptionGroup that carries every per-reader failure for diagnostic purposes.

The FITS branch sniffs the first 9 bytes for b'SIMPLE  =' before calling astropy.io.fits.open() so that wrong-extension files do not trigger astropy’s expensive parser, and so that warnings raised from inside astropy.io.fits.open() are converted to exceptions by warnings.catch_warnings + filterwarnings('error') and swallowed at the branch boundary.

picmaker.io.read_image_array() is the multi-file wrapper: it delegates to read_one_image_array() per file and stacks the resulting arrays along the band axis with numpy.vstack(). The combined result inherits the default_is_up and filter_info of the first file.

picmaker.io.read_pds_labeled_image_array() handles the PDS3 label / detached-data case. Pointer resolution lives here: ^IMAGE = 2 (attached integer offset), ^IMAGE = "data.dat" (detached, full file), and ^IMAGE = ("data.dat", 3) (detached with record offset) are all distinct branches.

picmaker.io.read_pil() and picmaker.io.read_array() are the Pillow-side helpers used by PIL-readable inputs and by read_one_image_array()’s PIL branch.

Path planning

picmaker.io.get_outfile() derives the output file path for one input. It honors four replace= policies ('all' — silent overwrite; 'none' — return '' to signal the loop should skip; 'warn' — overwrite and emit UserWarning; 'error' — raise OSError). It creates the parent directory tree if it does not already exist.

picmaker.pipeline.find_common_path() derives the recursive output tree’s root by calling os.path.commonpath() over the input directories. The legacy hand-rolled version of this function used / as a literal separator and was wrong on Windows; the current implementation handles platform separators correctly and returns '' when the inputs share only the root.

Per-image pipeline

picmaker.pipeline.images_to_pics() runs the per-image pipeline shown in the flowchart above. The body is now a thin loop that builds a PicmakerOptions, backfills the legacy None-means-default kwargs, and delegates each filename to picmaker.pipeline._process_one_image(). That helper runs the following phases for one input file:

  1. Build the output path (get_outfile()); skip if replace='none' returned ''.

  2. Read the array (read_image_array()), with PDS3 detached-label pointer resolution delegated to picmaker.pipeline._pds3_resolve_pointer(). The caller’s reuse tuple short-circuits the read for the single-file batches that process_images() builds per option_dict.

  3. If hst=True and the instrument is ACS/WFC or WFPC2, dispatch to picmaker.pipeline._hst_mosaic_rgb() for the per-detector stack-and-mosaic flow.

  4. Otherwise: slice (slice_array()), optionally fill zebra stripes (fill_zebra_stripes()), compute limits (get_limits()), apply the colormap (apply_colormap()).

  5. Apply the orientation override (rotate_array_rgb()) and gamma (apply_gamma()).

  6. Convert to a PIL image (array_to_pil()), apply the PIL filter (filter_image()), resize (resize_image()), optionally wrap (wrap_image()), optionally pad (pad_image()).

  7. Write (write_pil()), which dispatches 16-bit output to picmaker.tiff16.WriteTiff16() and everything else to PIL.Image.Image.save().

The function returns (low, high, reuse) so callers (or the --movie second pass) can either consume the limits or replay the read.

picmaker.pipeline._hst_mosaic_rgb() itself further delegates the panel-assembly geometry to two private helpers, picmaker.pipeline._hst_wfpc2_mosaic() (four detectors, PC1/WF2/WF3/WF4 in a 2x2 quadrant) and picmaker.pipeline._hst_acs_panel_mosaic() (two detectors, WFC1 above and WFC2 below). Each helper is unit-tested directly in tests/test_pipeline_helpers.py.

picmaker.pipeline.process_images() is the thin loop that drives images_to_pics() per file; its only real job is the movie-mode two-pass dance described above.

Enhancement helpers

picmaker.enhance.get_limits() is the most option-heavy function in the codebase. It supports four ways of choosing the stretch range that can be combined:

  • Explicit limits=(lo, hi) — passed through unchanged.

  • percentiles=(lo%, hi%) — uses numpy.histogram() over the valid pixels and linear interpolation to find the corresponding DN values.

  • trim=N — drop N pixels from each edge before computing.

  • trim_zeros=True — peel all-zero exterior rows and columns before computing.

  • footprint=D — apply a circular median filter (footprint diameter D) and tighten the limits to the filter output.

picmaker.enhance.apply_colormap() maps a 2-D stretched array to a 3-D RGB array using either a named hyphen-separated colormap (e.g. 'red-blue'), a list of (R, G, B) tuples, or the per-instrument tint from picmaker.color.tinted_colormap(). It also handles the out-of-range and invalid-pixel highlight colors.

picmaker.enhance.apply_gamma() is the final power-law correction (array ** gamma).

picmaker.enhance.fill_zebra_stripes() cleans up leading- and trailing-zero artifacts in compressed spacecraft images. The implementation is currently a Python pixel loop; vectorization is tracked in issue #18.

Geometry helpers

picmaker.geometry.slice_array() takes the raw 3-D (bands, lines, samples) array and returns a 2-D band-averaged array plus an optional invalid-pixel mask. It honors samples, lines, bands, valid, and crop slice arguments.

picmaker.geometry.crop_array() strips constant-value borders (typically crop=0 for zero-padded fields). It returns the input unchanged if the whole array equals the crop value.

picmaker.geometry.rotate_array_rgb() applies the --rotate {fliplr,fliptb,rot90,rot180,rot270} choice and the display_upward override.

picmaker.geometry.get_size() is the resize planner. It returns (unwrapped_size, wrapped_size, sections, wrap_axis); the caller uses unwrapped_size to resize the image and the remaining three fields to wrap it (when sections > 1).

picmaker.geometry.resize_image(), picmaker.geometry.wrap_image(), and picmaker.geometry.pad_image() execute the plan against a PIL image. resize_image() upscales with NEAREST and downscales with LANCZOS so the output is pixel-art-friendly for small inputs and Lanczos-smoothed for large ones.

Color, filter, and PIL bridges

picmaker.color.tinted_colormap() is the entry point for per-filter tinting. It normalizes HST’s CL1 / CL2 / CLEAR* / N/A filter-tuple quirks, picks the right instrument module via picmaker.instruments.lookup(), and delegates to its tint_for callable (e.g. picmaker.instruments.cassini.tint_for()).

picmaker._filters.filter_image() applies one of the picmaker._filters.FILTER_DICT PIL presets to a PIL image, or raises KeyError if the case-folded filter name is not in the dict.

picmaker.pil_utils.array_to_pil(), picmaker.pil_utils.pil_to_array(), and picmaker.pil_utils.write_pil() are the three numpy ↔ PIL bridges. write_pil() dispatches 16-bit output (list-of-three 'I'-mode images, or a single 'I'-mode image) through picmaker.tiff16.WriteTiff16() and the 8-bit path through PIL.Image.Image.save().