ARGUS-16 Imager · Photometry Pipeline · Migration Review

Moving nightly photometry from the legacy scripts to prism v2

Where the migration stands after reprocessing the full 2024–2025 archive: what improved, what broke, the three decisions the instrument team owes, and the validation night that gates the switch.

Status awaiting review — nothing switched over Reprocessed 412 nights, 2024A–2025B Verified against prism @ 4c1f9e2, 2026-07-18 Author pipeline working group

Technique 01 Masthead & provenance row Eyebrow → title → dek → metadata strip. The provenance line ("verified against commit X on date Y") is what makes a shared plan trustworthy after it goes stale.

Zero-point scatter

Improved 12 → 7 mmag

Nightly RMS across 412 reprocessed nights, r-band

Nightly jobs

Green 409 / 412

3 nights fail on the crowded-field blocker below

Runtime per night

Slower 41 min → 58 min

PSF fitting dominates; budget is 90 min before morning

Gap to switchover

3 items

1 blocker bug · 2 team decisions

Technique 02 Stat strip with status pills The whole health picture as numbers-first tiles before the first paragraph. Green/amber/red pills classify each figure so the eye finds trouble without reading.

The migration is a win on precision and a loss on runtime, and both are quantified. PSF fitting cuts the nightly zero-point scatter almost in half and recovers ~1.1 mag of depth in crowded fields — but it runs 40% slower and currently falls over on the three densest Galactic-plane pointings.

Recommendation: fix the deblender ceiling, then switch. The remaining work is one bounded bug plus two policy calls (aperture-compatibility columns, archive re-ingest order). Nothing else in the diff blocks first light on v2.

Technique 03 TL;DR / thesis panel The inverted pyramid, enforced by layout — an accent-washed box the reader cannot miss, holding the whole verdict in two sentences. A markdown "**TL;DR**" heading has no such gravity.

Needs the instrument team — in priority order

1 · Approve the deblender fix scope. Cap the per-tile source count at 40,000 and fall back to aperture photometry above it (proposed), or fund the full iterative-deblend rewrite (~3 weeks). The three failing nights all sit in Cygnus.
2 · Compatibility columns. Do v2 catalogs keep the legacy flux_ap3 / flux_ap5 aperture columns alongside PSF fluxes for one semester, or cut over clean? Ongoing monitoring programs vote here.
3 · Re-ingest order. The archive team can re-ingest 2024A–2025B oldest-first (safe, slow) or newest-first (monitoring programs see benefits immediately). Pick one; the script takes either.

Technique 04 "Needs you" priority queue LLM plans bury their asks. One loud, ranked block turns "read 4,000 words" into "answer 3 questions" — the artifact as a to-do list for the human, not the machine.

60-second primer: the life of a frame

Everything below makes sense once you know this path. A raw exposure becomes catalog rows like so:

raw frame ──▶ calibrate (bias · dark · flat) ──▶ extract sources ──▶ fit PSF ──▶ cross-match Gaia DR3 ──▶ zero-point ──▶ catalog row

Zero-point: The nightly constant that converts instrumental counts to calibrated magnitudes. Its scatter is the single best one-number health metric for a photometry pipeline.
PSF fitting vs. aperture photometry: Aperture sums the pixels in a circle; PSF fitting models the star's actual shape. PSF wins in crowded fields and poor seeing, at the cost of compute and a model that can fail.
Deblender: The stage that splits overlapping sources into separate detections. It is where crowded fields go to die — see finding 1.

Technique 05 Primer / glossary panel Artifacts get shared to people who weren't in the session. Front-loading the four terms the page depends on makes it readable with zero prerequisites — the thing a chat transcript can never be.

ARGUS-16 at a glance

The fixed instrument parameters the rest of this review leans on — a reference record, not a status board.

Primary mirror1.83 m

Optical designPrime-focus corrector, f/2.9 — fast and wide by design

Detector mosaic16 × (6k × 6k) CCDs — the sixteen in ARGUS-16; 576 Mpix/frame

Pixel scale0.31 ″/px — coarse, so close pairs share pixels

Field of view2.1° × 2.1° — one pointing swallows a whole Galactic-plane field

Filter complementu g r i z · Hα · Hβ

Median seeing1.1″ — the PSF model has to track this nightly

Nightly data volume≈ 340 GB — 412 nights reprocessed for this review

The three accented values are the whole story of the crowded-field blocker: a fast, wide field with coarse pixels packs enormous source counts into one tile.

Technique 06 Spec / key-value record Structured records want a record layout, not prose. The label→value grid collapses to a clean stack on a phone (narrow the window to watch it) — responsive behavior a markdown table can't do — and the accented values are the ones that explain the hard part of the job.

Where v2 changes the pipeline

Ingestraw FITS + headers, unchanged
Calibratesame master bias/dark/flat logic
Extract + deblendnew: image segmentation replaces threshold boxes
PSF fitnew: per-quadrant PSF model, replaces fixed apertures
Cross-matchGaia DR3, radius tightened 1.5″ → 0.8″
Zero-pointnew: sigma-clipped per-quadrant solution
Publishcatalog + QA plots to the archive

Outlined stages are the migration surface; everything else is byte-identical to v1 by construction.

Technique 07 Pipeline diagram in pure CSS Architecture is spatial. Numbered flex cards with a highlighted "changed" state show the migration surface at a glance — pure HTML, so it inherits the theme and stays editable text.

The review loop, as a graph

flowchart LR
    A[nightly run] --> B{QA gates pass?}
    B -- yes --> C[publish to archive]
    B -- no --> D[quarantine night]
    D --> E[triage: pipeline bug or sky?]
    E -- pipeline --> F[fix + reprocess]
    E -- sky --> G[flag night, publish with caveats]
    F --> A

Technique 08 Native mermaid diagrams Artifacts render <pre class="mermaid"> blocks as real diagrams with no external libraries — the fastest way to get a flowchart, state machine, or sequence diagram into a shared page.

Ground truth: what the code actually does today

prism/photometry/psf.py:214–261 The PSF model is per-quadrant. Four independent models per frame, refit nightly. This is what buys the zero-point improvement — the old single-model assumption was the dominant error term in the corners.

prism/match/gaia.py:88 Cross-match radius is now 0.8″. Tightened from 1.5″; false-match rate in test fields drops from 2.1% to 0.4% with no loss of real matches brighter than r = 20.5.

prism/extract/deblend.py:141 Trap: the deblender allocates per-source, unbounded. Memory grows linearly with detected sources; above ~55,000 per tile the job is OOM-killed. The three Cygnus nights hit 70k+. This is the blocker.

prism/zeropoint/solve.py:52 Zero-point rejects variables correctly. Known variables from the reference catalog are sigma-clipped before the solve — verified against 30 injected test variables, all excluded.

Technique 09 Ground-truth fact cards, with a trap card Verified observations get mono file:line locators and their own cards, structurally separated from design opinion — and the land-mine is tinted red so nobody skims past it. Credibility lives here.

Aperture vs. PSF, honestly

	v1 · Aperture	v2 · PSF fitting
Crowded-field photometry	Blends unresolved	Deblended, +1.1 mag depth
Zero-point scatter (r-band)	12 mmag	7 mmag
Runtime per night	41 min	58 min
Fails on dense tiles	Never	3 / 412 nights (blocker)
Poor-seeing nights	Aperture losses uncorrected	Model absorbs seeing
Downstream compatibility	Every consumer works today	Needs compat columns (decision 2)

Technique 10 Comparison table with verdict cells Tradeoffs are two-dimensional; prose flattens them. Colored yes/no cells add a third channel (the verdict), and the overflow-x wrapper means wide data scrolls in place instead of breaking the page on a phone.

Switchover: who does what

Observer on shift — one-time, at the telescope Pipeline, automated — every night Archive team — one-time, during re-ingest

Observer on shift

Run the validation-night checklist (below) on the first v2 night — human eyes on the QA plots before anyone trusts the automation
Keep the v1 fallback command taped to the console for two weeks

Pipeline, automated

Nightly: calibrate → extract → fit → match → solve → publish, with QA gates at each stage
On any gate failure: quarantine the night, page nobody before 08:00 — nothing here is worth waking an astronomer for

Archive team

Re-ingest 412 reprocessed nights in the order decision 3 picks
Keep v1 catalogs queryable under argus_dr2_legacy for one year

Technique 11 Actor-colored lanes "Who does what" is the question every plan must answer. Encoding who as a persistent color + lane lets a reader trace one actor's duties through the whole document — impossible in flat markdown.

Review findings

Showing 3 of 3

BlockerUnbounded deblender memory kills dense-field nights

Per-source allocation with no ceiling (deblend.py:141). The three Galactic-plane pointings exceed 70,000 sources per tile and the job is OOM-killed at ~55,000. Until this is capped or rewritten, v2 cannot be the default — those fields are in two active programs.

MajorRuntime eats 64% of the morning budget

58 of 90 minutes, and the margin shrinks in poor seeing (bigger PSF stamps). Not a blocker, but the composition bar below says where the time goes — the PSF stage is the only lever worth pulling.

MinorQA plot axes unlabeled in two panels

The seeing-vs-zero-point and airmass panels ship without units. Cosmetic, but these plots are what the observer checks at 07:00 — fix before the validation night.

Technique 12 Finding cards with severity rails Review output is a list of findings with severities. The colored left rail + chip gives each one a scannable identity, and the repeated card anatomy makes ten findings read as one shape, not ten essays.

Technique 13 Live client-side filter The pills above the findings aren't decoration — they compute. Click "Blockers only" and the page hides everything else and updates the count, on the reader's click. A markdown file or a PDF can't do this; an artifact is a real web page, so it can filter, toggle, and count. Used with restraint — one control that answers one real question ("what actually stops the switchover?").

Validation night: the guided checklist

Work top to bottom on the first v2 night. Each step says what to do and exactly what you should see. Checkboxes are for your tracking only — they don't persist.

Open the nightly QA page after the 06:30 run completes.

All seven stage gates show green. The zero-point panel reports RMS ≤ 9 mmag for a photometric night.

Pick one standard field and compare v2 PSF magnitudes to the v1 catalog for 20 bright, isolated stars.

Median offset < 5 mmag, no trend with magnitude. A constant offset is fine (zero-point definition changed); a slope is not.

Skip stars within 10″ of a neighbor — you'd be comparing deblended v2 photometry against blended v1 photometry, and the disagreement is the improvement, not a bug.

Check the quarantine directory.

Empty on a clear night. If a night is quarantined, the triage note names the failed gate — report the gate name, not "it broke."

Technique 14 Checklist with do / expect anatomy A test plan is something you execute. Real checkboxes give working state; the bold "do" / muted "expect" split separates the action from its oracle; the amber note pre-empts the false bug report before it's filed.

The fix, annotated

# prism/extract/deblend.py — the proposed cap
MAX_SOURCES_PER_TILE = 40_000   # decision 1: team approves this number

def deblend(tile):
    if tile.n_detected > MAX_SOURCES_PER_TILE:
        return aperture_fallback(tile, flag="DEBLEND_CAPPED")  # bounded, flagged, honest
    return iterative_deblend(tile)

And the log line that found the blocker — one highlighted line in four is the whole diagnosis:

02:41:07  extract  tile 118  detected 71,204 sources
02:41:31  deblend  tile 118  allocating models…
02:43:55  deblend  tile 118  OOM-killed at 54,880 models (rss 31.2 GB)
02:43:56  night    2025-06-02  QUARANTINED at stage: extract

Technique 15 Annotated code & highlighted log line Markdown fences are monochrome. Coloring the load-bearing constant, the fix, and the one fatal log line is the difference between showing code and teaching from it.

Where the 58 minutes go

PSF fitting 62% Calibration + I/O 23% Cross-match + zero-point 15%

One bar replaces a paragraph of percentages — and says the PSF stage is the only optimization target worth anyone's time.

Technique 16 Composition bar A part-to-whole in one CSS flex row: segment widths are the data, 2px gaps separate the fills, the key carries labels in text ink. No chart library, fully theme-aware.

The long tail, tucked away

Full QA gate definitions (7 gates, reference only)

Gate 1 — calibration residuals: master-flat residual RMS < 0.4%. Gate 2 — extraction: detected source count within 3σ of the field's historical envelope. Gate 3 — PSF: model FWHM within 15% of the seeing monitor. Gate 4 — cross-match: match fraction > 92% for r < 19. Gate 5 — zero-point: solution converged, RMS < 15 mmag. Gate 6 — astrometry: median residual vs. Gaia < 60 mas. Gate 7 — publish: catalog row count equals extraction count minus rejects, exactly.

Why the cross-match radius tightened to 0.8″

With per-quadrant PSF astrometry the positional scatter dropped enough that 1.5″ was admitting field contaminants at 2.1%. The 0.8″ radius was chosen from the match-fraction elbow plot — the largest radius at which the false-match rate stays under 0.5%.

Technique 17 Progressive disclosure <details>/<summary> keeps reference-grade detail one click away without taxing the main read. Use sparingly — strong hierarchy usually beats hiding — but it's the right home for the appendix.

What you just read is the demo. Every scenario detail on this page is fictional, but every formatting device is real and reusable: any Claude session can produce a page like this from the work you just did with it — hosted privately, shared by link, readable by someone who wasn't there. If a plan matters enough to be read by a second person, it's worth an artifact.

Demonstration artifact for the ARGUS-16 pipeline scenario (fictional). Each dashed tag names the formatting technique the section above it demonstrates. Dual light/dark theme via CSS tokens + prefers-color-scheme + the viewer's theme toggle; no external resources; wide content scrolls inside its own container.