Catalogue-type profiles — adding a new survey family

cosmotron_mcp/catalogue_profiles.yaml is a small, hand-curated YAML of known survey families. A profile supplies default pipeline parameters and the catalogue's column-name conventions, so a recognised survey resolves with no back-and-forth. An unrecognised survey falls back to fully dynamic auto-detection — that's the correct default for foreign data, not an error state.

Profiles are matched only by an explicit survey= argument to bootstrap_session / ingest_to_session (against a profile's name or aliases) — never guessed from the data or the task text.

Adding a profile

Add an entry to cosmotron_mcp/catalogue_profiles.yaml. No code change is needed — new entries are picked up automatically and validated against cosmotron_mcp.schemas.CatalogueProfile.

Core fields (every profile):

Field	Meaning
name	Canonical key — also the survey= lookup name
aliases	Other accepted names/spellings
description	Free text — what the survey is, catalogue conventions
coordinate_convention	"equatorial" — the only value supported currently; a profile declaring galactic/ecliptic is rejected with a clear error rather than silently mis-projecting
columns	{ra, dec, z, weight} — the actual column names in the file
defaults	Any subset of {nside, lmax, lmin, n_bandpowers, full_sky}

Survey-isolation fields (real-catalogue / Phase B additions, all optional):

Field	Meaning
probe	galaxy_density or galaxy_shear — pins the probe for this profile
selection	[{col, op, value}, ...] — declarative default row cuts, AND-combined
mask_kind	"binary" | "weight"
mask_from_weights	Build the shear mask as per-pixel Σw rather than a supplied footprint
completeness_map_pattern	Discovery regex for a HEALPix completeness map
calibration	{m: [...], sigma_m: [...], m_source: str} — per-bin shear multiplicative bias
derived_weight	{product: [...], sum_minus_one: [...]} — combine several columns into one weight
hdf5_group	Column-name prefix for grouped HDF5 files
columns.psf_e1 / columns.psf_e2	PSF ellipticity columns (α-leakage diagnostic)
columns.bin_id	Integer tomographic bin-ID column (e.g. TOM_BIN_ID, ZBIN)
defaults.ell_binning	"linear" | "log" (global per session)
defaults.bin_values	Pre-assigned integer bin IDs

Example — KiDS-1000

A real profile from the repo, showing most of the optional fields in use:

kids1000:
  name: kids1000
  aliases: [kids, kids-1000, kids_legacy, kids-legacy]
  description: >
    KiDS-1000 (DR4.1) cosmic-shear gold catalogue. Lensfit shear, per-object
    lensfit weight; the shear mask is the per-pixel sum of weights. Cosmic
    shear defaults to log-spaced bandpowers.
  coordinate_convention: equatorial
  probe: galaxy_shear
  columns:
    ra: ALPHA_J2000
    dec: DELTA_J2000
    z: Z_B
    weight: weight
    e1: e1
    e2: e2
    psf_e1: PSF_e1
    psf_e2: PSF_e2
  # Lensfit zero-weight objects are non-detections — drop them.
  selection:
    - {col: weight, op: ">", value: 0}
  mask_from_weights: true
  mask_kind: weight

Using it: bootstrap_session(data_dir, task, survey="kids1000") (or survey="kids") resolves probe, the column map, the default selection cut, and the weight-mask construction — no per-run overrides needed unless the task explicitly wants to deviate.

Verifying against the real file

Column names in a profile are a claim about a specific data release. Before trusting a profile against a real download, check the actual FITS/HDF5 header — a public catalogue can rename columns between releases (the KiDS-1000 profile above carries exactly this warning in its comments: some releases expose RAJ2000/DECJ2000 instead of ALPHA_J2000/DELTA_J2000). Override with ra_col=/dec_col=/etc. at ingest time if they differ — this does not require editing the profile.

Variants are distinct datasets

A survey's single-bin, tomographic, masked, and cosmic-shear variants each get registered separately in the dataset registry — they share a profile but are different datasets, distinguished by the register_as/name/version you give them at registration time.