Skip to content

Multi-format ingestion

Catalogues may be FITS, HDF5, or CSV. A typed reader registry (cosmotron_mcp/tools/readers.py) dispatches by file suffix and standardises all three into the same canonical FITS form that every downstream tool consumes — the rest of the pipeline never needs to know what format the raw catalogue arrived in.

Column resolution

For each required column (RA, DEC, Z, weight, and for shear E1/E2/PSF_E1/ PSF_E2), resolution tries, in order:

  1. Explicit argument — ra_col=, dec_col=, e1_col=, etc., passed to ingest_to_session.
  2. Profile — a catalogue-type profile's columns block, if survey= was given.
  3. Pattern auto-detection — common naming conventions (RA/ra/ ALPHA_J2000, …).

If a column can't be resolved by any of the three, ingest_to_session returns {"needs_input": ..., "columns": [...]} naming the missing role — it never guesses at a column mapping.

Ancillary files

  • Masks. Multiple footprint masks discovered in a data directory are auto-combined into one (element-wise max), recorded in manifest.mask_components.
  • n(z). For tomography, one n(z) file is matched per bin (by filename pattern ..._bin_<i> or by position). Multiple n(z) candidates are not guessed at — pass nz_paths=[one per bin] explicitly, or ingest_to_session returns {"needs_input": ..., "nz_candidates": [...]}.
  • Theory Cℓ. Same pattern: theory_path / theory_paths=[...], one per bin for tomography; multiple candidates return needs_input rather than a guess.

Shear (spin-2) ingestion

probe="galaxy_shear" plus e1_col/e2_col (also auto-detected from e1/e2/g1/g2) writes standardised E1/E2 columns, tags every bin spin=2, and records per-bin sigma_e — the user's value if given, else the measured Heymans et al. (2012) estimator — plus n_eff_arcmin2.

Multi-probe (3×2pt) ingestion

Call ingest_to_session once per catalogue: the first call ingests the session's primary catalogue, each subsequent call passes append=True, catalogue_path=<other>, probe=<distinct>. catalogue_path accepts an absolute path, a data_folder-relative path, or a repo-root- relative path — tried in that order by existence, same rule as mask_path/ completeness_path. Appended bins continue the global bin_id sequence and accumulate in manifest.catalogues — one entry per ingested catalogue/probe. Reuse (reuse_dataset_id=/dataset_tag=) works with append=True too — see Dataset registry: multi-catalogue reuse.

Full tool signature: ingest_to_session.