Skip to content

Session model

A run lives in workspace/{YYYY-MM-DD}_{data_folder_stem}/ — named after the data folder, not a catalogue file (a multi-catalogue 3×2pt directory has no single "the" catalogue to name it after) — and is described by three JSON files, each with a different owner and a different lifetime:

workspace/YYYY-MM-DD_{data_folder_stem}/
├── DATA_DESCRIPTION.md       # user-supplied — see "Writing DATA_DESCRIPTION.md"
├── session_context.json      # pinned pipeline parameters — written once, before any agent
├── remote_state.json         # (remote runs only) {site, remote_session_dir, pinned_at}
├── manifest.json             # written by ingest_to_session — per-bin paths + stats
├── results/
│   ├── catalogue/             maps/           spectra/         covariance/
│   │   systematics/           data_vector_sacc.fits (root — final product)
│   │   artefact_registry.json (root — meta)
├── plots/                     # PNGs — sibling of results/, not nested under it
└── scripts/                   # written by run_analysis_script, when used

workspace/ is disposable. Anything worth keeping across sessions goes in the dataset registry, not here.

session_context.json — read-only for agents

The anti-hallucination anchor. Pins the catalogue path and every pipeline parameter, written once at bootstrap_session time and never hand-edited afterwards.

{
  "catalogue_path": "data/.../catalogue.fits",
  "data_folder": "data/...",
  "pipeline_config": {
    "nside": 512,
    "lmax": 1535,
    "lmin": 2,
    "n_bandpowers": 20,
    "bandwidth": 77,
    "apodisation_scale": 3.0,
    "apodisation_type": "C1",
    "full_sky": false,
    "spin": 0,
    "probes": {
      "galaxy_density": {"spin": 0, "galaxy_bias": 1.8},
      "galaxy_shear": {"spin": 2, "sigma_e": 0.28, "lmax": 700}
    },
    "pairs": "all",
    "remote": {"site": "sunrise", "steps": ["covariance", "inference"]}
  },
  "pipeline_config_provenance": {
    "nside": {"source": "user"},
    "lmax":  {"source": "default"}
  },
  "execution_site": "sunrise",
  "required_plots": []
}
  • nside is global — cross-correlations require every probe's maps at the same resolution. It has no default; if nothing resolves it, bootstrap_session returns nside_unresolved and the bootstrapper agent has to ask.
  • probes overrides only spin/lmax/lmin/galaxy_bias/mask_path/ sigma_e per probe — everything else stays global.
  • pairs (from ## correlations: in the task, see Writing TASK.md) is all / auto / adjacent:N / an explicit [[i,j], ...] list. It's applied early, so spectra, covariance, and the SACC assembly only ever build the selected pairs — no giant covariance you then throw half of away.
  • Spin/probe are set only by an explicit declaration (spin=, probe=, or a ## probes: block) — never inferred from prose. A shear-sounding task with no declaration is gated (probe_unresolved) rather than guessed.
  • pipeline_config.remote ({site, steps}, only present when the task has a ## remote:/## site: block or site:path data) and the top-level execution_site record where heavy steps run — the remote-capable tools auto-attach a jobspec from this, and the session is pinned to the site via remote_state.json. Absent → everything runs locally. See Remote execution.

Read it with read_session_context_tool. See Parameter resolution for how each value gets its source.

manifest.json — written by ingestion

Per-bin catalogue paths and statistics. Carries only nside and lmax among the scalar pipeline parameters — everything else (n_bandpowers, lmin, bandwidth, apodisation_scale, full_sky) lives in pipeline_config above, never here. Each bin carries its own spin/probe/sigma_e/n_eff_arcmin2/mask_path; a multi-probe (3×2pt) session also carries a top-level catalogues list, one entry per ingested catalogue.

{
  "session_dir": "workspace/2026-07-02_3x2pt",
  "survey": "mock3x2pt",
  "raw_path": "data/.../clustering.fits",
  "mask_path": "results/maps/combined_mask.fits",
  "nside": 512,
  "lmax": 1535,
  "tomographic_bins": [
    {
      "bin_id": 0, "z_min": 0.2, "z_max": 0.5,
      "fits_path": "results/catalogue/bin_00_catalogue.fits",
      "n_objects": 568776, "spin": 0, "probe": "galaxy_density",
      "catalogue_stats": {"n_objects": 568776, "f_sky": 0.24, "n_bar": 12.3}
    },
    {
      "bin_id": 2, "z_min": 0.2, "z_max": 1.2,
      "fits_path": "results/catalogue/bin_02_galaxy_shear_catalogue.fits",
      "spin": 2, "probe": "galaxy_shear", "sigma_e": 0.28,
      "n_eff_arcmin2": 1.5, "m": 0.012, "sigma_m": 0.02
    }
  ],
  "catalogues": [
    {"probe": "galaxy_density", "raw_path": "data/.../clustering.fits"},
    {"probe": "galaxy_shear", "raw_path": "data/.../shear.fits"}
  ]
}

n_bar = n_objects / (f_sky * 12 * nside**2) — mean counts per HEALPix pixel. An empty bin (f_sky == 0 or n_objects == 0) raises rather than silently writing n_bar = 0.0.

artefact_registry.json

Every output written by the save_* tools or a compute_*_from_session call auto-registers here:

{
  "type": "cls_json",
  "path": "results/spectra/cls_bin_00_x_00.json",
  "generated_by": "step_02_compute_cls.py",
  "agent_name": "analysis_coder",
  "bin_index": 0,
  "metadata": {}
}

Read it to confirm completion and to source exact output paths — never type or guess a result filename. Call register_artefact manually only for outputs written outside results/.

Canonical output filenames

Zero-padded bin indices throughout (bin_00, bin_01, …); pipeline-stage subfolders under results/; single-file final products (the SACC, the registry itself) stay loose at results/ root.

Artefact Path
Standardised catalogue (per bin) results/catalogue/bin_{i:02d}_catalogue.fits
Power spectrum (auto) results/spectra/cls_bin_{i:02d}_x_{i:02d}.json
Power spectrum (cross) results/spectra/cls_bin_{i:02d}_x_{j:02d}.json (i<j)
Cached NaMaster MCM workspaces results/spectra/workspaces/mcm_{key}.fits
Covariance (per bin) results/covariance/covariance_bin_{i:02d}_x_{i:02d}.json
Covariance (joint, EE) results/covariance/covariance_all_bins.json
B-mode validation covariance results/covariance/covariance_validation_{BB,EB}.json
Null test results/systematics/null_test_bin_{i:02d}[_{BB,EB}].json
Systematics gate results/systematics/systematics_gate.json
Human gates results/gates/<gate_id>.json — see Human gates
HEALPix map (clustering) results/maps/delta_map_bin_{i:02d}.fits
HEALPix maps (shear) results/maps/shear_q_bin_{i:02d}.fits, shear_u_bin_{i:02d}.fits
SACC data vector results/data_vector_sacc.fits
Plots plots/*.png

Power spectra are always JSON, never .fits. Binned bandpowers are never labelled "unbinned".