New issue with cosmosis campaign env varialbes

[jessmuir]

I recently updated cosmosis to version 3.8 and have encountered a new error when using env variables with cosmosis campaign.

The yml and ini files I'm working with ran ok before I updated cosmosis. Unfortunately I'm not sure what the previous version was before I updated to 3.8, so I'm not sure exactly what change interfered with this.

My setup is that I have an ini files specifying a pipeline, where a 2pt_like module definition includes the line

%include scale_cuts/${SCALE_CUTS_FILE}

That bash varialbe is specified in a run definition in the campaign file, using

  # Fiducial 3x2pt lcdm data run.
  - name: 32pt_lcdm_fidsimy6
    base: ini_files/main_32pt.ini
    params:
    - pipeline.fast_slow = T
    - sampler = polychord
    env:
      # setting these here have the same effect as
      # defining bash variables in an job submission script
      DATA_VECTOR : sim_baseline_32_lcdm.fits
      SCALE_CUTS_FILE : scale_cut_file.ini

When run cosmosis-campaign, either attempting to do a test run or looking at available runs using cosmosis-campaign -l, I get the error:

(/Users/jmuir/workspace/cosmosis3/env) y6simruns$ cosmosis-campaign -l campaign_sim_xlens.yml
Traceback (most recent call last):
  File "/Users/jmuir/workspace/cosmosis3/env/bin/cosmosis-campaign", line 4, in <module>
    cosmosis.campaign.main(args)
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/campaign.py", line 816, in main
    runs, _ = parse_yaml_run_file(args.run_config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/campaign.py", line 608, in parse_yaml_run_file
    inc_runs, inc_comps = parse_yaml_run_file(include_file)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/campaign.py", line 625, in parse_yaml_run_file
    runs[name] = build_run(name, run_dict, runs, components, output_dir, submission_info, output_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/campaign.py", line 417, in build_run
    params = Inifile(run_info["base"], print_include_messages=False, no_expand_vars=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/runtime/config.py", line 136, in __init__
    self.read(filename)
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/configparser.py", line 713, in read
    self._read(fp, filename)
  File "/Users/jmuir/workspace/cosmosis3/env/lib/python3.11/site-packages/cosmosis/runtime/config.py", line 66, in _read
    raise ValueError(f"Tried to include non-existent file {filename}")
ValueError: Tried to include non-existent file scale_cuts/${SCALE_CUTS_FILE}

Perhaps a recent update to ini file parsing changed this behavior? As I said, this setup didn't cause problems before yesterday when I updated cosmosis?

[jessmuir]

Note that this isn't really a bug: one could argue that it's better to avoid using this bash variable in the original ini file, and then use a cosmosis campaign component to overwrite the scale cuts instead of changing what scale cut ini file is included. I mainly just wanted to flag the fact that this behavior changed.

On a related note: for a different element of the same project I've written a couple utility functions that generate a cosmosis-campaign component for a yml file given scale cuts specified in an ini file, and vice versa. I don't know if there's a sensible way to incorporate that into the campaign structure, but since testing how runs respond to different scale cuts is a somewhat common test, and since transferring scale cut info between ini and yml formats is fairly tedious (and risks of typos), others might find this automation useful. Let me know if there's a helpful way to share this.

[joezuntz]

Hi Jessie - thanks for flagging this. I made a change so that campaign environment variables were only supported in values, not in keys or section names. I had thought that was the best idea, but it does mean that there is an inconsistency between the behaviour when using campaigns and when using the original ini files, which certainly isn't good. But at least npthing should silently go wrong.

I think I'd have to completely rewrite how the runs are stored in order to make this work while still allowing child runs to override the environment variables of their parents. I think I'd like to do that at some point in order to make Runs and Campaigns more usable from scripts, but that will take some time, and I think you're right that ultimately it would be better to avoid env vars here (and indeed everywhere) in favour of using components.

So I'm not sure what to do here. Any thoughts welcome!

[jessmuir]

I think the change is reasonable, and if having both behaviors (being able to specify filenames with env variables as well as letting children override parents' env) isn't easy to implement.

It'll require people to make some changes to workflow in terms of how people often set choices for 2pt scale cuts, and thus will mean a bit more legwork for folks transiting from existing lots-of-related-ini-fiels setups to campaign setups. That's not a show stopper, but could maybe be worth adding a note about it to the campaign setup documentation?