Make environment squashing optional

[lclanzi]

Add a skip_env_squashfs boolean flag to RunConfig that allows bypassing squashfs image creation and using the virtual environment directly. I believe this can be useful during development or quick iterations where the ~5–10 min mksquashfs step is a bottleneck.

Config example:

  - forecaster:
      checkpoint: https://servicedepl.meteoswiss.ch/mlstore#/experiments/409/runs/b30acf68520a4bbd8324c44666561696
      label: stage_C_icon_1km
      steps: 0/120/6
      skip_env_squashfs: true
      config: resources/inference/configs/sgm-forecaster-global-ich1-oper.yaml

[dnerini]

thanks for the nice contribution! I can see how this is really useful for debugging, although I'm slightly concerned that the option will be used a lot once we realize that it makes the code faster (the squash-fs step follows best practices from CSCS concerning python virtual envs)

[dnerini]

mmmh I'm not really happy with all this code duplication. Can we try avoid it?

[lclanzi]

Updated in latest commit

[dnerini]

I wonder if we should phrase it in a "positive" sense and more in the uenv sense: using e.g. a --build-uenv flag that is true by default?

[lclanzi]

Updated in latest commit

[lclanzi]

thanks for the nice contribution! I can see how this is really useful for debugging, although I'm slightly concerned that the option will be used a lot once we realize that it makes the code faster (the squash-fs step follows best practices from CSCS concerning python virtual envs)

I see, that's a valid point. If this is the case, feel free to reject the PR

[dnerini]

thanks for the nice contribution! I can see how this is really useful for debugging, although I'm slightly concerned that the option will be used a lot once we realize that it makes the code faster (the squash-fs step follows best practices from CSCS concerning python virtual envs)

I see, that's a valid point. If this is the case, feel free to reject the PR

maybe it's enough to hide it well ;) jokes asides, I really think this can be useful, so good documentation should be enough