Skip to content

env_params: support full dot-path keys from TestDefinition (cmd_args.foo.bar, bench_cmd_args.*) #950

Description

@rutayan-nv

Summary

Today env_params keys must name a top-level cmd_args field. They cannot address nested attributes (cmd_args.foo.bar) or other CLI argument bags some workloads expose (e.g. vllm's bench_cmd_args). This makes the env_params path semantics inconsistent with the rest of the parsing surface.

Proposed by @podkidyshev in review of #901 (thread). Explicitly out of scope for #901; tracked here as a follow-up.

Current behavior

TestDefinition.validate_env_params rejects any key that is not a top-level cmd_args field:

unknown = sorted(k for k in self.env_params if k not in cmd_args_fields)
if unknown:
    raise ValueError(f"env_params keys {unknown} are not cmd_args fields on ...")

and is_env_sampled / is_dse_job match on the bare key, so nested and non-cmd_args paths are unreachable.

Proposed behavior

Let an env_params key be a full dot-path rooted at TestDefinition, so it can target any nested attribute consistently:

num_nodes = 2

[cmd_args]
  foo = 1

  [cmd_args.args]
  bar = [1, 2, 3]

[env_params]
"cmd_args.args.bar" = { weights = [0.1, 0.1, 0.8] }

This makes env_params paths consistent with how other features address nested config, and unblocks workloads whose sweepable knobs live outside the root cmd_args (e.g. bench_cmd_args).

Scope notes

  • Path resolution must be shared with is_env_sampled / is_dse_job / param_space / apply_params_set so all of them agree on what a key addresses.
  • Keep the existing leaf/list/weights validation (a key must resolve to a leaf candidate list).
  • Backward compatibility: bare top-level keys should keep working (or a clear migration).

cc @podkidyshev (offered to take this).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions