feat(loader): add pipeline_tag fallback to task detection#1113
Conversation
When architecture-based task detection (Stage 1c) fails because config.architectures contains a generic name like 'Model', fall back to querying the HuggingFace Hub pipeline_tag (Stage 1e) before the last-resort feature-extraction default (Stage 1d). - Add TaskSource.PIPELINE_TAG enum value for provenance tracking - Add Stage 1e in resolve_task() between 1c and 1d: - Checks model_id is not a local path via _is_local_path - Queries HfApi().model_info(model_id).pipeline_tag - Validates against KNOWN_TASKS after normalize_task() - Wrapped in try/except for graceful fallthrough on errors - Add 4 tests covering valid tag, invalid task, API failure, local path Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Addresses CodeQL 'Empty except' finding by documenting that the pass is intentional — network errors, invalid model IDs, and missing pipeline_tag all gracefully fall through to Stage 1d. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
a follow up of #1094 tested via winml export -m audeering/wav2vec2-large-robust-24-ft-age-gender -o temp/wav2vec2.onnx |
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Thanks for the pipeline_tag fallback — the overall shape is right: it only runs on the last-resort path (opt_task is None), guards against local paths, and falls through gracefully. A few things worth addressing before merge, left inline:
- resolve_task now does blocking network I/O on a path that used to be purely local — no timeout, no caching, and the docstring's stage list wasn't updated.
- The bare
except Exception: passdrops all diagnostics. - The local-path negative test can't actually fail, because the AssertionError it raises is swallowed by that same
except Exception.
…ging
- Extract get_pipeline_tag() helper in hub_utils.py:
- Encapsulates _is_local_path check, HfApi call with 10s timeout,
and logger.debug on failure (matching existing hub_utils conventions)
- Simplify Stage 1e in resolution.py to call get_pipeline_tag() directly
- Update resolve_task docstring to mention pipeline-tag stage
- Rewrite tests to mock get_pipeline_tag at source, verify call args
Addresses review feedback:
- Timeout on HfApi().model_info() (was unbounded)
- logger.debug breadcrumb instead of bare pass
- Centralized Hub access (no more reaching into private _is_local_path)
- Stronger test assertion (mock_tag.assert_called_once_with)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Reviewed the pipeline_tag fallback — the core idea is sound and the resolve_task dispatch is well covered. A few things inline worth a look: the KNOWN_TASKS gate admits display-only tasks that aren't ONNX-exportable; this adds a Hub network call to the shared resolve_task path (build/eval/inference, not just inspect); the 1e/1d labels read out of execution order; and the local-path test mocks the function under test so it doesn't cover the skip.
One more, not inline: the PR description looks slightly stale — it says the HfApi().model_info(...) call is inlined in resolution.py and lists 2 changed files, but the implementation factors it into a get_pipeline_tag helper in utils/hub_utils.py (3 files). Worth updating so the description matches.
… fix - Gate Stage 1d pipeline_tag on the model-type's ONNX-exportable set (get_supported_tasks) instead of the full KNOWN_TASKS display taxonomy, so a non-exportable pipeline label (text-to-image, reinforcement-learning, ...) degrades to the last-resort default instead of failing at Stage 2/3. - Lower _PIPELINE_TAG_TIMEOUT from 10s to 5s (best-effort fallback shouldn't block long on unreachable/slow networks). - Rename inline stage labels so they match execution order: pipeline_tag is now 1d and last-resort default is 1e (was mislabeled 1e/1d). - Export get_pipeline_tag from utils package (public API). - Add focused unit tests for get_pipeline_tag itself (local-path skip without network, tag extraction, missing tag, API-error fallthrough). - Update Stage 1d resolver tests for the exportable-set gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Main advanced 4 commits (dynamic export controls in the config command #1106, pipeline_tag task fallback #1113, dynamic-axes e2e tests, a recipe), re- conflicting the PR and blocking CI. Re-merge to restore mergeability. Conflicts: - utils/__init__.py: take origin's fuller barrel (hub_utils + model_input + the new get_pipeline_tag) plus this branch's manifest exports. Origin's is a superset; this branch's consumers use submodule imports, so nothing breaks, and main's public API is preserved. - commands/config.py: keep origin's new dynamic-export options (--input-specs/--export-config/--dynamic-axes; signature+body already auto-merged them) alongside this branch's inline --device Choice and EpAtSourceParamType --ep. `winml config --help` renders all options. Claude-Session: https://claude.ai/code/session_01CRVeL37EPZvGhMpQNj1hTF
Summary
When architecture-based task detection (Stage 1c) fails because
config.architecturescontains a generic name like'Model'that doesn't map to any real transformers class, task resolution falls through to the last-resortfeature-extractiondefault. However, HuggingFace Hub knows the correct task viapipeline_tag(e.g.audio-classificationforaudeering/wav2vec2-large-robust-24-ft-age-gender).This PR adds a Stage 1e between Stage 1c (TasksManager architecture detection) and Stage 1d (last-resort default) that queries the Hub's
pipeline_tagas a fallback.Changes
src/winml/modelkit/loader/resolution.pyTaskSource.PIPELINE_TAG = "pipeline-tag"enum value for provenance trackingKNOWN_TASKSto imports from.taskresolve_task():opt_task is None(architecture detection failed) andmodel_idexistsmodel_idis not a local path via_is_local_pathfromhub_utilsHfApi().model_info(model_id).pipeline_tagKNOWN_TASKStry/except Exceptionfor graceful fallthrough on network errors or invalid model IDstests/unit/loader/test_detect_task.pyAdded 4 tests: