Add single-use callback token for deadline callback context fetch#69840
Draft
seanghaeli wants to merge 2 commits into
Draft
Add single-use callback token for deadline callback context fetch#69840seanghaeli wants to merge 2 commits into
seanghaeli wants to merge 2 commits into
Conversation
boring-cyborg
Bot
added
area:API
seanghaeli
force-pushed
the
callback-token-apache
branch
from
fa89c3b to
f97d103
Compare
seanghaeli
changed the title
Deadline callbacks run in a subprocess that needs to read the DagRun context (and connections/variables/xcoms) from the Execution API. PR also accept the long-lived ``workload`` token via ``token:workload`` opt-ins. That over-broadened the workload token's reach (scope creep): a long-lived token could read arbitrary DagRun/connection/variable/xcom data for the whole queue-wait lifetime, and Ash asked for a single-use credential instead. This re-lands the security core of apache#66608 with a tighter design: * New ``callback`` token scope. The executor mints a ``callback``-scoped token whose ``sub`` is the callback id. That token is accepted ONLY on the new ``PATCH /callbacks/{id}/run`` exchange endpoint (pinned by a ``callback:self`` scope) and nowhere else. * Single-use exchange. ``run_callback`` performs an atomic ``QUEUED -> RUNNING`` transition under ``SELECT ... FOR UPDATE`` and returns a fresh short-lived ``execution`` token via the ``Refreshed-API-Token`` header. A second presentation of the same callback token finds the row already RUNNING and gets 409, so the token is exchangeable exactly once. This mirrors the Task Instance ``/run`` pattern. The subprocess calls the exchange once, before any context read, then uses the execution token for the actual reads. * Reverted the apache#66608 ``token:workload`` opt-ins on the DagRun, connections, variables, and xcoms GET endpoints back to execution-only. * ``JWTReissueMiddleware`` skips ``callback`` tokens (as it already does for ``workload``), since they are long-lived and exchanged exactly once. Two shared-helper refactors keep the new code DRY and also clean up pre-existing duplication: * ``require_auth`` now drives the three ``*:self`` scope checks (``ti:self`` / ``ct:self`` / ``callback:self``) from a single ``SELF_SCOPE_PATH_PARAMS`` mapping + loop instead of three near-identical branches. Behavior for ``ti:self`` and ``ct:self`` is unchanged. * New ``issue_execution_token`` helper mints an execution-scope token and sets the ``Refreshed-API-Token`` header. It replaces the duplicated mint block in the Task Instance ``/run`` handler (still guarded by the ``workload`` scope check) and the new callback exchange endpoint.
seanghaeli
force-pushed
the
callback-token-apache
branch
from
f97d103 to
02af3fd
Compare
The new PATCH /callbacks/{callback_id}/run route accepts the single-use
'callback' token, a non-default (non-execution) scope. The token scope
boundary regression guard requires such routes to be declared explicitly
in NON_DEFAULT_TOKEN_POLICY; add it there per the test's documented
maintenance step.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds single-use
callbacktoken so a deadline-callback subprocess can fetch its DagRun context from the Execution API without holding a broadworkloadtoken (the scope creep that got #66608 reverted).PATCH /callbacks/{id}/run: single-use exchange gated on an atomicCallback.stateQUEUED→RUNNING transition (409 on reuse), returning a short-livedexecutiontoken viaRefreshed-API-Token. Mirrors the TI/runpattern.callbacktoken scope +callback:selfcheck (token subject must match the callback id).issue_execution_tokenhelper used by both the callback exchange and TI/run, and collapses the*:selfscope checks into one parameterized check.