Skip to content

Should unregistering a tool cancel an in-flight executions? #218

Description

@domfarolino

How should tool unregistration interact with a tool execution that is already in-flight? This arises in two common patterns:

  1. Declarative Tools (React / SPA Form Detachment):
    When a declarative form tool (<form toolname="...">) is submitted, the site calls e.respondWith(promise). After this method is called, apps built with frameworks like React might unmount the form element after submission, and render a success message instead. This removes the <form> from the DOM while promise is still pending. The plan is to make DOM removal trigger tool unregistration (what Chromium does now), which means we'd end up synchronously unregistering the tool before promise resolves, and before the invoker of the tool gets any response.
  2. Imperative Tools:
    A tool caller invokes modelContext.executeTool() (soon to be spec'd) and gets a Promise representing the result. After tool call execution has started, but before the ToolExecuteCallback Promise resolves, the site hosting the tool aborts the registration signal, thus unregistering the tool.

Right now in Chromium, unregistering a tool unconditionally aborts any in-flight executions. The result is a tricky footgun we're observing: any declarative form using e.respondWith() will fail in React or SPAs that unmount forms immediately upon submission.

The question

Should we cancel pending executions for a tool when it is unregistered, or should we let them settle naturally? I personally think that tool unregistration should cancel pending invocations, to keep things simple, but I'm also not sure how big of a footgun this is for apps using frameworks.

We first observed this issue in sites using React. The pattern was something like:

import { useState } from 'react';

export default function ToggleForm() {
  const [isSubmitted, setIsSubmitted] = useState(false);

  const handleSubmit = (e: React.SubmitEvent) => {
    e.preventDefault();
    if (e.nativeEvent.agentInvoked) {
      e.nativeEvent.respondWith('Your submission was successful');
    }
    setIsSubmitted(true);
  };

  // If submitted, show a success message instead of the form
  if (isSubmitted) {
    return <p>Your submission was successful and the form has been removed.</p>;
  }

Initially, our thought was just to gate the setIsSubmitted(true); behind a macrotask to let the Promise generated by respondWith() resolve first:

-    setIsSubmitted(true);
+     setTimeout(() => {
+       setIsSubmitted(true);
+     }, 0);

But I noted that for any serious tool call, the data piped through respondWith() may be async, so we'll need to generalize this advice to something like:

  const handleSubmit = (e: React.SubmitEvent) => {
    let p = Promise.resolve();
    e.preventDefault();
    if (e.nativeEvent.agentInvoked) {
      p = fetch('/get-async-response').then(r => r.json());
      e.nativeEvent.respondWith(p);
    }

    p.then(() => {
      setIsSubmitted(true);
    });
  };

Is this reasonable advice to give developer? Should we solve it at the platform layer by decoupling tool unregistration and in-flight execution?

/cc @sdras, @beaufortfrancois, @juanferrub

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions