Add consequentialHint to ToolAnnotations#217
Conversation
|
@domfarolino Please review |
NoClueMike
left a comment
NoClueMike
left a comment
There was a problem hiding this comment.
When did "hints" deserve the time spent over actual communication. In the clients world, they can't even enable api"s for tools.
victorhuangwq
left a comment
victorhuangwq
left a comment
There was a problem hiding this comment.
@beaufortfrancois thanks for adding this!
On a broader note, I'm starting to think if all ToolAnnotations should default to null instead of just false?
For example, I'm not sure if consequential not being set is the same as false.
|
|
||
| **Threats addressed:** [[#misrepresentation-of-intent]] | ||
|
|
||
| **How:** A boolean {{ToolAnnotations/consequentialHint}} annotation acts as a signal to the client or agent that the tool performs a consequential action, such as booking a flight or transferring money. This way they can selectively enforce mandatory user confirmation prompts before executing high-stakes tools, directly mitigating the risk of accidental or malicious misrepresentation of intent. |
There was a problem hiding this comment.
I agree that consequentialHint helps with the accidental case. Using the finalizeCart example, the name and description may be ambiguous about whether the tool merely prepares the cart or completes the purchase. A non-malicious developer still knows that execute triggers a purchase, so they will likely be marking the tool as consequential. This would give the client or agent an explicit signal that it should request confirmation rather than relying only on its interpretation of “finalize.”
I’m less convinced it mitigates the malicious case described here. A malicious developer controls the name, description, implementation, and annotations. If they deliberately describe finalizeCart ambiguously to conceal a purchase, they will omit consequentialHint or set it to false.
3 participants
Fix #176
Preview | Diff