Skip to content

fix: after_llm_call hooks no longer break native tool execution#6531

Merged
lucasgomide merged 3 commits into
mainfrom
luzk/fix-after-hook-tool-calls
Jul 14, 2026
Merged

fix: after_llm_call hooks no longer break native tool execution#6531
lucasgomide merged 3 commits into
mainfrom
luzk/fix-after-hook-tool-calls

Conversation

@lucasgomide

@lucasgomide lucasgomide commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Registering any after_llm_call hook broke native tool execution: the executor runs _setup_after_llm_call_hooks on the intermediate response that carries the model's tool calls, the structured payload got stringified for the response-rewrite pass, and the executor then treated that string as the final answer instead of executing the tools. Structured payloads (neither str nor BaseModel) now pass through untouched, matching the guard the direct-call path already applies in _invoke_after_llm_call_hooks; hooks still fire on the follow-up textual response.

Fixes #6529


Note

Low Risk
Small, targeted guard in hook setup with a regression test; behavior change only affects the tool-call intermediate response when after hooks are registered.

Overview
Fixes a regression where any registered after_llm_call hook broke native tool execution: _setup_after_llm_call_hooks ran on the LLM’s intermediate tool-call payload, stringified it for the hook rewrite path, and the executor then treated that string as the final answer instead of running tools (#6529).

_setup_after_llm_call_hooks now returns structured responses unchanged when the answer is neither str nor BaseModel (e.g. native tool-call lists). Hooks still run on normal text and Pydantic responses. This mirrors the guard the direct llm.call() path already uses via _invoke_after_llm_call_hooks.

Adds a regression test ensuring tool-call payloads pass through untouched while hooks still observe textual responses.

Reviewed by Cursor Bugbot for commit 12356b4. Bugbot is set up for automated code reviews on this repo. Configure here.

Registering any `after_llm_call` hook broke native tool execution: the
executor invokes `_setup_after_llm_call_hooks` on the intermediate
response that carries the model's tool calls, the non-str payload was
stringified for the response-rewrite pass, and the executor then treated
that string as a final answer instead of executing the tools. Structured
payloads (neither `str` nor `BaseModel`) now pass through untouched,
mirroring the isinstance guard `_invoke_after_llm_call_hooks` already
applies on the direct-call path; hooks still fire on the follow-up
textual response.

Fixes #6529
@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Tool-call hook preservation

Layer / File(s) Summary
After-LLM response guard and regression coverage
lib/crewai/src/crewai/utilities/agent_utils.py, lib/crewai/tests/hooks/test_llm_hooks.py
The after-LLM hook pipeline bypasses rewriting for non-string, non-BaseModel responses, while regression coverage verifies structured tool-call passthrough and textual hook execution.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address #6529 by letting structured tool-call payloads pass through unchanged so tools can execute normally.
Out of Scope Changes check ✅ Passed The PR stays focused on the hook serialization fix and its regression test, with no obvious unrelated changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title clearly describes the main fix: after_llm_call hooks no longer interfere with native tool execution.
Description check ✅ Passed The description matches the changeset and explains the hook/tool-call regression, the guard, and the regression test.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch luzk/fix-after-hook-tool-calls

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/crewai/src/crewai/utilities/agent_utils.py (1)

1754-1760: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Guard logic is correct; type annotations need widening.

The early-return guard correctly prevents stringification of structured tool-call payloads. However, the function signature at lines 1736 and 1739 still types answer as str | BaseModel and the return as str | BaseModel, even though the guard now allows arbitrary objects (e.g., list[Mock()] in the test) to pass through unchanged. Update the annotations to reflect reality.

♻️ Proposed type annotation update
 def _setup_after_llm_call_hooks(
     executor_context: CrewAgentExecutor | AgentExecutor | LiteAgent | None,
-    answer: str | BaseModel,
+    answer: str | BaseModel | Any,
     printer: Printer,
     verbose: bool = True,
-) -> str | BaseModel:
+) -> str | BaseModel | Any:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/utilities/agent_utils.py` around lines 1754 - 1760,
Widen the answer parameter and return annotations of the function containing the
new isinstance guard to include arbitrary object values, matching its
passthrough behavior for non-str/BaseModel responses. Keep the existing guard
and textual/BaseModel handling unchanged, and use the project’s established
broad object type annotation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/crewai/src/crewai/utilities/agent_utils.py`:
- Around line 1754-1760: Widen the answer parameter and return annotations of
the function containing the new isinstance guard to include arbitrary object
values, matching its passthrough behavior for non-str/BaseModel responses. Keep
the existing guard and textual/BaseModel handling unchanged, and use the
project’s established broad object type annotation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bf19ddef-bd25-401f-af1d-d582a0e8367e

📥 Commits

Reviewing files that changed from the base of the PR and between 9d72e26 and 3bdd042.

📒 Files selected for processing (2)
  • lib/crewai/src/crewai/utilities/agent_utils.py
  • lib/crewai/tests/hooks/test_llm_hooks.py

@lucasgomide lucasgomide merged commit 6452608 into main Jul 14, 2026
55 of 56 checks passed
@lucasgomide lucasgomide deleted the luzk/fix-after-hook-tool-calls branch July 14, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

after_llm_call hook prevents agents from executing tools

2 participants