OpenAI compatibility#21
Conversation
Keeps a singular tool, but with a more structured tool schema. The "flat schema of all possible fields" is an artifact of trying to fit anthropic's internal schema into langgraph's infrastructure. OpenAI we're defining the tooling, so we can do better.
Paper over differences in "standardized" (lol) usage metrics, and normalize `list[dict | str]` and `list[dict]` (openai doesn't let you intermix string and dicts within a list of content, booo)
|
|
||
| def _dispatch_openai_op[R]( | ||
| backend: MemoryToolImpl[R], | ||
| op: _CreateOp | _ViewOp | _StrReplaceOp | _InsertOp | _DeleteOp | _RenameOp, |
There was a problem hiding this comment.
| op: _CreateOp | _ViewOp | _StrReplaceOp | _InsertOp | _DeleteOp | _RenameOp, | |
| op: _MemoryOpUnion, |
?
| if vr is not None and len(vr) >= 2: | ||
| rng = (vr[0], vr[1]) |
There was a problem hiding this comment.
is there validation somewhere that this is either None or of length exactly 2?
There was a problem hiding this comment.
er, no, I guess we can tighten the validation here.
| range = (args.view_range[0], args.view_range[1]) | ||
| return backend.view(args.path, range) | ||
|
|
||
| def async_memory_tool(backend: MemoryToolImpl[Awaitable[str]]) -> BaseTool: |
There was a problem hiding this comment.
do you want to rename this to async_anthropic_memory_tool or something?
| ) | ||
| return MemoryTool.as_tool("memory") | ||
|
|
||
| def memory_tool(backend: MemoryToolImpl[str]) -> BaseTool: |
| return to_ret | ||
|
|
||
| to_ret["total_input_tokens"] = usage["input_tokens"] | ||
| to_ret["total_output_tokens"] = usage["output_tokens"] |
There was a problem hiding this comment.
so the logic here is different than before - total used to be input+chached, now it's just input. Why the change?
There was a problem hiding this comment.
The previous logic we had would pull from anthropic specific response metadata. langchain actually exposes a "normalized" token usage metadata object, but its definition of "input tokens" is different from anthropic. Namely, what anthropic calls input tokens are "all tokens which aren't cache writes or reads." Langchain's "input tokens" are "all input tokens" (regardless of cache status); each model provider integration in LC writes code to translate the provider specific metadata to this representation. We want to move to provider agnostic code where possible, but I wasn't willing to break BC and change the meaning of "input tokens" in the existing APIs. Hence this new api, and the new interpretation of "input tokens" (to match LC).
| """ | ||
|
|
||
|
|
||
| def openai_async_memory_tool( |
There was a problem hiding this comment.
I probably just don't understand at all this implementation, but I expected there to be some abstraction layer that dispatches to the correct memory_tool implementation depending on the model. Where is that going to live?
There was a problem hiding this comment.
Yeah, its all upstream: Certora/AIComposer#105
2 participants
invokeandainvokewrappers to paper over the difference in input list schemas (anthropic allows bare strings in lists, openai doesn't)