feat: bootstrap RAG knowledge base and add think-stage intent eval suite#1496
Open
YuZhangLarry wants to merge 1 commit into
Open
feat: bootstrap RAG knowledge base and add think-stage intent eval suite#1496YuZhangLarry wants to merge 1 commit into
YuZhangLarry wants to merge 1 commit into
Conversation
Task A — seed data: bundle 5 curated Dubbo/Dubbo-Admin knowledge docs (component/rag/seeds/*.md), embed them via //go:embed, and index on startup (best-effort, batched ≤10 to respect the dashscope embedding per-request cap). Adds seed.enabled config + schema; fixes cmd/index.go to target the default index the runtime retriever reads from. Task B — call timing: rewrite agentThink prompt to answer directly by default and only suggest query_knowledge_base for version-sensitive / precise-fact lookups. Fix think→act type assertion (ParseThinkOutput returns a pointer) so the think stage's no-tool decision is honored — without this the prompt change had no effect. Lower max_iterations 10→3 and timeout 60→30 for faster fallback. Tests: multi-turn e2e validates the seed retrieval path end-to-end; new isolated single-turn think test (ThinkOnce) asserts documentation questions classify as DOCUMENTATION_QUERY free of multi-turn bias. Also ignore /.env.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ai/component/rag/): bundled seed docs (seeds/*.md— Dubbooverview, configuration, registry, traffic governance, Dubbo Admin) are
embedded via
//go:embedand indexed at startup, so a fresh deployment cananswer documentation-grade questions out of the box. Indexing is best-effort
and idempotent (dedup by content hash) and batched to the embedding backend's
per-request input limit; a missing embedder key degrades gracefully instead of
failing startup, and reranking falls back to raw vector results when no rerank
key is configured.
ai/component/agent/,ai/prompts/agentThink.txt):the think stage now prefers answering directly (
suggested_tools: []) and nolonger confuses objective documentation lookups (config keys, defaults,
required addresses) with memory search mid-conversation. Adds a
ThinkOnceentry point so the stage can be evaluated in isolation.
ai/test/e2e/):TestThinkClassificationIsolatedruns each documentation question in a freshsession (no multi-turn bias) and repeats it to surface nondeterminism as a
distribution, asserting a stable
DOCUMENTATION_QUERYclassification routed toquery_knowledge_base;TestMultiTurnConversationexercises the fullthink→act→observe loop end to end.