docs: 交付文档、维护指引与端到端回归
补齐 README、开发计划、维护交接与 E2E 用例,执行最终验收。
This commit is contained in:
1 parent
18732801b5
commit
6cd904d460
216 files changed
+37038
No files matched your search
@@ -0,0 +1,27 @@
|
||||
import { test, expect, addKnowledgeText, unique, AI_REPLY } from '@e2e'
|
||||
|
||||
test.describe('knowledge base', () => {
|
||||
test('an answer is grounded in text added to the knowledge base @smoke @knowledge', async ({ agent, widget, llm, adminApi, org }) => {
|
||||
// A token no model could guess, so finding it in the search result proves the
|
||||
// text was indexed and retrieved — not echoed from the prompt.
|
||||
const policyCode = unique('RET')
|
||||
await addKnowledgeText(adminApi, {
|
||||
orgId: org.orgId,
|
||||
agentId: agent.id,
|
||||
title: `Returns policy ${policyCode}`, // titles are unique per org
|
||||
content: `Returns policy: items can be returned within 45 days of delivery. Quote reference ${policyCode} on the return label.`,
|
||||
})
|
||||
await llm.script(agent, (s) => s
|
||||
.when(/return/i).callTool('search_knowledge_base', { query: 'returns policy' })
|
||||
.thenReply('You can return items within 45 days of delivery.')
|
||||
.otherwise().reply('Happy to help!'))
|
||||
|
||||
await widget.embed()
|
||||
await widget.open()
|
||||
await widget.send('What is your returns policy?')
|
||||
|
||||
await expect(widget.lastAgentMessage()).toHaveText('You can return items within 45 days of delivery.', AI_REPLY)
|
||||
const searchResults = (await llm.requests(agent)).flatMap((request) => request.toolResults)
|
||||
expect(searchResults.join('\n')).toContain(policyCode)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,41 @@
|
||||
import { test, expect, addKnowledgeText, deleteKnowledge, unique, AI_REPLY, type TestAgent, type FakeLlm, type ChatWidget } from '@e2e'
|
||||
|
||||
// What the model is shown is what the knowledge base holds for *this* agent, now.
|
||||
|
||||
async function askAboutReturns(llm: FakeLlm, agent: TestAgent, widget: ChatWidget): Promise<string> {
|
||||
await llm.script(agent, (s) => s
|
||||
.when(/return/i).callTool('search_knowledge_base', { query: 'returns policy' })
|
||||
.thenReply('Let me check our policy.')
|
||||
.otherwise().reply('Happy to help!'))
|
||||
await widget.embed()
|
||||
await widget.open()
|
||||
await widget.send('What is your returns policy?')
|
||||
await expect(widget.lastAgentMessage()).toHaveText('Let me check our policy.', AI_REPLY)
|
||||
return (await llm.requests(agent)).flatMap((r) => r.toolResults).join('\n')
|
||||
}
|
||||
|
||||
test.describe('knowledge base lifecycle', () => {
|
||||
test('deleted knowledge is no longer what the AI is shown @knowledge', async ({ agent, widget, llm, adminApi, org, makeWidget }) => {
|
||||
const code = unique('RET')
|
||||
const id = await addKnowledgeText(adminApi, {
|
||||
orgId: org.orgId, agentId: agent.id, title: `Returns policy ${code}`,
|
||||
content: `Returns policy: items can be returned within 45 days. Reference ${code}.`,
|
||||
})
|
||||
expect(await askAboutReturns(llm, agent, widget)).toContain(code)
|
||||
|
||||
await deleteKnowledge(adminApi, id)
|
||||
const freshVisitor = await makeWidget(agent.id)
|
||||
expect(await askAboutReturns(llm, agent, freshVisitor)).not.toContain(code)
|
||||
})
|
||||
|
||||
test('one agent\'s knowledge never reaches another agent\'s answers @knowledge', async ({ agent, widget, llm, adminApi, org, makeAgent }) => {
|
||||
const other = await makeAgent()
|
||||
const secret = unique('INTERNAL')
|
||||
await addKnowledgeText(adminApi, {
|
||||
orgId: org.orgId, agentId: other.id, title: `Staff returns rules ${secret}`,
|
||||
content: `Staff-only returns rules: discount code ${secret} for internal returns.`,
|
||||
})
|
||||
// `agent` searches the same org's knowledge base, for the same words.
|
||||
expect(await askAboutReturns(llm, agent, widget)).not.toContain(secret)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,24 @@
|
||||
import { test, expect, runWorkerOnce, textPdf, unique, unfinishedKnowledge, uploadKnowledgePdf, AI_REPLY, KNOWLEDGE_INDEXED } from '@e2e'
|
||||
|
||||
test('an uploaded PDF becomes knowledge the AI answers from @knowledge @workers', async ({ agent, widget, llm, adminApi, org }) => {
|
||||
const code = unique('WARRANTY')
|
||||
await uploadKnowledgePdf(adminApi, {
|
||||
orgId: org.orgId, agentId: agent.id, filename: `${code}.pdf`,
|
||||
pdf: textPdf(['Warranty policy', `Every product carries a two-year warranty. Reference ${code}.`]),
|
||||
})
|
||||
// Production indexes it on the knowledge worker's next poll; here, now.
|
||||
await runWorkerOnce('knowledge')
|
||||
// Tests run in parallel: another test's pass may be the one indexing it.
|
||||
await expect.poll(() => unfinishedKnowledge(adminApi, agent.id), KNOWLEDGE_INDEXED).toEqual([])
|
||||
|
||||
await llm.script(agent, (s) => s
|
||||
.when(/warranty/i).callTool('search_knowledge_base', { query: 'warranty policy' })
|
||||
.thenReply('Every product has a two-year warranty.')
|
||||
.otherwise().reply('Happy to help!'))
|
||||
await widget.embed()
|
||||
await widget.open()
|
||||
await widget.send('What warranty do you offer?')
|
||||
|
||||
await expect(widget.lastAgentMessage()).toHaveText('Every product has a two-year warranty.', AI_REPLY)
|
||||
expect((await llm.requests(agent)).flatMap((r) => r.toolResults).join('\n')).toContain(code)
|
||||
})
|
||||
Reference in new issue
Block a user