Align folder structure of EN documentations with CN documentations
Mark Sun
parent
d8a8900091
commit
a9efced76e
Binary file not shown.
|
After Width: | Height: | Size: 552 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 229 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 106 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 140 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 492 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 223 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 94 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 69 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 370 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 236 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 406 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 229 KiB |
219
en/SUMMARY.md
219
en/SUMMARY.md
|
|
@ -2,117 +2,124 @@
|
|||
|
||||
## Getting Started
|
||||
|
||||
* [Welcome to Dify!](README.md)
|
||||
* [Technical Spec](getting-started/readme/specifications-and-technical-features.md)
|
||||
* [Model Providers](getting-started/readme/model-providers.md)
|
||||
* [Using Dify Cloud](getting-started/cloud.md)
|
||||
* [Install (Self hosted)](getting-started/install-self-hosted/README.md)
|
||||
* [Welcome to Dify](README.md)
|
||||
* [Features and Specifications](getting-started/readme/features-and-specifications.md)
|
||||
* [Model Providers List](getting-started/readme/model-providers.md)
|
||||
* [Cloud Services](getting-started/cloud.md)
|
||||
* [Community Edition](getting-started/install-self-hosted/README.md)
|
||||
* [Docker Compose Deployment](getting-started/install-self-hosted/docker-compose.md)
|
||||
* [Local Source Code Start](getting-started/install-self-hosted/local-source-code.md)
|
||||
* [Start the frontend Docker container separately](getting-started/install-self-hosted/start-the-frontend-docker-container.md)
|
||||
* [Environments](getting-started/install-self-hosted/environments.md)
|
||||
* [FAQ](getting-started/install-self-hosted/install-faq.md)
|
||||
* [What is LLMOps?](getting-started/what-is-llmops.md)
|
||||
* [Local Source Code Startup](getting-started/install-self-hosted/local-source-code.md)
|
||||
* [Start Frontend Docker Container Separately](getting-started/install-self-hosted/start-the-frontend-docker-container.md)
|
||||
* [Environment Variables Explanation](getting-started/install-self-hosted/environments.md)
|
||||
|
||||
## User Guide
|
||||
## Guides
|
||||
|
||||
* [Creating Dify Apps](user-guide/creating-dify-apps/README.md)
|
||||
* [Quickstart](user-guide/creating-dify-apps/creating-an-application.md)
|
||||
* [Overview](user-guide/creating-dify-apps/overview.md)
|
||||
* [Setting Prompts](user-guide/creating-dify-apps/prompt-engineering/README.md)
|
||||
* [Agent Assistant](user-guide/creating-dify-apps/prompt-engineering/agent-assistant.md)
|
||||
* [Chat App](user-guide/creating-dify-apps/prompt-engineering/conversation-application.md)
|
||||
* [Text Generator](user-guide/creating-dify-apps/prompt-engineering/text-generation-application.md)
|
||||
* [FAQ](user-guide/creating-dify-apps/llms-use-faq.md)
|
||||
* [Use Cases](user-guide/creating-dify-apps/use-cases/README.md)
|
||||
* [Notion AI Assistant Based on Your Own Notes](user-guide/creating-dify-apps/use-cases/build-an-notion-ai-assistant.md)
|
||||
* [AI ChatBot with Business Data](user-guide/creating-dify-apps/use-cases/create-an-ai-chatbot-with-business-data-in-minutes.md)
|
||||
* [Midjourney Prompt Bot](user-guide/creating-dify-apps/use-cases/create-a-midjourney-prompt-bot-with-dify.md)
|
||||
* [Launching Dify Apps](user-guide/launching-dify-apps/README.md)
|
||||
* [Quickstart](user-guide/launching-dify-apps/launch-webapp.md)
|
||||
* [Developing with APIs](user-guide/launching-dify-apps/developing-with-apis/README.md)
|
||||
* [FAQ](user-guide/launching-dify-apps/developing-with-apis/api-use-faq.md)
|
||||
* [Using Dify Apps](user-guide/using-dify-apps/README.md)
|
||||
* [Text Generator](user-guide/using-dify-apps/text-generator.md)
|
||||
* [Chat App](user-guide/using-dify-apps/conversation-application.md)
|
||||
* [Further Chat App Settings](user-guide/using-dify-apps/chat.md)
|
||||
|
||||
## Features
|
||||
|
||||
* [Prompting Expert Mode](features/prompt-engineering/README.md)
|
||||
* [Prompt Template](features/prompt-engineering/prompt-template.md)
|
||||
* [Workflow](features/workflow/README.md)
|
||||
* [Introduce](features/workflow/introduce.md)
|
||||
* [Key Concept](features/workflow/key-concept.md)
|
||||
* [Nodes](features/workflow/node/README.md)
|
||||
* [Start](features/workflow/node/start.md)
|
||||
* [End](features/workflow/node/end.md)
|
||||
* [Answer](features/workflow/node/answer.md)
|
||||
* [LLM](features/workflow/node/llm.md)
|
||||
* [Knowledge Retrieval](features/workflow/node/knowledge-retrieval.md)
|
||||
* [Question Classifier](features/workflow/node/question-classifier.md)
|
||||
* [IF/ELSE](features/workflow/node/if-else.md)
|
||||
* [Code](features/workflow/node/code.md)
|
||||
* [Template](features/workflow/node/template.md)
|
||||
* [Variable Assigner](features/workflow/node/variable-assigner.md)
|
||||
* [HTTP Request](features/workflow/node/http-request.md)
|
||||
* [Tools](features/workflow/node/tools.md)
|
||||
* [Preview\&Run](features/workflow/preview-and-run/README.md)
|
||||
* [Preview\&Run](features/workflow/preview-and-run/preview-and-run.md)
|
||||
* [Step Test](features/workflow/preview-and-run/step-test.md)
|
||||
* [Log](features/workflow/preview-and-run/log.md)
|
||||
* [Checklist](features/workflow/preview-and-run/checklist.md)
|
||||
* [History](features/workflow/preview-and-run/history.md)
|
||||
* [Publish](features/workflow/publish.md)
|
||||
* [Export/Import](features/workflow/export-import.md)
|
||||
* [RAG (Retrieval Augmented Generation)](features/retrieval-augment/README.md)
|
||||
* [Hybrid Search](features/retrieval-augment/hybrid-search.md)
|
||||
* [Rerank](features/retrieval-augment/rerank.md)
|
||||
* [Retrieval](features/retrieval-augment/retrieval.md)
|
||||
* [Knowledge Import](features/datasets/README.md)
|
||||
* [Sync from Notion](features/datasets/sync-from-notion.md)
|
||||
* [Maintain Knowledge Via Api](features/datasets/maintain-dataset-via-api.md)
|
||||
* [External Data Tool](features/external\_data\_tool.md)
|
||||
* [Annotation Reply](features/annotation-reply.md)
|
||||
* [Logs & Annotations](features/logs.md)
|
||||
* [Plugins](features/ai-plugins/README.md)
|
||||
* [Based on WebApp Template](features/ai-plugins/based-on-frontend-templates.md)
|
||||
* [More Integration](features/more-integration.md)
|
||||
* [Extension](features/extension/README.md)
|
||||
* [API Based Extension](features/extension/api\_based\_extension/README.md)
|
||||
* [External\_data\_tool](features/extension/api\_based\_extension/external\_data\_tool.md)
|
||||
* [Moderation Extension](features/extension/api\_based\_extension/moderation-extension.md)
|
||||
* [Code-based Extension](features/extension/code-based-extension.md)
|
||||
* [Moderation](features/moderation\_tool.md)
|
||||
|
||||
## workspace
|
||||
|
||||
* [Discovery](workspace/app.md)
|
||||
* [Billing](workspace/billing.md)
|
||||
|
||||
## Tutorials
|
||||
|
||||
* [Quick Tool Integration](tutorials/quick-tool-integration.md)
|
||||
* [Advanced Tool Integration](tutorials/advanced-tool-integration.md)
|
||||
* [Expose API Extension on public Internet using Cloudflare Workers](tutorials/cloudflare\_workers.md)
|
||||
* [Connecting with Different Models](tutorials/model-configuration/README.md)
|
||||
* [Hugging Face](tutorials/model-configuration/hugging-face.md)
|
||||
* [Replicate](tutorials/model-configuration/replicate.md)
|
||||
* [Xinference](tutorials/model-configuration/xinference.md)
|
||||
* [OpenLLM](tutorials/model-configuration/openllm.md)
|
||||
* [LocalAI](tutorials/model-configuration/localai.md)
|
||||
* [Ollama](tutorials/model-configuration/ollama.md)
|
||||
* [Vector Database Migrate Tool](tutorials/vector-database-migrate-tool.md)
|
||||
* [Connecting with Different Tools](tutorials/tool-configuration/README.md)
|
||||
* [Stable Diffusion](tutorials/tool-configuration/stable-diffusion.md)
|
||||
* [SearXNG](tutorials/tool-configuration/searxng.md)
|
||||
* [Model](guides/model-configuration/README.md)
|
||||
* [Add New Provider](guides/model-configuration/new-provider.md)
|
||||
* [Predefined Model Integration](guides/model-configuration/predefined-model.md)
|
||||
* [Customizable Model Integration](guides/model-configuration/customizable-model.md)
|
||||
* [Integrate Open Source Models from Hugging Face](guides/model-configuration/hugging-face.md)
|
||||
* [Integrate Open Source Models from Replicate](guides/model-configuration/replicate.md)
|
||||
* [Integrate Local Models Deployed by Xinference](guides/model-configuration/xinference.md)
|
||||
* [Integrate Local Models Deployed by OpenLLM](guides/model-configuration/openllm.md)
|
||||
* [Integrate Local Models Deployed by LocalAI](guides/model-configuration/localai.md)
|
||||
* [Integrate Local Models Deployed by Ollama](guides/model-configuration/ollama.md)
|
||||
* [Load Balancing](guides/model-configuration/load-balancing.md)
|
||||
* [Build](guides/application-design/README.md)
|
||||
* [Create an Application](guides/application-design/creating-an-application.md)
|
||||
* [Chat Assistant](guides/application-design/conversation-application.md)
|
||||
* [Agent](guides/application-design/agent-assistant.md)
|
||||
* [Text Generation Application](guides/application-design/text-generation-application.md)
|
||||
* [Prompt Engineering](guides/application-design/prompt-engineering/README.md)
|
||||
* [Prompt Engineering Expert Mode (Deprecated)](guides/application-design/prompt-engineering/prompt-engineering-expert-mode.md)
|
||||
* [Prompt Template Reference](guides/application-design/prompt-engineering/prompt-template.md)
|
||||
* [Application Extra Features](guides/application-design/application-extra-features/README.md)
|
||||
* [Moderation Tool](guides/application-design/application-extra-features/moderation-tool.md)
|
||||
* [Workflow](guides/workflow/README.md)
|
||||
* [Key Concepts](guides/workflow/key-concept.md)
|
||||
* [Node Descriptions](guides/workflow/node/README.md)
|
||||
* [Start](guides/workflow/node/start.md)
|
||||
* [End](guides/workflow/node/end.md)
|
||||
* [Answer](guides/workflow/node/answer.md)
|
||||
* [LLM](guides/workflow/node/llm.md)
|
||||
* [Knowledge Retrieval](guides/workflow/node/knowledge-retrieval.md)
|
||||
* [Question Classifier](guides/workflow/node/question-classifier.md)
|
||||
* [IF/ELSE](guides/workflow/node/if-else.md)
|
||||
* [Code](guides/workflow/node/code.md)
|
||||
* [Template](guides/workflow/node/template.md)
|
||||
* [Variable Aggregator](guides/workflow/node/variable-assigner.md)
|
||||
* [Iteration](guides/workflow/node/iteration.md)
|
||||
* [Parameter Extractor](guides/workflow/node/parameter-extractor.md)
|
||||
* [HTTP Request](guides/workflow/node/http-request.md)
|
||||
* [Tools](guides/workflow/node/tools.md)
|
||||
* [Debug and Preview](guides/workflow/debug-and-preview/README.md)
|
||||
* [Preview and Run](guides/workflow/debug-and-preview/preview-and-run.md)
|
||||
* [Step Test](guides/workflow/debug-and-preview/step-test.md)
|
||||
* [Logs](guides/workflow/debug-and-preview/log.md)
|
||||
* [Checklist](guides/workflow/debug-and-preview/checklist.md)
|
||||
* [Run History](guides/workflow/debug-and-preview/history.md)
|
||||
* [Application Publishing](guides/workflow/publish.md)
|
||||
* [Export/Import Templates](guides/workflow/export-import.md)
|
||||
* [Knowledge Base](guides/knowledge-base/README.md)
|
||||
* [Create Knowledge Base & Upload Documents](guides/knowledge-base/create-knowledge-and-upload-documents.md)
|
||||
* [Knowledge Base and Document Maintenance](guides/knowledge-base/knowledge-and-documents-maintenance.md)
|
||||
* [Integrate Knowledge Base within Application](guides/knowledge-base/integrate-knowledge-within-application.md)
|
||||
* [Retrieval Test/Citation](guides/knowledge-base/retrieval-test-and-citation.md)
|
||||
* [Sync Data from Notion](guides/knowledge-base/sync-from-notion.md)
|
||||
* [Maintain Knowledge Base via API](guides/knowledge-base/maintain-dataset-via-api.md)
|
||||
* [External Data Tool](guides/knowledge-base/external-data-tool.md)
|
||||
* [Tools](guides/tools/README.md)
|
||||
* [Quick Tool Integration](guides/tools/quick-tool-integration.md)
|
||||
* [Advanced Tool Integration](guides/tools/advanced-tool-integration.md)
|
||||
* [Tool Configuration](guides/tools/tool-configuration/README.md)
|
||||
* [StableDiffusion](guides/tools/tool-configuration/stable-diffusion.md)
|
||||
* [SearXNG](guides/tools/tool-configuration/searxng.md)
|
||||
* [Publishing](guides/application-publishing/README.md)
|
||||
* [Launch Your Webapp Quickly](guides/application-publishing/launch-your-webapp-quickly/README.md)
|
||||
* [Web App Settings](guides/application-publishing/launch-your-webapp-quickly/web-app-settings.md)
|
||||
* [Text Generator Application](guides/application-publishing/launch-your-webapp-quickly/text-generator.md)
|
||||
* [Conversation Application](guides/application-publishing/launch-your-webapp-quickly/conversation-application.md)
|
||||
* [Developing with APIs](guides/application-publishing/developing-with-apis/README.md)
|
||||
* [Re-develop Based on Frontend Components](guides/application-publishing/based-on-frontend-templates.md)
|
||||
* [Annotation](guides/biao-zhu/README.md)
|
||||
* [Logs and Annotation](guides/biao-zhu/logs.md)
|
||||
* [Annotation Reply](guides/biao-zhu/annotation-reply.md)
|
||||
* [Monitoring](guides/monitoring/README.md)
|
||||
* [Data Analysis](guides/monitoring/analysis.md)
|
||||
* [Extension](guides/extension/README.md)
|
||||
* [API-based Extension](guides/extension/api-based-extension/README.md)
|
||||
* [External Data Tool](guides/extension/api-based-extension/external-data-tool.md)
|
||||
* [Deploy API Tools using Cloudflare Workers](guides/extension/api-based-extension/cloudflare-workers.md)
|
||||
* [Moderation](guides/extension/api-based-extension/moderation.md)
|
||||
* [Code-based Extension](guides/extension/code-based-extension/README.md)
|
||||
* [External Data Tool](guides/extension/code-based-extension/external-data-tool.md)
|
||||
* [Moderation](guides/extension/code-based-extension/moderation.md)
|
||||
* [Collaboration](guides/workspace/README.md)
|
||||
* [Invite and Manage Members](guides/workspace/invite-and-manage-members.md)
|
||||
* [Billings](guides/workspace/billing.md)
|
||||
* [Explore](guides/workspace/explore.md)
|
||||
|
||||
## Community
|
||||
|
||||
* [Contributing](community/contributing.md)
|
||||
* [Support](community/support.md)
|
||||
* [Seek Support](community/support.md)
|
||||
* [Become a Contributor](community/contribution.md)
|
||||
|
||||
## User Agreement
|
||||
## Learn More
|
||||
|
||||
* [Open-Source License](user-agreement/open-source.md)
|
||||
* [Data Security](user-agreement/data-security.md)
|
||||
* [Use Cases](learn-more/use-cases/README.md)
|
||||
* [Extended Reading](learn-more/extended-reading/README.md)
|
||||
* [What is LLMOps?](learn-more/extended-reading/what-is-llmops.md)
|
||||
* [Retrieval-Augmented Generation (RAG)](learn-more/extended-reading/retrieval-augment/README.md)
|
||||
* [Hybrid Search](learn-more/extended-reading/retrieval-augment/hybrid-search.md)
|
||||
* [Re-ranking](learn-more/extended-reading/retrieval-augment/rerank.md)
|
||||
* [Retrieval Modes](learn-more/extended-reading/retrieval-augment/retrieval.md)
|
||||
* [FAQ](learn-more/faq/README.md)
|
||||
* [Self-Host Related](learn-more/faq/self-host-faq.md)
|
||||
* [LLM Configuration and Usage](learn-more/faq/use-llms-faq.md)
|
||||
|
||||
## Policies
|
||||
|
||||
* [Open Source License](policies/open-source.md)
|
||||
* [User Agreement and Data Security](policies/agreement/README.md)
|
||||
* [Terms of Service](https://dify.ai/terms)
|
||||
* [Privacy Policy](https://dify.ai/privacy)
|
||||
|
|
@ -0,0 +1,118 @@
|
|||
# Table of contents
|
||||
|
||||
## Getting Started
|
||||
|
||||
* [Welcome to Dify!](README.md)
|
||||
* [Technical Spec](getting-started/readme/specifications-and-technical-features.md)
|
||||
* [Model Providers](getting-started/readme/model-providers.md)
|
||||
* [Using Dify Cloud](getting-started/cloud.md)
|
||||
* [Install (Self hosted)](getting-started/install-self-hosted/README.md)
|
||||
* [Docker Compose Deployment](getting-started/install-self-hosted/docker-compose.md)
|
||||
* [Local Source Code Start](getting-started/install-self-hosted/local-source-code.md)
|
||||
* [Start the frontend Docker container separately](getting-started/install-self-hosted/start-the-frontend-docker-container.md)
|
||||
* [Environments](getting-started/install-self-hosted/environments.md)
|
||||
* [FAQ](getting-started/install-self-hosted/install-faq.md)
|
||||
* [What is LLMOps?](getting-started/what-is-llmops.md)
|
||||
|
||||
## User Guide
|
||||
|
||||
* [Creating Dify Apps](user-guide/creating-dify-apps/README.md)
|
||||
* [Quickstart](user-guide/creating-dify-apps/creating-an-application.md)
|
||||
* [Overview](user-guide/creating-dify-apps/overview.md)
|
||||
* [Setting Prompts](user-guide/creating-dify-apps/prompt-engineering/README.md)
|
||||
* [Agent Assistant](user-guide/creating-dify-apps/prompt-engineering/agent-assistant.md)
|
||||
* [Chat App](user-guide/creating-dify-apps/prompt-engineering/conversation-application.md)
|
||||
* [Text Generator](user-guide/creating-dify-apps/prompt-engineering/text-generation-application.md)
|
||||
* [FAQ](user-guide/creating-dify-apps/llms-use-faq.md)
|
||||
* [Use Cases](user-guide/creating-dify-apps/use-cases/README.md)
|
||||
* [Notion AI Assistant Based on Your Own Notes](user-guide/creating-dify-apps/use-cases/build-an-notion-ai-assistant.md)
|
||||
* [AI ChatBot with Business Data](user-guide/creating-dify-apps/use-cases/create-an-ai-chatbot-with-business-data-in-minutes.md)
|
||||
* [Midjourney Prompt Bot](user-guide/creating-dify-apps/use-cases/create-a-midjourney-prompt-bot-with-dify.md)
|
||||
* [Launching Dify Apps](user-guide/launching-dify-apps/README.md)
|
||||
* [Quickstart](user-guide/launching-dify-apps/launch-webapp.md)
|
||||
* [Developing with APIs](user-guide/launching-dify-apps/developing-with-apis/README.md)
|
||||
* [FAQ](user-guide/launching-dify-apps/developing-with-apis/api-use-faq.md)
|
||||
* [Using Dify Apps](user-guide/using-dify-apps/README.md)
|
||||
* [Text Generator](user-guide/using-dify-apps/text-generator.md)
|
||||
* [Chat App](user-guide/using-dify-apps/conversation-application.md)
|
||||
* [Further Chat App Settings](user-guide/using-dify-apps/chat.md)
|
||||
|
||||
## Features
|
||||
|
||||
* [Prompting Expert Mode](features/prompt-engineering/README.md)
|
||||
* [Prompt Template](features/prompt-engineering/prompt-template.md)
|
||||
* [Workflow](features/workflow/README.md)
|
||||
* [Introduce](features/workflow/introduce.md)
|
||||
* [Key Concept](features/workflow/key-concept.md)
|
||||
* [Nodes](features/workflow/node/README.md)
|
||||
* [Start](features/workflow/node/start.md)
|
||||
* [End](features/workflow/node/end.md)
|
||||
* [Answer](features/workflow/node/answer.md)
|
||||
* [LLM](features/workflow/node/llm.md)
|
||||
* [Knowledge Retrieval](features/workflow/node/knowledge-retrieval.md)
|
||||
* [Question Classifier](features/workflow/node/question-classifier.md)
|
||||
* [IF/ELSE](features/workflow/node/if-else.md)
|
||||
* [Code](features/workflow/node/code.md)
|
||||
* [Template](features/workflow/node/template.md)
|
||||
* [Variable Assigner](features/workflow/node/variable-assigner.md)
|
||||
* [HTTP Request](features/workflow/node/http-request.md)
|
||||
* [Tools](features/workflow/node/tools.md)
|
||||
* [Preview\&Run](features/workflow/preview-and-run/README.md)
|
||||
* [Preview\&Run](features/workflow/preview-and-run/preview-and-run.md)
|
||||
* [Step Test](features/workflow/preview-and-run/step-test.md)
|
||||
* [Log](features/workflow/preview-and-run/log.md)
|
||||
* [Checklist](features/workflow/preview-and-run/checklist.md)
|
||||
* [History](features/workflow/preview-and-run/history.md)
|
||||
* [Publish](features/workflow/publish.md)
|
||||
* [Export/Import](features/workflow/export-import.md)
|
||||
* [RAG (Retrieval Augmented Generation)](features/retrieval-augment/README.md)
|
||||
* [Hybrid Search](features/retrieval-augment/hybrid-search.md)
|
||||
* [Rerank](features/retrieval-augment/rerank.md)
|
||||
* [Retrieval](features/retrieval-augment/retrieval.md)
|
||||
* [Knowledge Import](features/datasets/README.md)
|
||||
* [Sync from Notion](features/datasets/sync-from-notion.md)
|
||||
* [Maintain Knowledge Via Api](features/datasets/maintain-dataset-via-api.md)
|
||||
* [External Data Tool](features/external\_data\_tool.md)
|
||||
* [Annotation Reply](features/annotation-reply.md)
|
||||
* [Logs & Annotations](features/logs.md)
|
||||
* [Plugins](features/ai-plugins/README.md)
|
||||
* [Based on WebApp Template](features/ai-plugins/based-on-frontend-templates.md)
|
||||
* [More Integration](features/more-integration.md)
|
||||
* [Extension](features/extension/README.md)
|
||||
* [API Based Extension](features/extension/api\_based\_extension/README.md)
|
||||
* [External\_data\_tool](features/extension/api\_based\_extension/external\_data\_tool.md)
|
||||
* [Moderation Extension](features/extension/api\_based\_extension/moderation-extension.md)
|
||||
* [Code-based Extension](features/extension/code-based-extension.md)
|
||||
* [Moderation](features/moderation\_tool.md)
|
||||
|
||||
## workspace
|
||||
|
||||
* [Discovery](workspace/app.md)
|
||||
* [Billing](workspace/billing.md)
|
||||
|
||||
## Tutorials
|
||||
|
||||
* [Quick Tool Integration](tutorials/quick-tool-integration.md)
|
||||
* [Advanced Tool Integration](tutorials/advanced-tool-integration.md)
|
||||
* [Expose API Extension on public Internet using Cloudflare Workers](tutorials/cloudflare\_workers.md)
|
||||
* [Connecting with Different Models](tutorials/model-configuration/README.md)
|
||||
* [Hugging Face](tutorials/model-configuration/hugging-face.md)
|
||||
* [Replicate](tutorials/model-configuration/replicate.md)
|
||||
* [Xinference](tutorials/model-configuration/xinference.md)
|
||||
* [OpenLLM](tutorials/model-configuration/openllm.md)
|
||||
* [LocalAI](tutorials/model-configuration/localai.md)
|
||||
* [Ollama](tutorials/model-configuration/ollama.md)
|
||||
* [Vector Database Migrate Tool](tutorials/vector-database-migrate-tool.md)
|
||||
* [Connecting with Different Tools](tutorials/tool-configuration/README.md)
|
||||
* [Stable Diffusion](tutorials/tool-configuration/stable-diffusion.md)
|
||||
* [SearXNG](tutorials/tool-configuration/searxng.md)
|
||||
|
||||
## Community
|
||||
|
||||
* [Contributing](community/contributing.md)
|
||||
* [Support](community/support.md)
|
||||
|
||||
## User Agreement
|
||||
|
||||
* [Open-Source License](user-agreement/open-source.md)
|
||||
* [Data Security](user-agreement/data-security.md)
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 228 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 137 KiB |
|
|
@ -1,5 +0,0 @@
|
|||
# Plugins
|
||||
|
||||
{% hint style="info" %}
|
||||
Plugins is an upcoming feature of Dify. You can incorporate plugins into your App orchestration and access AI applications with plugin capabilities through an API or WebApp. Dify is compatible with the ChatGPT Plugins standard and provides some native plugins.
|
||||
{% endhint %}
|
||||
|
|
@ -1,134 +0,0 @@
|
|||
# Knowledge Import
|
||||
|
||||
Most language models use outdated training data and have length limitations for the context of each request. For example, GPT-3.5 is trained on corpora from 2021 and has a limit of approximately 4k tokens per request. This means that developers who want their AI applications to be based on the latest and private context conversations must use techniques like embedding.
|
||||
|
||||
Dify' knowledge feature allows developers (and even non-technical users) to easily manage knowledge and automatically integrate them into AI applications. All you need to do is prepare text content, such as:
|
||||
|
||||
* Long text content (TXT, Markdown, DOCX, HTML, JSONL, or even PDF files)
|
||||
* Structured data (CSV, Excel, etc.)
|
||||
|
||||
Additionally, we are gradually supporting syncing data from various data sources to knowledge, including:
|
||||
|
||||
* GitHub
|
||||
* Databases
|
||||
* Webpages
|
||||
* ...
|
||||
|
||||
{% hint style="info" %}
|
||||
**Practice**: If your company wants to build an AI customer service assistant based on existing knowledge bases and product documentation, you can upload the documents to a knowledge base in Dify and create a conversational application. This might have taken you several weeks in the past and been difficult to maintain continuously.
|
||||
{% endhint %}
|
||||
|
||||
### Knowledge and Documents
|
||||
|
||||
In Dify, knowledge bases are collections of documents. A knowledge base can be integrated as a whole into an application to be used as context. Documents can be uploaded by developers or operations staff, or synced from other data sources (typically corresponding to a file unit in the data source).
|
||||
|
||||
**Steps to upload a document:**
|
||||
|
||||
1. Upload your file, usually a long text file or a spreadsheet
|
||||
2. Segment, clean, and preview
|
||||
3. Dify submits it to the LLM provider for embedding as vector data and storage
|
||||
4. Set metadata for the document
|
||||
5. Ready to use in the application!
|
||||
|
||||
#### How to write a good knowledge description
|
||||
|
||||
When multiple knowledge bases are referenced in an application, AI uses the description of the knowledge and the user's question to determine which knowledge base to use to answer the user's question. Therefore, a well-written knowledge description can improve the accuracy of AI in selecting knowledge.
|
||||
|
||||
The key to writing a good knowledge description is to clearly describe the content and characteristics of the knowledge. **It is recommended that the knowledge description begin with this: `Useful only when the question you want to answer is about the following: specific description`**. Here is an example of a real estate knowledge description:
|
||||
|
||||
> Useful only when the question you want to answer is about the following: global real estate market data from 2010 to 2020. This data includes information such as the average housing price, property sales volume, and housing types for each city. In addition, this knowledge base also includes some economic indicators such as GDP and unemployment rate, as well as some social indicators such as population and education level. These indicators can help analyze the trends and influencing factors of the real estate market. With this data, we can understand the development trends of the global real estate market, analyze the changes in housing prices in various cities, and understand the impact of economic and social factors on the real estate market.
|
||||
|
||||
### Create a knowledge
|
||||
|
||||
1. Click on knowledge in the main navigation bar of Dify. On this page, you can see the existing knowledge bases. Click on "Create Knowledge" to enter the creation wizard.
|
||||
2. If you have already prepared your files, you can start by uploading the files.
|
||||
3. If you haven't prepared your documents yet, you can create an empty knowledge base first.
|
||||
|
||||
### Uploading Documents By upload file
|
||||
|
||||
1. Select the file you want to upload.We support batch uploads
|
||||
2. Preview the full text
|
||||
3. Perform segmentation and cleaning
|
||||
4. Wait for Dify to process the data for you; this step usually consumes tokens in the LLM provider
|
||||
|
||||
### Text Preprocessing and Cleaning
|
||||
|
||||
Text Preprocessing and cleaning refers to Dify automatically segmenting and vectorizing your data documents so that user's questions (input) can match relevant paragraphs (Q to P), and generate results.
|
||||
|
||||
When uploading a knowledge base, you need to select a **indexing mode** to specify how data is matched. This affects the accuracy of AI replies.
|
||||
|
||||
In **High Quality mode**, OpenAI's embedding API is used for higher accuracy in user queries.
|
||||
|
||||
In **Economic mode**, offline vector engines, keyword indexing etc. are used to reduce costs at the expense of lower accuracy.
|
||||
|
||||
In **Segmenting in Question & Answer format**, instead of normal "Q to P" (question matches paragraphs), it uses "Q to Q" (question matches question) matching. After segmentation, Q\&A pairs are generated for each passage. When users ask questions, the system finds the most similar question and returns the corresponding passage as the answer. This is more precise because it directly matches the user's question and retrieves the information they need.
|
||||
|
||||
> Questions have complete syntax while keywords lack semantics and context. So Q to Q improves clarity and handles similar high-frequency questions better.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/segment-list.png" alt=""><figcaption><p>In Segmenting in Question & Answer format, the text is summarized into multiple QA pairs</p></figcaption></figure>
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (63).png" alt=""><figcaption><p>The difference between Q to P and Q to Q indexing modes</p></figcaption></figure>
|
||||
|
||||
### Modify Documents
|
||||
|
||||
Modify Documents For technical reasons, if developers make the following changes to documents, Dify will create a new document for you, and the old document will be archived and deactivated:
|
||||
|
||||
1. Adjust segmentation and cleaning settings
|
||||
2. Re-upload the file
|
||||
|
||||
Dify support customizing the segmented and cleaned text by adding, deleting, and editing paragraphs. You can dynamically adjust your segmentation to make your knowledge more accurate. Click **Document --> paragraph --> Edit** in the knowledge to modify paragraphs content and custom keywords. Click **Document --> paragraph --> Add segment --> Add a segment** to manually add new paragraph. Or click **Document --> paragraph --> Add segment --> Batch add** to batch add new paragraph.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (3) (1) (1) (1) (1).png" alt=""><figcaption><p>Edit</p></figcaption></figure>
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>add</p></figcaption></figure>
|
||||
|
||||
### Disabling and Archiving of Documents
|
||||
|
||||
* **Disable, cancel disable**: The knowledge supports disabling documents or chunks that you temporarily do not want indexed. In the knowledge's document list, click the Disable button and the document will be disabled. You can also click the Disable button in the document details to disable the entire document or a segment. Disabled documents will not be indexed. To cancel the disable, click Enable on a disabled document.
|
||||
* **Archive, Unarchive:** Some unused old document data can be archived if you don't want to delete it. After archiving, the data can only be viewed or deleted, not edited. In the document list of the knowledge, click the Archive button to archive the document. Documents can also be archived in the document details page. Archived documents will not be indexed. Archived documents can also be unarchived by clicking the Unarchive button.
|
||||
|
||||
### Maintain Knowledge via API
|
||||
|
||||
Head to [maintain-dataset-via-api.md](maintain-dataset-via-api.md "mention").
|
||||
|
||||
### Knowledge Settings
|
||||
|
||||
Click **Settings** in the left navigation of the knowledge. You can change the following settings for the knowledge:
|
||||
|
||||
* Knowledge **name** for identifying a knowledge base
|
||||
* Knowledge **description** to allow AI to better use the knowledge appropriately. If the description is empty, Dify's automatic indexing strategy will be used.
|
||||
* **Permissions** can be set to Only Me or All Team Members. Those without permissions cannot view and edit the knowledge.
|
||||
* **Indexing mode**: In High Quality mode, OpenAI's embedding interface will be called to process and provide higher accuracy when users query. In Economic mode, offline vector engines, keyword indexing, etc. will be used to reduce accuracy without consuming tokens.
|
||||
|
||||
Note: Upgrading the indexing mode from Economic to High Quality will incur additional token consumption. Downgrading from High Quality to Economic will not consume tokens.
|
||||
|
||||
### Integrate into Applications
|
||||
|
||||
Once the knowledge base is ready, it needs to be integrated into the application. When the AI application processes will automatically use the associated knowledge content as a reference context.
|
||||
|
||||
1. Go to the application - Prompt Arrangement page
|
||||
2. In the context options, select the knowledge you want to integrate
|
||||
3. Save the settings to complete the integration
|
||||
|
||||
### Q\&A
|
||||
|
||||
**Q: What should I do if the PDF upload is garbled?**
|
||||
|
||||
A: If your PDF parsing appears garbled under certain formatted contents, you could consider converting the PDF to Markdown format, which currently offers higher accuracy, or you could reduce the use of images, tables, and other formatted content in the PDF. We are researching ways to optimize the experience of using PDFs.
|
||||
|
||||
**Q: How does the consumption mechanism of context work?** A: With a knowledge base added, each query will consume segmented content (currently embedding two chunks) + question + prompt + chat history combined. However, it will not exceed model limitations, such as 4096.
|
||||
|
||||
**Q: Where does the embedded knowledge appear when asking questions?** A: It will be embedded as context before the question.
|
||||
|
||||
**Q: Is there any priority between the added knowledge and OpenAI's answers?** A: The knowledge serves as context and is used together with questions for LLM to understand and answer; there is no priority relationship.
|
||||
|
||||
**Q: Why can I hit in test but not in application?** A: You can troubleshoot issues by following these steps:
|
||||
|
||||
1. Make sure you have added text on the prompt page and clicked on the save button in the top right corner.
|
||||
2. Test whether it responds normally in the prompt debugging interface.
|
||||
3. Try again in a new WebApp session window.
|
||||
4. Optimize your data format and quality. For practice reference, visit [https://github.com/langgenius/dify/issues/90](https://github.com/langgenius/dify/issues/90) If none of these steps solve your problem, please join our community for help.
|
||||
|
||||
**Q: Will APIs related to hit testing be opened up so that dify can access knowledge bases and implement dialogue generation using custom models?** A: We plan to open up Webhooks later on; however, there are no current plans for this feature. You can achieve your requirements by connecting to any vector database.
|
||||
|
||||
**Q: How do I add multiple knowledge bases?** A: Due to short-term performance considerations, we currently only support one knowledge base. If you have multiple sets of data, you can upload them within the same knowledge base for use.
|
||||
|
|
@ -1,74 +0,0 @@
|
|||
# Sync from Notion
|
||||
|
||||
Dify knowledge supports importing from Notion and setting up **Sync** so that data is automatically synced to Dify after updates in Notion.
|
||||
|
||||
### Authorization verification
|
||||
|
||||
1. When creating a knowledge base, select the data source, click **Sync from Notion--Go to connect**, and complete the authorization verification according to the prompt.
|
||||
2. You can also: click **Settings--Data Sources--Add a Data Source**, click Notion Source **Connect** to complete authorization verification.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/notion-connect.png" alt=""><figcaption><p>Connect Notion</p></figcaption></figure>
|
||||
|
||||
### Import Notion data
|
||||
|
||||
After completing authorization verification, go to the knowledge creation page, click **Sync from Notion**, and select the required authorization page to import.
|
||||
|
||||
### Segmentation and cleaning
|
||||
|
||||
Next, select your **segmentation settings** and **indexing method**, **save and process**. Wait for Dify to process this data, usually this step requires token consumption in LLM providers. Dify not only supports importing ordinary page types but also summarizes and saves the page attributes under the database type.
|
||||
|
||||
_**Note: Images and files are not currently supported for import. Table data will be converted to text.**_
|
||||
|
||||
### Sync Notion data
|
||||
|
||||
If your Notion content has been modified, you can click Sync directly on the Dify knowledge document list page to sync the data with one click(Please note that each time you click, the current content will be synchronized). This step requires token consumption.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/sync-notion-data.png" alt=""><figcaption><p>Sync Notion data</p></figcaption></figure>
|
||||
|
||||
### (Community Edition) Notion Integration Configuration Guide
|
||||
|
||||
Notion integration is divided into two ways: **internal integration** and **public integration** . It can be configured in Dify on demand.
|
||||
|
||||
For the specific differences between the two integration methods, please refer to the [official doc of Notion](https://developers.notion.com/docs/authorization).
|
||||
|
||||
#### 1. **Use internal integration**
|
||||
|
||||
Create an integration in your [integration's settings](https://www.notion.so/my-integrations) page. By default, all integrations start with an internal integration; internal integrations will be associated with a workspace of your choice, so you need to be the workspace owner to create an integration.
|
||||
|
||||
**Specific operation steps:**
|
||||
|
||||
Click the " **New integration** " button, the type is Internal by default (cannot be modified), select the associated space, enter the name and upload the logo, and click "**Submit**" to create the integration successfully.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (4) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Once the integration is created, you can update its settings as needed under the **Capabilities** tab and click the "**Show**" button under **Secrets** and then copy the Secrets.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Copy it and back to the Dify source code , in the **.env** file configuration related environment variables, environment variables as follows:
|
||||
|
||||
**NOTION\_INTEGRATION\_TYPE** = internal or **NOTION\_INTEGRATION\_TYPE** = public
|
||||
|
||||
**NOTION\_INTERNAL\_SECRET**=you-internal-secret
|
||||
|
||||
#### 2. **Use public integration**
|
||||
|
||||
**You need to upgrade the internal integration to public integration** , navigate to the integrated Distribution page, and toggle the switch to expose the integration.
|
||||
|
||||
To toggle the switch to public settings, you need to **fill in additional information in the Organization Information** form below, including your company name, website, and Retargeting URL, and click the "Submit" button.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
After your integration has been successfully made public in your [integration’s settings page](https://www.notion.so/my-integrations), you will be able to access the integration’s secrets in the Secrets tab.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (3) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Back to the Dify source code , in the **.env** file configuration related environment variables , environment variables as follows:
|
||||
|
||||
**NOTION\_INTEGRATION\_TYPE**=public
|
||||
|
||||
**NOTION\_CLIENT\_SECRET**=you-client-secret
|
||||
|
||||
**NOTION\_CLIENT\_ID**=you-client-id
|
||||
|
||||
Once configured, you will be able to utilize Notion data import and sync functions in the knowledge section.
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
# External Data Tool
|
||||
|
||||
Previously, [datasets](datasets/ "mention") allowed developers to directly upload long texts in various formats and structured data to build knowledge, enabling AI applications to converse based on the latest context uploaded by users. With this update, the external data tool empowers developers to use their own search capabilities or external data such as internal knowledge bases as the context for LLMs. This is achieved by extending APIs to fetch external data and embedding it into Prompts. Compared to uploading knowledge to the cloud, using external data tools offers significant advantages in ensuring the security of private data, customizing searches, and obtaining real-time data.
|
||||
|
||||
## What does it do?
|
||||
|
||||
When end-users make a request to the conversational system, the platform backend triggers the external data tool (i.e., calling its own API), which queries external information related to the user's question, such as employee profiles, real-time records, etc. The tool then returns through the API the portions relevant to the current request. The platform backend will assemble the returned results into text as context injected into the Prompt, in order to produce replies that are more personalized and meet user needs more accurately.
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Before using the external data tool, you need to prepare an API and an API Key for authentication. Head to [external\_data\_tool.md](extension/api\_based\_extension/external\_data\_tool.md "mention").
|
||||
2. Dify offers centralized API management; After adding API extension configurations in the settings interface, they can be directly utilized across various applications on Dify.
|
||||
|
||||
<figure><img src="../.gitbook/assets/api_based.png" alt=""><figcaption><p>API-based Extension<br></p></figcaption></figure>
|
||||
|
||||
3. Taking "Query Weather" as an example, enter the name, API endpoint, and API Key in the "Add New API-based Extension" dialog box. After saving, we can then call the API.
|
||||
|
||||
<figure><img src="../.gitbook/assets/api_based_extension.png" alt=""><figcaption><p>Weather Inquiry</p></figcaption></figure>
|
||||
|
||||
4. On the prompt orchestration page, click the "+ Add" button to the right of "Tools," and in the "Add Tool" dialog that opens, fill in the name and variable name (the variable name will be referenced in the Prompt, so please use English), as well as select the API-based extension added in Step 2.
|
||||
|
||||
<figure><img src="../.gitbook/assets/api_based_extension1.png" alt=""><figcaption><p>External_data_tool</p></figcaption></figure>
|
||||
|
||||
5. In the prompt orchestration box, we can assemble the queried external data into the Prompt. For instance, if we want to query today's weather in London, we can add a variable named `location`, enter "London", and combine it with the external data tool's extension variable name `weather_data`. The debug output would be as follows:
|
||||
|
||||
<figure><img src="../.gitbook/assets/Weather_search_tool.jpeg" alt=""><figcaption><p>Weather_search_tool</p></figcaption></figure>
|
||||
|
||||
In the Prompt Log, we can also see the real-time data returned by the API:
|
||||
|
||||
<figure><img src="../.gitbook/assets/log.jpeg" alt="" width="335"><figcaption><p>Prompt Log</p></figcaption></figure>
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
# More Integration
|
||||
|
||||
TODO
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
# RAG (Retrieval Augmented Generation)
|
||||
|
||||
## The concept of the RAG
|
||||
|
||||
The RAG architecture, with vector retrieval at its core, has become the leading technological framework for addressing two major challenges of large models: acquiring the latest external knowledge and mitigating issues of generating hallucinations. This architecture has been widely implemented in numerous practical application scenarios.
|
||||
|
||||
Developers can utilize this technology to cost-effectively build AI-powered customer service bots, corporate knowledge bases, AI search engines, etc. These systems interact with various forms of organized knowledge through natural language input. A representative example of a RAG application is as follows:
|
||||
|
||||
In the diagram below, when a user asks, "Who is the President of the United States?", the system doesn't directly relay the question to the large model for an answer. Instead, it first conducts a vector search in a knowledge base (like Wikipedia, as shown in the diagram) for the user's query. It finds relevant content through semantic similarity matching (for instance, "Biden is the current 46th President of the United States…"), and then provides the user's question along with the found knowledge to the large model. This enables the model to have sufficient and complete knowledge to answer the question, thereby yielding a more reliable response.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Basic Architecture of RAG</p></figcaption></figure>
|
||||
|
||||
## Why is this necessary?
|
||||
|
||||
We can liken a large model to a super-expert, knowledgeable in various human domains. However, this expert has its limitations; for example, it doesn't know your personal situation, as such information is private and not publicly available on the internet, and therefore, it hasn't had the opportunity to learn it beforehand.
|
||||
|
||||
When you want to hire this super-expert as your family financial advisor, you need to allow them to review your investment records, household expenses, and other relevant data before they can respond to your inquiries. This enables them to provide professional advice tailored to your personal circumstances.
|
||||
|
||||
**This is what the RAG system does: it helps the large model temporarily acquire external knowledge it doesn't possess, allowing it to search for answers before responding to a question.**
|
||||
|
||||
Based on this example, it's evident that the most critical aspect of the RAG system is the retrieval of external knowledge. The expert's ability to provide professional financial advice depends on accurately finding the necessary information. If the expert retrieves information unrelated to financial investments, like a family weight loss plan, even the most capable expert would be ineffective.
|
||||
|
|
@ -1,80 +0,0 @@
|
|||
# Hybrid Search
|
||||
|
||||
## Why is Hybrid Search Necessary?
|
||||
|
||||
The mainstream method in the RAG retrieval phase is Vector Search, which is based on semantic relevance matching. The technical principle involves initially dividing documents from external knowledge bases into semantically complete paragraphs or sentences, and then converting them through a process known as embedding into a series of numerical expressions (multidimensional vectors) that computers can understand. The user's query undergoes a similar conversion.
|
||||
|
||||
Computers can detect subtle semantic correlations between user queries and sentences. For example, the semantic relevance between "a cat chases a mouse" and "a kitten hunting a mouse" is higher than between "a cat chases a mouse" and "I like to eat ham." After identifying the text with the highest relevance, the RAG system provides it as context alongside the user's query to the large model, aiding in answering the question.
|
||||
|
||||
Besides complex semantic text searches, Vector Search has other advantages:
|
||||
|
||||
* Understanding of similar semantics (e.g., mouse/mousetrap/cheese, Google/Bing/search engine)
|
||||
* Multilingual comprehension (e.g., matching Chinese input with English content)
|
||||
* Multimodal understanding (supports matching text, images, audio, and video)
|
||||
* Fault tolerance (handles spelling mistakes, vague descriptions)
|
||||
|
||||
However, Vector Search might underperform in certain scenarios, like:
|
||||
|
||||
* Searching names of people or objects (e.g., Elon Musk, iPhone 15)
|
||||
* Searching acronyms or short phrases (e.g., RAG, RLHF)
|
||||
* Searching IDs (e.g., `gpt-3.5-turbo`, `titan-xlarge-v1.01`)
|
||||
|
||||
These limitations are precisely where traditional keyword search excels, being adept at:
|
||||
|
||||
* Precise matching (e.g., product names, personal names, product numbers)
|
||||
* Matching a small number of characters (vector search performs poorly with few characters, but users often input just a few keywords)
|
||||
* Matching low-frequency vocabulary (which often carries more significant meanings, like in “Do you want to go for a coffee with me?”, words like “drink” and “coffee” carry more weight than “you”, “want”, “me”)
|
||||
|
||||
In most text search scenarios, it's crucial to ensure that the most relevant results appear in the candidates. Vector and keyword searches each have their strengths in the search domain. Hybrid Search combines the advantages of both techniques while compensating for their respective shortcomings.
|
||||
|
||||
In Hybrid Search, vector and keyword indices are pre-established in the database. Upon user query input, the system searches for the most relevant text in documents using both search methods.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1).png" alt=""><figcaption><p>Hybrid Search</p></figcaption></figure>
|
||||
|
||||
"Hybrid Search" doesn't have a definitive definition; this article exemplifies it as a combination of Vector Search and Keyword Search. However, the term can also apply to other combinations of search algorithms. For instance, we could combine knowledge graph technology, used for retrieving entity relationships, with Vector Search.
|
||||
|
||||
Different search systems each excel at uncovering various subtle connections within texts (paragraphs, sentences, words), including precise relationships, semantic relationships, thematic relationships, structural relationships, entity relationships, temporal relationships, and event relationships. It's safe to say that no single search mode is suitable for all scenarios. **Hybrid Search, by integrating multiple search systems, achieves a complementarity among various search technologies.**
|
||||
|
||||
## Vector Search
|
||||
|
||||
Definition: Vector Search involves generating query embeddings and then searching for text chunks that most closely match these embeddings in terms of vector representation.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-174228.png" alt=""><figcaption><p>Settings for Vector Search</p></figcaption></figure>
|
||||
|
||||
**TopK:** This setting is used to filter text chunks that have the highest similarity to the user's query. The system also dynamically adjusts the number of chunks based on the context window size of the selected model. The default value for this setting is 3.
|
||||
|
||||
**Score Threshold:** This setting is used to establish a similarity threshold for the selection of text chunks. It means that only text chunks exceeding the set score are recalled. By default, this setting is turned off, meaning that the system does not filter the similarity values of the recalled text chunks. When activated, the default value is set to 0.5.
|
||||
|
||||
**Rerank Model:** After configuring the Rerank model's API key on the "Model Provider" page, you can enable the "Rerank Model" in the search settings. The system then performs a semantic re-ranking of the document results that have been recalled after semantic search, optimizing the order of these results. Once the Rerank model is set up, the TopK and Score threshold settings are only effective in the Rerank step.
|
||||
|
||||
## Full-Text Search
|
||||
|
||||
Definition: Full-Text Search involves indexing all the words in a document, enabling users to query any term and retrieve text chunks that contain these terms.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-174610.png" alt=""><figcaption><p>Settings for Full-Text Search</p></figcaption></figure>
|
||||
|
||||
**TopK:** This setting is utilized to select text chunks that most closely match the user's query in terms of similarity. The system also dynamically adjusts the number of chunks based on the context window size of the chosen model. The default value for TopK is set at 3.
|
||||
|
||||
**Rerank Model:** After configuring the API key for the Rerank model on the "Model Provider" page, you can activate the "Rerank Model" in the search settings. The system will then perform a semantic re-ranking of the document results retrieved through full-text search, optimizing the order of these results. Once the Rerank model is configured, the TopK and any Score threshold settings will only be effective during the Rerank step.
|
||||
|
||||
## Hybrid Search
|
||||
|
||||
Hybrid Search operates by concurrently executing Full-Text Search and Vector Search. It then applies a re-ranking step to choose the best results that match the user's query from both types of search results. To effectively use this feature, it is necessary to configure the Rerank Model API.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-175216.png" alt=""><figcaption><p>Settings for Hybrid Search</p></figcaption></figure>
|
||||
|
||||
**TopK:** This setting is used for filtering text chunks that have the highest similarity to the user's query. The system will dynamically adjust the number of chunks based on the context window size of the model in use. The default value for TopK is set at 3.
|
||||
|
||||
**Rerank Model:** After configuring the Rerank model's API key on the "Model Supplier" page, you can enable the "Rerank Model" in the search settings. The system will perform a semantic re-ranking of the document results retrieved through hybrid search, thereby optimizing the order of these results. Once the Rerank model is set up, the TopK and any Score threshold settings are only applicable during the Rerank step.
|
||||
|
||||
## Setting the Search Mode When Creating a Knowledge
|
||||
|
||||
To set the search mode when creating a knowledge base, navigate to the "Knowledge -> Create Knowledge" page. There, you can configure different search modes in the retrieval settings section.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-175958.png" alt=""><figcaption><p>Setting the Search Mode When Creating a Knowledge base</p></figcaption></figure>
|
||||
|
||||
## Modifying the Search Mode in Prompt Engineering
|
||||
|
||||
You can modify the search mode during application creation by navigating to the "Prompt Engineering -> Context -> Select Knowledge -> Settings" page. This allows for adjustments to different search modes within the prompt arrangement phase.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-182704.png" alt=""><figcaption><p>Modifying the Search Mode in Prompt Engineering</p></figcaption></figure>
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
# Rerank
|
||||
|
||||
## Why is Rerank Necessary? 
|
||||
|
||||
Hybrid Search combines the advantages of various search technologies to achieve better recall results. However, results from different search modes need to be merged and normalized (converting data into a uniform standard range or distribution for better comparison, analysis, and processing) before being collectively provided to the large model. This necessitates the introduction of a scoring system: Rerank Model.
|
||||
|
||||
The Rerank Model works by reordering the list of candidate documents based on their semantic match with the user's question, thus improving the results of semantic sorting. It does this by calculating a relevance score between the user's question and each candidate document, returning a list of documents sorted by relevance from high to low. Common Rerank models include Cohere rerank, bge-reranker, and others.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/spaces_CdDIVDY6AtAz028MFT4d_uploads_ohgmBurknjsKmg53Z00U_image.webp" alt=""><figcaption><p>Hybrid Search + Rerank</p></figcaption></figure>
|
||||
|
||||
In most cases, there is an initial search before rerank because calculating the relevance score between a query and millions of documents is highly inefficient. **Therefore, rerank is typically placed at the end of the search process, making it very suitable for merging and sorting results from different search systems.**
|
||||
|
||||
However, rerank is not only applicable to merging results from different search systems. Even in a single search mode, introducing a rerank step can effectively improve the recall of documents, such as adding semantic rerank after keyword search.
|
||||
|
||||
In practice, apart from normalizing results from multiple queries, we usually limit the number of text chunks passed to the large model before providing the relevant text chunks (i.e., TopK, which can be set in the rerank model parameters). This is done because the input window of the large model has size limitations (generally 4K, 8K, 16K, 128K Token counts), and you need to select an appropriate segmentation strategy and TopK value based on the size limitation of the chosen model's input window.
|
||||
|
||||
It should be noted that even if the model's context window is sufficiently large, too many recalled chunks may introduce content with lower relevance, thus degrading the quality of the answer. Therefore, the TopK parameter for rerank is not necessarily better when larger.
|
||||
|
||||
Rerank is not a substitute for search technology but an auxiliary tool to enhance existing search systems. **Its greatest advantage is that it not only offers a simple and low-complexity method to improve search results but also allows users to integrate semantic relevance into existing search systems without the need for significant infrastructure modifications.**
|
||||
|
||||
## How to Obtain the Cohere Rerank Model? 
|
||||
|
||||
Visit [https://cohere.com/rerank](https://cohere.com/rerank), register on the page, and apply for usage rights for the Rerank model to obtain the API key.
|
||||
|
||||
## Setting the Rerank Model in Knowledge Search Mode 
|
||||
|
||||
Access the Rerank settings by navigating to “Knowledge -> Create Knowledge -> Retrieval Settings”. Besides setting Rerank during knowledge creation, you can also modify the Rerank configuration in the settings of an already created knowledge base, and change the Rerank configuration in the knowledge recall mode settings of application arrangement.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-191016.png" alt=""><figcaption><p>Setting the Rerank Model in Knowledge Search Mode </p></figcaption></figure>
|
||||
|
||||
**TopK:** Used to set the number of relevant documents returned after Rerank. 
|
||||
|
||||
**Score Threshold:** Used to set the minimum score for relevant documents to be returned after Rerank. After setting the Rerank model, the TopK and Score threshold settings are only effective in the Rerank step.
|
||||
|
||||
## Setting the Rerank Model in Multi-path retrieval
|
||||
|
||||
 Recall Mode Enable the Rerank model by setting it to Multi-path retrieval mode in the “Prompt Engineering -> Context -> Settings” page. 
|
||||
|
||||
Explanation of Multi-path retrieval Mode: 🔗
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-191531.png" alt=""><figcaption><p>Setting the Rerank Model in Multi-path retrieval</p></figcaption></figure>
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
# Retrieval
|
||||
|
||||
When users build knowledge base Q\&A AI applications, if multiple knowledge bases are associated within the application, Dify supports two retrieval modes: N-to-1 retrieval and Multi-path retrieval.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/screenshot-20231119-191531.png" alt=""><figcaption><p>Retrieval Settings</p></figcaption></figure>
|
||||
|
||||
## Retrieval **Settings**
|
||||
|
||||
### **N-to-1 Retrieval** 
|
||||
|
||||
Based on user intent and knowledge description, the Agent independently determines and selects the most matching single knowledge base for querying relevant text. This mode is suitable for applications with distinct knowledge and a smaller number of knowledge bases. N-to-1 retrieval relies on the model's inference capability to choose the most relevant knowledge base based on user intent. When inferring the knowledge, the knowledge serves as a tool for the Agent, chosen through intent inference; the tool description is essentially the knowledge description.
|
||||
|
||||
When users upload knowledge, the system automatically creates a summary description of each knowledge base. To achieve the best retrieval results in this mode, you can view the system-generated summary description under “Knowledge -> Settings -> Knowledge Description” and check if this content clearly summarizes the knowledge's content.
|
||||
|
||||
Here is the technical flowchart for N-to-1 retrieval:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/spaces_CdDIVDY6AtAz028MFT4d_uploads_LgAOVtxy9kQ0B8e2qaQl_image.webp" alt=""><figcaption><p>N-to-1 Retrieval </p></figcaption></figure>
|
||||
|
||||
Therefore, this mode's recall effectiveness can be impacted when there are too many knowledge bases or when the knowledge descriptions lack sufficient distinction. This mode is more suitable for applications with fewer knowledge bases. 
|
||||
|
||||
Tip: OpenAI Function Call already supports multiple tool calls, and Dify plans to upgrade this mode to "N-to-M retrieval" in future versions.
|
||||
|
||||
### Multi-path Retrieval
|
||||
|
||||
Based on user intent, this mode matches all knowledge bases simultaneously, queries relevant text chunks from multiple knowledge bases, and after a re-ranking step, selects the best results matching the user's question from the multi-path query results. Configuring the Rerank model API is required. In Multi-path retrieval mode, the search engine retrieves text content related to the user's query from all knowledge bases associated with the application, merges the results from multi-path recall, and re-ranks the retrieved documents semantically using the Rerank model.
|
||||
|
||||
In Multi-path retrieval mode, configuring the Rerank model is necessary. How to configure the Rerank model: 🔗
|
||||
|
||||
Here is the technical flowchart for Multi-path retrieval: 
|
||||
|
||||
<figure><img src="../../.gitbook/assets/spaces_CdDIVDY6AtAz028MFT4d_uploads_xfMNnsyD506TOoynHdgU_image.webp" alt=""><figcaption><p>Multi-path retrieval</p></figcaption></figure>
|
||||
|
||||
As Multi-path retrieval does not rely on the model's inferencing capability or knowledge descriptions, this mode can achieve higher quality recall results in multi-knowledge searches. Additionally, incorporating the Rerank step can effectively improve document recall. Therefore, when creating a knowledge base Q\&A application associated with multiple knowledge bases, we recommend configuring the retrieval mode as Multi-path retrieval.
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
# Nodes
|
||||
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
---
|
||||
description: Answer
|
||||
---
|
||||
|
||||
# Answer
|
||||
|
||||
Defining Reply Content in a Chatflow Process. In a text editor, you have the flexibility to determine the reply format. This includes crafting a fixed block of text, utilizing output variables from preceding steps as the reply content, or merging custom text with variables for the response.
|
||||
|
||||
Answer node can be seamlessly integrated at any point to dynamically deliver content into the dialogue responses. This setup supports a live-editing configuration mode, allowing for both text and image content to be arranged together. The configurations include:
|
||||
|
||||
1. Outputting the reply content from a Language Model (LLM) node.
|
||||
2. Outputting generated images.
|
||||
3. Outputting plain text.
|
||||
|
||||
Example 1: Output plain text.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (8) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Example 2: Output image and LLM reply.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (6) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (7) (1).png" alt="" width="275"><figcaption></figcaption></figure>
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
# Code
|
||||
|
||||
## Introduction
|
||||
|
||||
The Code node lets you write Python / NodeJS code to perform data transformations not achievable via pre-defined nodes. It lends a lot of flexibility to your workflows and can be used to perform calculations, process JSON data, transform texts, and more.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (157).png" alt="" width="375"><figcaption></figcaption></figure>
|
||||
|
||||
## Including variables
|
||||
If you need to use variables from other nodes within your code, you need to define the variable names in `input variables` and reference these variables, see [Variable Reference](../key_concept.md#variable) for details.
|
||||
|
||||
You will need to define the type of the output variable for
|
||||
|
||||
## Use Cases
|
||||
With the code node, you can perform the following common operations:
|
||||
|
||||
### Structured Data Processing
|
||||
In workflows, it's often necessary to deal with unstructured data processing, such as parsing, extracting, and transforming JSON strings. A typical example is dealing with output from the [HTTP Request node](./http-request.md), where the data you need to extract might be nested under multiple layers within a JSON object. The code node can help you accomplish such tasks. Here's a simple example that extracts the `data.name` field from a JSON string returned by an HTTP node:
|
||||
|
||||
```python
|
||||
def main(http_response: str) -> str:
|
||||
import json
|
||||
data = json.loads(http_response)
|
||||
return data['data']['name']
|
||||
```
|
||||
|
||||
### Mathematical Calculations
|
||||
You can also write custom code to calculate a complex mathematical formula or perform some statistical analysis on certain data. Here is a simple example that calculates the variance of a list:
|
||||
|
||||
```python
|
||||
def main(x: list) -> float:
|
||||
return sum([(i - sum(x) / len(x)) ** 2 for i in x]) / len(x)
|
||||
```
|
||||
|
||||
### Data Transformations
|
||||
|
||||
```python
|
||||
def main(knowledge1: list, knowledge2: list) -> list:
|
||||
return knowledge1 + knowledge2
|
||||
```
|
||||
|
||||
## Limitations
|
||||
The execution environment is sandboxed for both Python and Javascript (If you are self-hosting Dify, a sandbox service would be started with Docker).
|
||||
|
||||
This means that certain functionalities that require extensive system resources or pose security risks are not available. This includes, but is not limited to, direct file system access, network calls, and operating system-level commands.
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
# End
|
||||
|
||||
Defining the Final Output Content of a Workflow Process. Every workflow needs at least one "End" node to output the final result after full execution. 
|
||||
|
||||
The "End" node serves as the termination point of the process, beyond which no further nodes can be added. In workflow applications, execution results are only output when the process reaches the "End" node. If the process involves conditional branching, multiple "End" nodes must be defined.
|
||||
|
||||
Single-Path Execution Example:
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (2) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Multi-Path Execution Example:
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (5) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
# HTTP Request
|
||||
|
||||
HTTP Request node lets you craft and dispatch HTTP requests to specified endpoints, enabling a wide range of integrations and data exchanges with external services. The node supports all common HTTP request methods, and lets you fully customize over the URL, headers, query parameters, body content, and authorization details of the request.
|
||||
|
||||
<figure><img src="https://langgenius.feishu.cn/space/api/box/stream/download/asynccode/?code=YjQ4Y2EyNjhlNWQ3NDA0ZGJiMWUzYTYyMWFkNWRlOThfek1Dd2c1Z3VwdU1jVGpqMkNrM2hzUUZmMXFEUldaOGpfVG9rZW46WGJwOGJuQ0pJb245TFN4aUtXUmNuUktFblVjXzE3MTI1OTc2ODg6MTcxMjYwMTI4OF9WNA" alt="" width="375"><figcaption></figcaption></figure>
|
||||
|
||||
A really handy feature with HTTP request is the ability to dynamically construct the request by inserting variables in different fields. For instance, in a customer support scenario, variables such as username or customer ID can be used to personalize automated responses sent via a POST request, or retrieve individual-specific information related to the customer.The HTTP request returns `body`, `status_code`, `headers`, and `files` as outputs. If the response includes files of [MIME](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics\_of\_HTTP/MIME\_types/Common\_types) types (currently limited to images), the node automatically saves these as `files` for downstream use.
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
# IF/ELSE
|
||||
|
||||
The IF/ELSE Node allows you to split a workflow into two branches based on if/else conditions. In this node, you can set one or more IF conditions. When the IF condition(s) are met, the workflow proceeds to the next step under the "IS TRUE" branch. If the IF condition(s) are not met, the workflow triggers the next step under the "IS FALSE" branch.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (58).png" alt=""><figcaption></figcaption></figure>
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
# Knowledge Retrieval
|
||||
|
||||
\
|
||||
The Knowledge Base Retrieval Node is designed to query text content related to user questions from the Dify Knowledge Base, which can then be used as context for subsequent answers by the Large Language Model (LLM).
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (44).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Configuring the Knowledge Base Retrieval Node involves three main steps:
|
||||
|
||||
1. **Selecting the Query Variable**
|
||||
2. **Choosing the Knowledge Base for Query**
|
||||
3. **Configuring the Retrieval Strategy**
|
||||
|
||||
**Selecting the Query Variable**
|
||||
|
||||
In knowledge base retrieval scenarios, the query variable typically represents the user's input question. In the "Start" node of conversational applications, the system pre-sets "sys.query" as the user input variable. This variable can be used to query the knowledge base for text segments most closely related to the user's question.
|
||||
|
||||
**Choosing the Knowledge Base for Query**
|
||||
|
||||
Within the knowledge base retrieval node, you can add an existing knowledge base from Dify. For instructions on creating a knowledge base within Dify, please refer to the knowledge base [help documentation](https://docs.dify.ai/v/zh-hans/guides/knowledge-base).
|
||||
|
||||
**Configuring the Retrieval Strategy**
|
||||
|
||||
It's possible to modify the indexing strategy and retrieval mode for an individual knowledge base within the node. For a detailed explanation of these settings, refer to the knowledge base [help documentation](https://docs.dify.ai/v/zh-hans/learn-more/extended-reading/retrieval-augment/hybrid-search).
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (49).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Dify offers two recall strategies for different knowledge base retrieval scenarios: "N-choose-1 Recall" and "Multi-way Recall". In the N-choose-1 mode, knowledge base queries are executed through function calling, requiring the selection of a system reasoning model. In the multi-way recall mode, a Rerank model needs to be configured for result re-ranking. For a detailed explanation of these two recall strategies, refer to the retrieval mode explanation in the [help documentation](https://docs.dify.ai/v/zh-hans/learn-more/extended-reading/retrieval-augment/retrieval).
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (51).png" alt=""><figcaption></figcaption></figure>
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
# LLM
|
||||
|
||||
Invoking a Large Language Model for Question Answering or Natural Language Processing. Within an LLM node, you can select an appropriate model, compose prompts, set the context referenced in the prompts, configure memory settings, and adjust the memory window size.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (9) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Configuring an LLM node primarily involves two steps:
|
||||
|
||||
1. Selecting a model
|
||||
2. Composing system prompts
|
||||
|
||||
**Model Configuration** 
|
||||
|
||||
Before selecting a model suitable for your task, you must complete the model configuration in "System Settings—Model Provider". The specific configuration method can be referenced in the [model configuration instructions](https://docs.dify.ai/v/zh-hans/guides/model-configuration). After selecting a model, you can configure its parameters.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (10) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
**Write Prompts**
|
||||
|
||||
Within an LLM node, you can customize the model input prompts. If you choose a conversational model, you can customize the content of system prompts, user messages, and assistant messages. 
|
||||
|
||||
For instance, in a knowledge base Q\&A scenario, after linking the "Result" variable from the knowledge base retrieval node in "Context", inserting the "Context" special variable in the prompts will use the text retrieved from the knowledge base as the context background information for the model input.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (12) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
In the prompt editor, you can bring up the variable insertion menu by typing "/" or "{" to insert special variable blocks or variables from preceding flow nodes into the prompts as context content.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (13) (1).png" alt="" width="375"><figcaption></figcaption></figure>
|
||||
|
||||
If you opt for a completion model, the system provides preset prompt templates for conversational applications. You can customize the content of the prompts and insert special variable blocks like "Conversation History" and "Context" at appropriate positions by typing "/" or "{", enabling richer conversational functionalities.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (14) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
**Memory Toggle Settings** In conversational applications (Chatflow), the LLM node defaults to enabling system memory settings. In multi-turn dialogues, the system stores historical dialogue messages and passes them into the model. In workflow applications (Workflow), system memory is turned off by default, and no memory setting options are provided.
|
||||
|
||||
**Memory Window Settings** If the memory window setting is off, the system dynamically passes historical dialogue messages according to the model's context window. With the memory window setting on, you can configure the number of historical dialogue messages to pass based on your needs.
|
||||
|
||||
**Dialogue Role Name Settings** Due to differences in model training phases, different models adhere to role name commands to varying degrees, such as Human/Assistant, Human/AI, 人类/助手, etc. To adapt to the prompt response effects of multiple models, the system allows setting dialogue role names, modifying the role prefix in conversation history.
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
# Question Classifier
|
||||
|
||||
Question Classifier node defines the categorization conditions for user queries, enabling the LLM to dictate the progression of the dialogue based on these categorizations. As illustrated in a typical customer service robot scenario, the question classifier can serve as a preliminary step to knowledge base retrieval, identifying user intent. Classifying user intent before retrieval can significantly enhance the recall efficiency of the knowledge base.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (54).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Configuring the Question Classifier Node involves three main components:
|
||||
|
||||
1. **Selecting the Input Variable**
|
||||
2. **Configuring the Inference Model**
|
||||
3. **Writing the Classification Method**
|
||||
|
||||
**Selecting the Input Variable** In conversational customer scenarios, you can use the user input variable from the "Start Node" (sys.query) as the input for the question classifier. In automated/batch processing scenarios, customer feedback or email content can be utilized as input variables.
|
||||
|
||||
**Configuring the Inference Model** The question classifier relies on the natural language processing capabilities of the LLM to categorize text. You will need to configure an inference model for the classifier. Before configuring this model, you might need to complete the model setup in "System Settings - Model Provider". The specific configuration method can be found in the [model configuration instructions](https://docs.dify.ai/v/zh-hans/guides/model-configuration). After selecting a suitable model, you can configure its parameters.
|
||||
|
||||
**Writing Classification Conditions** You can manually add multiple classifications by composing keywords or descriptive sentences that fit each classification. Based on the descriptions of these conditions, the question classifier can route the dialogue to the appropriate process path according to the semantics of the user's input.
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
# Start
|
||||
|
||||
Defining initial parameters for a workflow process initiation allows for customization at the start node, where you input variables to kick-start the workflow. Every workflow necessitates a start node, acting as the entry point and foundation for the workflow's execution path.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (20).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Within the "Start" node, you can define input variables of four types:
|
||||
|
||||
* **Text**: For short, simple text inputs like names, identifiers, or any other concise data.
|
||||
* **Paragraph**: Supports longer text entries, suitable for descriptions, detailed queries, or any extensive textual data.
|
||||
* **Dropdown Options**: Allows the selection from a predefined list of options, enabling users to choose from a set of predetermined values.
|
||||
* **Number**: For numeric inputs, whether integers or decimals, to be used in calculations, quantities, identifiers, etc.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (24).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Once the configuration is completed, the workflow's execution will prompt for the values of the variables defined in the start node. This step ensures that the workflow has all the necessary information to proceed with its designated processes.
|
||||
|
||||
<figure><img src="../../../.gitbook/assets/image (37).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
**Tip:** In Chatflow, the start node provides system-built variables: `sys.query` and `sys.files`. `sys.query` is utilized for user question input in conversational applications, enabling the system to process and respond to user queries. `sys.files` is used for file uploads within the conversation, such as uploading an image to understand its content. This requires the integration of image understanding models or tools designed for processing image inputs, allowing the workflow to interpret and act upon the uploaded files effectively.
|
||||
|
|
@ -1,29 +0,0 @@
|
|||
# Template
|
||||
|
||||
Template lets you dynamically format and combine variables from previous nodes into a single text-based output using Jinja2, a powerful templating syntax for Python. It's useful for combining data from multiple sources into a specific structure required by subsequent nodes. The simple example below shows how to assemble an article by piecing together various previous outputs:
|
||||
|
||||
<figure><img src="https://langgenius.feishu.cn/space/api/box/stream/download/asynccode/?code=MzE1NTFmODViNTFmMDQzNTg5YTZhMjFiODdlYjI4ZTFfRHRJVEJpNlNWVWVxYWs0c1I3c09OTzFCUUJoWURndkpfVG9rZW46R210aGJUa3R0b3ByTVV4QVlwc2NXNTFRbnZjXzE3MTI1OTczOTM6MTcxMjYwMDk5M19WNA" alt="" width="375"><figcaption></figcaption></figure>
|
||||
|
||||
Beyond naive use cases, you can create more complex templates as per Jinja's [documentation](https://jinja.palletsprojects.com/en/3.1.x/templates/) for a variety of tasks. Here's one template that structures retrieved chunks and their relevant metadata from a knowledge retrieval node into a formatted markdown:
|
||||
|
||||
```Plain
|
||||
{% raw %}
|
||||
{% for item in chunks %}
|
||||
### Chunk {{ loop.index }}.
|
||||
### Similarity: {{ item.metadata.score | default('N/A') }}
|
||||
|
||||
#### {{ item.title }}
|
||||
|
||||
##### Content
|
||||
{{ item.content | replace('\n', '\n\n') }}
|
||||
|
||||
---
|
||||
{% endfor %}
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
<figure><img src="https://langgenius.feishu.cn/space/api/box/stream/download/asynccode/?code=ZjFhMzg4MzVkNTU2MmY1ZDg0NTVjY2RiMWM5MDU4YmVfRUwybmxWQlZHc0pxNE5CVGx0b0JaYmVDdzlFeEJUMDBfVG9rZW46SktUN2JKaFhLb3hEemV4NVZQMmN1TGhsbmlmXzE3MTI1OTczOTM6MTcxMjYwMDk5M19WNA" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
This template node can then be used within a Chatflow to return intermediate outputs to the end user, before a LLM response is initiated.
|
||||
|
||||
> The `Answer` node in a Chatflow is non-terminal. It can be inserted anywhere to output responses at multiple points within the flow.
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
# Variable Assigner
|
||||
|
||||
The Variable Assigner node serves as a hub for collecting branch outputs within the workflow, ensuring that regardless of which branch is taken, the output can be referenced by a single variable. The output can subsequently be manipulated by nodes downstream.
|
||||
|
||||
In the example below, a branch is created to produce different LLM responses based on different user questions: A knowledge base is used in one branch, but not the other. The output variables from both branches are squashed by a Variable Assigner node into a single output based on the branch taken. This has a few benefits:
|
||||
|
||||
1. Avoids having to create duplicate downstream logic for each branch
|
||||
2. When modifying the name of an output variable, you don't need to locate everywhere it is referenced downstream and update the variable name.
|
||||
|
||||
<figure><img src="https://langgenius.feishu.cn/space/api/box/stream/download/asynccode/?code=YWFlNDlhZWRmN2U5NmI1MzIyNzcwNDIxOGRjNmFmMjVfV29TVmt0SHdMdjlXRGR2TFdGdFdraGV2SFIxWmt3OFpfVG9rZW46WVJKZ2I5a1Ztb0RMRnp4ZnpQZGN5Z1Y5bnliXzE3MTI1OTc2NDg6MTcxMjYwMTI0OF9WNA" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Variable Assigner supports multiple types of output variables including `String`,`Number`, `Object`, and `Array`. Given the specified output type, you may add input variables from the dropdown list of variables to the node. The list of variables is derived from previous branch outputs and autofiltered based on the specified type.
|
||||
|
||||
<figure><img src="https://langgenius.feishu.cn/space/api/box/stream/download/asynccode/?code=MmQ5OGY3OTBiZjdkYjUzM2M4ODBmMWFmYTc0YzUwMzJfY0JJWkJBcWd1WlVXQTJ4RW56Uk11WE1pOEtZeXN6dzJfVG9rZW46TjA5TGJqWmF4b3VIb3R4eUI4ZWNkUWVkbnNmXzE3MTI1OTc0Mjg6MTcxMjYwMTAyOF9WNA" alt="" width="375"><figcaption></figcaption></figure>
|
||||
|
||||
Variable Assigner gives a single `output` variable of the specified type for downstream use.
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
# Preview\&Run
|
||||
|
||||
|
|
@ -0,0 +1 @@
|
|||
## 🚧
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
### Literal Translation
|
||||
# Knowledge Base
|
||||
|
||||
The training data for large language models is generally based on publicly available data, and each training session requires a significant amount of computational power. This means that the knowledge of the models generally does not include private domain knowledge, and there is a certain delay in the public knowledge domain. To solve this problem, the current common solution is to use RAG (Retrieval-Augmented Generation) technology, which uses users' questions to match the most relevant external data, and after retrieving the relevant content, reorganize and insert the response back as the context of the model prompt.
|
||||
|
||||
{% hint style="info" %}
|
||||
To learn more, please check the extended reading on [Retrieval-Augmented Generation (RAG)](../../learn-more/extended-reading/retrieval-augment/)
|
||||
{% endhint %}
|
||||
<!-- TODO: check link -->
|
||||
|
||||
Dify's knowledge base feature visualizes each step in the RAG pipeline, providing a simple and easy-to-use user interface to help application builders in managing personal or team knowledge bases, and quickly integrating them into AI applications. You only need to prepare text content, such as:
|
||||
|
||||
* Long text content (TXT, Markdown, DOCX, HTML, JSONL, or even PDF files)
|
||||
* Structured data (CSV, Excel, etc.)
|
||||
|
||||
Additionally, we are gradually supporting synchronizing data from various data sources to datasets, including:
|
||||
|
||||
* Web pages
|
||||
* Notion
|
||||
* Github
|
||||
* Databases
|
||||
* ……
|
||||
|
||||
{% hint style="info" %}
|
||||
**Scenario**: If your company wants to establish an AI customer service assistant based on the existing knowledge base and product documentation, you can upload the documents to the dataset in Dify and build a chatbot. In the past, this might have taken you weeks and been difficult to maintain continuously.
|
||||
{% endhint %}
|
||||
|
||||
### Knowledge Base and Documents
|
||||
|
||||
In Dify, Knowledge is a collection of documents. A knowledge base can be integrated into an application as a retrieval context. Documents can be uploaded by developers or a member of operation team, or synchronized from other data sources (usually corresponding to one unit file in the data source).
|
||||
|
|
@ -0,0 +1,151 @@
|
|||
# Creating a Knowledge Base & Uploading Documents
|
||||
|
||||
### 1 Creating a Knowledge Base
|
||||
|
||||
Click on Knowledge in the main navigation bar of Dify. On this page, you can see your existing knowledge bases. Click **Create Knowledge** to enter the setup wizard:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/create-knowledge.png" alt=""><figcaption><p>Creating Knowledge</p></figcaption></figure>
|
||||
|
||||
* If you have already prepared the files, you can start by uploading them;
|
||||
* If you have not prepared any documents yet, you can first create an empty dataset;
|
||||
|
||||
<figure><img src="../../.gitbook/assets/create-knowledge-2.png" alt=""><figcaption><p>Creating Knowledge Base</p></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
If you choose to use an external data source when creating a dataset, the type of knowledge cannot be changed. This is to prevent difficulties in managing datasets caused by having multiple data sources in a single knowledge base. If you need to use multiple data sources, it is recommended to create multiple knowledge.
|
||||
{% endhint %}
|
||||
|
||||
***
|
||||
|
||||
### 2 Uploading Documents
|
||||
|
||||
**Steps to upload documents into Knowledge:**
|
||||
|
||||
1. Select the document you need to upload from your local files;
|
||||
2. Segment and clean the document, and preview the effect;
|
||||
3. Choose and configure Index Mode and Retreival Settings;
|
||||
4. Wait for the chunks to be embedded;
|
||||
5. Upload completed, now you can use it in your applications 🎉
|
||||
|
||||
**Limitations for uploading documents:**
|
||||
|
||||
* The upload size limit for a single document is 15MB;
|
||||
* The maximum number of files for a single batch upload is 20;
|
||||
* Different [subscription plans](https://dify.ai/pricing) for the SaaS version limit **batch upload numbers, total document uploads, and vector storage**;
|
||||
|
||||
### 3 Segmenting and Cleaning
|
||||
|
||||
**Segmenting**: Large language models have a limited context window, usually requiring the entire text to be segmented and then recalling the most relevant segments to the user’s question, known as the segment TopK recall mode. Additionally, appropriate segment sizes help match the most relevant text content and reduce information noise when semantically matching user questions with text segments.
|
||||
|
||||
**Cleaning**: To ensure the quality of text recall, it is usually necessary to clean the data before passing it into the model. For example, unwanted characters or blank lines in the output may affect the quality of the response. To help users solve this problem, Dify provides various cleaning methods to help clean the output before sending it to downstream applications.
|
||||
|
||||
Segmentation and cleaning support two configuration strategies:
|
||||
|
||||
* Automatic mode (to be phased out)
|
||||
* Custom mode
|
||||
|
||||
<figure><img src="../../.gitbook/assets/custom-chunk-settings.png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
In custom mode, users can configure chunk settings and cleaning settings according to different document formats and scenario requirements.
|
||||
|
||||
**Segmentation rules:**
|
||||
|
||||
* Segmentation identifier, set an identifier such as "\n", and the system will segment the text when the identifier appears in the text;
|
||||
* Maximum segment length, segment based on the maximum character limit of the text, forcibly segmenting when exceeding this length;
|
||||
* Segment overlap length, set the number of overlapping characters between segments, it is recommended to set it to 10-25% of the segment length, which helps retain semantic relevance between segments and improves recall results during multi-segment recall.
|
||||
|
||||
**Preprocessing rules:**
|
||||
|
||||
* Replace continuous spaces, newlines, and tabs;
|
||||
* Delete all URLs and email addresses;
|
||||
|
||||
***
|
||||
|
||||
### 4 Optional ETL Configuration
|
||||
|
||||
In production-level applications of RAG, to achieve better data recall, multi-source data needs to be preprocessed and cleaned, i.e., ETL (extract, transform, load). To enhance the preprocessing capabilities of unstructured/semi-structured data, Dify supports optional ETL solutions: **Dify ETL** and [**Unstructured ETL**](https://unstructured.io/).
|
||||
|
||||
> Unstructured can efficiently extract and transform your data into clean data for subsequent steps.
|
||||
|
||||
ETL solution choices in different versions of Dify:
|
||||
|
||||
* The SaaS version defaults to using Unstructured ETL and cannot be changed;
|
||||
* The community version defaults to using Dify ETL but can enable Unstructured ETL through [environment variables](../../getting-started/install-self-hosted/environments.md#zhi-shi-ku-pei-zhi);
|
||||
<!-- TODO: change link -->
|
||||
|
||||
Differences in supported file formats for parsing:
|
||||
|
||||
| DIFY ETL | Unstructured ETL |
|
||||
| ---------------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| txt, markdown, md, pdf, html, htm, xlsx, xls, docx, csv | txt, markdown, md, pdf, html, htm, xlsx, xls, docx, csv, eml, msg, pptx, ppt, xml, epub |
|
||||
|
||||
{% hint style="info" %}
|
||||
Different ETL solutions may have differences in file extraction effects. For more information on Unstructured ETL’s data processing methods, please refer to the [official documentation](https://docs.unstructured.io/open-source/core-functionality/partitioning).
|
||||
{% endhint %}
|
||||
|
||||
***
|
||||
|
||||
### 5 Indexing Methods
|
||||
|
||||
You need to choose the **indexing method** for the text to specify the data matching method. The indexing strategy is often related to the retrieval method, and you need to choose the appropriate indexing method according to the scenario.
|
||||
|
||||
**High-Quality Mode**: Calls OpenAI's embedding interface for processing, providing higher accuracy during user queries.
|
||||
|
||||
**Economy Mode**: Uses keyword indexing, reducing accuracy but not requiring token costs.
|
||||
|
||||
**Q\&A Mode (community version only)**: The Q\&A segment mode function differs from the ordinary "Q to P" (question to paragraph) matching mode mentioned above. It uses the "Q to Q" (question to question) matching mode. After the document is segmented, each segment generates a Q\&A matching pair through summarization. When a user asks a question, the system finds the most similar question and returns the corresponding segment as the answer. This method is more precise because it directly matches the user’s question, accurately obtaining the information the user truly needs.
|
||||
|
||||
When uploading documents to the knowledge base, the system segments the text so that the user's questions (input) can match the relevant text segments (Q to P), and finally output the result.
|
||||
|
||||
> Question text is natural language with complete grammatical structure, not just some keywords in a document retrieval task. Therefore, the Q to Q (question matching question) mode makes semantics and matching clearer and meets the needs of high-frequency and high-similarity question scenarios.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/Q&A-pair" alt=""><figcaption><p>Texts summarized into multiple Q&A pairs in Q&A segment mode</p></figcaption></figure>
|
||||
|
||||
<figure><img src="../../.gitbook/assets/q2p-and-q2q" alt=""><figcaption><p>Difference between Q to P and Q to Q indexing modes</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 6 Retrieval Settings
|
||||
|
||||
In high-quality indexing mode, Dify offers three retrieval options:
|
||||
|
||||
* **Vector Search**, generating query embeddings and searching for the text chunk most similar to its vector representation.
|
||||
* **Full-Text Search**, indexing all terms in the document, allowing users to search any term and retrieve relevant text chunk containing those terms.
|
||||
* **Hybrid Search**, executing full-text search and vector searches simultaneously, re-rank to select the best match for the user's query. Configuration of the Rerank model APIis necessary.
|
||||
|
||||
The specific configurations for the three retrieval methods are as follows:
|
||||
|
||||
#### **Vector Search**
|
||||
|
||||
Definition: By generating query embeddings to search the most similar text chunk to the query's vector representation.
|
||||
<!-- TODO: needs refinement -->
|
||||
|
||||
<figure><img src="../../.gitbook/assets/vector-search.png" alt="" width="563"><figcaption><p>Vector Search Settings</p></figcaption></figure>
|
||||
|
||||
TopK: Used to filter the text chunk most similar to the user’s query. The system will dynamically adjust the number of chunks based on the context window size of the selected model. The default value is 3.
|
||||
|
||||
Score threshold: Used to set the similarity threshold for filtering text fragments, i.e., only recall text fragments that exceed the set score. The system disables this setting by default, meaning no filtering is applied to the similarity values of the recalled text fragments. The default value is 0.5 when enabled.
|
||||
|
||||
Rerank model: After configuring the API key for the Rerank model in the “Model Provider” page, you can
|
||||
|
||||
enable the “Rerank model” in the retrieval settings. The system will semantically rerank the recalled document results after vector retrieval to optimize the ranking results. After setting the Rerank model, TopK and Score threshold settings only take effect in the Rerank step.
|
||||
|
||||
#### **Full-Text Search**
|
||||
|
||||
Definition: Indexing all terms in the document, allowing users to query any terms and return text fragments containing those terms.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/full-text-search.png" alt="" width="563"><figcaption><p>Full-Text Search Settings</p></figcaption></figure>
|
||||
|
||||
TopK: Used to filter the text fragments most similar to the user’s query. The system will dynamically adjust the number of fragments based on the context window size of the selected model. The default value is 3.
|
||||
|
||||
Rerank model: After configuring the API key for the Rerank model in the “Model Provider” page, you can enable the “Rerank model” in the retrieval settings. The system will semantically rerank the recalled document results after full-text retrieval to optimize the ranking results. After setting the Rerank model, TopK and Score threshold settings only take effect in the Rerank step.
|
||||
|
||||
#### **Hybrid Search**
|
||||
|
||||
Performs full-text and vector search simultaneously, with an additional reranking step to select the best match for the user’s query from the two types of query results, requiring Rerank model API configuration.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/hybrid-search.png" alt="" width="563"><figcaption><p>Hybrid Search Settings</p></figcaption></figure>
|
||||
|
||||
TopK: Used to filter the text fragments most similar to the user’s query. The system will dynamically adjust the number of fragments based on the context window size of the selected model. The default value is 3.
|
||||
|
||||
Rerank model: After configuring the API key for the Rerank model in the “Model Provider” page, you can enable the “Rerank model” in the retrieval settings. The system will semantically rerank the recalled document results after hybrid retrieval to optimize the ranking results. After setting the Rerank model, TopK and Score threshold settings only take effect in the Rerank step.
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# External Data Tool
|
||||
|
||||
## Feature Introduction
|
||||
|
||||
Previously, the [.](./ "mention") feature allowed developers to directly upload various formats of long texts and structured data to build datasets, enabling AI applications to engage in conversations based on the latest context provided by the user.
|
||||
|
||||
With this update, the **External Data Tool** empowers developers to use their own search capabilities or internal knowledge bases as external data for the LLM's context. This is achieved by extending the API to retrieve and embed external data into prompts. Compared to uploading datasets to the cloud, using the **External Data Tool** offers significant advantages in ensuring private data security, customizing searches, and accessing real-time data.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
When an end-user makes a request to the dialogue system, the platform's backend will trigger the external data tool (i.e., call its own API). It will query external information related to the user's question, such as employee profiles or real-time records, and return relevant parts via the API. The platform's backend will assemble the returned results into text and inject it into the prompt as context to provide more personalized and user-need-aligned responses.
|
||||
|
||||
## Operating Instructions
|
||||
|
||||
1. Before using the external data tool, you need to prepare an API and an API Key for authentication. Please read [external\_data\_tool.md](../extension/api\_based\_extension/external\_data\_tool.md "mention")
|
||||
2. Dify provides centralized API management. After adding the API extension configuration in the settings interface, it can be used directly in various applications on Dify.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/api_based.png" alt=""><figcaption><p>API-based Extension</p></figcaption></figure>
|
||||
|
||||
3. For example, to "query the weather," enter the name, API endpoint, and API Key in the "Add API-based Extension" dialog box. After saving, we can call the API.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/weather inquiry.png" alt=""><figcaption><p>Weather Inquiry</p></figcaption></figure>
|
||||
|
||||
4. On the prompt orchestration page, click the "+ Add" button next to "Tools." In the opened "Add Tool" dialog box, fill in the name and variable name (the variable name will be referenced in the prompt, please fill in English), and select the API-based extension added in step 2.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/api_based_extension1.png" alt=""><figcaption><p>External_data_tool</p></figcaption></figure>
|
||||
|
||||
5. In this way, we can assemble the queried external data into the prompt. For example, to query today's weather in London, you can add the `location` variable, input "London," combine it with the external data tool's extension variable name `weather_data`, and the debug output will be as follows:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/Weather_search_tool.jpeg" alt=""><figcaption><p>Weather_search_tool</p></figcaption></figure>
|
||||
|
||||
In the dialogue logs, we can also see the real-time data returned by the API:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/log.jpeg" alt="" width="335"><figcaption><p>Prompt Log</p></figcaption></figure>
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
# Integrating Knowledge Base within the Application
|
||||
|
||||
### 1. Creating a Knowledge Base Application
|
||||
|
||||
A knowledge base can be used as external knowledge to provide precise answers to user questions through a large language model. You can associate an existing knowledge base with any application type in Dify.
|
||||
|
||||
Taking a chat assistant as an example, the process is as follows:
|
||||
|
||||
1. Go to **Studio -- Create Application -- Create Chat Assistant**
|
||||
2. Enter **Context Settings**, click **Add**, and select the already created knowledge base
|
||||
3. In **Context Settings -- Parameter Settings**, configure the **Recall Strategy**
|
||||
4. Enable **Citation and Attribution** in **Add Features**
|
||||
5. In **Debug and Preview**, input user questions related to the knowledge base for debugging
|
||||
6. After debugging, **Save and Publish** as an AI knowledge base Q&A application
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (187).png" alt=""><figcaption><p>Associating a knowledge base within the application</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 2. Recall Modes
|
||||
|
||||
Enter **Context -- Parameter Settings -- Recall Settings** to choose the recall mode for the knowledge base.
|
||||
|
||||
**N-Choose-1 Recall**: Based on user intent and knowledge base description, the LLM autonomously selects the most relevant single knowledge base to query for related text.
|
||||
|
||||
**Multi-Path Recall**: Matches all knowledge bases simultaneously based on user intent, querying related text fragments from multiple knowledge bases. After a re-ranking step, the best result matching the user question is selected from the multi-path query results. Requires configuration of the Rerank model API.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (189).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
**How to Choose a Recall Mode**
|
||||
|
||||
N-Choose-1 Recall is driven by Function Call/ReAct, where each associated knowledge base acts as a tool function. The LLM autonomously selects the most relevant knowledge base to query based on the semantic match between the user question and the knowledge base description.
|
||||
|
||||
The effectiveness of the N-Choose-1 mode is influenced by three main factors:
|
||||
|
||||
* **The reasoning capability of the system model:** Some models have unstable adherence to Function Call/ReAct instructions.
|
||||
* **Clarity of the knowledge base description:** The description content affects the LLM's reasoning about the user question and the related knowledge base.
|
||||
* **Number of knowledge bases:** Too many knowledge bases can affect the LLM's reasoning accuracy and may exceed the context window length of the reasoning model.
|
||||
|
||||
**Recommended Configuration for N-Choose-1 Mode:** Choose a system reasoning model with better performance, associate as few knowledge bases as possible, and provide precise knowledge base descriptions.
|
||||
|
||||
When users upload a knowledge base, the system reasoning model automatically generates a summary description for the knowledge base. To achieve the best recall effect in this mode, you can check the system-generated summary description in "Knowledge Base -> Settings -> Knowledge Base Description" to ensure it clearly summarizes the content of the knowledge base.
|
||||
|
||||
Below is the technical flowchart for the N-Choose-1 Recall mode:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (190).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
N-Choose-1 Recall relies on the reasoning capability of the model and has many usage restrictions. The recall strategy for this mode is planned to be adjusted in Q3 2024.
|
||||
{% endhint %}
|
||||
|
||||
#### Multi-Path Recall Mode (Recommended)
|
||||
|
||||
In Multi-Path Recall mode, the retriever searches for text content related to the user question across all knowledge bases associated with the application. The results of the multi-path recall are merged, and a subsequent re-ranking (Rerank) step reorders the retrieved documents semantically.
|
||||
|
||||
Below is the technical flowchart for the Multi-Path Recall mode:
|
||||
|
||||
<figure><img src="https://docs.dify.ai/~gitbook/image?url=https%3A%2F%2F1288284732-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FCdDIVDY6AtAz028MFT4d%252Fuploads%252Fgit-blob-9bb237ea9a2b4cc09637e951e696d5b52eb31033%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=0790e257848b5e6c45ce226109aa1c2f5d54bae1c04d1e14dec9fa6a46bdee17" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
Multi-Path Recall mode requires the configuration of a Rerank model.
|
||||
{% endhint %}
|
||||
|
||||
The Multi-Path Recall mode does not depend on the reasoning capability of the model or the knowledge base description. This mode can achieve higher quality recall effects when retrieving from multiple knowledge bases. Therefore, it is **recommended to set the recall mode to Multi-Path Recall**.
|
||||
|
||||
***
|
||||
|
||||
### 3. Re-Ranking (Rerank)
|
||||
|
||||
The re-ranking model improves the results of semantic sorting by reordering the candidate document list based on the semantic match between the user question and the documents. It calculates the relevance score between the user question and each candidate document and returns a list of documents sorted by relevance from high to low.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (128).png" alt=""><figcaption><p>Hybrid Retrieval + Re-Ranking</p></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
For more information about Rerank, please refer to the extended reading [Re-Ranking](integrate\_knowledge\_within\_application.md#zhong-pai-xu-rerank).
|
||||
{% endhint %}
|
||||
|
||||
#### How to Configure the Rerank Model?
|
||||
|
||||
Dify currently supports the Cohere Rerank model. Enter the "Model Provider -> Cohere" page and fill in the Rerank model's API key:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (112).png" alt=""><figcaption><p>Configuring the Cohere Rerank model in the model provider</p></figcaption></figure>
|
||||
|
||||
How to obtain the Cohere Rerank model?
|
||||
|
||||
Log in to [https://cohere.com/rerank](https://cohere.com/rerank), register on the page, apply for the usage qualification of the Rerank model, and obtain the API key.
|
||||
|
||||
{% hint style="info" %}
|
||||
Besides supporting the Cohere Rerank API, you can also use local inference frameworks like Ollama and Xinference to deploy local Rerank models such as bge-reranker.
|
||||
{% endhint %}
|
||||
|
||||
#### Setting the Rerank Model
|
||||
|
||||
Go to the "Dataset -> Create Dataset -> Retrieval Settings" page and add the Rerank settings. In addition to setting Rerank when creating a dataset, you can also change the Rerank configuration in the settings of an already created dataset. Change the Rerank configuration in the recall mode settings of the application orchestration dataset.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt="" width="563"><figcaption><p>Setting the Rerank model in the dataset retrieval mode</p></figcaption></figure>
|
||||
|
||||
**TopK**: Used to set the number of relevant documents returned after Rerank.
|
||||
|
||||
**Score Threshold**: Used to set the minimum score of relevant documents returned after Rerank. After setting the Rerank model, the TopK and Score Threshold settings only take effect in the Rerank step.
|
||||
|
||||
When setting the recall mode to Multi-Path Recall in the "Prompt Orchestration -> Context -> Settings" page, you need to enable the Rerank model.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Setting the Rerank model in the dataset multi-path recall mode</p></figcaption></figure>
|
||||
|
|
@ -0,0 +1,111 @@
|
|||
# Knowledge Base and Document Maintenance
|
||||
|
||||
### 1. Viewing Text Segments
|
||||
|
||||
Each document uploaded to the knowledge base is stored in the form of text segments (Chunks). You can view the specific text content of each segment in the segment list.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (3) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Viewing uploaded document segments</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 2. Checking Segment Quality
|
||||
|
||||
The quality of document segmentation significantly affects the Q&A performance of the knowledge base application. It is recommended to manually check the segment quality before associating the knowledge base with the application.
|
||||
|
||||
Although automated segmentation methods based on character length, identifiers, or NLP semantic segmentation can significantly reduce the workload of large-scale text segmentation, the quality of segmentation is related to the text structure of different document formats and the semantic context. Manual checking and correction can effectively compensate for the shortcomings of machine segmentation in semantic recognition.
|
||||
|
||||
When checking segment quality, pay attention to the following situations:
|
||||
|
||||
* **Overly short text segments**, leading to semantic loss;
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (183).png" alt="" width="373"><figcaption><p>Overly short text segments</p></figcaption></figure>
|
||||
|
||||
* **Overly long text segments**, leading to semantic noise affecting matching accuracy;
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (186).png" alt="" width="375"><figcaption><p>Overly long text segments</p></figcaption></figure>
|
||||
|
||||
* **Obvious semantic truncation**, which occurs when using maximum segment length limits, leading to forced semantic truncation and missing content during recall;
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (185).png" alt="" width="357"><figcaption><p>Obvious semantic truncation</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 3. Adding Text Segments
|
||||
|
||||
In the segment list, click "Add Segment" to add one or multiple custom segments to the document.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
When adding segments in bulk, you need to first download the CSV format segment upload template, edit all the segment content in Excel according to the template format, save the CSV file, and then upload it.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (4) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Bulk adding custom segments</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 4. Editing Text Segments
|
||||
|
||||
In the segment list, you can directly edit the content of the added segments, including the text content and keywords of the segments.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (5) (1).png" alt=""><figcaption><p>Editing document segments</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 5. Metadata Management
|
||||
|
||||
In addition to marking metadata information from different source documents, such as the title, URL, keywords, and description of web data, metadata will be used in the segment recall process of the knowledge base as structured fields for recall filtering or displaying citation sources.
|
||||
|
||||
{% hint style="info" %}
|
||||
The metadata filtering and citation source functions are not yet supported in the current version.
|
||||
{% endhint %}
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (179).png" alt=""><figcaption><p>Metadata management</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 6. Adding Documents
|
||||
|
||||
In "Knowledge Base > Document List," click "Add File" to upload new documents or [Notion pages](sync-from-notion.md) to the created knowledge base.
|
||||
|
||||
A knowledge base (Knowledge) is a collection of documents (Documents). Documents can be uploaded by developers or operators, or synchronized from other data sources (usually corresponding to a file unit in the data source).
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (181).png" alt=""><figcaption><p>Uploading new documents to the knowledge base</p></figcaption></figure>
|
||||
|
||||
***
|
||||
|
||||
### 7. Document Disable and Archive
|
||||
|
||||
**Disable**: The dataset supports disabling documents or segments that are temporarily not to be indexed. In the dataset document list, click the disable button to disable the document. You can also disable an entire document or a specific segment in the document details. Disabled documents will not be indexed. Click enable on the disabled documents to cancel the disable status.
|
||||
|
||||
**Archive**: Old document data that is no longer in use can be archived if you do not want to delete it. Archived data can only be viewed or deleted, not edited. In the dataset document list, click the archive button to archive the document. You can also archive documents in the document details. Archived documents will not be indexed. Archived documents can also be unarchived.
|
||||
|
||||
***
|
||||
|
||||
### 8. Knowledge Base Settings
|
||||
|
||||
Click **Settings** in the left navigation of the knowledge base to change the following settings:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (182).png" alt=""><figcaption><p>Knowledge base settings</p></figcaption></figure>
|
||||
|
||||
**Knowledge Base Name**: Define a name to identify a knowledge base.
|
||||
|
||||
**Knowledge Base Description**: Used to describe the information represented by the documents in the knowledge base.
|
||||
|
||||
{% hint style="info" %}
|
||||
When the recall mode of the knowledge base is N-Choose-1, the knowledge base is provided as a tool for LLM reasoning calls. The reasoning is based on the description of the knowledge base. If the description is empty, Dify's automatic indexing strategy will be used.
|
||||
{% endhint %}
|
||||
|
||||
**Visibility Permissions**: You can choose "Only Me" or "All Team Members." People without permissions will not be able to view and edit the dataset.
|
||||
|
||||
**Index Mode**: [Reference Document](create\_knowledge\_and\_upload\_documents.md#index-mode)
|
||||
|
||||
**Embedding Model**: Modify the embedding model of the knowledge base. Changing the embedding model will re-embed all documents in the knowledge base, and the original embeddings will be deleted.
|
||||
|
||||
**Retrieval Settings**: [Reference Document](create\_knowledge\_and\_upload\_documents.md#retrieval-settings)
|
||||
|
||||
***
|
||||
|
||||
### 9. Knowledge Base API Management
|
||||
|
||||
Dify Knowledge Base provides a complete set of standard APIs. Developers can use API calls to perform daily management and maintenance operations such as adding, deleting, modifying, and querying documents and segments in the knowledge base. Please refer to the [Knowledge Base API Documentation](maintain-dataset-via-api.md).
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (180).png" alt=""><figcaption><p>Knowledge base API management</p></figcaption></figure>
|
||||
|
|
@ -1,25 +1,26 @@
|
|||
# Maintain Knowledge via API
|
||||
# Maintaining Datasets via API
|
||||
|
||||
> Authentication, invocation method and application Service API remain consistent. The difference is that a knowledge API token can operate on all knowledge bases.
|
||||
> Authentication and invocation methods are consistent with the application Service API. The difference is that a single dataset API token can operate on all datasets.
|
||||
|
||||
### Benefits of Using the Knowledge API
|
||||
* Sync your data systems to Dify knowledge to create powerful workflows.
|
||||
* Provide knowledge list and document list APIs as well as detail query interfaces, to facilitate building your own data management page.
|
||||
* Support both plain text and file uploads/updates documents, as well as batch additions and modifications, to simplify your sync process.
|
||||
* Reduce manual document handling and syncing time, improving visibility of Dify's software and services.
|
||||
### Advantages of Using Dataset API
|
||||
|
||||
### How to use
|
||||
* Synchronize your data system with Dify datasets to create powerful workflows.
|
||||
* Provide dataset list, document list, and detail queries to facilitate building your own data management page.
|
||||
* Support both plain text and file uploads and updates for documents, and support batch addition and modification at the segment level to streamline your synchronization process.
|
||||
* Reduce the time spent on manual document processing and synchronization, enhancing your visibility into Dify's software and services.
|
||||
|
||||
Please go to the knowledge page, you can switch tap to the API page in the navigation on the left side. On this page, you can view the API documentation provided by Dify and manage credentials for accessing the Knowledge API.
|
||||
### How to Use
|
||||
|
||||
Navigate to the dataset page, and you can switch to the **API** page from the left navigation. On this page, you can view the dataset API documentation provided by Dify and manage the credentials for accessing the dataset API in **API Keys**.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/dataset-api-token.png" alt=""><figcaption><p>Knowledge API Document</p></figcaption></figure>
|
||||
|
||||
## **Create Empty Knowledge**
|
||||
### API Call Examples
|
||||
|
||||
**`POST /datasets`**
|
||||
#### **Create an Empty Dataset**
|
||||
|
||||
{% hint style="warning" %}
|
||||
Used only to create an empty dataset
|
||||
Only used to create an empty dataset
|
||||
{% endhint %}
|
||||
|
||||
```
|
||||
|
|
@ -27,24 +28,21 @@ curl --location --request POST 'https://api.dify.ai/v1/datasets' \
|
|||
--header 'Authorization: Bearer {api_key}' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{"name": "name"}'
|
||||
|
||||
```
|
||||
|
||||
#### **List of Knowledge**
|
||||
|
||||
#### **Dataset List**
|
||||
|
||||
```
|
||||
curl --location --request GET 'https://api.dify.ai/v1/datasets?page=1&limit=20' \
|
||||
--header 'Authorization: Bearer {api_key}'
|
||||
|
||||
```
|
||||
|
||||
#### **Create A Document From Text**
|
||||
#### **Create Document by Text**
|
||||
|
||||
```
|
||||
curl --location --request POST '<https://api.dify.ai/v1/datasets/<uuid:dataset_id>/document/create_by_text>' \\
|
||||
--header 'Authorization: Bearer {api_key}' \\
|
||||
--header 'Content-Type: application/json' \\
|
||||
curl --location --request POST 'https://api.dify.ai/v1/datasets/<uuid:dataset_id>/document/create_by_text' \
|
||||
--header 'Authorization: Bearer {api_key}' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
"name": "Dify",
|
||||
"text": "Dify means Do it for you...",
|
||||
|
|
@ -66,10 +64,9 @@ curl --location --request POST '<https://api.dify.ai/v1/datasets/<uuid:dataset_i
|
|||
"mode": "custom"
|
||||
}
|
||||
}'
|
||||
|
||||
```
|
||||
|
||||
#### **Create A Document From File**
|
||||
#### **Create Document by File**
|
||||
|
||||
```
|
||||
curl --location POST 'https://api.dify.ai/v1/datasets/{dataset_id}/document/create_by_file' \
|
||||
|
|
@ -96,17 +93,15 @@ curl --location POST 'https://api.dify.ai/v1/datasets/{dataset_id}/document/crea
|
|||
}";
|
||||
type=text/plain' \
|
||||
--form 'file=@"/path/to/file"'
|
||||
|
||||
```
|
||||
|
||||
#### **Get Document Embedding Status**
|
||||
#### **Get Document Embedding Status (Progress)**
|
||||
|
||||
```
|
||||
curl --location --request GET 'https://api.dify.ai/v1/datasets/{dataset_id}/documents/{batch}/indexing-status' \
|
||||
--header 'Authorization: Bearer {api_key}'
|
||||
```
|
||||
|
||||
|
||||
#### **Delete Document**
|
||||
|
||||
```
|
||||
|
|
@ -114,39 +109,36 @@ curl --location --request DELETE 'https://api.dify.ai/v1/datasets/{dataset_id}/d
|
|||
--header 'Authorization: Bearer {api_key}'
|
||||
```
|
||||
|
||||
#### **Get Document List**
|
||||
#### **Dataset Document List**
|
||||
|
||||
```
|
||||
curl --location --request GET 'https://api.dify.ai/v1/datasets/{dataset_id}/documents' \
|
||||
--header 'Authorization: Bearer {api_key}'
|
||||
|
||||
```
|
||||
|
||||
#### **Add New Segment**
|
||||
#### **Add Segments**
|
||||
|
||||
```
|
||||
curl 'https://api.dify.ai/v1/datasets/aac47674-31a8-4f12-aab2-9603964c4789/documents/2034e0c1-1b75-4532-849e-24e72666595b/segment' \
|
||||
--header 'Authorization: Bearer {api_key}' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw $'"segments":[
|
||||
--data-raw $'"chunks":[
|
||||
{"content":"Dify means Do it for you",
|
||||
"keywords":["Dify","Do"]
|
||||
}
|
||||
]'
|
||||
--compressed
|
||||
|
||||
```
|
||||
|
||||
### Error Messages
|
||||
|
||||
### Error Message
|
||||
|
||||
- `document_indexing`，document is in indexing status
|
||||
- `provider_not_initialize`， Embedding model is not configured
|
||||
- `not_found`，document not exist
|
||||
- `dataset_name_duplicate` ，have existing knowledge name
|
||||
- `provider_quota_exceeded`，The model quota has exceeded the limit
|
||||
- `dataset_not_initialized`，The knowledge has not been initialized
|
||||
- `unsupported_file_type`，Unsupported file type
|
||||
- support file type：txt, markdown, md, pdf, html, htm, xlsx, docx, csv
|
||||
- `too_many_files`，The number of files is too large, and only single file upload is temporarily supported
|
||||
- `file_too_large`，The file is too large, supporting files under 15M
|
||||
* `document_indexing`: Document indexing failed
|
||||
* `provider_not_initialize`: Embedding model not configured
|
||||
* `not_found`: Document not found
|
||||
* `dataset_name_duplicate`: Dataset name duplicate
|
||||
* `provider_quota_exceeded`: Model quota exceeded
|
||||
* `dataset_not_initialized`: Dataset not initialized
|
||||
* `unsupported_file_type`: Unsupported file type
|
||||
* Currently supported: txt, markdown, md, pdf, html, htm, xlsx, docx, csv
|
||||
* `too_many_files`: Too many files, currently only single file uploads are supported
|
||||
* `file_too_large`: File too large, supports files under 15MB
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# Recall Testing/Citation Attribution
|
||||
|
||||
### 1. Recall Testing
|
||||
|
||||
The Dify Knowledge Base provides a text recall testing feature to debug the recall effects under different retrieval methods and parameter configurations. You can enter common user questions in the **Source Text** input box, click **Test**, and view the recall results in the **Recalled Paragraph** section on the right. The **Recent Queries** section allows you to view the history of query records; if the knowledge base is linked to an application, queries triggered from within the application can also be viewed here.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Recall Testing</p></figcaption></figure>
|
||||
|
||||
Clicking the icon in the upper right corner of the source text input box allows you to change the retrieval method and specific parameters of the current knowledge base. **Changes will only take effect during the recall testing process.** After completing the recall test and confirming changes to the retrieval parameters of the knowledge base, you need to make changes in [Knowledge Base Settings > Retrieval Settings](retrieval\_test\_and\_citation.md#zhi-shi-ku-she-zhi).
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Recall Testing - Retrieval Settings</p></figcaption></figure>
|
||||
|
||||
**Suggested Steps for Recall Testing:**
|
||||
|
||||
1. Design and organize test cases/test question sets covering common user questions.
|
||||
2. Choose an appropriate retrieval strategy: vector search/full-text search/hybrid search. For the pros and cons of different retrieval methods, please refer to the extended reading [Retrieval-Augmented Generation (RAG)](../../learn-more/extended-reading/retrieval-augment/).
|
||||
3. Debug the number of recall segments (TopK) and the recall score threshold (Score). Choose appropriate parameter combinations based on the application scenario, including the quality of the documents themselves.
|
||||
|
||||
**How to Configure TopK Value and Recall Threshold (Score)**
|
||||
|
||||
* **TopK represents the maximum number of recall segments when sorted in descending order of similarity scores.** A smaller TopK value will recall fewer segments, which may result in incomplete recall of relevant texts; a larger TopK value will recall more segments, which may result in recalling segments with lower semantic relevance, reducing the quality of LLM responses.
|
||||
* **The recall threshold (Score) represents the minimum similarity score allowed for recall segments.** A smaller recall score will recall more segments, which may result in recalling less relevant segments; a larger recall score threshold will recall fewer segments, and if too large, may result in missing relevant segments.
|
||||
|
||||
***
|
||||
|
||||
### 2. Citation and Attribution
|
||||
|
||||
When testing the knowledge base effect within the application, you can go to **Workspace -- Add Function -- Citation Attribution** to enable the citation attribution feature.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>Enable Citation and Attribution Feature</p></figcaption></figure>
|
||||
|
||||
After enabling the feature, when the large language model responds to a question by citing content from the knowledge base, you can view specific citation paragraph information below the response content, including **original segment text, segment number, matching degree**, etc. Clicking **Jump to Knowledge Base** above the cited segment allows quick access to the segment list in the knowledge base, facilitating developers in debugging and editing.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption><p>View Citation Information in Response Content</p></figcaption></figure>
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
# Importing Data from Notion
|
||||
|
||||
Dify datasets support importing from Notion and setting up **synchronization** so that data updates in Notion are automatically synced to Dify.
|
||||
|
||||
### Authorization Verification
|
||||
|
||||
1. When creating a dataset and selecting the data source, click **Sync from Notion Content -- Bind Now** and follow the prompts to complete the authorization verification.
|
||||
2. Alternatively, you can go to **Settings -- Data Sources -- Add Data Source**, click on the Notion source **Bind**, and complete the authorization verification.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (46).png" alt=""><figcaption><p>Binding Notion</p></figcaption></figure>
|
||||
|
||||
### Importing Notion Data
|
||||
|
||||
After completing the authorization verification, go to the create dataset page, click **Sync from Notion Content**, and select the authorized pages you need to import.
|
||||
|
||||
### Segmentation and Cleaning
|
||||
|
||||
Next, choose your **segmentation settings** and **indexing method**, then **Save and Process**. Wait for Dify to process this data for you, which typically requires token consumption in the LLM provider. Dify supports importing not only standard page types but also aggregates and saves page properties under the database type.
|
||||
|
||||
_**Please note: Images and files are not currently supported for import, and tabular data will be converted to text display.**_
|
||||
|
||||
### Synchronizing Notion Data
|
||||
|
||||
If your Notion content is modified, you can directly click **Sync** in the Dify dataset **Document List Page** to perform a one-click data synchronization. This step requires token consumption.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/sync-notion.png" alt=""><figcaption><p>Sync Notion Content</p></figcaption></figure>
|
||||
|
||||
### Integration Configuration Method for Community Edition Notion
|
||||
|
||||
Notion integration can be done in two ways: **internal integration** and **public integration**. You can configure them as needed in Dify. For specific differences between the two integration methods, please refer to [Notion Official Documentation](https://developers.notion.com/docs/authorization).
|
||||
|
||||
### 1. **Using Internal Integration**
|
||||
|
||||
First, create an integration in the integration settings page [Create Integration](https://www.notion.so/my-integrations). By default, all integrations start as internal integrations; internal integrations will be associated with the workspace you choose, so you need to be the workspace owner to create an integration.
|
||||
|
||||
Specific steps:
|
||||
|
||||
Click the **New integration** button. The type is **Internal** by default (cannot be modified). Select the associated space, enter the integration name, upload a logo, and click **Submit** to create the integration successfully.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
After creating the integration, you can update its settings as needed under the Capabilities tab and click the **Show** button under Secrets to copy the secrets.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (3) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
After copying, go back to the Dify source code, and configure the relevant environment variables in the **.env** file. The environment variables are as follows:
|
||||
|
||||
**NOTION\_INTEGRATION\_TYPE**=internal or **NOTION\_INTEGRATION\_TYPE**=public
|
||||
|
||||
**NOTION\_INTERNAL\_SECRET**=your-internal-secret
|
||||
|
||||
### 2. **Using Public Integration**
|
||||
|
||||
**You need to upgrade the internal integration to a public integration.** Navigate to the Distribution page of the integration, and toggle the switch to make the integration public. When switching to the public setting, you need to fill in additional information in the Organization Information form below, including your company name, website, and redirect URL, then click the **Submit** button.
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (6) (1) (1) (1) (1).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
After successfully making the integration public on the integration settings page, you will be able to access the integration key in the Keys tab:
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (17).png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
Go back to the Dify source code, and configure the relevant environment variables in the **.env** file. The environment variables are as follows:
|
||||
|
||||
**NOTION\_INTEGRATION\_TYPE**=public
|
||||
|
||||
**NOTION\_CLIENT\_SECRET**=your-client-secret
|
||||
|
||||
**NOTION\_CLIENT\_ID**=your-client-id
|
||||
|
||||
After configuration, you can operate the Notion data import and synchronization functions in the dataset.
|
||||
|
|
@ -0,0 +1,300 @@
|
|||
# 自定义模型接入
|
||||
|
||||
### 介绍
|
||||
|
||||
供应商集成完成后，接下来为供应商下模型的接入，为了帮助理解整个接入过程，我们以`Xinference`为例，逐步完成一个完整的供应商接入。
|
||||
|
||||
需要注意的是，对于自定义模型，每一个模型的接入都需要填写一个完整的供应商凭据。
|
||||
|
||||
而不同于预定义模型，自定义供应商接入时永远会拥有如下两个参数，不需要在供应商 yaml 中定义。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/Feb 4,2.png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
在前文中，我们已经知道了供应商无需实现`validate_provider_credential`，Runtime会自行根据用户在此选择的模型类型和模型名称调用对应的模型层的`validate_credentials`来进行验证。
|
||||
|
||||
#### 编写供应商 yaml
|
||||
|
||||
我们首先要确定，接入的这个供应商支持哪些类型的模型。
|
||||
|
||||
当前支持模型类型如下：
|
||||
|
||||
* `llm` 文本生成模型
|
||||
* `text_embedding` 文本 Embedding 模型
|
||||
* `rerank` Rerank 模型
|
||||
* `speech2text` 语音转文字
|
||||
* `tts` 文字转语音
|
||||
* `moderation` 审查
|
||||
|
||||
`Xinference`支持`LLM`、`Text Embedding`和`Rerank`，那么我们开始编写`xinference.yaml`。
|
||||
|
||||
```yaml
|
||||
provider: xinference #确定供应商标识
|
||||
label: # 供应商展示名称，可设置 en_US 英文、zh_Hans 中文两种语言，zh_Hans 不设置将默认使用 en_US。
|
||||
en_US: Xorbits Inference
|
||||
icon_small: # 小图标，可以参考其他供应商的图标，存储在对应供应商实现目录下的 _assets 目录，中英文策略同 label
|
||||
en_US: icon_s_en.svg
|
||||
icon_large: # 大图标
|
||||
en_US: icon_l_en.svg
|
||||
help: # 帮助
|
||||
title:
|
||||
en_US: How to deploy Xinference
|
||||
zh_Hans: 如何部署 Xinference
|
||||
url:
|
||||
en_US: https://github.com/xorbitsai/inference
|
||||
supported_model_types: # 支持的模型类型，Xinference同时支持LLM/Text Embedding/Rerank
|
||||
- llm
|
||||
- text-embedding
|
||||
- rerank
|
||||
configurate_methods: # 因为Xinference为本地部署的供应商，并且没有预定义模型，需要用什么模型需要根据Xinference的文档自己部署，所以这里只支持自定义模型
|
||||
- customizable-model
|
||||
provider_credential_schema:
|
||||
credential_form_schemas:
|
||||
```
|
||||
|
||||
随后，我们需要思考在 Xinference 中定义一个模型需要哪些凭据
|
||||
|
||||
* 它支持三种不同的模型，因此，我们需要有`model_type`来指定这个模型的类型，它有三种类型，所以我们这么编写
|
||||
|
||||
```yaml
|
||||
provider_credential_schema:
|
||||
credential_form_schemas:
|
||||
- variable: model_type
|
||||
type: select
|
||||
label:
|
||||
en_US: Model type
|
||||
zh_Hans: 模型类型
|
||||
required: true
|
||||
options:
|
||||
- value: text-generation
|
||||
label:
|
||||
en_US: Language Model
|
||||
zh_Hans: 语言模型
|
||||
- value: embeddings
|
||||
label:
|
||||
en_US: Text Embedding
|
||||
- value: reranking
|
||||
label:
|
||||
en_US: Rerank
|
||||
```
|
||||
|
||||
* 每一个模型都有自己的名称`model_name`，因此需要在这里定义
|
||||
|
||||
```yaml
|
||||
- variable: model_name
|
||||
type: text-input
|
||||
label:
|
||||
en_US: Model name
|
||||
zh_Hans: 模型名称
|
||||
required: true
|
||||
placeholder:
|
||||
zh_Hans: 填写模型名称
|
||||
en_US: Input model name
|
||||
```
|
||||
|
||||
* 填写 Xinference 本地部署的地址
|
||||
|
||||
```yaml
|
||||
- variable: server_url
|
||||
label:
|
||||
zh_Hans: 服务器URL
|
||||
en_US: Server url
|
||||
type: text-input
|
||||
required: true
|
||||
placeholder:
|
||||
zh_Hans: 在此输入Xinference的服务器地址，如 https://example.com/xxx
|
||||
en_US: Enter the url of your Xinference, for example https://example.com/xxx
|
||||
```
|
||||
|
||||
* 每个模型都有唯一的 model\_uid，因此需要在这里定义
|
||||
|
||||
```yaml
|
||||
- variable: model_uid
|
||||
label:
|
||||
zh_Hans: 模型 UID
|
||||
en_US: Model uid
|
||||
type: text-input
|
||||
required: true
|
||||
placeholder:
|
||||
zh_Hans: 在此输入您的 Model UID
|
||||
en_US: Enter the model uid
|
||||
```
|
||||
|
||||
现在，我们就完成了供应商的基础定义。
|
||||
|
||||
#### 编写模型代码
|
||||
|
||||
然后我们以`llm`类型为例，编写`xinference.llm.llm.py`
|
||||
|
||||
在 `llm.py` 中创建一个 Xinference LLM 类，我们取名为 `XinferenceAILargeLanguageModel`（随意），继承 `__base.large_language_model.LargeLanguageModel` 基类，实现以下几个方法：
|
||||
|
||||
* LLM 调用
|
||||
|
||||
实现 LLM 调用的核心方法，可同时支持流式和同步返回。
|
||||
|
||||
```python
|
||||
def _invoke(self, model: str, credentials: dict,
|
||||
prompt_messages: list[PromptMessage], model_parameters: dict,
|
||||
tools: Optional[list[PromptMessageTool]] = None, stop: Optional[List[str]] = None,
|
||||
stream: bool = True, user: Optional[str] = None) \
|
||||
-> Union[LLMResult, Generator]:
|
||||
"""
|
||||
Invoke large language model
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:param prompt_messages: prompt messages
|
||||
:param model_parameters: model parameters
|
||||
:param tools: tools for tool calling
|
||||
:param stop: stop words
|
||||
:param stream: is stream response
|
||||
:param user: unique user id
|
||||
:return: full response or stream response chunk generator result
|
||||
"""
|
||||
```
|
||||
|
||||
在实现时，需要注意使用两个函数来返回数据，分别用于处理同步返回和流式返回，因为Python会将函数中包含 `yield` 关键字的函数识别为生成器函数，返回的数据类型固定为 `Generator`，因此同步和流式返回需要分别实现，就像下面这样（注意下面例子使用了简化参数，实际实现时需要按照上面的参数列表进行实现）：
|
||||
|
||||
```python
|
||||
def _invoke(self, stream: bool, **kwargs) \
|
||||
-> Union[LLMResult, Generator]:
|
||||
if stream:
|
||||
return self._handle_stream_response(**kwargs)
|
||||
return self._handle_sync_response(**kwargs)
|
||||
|
||||
def _handle_stream_response(self, **kwargs) -> Generator:
|
||||
for chunk in response:
|
||||
yield chunk
|
||||
def _handle_sync_response(self, **kwargs) -> LLMResult:
|
||||
return LLMResult(**response)
|
||||
```
|
||||
* 预计算输入 tokens
|
||||
|
||||
若模型未提供预计算 tokens 接口，可直接返回 0。
|
||||
|
||||
```python
|
||||
def get_num_tokens(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
|
||||
tools: Optional[list[PromptMessageTool]] = None) -> int:
|
||||
"""
|
||||
Get number of tokens for given prompt messages
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:param prompt_messages: prompt messages
|
||||
:param tools: tools for tool calling
|
||||
:return:
|
||||
"""
|
||||
```
|
||||
|
||||
有时候，也许你不需要直接返回0，所以你可以使用`self._get_num_tokens_by_gpt2(text: str)`来获取预计算的tokens，这个方法位于`AIModel`基类中，它会使用GPT2的Tokenizer进行计算，但是只能作为替代方法，并不完全准确。
|
||||
* 模型凭据校验
|
||||
|
||||
与供应商凭据校验类似，这里针对单个模型进行校验。
|
||||
|
||||
```python
|
||||
def validate_credentials(self, model: str, credentials: dict) -> None:
|
||||
"""
|
||||
Validate model credentials
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:return:
|
||||
"""
|
||||
```
|
||||
* 模型参数 Schema
|
||||
|
||||
与自定义类型不同，由于没有在 yaml 文件中定义一个模型支持哪些参数，因此，我们需要动态时间模型参数的Schema。
|
||||
|
||||
如Xinference支持`max_tokens` `temperature` `top_p` 这三个模型参数。
|
||||
|
||||
但是有的供应商根据不同的模型支持不同的参数，如供应商`OpenLLM`支持`top_k`，但是并不是这个供应商提供的所有模型都支持`top_k`，我们这里举例 A 模型支持`top_k`，B模型不支持`top_k`，那么我们需要在这里动态生成模型参数的 Schema，如下所示：
|
||||
|
||||
```python
|
||||
def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity | None:
|
||||
"""
|
||||
used to define customizable model schema
|
||||
"""
|
||||
rules = [
|
||||
ParameterRule(
|
||||
name='temperature', type=ParameterType.FLOAT,
|
||||
use_template='temperature',
|
||||
label=I18nObject(
|
||||
zh_Hans='温度', en_US='Temperature'
|
||||
)
|
||||
),
|
||||
ParameterRule(
|
||||
name='top_p', type=ParameterType.FLOAT,
|
||||
use_template='top_p',
|
||||
label=I18nObject(
|
||||
zh_Hans='Top P', en_US='Top P'
|
||||
)
|
||||
),
|
||||
ParameterRule(
|
||||
name='max_tokens', type=ParameterType.INT,
|
||||
use_template='max_tokens',
|
||||
min=1,
|
||||
default=512,
|
||||
label=I18nObject(
|
||||
zh_Hans='最大生成长度', en_US='Max Tokens'
|
||||
)
|
||||
)
|
||||
]
|
||||
|
||||
# if model is A, add top_k to rules
|
||||
if model == 'A':
|
||||
rules.append(
|
||||
ParameterRule(
|
||||
name='top_k', type=ParameterType.INT,
|
||||
use_template='top_k',
|
||||
min=1,
|
||||
default=50,
|
||||
label=I18nObject(
|
||||
zh_Hans='Top K', en_US='Top K'
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
"""
|
||||
some NOT IMPORTANT code here
|
||||
"""
|
||||
|
||||
entity = AIModelEntity(
|
||||
model=model,
|
||||
label=I18nObject(
|
||||
en_US=model
|
||||
),
|
||||
fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
|
||||
model_type=model_type,
|
||||
model_properties={
|
||||
ModelPropertyKey.MODE: ModelType.LLM,
|
||||
},
|
||||
parameter_rules=rules
|
||||
)
|
||||
|
||||
return entity
|
||||
```
|
||||
* 调用异常错误映射表
|
||||
|
||||
当模型调用异常时需要映射到 Runtime 指定的 `InvokeError` 类型，方便 Dify 针对不同错误做不同后续处理。
|
||||
|
||||
Runtime Errors:
|
||||
|
||||
* `InvokeConnectionError` 调用连接错误
|
||||
* `InvokeServerUnavailableError` 调用服务方不可用
|
||||
* `InvokeRateLimitError` 调用达到限额
|
||||
* `InvokeAuthorizationError` 调用鉴权失败
|
||||
* `InvokeBadRequestError` 调用传参有误
|
||||
|
||||
```python
|
||||
@property
|
||||
def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
|
||||
"""
|
||||
Map model invoke error to unified error
|
||||
The key is the error type thrown to the caller
|
||||
The value is the error type thrown by the model,
|
||||
which needs to be converted into a unified error type for the caller.
|
||||
|
||||
:return: Invoke error mapping
|
||||
"""
|
||||
```
|
||||
|
||||
接口方法说明见：[Interfaces](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/interfaces.md)，具体实现可参考：[llm.py](https://github.com/langgenius/dify-runtime/blob/main/lib/model\_providers/anthropic/llm/llm.py)。
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
# 负载均衡
|
||||
|
||||
模型速率限制（Rate limits）是模型厂商对用户或客户在指定时间内访问 API 服务次数所添加的限制。它有助于防止 API 的滥用或误用，有助于确保每个用户都能公平地访问 API，控制基础设施的总体负载。
|
||||
|
||||
在企业级大规模调用模型 API 时，高并发请求会导致超过请求速率限制并影响用户访问。负载均衡可以通过在多个 API 端点之间分配 API 请求，确保所有用户都能获得最快的响应和最高的模型调用吞吐量，保障业务稳定运行。
|
||||
|
||||
你可以在 **模型供应商 -- 模型列表 -- 设置模型负载均衡** 打开该功能，并在同一个模型上添加多个凭据 (API key)。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (2) (1) (1).png" alt="" width="563"><figcaption><p>模型负载均衡</p></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
模型负载均衡为付费特性，您可以通过[订阅 SaaS 付费服务](../../getting-started/cloud.md#ding-yue-ji-hua)或者购买企业版来开启该功能。
|
||||
{% endhint %}
|
||||
|
||||
默认配置中的 API Key 为初次配置模型供应商时添加的凭据，您需要点击 **增加配置** 添加同一模型的不同 API Key 来正常使用负载均衡功能。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (3) (1) (1).png" alt="" width="563"><figcaption><p>配置负载均衡</p></figcaption></figure>
|
||||
|
||||
**需要额外添加至少 1 个模型凭据**即可保存并开启负载均衡。
|
||||
|
||||
你也可以将已配置的凭据**临时停用**或者**删除**。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (7).png" alt="" width="563"><figcaption></figcaption></figure>
|
||||
|
||||
配置完成后再模型列表内会显示所有已开启负载均衡的模型。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (6).png" alt="" width="563"><figcaption><p>开启负载均衡</p></figcaption></figure>
|
||||
|
||||
{% hint style="info" %}
|
||||
默认情况下，负载均衡使用 Round-robin 策略。如果触发速率限制，将应用 1 分钟的冷却时间。
|
||||
{% endhint %}
|
||||
|
||||
你也可以从 **添加模型** 配置负载均衡，配置流程与上面一致。
|
||||
|
||||
<figure><img src="../../.gitbook/assets/image (4).png" alt="" width="563"><figcaption><p>从添加模型配置负载均衡</p></figcaption></figure>
|
||||
|
|
@ -0,0 +1,190 @@
|
|||
# 增加新供应商
|
||||
|
||||
供应商支持三种模型配置方式：
|
||||
|
||||
* `predefined-model`预定义模型
|
||||
|
||||
表示用户只需要配置统一的供应商凭据即可使用供应商下的预定义模型。
|
||||
* `customizable-model`自定义模型
|
||||
|
||||
用户需要新增每个模型的凭据配置，如 Xinference，它同时支持 LLM 和 Text Embedding，但是每个模型都有唯一的 **model\_uid**，如果想要将两者同时接入，就需要为每个模型配置一个 **model\_uid**。
|
||||
* `fetch-from-remote`从远程获取
|
||||
|
||||
与 `predefined-model`配置方式一致，只需要配置统一的供应商凭据即可，模型通过凭据信息从供应商获取。
|
||||
|
||||
如OpenAI，我们可以基于 gpt-turbo-3.5 来 Fine Tune 多个模型，而他们都位于同一个 **api\_key** 下，当配置为`fetch-from-remote`时，开发者只需要配置统一的 **api\_key** 即可让 Dify Runtime 获取到开发者所有的微调模型并接入 Dify。
|
||||
|
||||
这三种配置方式**支持共存**，即存在供应商支持`predefined-model` + `customizable-model` 或 `predefined-model` + `fetch-from-remote`等，也就是配置了供应商统一凭据可以使用预定义模型和从远程获取的模型，若新增了模型，则可以在此基础上额外使用自定义的模型。
|
||||
|
||||
### 开始
|
||||
|
||||
#### 介绍
|
||||
|
||||
**名词解释**
|
||||
|
||||
* `module`: 一个`module`即为一个 Python Package，或者通俗一点，称为一个文件夹，里面包含了一个`__init__.py`文件，以及其他的`.py`文件。
|
||||
|
||||
**步骤**
|
||||
|
||||
新增一个供应商主要分为几步，这里简单列出，帮助大家有一个大概的认识，具体的步骤会在下面详细介绍。
|
||||
|
||||
* 创建供应商 yaml 文件，根据 [Provider Schema](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/schema.md) 编写。
|
||||
* 创建供应商代码，实现一个`class`。
|
||||
* 根据模型类型，在供应商`module`下创建对应的模型类型 `module`，如`llm`或`text_embedding`。
|
||||
* 根据模型类型，在对应的模型`module`下创建同名的代码文件，如`llm.py`，并实现一个`class`。
|
||||
* 如果有预定义模型，根据模型名称创建同名的yaml文件在模型`module`下，如`claude-2.1.yaml`，根据 [AI Model Entity](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/schema.md) 编写。
|
||||
* 编写测试代码，确保功能可用。
|
||||
|
||||
#### 开始吧
|
||||
|
||||
增加一个新的供应商需要先确定供应商的英文标识，如 `anthropic`，使用该标识在 `model_providers` 创建以此为名称的 `module`。
|
||||
|
||||
在此 `module` 下，我们需要先准备供应商的 YAML 配置。
|
||||
|
||||
**准备供应商 YAML**
|
||||
|
||||
此处以 `Anthropic` 为例，预设了供应商基础信息、支持的模型类型、配置方式、凭据规则。
|
||||
|
||||
```YAML
|
||||
provider: anthropic # 供应商标识
|
||||
label: # 供应商展示名称，可设置 en_US 英文、zh_Hans 中文两种语言，zh_Hans 不设置将默认使用 en_US。
|
||||
en_US: Anthropic
|
||||
icon_small: # 供应商小图标，存储在对应供应商实现目录下的 _assets 目录，中英文策略同 label
|
||||
en_US: icon_s_en.png
|
||||
icon_large: # 供应商大图标，存储在对应供应商实现目录下的 _assets 目录，中英文策略同 label
|
||||
en_US: icon_l_en.png
|
||||
supported_model_types: # 支持的模型类型，Anthropic 仅支持 LLM
|
||||
- llm
|
||||
configurate_methods: # 支持的配置方式，Anthropic 仅支持预定义模型
|
||||
- predefined-model
|
||||
provider_credential_schema: # 供应商凭据规则，由于 Anthropic 仅支持预定义模型，则需要定义统一供应商凭据规则
|
||||
credential_form_schemas: # 凭据表单项列表
|
||||
- variable: anthropic_api_key # 凭据参数变量名
|
||||
label: # 展示名称
|
||||
en_US: API Key
|
||||
type: secret-input # 表单类型，此处 secret-input 代表加密信息输入框，编辑时只展示屏蔽后的信息。
|
||||
required: true # 是否必填
|
||||
placeholder: # PlaceHolder 信息
|
||||
zh_Hans: 在此输入您的 API Key
|
||||
en_US: Enter your API Key
|
||||
- variable: anthropic_api_url
|
||||
label:
|
||||
en_US: API URL
|
||||
type: text-input # 表单类型，此处 text-input 代表文本输入框
|
||||
required: false
|
||||
placeholder:
|
||||
zh_Hans: 在此输入您的 API URL
|
||||
en_US: Enter your API URL
|
||||
```
|
||||
|
||||
如果接入的供应商提供自定义模型，比如`OpenAI`提供微调模型，那么我们就需要添加[`model_credential_schema`](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/schema.md)，以`OpenAI`为例：
|
||||
|
||||
```yaml
|
||||
model_credential_schema:
|
||||
model: # 微调模型名称
|
||||
label:
|
||||
en_US: Model Name
|
||||
zh_Hans: 模型名称
|
||||
placeholder:
|
||||
en_US: Enter your model name
|
||||
zh_Hans: 输入模型名称
|
||||
credential_form_schemas:
|
||||
- variable: openai_api_key
|
||||
label:
|
||||
en_US: API Key
|
||||
type: secret-input
|
||||
required: true
|
||||
placeholder:
|
||||
zh_Hans: 在此输入您的 API Key
|
||||
en_US: Enter your API Key
|
||||
- variable: openai_organization
|
||||
label:
|
||||
zh_Hans: 组织 ID
|
||||
en_US: Organization
|
||||
type: text-input
|
||||
required: false
|
||||
placeholder:
|
||||
zh_Hans: 在此输入您的组织 ID
|
||||
en_US: Enter your Organization ID
|
||||
- variable: openai_api_base
|
||||
label:
|
||||
zh_Hans: API Base
|
||||
en_US: API Base
|
||||
type: text-input
|
||||
required: false
|
||||
placeholder:
|
||||
zh_Hans: 在此输入您的 API Base
|
||||
en_US: Enter your API Base
|
||||
```
|
||||
|
||||
也可以参考`model_providers`目录下其他供应商目录下的 [YAML 配置信息](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/schema.md)。
|
||||
|
||||
**实现供应商代码**
|
||||
|
||||
我们需要在`model_providers`下创建一个同名的python文件，如`anthropic.py`，并实现一个`class`，继承`__base.provider.Provider`基类，如`AnthropicProvider`。
|
||||
|
||||
**自定义模型供应商**
|
||||
|
||||
当供应商为 Xinference 等自定义模型供应商时，可跳过该步骤，仅创建一个空的`XinferenceProvider`类即可，并实现一个空的`validate_provider_credentials`方法，该方法并不会被实际使用，仅用作避免抽象类无法实例化。
|
||||
|
||||
```python
|
||||
class XinferenceProvider(Provider):
|
||||
def validate_provider_credentials(self, credentials: dict) -> None:
|
||||
pass
|
||||
```
|
||||
|
||||
**预定义模型供应商**
|
||||
|
||||
供应商需要继承 `__base.model_provider.ModelProvider` 基类，实现 `validate_provider_credentials` 供应商统一凭据校验方法即可，可参考 [AnthropicProvider](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/anthropic/anthropic.py)。
|
||||
|
||||
```python
|
||||
def validate_provider_credentials(self, credentials: dict) -> None:
|
||||
"""
|
||||
Validate provider credentials
|
||||
You can choose any validate_credentials method of model type or implement validate method by yourself,
|
||||
such as: get model list api
|
||||
|
||||
if validate failed, raise exception
|
||||
|
||||
:param credentials: provider credentials, credentials form defined in `provider_credential_schema`.
|
||||
"""
|
||||
```
|
||||
|
||||
当然也可以先预留 `validate_provider_credentials` 实现，在模型凭据校验方法实现后直接复用。
|
||||
|
||||
**增加模型**
|
||||
|
||||
[**增加预定义模型** ](https://docs.dify.ai/v/zh-hans/guides/model-configuration/predefined-model)**👈🏻**
|
||||
|
||||
对于预定义模型，我们可以通过简单定义一个 yaml，并通过实现调用代码来接入。
|
||||
|
||||
[**增加自定义模型**](https://docs.dify.ai/v/zh-hans/guides/model-configuration/customizable-model) **👈🏻**
|
||||
|
||||
对于自定义模型，我们只需要实现调用代码即可接入，但是它需要处理的参数可能会更加复杂。
|
||||
|
||||
***
|
||||
|
||||
#### 测试
|
||||
|
||||
为了保证接入供应商/模型的可用性，编写后的每个方法均需要在 `tests` 目录中编写对应的集成测试代码。
|
||||
|
||||
依旧以 `Anthropic` 为例。
|
||||
|
||||
在编写测试代码前，需要先在 `.env.example` 新增测试供应商所需要的凭据环境变量，如：`ANTHROPIC_API_KEY`。
|
||||
|
||||
在执行前需要将 `.env.example` 复制为 `.env` 再执行。
|
||||
|
||||
**编写测试代码**
|
||||
|
||||
在 `tests` 目录下创建供应商同名的 `module`: `anthropic`，继续在此模块中创建 `test_provider.py` 以及对应模型类型的 test py 文件，如下所示：
|
||||
|
||||
```shell
|
||||
.
|
||||
├── __init__.py
|
||||
├── anthropic
|
||||
│ ├── __init__.py
|
||||
│ ├── test_llm.py # LLM 测试
|
||||
│ └── test_provider.py # 供应商测试
|
||||
```
|
||||
|
||||
针对上面实现的代码的各种情况进行测试代码编写，并测试通过后提交代码。
|
||||
|
|
@ -0,0 +1,197 @@
|
|||
# 预定义模型接入
|
||||
|
||||
供应商集成完成后，接下来为供应商下模型的接入。
|
||||
|
||||
我们首先需要确定接入模型的类型，并在对应供应商的目录下创建对应模型类型的 `module`。
|
||||
|
||||
当前支持模型类型如下：
|
||||
|
||||
* `llm` 文本生成模型
|
||||
* `text_embedding` 文本 Embedding 模型
|
||||
* `rerank` Rerank 模型
|
||||
* `speech2text` 语音转文字
|
||||
* `tts` 文字转语音
|
||||
* `moderation` 审查
|
||||
|
||||
依旧以 `Anthropic` 为例，`Anthropic` 仅支持 LLM，因此在 `model_providers.anthropic` 创建一个 `llm` 为名称的 `module`。
|
||||
|
||||
对于预定义的模型，我们首先需要在 `llm` `module` 下创建以模型名为文件名称的 YAML 文件，如：`claude-2.1.yaml`。
|
||||
|
||||
#### 准备模型 YAML
|
||||
|
||||
```yaml
|
||||
model: claude-2.1 # 模型标识
|
||||
# 模型展示名称，可设置 en_US 英文、zh_Hans 中文两种语言，zh_Hans 不设置将默认使用 en_US。
|
||||
# 也可不设置 label，则使用 model 标识内容。
|
||||
label:
|
||||
en_US: claude-2.1
|
||||
model_type: llm # 模型类型，claude-2.1 为 LLM
|
||||
features: # 支持功能，agent-thought 为支持 Agent 推理，vision 为支持图片理解
|
||||
- agent-thought
|
||||
model_properties: # 模型属性
|
||||
mode: chat # LLM 模式，complete 文本补全模型，chat 对话模型
|
||||
context_size: 200000 # 支持最大上下文大小
|
||||
parameter_rules: # 模型调用参数规则，仅 LLM 需要提供
|
||||
- name: temperature # 调用参数变量名
|
||||
# 默认预置了 5 种变量内容配置模板，temperature/top_p/max_tokens/presence_penalty/frequency_penalty
|
||||
# 可在 use_template 中直接设置模板变量名，将会使用 entities.defaults.PARAMETER_RULE_TEMPLATE 中的默认配置
|
||||
# 若设置了额外的配置参数，将覆盖默认配置
|
||||
use_template: temperature
|
||||
- name: top_p
|
||||
use_template: top_p
|
||||
- name: top_k
|
||||
label: # 调用参数展示名称
|
||||
zh_Hans: 取样数量
|
||||
en_US: Top k
|
||||
type: int # 参数类型，支持 float/int/string/boolean
|
||||
help: # 帮助信息，描述参数作用
|
||||
zh_Hans: 仅从每个后续标记的前 K 个选项中采样。
|
||||
en_US: Only sample from the top K options for each subsequent token.
|
||||
required: false # 是否必填，可不设置
|
||||
- name: max_tokens_to_sample
|
||||
use_template: max_tokens
|
||||
default: 4096 # 参数默认值
|
||||
min: 1 # 参数最小值，仅 float/int 可用
|
||||
max: 4096 # 参数最大值，仅 float/int 可用
|
||||
pricing: # 价格信息
|
||||
input: '8.00' # 输入单价，即 Prompt 单价
|
||||
output: '24.00' # 输出单价，即返回内容单价
|
||||
unit: '0.000001' # 价格单位，即上述价格为每 100K 的单价
|
||||
currency: USD # 价格货币
|
||||
```
|
||||
|
||||
建议将所有模型配置都准备完毕后再开始模型代码的实现。
|
||||
|
||||
同样，也可以参考 `model_providers` 目录下其他供应商对应模型类型目录下的 YAML 配置信息，完整的 YAML 规则见：Schema[^1]。
|
||||
|
||||
#### 实现模型调用代码
|
||||
|
||||
接下来需要在 `llm` `module` 下创建一个同名的 python 文件 `llm.py` 来编写代码实现。
|
||||
|
||||
在 `llm.py` 中创建一个 Anthropic LLM 类，我们取名为 `AnthropicLargeLanguageModel`（随意），继承 `__base.large_language_model.LargeLanguageModel` 基类，实现以下几个方法：
|
||||
|
||||
* LLM 调用
|
||||
|
||||
实现 LLM 调用的核心方法，可同时支持流式和同步返回。
|
||||
|
||||
```python
|
||||
def _invoke(self, model: str, credentials: dict,
|
||||
prompt_messages: list[PromptMessage], model_parameters: dict,
|
||||
tools: Optional[list[PromptMessageTool]] = None, stop: Optional[List[str]] = None,
|
||||
stream: bool = True, user: Optional[str] = None) \
|
||||
-> Union[LLMResult, Generator]:
|
||||
"""
|
||||
Invoke large language model
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:param prompt_messages: prompt messages
|
||||
:param model_parameters: model parameters
|
||||
:param tools: tools for tool calling
|
||||
:param stop: stop words
|
||||
:param stream: is stream response
|
||||
:param user: unique user id
|
||||
:return: full response or stream response chunk generator result
|
||||
"""
|
||||
```
|
||||
|
||||
在实现时，需要注意使用两个函数来返回数据，分别用于处理同步返回和流式返回，因为Python会将函数中包含 `yield` 关键字的函数识别为生成器函数，返回的数据类型固定为 `Generator`，因此同步和流式返回需要分别实现，就像下面这样（注意下面例子使用了简化参数，实际实现时需要按照上面的参数列表进行实现）：
|
||||
|
||||
```python
|
||||
def _invoke(self, stream: bool, **kwargs) \
|
||||
-> Union[LLMResult, Generator]:
|
||||
if stream:
|
||||
return self._handle_stream_response(**kwargs)
|
||||
return self._handle_sync_response(**kwargs)
|
||||
|
||||
def _handle_stream_response(self, **kwargs) -> Generator:
|
||||
for chunk in response:
|
||||
yield chunk
|
||||
def _handle_sync_response(self, **kwargs) -> LLMResult:
|
||||
return LLMResult(**response)
|
||||
```
|
||||
* 预计算输入 tokens
|
||||
|
||||
若模型未提供预计算 tokens 接口，可直接返回 0。
|
||||
|
||||
```python
|
||||
def get_num_tokens(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
|
||||
tools: Optional[list[PromptMessageTool]] = None) -> int:
|
||||
"""
|
||||
Get number of tokens for given prompt messages
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:param prompt_messages: prompt messages
|
||||
:param tools: tools for tool calling
|
||||
:return:
|
||||
"""
|
||||
```
|
||||
* 模型凭据校验
|
||||
|
||||
与供应商凭据校验类似，这里针对单个模型进行校验。
|
||||
|
||||
```python
|
||||
def validate_credentials(self, model: str, credentials: dict) -> None:
|
||||
"""
|
||||
Validate model credentials
|
||||
|
||||
:param model: model name
|
||||
:param credentials: model credentials
|
||||
:return:
|
||||
"""
|
||||
```
|
||||
* 调用异常错误映射表
|
||||
|
||||
当模型调用异常时需要映射到 Runtime 指定的 `InvokeError` 类型，方便 Dify 针对不同错误做不同后续处理。
|
||||
|
||||
Runtime Errors:
|
||||
|
||||
* `InvokeConnectionError` 调用连接错误
|
||||
* `InvokeServerUnavailableError` 调用服务方不可用
|
||||
* `InvokeRateLimitError` 调用达到限额
|
||||
* `InvokeAuthorizationError` 调用鉴权失败
|
||||
* `InvokeBadRequestError` 调用传参有误
|
||||
|
||||
```python
|
||||
@property
|
||||
def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
|
||||
"""
|
||||
Map model invoke error to unified error
|
||||
The key is the error type thrown to the caller
|
||||
The value is the error type thrown by the model,
|
||||
which needs to be converted into a unified error type for the caller.
|
||||
|
||||
:return: Invoke error mapping
|
||||
"""
|
||||
```
|
||||
|
||||
接口方法说明见：[Interfaces](https://github.com/langgenius/dify/blob/main/api/core/model\_runtime/docs/zh\_Hans/interfaces.md)，具体实现可参考：[llm.py](https://github.com/langgenius/dify-runtime/blob/main/lib/model\_providers/anthropic/llm/llm.py)。
|
||||
|
||||
[^1]: #### Provider
|
||||
|
||||
* `provider` (string) 供应商标识，如：`openai`
|
||||
* `label` (object) 供应商展示名称，i18n，可设置 `en_US` 英文、`zh_Hans` 中文两种语言
|
||||
* `zh_Hans` (string) \[optional] 中文标签名，`zh_Hans` 不设置将默认使用 `en_US`。
|
||||
* `en_US` (string) 英文标签名
|
||||
* `description` (object) \[optional] 供应商描述，i18n
|
||||
* `zh_Hans` (string) \[optional] 中文描述
|
||||
* `en_US` (string) 英文描述
|
||||
* `icon_small` (string) \[optional] 供应商小 ICON，存储在对应供应商实现目录下的 `_assets` 目录，中英文策略同 `label`
|
||||
* `zh_Hans` (string) \[optional] 中文 ICON
|
||||
* `en_US` (string) 英文 ICON
|
||||
* `icon_large` (string) \[optional] 供应商大 ICON，存储在对应供应商实现目录下的 \_assets 目录，中英文策略同 label
|
||||
* `zh_Hans` (string) \[optional] 中文 ICON
|
||||
* `en_US` (string) 英文 ICON
|
||||
* `background` (string) \[optional] 背景颜色色值，例：#FFFFFF，为空则展示前端默认色值。
|
||||
* `help` (object) \[optional] 帮助信息
|
||||
* `title` (object) 帮助标题，i18n
|
||||
* `zh_Hans` (string) \[optional] 中文标题
|
||||
* `en_US` (string) 英文标题
|
||||
* `url` (object) 帮助链接，i18n
|
||||
* `zh_Hans` (string) \[optional] 中文链接
|
||||
* `en_US` (string) 英文链接
|
||||
* `supported_model_types` (array\[ModelType]) 支持的模型类型
|
||||
* `configurate_methods` (array\[ConfigurateMethod]) 配置方式
|
||||
* `provider_credential_schema` (ProviderCredentialSchema) 供应商凭据规格
|
||||
* `model_credential_schema` (ModelCredentialSchema) 模型凭据规格
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Under Maintenance
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue