diff --git a/en/.gitbook/assets/content_moderation.png b/en/.gitbook/assets/content_moderation.png new file mode 100644 index 0000000..2127169 Binary files /dev/null and b/en/.gitbook/assets/content_moderation.png differ diff --git a/en/.gitbook/assets/moderation2.png b/en/.gitbook/assets/moderation2.png new file mode 100644 index 0000000..26819fd Binary files /dev/null and b/en/.gitbook/assets/moderation2.png differ diff --git a/en/.gitbook/assets/moderation3.png b/en/.gitbook/assets/moderation3.png new file mode 100644 index 0000000..82092ad Binary files /dev/null and b/en/.gitbook/assets/moderation3.png differ diff --git a/en/.gitbook/assets/moderation4.png b/en/.gitbook/assets/moderation4.png new file mode 100644 index 0000000..5c997a6 Binary files /dev/null and b/en/.gitbook/assets/moderation4.png differ diff --git a/en/SUMMARY.md b/en/SUMMARY.md index 0a4b1ff..03473a2 100644 --- a/en/SUMMARY.md +++ b/en/SUMMARY.md @@ -23,6 +23,7 @@ * [Text Generator](application/prompt-engineering/text-generation-application.md) * [Conversation Application](application/prompt-engineering/conversation-application.md) * [External-data-tool](application/prompt-engineering/external\_data\_tool.md) + * [Moderation](application/prompt-engineering/moderation\_tool.md) * [Developing with APIs](application/developing-with-apis.md) * [Logs & Annotations](application/logs.md) @@ -56,7 +57,7 @@ * [Extension](advanced/extension/README.md) * [API-based Extension](advanced/extension/api\_based\_extension/README.md) * [External\_data\_tool](advanced/extension/api\_based\_extension/external\_data\_tool.md) - * [Moderation](advanced/extension/api\_based\_extension/moderation.md) + * [Moderation Extension](advanced/extension/api\_based\_extension/moderation-extension.md) * [Code-based Extension](advanced/extension/code-based-extension.md) ## use cases diff --git a/en/advanced/extension/api_based_extension/moderation.md b/en/advanced/extension/api_based_extension/moderation-extension.md similarity index 100% rename from en/advanced/extension/api_based_extension/moderation.md rename to en/advanced/extension/api_based_extension/moderation-extension.md diff --git a/en/application/prompt-engineering/moderation_tool.md b/en/application/prompt-engineering/moderation_tool.md new file mode 100644 index 0000000..a64d395 --- /dev/null +++ b/en/application/prompt-engineering/moderation_tool.md @@ -0,0 +1,26 @@ +# Moderation + +In our interactions with AI applications, we often have stringent requirements in terms of content security, user experience, and legal regulations. At this point, we need the "Sensitive Word Review" feature to create a better interactive environment for end-users. On the prompt orchestration page, click "Add Function" and locate the "Content Review" toolbox at the bottom: + +

Content moderation

+ +## Call the OpenAI Moderation API + +OpenAI, along with most companies providing LLMs, includes content moderation features in their models to ensure that outputs do not contain controversial content, such as violence, sexual content, and illegal activities. Additionally, OpenAI has made this content moderation capability available, which you can refer to at [https://platform.openai.com/docs/guides/moderation/overview](https://platform.openai.com/docs/guides/moderation/overview). + +Now you can also directly call the OpenAI Moderation API on Dify; you can review either input or output content simply by entering the corresponding "preset reply." + +

OpenAI Moderation

+ +## Keywords + +Developers can customize the sensitive words they need to review, such as using "kill" as a keyword to perform an audit action when users input. The preset reply content should be "The content is violating usage policies." It can be anticipated that when a user inputs a text snippet containing "kill" at the terminal, it will trigger the sensitive word review tool and return the preset reply content. + +

Keywords

+ +## Moderation Extension + +Different enterprises often have their own mechanisms for sensitive word moderation. When developing their own AI applications, such as an internal knowledge base ChatBot, enterprises need to moderate the query content input by employees for sensitive words. For this purpose, developers can write an API extension based on their enterprise's internal sensitive word moderation mechanisms, specifically referring to [moderation-extension.md](../../advanced/extension/api\_based\_extension/moderation-extension.md "mention"), which can then be called on Dify to achieve a high degree of customization and privacy protection for sensitive word review. + +

Moderation Extension

+