seg_yinzy
/
dify-docs
mirror of https://github.com/langgenius/dify-docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

doc: xin & localai

pull/33/head
crazywoola 2024-01-11 22:19:23 +08:00
parent a5479c106d
commit e85ca966b9
4 changed files with 10 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
en/advanced/model-configuration/localai.md
View File

@ -6,26 +6,6 @@ Dify allows integration with LocalAI for local deployment of large language mode
## Deploying LocalAI
### Before you start
When using Docker to deploy a private model locally, you might need to access the service via the container's IP address instead of `127.0.0.1`. This is because `127.0.0.1` or `localhost` by default points to your host system and not the internal network of the Docker container. To retrieve the IP address of your Docker container, you can follow these steps:
1. First, determine the name or ID of your Docker container. You can list all active containers using the following command:
```bash
docker ps
```
2. Then, use the command below to obtain detailed information about a specific container, including its IP address:
```bash
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_ID
```
Please note that you usually do not need to manually find the IP address of the Docker container to access the service, because Docker offers a port mapping feature. This allows you to map the container ports to local machine ports, enabling access via your local address. For example, if you used the `-p 80:80` parameter when running the container, you can access the service inside the container by visiting `http://localhost:80` or `http://127.0.0.1:80`.
If you do need to use the container's IP address directly, the steps above will assist you in obtaining this information.
### Starting LocalAI
You can refer to the official [Getting Started](https://localai.io/basics/getting_started/) guide for deployment, or quickly integrate following the steps below:
@ -79,6 +59,7 @@ You can refer to the official [Getting Started](https://localai.io/basics/gettin
- Embedding Model: `all-MiniLM-L6-v2`
External access name: `text-embedding-ada-002` (This name is customizable and can be configured in `models/embeddings.yaml`).
> If you use the Dify Docker deployment method, you need to pay attention to the network configuration to ensure that the Dify container can access the endpoint of LocalAI. The Dify container cannot access localhost inside, and you need to use the host IP address.
5. Integrate the models into Dify.
@ -104,7 +85,7 @@ You can refer to the official [Getting Started](https://localai.io/basics/gettin
- Server URL: http://127.0.0.1:8080
If Dify is deployed via docker, fill in the host domain: `http://<your-LocalAI-endpoint-domain>:8080`, which can be a LAN IP address, like: `http://192.168.1.100:8080`
> If Dify is deployed via docker, fill in the host domain: `http://<your-LocalAI-endpoint-domain>:8080`, which can be a LAN IP address, like: `http://192.168.1.100:8080`
Click "Save" to use the model in the application.

20
en/advanced/model-configuration/xinference.md
View File

@ -4,22 +4,6 @@
## Deploy Xinference
### Before you start
When using Docker to deploy a private model locally, you might need to access the service via the container's IP address instead of `127.0.0.1`. This is because `127.0.0.1` or `localhost` by default points to your host system and not the internal network of the Docker container. To retrieve the IP address of your Docker container, you can follow these steps:
1. First, determine the name or ID of your Docker container. You can list all active containers using the following command:
```bash
docker ps
```
2. Then, use the command below to obtain detailed information about a specific container, including its IP address:
```bash
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_ID
```
Please note that you usually do not need to manually find the IP address of the Docker container to access the service, because Docker offers a port mapping feature. This allows you to map the container ports to local machine ports, enabling access via your local address. For example, if you used the `-p 80:80` parameter when running the container, you can access the service inside the container by visiting `http://localhost:80` or `http://127.0.0.1:80`.
If you do need to use the container's IP address directly, the steps above will assist you in obtaining this information.
@ -43,6 +27,8 @@ There are two ways to deploy Xinference, namely [local deployment](https://githu
```
Xinference will start a worker locally by default, with the endpoint: `http://127.0.0.1:9997`, and the default port is `9997`. By default, access is limited to the local machine only, but it can be configured with `-H 0.0.0.0` to allow access from any non-local client. To modify the host or port, you can refer to xinference's help information: `xinference-local --help`.
> If you use the Dify Docker deployment method, you need to pay attention to the network configuration to ensure that the Dify container can access the endpoint of Xinference. The Dify container cannot access localhost inside, and you need to use the host IP address.
3. Create and deploy the model
Visit `http://127.0.0.1:9997`, select the model and specification you need to deploy, as shown below:
@ -58,7 +44,7 @@ There are two ways to deploy Xinference, namely [local deployment](https://githu
In `Settings > Model Providers > Xinference`, enter:
* Model name: `vicuna-v1.3`
* Server URL: `http://127.0.0.1:9997`
* Server URL: `http://<Machine_IP>:9997` **Replace with your machine IP address**
* Model UID: `2c886330-8849-11ee-9518-43b0b8f40bea`
Click "Save" to use the model in the dify application.

3
zh_CN/guides/model-configuration/localai.md
View File

@ -72,6 +72,7 @@ Dify 支持以本地部署的方式接入 LocalAI 部署的大型语言模型推
- Embedding 模型：`all-MiniLM-L6-v2`
对外访问名称：`text-embedding-ada-002`（该名称可自定义，在 `models/embeddings.yaml` 中配置。
> 使用 Dify Docker 部署方式的需要注意网络配置，确保 Dify 容器可以访问到 Xinference 的端点，Dify 容器内部无法访问到 localhost，需要使用宿主机 IP 地址。
5. LocalAI API 服务部署完毕，在 Dify 中使用接入模型
@ -97,7 +98,7 @@ Dify 支持以本地部署的方式接入 LocalAI 部署的大型语言模型推
- 服务器 URL：http://127.0.0.1:8080
若 Dify 为 docker 部署，请填入 host 域名：`http://<your-LocalAI-endpoint-domain>:8080`，可填写局域网 IP 地址，如：`http://192.168.1.100:8080`
> 若 Dify 为 docker 部署，请填入 host 域名：`http://<your-LocalAI-endpoint-domain>:8080`，可填写局域网 IP 地址，如：`http://192.168.1.100:8080`
"保存" 后即可在应用中使用该模型。

7
zh_CN/guides/model-configuration/xinference.md
View File

@ -4,9 +4,6 @@
## 部署 Xinference
### 使用前注意事项
### 开始部署
部署 Xinference 有两种方式，分别为[本地部署](https://github.com/xorbitsai/inference/blob/main/README\_zh\_CN.md#%E6%9C%AC%E5%9C%B0%E9%83%A8%E7%BD%B2)和[分布式部署](https://github.com/xorbitsai/inference/blob/main/README\_zh\_CN.md#%E5%88%86%E5%B8%83%E5%BC%8F%E9%83%A8%E7%BD%B2)，以下以本地部署为例。
@ -26,6 +23,8 @@
```
Xinference 默认会在本地启动一个 worker，端点为：`http://127.0.0.1:9997`，端口默认为 `9997`。 默认只可本机访问，可配置 `-H 0.0.0.0`，非本地客户端可任意访问。 如需进一步修改 host 或 port，可查看 xinference 的帮助信息：`xinference-local --help`。
> 使用 Dify Docker 部署方式的需要注意网络配置，确保 Dify 容器可以访问到 Xinference 的端点，Dify 容器内部无法访问到 localhost，需要使用宿主机 IP 地址。
3. 创建并部署模型
进入 `http://127.0.0.1:9997` 选择需要部署的模型和规格进行部署，如下图所示：
@ -41,7 +40,7 @@
在 `设置 > 模型供应商 > Xinference` 中填入：
* 模型名称：`vicuna-v1.3`
* 服务器 URL：`http://127.0.0.1:9997`
* 服务器 URL：`http://<Machine_IP>:9997` **替换成您的机器 IP 地址**
* 模型 UID：`2c886330-8849-11ee-9518-43b0b8f40bea`
"保存" 后即可在应用中使用该模型。