Commit 7fbca0d72a278e9acefa525cccb2e01a7de2a8c7
1 parent
02c40701
启动脚本优化
Showing
21 changed files
with
135 additions
and
430 deletions
Show diff stats
.env.example
CLIP_SERVICE_README.md
| 1 | -## 基于 `clip-server` 的向量服务(平级替代 `embeddings`) | 1 | +## CLIP 服务说明(已收敛到 CN-CLIP) |
| 2 | 2 | ||
| 3 | -本模块说明如何在 **独立环境** 中部署基于 `jina-ai/clip-as-service` 仓库的向量服务(实际安装包为 `clip-server` / `clip-client`),用于替代当前仓库里的本地 `embeddings` 服务(`embeddings/server.py`)。 | 3 | +原有 `start_clip_service.sh / stop_clip_service.sh` 已移除,避免与 `cnclip` 重名和端口状态混淆。 |
| 4 | 4 | ||
| 5 | -> 设计目标: | ||
| 6 | -> - 与项目主环境(`searchengine` conda env)**完全隔离** | ||
| 7 | -> - 使用官方开源项目 [`jina-ai/clip-as-service`](https://github.com/jina-ai/clip-as-service)(对应 PyPI 包:`clip-server` / `clip-client`) | ||
| 8 | -> - 提供简单的 **安装 / 启动 / 停止脚本** | ||
| 9 | - | ||
| 10 | ---- | ||
| 11 | - | ||
| 12 | -## 1. 环境准备(独立环境) | ||
| 13 | - | ||
| 14 | -推荐使用 Conda 新建一个专用环境(与本项目的 `searchengine` 环境隔离): | ||
| 15 | - | ||
| 16 | -```bash | ||
| 17 | -# 1)加载 conda(你的 conda 是 ~/anaconda3/bin/conda → CONDA_ROOT=~/anaconda3) | ||
| 18 | -export CONDA_ROOT=${CONDA_ROOT:-$HOME/anaconda3} # 或你的 Conda 安装路径 | ||
| 19 | -source "$CONDA_ROOT/etc/profile.d/conda.sh" | ||
| 20 | - | ||
| 21 | -# 2)创建 clip 向量服务专用环境 | ||
| 22 | -conda create -n clip_service python=3.9 -y | ||
| 23 | - | ||
| 24 | -# 3)激活环境 | ||
| 25 | -conda activate clip_service | ||
| 26 | - | ||
| 27 | -# 4)安装 clip-server / clip-client(其内部依赖 jina) | ||
| 28 | -# 如需绕过镜像问题,可显式使用官方 PyPI 源: | ||
| 29 | -# pip install -i https://pypi.org/simple "clip-server" "clip-client" | ||
| 30 | -pip install "clip-server" "clip-client" | ||
| 31 | -``` | ||
| 32 | - | ||
| 33 | -> 如果你不使用 Conda,也可以改用 `python -m venv` 创建虚拟环境, | ||
| 34 | -> 但务必保证 **不要与主项目共用同一个 Python 环境**。 | ||
| 35 | - | ||
| 36 | ---- | ||
| 37 | - | ||
| 38 | -## 2. 启动 / 停止脚本 | ||
| 39 | - | ||
| 40 | -本仓库在 `scripts/` 目录下提供了两个脚本(需要手动赋权一次): | ||
| 41 | - | ||
| 42 | -```bash | ||
| 43 | -chmod +x scripts/start_clip_service.sh | ||
| 44 | -chmod +x scripts/stop_clip_service.sh | ||
| 45 | -``` | ||
| 46 | - | ||
| 47 | -### 2.1 启动服务 | ||
| 48 | - | ||
| 49 | -```bash | ||
| 50 | -cd /data/saas-search | ||
| 51 | -./scripts/start_clip_service.sh | ||
| 52 | -``` | ||
| 53 | - | ||
| 54 | -脚本行为: | ||
| 55 | - | ||
| 56 | -- 自动 `cd` 到仓库根目录 | ||
| 57 | -- 尝试加载 `$CONDA_ROOT/etc/profile.d/conda.sh` 并激活 `clip_service` 环境(可通过 `export CONDA_ROOT=...` 适配新机器) | ||
| 58 | -- 使用 `nohup python -m clip_server` 启动服务到后台 | ||
| 59 | -- 将日志写入 `logs/clip_service.log` | ||
| 60 | -- 将进程号写入 `logs/clip_service.pid` | ||
| 61 | - | ||
| 62 | -默认情况下,`clip-server` 会监听在 **`grpc://0.0.0.0:51000`**(gRPC 协议,端口 51000)。 | ||
| 63 | - | ||
| 64 | -> ⚠️ **重要**:客户端连接时请使用端口 **51000**,不是 23456 或其他端口。 | ||
| 65 | - | ||
| 66 | -### 2.2 停止服务 | ||
| 67 | - | ||
| 68 | -```bash | ||
| 69 | -cd /data/saas-search | ||
| 70 | -./scripts/stop_clip_service.sh | ||
| 71 | -``` | ||
| 72 | - | ||
| 73 | -脚本行为: | ||
| 74 | - | ||
| 75 | -- 读取 `logs/clip_service.pid` 中的 PID | ||
| 76 | -- 如果进程存在则发送 `kill` 终止 | ||
| 77 | -- 清理 `logs/clip_service.pid` | ||
| 78 | - | ||
| 79 | ---- | ||
| 80 | - | ||
| 81 | -## 3. 与现有 `embeddings` 服务的关系 | ||
| 82 | - | ||
| 83 | -- 现有本地向量服务: | ||
| 84 | - - 启动脚本:`./scripts/start_embedding_service.sh` | ||
| 85 | - - 实现:`embeddings/server.py`(FastAPI + 本地模型 `qwen3_model.py` / `clip_model.py`) | ||
| 86 | -- 新增的 `clip-server`: | ||
| 87 | - - 使用官方实现,单独进程、单独环境 | ||
| 88 | - - 面向图像 / 文本的 CLIP 向量化服务 | ||
| 89 | - | ||
| 90 | -### 使用建议 | ||
| 91 | - | ||
| 92 | -- 如果你想继续使用本仓库自带的本地模型服务,保持原有脚本不变即可: | ||
| 93 | - - `./scripts/start_embedding_service.sh` | ||
| 94 | -- 如果你想用 `clip-as-service` 替代原来的本地服务,可以: | ||
| 95 | - - 在上游调用代码中,将向量请求切换到 `clip-as-service` 对应的端口 / 接口 | ||
| 96 | - - 或者增加一个适配层,将 `clip-as-service` 封装成与 `POST /embed/text` / `POST /embed/image` 相同的接口(视具体场景而定) | ||
| 97 | - | ||
| 98 | ---- | ||
| 99 | - | ||
| 100 | -## 4. 基本验证 | ||
| 101 | - | ||
| 102 | -1. 确认 `clip_service` 环境创建并安装成功: | ||
| 103 | - | ||
| 104 | - ```bash | ||
| 105 | - export CONDA_ROOT=${CONDA_ROOT:-$HOME/anaconda3} | ||
| 106 | - source "$CONDA_ROOT/etc/profile.d/conda.sh" | ||
| 107 | - conda activate clip_service | ||
| 108 | - python -c "import jina; print('jina version:', jina.__version__)" | ||
| 109 | - ``` | ||
| 110 | - | ||
| 111 | -2. 启动服务并查看日志: | ||
| 112 | - | ||
| 113 | - ```bash | ||
| 114 | - cd /data/saas-search | ||
| 115 | - ./scripts/start_clip_service.sh | ||
| 116 | - tail -f logs/clip_service.log | ||
| 117 | - ``` | ||
| 118 | - | ||
| 119 | - 服务启动后,默认监听在 **`grpc://0.0.0.0:51000`**(gRPC 协议,端口 51000)。 | ||
| 120 | - | ||
| 121 | -3. 测试客户端连接(在 `clip_service` 环境中): | ||
| 122 | - | ||
| 123 | - ```python | ||
| 124 | - from clip_client import Client | ||
| 125 | - | ||
| 126 | - # 注意:默认端口是 51000,不是 23456 | ||
| 127 | - c = Client('grpc://0.0.0.0:51000') | ||
| 128 | - | ||
| 129 | - # 测试连接 | ||
| 130 | - c.profile() | ||
| 131 | - | ||
| 132 | - # 测试文本向量化 | ||
| 133 | - r = c.encode(['First do it', 'then do it right', 'then do it better']) | ||
| 134 | - print(r.shape) # 应该输出 [3, 512] 或类似形状 | ||
| 135 | - | ||
| 136 | - # 测试图像向量化 | ||
| 137 | - r = c.encode(['https://picsum.photos/200']) | ||
| 138 | - print(r.shape) # 应该输出 [1, 512] 或类似形状 | ||
| 139 | - ``` | ||
| 140 | - | ||
| 141 | -4. 如果不再需要服务,执行: | ||
| 142 | - | ||
| 143 | - ```bash | ||
| 144 | - ./scripts/stop_clip_service.sh | ||
| 145 | - ``` | ||
| 146 | - | ||
| 147 | -### 常见问题 | ||
| 148 | - | ||
| 149 | -**Q: 连接被拒绝(Connection refused)?** | ||
| 150 | -A: 请确认: | ||
| 151 | -- 服务已启动(检查 `logs/clip_service.log` 和进程) | ||
| 152 | -- 客户端使用的端口是 **51000**(不是 23456) | ||
| 153 | -- 客户端地址格式正确:`grpc://0.0.0.0:51000` 或 `grpc://localhost:51000` | ||
| 154 | - | ||
| 155 | -**Q: Gateway 启动了但 worker 连接失败?** | ||
| 156 | -A: 可能原因: | ||
| 157 | -- Worker 进程(clip_t)还在启动中,模型加载需要时间(首次启动可能需要下载模型) | ||
| 158 | -- 检查日志中是否有模型下载或加载错误: | ||
| 159 | - ```bash | ||
| 160 | - tail -f logs/clip_service.log | grep -E "(ERROR|WARNING|model|download)" | ||
| 161 | - ``` | ||
| 162 | -- 如果持续失败,尝试重启服务: | ||
| 163 | - ```bash | ||
| 164 | - ./scripts/stop_clip_service.sh | ||
| 165 | - ./scripts/start_clip_service.sh | ||
| 166 | - ``` | ||
| 167 | - | ||
| 168 | -**Q: 如何查看服务实际监听的端口?** | ||
| 169 | -A: 查看启动日志: | ||
| 170 | -```bash | ||
| 171 | -tail -f logs/clip_service.log | grep "bound to" | ||
| 172 | -``` | ||
| 173 | -或检查进程监听的端口: | ||
| 174 | -```bash | ||
| 175 | -lsof -i :51000 | ||
| 176 | -# 或 | ||
| 177 | -netstat -tlnp | grep 51000 | ||
| 178 | -``` | ||
| 179 | - | ||
| 180 | -**Q: 如何确认服务完全就绪?** | ||
| 181 | -A: 查看日志,确认看到类似输出: | ||
| 182 | -``` | ||
| 183 | -INFO gateway/rep-0@XXXXX start server bound to 0.0.0.0:51000 | ||
| 184 | -``` | ||
| 185 | -然后等待几秒让 worker 进程启动,再测试客户端连接。 | ||
| 186 | - | ||
| 187 | ---- | ||
| 188 | - | ||
| 189 | -## 5. 参考 | ||
| 190 | - | ||
| 191 | -- 项目地址:`https://github.com/jina-ai/clip-as-service` | ||
| 192 | -- 本项目向量模块文档:`embeddings/README.md` | 5 | +当前统一入口: |
| 193 | 6 | ||
| 7 | +- 环境:`./scripts/setup_cnclip_venv.sh` | ||
| 8 | +- 启动:`./scripts/start_cnclip_service.sh` | ||
| 9 | +- 停止:`./scripts/stop_cnclip_service.sh` | ||
| 10 | +- 编排:`./scripts/service_ctl.sh start cnclip` | ||
| 194 | 11 | ||
| 12 | +请以 `docs/CNCLIP_SERVICE说明文档.md` 为唯一文档入口。 |
README.md
| @@ -28,8 +28,8 @@ source activate.sh | @@ -28,8 +28,8 @@ source activate.sh | ||
| 28 | # 启动核心服务(backend/indexer/frontend) | 28 | # 启动核心服务(backend/indexer/frontend) |
| 29 | ./run.sh | 29 | ./run.sh |
| 30 | 30 | ||
| 31 | -# 可选:附加能力服务 | ||
| 32 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 ./run.sh | 31 | +# 可选:附加能力服务(按需开启) |
| 32 | +START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./run.sh | ||
| 33 | 33 | ||
| 34 | # 查看状态 | 34 | # 查看状态 |
| 35 | ./scripts/service_ctl.sh status | 35 | ./scripts/service_ctl.sh status |
docs/CNCLIP_SERVICE说明文档.md
| @@ -70,11 +70,9 @@ cd /data/saas-search | @@ -70,11 +70,9 @@ cd /data/saas-search | ||
| 70 | ### 6.1 通过统一编排启动 | 70 | ### 6.1 通过统一编排启动 |
| 71 | 71 | ||
| 72 | ```bash | 72 | ```bash |
| 73 | -START_EMBEDDING=1 START_TEI=1 ./scripts/service_ctl.sh start | 73 | +START_EMBEDDING=1 START_TEI=1 START_CNCLIP=1 ./scripts/service_ctl.sh start |
| 74 | ``` | 74 | ``` |
| 75 | 75 | ||
| 76 | -当 `USE_CLIP_AS_SERVICE=true` 且 `START_EMBEDDING=1` 时,`service_ctl` 会自动拉起 `cnclip`。 | ||
| 77 | - | ||
| 78 | ### 6.2 设备选择优先级 | 76 | ### 6.2 设备选择优先级 |
| 79 | 77 | ||
| 80 | - 显式传入 `CNCLIP_DEVICE` 时,以该值为准: | 78 | - 显式传入 `CNCLIP_DEVICE` 时,以该值为准: |
| @@ -153,4 +151,3 @@ curl -sS -X POST "http://127.0.0.1:6005/embed/image" \ | @@ -153,4 +151,3 @@ curl -sS -X POST "http://127.0.0.1:6005/embed/image" \ | ||
| 153 | - TEI 专项:`docs/TEI_SERVICE说明文档.md` | 151 | - TEI 专项:`docs/TEI_SERVICE说明文档.md` |
| 154 | - 体系规范:`docs/DEVELOPER_GUIDE.md` | 152 | - 体系规范:`docs/DEVELOPER_GUIDE.md` |
| 155 | - embedding 模块:`embeddings/README.md` | 153 | - embedding 模块:`embeddings/README.md` |
| 156 | - |
docs/QUICKSTART.md
| @@ -66,10 +66,9 @@ source activate.sh | @@ -66,10 +66,9 @@ source activate.sh | ||
| 66 | ```bash | 66 | ```bash |
| 67 | ./run.sh | 67 | ./run.sh |
| 68 | # 启动全部能力 | 68 | # 启动全部能力 |
| 69 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh | 69 | +START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./run.sh |
| 70 | # 等价方式(直接使用服务控制器) | 70 | # 等价方式(直接使用服务控制器) |
| 71 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./scripts/service_ctl.sh start | ||
| 72 | -# 说明:当 USE_CLIP_AS_SERVICE=true(默认)且 START_EMBEDDING=1 时,会自动启动 cnclip(51000) | 71 | +START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./scripts/service_ctl.sh start |
| 73 | # 说明: | 72 | # 说明: |
| 74 | # - reranker 为 GPU 强制模式(资源不足会直接启动失败) | 73 | # - reranker 为 GPU 强制模式(资源不足会直接启动失败) |
| 75 | # - TEI 默认使用 GPU;当 TEI_USE_GPU=1 且 GPU 不可用时会直接失败(不会自动降级到 CPU) | 74 | # - TEI 默认使用 GPU;当 TEI_USE_GPU=1 且 GPU 不可用时会直接失败(不会自动降级到 CPU) |
docs/TEI_SERVICE说明文档.md
| @@ -152,7 +152,7 @@ curl -sS -X POST "http://127.0.0.1:6005/embed/text" \ | @@ -152,7 +152,7 @@ curl -sS -X POST "http://127.0.0.1:6005/embed/text" \ | ||
| 152 | 启动全套(含 TEI): | 152 | 启动全套(含 TEI): |
| 153 | 153 | ||
| 154 | ```bash | 154 | ```bash |
| 155 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 TEI_USE_GPU=1 ./scripts/service_ctl.sh start | 155 | +START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 TEI_USE_GPU=1 ./scripts/service_ctl.sh start |
| 156 | ``` | 156 | ``` |
| 157 | 157 | ||
| 158 | 仅启动 TEI: | 158 | 仅启动 TEI: |
docs/Usage-Guide.md
| @@ -135,10 +135,10 @@ cd /data/saas-search | @@ -135,10 +135,10 @@ cd /data/saas-search | ||
| 135 | - **API文档**: http://localhost:6002/docs | 135 | - **API文档**: http://localhost:6002/docs |
| 136 | - **索引API**: http://localhost:6004/docs | 136 | - **索引API**: http://localhost:6004/docs |
| 137 | 137 | ||
| 138 | -可选:全功能模式(同时启动 embedding/translator/reranker/tei): | 138 | +可选:全功能模式(同时启动 embedding/translator/reranker/tei/cnclip): |
| 139 | 139 | ||
| 140 | ```bash | 140 | ```bash |
| 141 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh | 141 | +START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./run.sh |
| 142 | ``` | 142 | ``` |
| 143 | 143 | ||
| 144 | ### 方式2: 统一控制脚本(推荐) | 144 | ### 方式2: 统一控制脚本(推荐) |
| @@ -151,7 +151,7 @@ START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh | @@ -151,7 +151,7 @@ START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh | ||
| 151 | ./scripts/service_ctl.sh start | 151 | ./scripts/service_ctl.sh start |
| 152 | 152 | ||
| 153 | # 启动指定服务 | 153 | # 启动指定服务 |
| 154 | -./scripts/service_ctl.sh start backend indexer frontend translator reranker tei | 154 | +./scripts/service_ctl.sh start backend indexer frontend translator reranker tei cnclip |
| 155 | 155 | ||
| 156 | # 停止全部服务(含可选服务) | 156 | # 停止全部服务(含可选服务) |
| 157 | ./scripts/service_ctl.sh stop | 157 | ./scripts/service_ctl.sh stop |
| @@ -311,6 +311,8 @@ RERANKER_PORT=6007 | @@ -311,6 +311,8 @@ RERANKER_PORT=6007 | ||
| 311 | START_EMBEDDING=0 | 311 | START_EMBEDDING=0 |
| 312 | START_TRANSLATOR=0 | 312 | START_TRANSLATOR=0 |
| 313 | START_RERANKER=0 | 313 | START_RERANKER=0 |
| 314 | +START_TEI=0 | ||
| 315 | +START_CNCLIP=0 | ||
| 314 | ``` | 316 | ``` |
| 315 | 317 | ||
| 316 | ### 修改配置 | 318 | ### 修改配置 |
docs/搜索API对接指南.md
| @@ -1726,7 +1726,7 @@ curl "http://localhost:6005/health" | @@ -1726,7 +1726,7 @@ curl "http://localhost:6005/health" | ||
| 1726 | 1726 | ||
| 1727 | ```bash | 1727 | ```bash |
| 1728 | ./scripts/start_tei_service.sh | 1728 | ./scripts/start_tei_service.sh |
| 1729 | -START_TEI=0 ./scripts/service_ctl.sh restart embedding | 1729 | +./scripts/service_ctl.sh restart embedding |
| 1730 | ``` | 1730 | ``` |
| 1731 | 1731 | ||
| 1732 | 默认端口: | 1732 | 默认端口: |
restart.sh
| @@ -4,7 +4,4 @@ | @@ -4,7 +4,4 @@ | ||
| 4 | 4 | ||
| 5 | cd "$(dirname "$0")" | 5 | cd "$(dirname "$0")" |
| 6 | 6 | ||
| 7 | -START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 CNCLIP_DEVICE=cuda TEI_USE_GPU=1 ./scripts/service_ctl.sh restart | ||
| 8 | - | ||
| 9 | -# ./scripts/service_ctl.sh restart | ||
| 10 | - | 7 | +./scripts/service_ctl.sh restart |
scripts/frontend_server.py
| @@ -27,7 +27,7 @@ frontend_dir = os.path.join(os.path.dirname(__file__), '../frontend') | @@ -27,7 +27,7 @@ frontend_dir = os.path.join(os.path.dirname(__file__), '../frontend') | ||
| 27 | os.chdir(frontend_dir) | 27 | os.chdir(frontend_dir) |
| 28 | 28 | ||
| 29 | # Get port from environment variable or default | 29 | # Get port from environment variable or default |
| 30 | -PORT = int(os.getenv('PORT', 6003)) | 30 | +PORT = int(os.getenv('FRONTEND_PORT', 6003)) |
| 31 | 31 | ||
| 32 | # Configure logging to suppress scanner noise | 32 | # Configure logging to suppress scanner noise |
| 33 | logging.basicConfig(level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s') | 33 | logging.basicConfig(level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s') |
scripts/service_ctl.sh
| @@ -15,11 +15,10 @@ mkdir -p "${LOG_DIR}" | @@ -15,11 +15,10 @@ mkdir -p "${LOG_DIR}" | ||
| 15 | source "${PROJECT_ROOT}/scripts/lib/load_env.sh" | 15 | source "${PROJECT_ROOT}/scripts/lib/load_env.sh" |
| 16 | 16 | ||
| 17 | CORE_SERVICES=("backend" "indexer" "frontend") | 17 | CORE_SERVICES=("backend" "indexer" "frontend") |
| 18 | -OPTIONAL_SERVICES=("embedding" "translator" "reranker" "tei") | ||
| 19 | -LEGACY_SERVICES=("clip" "cnclip") | 18 | +OPTIONAL_SERVICES=("embedding" "translator" "reranker" "tei" "cnclip") |
| 20 | 19 | ||
| 21 | all_services() { | 20 | all_services() { |
| 22 | - echo "${CORE_SERVICES[@]} ${OPTIONAL_SERVICES[@]} ${LEGACY_SERVICES[@]}" | 21 | + echo "${CORE_SERVICES[@]} ${OPTIONAL_SERVICES[@]}" |
| 23 | } | 22 | } |
| 24 | 23 | ||
| 25 | get_port() { | 24 | get_port() { |
| @@ -32,7 +31,6 @@ get_port() { | @@ -32,7 +31,6 @@ get_port() { | ||
| 32 | translator) echo "${TRANSLATION_PORT:-${TRANSLATOR_PORT:-6006}}" ;; | 31 | translator) echo "${TRANSLATION_PORT:-${TRANSLATOR_PORT:-6006}}" ;; |
| 33 | reranker) echo "${RERANKER_PORT:-6007}" ;; | 32 | reranker) echo "${RERANKER_PORT:-6007}" ;; |
| 34 | tei) echo "${TEI_PORT:-8080}" ;; | 33 | tei) echo "${TEI_PORT:-8080}" ;; |
| 35 | - clip) echo "${CLIP_PORT:-51000}" ;; | ||
| 36 | cnclip) echo "${CNCLIP_PORT:-51000}" ;; | 34 | cnclip) echo "${CNCLIP_PORT:-51000}" ;; |
| 37 | *) echo "" ;; | 35 | *) echo "" ;; |
| 38 | esac | 36 | esac |
| @@ -41,7 +39,6 @@ get_port() { | @@ -41,7 +39,6 @@ get_port() { | ||
| 41 | pid_file() { | 39 | pid_file() { |
| 42 | local service="$1" | 40 | local service="$1" |
| 43 | case "${service}" in | 41 | case "${service}" in |
| 44 | - clip) echo "${LOG_DIR}/clip_service.pid" ;; | ||
| 45 | cnclip) echo "${LOG_DIR}/cnclip_service.pid" ;; | 42 | cnclip) echo "${LOG_DIR}/cnclip_service.pid" ;; |
| 46 | *) echo "${LOG_DIR}/${service}.pid" ;; | 43 | *) echo "${LOG_DIR}/${service}.pid" ;; |
| 47 | esac | 44 | esac |
| @@ -62,7 +59,6 @@ service_start_cmd() { | @@ -62,7 +59,6 @@ service_start_cmd() { | ||
| 62 | translator) echo "./scripts/start_translator.sh" ;; | 59 | translator) echo "./scripts/start_translator.sh" ;; |
| 63 | reranker) echo "./scripts/start_reranker.sh" ;; | 60 | reranker) echo "./scripts/start_reranker.sh" ;; |
| 64 | tei) echo "./scripts/start_tei_service.sh" ;; | 61 | tei) echo "./scripts/start_tei_service.sh" ;; |
| 65 | - clip) echo "./scripts/start_clip_service.sh" ;; | ||
| 66 | cnclip) echo "./scripts/start_cnclip_service.sh" ;; | 62 | cnclip) echo "./scripts/start_cnclip_service.sh" ;; |
| 67 | *) return 1 ;; | 63 | *) return 1 ;; |
| 68 | esac | 64 | esac |
| @@ -158,7 +154,10 @@ start_one() { | @@ -158,7 +154,10 @@ start_one() { | ||
| 158 | local service="$1" | 154 | local service="$1" |
| 159 | cd "${PROJECT_ROOT}" | 155 | cd "${PROJECT_ROOT}" |
| 160 | local cmd | 156 | local cmd |
| 161 | - cmd="$(service_start_cmd "${service}")" | 157 | + if ! cmd="$(service_start_cmd "${service}")"; then |
| 158 | + echo "[error] unknown service: ${service}" >&2 | ||
| 159 | + return 1 | ||
| 160 | + fi | ||
| 162 | local pf lf | 161 | local pf lf |
| 163 | pf="$(pid_file "${service}")" | 162 | pf="$(pid_file "${service}")" |
| 164 | lf="$(log_file "${service}")" | 163 | lf="$(log_file "${service}")" |
| @@ -186,7 +185,7 @@ start_one() { | @@ -186,7 +185,7 @@ start_one() { | ||
| 186 | fi | 185 | fi |
| 187 | 186 | ||
| 188 | case "${service}" in | 187 | case "${service}" in |
| 189 | - clip|cnclip|tei) | 188 | + cnclip|tei) |
| 190 | echo "[start] ${service} (managed by native script)" | 189 | echo "[start] ${service} (managed by native script)" |
| 191 | if [ "${service}" = "cnclip" ]; then | 190 | if [ "${service}" = "cnclip" ]; then |
| 192 | CNCLIP_DEVICE="${CNCLIP_DEVICE:-cuda}" bash -lc "${cmd}" >> "${lf}" 2>&1 | 191 | CNCLIP_DEVICE="${CNCLIP_DEVICE:-cuda}" bash -lc "${cmd}" >> "${lf}" 2>&1 |
| @@ -248,11 +247,6 @@ start_one() { | @@ -248,11 +247,6 @@ start_one() { | ||
| 248 | stop_one() { | 247 | stop_one() { |
| 249 | local service="$1" | 248 | local service="$1" |
| 250 | cd "${PROJECT_ROOT}" | 249 | cd "${PROJECT_ROOT}" |
| 251 | - if [ "${service}" = "clip" ]; then | ||
| 252 | - echo "[stop] clip (managed by native script)" | ||
| 253 | - bash -lc "./scripts/stop_clip_service.sh" || true | ||
| 254 | - return 0 | ||
| 255 | - fi | ||
| 256 | if [ "${service}" = "cnclip" ]; then | 250 | if [ "${service}" = "cnclip" ]; then |
| 257 | echo "[stop] cnclip (managed by native script)" | 251 | echo "[stop] cnclip (managed by native script)" |
| 258 | bash -lc "./scripts/stop_cnclip_service.sh" || true | 252 | bash -lc "./scripts/stop_cnclip_service.sh" || true |
| @@ -347,17 +341,9 @@ resolve_targets() { | @@ -347,17 +341,9 @@ resolve_targets() { | ||
| 347 | case "${scope}" in | 341 | case "${scope}" in |
| 348 | start) | 342 | start) |
| 349 | local targets=("${CORE_SERVICES[@]}") | 343 | local targets=("${CORE_SERVICES[@]}") |
| 350 | - # Start TEI before embedding when both are enabled, because embedding | ||
| 351 | - # tei backend performs strict startup health checks against TEI. | ||
| 352 | if [ "${START_TEI:-0}" = "1" ]; then targets+=("tei"); fi | 344 | if [ "${START_TEI:-0}" = "1" ]; then targets+=("tei"); fi |
| 353 | - if [ "${START_EMBEDDING:-0}" = "1" ]; then | ||
| 354 | - local use_clip="${USE_CLIP_AS_SERVICE:-true}" | ||
| 355 | - use_clip="$(echo "${use_clip}" | tr '[:upper:]' '[:lower:]')" | ||
| 356 | - if [[ "${use_clip}" == "1" || "${use_clip}" == "true" || "${use_clip}" == "yes" ]]; then | ||
| 357 | - targets+=("cnclip") | ||
| 358 | - fi | ||
| 359 | - targets+=("embedding") | ||
| 360 | - fi | 345 | + if [ "${START_CNCLIP:-0}" = "1" ]; then targets+=("cnclip"); fi |
| 346 | + if [ "${START_EMBEDDING:-0}" = "1" ]; then targets+=("embedding"); fi | ||
| 361 | if [ "${START_TRANSLATOR:-0}" = "1" ]; then targets+=("translator"); fi | 347 | if [ "${START_TRANSLATOR:-0}" = "1" ]; then targets+=("translator"); fi |
| 362 | if [ "${START_RERANKER:-0}" = "1" ]; then targets+=("reranker"); fi | 348 | if [ "${START_RERANKER:-0}" = "1" ]; then targets+=("reranker"); fi |
| 363 | echo "${targets[@]}" | 349 | echo "${targets[@]}" |
| @@ -366,8 +352,7 @@ resolve_targets() { | @@ -366,8 +352,7 @@ resolve_targets() { | ||
| 366 | echo "$(all_services)" | 352 | echo "$(all_services)" |
| 367 | ;; | 353 | ;; |
| 368 | restart) | 354 | restart) |
| 369 | - # Restart with no explicit services should preserve start-order dependency | ||
| 370 | - # behavior (e.g. tei/cnclip before embedding). | 355 | + # Restart with no explicit services uses the same explicit start target order. |
| 371 | echo "$(resolve_targets start)" | 356 | echo "$(resolve_targets start)" |
| 372 | ;; | 357 | ;; |
| 373 | *) | 358 | *) |
| @@ -391,9 +376,8 @@ Default target set (when no service provided): | @@ -391,9 +376,8 @@ Default target set (when no service provided): | ||
| 391 | status -> all known services | 376 | status -> all known services |
| 392 | 377 | ||
| 393 | Optional startup flags: | 378 | Optional startup flags: |
| 394 | - START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh | ||
| 395 | - START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./scripts/service_ctl.sh start | ||
| 396 | - # when USE_CLIP_AS_SERVICE=true (default), START_EMBEDDING=1 will auto-start cnclip | 379 | + START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./run.sh |
| 380 | + START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./scripts/service_ctl.sh start | ||
| 397 | CNCLIP_DEVICE=cuda|cpu ./scripts/service_ctl.sh start cnclip | 381 | CNCLIP_DEVICE=cuda|cpu ./scripts/service_ctl.sh start cnclip |
| 398 | EOF | 382 | EOF |
| 399 | } | 383 | } |
| @@ -411,7 +395,7 @@ main() { | @@ -411,7 +395,7 @@ main() { | ||
| 411 | local stop_targets="" | 395 | local stop_targets="" |
| 412 | local targets | 396 | local targets |
| 413 | # For restart without explicit services, stop everything first, then start | 397 | # For restart without explicit services, stop everything first, then start |
| 414 | - # with dependency-aware start targets. | 398 | + # with the default start target order. |
| 415 | if [ "${action}" = "restart" ] && [ "$#" -eq 0 ]; then | 399 | if [ "${action}" = "restart" ] && [ "$#" -eq 0 ]; then |
| 416 | stop_targets="$(resolve_targets stop)" | 400 | stop_targets="$(resolve_targets stop)" |
| 417 | fi | 401 | fi |
scripts/start.sh
| @@ -12,7 +12,7 @@ echo "saas-search 服务启动" | @@ -12,7 +12,7 @@ echo "saas-search 服务启动" | ||
| 12 | echo "========================================" | 12 | echo "========================================" |
| 13 | echo "默认启动核心服务: backend/indexer/frontend" | 13 | echo "默认启动核心服务: backend/indexer/frontend" |
| 14 | echo "可选服务通过环境变量开启:" | 14 | echo "可选服务通过环境变量开启:" |
| 15 | -echo " START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 ./run.sh" | 15 | +echo " START_EMBEDDING=1 START_TRANSLATOR=1 START_RERANKER=1 START_TEI=1 START_CNCLIP=1 ./run.sh" |
| 16 | echo | 16 | echo |
| 17 | 17 | ||
| 18 | ./scripts/service_ctl.sh start | 18 | ./scripts/service_ctl.sh start |
scripts/start_clip_service.sh deleted
| @@ -1,57 +0,0 @@ | @@ -1,57 +0,0 @@ | ||
| 1 | -#!/bin/bash | ||
| 2 | -# | ||
| 3 | -# Start CLIP vector service (clip-server) in an independent environment. | ||
| 4 | -# | ||
| 5 | -# This service is designed to be a drop-in alternative to the local | ||
| 6 | -# `embeddings` service, but runs in its own Python environment and depends | ||
| 7 | -# on `jina` via `clip-server`. | ||
| 8 | -# | ||
| 9 | -set -e | ||
| 10 | - | ||
| 11 | -cd "$(dirname "$0")/.." | ||
| 12 | - | ||
| 13 | -LOG_DIR="$(pwd)/logs" | ||
| 14 | -mkdir -p "${LOG_DIR}" | ||
| 15 | -PID_FILE="${LOG_DIR}/clip_service.pid" | ||
| 16 | -LOG_FILE="${LOG_DIR}/clip_service.log" | ||
| 17 | - | ||
| 18 | -echo "========================================" | ||
| 19 | -echo "Starting CLIP vector service (clip-server)" | ||
| 20 | -echo "========================================" | ||
| 21 | - | ||
| 22 | -# Force isolated CN-CLIP runtime env | ||
| 23 | -CLIP_VENV="$(pwd)/.venv-cnclip" | ||
| 24 | -if [ ! -x "${CLIP_VENV}/bin/python" ]; then | ||
| 25 | - echo "Error: isolated clip runtime not found: ${CLIP_VENV}" >&2 | ||
| 26 | - echo "Please run: ./scripts/setup_cnclip_venv.sh" >&2 | ||
| 27 | - exit 1 | ||
| 28 | -fi | ||
| 29 | -PYTHON_BIN="${CLIP_VENV}/bin/python" | ||
| 30 | - | ||
| 31 | -if [ -f "${PID_FILE}" ]; then | ||
| 32 | - EXISTING_PID="$(cat "${PID_FILE}")" | ||
| 33 | - if ps -p "${EXISTING_PID}" > /dev/null 2>&1; then | ||
| 34 | - echo "clip-server already appears to be running with PID ${EXISTING_PID}." | ||
| 35 | - echo "If this is incorrect, remove ${PID_FILE} and try again." | ||
| 36 | - exit 0 | ||
| 37 | - else | ||
| 38 | - echo "Stale PID file found at ${PID_FILE}, removing..." | ||
| 39 | - rm -f "${PID_FILE}" | ||
| 40 | - fi | ||
| 41 | -fi | ||
| 42 | - | ||
| 43 | -echo "Log file: ${LOG_FILE}" | ||
| 44 | -echo "PID file: ${PID_FILE}" | ||
| 45 | -echo | ||
| 46 | -echo "Starting clip-server in background..." | ||
| 47 | - | ||
| 48 | -nohup "${PYTHON_BIN}" -m clip_server > "${LOG_FILE}" 2>&1 & | ||
| 49 | -SERVICE_PID=$! | ||
| 50 | -echo "${SERVICE_PID}" > "${PID_FILE}" | ||
| 51 | - | ||
| 52 | -echo "clip-server started with PID ${SERVICE_PID}." | ||
| 53 | -echo "You can check logs with:" | ||
| 54 | -echo " tail -f ${LOG_FILE}" | ||
| 55 | - | ||
| 56 | - | ||
| 57 | - |
scripts/start_embedding_service.sh
| @@ -21,32 +21,10 @@ if [[ ! -x "${PYTHON_BIN}" ]]; then | @@ -21,32 +21,10 @@ if [[ ! -x "${PYTHON_BIN}" ]]; then | ||
| 21 | exit 1 | 21 | exit 1 |
| 22 | fi | 22 | fi |
| 23 | 23 | ||
| 24 | -# Load .env if present (same behavior as activate.sh, without activating main venv) | ||
| 25 | -ENV_FILE="${PROJECT_ROOT}/.env" | ||
| 26 | -if [ -f "${ENV_FILE}" ]; then | ||
| 27 | - while IFS= read -r line || [ -n "${line}" ]; do | ||
| 28 | - line="${line%$'\r'}" | ||
| 29 | - [[ -z "${line//[[:space:]]/}" ]] && continue | ||
| 30 | - [[ "${line}" =~ ^[[:space:]]*# ]] && continue | ||
| 31 | - [[ "${line}" != *=* ]] && continue | ||
| 32 | - | ||
| 33 | - key="${line%%=*}" | ||
| 34 | - value="${line#*=}" | ||
| 35 | - key="${key#"${key%%[![:space:]]*}"}" | ||
| 36 | - key="${key%"${key##*[![:space:]]}"}" | ||
| 37 | - value="${value#"${value%%[![:space:]]*}"}" | ||
| 38 | - | ||
| 39 | - if [[ ${#value} -ge 2 ]]; then | ||
| 40 | - first="${value:0:1}" | ||
| 41 | - last="${value: -1}" | ||
| 42 | - if [[ ("${first}" == '"' && "${last}" == '"') || ("${first}" == "'" && "${last}" == "'") ]]; then | ||
| 43 | - value="${value:1:${#value}-2}" | ||
| 44 | - fi | ||
| 45 | - fi | ||
| 46 | - | ||
| 47 | - export "${key}=${value}" | ||
| 48 | - done < "${ENV_FILE}" | ||
| 49 | -fi | 24 | +# Load .env without activating main venv. |
| 25 | +# shellcheck source=scripts/lib/load_env.sh | ||
| 26 | +source "${PROJECT_ROOT}/scripts/lib/load_env.sh" | ||
| 27 | +load_env_file "${PROJECT_ROOT}/.env" | ||
| 50 | 28 | ||
| 51 | DEFAULT_EMBEDDING_SERVICE_HOST=$("${PYTHON_BIN}" -c "from embeddings.config import CONFIG; print(CONFIG.HOST)") | 29 | DEFAULT_EMBEDDING_SERVICE_HOST=$("${PYTHON_BIN}" -c "from embeddings.config import CONFIG; print(CONFIG.HOST)") |
| 52 | DEFAULT_EMBEDDING_SERVICE_PORT=$("${PYTHON_BIN}" -c "from embeddings.config import CONFIG; print(CONFIG.PORT)") | 30 | DEFAULT_EMBEDDING_SERVICE_PORT=$("${PYTHON_BIN}" -c "from embeddings.config import CONFIG; print(CONFIG.PORT)") |
scripts/start_frontend.sh
| @@ -15,13 +15,14 @@ echo -e "${GREEN}========================================${NC}" | @@ -15,13 +15,14 @@ echo -e "${GREEN}========================================${NC}" | ||
| 15 | echo -e "${GREEN}Starting Frontend Server${NC}" | 15 | echo -e "${GREEN}Starting Frontend Server${NC}" |
| 16 | echo -e "${GREEN}========================================${NC}" | 16 | echo -e "${GREEN}========================================${NC}" |
| 17 | 17 | ||
| 18 | -PORT=6003 | 18 | +FRONTEND_PORT="${FRONTEND_PORT:-6003}" |
| 19 | +API_PORT="${API_PORT:-6002}" | ||
| 19 | 20 | ||
| 20 | echo -e "\n${YELLOW}Frontend will be available at:${NC}" | 21 | echo -e "\n${YELLOW}Frontend will be available at:${NC}" |
| 21 | -echo -e " ${GREEN}http://localhost:$PORT${NC}" | 22 | +echo -e " ${GREEN}http://localhost:${FRONTEND_PORT}${NC}" |
| 22 | echo "" | 23 | echo "" |
| 23 | echo -e "${YELLOW}Make sure the backend API is running at:${NC}" | 24 | echo -e "${YELLOW}Make sure the backend API is running at:${NC}" |
| 24 | -echo -e " ${GREEN}http://localhost:6002${NC}" | 25 | +echo -e " ${GREEN}http://localhost:${API_PORT}${NC}" |
| 25 | echo "" | 26 | echo "" |
| 26 | 27 | ||
| 27 | python scripts/frontend_server.py | 28 | python scripts/frontend_server.py |
scripts/start_reranker.sh
| @@ -16,32 +16,10 @@ if [[ ! -x "${PYTHON_BIN}" ]]; then | @@ -16,32 +16,10 @@ if [[ ! -x "${PYTHON_BIN}" ]]; then | ||
| 16 | exit 1 | 16 | exit 1 |
| 17 | fi | 17 | fi |
| 18 | 18 | ||
| 19 | -# Load .env if present (without activating main venv) | ||
| 20 | -ENV_FILE="${PROJECT_ROOT}/.env" | ||
| 21 | -if [ -f "${ENV_FILE}" ]; then | ||
| 22 | - while IFS= read -r line || [ -n "${line}" ]; do | ||
| 23 | - line="${line%$'\r'}" | ||
| 24 | - [[ -z "${line//[[:space:]]/}" ]] && continue | ||
| 25 | - [[ "${line}" =~ ^[[:space:]]*# ]] && continue | ||
| 26 | - [[ "${line}" != *=* ]] && continue | ||
| 27 | - | ||
| 28 | - key="${line%%=*}" | ||
| 29 | - value="${line#*=}" | ||
| 30 | - key="${key#"${key%%[![:space:]]*}"}" | ||
| 31 | - key="${key%"${key##*[![:space:]]}"}" | ||
| 32 | - value="${value#"${value%%[![:space:]]*}"}" | ||
| 33 | - | ||
| 34 | - if [[ ${#value} -ge 2 ]]; then | ||
| 35 | - first="${value:0:1}" | ||
| 36 | - last="${value: -1}" | ||
| 37 | - if [[ ("${first}" == '"' && "${last}" == '"') || ("${first}" == "'" && "${last}" == "'") ]]; then | ||
| 38 | - value="${value:1:${#value}-2}" | ||
| 39 | - fi | ||
| 40 | - fi | ||
| 41 | - | ||
| 42 | - export "${key}=${value}" | ||
| 43 | - done < "${ENV_FILE}" | ||
| 44 | -fi | 19 | +# Load .env without activating main venv. |
| 20 | +# shellcheck source=scripts/lib/load_env.sh | ||
| 21 | +source "${PROJECT_ROOT}/scripts/lib/load_env.sh" | ||
| 22 | +load_env_file "${PROJECT_ROOT}/.env" | ||
| 45 | 23 | ||
| 46 | RERANKER_HOST="${RERANKER_HOST:-0.0.0.0}" | 24 | RERANKER_HOST="${RERANKER_HOST:-0.0.0.0}" |
| 47 | RERANKER_PORT="${RERANKER_PORT:-6007}" | 25 | RERANKER_PORT="${RERANKER_PORT:-6007}" |
scripts/start_tei_service.sh
| @@ -7,32 +7,10 @@ set -euo pipefail | @@ -7,32 +7,10 @@ set -euo pipefail | ||
| 7 | PROJECT_ROOT="$(cd "$(dirname "$0")/.." && pwd)" | 7 | PROJECT_ROOT="$(cd "$(dirname "$0")/.." && pwd)" |
| 8 | cd "${PROJECT_ROOT}" | 8 | cd "${PROJECT_ROOT}" |
| 9 | 9 | ||
| 10 | -# Load .env if present | ||
| 11 | -ENV_FILE="${PROJECT_ROOT}/.env" | ||
| 12 | -if [ -f "${ENV_FILE}" ]; then | ||
| 13 | - while IFS= read -r line || [ -n "${line}" ]; do | ||
| 14 | - line="${line%$'\r'}" | ||
| 15 | - [[ -z "${line//[[:space:]]/}" ]] && continue | ||
| 16 | - [[ "${line}" =~ ^[[:space:]]*# ]] && continue | ||
| 17 | - [[ "${line}" != *=* ]] && continue | ||
| 18 | - | ||
| 19 | - key="${line%%=*}" | ||
| 20 | - value="${line#*=}" | ||
| 21 | - key="${key#"${key%%[![:space:]]*}"}" | ||
| 22 | - key="${key%"${key##*[![:space:]]}"}" | ||
| 23 | - value="${value#"${value%%[![:space:]]*}"}" | ||
| 24 | - | ||
| 25 | - if [[ ${#value} -ge 2 ]]; then | ||
| 26 | - first="${value:0:1}" | ||
| 27 | - last="${value: -1}" | ||
| 28 | - if [[ ("${first}" == '"' && "${last}" == '"') || ("${first}" == "'" && "${last}" == "'") ]]; then | ||
| 29 | - value="${value:1:${#value}-2}" | ||
| 30 | - fi | ||
| 31 | - fi | ||
| 32 | - | ||
| 33 | - export "${key}=${value}" | ||
| 34 | - done < "${ENV_FILE}" | ||
| 35 | -fi | 10 | +# Load .env. |
| 11 | +# shellcheck source=scripts/lib/load_env.sh | ||
| 12 | +source "${PROJECT_ROOT}/scripts/lib/load_env.sh" | ||
| 13 | +load_env_file "${PROJECT_ROOT}/.env" | ||
| 36 | 14 | ||
| 37 | if ! command -v docker >/dev/null 2>&1; then | 15 | if ! command -v docker >/dev/null 2>&1; then |
| 38 | echo "ERROR: docker is required to run TEI service." >&2 | 16 | echo "ERROR: docker is required to run TEI service." >&2 |
scripts/stop_clip_service.sh deleted
| @@ -1,49 +0,0 @@ | @@ -1,49 +0,0 @@ | ||
| 1 | -#!/bin/bash | ||
| 2 | -# | ||
| 3 | -# Stop CLIP vector service (clip-as-service) started by start_clip_service.sh | ||
| 4 | -# | ||
| 5 | -set -e | ||
| 6 | - | ||
| 7 | -PROJECT_ROOT="$(cd "$(dirname "$0")/.." && pwd)" | ||
| 8 | -LOG_DIR="${PROJECT_ROOT}/logs" | ||
| 9 | -PID_FILE="${LOG_DIR}/clip_service.pid" | ||
| 10 | - | ||
| 11 | -echo "========================================" | ||
| 12 | -echo "Stopping CLIP vector service (clip-as-service)" | ||
| 13 | -echo "========================================" | ||
| 14 | - | ||
| 15 | -if [ ! -f "${PID_FILE}" ]; then | ||
| 16 | - echo "No PID file found at ${PID_FILE}." | ||
| 17 | - echo "clip-as-service may not be running (or was not started via start_clip_service.sh)." | ||
| 18 | - exit 0 | ||
| 19 | -fi | ||
| 20 | - | ||
| 21 | -PID="$(cat "${PID_FILE}")" | ||
| 22 | - | ||
| 23 | -if [ -z "${PID}" ]; then | ||
| 24 | - echo "PID file exists but is empty. Removing it." | ||
| 25 | - rm -f "${PID_FILE}" | ||
| 26 | - exit 0 | ||
| 27 | -fi | ||
| 28 | - | ||
| 29 | -if ps -p "${PID}" > /dev/null 2>&1; then | ||
| 30 | - echo "Sending SIGTERM to clip-as-service (PID ${PID})..." | ||
| 31 | - kill "${PID}" || true | ||
| 32 | - sleep 1 | ||
| 33 | - | ||
| 34 | - if ps -p "${PID}" > /dev/null 2>&1; then | ||
| 35 | - echo "Process still alive, sending SIGKILL..." | ||
| 36 | - kill -9 "${PID}" || true | ||
| 37 | - fi | ||
| 38 | - | ||
| 39 | - echo "clip-as-service (PID ${PID}) has been stopped." | ||
| 40 | -else | ||
| 41 | - echo "No process with PID ${PID} found. Assuming it's already stopped." | ||
| 42 | -fi | ||
| 43 | - | ||
| 44 | -rm -f "${PID_FILE}" | ||
| 45 | -echo "PID file removed: ${PID_FILE}" | ||
| 46 | - | ||
| 47 | - | ||
| 48 | - | ||
| 49 | - |
search/es_query_builder.py
| @@ -277,6 +277,17 @@ class ESQueryBuilder: | @@ -277,6 +277,17 @@ class ESQueryBuilder: | ||
| 277 | "num_candidates": knn_num_candidates, | 277 | "num_candidates": knn_num_candidates, |
| 278 | "boost": knn_boost | 278 | "boost": knn_boost |
| 279 | } | 279 | } |
| 280 | + # Top-level knn does not inherit query.bool.filter automatically. | ||
| 281 | + # Apply conjunctive + range filters here so vector recall respects hard filters. | ||
| 282 | + if filter_clauses: | ||
| 283 | + if len(filter_clauses) == 1: | ||
| 284 | + knn_clause["filter"] = filter_clauses[0] | ||
| 285 | + else: | ||
| 286 | + knn_clause["filter"] = { | ||
| 287 | + "bool": { | ||
| 288 | + "filter": filter_clauses | ||
| 289 | + } | ||
| 290 | + } | ||
| 280 | es_query["knn"] = knn_clause | 291 | es_query["knn"] = knn_clause |
| 281 | 292 | ||
| 282 | # 5. Add post_filter for disjunctive (multi-select) filters | 293 | # 5. Add post_filter for disjunctive (multi-select) filters |
search/searcher.py
| @@ -593,11 +593,14 @@ class Searcher: | @@ -593,11 +593,14 @@ class Searcher: | ||
| 593 | if filters or range_filters: | 593 | if filters or range_filters: |
| 594 | filter_clauses = self.query_builder._build_filters(filters, range_filters) | 594 | filter_clauses = self.query_builder._build_filters(filters, range_filters) |
| 595 | if filter_clauses: | 595 | if filter_clauses: |
| 596 | - es_query["query"] = { | ||
| 597 | - "bool": { | ||
| 598 | - "filter": filter_clauses | 596 | + if len(filter_clauses) == 1: |
| 597 | + es_query["knn"]["filter"] = filter_clauses[0] | ||
| 598 | + else: | ||
| 599 | + es_query["knn"]["filter"] = { | ||
| 600 | + "bool": { | ||
| 601 | + "filter": filter_clauses | ||
| 602 | + } | ||
| 599 | } | 603 | } |
| 600 | - } | ||
| 601 | 604 | ||
| 602 | # Execute search | 605 | # Execute search |
| 603 | es_response = self.es_client.search( | 606 | es_response = self.es_client.search( |
| @@ -0,0 +1,64 @@ | @@ -0,0 +1,64 @@ | ||
| 1 | +from types import SimpleNamespace | ||
| 2 | + | ||
| 3 | +import numpy as np | ||
| 4 | + | ||
| 5 | +from search.es_query_builder import ESQueryBuilder | ||
| 6 | + | ||
| 7 | + | ||
| 8 | +def _builder() -> ESQueryBuilder: | ||
| 9 | + return ESQueryBuilder( | ||
| 10 | + match_fields=["title.en^3.0", "brief.en^1.0"], | ||
| 11 | + text_embedding_field="title_embedding", | ||
| 12 | + default_language="en", | ||
| 13 | + ) | ||
| 14 | + | ||
| 15 | + | ||
| 16 | +def test_knn_prefilter_includes_range_filters(): | ||
| 17 | + qb = _builder() | ||
| 18 | + q = qb.build_query( | ||
| 19 | + query_text="bags", | ||
| 20 | + query_vector=np.array([0.1, 0.2, 0.3]), | ||
| 21 | + range_filters={"min_price": {"gte": 50, "lt": 100}}, | ||
| 22 | + enable_knn=True, | ||
| 23 | + ) | ||
| 24 | + | ||
| 25 | + assert "knn" in q | ||
| 26 | + assert q["knn"]["filter"] == {"range": {"min_price": {"gte": 50, "lt": 100}}} | ||
| 27 | + | ||
| 28 | + | ||
| 29 | +def test_knn_prefilter_uses_only_conjunctive_filters_when_disjunctive_present(): | ||
| 30 | + qb = _builder() | ||
| 31 | + facets = [SimpleNamespace(field="category_name", disjunctive=True)] | ||
| 32 | + q = qb.build_query( | ||
| 33 | + query_text="bags", | ||
| 34 | + query_vector=np.array([0.1, 0.2, 0.3]), | ||
| 35 | + filters={"category_name": ["A", "B"], "vendor": "Nike"}, | ||
| 36 | + range_filters={"min_price": {"gte": 50, "lt": 100}}, | ||
| 37 | + facet_configs=facets, | ||
| 38 | + enable_knn=True, | ||
| 39 | + ) | ||
| 40 | + | ||
| 41 | + assert "knn" in q | ||
| 42 | + assert "filter" in q["knn"] | ||
| 43 | + knn_filter = q["knn"]["filter"] | ||
| 44 | + assert knn_filter == { | ||
| 45 | + "bool": { | ||
| 46 | + "filter": [ | ||
| 47 | + {"term": {"vendor": "Nike"}}, | ||
| 48 | + {"range": {"min_price": {"gte": 50, "lt": 100}}}, | ||
| 49 | + ] | ||
| 50 | + } | ||
| 51 | + } | ||
| 52 | + assert q["post_filter"] == {"terms": {"category_name": ["A", "B"]}} | ||
| 53 | + | ||
| 54 | + | ||
| 55 | +def test_knn_prefilter_not_added_without_filters(): | ||
| 56 | + qb = _builder() | ||
| 57 | + q = qb.build_query( | ||
| 58 | + query_text="bags", | ||
| 59 | + query_vector=np.array([0.1, 0.2, 0.3]), | ||
| 60 | + enable_knn=True, | ||
| 61 | + ) | ||
| 62 | + | ||
| 63 | + assert "knn" in q | ||
| 64 | + assert "filter" not in q["knn"] |