Commit 7b8d9e1aab3d53cd620035bd600c85b9f29e7524
1 parent
f8e7cb97
评估框架的启动脚本
Showing
11 changed files
with
70 additions
and
18 deletions
Show diff stats
.env.example
| ... | ... | @@ -27,6 +27,9 @@ EMBEDDING_HOST=0.0.0.0 |
| 27 | 27 | |
| 28 | 28 | # Optional service ports |
| 29 | 29 | FRONTEND_PORT=6003 |
| 30 | +# Search evaluation web UI (scripts/evaluation, service name: eval-web) | |
| 31 | +EVAL_WEB_HOST=0.0.0.0 | |
| 32 | +EVAL_WEB_PORT=6010 | |
| 30 | 33 | EMBEDDING_PORT=6005 |
| 31 | 34 | TEI_PORT=8080 |
| 32 | 35 | CNCLIP_PORT=51000 | ... | ... |
CLAUDE.md
| ... | ... | @@ -54,6 +54,7 @@ password: P89cZHS5d7dFyc9R |
| 54 | 54 | **Service Endpoints:** |
| 55 | 55 | - **Backend API**: http://localhost:6002 (FastAPI) |
| 56 | 56 | - **Frontend UI**: http://localhost:6003 (Debug interface) |
| 57 | +- **Search evaluation UI**: http://localhost:6010 (`eval-web`, `scripts/evaluation/`; start via `./scripts/service_ctl.sh start eval-web` or included in `./run.sh all`) | |
| 57 | 58 | - **Elasticsearch**: http://localhost:9200 |
| 58 | 59 | - **API Documentation**: http://localhost:6002/docs |
| 59 | 60 | ... | ... |
README.md
| ... | ... | @@ -46,6 +46,7 @@ source activate.sh |
| 46 | 46 | - `6002` backend（`/search/*`, `/admin/*`） |
| 47 | 47 | - `6004` indexer（`/indexer/*`） |
| 48 | 48 | - `6003` frontend |
| 49 | +- `6010` eval-web（搜索评估 UI，`./scripts/service_ctl.sh` 服务名 `eval-web`） | |
| 49 | 50 | - `6005` embedding（可选） |
| 50 | 51 | - `6006` translator（可选） |
| 51 | 52 | - `6007` reranker（可选） | ... | ... |
docs/DEVELOPER_GUIDE.md
| ... | ... | @@ -92,13 +92,14 @@ MySQL (店匠 SPU/SKU) |
| 92 | 92 | | backend | 6002 | 搜索 API（含 admin） | ✓ | |
| 93 | 93 | | indexer | 6004 | 索引 API（reindex/build-docs 等） | ✓ | |
| 94 | 94 | | frontend | 6003 | 调试 UI | ✓ | |
| 95 | +| eval-web | 6010 | 搜索评估 Web（`scripts/evaluation/`） | ✓ | | |
| 95 | 96 | | embedding | 6005 | 文本向量服务 | 可选 | |
| 96 | 97 | | embedding-image | 6008 | 图片向量服务 | 可选 | |
| 97 | 98 | | translator | 6006 | 翻译服务（`POST /translate` 支持单条或批量 list；批量失败用 `null` 占位） | 可选 | |
| 98 | 99 | | reranker | 6007 | 重排服务 | 可选 | |
| 99 | 100 | |
| 100 | -- 启动：`./run.sh` 仅启动 backend / indexer / frontend；需全功能时通过环境变量或脚本另行启动 embedding / translator / reranker。 | |
| 101 | -- 停止：统一使用 `./scripts/stop.sh`（会停止上述所有端口上的进程）。 | |
| 101 | +- 启动：`./run.sh all` 会按 `scripts/service_ctl.sh` 定义拉起可选依赖（tei、cnclip、embedding 等）与核心服务（backend、indexer、frontend、**eval-web**）；仅需核心时可 `./run.sh backend indexer frontend`（不启 eval-web）。 | |
| 102 | +- 停止：统一使用 `./scripts/stop.sh all`（或 `service_ctl.sh down`），会停止当前目标集合内各服务占用的端口与 monitor。 | |
| 102 | 103 | - 详见 [QUICKSTART.md](./QUICKSTART.md) 与 [Usage-Guide.md](./Usage-Guide.md)。 |
| 103 | 104 | |
| 104 | 105 | ### 3.3 仓库目录结构（与架构对应） | ... | ... |
docs/QUICKSTART.md
| ... | ... | @@ -57,6 +57,7 @@ source activate.sh |
| 57 | 57 | | backend | 6002 | ✓ | 搜索 API（`/search/*`）+ 管理接口（`/admin/*`） | |
| 58 | 58 | | indexer | 6004 | ✓ | 索引 API（`/indexer/*`） | |
| 59 | 59 | | frontend | 6003 | ✓ | 调试 UI | |
| 60 | +| eval-web | 6010 | ✓ | 搜索评估 Web（`scripts/evaluation/`，依赖 backend 联调） | | |
| 60 | 61 | | embedding | 6005 | - | 文本向量服务（`/embed/text`） | |
| 61 | 62 | | embedding-image | 6008 | - | 图片向量服务（`/embed/image`） | |
| 62 | 63 | | translator | 6006 | - | 翻译服务（`/translate`） | |
| ... | ... | @@ -68,7 +69,7 @@ source activate.sh |
| 68 | 69 | ./run.sh all |
| 69 | 70 | # 仅为薄封装：等价于 ./scripts/service_ctl.sh up all |
| 70 | 71 | # 说明： |
| 71 | -# - all = tei cnclip embedding translator reranker backend indexer frontend | |
| 72 | +# - all = tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web | |
| 72 | 73 | # - up 会同时启动 monitor daemon（运行期连续失败自动重启） |
| 73 | 74 | # - reranker 为 GPU 强制模式（资源不足会直接启动失败） |
| 74 | 75 | # - TEI 默认使用 GPU；当 TEI_DEVICE=cuda 且 GPU 不可用时会直接失败（不会自动降级到 CPU） |
| ... | ... | @@ -301,6 +302,7 @@ DEEPL_AUTH_KEY=your-key |
| 301 | 302 | # API |
| 302 | 303 | API_HOST=0.0.0.0 |
| 303 | 304 | API_PORT=6002 |
| 305 | +EVAL_WEB_PORT=6010 | |
| 304 | 306 | ``` |
| 305 | 307 | |
| 306 | 308 | > 生产环境请妥善保管凭证；本地/测试可改用上述值或自建实例。 |
| ... | ... | @@ -536,6 +538,8 @@ curl http://localhost:6005/health |
| 536 | 538 | curl http://localhost:6008/health |
| 537 | 539 | curl http://localhost:6006/health |
| 538 | 540 | curl http://localhost:6007/health |
| 541 | +# eval-web（若已启动）：根路径可访问即正常 | |
| 542 | +curl -sf -o /dev/null http://localhost:6010/ || true | |
| 539 | 543 | ``` |
| 540 | 544 | |
| 541 | 545 | ### 5.2 常看日志 |
| ... | ... | @@ -554,6 +558,7 @@ curl http://localhost:6007/health |
| 554 | 558 | curl http://localhost:9200 |
| 555 | 559 | lsof -i :6002 |
| 556 | 560 | lsof -i :6004 |
| 561 | +lsof -i :6010 | |
| 557 | 562 | ``` |
| 558 | 563 | |
| 559 | 564 | 更完整的运行排障（多环境切换、Suggestion 构建、FAQ）见 `docs/Usage-Guide.md`。 | ... | ... |
docs/Usage-Guide.md
| ... | ... | @@ -126,13 +126,14 @@ cd /data/saas-search |
| 126 | 126 | |
| 127 | 127 | 这个脚本会自动： |
| 128 | 128 | 1. 创建日志目录 |
| 129 | -2. 按目标启动服务（`all`：`tei cnclip embedding translator reranker backend indexer frontend`） | |
| 129 | +2. 按目标启动服务（`all`：`tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web`） | |
| 130 | 130 | 3. 写入 PID 到 `logs/*.pid` |
| 131 | 131 | 4. 执行健康检查 |
| 132 | 132 | 5. 启动 monitor daemon（运行期连续失败自动重启） |
| 133 | 133 | |
| 134 | 134 | 启动完成后，访问： |
| 135 | 135 | - **前端界面**: http://localhost:6003 |
| 136 | +- **搜索评估 UI**: http://localhost:6010（`eval-web`，依赖 backend 提供实时搜索） | |
| 136 | 137 | - **后端API**: http://localhost:6002 |
| 137 | 138 | - **API文档**: http://localhost:6002/docs |
| 138 | 139 | - **索引API**: http://localhost:6004/docs |
| ... | ... | @@ -335,6 +336,7 @@ python -m http.server 6003 |
| 335 | 336 | | Backend API | 6002 | http://localhost:6002 | |
| 336 | 337 | | Indexer API | 6004 | http://localhost:6004 | |
| 337 | 338 | | Frontend Web | 6003 | http://localhost:6003 | |
| 339 | +| Search eval web (`eval-web`) | 6010 | http://localhost:6010 | | |
| 338 | 340 | | Embedding Text (optional) | 6005 | http://localhost:6005 | |
| 339 | 341 | | Embedding Image (optional) | 6008 | http://localhost:6008 | |
| 340 | 342 | | TEI (optional) | 8080 | http://localhost:8080 | |
| ... | ... | @@ -381,6 +383,7 @@ INDEXER_PORT=6004 |
| 381 | 383 | |
| 382 | 384 | # Optional service ports |
| 383 | 385 | FRONTEND_PORT=6003 |
| 386 | +EVAL_WEB_PORT=6010 | |
| 384 | 387 | EMBEDDING_TEXT_PORT=6005 |
| 385 | 388 | EMBEDDING_IMAGE_PORT=6008 |
| 386 | 389 | TEI_PORT=8080 |
| ... | ... | @@ -691,6 +694,7 @@ cat .env | grep ES_ |
| 691 | 694 | # 查看占用端口的进程 |
| 692 | 695 | lsof -i :6002 # 后端 |
| 693 | 696 | lsof -i :6003 # 前端 |
| 697 | +lsof -i :6010 # 搜索评估 UI（eval-web） | |
| 694 | 698 | lsof -i :9200 # ES |
| 695 | 699 | |
| 696 | 700 | # 杀掉进程 | ... | ... |
docs/工作总结-微服务性能优化与架构.md
| ... | ... | @@ -125,11 +125,11 @@ instruction: "Given a shopping query, rank product titles by relevance" |
| 125 | 125 | |
| 126 | 126 | - **脚本**：`scripts/service_ctl.sh` 统一负责各服务的生命周期与监控；依赖 `scripts/lib/load_env.sh` 与项目根目录 `.env`。 |
| 127 | 127 | - **服务与端口**： |
| 128 | - - 核心：backend **6002**、indexer **6004**、frontend **6003**。 | |
| 128 | + - 核心：backend **6002**、indexer **6004**、frontend **6003**、eval-web **6010**（搜索评估 UI）。 | |
| 129 | 129 | - 可选：embedding(text) **6005**、embedding-image **6008**、translator **6006**、reranker **6007**、tei **8080**、cnclip **51000**。 |
| 130 | - - 端口可由环境变量覆盖：`API_PORT`、`INDEXER_PORT`、`FRONTEND_PORT`、`EMBEDDING_TEXT_PORT`、`EMBEDDING_IMAGE_PORT`、`TRANSLATION_PORT`、`RERANKER_PORT`、`TEI_PORT`、`CNCLIP_PORT`。 | |
| 130 | + - 端口可由环境变量覆盖：`API_PORT`、`INDEXER_PORT`、`FRONTEND_PORT`、`EVAL_WEB_PORT`、`EMBEDDING_TEXT_PORT`、`EMBEDDING_IMAGE_PORT`、`TRANSLATION_PORT`、`RERANKER_PORT`、`TEI_PORT`、`CNCLIP_PORT`。 | |
| 131 | 131 | - **命令**： |
| 132 | - - `./scripts/service_ctl.sh start [service...]` 或 `start all`（all = tei cnclip embedding translator reranker backend indexer frontend，按依赖顺序）；`stop`、`restart` 同参数；`status` 默认列出所有服务。 | |
| 132 | + - `./scripts/service_ctl.sh start [service...]` 或 `up all` / `start all`（all 含 tei、cnclip、embedding、embedding-image、translator、reranker、reranker-fine、backend、indexer、frontend、eval-web，按依赖顺序）；`stop`、`restart`、`down` 同参数；`status` 默认列出所有服务。 | |
| 133 | 133 | - 启动时：backend/indexer/frontend/embedding/translator/reranker 会写 pid 到 `logs/<service>.pid`，并执行 `wait_for_health`（GET `http://127.0.0.1:<port>/health`）；reranker 健康重试 90 次，其余 30 次；TEI 校验 Docker 容器存在且 `/health` 成功；cnclip 无 HTTP 健康则仅校验进程/端口。 |
| 134 | 134 | - **监控常驻**： |
| 135 | 135 | - `./scripts/service_ctl.sh monitor-start <targets>` 启动后台监控进程，将 targets 写入 `logs/service-monitor.targets`，pid 写入 `logs/service-monitor.pid`，日志追加到 `logs/service-monitor.log`。 | ... | ... |
scripts/evaluation/README.md
| ... | ... | @@ -37,6 +37,8 @@ The framework supports four related tasks: |
| 37 | 37 | Requirement reference document. |
| 38 | 38 | - `quick_start_eval.sh` |
| 39 | 39 | Optional wrapper: `batch` (fill missing labels only), `batch-rebuild` (force full re-label), or `serve` (web UI). |
| 40 | +- `../start_eval_web.sh` | |
| 41 | + Same as `serve` but loads `activate.sh`; use with `./scripts/service_ctl.sh start eval-web` (port **`EVAL_WEB_PORT`**, default **6010**). `./run.sh all` starts **eval-web** with the rest of core services. | |
| 40 | 42 | |
| 41 | 43 | ## Quick start (from repo root) |
| 42 | 44 | |
| ... | ... | @@ -49,8 +51,9 @@ Set tenant if needed (`export TENANT_ID=163`). Requires live search API, DashSco |
| 49 | 51 | # Optional: full re-label of current top_k recall (expensive; use only when you intentionally rebuild the cache) |
| 50 | 52 | ./scripts/evaluation/quick_start_eval.sh batch-rebuild |
| 51 | 53 | |
| 52 | -# 2) Evaluation UI on http://127.0.0.1:6010/ | |
| 54 | +# 2) Evaluation UI on http://127.0.0.1:6010/ (override with EVAL_WEB_PORT / EVAL_WEB_HOST) | |
| 53 | 55 | ./scripts/evaluation/quick_start_eval.sh serve |
| 56 | +# Or: ./scripts/service_ctl.sh start eval-web | |
| 54 | 57 | ``` |
| 55 | 58 | |
| 56 | 59 | Equivalent explicit commands: | ... | ... |
scripts/evaluation/quick_start_eval.sh
| ... | ... | @@ -12,8 +12,8 @@ usage() { |
| 12 | 12 | echo "Usage: $0 batch|batch-rebuild|serve" |
| 13 | 13 | echo " batch — batch eval: live search every query, LLM only for missing labels (top_k=50, simple)" |
| 14 | 14 | echo " batch-rebuild — same as batch but --force-refresh-labels (re-LLM all top_k hits; expensive, overwrites cache)" |
| 15 | - echo " serve — eval UI on http://0.0.0.0:6010/" | |
| 16 | - echo "Env: TENANT_ID (default 163), REPO_EVAL_QUERIES (path to queries file)" | |
| 15 | + echo " serve — eval UI (default http://0.0.0.0:\${EVAL_WEB_PORT:-6010}/; also: ./scripts/start_eval_web.sh)" | |
| 16 | + echo "Env: TENANT_ID (default 163), REPO_EVAL_QUERIES, EVAL_WEB_HOST, EVAL_WEB_PORT (default 6010)" | |
| 17 | 17 | } |
| 18 | 18 | |
| 19 | 19 | case "${1:-}" in |
| ... | ... | @@ -35,11 +35,13 @@ case "${1:-}" in |
| 35 | 35 | --force-refresh-labels |
| 36 | 36 | ;; |
| 37 | 37 | serve) |
| 38 | + EVAL_WEB_PORT="${EVAL_WEB_PORT:-6010}" | |
| 39 | + EVAL_WEB_HOST="${EVAL_WEB_HOST:-0.0.0.0}" | |
| 38 | 40 | exec "$PY" scripts/evaluation/serve_eval_web.py serve \ |
| 39 | 41 | --tenant-id "$TENANT_ID" \ |
| 40 | 42 | --queries-file "$QUERIES" \ |
| 41 | - --host 0.0.0.0 \ | |
| 42 | - --port 6010 | |
| 43 | + --host "$EVAL_WEB_HOST" \ | |
| 44 | + --port "$EVAL_WEB_PORT" | |
| 43 | 45 | ;; |
| 44 | 46 | *) |
| 45 | 47 | usage | ... | ... |
scripts/service_ctl.sh
| ... | ... | @@ -15,10 +15,10 @@ mkdir -p "${LOG_DIR}" |
| 15 | 15 | # shellcheck source=scripts/lib/load_env.sh |
| 16 | 16 | source "${PROJECT_ROOT}/scripts/lib/load_env.sh" |
| 17 | 17 | |
| 18 | -CORE_SERVICES=("backend" "indexer" "frontend") | |
| 18 | +CORE_SERVICES=("backend" "indexer" "frontend" "eval-web") | |
| 19 | 19 | OPTIONAL_SERVICES=("tei" "cnclip" "embedding" "embedding-image" "translator" "reranker" "reranker-fine") |
| 20 | 20 | FULL_SERVICES=("${OPTIONAL_SERVICES[@]}" "${CORE_SERVICES[@]}") |
| 21 | -STOP_ORDER_SERVICES=("frontend" "indexer" "backend" "reranker-fine" "reranker" "translator" "embedding-image" "embedding" "cnclip" "tei") | |
| 21 | +STOP_ORDER_SERVICES=("frontend" "eval-web" "indexer" "backend" "reranker-fine" "reranker" "translator" "embedding-image" "embedding" "cnclip" "tei") | |
| 22 | 22 | |
| 23 | 23 | all_services() { |
| 24 | 24 | echo "${FULL_SERVICES[@]}" |
| ... | ... | @@ -80,6 +80,7 @@ get_port() { |
| 80 | 80 | ;; |
| 81 | 81 | tei) echo "${TEI_PORT:-8080}" ;; |
| 82 | 82 | cnclip) echo "${CNCLIP_PORT:-51000}" ;; |
| 83 | + eval-web) echo "${EVAL_WEB_PORT:-6010}" ;; | |
| 83 | 84 | *) echo "" ;; |
| 84 | 85 | esac |
| 85 | 86 | } |
| ... | ... | @@ -117,6 +118,7 @@ service_start_cmd() { |
| 117 | 118 | reranker-fine) echo "./scripts/start_reranker.sh" ;; |
| 118 | 119 | tei) echo "./scripts/start_tei_service.sh" ;; |
| 119 | 120 | cnclip) echo "./scripts/start_cnclip_service.sh" ;; |
| 121 | + eval-web) echo "./scripts/start_eval_web.sh" ;; | |
| 120 | 122 | *) return 1 ;; |
| 121 | 123 | esac |
| 122 | 124 | } |
| ... | ... | @@ -124,7 +126,7 @@ service_start_cmd() { |
| 124 | 126 | service_exists() { |
| 125 | 127 | local service="$1" |
| 126 | 128 | case "${service}" in |
| 127 | - backend|indexer|frontend|embedding|embedding-image|translator|reranker|reranker-fine|tei|cnclip) return 0 ;; | |
| 129 | + backend|indexer|frontend|eval-web|embedding|embedding-image|translator|reranker|reranker-fine|tei|cnclip) return 0 ;; | |
| 128 | 130 | *) return 1 ;; |
| 129 | 131 | esac |
| 130 | 132 | } |
| ... | ... | @@ -638,7 +640,7 @@ start_one() { |
| 638 | 640 | return 1 |
| 639 | 641 | fi |
| 640 | 642 | ;; |
| 641 | - backend|indexer|frontend|embedding|embedding-image|translator|reranker|reranker-fine) | |
| 643 | + backend|indexer|frontend|eval-web|embedding|embedding-image|translator|reranker|reranker-fine) | |
| 642 | 644 | echo "[start] ${service}" |
| 643 | 645 | local rerank_instance="" |
| 644 | 646 | rerank_instance="$(reranker_instance_for_service "${service}")" |
| ... | ... | @@ -922,8 +924,8 @@ Special targets: |
| 922 | 924 | |
| 923 | 925 | Examples: |
| 924 | 926 | ./scripts/service_ctl.sh up all |
| 925 | - ./scripts/service_ctl.sh up tei cnclip embedding embedding-image translator reranker reranker-fine | |
| 926 | - ./scripts/service_ctl.sh up backend indexer frontend | |
| 927 | + ./scripts/service_ctl.sh up tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web | |
| 928 | + ./scripts/service_ctl.sh up backend indexer frontend eval-web | |
| 927 | 929 | ./scripts/service_ctl.sh restart |
| 928 | 930 | ./scripts/service_ctl.sh monitor-start all |
| 929 | 931 | ./scripts/service_ctl.sh monitor-status | ... | ... |
| ... | ... | @@ -0,0 +1,30 @@ |
| 1 | +#!/bin/bash | |
| 2 | +# Search evaluation web UI (FastAPI). Managed by scripts/service_ctl.sh as service "eval-web". | |
| 3 | + | |
| 4 | +set -euo pipefail | |
| 5 | + | |
| 6 | +cd "$(dirname "$0")/.." | |
| 7 | +source ./activate.sh | |
| 8 | + | |
| 9 | +EVAL_WEB_PORT="${EVAL_WEB_PORT:-6010}" | |
| 10 | +EVAL_WEB_HOST="${EVAL_WEB_HOST:-0.0.0.0}" | |
| 11 | +TENANT_ID="${TENANT_ID:-163}" | |
| 12 | +QUERIES="${REPO_EVAL_QUERIES:-scripts/evaluation/queries/queries.txt}" | |
| 13 | + | |
| 14 | +GREEN='\033[0;32m' | |
| 15 | +YELLOW='\033[1;33m' | |
| 16 | +NC='\033[0m' | |
| 17 | + | |
| 18 | +echo -e "${GREEN}========================================${NC}" | |
| 19 | +echo -e "${GREEN}Starting Search Evaluation Web${NC}" | |
| 20 | +echo -e "${GREEN}========================================${NC}" | |
| 21 | +echo -e "\n${YELLOW}Evaluation UI:${NC} ${GREEN}http://localhost:${EVAL_WEB_PORT}/${NC}" | |
| 22 | +echo -e "${YELLOW}Requires backend for live search:${NC} ${GREEN}http://localhost:${API_PORT:-6002}${NC}\n" | |
| 23 | + | |
| 24 | +export EVAL_WEB_PORT EVAL_WEB_HOST TENANT_ID REPO_EVAL_QUERIES | |
| 25 | + | |
| 26 | +exec python scripts/evaluation/serve_eval_web.py serve \ | |
| 27 | + --tenant-id "${TENANT_ID}" \ | |
| 28 | + --queries-file "${QUERIES}" \ | |
| 29 | + --host "${EVAL_WEB_HOST}" \ | |
| 30 | + --port "${EVAL_WEB_PORT}" | ... | ... |