评估框架的启动脚本

tangwang
1 parent f8e7cb97
Showing 11 changed files with 70 additions and 18 deletions Show diff stats
.env.example
CLAUDE.md
README.md
docs/DEVELOPER_GUIDE.md
docs/QUICKSTART.md
docs/Usage-Guide.md
docs/工作总结-微服务性能优化与架构.md
scripts/evaluation/README.md
scripts/evaluation/quick_start_eval.sh
scripts/service_ctl.sh
scripts/start_eval_web.sh
@@ -27,6 +27,9 @@ EMBEDDING_HOST=0.0.0.0
 # Optional service ports
 FRONTEND_PORT=6003
+# Search evaluation web UI (scripts/evaluation, service name: eval-web)
+EVAL_WEB_HOST=0.0.0.0
+EVAL_WEB_PORT=6010
 EMBEDDING_PORT=6005
 TEI_PORT=8080
 CNCLIP_PORT=51000
@@ -54,6 +54,7 @@ password: P89cZHS5d7dFyc9R
 **Service Endpoints:**
 - **Backend API**: http://localhost:6002 (FastAPI)
 - **Frontend UI**: http://localhost:6003 (Debug interface)
+- **Search evaluation UI**: http://localhost:6010 (`eval-web`, `scripts/evaluation/`; start via `./scripts/service_ctl.sh start eval-web` or included in `./run.sh all`)
 - **Elasticsearch**: http://localhost:9200
 - **API Documentation**: http://localhost:6002/docs
@@ -46,6 +46,7 @@ source activate.sh
 - `6002` backend（`/search/*`, `/admin/*`）
 - `6004` indexer（`/indexer/*`）
 - `6003` frontend
+- `6010` eval-web（搜索评估 UI，`./scripts/service_ctl.sh` 服务名 `eval-web`）
 - `6005` embedding（可选）
 - `6006` translator（可选）
 - `6007` reranker（可选）
@@ -92,13 +92,14 @@ MySQL (店匠 SPU/SKU)
 | backend | 6002 | 搜索 API（含 admin） | ✓ |
 | indexer | 6004 | 索引 API（reindex/build-docs 等） | ✓ |
 | frontend | 6003 | 调试 UI | ✓ |
+| eval-web | 6010 | 搜索评估 Web（`scripts/evaluation/`） | ✓ |
 | embedding | 6005 | 文本向量服务 | 可选 |
 | embedding-image | 6008 | 图片向量服务 | 可选 |
 | translator | 6006 | 翻译服务（`POST /translate` 支持单条或批量 list；批量失败用 `null` 占位） | 可选 |
 | reranker | 6007 | 重排服务 | 可选 |
-- 启动：`./run.sh` 仅启动 backend / indexer / frontend；需全功能时通过环境变量或脚本另行启动 embedding / translator / reranker。
-- 停止：统一使用 `./scripts/stop.sh`（会停止上述所有端口上的进程）。
+- 启动：`./run.sh all` 会按 `scripts/service_ctl.sh` 定义拉起可选依赖（tei、cnclip、embedding 等）与核心服务（backend、indexer、frontend、**eval-web**）；仅需核心时可 `./run.sh backend indexer frontend`（不启 eval-web）。
+- 停止：统一使用 `./scripts/stop.sh all`（或 `service_ctl.sh down`），会停止当前目标集合内各服务占用的端口与 monitor。
 - 详见 [QUICKSTART.md](./QUICKSTART.md) 与 [Usage-Guide.md](./Usage-Guide.md)。
 ### 3.3 仓库目录结构（与架构对应）
@@ -57,6 +57,7 @@ source activate.sh
 | backend | 6002 | ✓ | 搜索 API（`/search/*`）+ 管理接口（`/admin/*`） |
 | indexer | 6004 | ✓ | 索引 API（`/indexer/*`） |
 | frontend | 6003 | ✓ | 调试 UI |
+| eval-web | 6010 | ✓ | 搜索评估 Web（`scripts/evaluation/`，依赖 backend 联调） |
 | embedding | 6005 | - | 文本向量服务（`/embed/text`） |
 | embedding-image | 6008 | - | 图片向量服务（`/embed/image`） |
 | translator | 6006 | - | 翻译服务（`/translate`） |
@@ -68,7 +69,7 @@ source activate.sh
 ./run.sh all
 # 仅为薄封装：等价于 ./scripts/service_ctl.sh up all
 # 说明：
-# - all = tei cnclip embedding translator reranker backend indexer frontend
+# - all = tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web
 # - up 会同时启动 monitor daemon（运行期连续失败自动重启）
 # - reranker 为 GPU 强制模式（资源不足会直接启动失败）
 # - TEI 默认使用 GPU；当 TEI_DEVICE=cuda 且 GPU 不可用时会直接失败（不会自动降级到 CPU）
@@ -301,6 +302,7 @@ DEEPL_AUTH_KEY=your-key
 # API
 API_HOST=0.0.0.0
 API_PORT=6002
+EVAL_WEB_PORT=6010
 ```
 > 生产环境请妥善保管凭证；本地/测试可改用上述值或自建实例。
@@ -536,6 +538,8 @@ curl http://localhost:6005/health
 curl http://localhost:6008/health
 curl http://localhost:6006/health
 curl http://localhost:6007/health
+# eval-web（若已启动）：根路径可访问即正常
+curl -sf -o /dev/null http://localhost:6010/ || true
 ```
 ### 5.2 常看日志
@@ -554,6 +558,7 @@ curl http://localhost:6007/health
 curl http://localhost:9200
 lsof -i :6002
 lsof -i :6004
+lsof -i :6010
 ```
 更完整的运行排障（多环境切换、Suggestion 构建、FAQ）见 `docs/Usage-Guide.md`。
@@ -126,13 +126,14 @@ cd /data/saas-search
 这个脚本会自动：
 1. 创建日志目录
-2. 按目标启动服务（`all`：`tei cnclip embedding translator reranker backend indexer frontend`）
+2. 按目标启动服务（`all`：`tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web`）
 3. 写入 PID 到 `logs/*.pid`
 4. 执行健康检查
 5. 启动 monitor daemon（运行期连续失败自动重启）
 启动完成后，访问：
 - **前端界面**: http://localhost:6003
+- **搜索评估 UI**: http://localhost:6010（`eval-web`，依赖 backend 提供实时搜索）
 - **后端API**: http://localhost:6002
 - **API文档**: http://localhost:6002/docs
 - **索引API**: http://localhost:6004/docs
@@ -335,6 +336,7 @@ python -m http.server 6003
 | Backend API | 6002 | http://localhost:6002 |
 | Indexer API | 6004 | http://localhost:6004 |
 | Frontend Web | 6003 | http://localhost:6003 |
+| Search eval web (`eval-web`) | 6010 | http://localhost:6010 |
 | Embedding Text (optional) | 6005 | http://localhost:6005 |
 | Embedding Image (optional) | 6008 | http://localhost:6008 |
 | TEI (optional) | 8080 | http://localhost:8080 |
@@ -381,6 +383,7 @@ INDEXER_PORT=6004
 # Optional service ports
 FRONTEND_PORT=6003
+EVAL_WEB_PORT=6010
 EMBEDDING_TEXT_PORT=6005
 EMBEDDING_IMAGE_PORT=6008
 TEI_PORT=8080
@@ -691,6 +694,7 @@ cat .env | grep ES_
 # 查看占用端口的进程
 lsof -i :6002  # 后端
 lsof -i :6003  # 前端
+lsof -i :6010  # 搜索评估 UI（eval-web）
 lsof -i :9200  # ES
 # 杀掉进程
@@ -125,11 +125,11 @@ instruction: &quot;Given a shopping query, rank product titles by relevance&quot;
 - **脚本**：`scripts/service_ctl.sh` 统一负责各服务的生命周期与监控；依赖 `scripts/lib/load_env.sh` 与项目根目录 `.env`。
 - **服务与端口**：
-  - 核心：backend **6002**、indexer **6004**、frontend **6003**。
+  - 核心：backend **6002**、indexer **6004**、frontend **6003**、eval-web **6010**（搜索评估 UI）。
   - 可选：embedding(text) **6005**、embedding-image **6008**、translator **6006**、reranker **6007**、tei **8080**、cnclip **51000**。
-  - 端口可由环境变量覆盖：`API_PORT`、`INDEXER_PORT`、`FRONTEND_PORT`、`EMBEDDING_TEXT_PORT`、`EMBEDDING_IMAGE_PORT`、`TRANSLATION_PORT`、`RERANKER_PORT`、`TEI_PORT`、`CNCLIP_PORT`。
+  - 端口可由环境变量覆盖：`API_PORT`、`INDEXER_PORT`、`FRONTEND_PORT`、`EVAL_WEB_PORT`、`EMBEDDING_TEXT_PORT`、`EMBEDDING_IMAGE_PORT`、`TRANSLATION_PORT`、`RERANKER_PORT`、`TEI_PORT`、`CNCLIP_PORT`。
 - **命令**：
-  - `./scripts/service_ctl.sh start [service...]` 或 `start all`（all = tei cnclip embedding translator reranker backend indexer frontend，按依赖顺序）；`stop`、`restart` 同参数；`status` 默认列出所有服务。
+  - `./scripts/service_ctl.sh start [service...]` 或 `up all` / `start all`（all 含 tei、cnclip、embedding、embedding-image、translator、reranker、reranker-fine、backend、indexer、frontend、eval-web，按依赖顺序）；`stop`、`restart`、`down` 同参数；`status` 默认列出所有服务。
   - 启动时：backend/indexer/frontend/embedding/translator/reranker 会写 pid 到 `logs/<service>.pid`，并执行 `wait_for_health`（GET `http://127.0.0.1:<port>/health`）；reranker 健康重试 90 次，其余 30 次；TEI 校验 Docker 容器存在且 `/health` 成功；cnclip 无 HTTP 健康则仅校验进程/端口。
 - **监控常驻**：
   - `./scripts/service_ctl.sh monitor-start <targets>` 启动后台监控进程，将 targets 写入 `logs/service-monitor.targets`，pid 写入 `logs/service-monitor.pid`，日志追加到 `logs/service-monitor.log`。
@@ -37,6 +37,8 @@ The framework supports four related tasks:
   Requirement reference document.
 - `quick_start_eval.sh`
   Optional wrapper: `batch` (fill missing labels only), `batch-rebuild` (force full re-label), or `serve` (web UI).
+- `../start_eval_web.sh`
+  Same as `serve` but loads `activate.sh`; use with `./scripts/service_ctl.sh start eval-web` (port **`EVAL_WEB_PORT`**, default **6010**). `./run.sh all` starts **eval-web** with the rest of core services.
 ## Quick start (from repo root)
@@ -49,8 +51,9 @@ Set tenant if needed (`export TENANT_ID=163`). Requires live search API, DashSco
 # Optional: full re-label of current top_k recall (expensive; use only when you intentionally rebuild the cache)
 ./scripts/evaluation/quick_start_eval.sh batch-rebuild
-# 2) Evaluation UI on http://127.0.0.1:6010/
+# 2) Evaluation UI on http://127.0.0.1:6010/ (override with EVAL_WEB_PORT / EVAL_WEB_HOST)
 ./scripts/evaluation/quick_start_eval.sh serve
+# Or: ./scripts/service_ctl.sh start eval-web
 ```
 Equivalent explicit commands:
@@ -12,8 +12,8 @@ usage() {
   echo "Usage: $0 batch|batch-rebuild|serve"
   echo "  batch          — batch eval: live search every query, LLM only for missing labels (top_k=50, simple)"
   echo "  batch-rebuild  — same as batch but --force-refresh-labels (re-LLM all top_k hits; expensive, overwrites cache)"
-  echo "  serve          — eval UI on http://0.0.0.0:6010/"
-  echo "Env: TENANT_ID (default 163), REPO_EVAL_QUERIES (path to queries file)"
+  echo "  serve          — eval UI (default http://0.0.0.0:\${EVAL_WEB_PORT:-6010}/; also: ./scripts/start_eval_web.sh)"
+  echo "Env: TENANT_ID (default 163), REPO_EVAL_QUERIES, EVAL_WEB_HOST, EVAL_WEB_PORT (default 6010)"
 }
 case "${1:-}" in
@@ -35,11 +35,13 @@ case &quot;${1:-}&quot; in
       --force-refresh-labels
     ;;
   serve)
+    EVAL_WEB_PORT="${EVAL_WEB_PORT:-6010}"
+    EVAL_WEB_HOST="${EVAL_WEB_HOST:-0.0.0.0}"
     exec "$PY" scripts/evaluation/serve_eval_web.py serve \
       --tenant-id "$TENANT_ID" \
       --queries-file "$QUERIES" \
-      --host 0.0.0.0 \
-      --port 6010
+      --host "$EVAL_WEB_HOST" \
+      --port "$EVAL_WEB_PORT"
     ;;
   *)
     usage
@@ -15,10 +15,10 @@ mkdir -p &quot;${LOG_DIR}&quot;
 # shellcheck source=scripts/lib/load_env.sh
 source "${PROJECT_ROOT}/scripts/lib/load_env.sh"
-CORE_SERVICES=("backend" "indexer" "frontend")
+CORE_SERVICES=("backend" "indexer" "frontend" "eval-web")
 OPTIONAL_SERVICES=("tei" "cnclip" "embedding" "embedding-image" "translator" "reranker" "reranker-fine")
 FULL_SERVICES=("${OPTIONAL_SERVICES[@]}" "${CORE_SERVICES[@]}")
-STOP_ORDER_SERVICES=("frontend" "indexer" "backend" "reranker-fine" "reranker" "translator" "embedding-image" "embedding" "cnclip" "tei")
+STOP_ORDER_SERVICES=("frontend" "eval-web" "indexer" "backend" "reranker-fine" "reranker" "translator" "embedding-image" "embedding" "cnclip" "tei")
 all_services() {
   echo "${FULL_SERVICES[@]}"
@@ -80,6 +80,7 @@ get_port() {
       ;;
     tei) echo "${TEI_PORT:-8080}" ;;
     cnclip) echo "${CNCLIP_PORT:-51000}" ;;
+    eval-web) echo "${EVAL_WEB_PORT:-6010}" ;;
     *) echo "" ;;
   esac
 }
@@ -117,6 +118,7 @@ service_start_cmd() {
     reranker-fine) echo "./scripts/start_reranker.sh" ;;
     tei) echo "./scripts/start_tei_service.sh" ;;
     cnclip) echo "./scripts/start_cnclip_service.sh" ;;
+    eval-web) echo "./scripts/start_eval_web.sh" ;;
     *) return 1 ;;
   esac
 }
@@ -124,7 +126,7 @@ service_start_cmd() {
 service_exists() {
   local service="$1"
   case "${service}" in
-    backend|indexer|frontend|embedding|embedding-image|translator|reranker|reranker-fine|tei|cnclip) return 0 ;;
+    backend|indexer|frontend|eval-web|embedding|embedding-image|translator|reranker|reranker-fine|tei|cnclip) return 0 ;;
     *) return 1 ;;
   esac
 }
@@ -638,7 +640,7 @@ start_one() {
         return 1
       fi
       ;;
-    backend|indexer|frontend|embedding|embedding-image|translator|reranker|reranker-fine)
+    backend|indexer|frontend|eval-web|embedding|embedding-image|translator|reranker|reranker-fine)
       echo "[start] ${service}"
       local rerank_instance=""
       rerank_instance="$(reranker_instance_for_service "${service}")"
@@ -922,8 +924,8 @@ Special targets:
 Examples:
   ./scripts/service_ctl.sh up all
-  ./scripts/service_ctl.sh up tei cnclip embedding embedding-image translator reranker reranker-fine
-  ./scripts/service_ctl.sh up backend indexer frontend
+  ./scripts/service_ctl.sh up tei cnclip embedding embedding-image translator reranker reranker-fine backend indexer frontend eval-web
+  ./scripts/service_ctl.sh up backend indexer frontend eval-web
   ./scripts/service_ctl.sh restart
   ./scripts/service_ctl.sh monitor-start all
   ./scripts/service_ctl.sh monitor-status
@@ -0,0 +1,30 @@
+#!/bin/bash
+# Search evaluation web UI (FastAPI). Managed by scripts/service_ctl.sh as service "eval-web".
+
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+source ./activate.sh
+
+EVAL_WEB_PORT="${EVAL_WEB_PORT:-6010}"
+EVAL_WEB_HOST="${EVAL_WEB_HOST:-0.0.0.0}"
+TENANT_ID="${TENANT_ID:-163}"
+QUERIES="${REPO_EVAL_QUERIES:-scripts/evaluation/queries/queries.txt}"
+
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+echo -e "${GREEN}========================================${NC}"
+echo -e "${GREEN}Starting Search Evaluation Web${NC}"
+echo -e "${GREEN}========================================${NC}"
+echo -e "\n${YELLOW}Evaluation UI:${NC} ${GREEN}http://localhost:${EVAL_WEB_PORT}/${NC}"
+echo -e "${YELLOW}Requires backend for live search:${NC} ${GREEN}http://localhost:${API_PORT:-6002}${NC}\n"
+
+export EVAL_WEB_PORT EVAL_WEB_HOST TENANT_ID REPO_EVAL_QUERIES
+
+exec python scripts/evaluation/serve_eval_web.py serve \
+  --tenant-id "${TENANT_ID}" \
+  --queries-file "${QUERIES}" \
+  --host "${EVAL_WEB_HOST}" \
+  --port "${EVAL_WEB_PORT}"