scripts/ 根目录主要保留启动/编排入口，其他脚本归到了几个固定子目录：

- 数据转换放到 scripts/data_import/README.md - 诊断巡检放到 scripts/inspect/README.md - 运维辅助放到 scripts/ops/README.md - 前端辅助服务放到 scripts/frontend/frontend_server.py - 翻译模型下载放到 scripts/translation/download_translation_models.py - 临时图片补 embedding 脚本收敛成 scripts/maintenance/embed_tenant_image_urls.py - Redis 监控脚本并入 redis/，现在是 scripts/redis/monitor_eviction.py 同时我把真实调用链都改到了新位置： - scripts/start_frontend.sh - scripts/start_cnclip_service.sh - scripts/service_ctl.sh - scripts/setup_translator_venv.sh - scripts/README.md 文档里涉及这些脚本的路径也同步修了，主要是 docs/QUICKSTART.md 和 translation/README.md。

scripts/ 根目录主要保留启动/编排入口，其他脚本归到了几个固定子目录：
- 数据转换放到 scripts/data_import/README.md - 诊断巡检放到 scripts/inspect/README.md - 运维辅助放到 scripts/ops/README.md - 前端辅助服务放到 scripts/frontend/frontend_server.py - 翻译模型下载放到 scripts/translation/download_translation_models.py - 临时图片补 embedding 脚本收敛成 scripts/maintenance/embed_tenant_image_urls.py - Redis 监控脚本并入 redis/，现在是 scripts/redis/monitor_eviction.py 同时我把真实调用链都改到了新位置： - scripts/start_frontend.sh - scripts/start_cnclip_service.sh - scripts/service_ctl.sh - scripts/setup_translator_venv.sh - scripts/README.md 文档里涉及这些脚本的路径也同步修了，主要是 docs/QUICKSTART.md 和 translation/README.md。
tangwang
1 parent 3abbc95a
Showing 28 changed files with 83 additions and 53 deletions Show diff stats
docs/QUICKSTART.md
docs/工作总结-微服务性能优化与架构.md
scripts/README.md
scripts/data_import/README.md
scripts/amazon_xlsx_to_shoplazza_xlsx.py -> scripts/data_import/amazon_xlsx_to_shoplazza_xlsx.py
scripts/competitor_xlsx_to_shoplazza_xlsx.py -> scripts/data_import/competitor_xlsx_to_shoplazza_xlsx.py
scripts/csv_to_excel.py -> scripts/data_import/csv_to_excel.py
scripts/csv_to_excel_multi_variant.py -> scripts/data_import/csv_to_excel_multi_variant.py
scripts/shoplazza_excel_template.py -> scripts/data_import/shoplazza_excel_template.py
scripts/shoplazza_import_template.py -> scripts/data_import/shoplazza_import_template.py
scripts/tenant3__csv_to_shoplazza_xlsx.sh -> scripts/data_import/tenant3_csv_to_shoplazza_xlsx.sh
scripts/frontend_server.py -> scripts/frontend/frontend_server.py
scripts/inspect/README.md
scripts/check_data_source.py -> scripts/inspect/check_data_source.py
scripts/check_es_data.py -> scripts/inspect/check_es_data.py
scripts/check_index_mapping.py -> scripts/inspect/check_index_mapping.py
scripts/compare_index_mappings.py -> scripts/inspect/compare_index_mappings.py
scripts/temp_embed_tenant_image_urls.py -> scripts/maintenance/embed_tenant_image_urls.py
scripts/ops/README.md
scripts/daily_log_router.sh -> scripts/ops/daily_log_router.sh
@@ -166,7 +166,7 @@ curl -X POST http://localhost:6008/embed/image \
 ```bash
 ./scripts/setup_translator_venv.sh
-./.venv-translator/bin/python scripts/download_translation_models.py --all-local   # 如需本地模型
+./.venv-translator/bin/python scripts/translation/download_translation_models.py --all-local   # 如需本地模型
 ./scripts/start_translator.sh
 curl -X POST http://localhost:6006/translate \
@@ -133,8 +133,8 @@ instruction: &quot;Given a shopping query, rank product titles by relevance&quot;
   - 启动时：backend/indexer/frontend/embedding/translator/reranker 会写 pid 到 `logs/<service>.pid`，并执行 `wait_for_health`（GET `http://127.0.0.1:<port>/health`）；reranker 健康重试 90 次，其余 30 次；TEI 校验 Docker 容器存在且 `/health` 成功；cnclip 无 HTTP 健康则仅校验进程/端口。
 - **监控常驻**：
   - `./scripts/service_ctl.sh monitor-start <targets>` 启动后台监控进程，将 targets 写入 `logs/service-monitor.targets`，pid 写入 `logs/service-monitor.pid`，日志追加到 `logs/service-monitor.log`。
-  - 轮询间隔 `MONITOR_INTERVAL_SEC` 默认 **10** 秒；连续 **3** 次（`MONITOR_FAIL_THRESHOLD`）健康失败则触发重启；重启冷却 `MONITOR_RESTART_COOLDOWN_SEC` 默认 **30** 秒；每小时最多重启 `MONITOR_MAX_RESTARTS_PER_HOUR` 默认 **6** 次；超限时调用 `scripts/wechat_alert.py` 告警（若存在）。
-- **日志**：各服务按日滚动到 `logs/<service>-<date>.log`，通过 `scripts/daily_log_router.sh` 与 `LOG_RETENTION_DAYS`（默认 30）控制保留。
+  - 轮询间隔 `MONITOR_INTERVAL_SEC` 默认 **10** 秒；连续 **3** 次（`MONITOR_FAIL_THRESHOLD`）健康失败则触发重启；重启冷却 `MONITOR_RESTART_COOLDOWN_SEC` 默认 **30** 秒；每小时最多重启 `MONITOR_MAX_RESTARTS_PER_HOUR` 默认 **6** 次；超限时调用 `scripts/ops/wechat_alert.py` 告警（若存在）。
+- **日志**：各服务按日滚动到 `logs/<service>-<date>.log`，通过 `scripts/ops/daily_log_router.sh` 与 `LOG_RETENTION_DAYS`（默认 30）控制保留。
 详见：`scripts/service_ctl.sh` 内注释及 `docs/Usage-Guide.md`。
 # Scripts
-`scripts/` 现在只保留当前架构下仍然有效的运行、运维、环境和数据处理脚本。
+`scripts/` 现在只保留当前架构下仍然有效的运行、运维、环境和数据处理脚本，并按职责拆到稳定子目录，避免继续在根目录平铺。
 ## 当前分类
@@ -20,6 +20,8 @@
   - `stop.sh`
   - `stop_tei_service.sh`
   - `stop_cnclip_service.sh`
+  - `frontend/`
+  - `ops/`
 - 环境初始化
   - `create_venv.sh`
@@ -33,11 +35,15 @@
   - `create_tenant_index.sh`
   - `build_suggestions.sh`
   - `mock_data.sh`
+  - `data_import/`
+  - `inspect/`
+  - `maintenance/`
 - 评估与专项工具
   - `evaluation/`
   - `redis/`
   - `debug/`
+  - `translation/`
 ## 已迁移
@@ -0,0 +1,13 @@
+# Data Import Scripts
+
+这一组脚本用于把外部商品数据或 CSV/XLSX 样本转换为 Shoplazza 导入格式。
+
+- `amazon_xlsx_to_shoplazza_xlsx.py`
+- `competitor_xlsx_to_shoplazza_xlsx.py`
+- `csv_to_excel.py`
+- `csv_to_excel_multi_variant.py`
+- `shoplazza_excel_template.py`
+- `shoplazza_import_template.py`
+- `tenant3_csv_to_shoplazza_xlsx.sh`
+
+这里是离线数据转换工具，不属于线上服务运维入口。
@@ -35,9 +35,10 @@ from pathlib import Path
 from openpyxl import load_workbook
-# Allow running as `python scripts/xxx.py` without installing as a package
-sys.path.insert(0, str(Path(__file__).resolve().parent))
-from shoplazza_excel_template import create_excel_from_template_fast
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from scripts.data_import.shoplazza_excel_template import create_excel_from_template_fast
 PREFERRED_OPTION_KEYS = [
@@ -612,4 +613,3 @@ def main():
 if __name__ == "__main__":
     main()
-
@@ -6,7 +6,7 @@ The input `data/mai_jia_jing_ling/products_data/*.xlsx` files are Amazon-format 
 (Parent/Child ASIN), not “competitor data”.
 Please use:
-  - `scripts/amazon_xlsx_to_shoplazza_xlsx.py`
+  - `scripts/data_import/amazon_xlsx_to_shoplazza_xlsx.py`
 This wrapper simply forwards all CLI args to the correctly named script, so you
 automatically get the latest performance improvements (fast read/write).
@@ -15,13 +15,12 @@ automatically get the latest performance improvements (fast read/write).
 import sys
 from pathlib import Path
-# Allow running as `python scripts/xxx.py` without installing as a package
-sys.path.insert(0, str(Path(__file__).resolve().parent))
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
-from amazon_xlsx_to_shoplazza_xlsx import main as amazon_main
+from scripts.data_import.amazon_xlsx_to_shoplazza_xlsx import main as amazon_main
 if __name__ == "__main__":
     amazon_main()
-
@@ -22,12 +22,12 @@ from openpyxl import load_workbook
 from openpyxl.styles import Font, Alignment
 from openpyxl.utils import get_column_letter
-# Shared helpers (keeps template writing consistent across scripts)
-from scripts.shoplazza_import_template import create_excel_from_template as _create_excel_from_template_shared
-from scripts.shoplazza_import_template import generate_handle as _generate_handle_shared
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
-# Add parent directory to path
-sys.path.insert(0, str(Path(__file__).parent.parent))
+# Shared helpers (keeps template writing consistent across scripts)
+from scripts.data_import.shoplazza_import_template import create_excel_from_template as _create_excel_from_template_shared
+from scripts.data_import.shoplazza_import_template import generate_handle as _generate_handle_shared
 def clean_value(value):
@@ -299,4 +299,3 @@ def main():
 if __name__ == '__main__':
     main()
-
@@ -22,12 +22,12 @@ import itertools
 from openpyxl import load_workbook
 from openpyxl.styles import Alignment
-# Shared helpers (keeps template writing consistent across scripts)
-from scripts.shoplazza_import_template import create_excel_from_template as _create_excel_from_template_shared
-from scripts.shoplazza_import_template import generate_handle as _generate_handle_shared
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
-# Add parent directory to path
-sys.path.insert(0, str(Path(__file__).parent.parent))
+# Shared helpers (keeps template writing consistent across scripts)
+from scripts.data_import.shoplazza_import_template import create_excel_from_template as _create_excel_from_template_shared
+from scripts.data_import.shoplazza_import_template import generate_handle as _generate_handle_shared
 # Color definitions
 COLORS = [
@@ -562,4 +562,3 @@ def main():
 if __name__ == '__main__':
     main()
-
@@ -5,16 +5,16 @@ cd &quot;$(dirname &quot;$0&quot;)/..&quot;
 source ./activate.sh
 # # 基本使用（生成所有数据）
-# python scripts/csv_to_excel.py
+# python scripts/data_import/csv_to_excel.py
 # # 指定输出文件
-# python scripts/csv_to_excel.py --output tenant3_imports.xlsx
+# python scripts/data_import/csv_to_excel.py --output tenant3_imports.xlsx
 # # 限制处理行数（用于测试）
-# python scripts/csv_to_excel.py --limit 100
+# python scripts/data_import/csv_to_excel.py --limit 100
 # 指定CSV文件和模板文件
-python scripts/csv_to_excel.py \
+python scripts/data_import/csv_to_excel.py \
     --csv-file data/customer1/goods_with_pic.5years_congku.csv.shuf.1w \
     --template docs/商品导入模板.xlsx \
-    --output tenant3_imports.xlsx
 \ No newline at end of file
+    --output tenant3_imports.xlsx
@@ -16,7 +16,7 @@ from pathlib import Path
 from dotenv import load_dotenv
 # Load .env file
-project_root = Path(__file__).parent.parent
+project_root = Path(__file__).resolve().parents[2]
 load_dotenv(project_root / '.env')
 # Get API_BASE_URL from environment（默认不注入，避免被旧 .env 覆盖同源策略）
@@ -27,7 +27,7 @@ INJECT_API_BASE_URL = os.getenv(&#39;FRONTEND_INJECT_API_BASE_URL&#39;, &#39;0&#39;) == &#39;1&#39;
 BACKEND_PROXY_URL = os.getenv('BACKEND_PROXY_URL', 'http://127.0.0.1:6002').rstrip('/')
 # Change to frontend directory
-frontend_dir = os.path.join(os.path.dirname(__file__), '../frontend')
+frontend_dir = os.path.join(project_root, 'frontend')
 os.chdir(frontend_dir)
 # FRONTEND_PORT is the canonical config; keep PORT as a secondary fallback.
@@ -0,0 +1,10 @@
+# Inspect Scripts
+
+这一组脚本用于做一次性诊断、索引检查和数据核对：
+
+- `check_data_source.py`
+- `check_es_data.py`
+- `check_index_mapping.py`
+- `compare_index_mappings.py`
+
+它们依赖真实 DB / ES 环境，不属于 CI 测试或 benchmark。
@@ -14,8 +14,8 @@ import argparse
 from pathlib import Path
 from sqlalchemy import create_engine, text
-# Add parent directory to path
-sys.path.insert(0, str(Path(__file__).parent.parent))
+# Add repo root to path
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
 from utils.db_connector import create_db_connection
@@ -298,4 +298,3 @@ def main():
 if __name__ == '__main__':
     sys.exit(main())
-
@@ -8,7 +8,7 @@ import os
 import argparse
 from pathlib import Path
-sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
 from utils.es_client import ESClient
@@ -265,4 +265,3 @@ def main():
 if __name__ == '__main__':
     sys.exit(main())
-
@@ -8,7 +8,7 @@ import sys
 import json
 from pathlib import Path
-sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
 from utils.es_client import get_es_client_from_env
 from indexer.mapping_generator import get_tenant_index_name
@@ -9,7 +9,7 @@ import json
 from pathlib import Path
 from typing import Dict, Any
-sys.path.insert(0, str(Path(__file__).parent.parent))
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
 from utils.es_client import get_es_client_from_env
@@ -186,4 +186,3 @@ def main():
 if __name__ == '__main__':
     sys.exit(main())
-
@@ -5,7 +5,7 @@
 用法:
   source activate.sh   # 会加载 .env，提供 ES_HOST / ES_USERNAME / ES_PASSWORD
-  python scripts/temp_embed_tenant_image_urls.py
+  python scripts/maintenance/embed_tenant_image_urls.py
 未 source 时脚本也会尝试加载项目根目录 .env。
 """
@@ -30,7 +30,7 @@ from elasticsearch.helpers import scan
 try:
     from dotenv import load_dotenv
-    _ROOT = Path(__file__).resolve().parents[1]
+    _ROOT = Path(__file__).resolve().parents[2]
     load_dotenv(_ROOT / ".env")
 except ImportError:
     pass
@@ -0,0 +1,8 @@
+# Ops Scripts
+
+这一组脚本是服务编排过程中的辅助脚本：
+
+- `daily_log_router.sh`：按天切日志
+- `wechat_alert.py`：监控告警发送
+
+如果其他启动脚本引用这些文件，应通过这里的固定路径，不要再复制出新的同类工具。
@@ -3,7 +3,7 @@
 # Route incoming log stream into per-day files.
 #
 # Usage:
-#   command 2>&1 | ./scripts/daily_log_router.sh <service> <log_dir> [retention_days]
+#   command 2>&1 | ./scripts/ops/daily_log_router.sh <service> <log_dir> [retention_days]
 #
 set -euo pipefail
@@ -6,7 +6,7 @@ This module is intentionally small and focused so that Bash-based monitors
 can invoke it without pulling in the full application stack.
 Usage example:
-  python scripts/wechat_alert.py --service backend --level error --message "backend restarted"
+  python scripts/ops/wechat_alert.py --service backend --level error --message "backend restarted"
 """
 import argparse
@@ -101,4 +101,3 @@ def main(argv: list[str] | None = None) -&gt; int:
 if __name__ == "__main__":
     raise SystemExit(main())
-
@@ -12,7 +12,7 @@ from pathlib import Path
 from datetime import datetime
 # 添加项目路径
-project_root = Path(__file__).parent.parent
+project_root = Path(__file__).resolve().parents[2]
 sys.path.insert(0, str(project_root))
 from config.env_config import REDIS_CONFIG
@@ -334,7 +334,7 @@ monitor_services() {
   local fail_threshold="${MONITOR_FAIL_THRESHOLD:-3}"
   local restart_cooldown_sec="${MONITOR_RESTART_COOLDOWN_SEC:-30}"
   local max_restarts_per_hour="${MONITOR_MAX_RESTARTS_PER_HOUR:-6}"
-  local wechat_alert_py="${PROJECT_ROOT}/scripts/wechat_alert.py"
+  local wechat_alert_py="${PROJECT_ROOT}/scripts/ops/wechat_alert.py"
   require_positive_int "MONITOR_INTERVAL_SEC" "${interval_sec}"
   require_positive_int "MONITOR_FAIL_THRESHOLD" "${fail_threshold}"
@@ -39,5 +39,5 @@ echo &quot;Using TMPDIR=${TMPDIR}&quot;
 echo
 echo "Done."
 echo "Translator venv: ${VENV_DIR}"
-echo "Download local models: ./.venv-translator/bin/python scripts/download_translation_models.py --all-local"
+echo "Download local models: ./.venv-translator/bin/python scripts/translation/download_translation_models.py --all-local"
 echo "Start service: ./scripts/start_translator.sh"
@@ -61,7 +61,7 @@ LOG_DIR=&quot;${PROJECT_ROOT}/logs&quot;
 PID_FILE="${LOG_DIR}/cnclip.pid"
 LOG_LINK="${LOG_DIR}/cnclip.log"
 LOG_FILE="${LOG_DIR}/cnclip-$(date +%F).log"
-LOG_ROUTER_SCRIPT="${PROJECT_ROOT}/scripts/daily_log_router.sh"
+LOG_ROUTER_SCRIPT="${PROJECT_ROOT}/scripts/ops/daily_log_router.sh"
 # 帮助信息
 show_help() {
@@ -27,4 +27,4 @@ echo -e &quot;  ${GREEN}http://localhost:${API_PORT}${NC}&quot;
 echo ""
 export FRONTEND_PORT API_PORT PORT
-exec python scripts/frontend_server.py
+exec python scripts/frontend/frontend_server.py
@@ -13,7 +13,7 @@ from typing import Iterable
 from huggingface_hub import snapshot_download
-PROJECT_ROOT = Path(__file__).resolve().parent.parent
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
 if str(PROJECT_ROOT) not in sys.path:
     sys.path.insert(0, str(PROJECT_ROOT))
 os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
@@ -11,7 +11,7 @@
 相关脚本与报告：
 - 启动脚本：[`scripts/start_translator.sh`](/data/saas-search/scripts/start_translator.sh)
 - 虚拟环境：[`scripts/setup_translator_venv.sh`](/data/saas-search/scripts/setup_translator_venv.sh)
-- 模型下载：[`scripts/download_translation_models.py`](/data/saas-search/scripts/download_translation_models.py)
+- 模型下载：[`scripts/translation/download_translation_models.py`](/data/saas-search/scripts/translation/download_translation_models.py)
 - 本地模型压测：[`benchmarks/translation/benchmark_translation_local_models.py`](/data/saas-search/benchmarks/translation/benchmark_translation_local_models.py)
 - 聚焦压测脚本：[`benchmarks/translation/benchmark_translation_local_models_focus.py`](/data/saas-search/benchmarks/translation/benchmark_translation_local_models_focus.py)
 - 基线性能报告：[`perf_reports/20260318/translation_local_models/README.md`](/data/saas-search/perf_reports/20260318/translation_local_models/README.md)
@@ -493,7 +493,7 @@ cd /data/saas-search
 下载全部本地模型：
 ```bash
-./.venv-translator/bin/python scripts/download_translation_models.py --all-local
+./.venv-translator/bin/python scripts/translation/download_translation_models.py --all-local
 ```
 下载完成后，默认目录应存在：