14 Nov, 2025

3 commits


13 Nov, 2025

8 commits

  • tangwang
     
  • tangwang
     
  • tangwang
     
  • tangwang
     
  • 创建统一配置文件 config/config.yaml(从 base 配置迁移,移除 customer_name)
    
    创建脚本体系
    启动、停止、重启、moc数据到mysql、从mysql灌入数据到ES 这些脚本
    restart.sh
    run.sh 内部调用 启动前后端
    scripts/mock_data.sh  mock数据 -> mysql
    scripts/ingest.sh  mysql->ES
    tangwang
     
  • tangwang
     
  • 主要变更:
    1. 创建.env文件,添加MySQL数据库配置(Shoplazza生产环境)
    2. 更新config/env_config.py,添加DB_CONFIG配置
    3. 创建demo_base.sh脚本,完整的演示流程:
       - 生成测试数据
       - 导入MySQL
       - 导入Elasticsearch
       - 启动后端服务
       - 启动前端服务
    4. 创建create_base_frontend.py,生成base配置专用的前端JS
    5. 创建frontend/base.html,base配置专用前端页面
    6. 更新frontend_server.py,支持base.html路由和PORT环境变量
    7. 创建stop_base.sh,停止演示服务脚本
    
    使用方式:
      bash scripts/demo_base.sh [tenant_id]
    
    访问地址:
      http://localhost:6003/base
    tangwang
     
  • 主要变更:
    1. 去掉数据源应用结构配置化,我们只针对店匠的spu sku表设计索引,数据灌入流程是写死的(只是满足测试需求,后面外层应用负责数据全量+增量灌入)。搜索系统主要关注如何适配外部搜索需求
    目前有两个数据灌入脚本,一种是之前的,一种是现在的从两个店匠的表sku表+spu表读取并且以spu为单位组织doc。
       - 配置只关注ES搜索相关配置,提高可维护性
       - 创建base配置(店匠通用配置)
    
    2. 索引结构重构(SPU维度)
       - 所有客户共享search_products索引,通过tenant_id隔离
       - 支持嵌套variants字段(SKU变体数组)
       - 创建SPUTransformer用于SPU数据转换
    
    3. API响应格式优化
       - 约定一套搜索结果的格式,而不是直接暴露ES doc的结构(_id _score _source内的字段)
       - 添加ProductResult和VariantResult模型
       - 添加suggestions和related_searches字段 (预留接口,逻辑暂未实现)
    
    4. 数据导入流程
       - 创建店匠数据导入脚本(ingest_shoplazza.py)
       - Pipeline层决定数据源,配置不包含数据源信息
       - 创建测试数据生成和导入脚本
    
    5. 文档更新
       - 更新设计文档,反映新架构
       - 创建BASE_CONFIG_GUIDE.md使用指南
    tangwang
     

11 Nov, 2025

4 commits

  • tangwang
     
  • tangwang
     
  • tangwang
     
  • ## 🎯 Major Features
    - Request context management system for complete request visibility
    - Structured JSON logging with automatic daily rotation
    - Performance monitoring with detailed stage timing breakdowns
    - Query analysis result storage and intermediate result tracking
    - Error and warning collection with context correlation
    
    ## 🔧 Technical Improvements
    - **Context Management**: Request-level context with reqid/uid correlation
    - **Performance Monitoring**: Automatic timing for all search pipeline stages
    - **Structured Logging**: JSON format logs with request context injection
    - **Query Enhancement**: Complete query analysis tracking and storage
    - **Error Handling**: Enhanced error tracking with context information
    
    ## 🐛 Bug Fixes
    - Fixed DeepL API endpoint (paid vs free API confusion)
    - Fixed vector generation (GPU memory cleanup)
    - Fixed logger parameter passing format (reqid/uid handling)
    - Fixed translation and embedding functionality
    
    ## 🌟 API Improvements
    - Simplified API interface (8→5 parameters, 37.5% reduction)
    - Made internal functionality transparent to users
    - Added performance info to API responses
    - Enhanced request correlation and tracking
    
    ## 📁 New Infrastructure
    - Comprehensive test suite (unit, integration, API tests)
    - CI/CD pipeline with automated quality checks
    - Performance monitoring and testing tools
    - Documentation and example usage guides
    
    ## 🔒 Security & Reliability
    - Thread-safe context management for concurrent requests
    - Automatic log rotation and structured output
    - Error isolation with detailed context information
    - Complete request lifecycle tracking
    
    🤖 Generated with Claude Code
    
    Co-Authored-By: Claude <noreply@anthropic.com>
    tangwang
     

10 Nov, 2025

2 commits


08 Nov, 2025

2 commits