eval tagger

tangwang
1 parent cdd8ee3a
Showing 2 changed files with 7 additions and 4 deletions Show diff stats
config/config.yaml
scripts/evaluation/eval_framework/prompts.py
@@ -152,6 +152,7 @@ query_config:
     multilingual_fields:
     - title
     - qanchors
+    - enriched_tags
     - category_path
     - category_name_text
     - brief
@@ -184,7 +184,8 @@ Typical examples:
    - If the attribute is not mentioned or cannot be confirmed, prefer **High Relevant**;
    - Only treat it as a conflict when the product information clearly shows the opposite of the query requirement.
-Query: {query}{intent_suffix}
+Query: {query}
+{intent_suffix}
 Products:
 {lines}
@@ -197,6 +198,7 @@ High Relevant
 Low Relevant
 Irrelevant
+Now, assign a relevance label to each result with respect to the query "{query}".
 The output lines must correspond to the products above in the same order.
 Do not output anything else.
 """
@@ -324,7 +326,8 @@ _CLASSIFY_TEMPLATE_ZH = &quot;&quot;&quot;ä½ æ˜¯ä¸€ä¸ªæœé¥°ç”µå•†æœç´¢ç³»ç»Ÿä¸çš„ç›¸å…³æ€§åˆ
    - æœªæåŠ / æ— æ³•ç¡®è®¤ï¼Œä¼˜å…ˆæŒ‰â€œåŸºæœ¬ç›¸å…³â€å¤„ç†ï¼›
    - åªæœ‰å½“å•†å“ä¿¡æ¯æ˜Žç¡®æ˜¾ç¤ºä¸ŽæŸ¥è¯¢è¦æ±‚ç›¸åæ—¶ï¼Œæ‰è§†ä¸ºå±žæ€§å†²çªã€‚
-æŸ¥è¯¢ï¼š{query}{intent_suffix}
+æŸ¥è¯¢ï¼š{query}
+{intent_suffix}
 å•†å“ï¼š
 {lines}
@@ -336,8 +339,7 @@ _CLASSIFY_TEMPLATE_ZH = &quot;&quot;&quot;ä½ æ˜¯ä¸€ä¸ªæœé¥°ç”µå•†æœç´¢ç³»ç»Ÿä¸çš„ç›¸å…³æ€§åˆ
 å¼±ç›¸å…³
 ä¸ç›¸å…³
-è¾“å‡ºè¡Œå¿…é¡»ä¸Žä¸Šæ–¹å•†å“é¡ºåºä¸€ä¸€å¯¹åº”ã€‚
-ä¸è¦è¾“å‡ºä»»ä½•å…¶ä»–å†…å®¹ã€‚
+çŽ°åœ¨è¯·æ ¹æ® queryâ€œ{query}â€ï¼Œä¸ºæ¯ä¸ªç»“æžœæ ‡æ³¨ç›¸å…³æ€§æ ‡ç¾ã€‚è¾“å‡ºè¡Œå¿…é¡»ä¸Žä¸Šæ–¹å•†å“é¡ºåºä¸€ä¸€å¯¹åº”ï¼Œä¸è¦è¾“å‡ºä»»ä½•å…¶ä»–å†…å®¹ã€‚
 """