ERR打分，停止概率的计算公式修改为通用方法： p(t) = (2^t - 1) / 2^{max_grade}

tangwang
1 parent b8ffa7e9
Showing 1 changed file with 4 additions and 3 deletions Show diff stats
scripts/evaluation/eval_framework/constants.py
@@ -35,10 +35,11 @@ RELEVANCE_GAIN_MAP = {
 }
  
 # P(stop | relevance) for ERR (Expected Reciprocal Rank); cascade model (Chapelle et al., 2009).
+# p(t) = (2^t - 1) / 2^{max_grade}
 STOP_PROB_MAP = {
-    RELEVANCE_LV3: 0.99,
-    RELEVANCE_LV2: 0.8,
-    RELEVANCE_LV1: 0.1,
+    RELEVANCE_LV3: 0.875,
+    RELEVANCE_LV2: 0.375,
+    RELEVANCE_LV1: 0.125,
     RELEVANCE_LV0: 0.0,
 }