diff --git a/docs/ES/ES_8.18/1_ES配置和使用.md b/docs/ES/ES_8.18/1_ES配置和使用.md index 6debb96..acb67bb 100644 --- a/docs/ES/ES_8.18/1_ES配置和使用.md +++ b/docs/ES/ES_8.18/1_ES配置和使用.md @@ -1,680 +1,680 @@ -# Elasticsearch 文档 - -## 相关链接 -- 接口文档：http://rap.essa.top:88/workspace/myWorkspace.do?projectId=78#2187 -- Kibana 控制台：http://43.166.252.75:5601/app/dev_tools#/console/shell - -## 分词方面 - -Ansj 分词插件安装 -ES可以用的中文分词，效果最好的是hanLP和ansj，其次是jieba。 - -我们老的搜索 solr 已经在几年前替代掉了ik，使用的是mmseg。但是我没找到mmseg的ES插件。 - -为了分词方面不至于比老版本差，这里先安装了ansj - -### 1. 下载插件 -从 [elasticsearch-analysis-ansj releases](https://github.com/NLPchina/elasticsearch-analysis-ansj/releases) 选择对应版本下载： - -- ES 8.18 版本： -```bash -wget https://github.com/NLPchina/elasticsearch-analysis-ansj/archive/refs/tags/v8.18.0.zip -``` - -- ES 8.17 版本： -```bash -wget https://github.com/NLPchina/elasticsearch-analysis-ansj/archive/refs/tags/v8.17.6.zip -``` - -### 2. 编译 -执行 `mvn package` 命令，编译成功后将在 `target/releases/` 目录生成插件压缩包： -`elasticsearch-analysis-ansj-<版本号>-release.zip` - -### 3. 安装步骤 -1. 进入 ES 安装路径（默认：`/usr/share/elasticsearch/`） -2. 执行安装命令： -```bash -bin/elasticsearch-plugin install file:///xxx/绝对路径到/elasticsearch-analysis-ansj-8.18.0.0-release.zip -``` -3. 重启服务： -```bash -systemctl restart elasticsearch -``` - -其他分词插件安装方法： -《3.1_hanlp安装.md》 -《3.2_jieba插件安装.md》 -在ES8上面安装过，但是没试过具体的版本 8.17 8.18 - -### 4. 配置说明 -停用词、同义词配置位于 `/config/elasticsearch-analysis-ansj/ansj.cfg.yml`（暂未使用） - -## 字段说明 - -```bash -需要的字段： -id 商品skuId -goods_id 商品spuId -buyer_id 所属专属采购商id -trader_buyer_ids 所属贸易商名下平台客户的专属采购商id -goods_certification_types 商品证书类型 -supplier_code 供应商编码 -supplier_name 供应商名称 -supplier_certification_code 供应商企业证书编码（列表） -auth_buyer_level_list 商品可见采购商等级（集合） -show_price_level_list 价格可见采购商等级（集合） -goods_composition 成分列表（材质） -compositions_main_secondary 物料主副（主：1，副：2），格式：物料代码_主副类型 -goods_key_word_zh 商品关键词中文 -goods_key_word_en 商品关键词英文 -goods_key_word_ru 商品关键词俄文 -goods_copyright 版权（自有、第三方、无授权、A货） -goods_main_material 主材质（字典：材质） -is_in_new_protect 是否在新品保护期（0否，1是） -goods_new_protect_date_stamp 新品保护期日期时间戳 -goods_attribute_name_zh spu属性中文（列表） -goods_attribute_name_en spu属性英文（列表） -goods_attribute_name_ru spu属性俄文（列表） -purchase_moq 采购MOQ -ts 触发索引的时间 -deliver_day 货期 -factory_no 工厂货号 -factory_no_buyer 工厂货号(客户) -fir_on_sell_time 首次上架时间 -fir_on_sell_time_stamp 首次上架时间timestamp -no 商品编码 -hs_no 宏升编码 -package_type 包装类型值（来自商品属性编码：PKG） -package_type_id 包装类型ID（来自商品属性编码：PKG） -labelId_by_skuId_essaone_* essaone商品标签，国家编码标识 -sale_goods_certificate_* 商品证书ID，国家编码标识 -labelId_by_skuId_essa_* essa商品标签，区域ID标识 -``` - -## Mapping 配置 -参考文件 `create_index.sh`： - -## 快速入门 - -### Shell -参考 [索引和查询测试](../docs/3.3_索引和查询测试.md) 包含了在ES服务器进行本地进行一些常用的查询操作。 - -### Python -- [test_index_and_search.py](../tests/test_index_and_search.py) 是一个简单的例子，创建索引，导入数据，查询数据 -- [batch_bulk_goods.py](../batch_bulk_goods.py) 功能是 通过sql 读取最近3年的所有数据，按batch（1000）通过bulk接口进行逐批入库，入库到goods索引。 - -### Kibana - -#### 分词相关 -```bash -# 索引分词 -GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车,搪胶雪宝宝&type=index_ansj - -# 查询分词 -GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车,搪胶雪宝宝&type=query_ansj - -# 查看配置 -GET /_cat/ansj/config -``` -#### 查询相关 -GET /goods/_search -{ - "query": { - "match_all": {} - }, - "size": 5 -} - -#### 1. 查看字段分词结果 -```bash -# 查看中文名称分词结果 -GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车&type=index_ansj - -# 查看英文名称分词结果 -GET /_cat/ansj?text=14 inch 4th generation real eye snow doll with manual cart&type=standard -``` - -#### 2. 查看索引随机10条内容 -```bash -GET /goods/_search -{ - "size": 10, - "query": { - "function_score": { - "query": { "match_all": {} }, - "random_score": {} - } - } -} -``` - -#### 3. 关键词查询 -```bash -# 简单关键词匹配 -GET /goods/_search -{ - "query": { - "match": { - "name_zh": "冰雪公仔" - } - } -} - -# 多字段关键词匹配 -GET /goods/_search -{ - "query": { - "multi_match": { - "query": "冰雪公仔", - "fields": ["name_zh", "sub_name_zh", "category_name_zh"] - } - } -} -``` - -#### 4. 向量查询 -```bash -# 使用向量相似度查询 -GET /goods/_search -{ - "query": { - "script_score": { - "query": { "match_all": {} }, - "script": { - "source": "cosineSimilarity(params.query_vector, 'name_prefix') + 1.0", - "params": { - "query_vector": [0.1, 0.2, ...] # 1024维向量 - } - } - } - } -} -``` - -#### 5. SKUID查询 -```bash -# 精确匹配SKUID -GET /goods/_search -{ - "query": { - "term": { - "goods_id": "2817667" - } - } -} -``` - -#### 6. 名称查询测试 -```bash -# 中文名称模糊匹配 -GET /goods/_search -{ - "query": { - "match": { - "name_zh": { - "query": "冰雪公仔", - "fuzziness": "AUTO" - } - } - } -} - -# 英文名称匹配 -GET /goods/_search -{ - "query": { - "match": { - "name_en": "snow doll" - } - } -} - -# 俄语名称匹配 -GET /goods/_search -{ - "query": { - "match": { - "name_ru": "снежная кукла" - } - } -} - -# 使用 match_phrase 进行短语匹配 -GET /goods/_search -{ - "query": { - "match_phrase": { - "name_zh": "冰雪公仔" - } - } -} - -# 使用 match_phrase 进行多语言短语匹配 -GET /goods/_search -{ - "query": { - "bool": { - "should": [ - { - "match_phrase": { - "name_zh": "冰雪公仔" - } - }, - { - "match_phrase": { - "name_en": "snow doll" - } - }, - { - "match_phrase": { - "name_ru": "снежная кукла" - } - } - ], - "minimum_should_match": 1 - } - } -} - -# 使用 match_phrase 配合 slop 参数进行模糊短语匹配 -GET /goods/_search -{ - "query": { - "match_phrase": { - "name_zh": { - "query": "冰雪公仔", - "slop": 2 - } - } - } -} - -# 多语言 match_phrase 配合 slop 参数 -GET /goods/_search -{ - "query": { - "bool": { - "should": [ - { - "match_phrase": { - "name_zh": { - "query": "冰雪公仔", - "slop": 2 - } - } - }, - { - "match_phrase": { - "name_en": { - "query": "snow doll", - "slop": 1 - } - } - }, - { - "match_phrase": { - "name_ru": { - "query": "снежная кукла", - "slop": 2 - } - } - } - ], - "minimum_should_match": 1 - } - } -} -``` - -#### 7. 多语言查询测试 -```bash -# 同时查询中英文名称 -GET /goods/_search -{ - "query": { - "bool": { - "should": [ - { - "match": { - "name_zh": "冰雪公仔" - } - }, - { - "match": { - "name_en": "snow doll" - } - } - ], - "minimum_should_match": 1 - } - } -} -``` - -#### 8. 向量索引查询测试 -注意：向量查询中的向量维度必须与索引中定义的维度匹配（1024维） -```bash -# 使用向量相似度进行商品推荐 -GET /goods/_search -{ - "query": { - "script_score": { - "query": { "match_all": {} }, - "script": { - "source": "cosineSimilarity(params.query_vector, 'ru_name') + 1.0", - "params": { - "query_vector": [0.1, 0.2, ...] # 1024维向量 - } - } - } - }, - "size": 10 -} -``` - -#### 9. 关键词+向量索引组合查询测试 -```bash -# 关键词搜索+向量相似度提权 -GET /goods/_search -{ - "query": { - "function_score": { - "query": { - "match": { - "name_zh": "冰雪公仔", - "boost": 1.0 - } - }, - "functions": [ - { - "script_score": { - "script": { - "source": "cosineSimilarity(params.query_vector, 'name_prefix') + 1.0", - "params": { - "query_vector": [0.1, 0.2, ...] # 1024维向量 - } - } - } - } - ], - "boost_mode": "multiply" - } - } -} - -# source 可以支持embedding为空 ： "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", - -两者乘起来： -{ - "query": { - "function_score": { - "score_mode": "sum", - "boost_mode": "multiply", - "query": { - "match": { - "content": { - "query": keywords, - "boost": 1.0 - } - } - }, - "functions": [ - { - "script_score": { - "script": { - "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", - "params": {"query_vector": context.embeddings[0][1]} - } - } - } - ] - } - } - } - -#### 9. 向量搜索+关键词搜索 -GET /goods/_search -{ - "query": { - "match": { - "content": { - "query": "玩具", - "boost": 1.0 - } - } - }, - "knn": { - "field": "name_prefix", - "query_vector": [-0.05291186273097992, ...], - "k": 5, - "num_candidates": 10, - "boost": 1.0 - } - } - - - - -参考代码： -```python - def execute_search(self, context, search_type="match_phrase", search_type_attachment=0, size=10): - query = context.query - normalized_query = context.normalized_query - core_term = context.core_term - keywords = context.keywords - knn_boost_keywords = core_term if core_term else keywords - expand = context.expand - - seen_queries = set() - unique_queries = [] - for q, weight in [(query, 1.0), (normalized_query, 1.0), (keywords, 0.5)]: - if q and q not in seen_queries: - unique_queries.append((q, weight)) - seen_queries.add(q) - - # 关于混合检索： - # knn和文本查询同时作用： - # 8.12之前query里面不能包含knn，kNN搜索作为查询已在 8.12 版本中引入: https://www.elastic.co/search-labs/blog/knn-query-elasticsearch - # { - # "size": 3, - # "query": { - # "bool": { - # "should": [ - # { - # "knn": { - # "field": "embedding", - # "query_vector": [2,2,2,0], - # "num_candidates": 10, - # "_name": "knn_query" - # } - # }, - # { - # "match": { - # "description": { - # "query": "luxury", - # "_name": "bm25query" - # } - # } - # } - # ] - # } - # - # knn里面不能包含query。 - # knn和query并列（hybrid search 混合检索），是求或的关系。 - # knn里面可以加filter，比如： "filter": {"match": {"my_label": "red"}} - - if search_type == "match_phrase": - body = { - "query": { - "bool": { - "should": [ - { - "match_phrase": { - "content": { - "query": unique_query, - "boost": weight, - "slop": search_type_attachment - } - } - } for unique_query, weight in unique_queries - ], - "minimum_should_match": 1 - } - } - } - # 纯关键词检索 2 - elif search_type == "match_keywords": - body = { - "query": { - "bool": { - "must": [ - { - "match": { - "content": {"query": core_term, "boost": 1.0} - } - } - ], - "should": [ - { - "match": { - "content": {"query": q, "boost": boost} - } - } for (q, boost) in [(keywords, 1.0), (expand, 0.6), (normalized_query, 1.0)] if q - ], - "minimum_should_match": 1 - } - } - } - # 关键词搜索+向量排序 - elif search_type == "match&boost": - body = { - "query": { - "function_score": { - "score_mode": "sum", - "boost_mode": "multiply", - "query": { - "match": { - "content": { - "query": keywords, - "boost": 1.0 - } - } - }, - "functions": [ - { - "script_score": { - "script": { - "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", - "params": {"query_vector": context.embeddings[0][1]} - } - } - } - ] - } - } - } - # 这个太慢 - elif search_type == "match&boost2": - body = { - "query": { - "script_score": { - "query": { - "match": { - "content": { - "query": keywords, - "boost": 1.0 - } - } - }, - "script": { - "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", - "params": {"query_vector": context.embeddings[0][1]} - } - } - } - } - # 向量搜索+关键词搜索 - elif search_type == "match&knn": - body = { - "query": { - "match": { - "content": { - "query": knn_boost_keywords, - "boost": 1.0 - } - } - }, - "knn": { - "field": "embedding", - "query_vector": context.embeddings[search_type_attachment][1], - "k": 5, - "num_candidates": 10, - "boost": 1.0 - } - } - # 纯向量搜索 - elif search_type == "knn": - body = { - "knn": { - "field": "embedding", - "query_vector": context.embeddings[search_type_attachment][1], - "k": 5, - "num_candidates": 10 - } - } - - need_embedding = (search_type == "match&boost") - need_highlights = (search_type != "knn") - - body["_source"] = {"excludes": ["keywords", "quotes"]} - if not need_embedding: - body["_source"]["excludes"].append("embedding") - # 在填充highlight之前写入search_from - search_from = f'searchtype[{search_type}],param[{search_type_attachment}],body:{body}' - - if need_highlights: - body["_source"]["excludes"] = [] - body["highlight"] = { - "pre_tags": [settings.HIGHTLIGHT_PRE_TAG], - "post_tags": [settings.HIGHTLIGHT_POST_TAG], - "fields": {"chapter_name": {}, "content": {}} - } - - body["size"] = size - - se_debug_info = '' - start_time = time.time() - try: - es_response = context.es.search(index=context.index_name, body=body) - except Exception as e: - se_debug_info = f'Error in executing search: {e}. request: {body}' - return None, se_debug_info - end_time = time.time() - elapsed_time = end_time - start_time - total_hits = es_response.get("hits", {}).get("total", {}).get("value", 0) - returned_hits = len(es_response.get("hits", {}).get("hits", [])) - - if not '"' in search_from: - search_from = search_from.replace('\'', '"') - search_from = search_from if len(search_from) < 400 else search_from[:400] + '...' - - str_body = str(body) - if not '"' in str_body: - str_body = str_body.replace('\'', '"') - se_debug_info = f'({elapsed_time:.2f} seconds. Total: {total_hits}. Returned: {returned_hits}) : {search_from[:400]}' - - if not 'hits' in es_response or not 'hits' in es_response['hits']: - se_debug_info += f' InvalidResponce in executing search: {e}. request: {body}' - return None, se_debug_info - - for hit in es_response['hits']['hits']: - hit['search_from'] = search_from - - return es_response, se_debug_info - -``` - - -# 测试向量： -# [-0.05291186273097992, 0.0274342093616724, -0.016730275005102158, 0.010487289167940617, -0.022640341892838478, -0.048682719469070435, 0.04544096067547798, 0.023079438135027885, 0.007221410982310772, 0.023566091433167458, 0.026696473360061646, 0.08252757787704468, -0.042835772037506104, 0.0009668126585893333, -0.02860398218035698, -0.004426108207553625, -0.002644421299919486, -0.027699561789631844, 0.005749804899096489, -0.04468372091650963, -0.0296687763184309, -0.009487600065767765, 0.020041221752762794, 0.00778265530243516, 0.008522099815309048, 0.03497027978301048, -0.021573258563876152, -0.028293319046497345, -8.54598984005861e-05, -0.03164539486169815, -0.017121458426117897, -0.0006902766763232648, 0.04650883004069328, -0.030234992504119873, -0.010207684710621834, -0.035288386046886444, -0.0047269039787352085, -0.0006454040994867682, -0.056146346032619476, 0.008901881985366344, 0.010757357813417912, -0.013022932223975658, 0.04627145081758499, -0.020669423043727875, -0.02031278982758522, -0.052186835557222366, -0.0148158585652709, -0.018267231062054634, -0.059003304690122604, -0.011793344281613827, 0.027096575126051903, 0.019299808889627457, 0.04161312058568001, -0.019393721595406532, -0.02361445501446724, 0.07711422443389893, -0.02068573422729969, -0.004702702630311251, -0.011135494336485863, 0.0101374052464962, -0.020808257162570953, 0.011924360878765583, -0.020093027502298355, -0.007138500455766916, 0.014727798290550709, 0.05770261213183403, 0.017841406166553497, 0.044339124113321304, -0.01490224339067936, -0.008343652822077274, -0.04842463508248329, 0.0336640290915966, -0.004893577191978693, -0.021536342799663544, -0.032384153455495834, -0.009452177211642265, -0.027460120618343353, -0.009426826611161232, 0.006357531528919935, 0.019494572654366493, 0.009722599759697914, -0.00497430982068181, 0.023032115772366524, 0.05221958085894585, -0.01671120524406433, 0.061740316450595856, -0.06789620220661163, -0.023851843550801277, -0.02249223366379738, -0.01231105625629425, -0.0499565526843071, 0.004251780919730663, 0.05466651916503906, -0.024449756368994713, -0.034151963889598846, 0.037387508898973465, -0.0016276679234579206, -0.02609393745660782, 0.01800747588276863, -0.0028136332985013723, -0.06036405637860298, 0.028903907164931297, 0.006318055558949709, 0.012870929203927517, -0.0021476889960467815, -0.012034566141664982, -0.008372323587536812, 0.024942906573414803, 0.08258169889450073, 0.006757829803973436, 0.032017264515161514, -0.012414710596203804, 0.014826267957687378, -0.040858786553144455, -0.0060302577912807465, 0.00843990221619606, -0.031066348776221275, -0.06313654035329819, 0.0056659989058971405, -0.007768781390041113, 0.011673268862068653, 0.007261875085532665, 0.006112886127084494, -0.07374890148639679, 0.06602894514799118, -0.05385972931981087, -0.0010994652984663844, 0.05939924344420433, 0.015503636561334133, 0.034621711820364, 0.008040975779294968, -0.023962488397955894, -0.06270411610603333, 0.00027893096557818353, -0.0436306893825531, -0.006309020332992077, 0.02416943572461605, -0.015391307882964611, -0.012442439794540405, -0.003181715961545706, -0.0021985983476042747, 0.008671553805470467, 0.004063367377966642, -0.02560708485543728, 0.03469422832131386, -0.04249674826860428, -0.013552767224609852, -0.052823010832071304, 0.014670411124825478, -0.011493593454360962, 0.024076055735349655, 0.056352417916059494, -0.008510314859449863, 0.015936613082885742, 0.003935575485229492, 0.0037949192337691784, 0.015074086375534534, 0.016583971679210663, -0.0057802870869636536, 0.005751866847276688, -0.009386995807290077, -0.03710195794701576, -0.03144300729036331, -0.07106415182352066, -0.003882911056280136, -0.010697683319449425, -0.014338435605168343, 0.007036983501166105, -0.035716522485017776, 0.06593189388513565, 0.007752529811114073, -0.030261363834142685, -0.02513342909514904, -0.039278656244277954, 0.015320679172873497, -0.012659071013331413, 0.014207725413143635, 0.010264124721288681, 0.01617652177810669, -0.022644126787781715, -0.031033707782626152, 0.04160666465759277, -0.05329348146915436, 0.02423500455915928, -0.019389694556593895, 0.008645910769701004, -0.005958682857453823, -0.03648180514574051, 0.011972597800195217, 0.037404924631118774, -0.007001751102507114, -0.05138246342539787, 0.0013400549069046974, -0.03268183395266533, 0.07687076926231384, -0.02033335529267788, -0.020667986944317818, 0.0038236891850829124, 0.029960744082927704, 0.015430699102580547, 0.05047214776277542, 0.0052254535257816315, 0.013995353132486343, -0.031164521351456642, -0.014291719533503056, 0.015829795971512794, -0.0013409113744273782, -0.044300951063632965, 0.045415859669446945, -0.005037966184318066, -0.03883415088057518, 0.027200160548090935, 0.008182630874216557, -0.046456750482320786, -0.029778052121400833, 0.02067168429493904, -0.006381513085216284, -0.04693000763654709, 0.009974686428904533, 0.03109011799097061, -0.012696364894509315, 0.030124813318252563, 0.02372679114341736, 0.06566771119832993, 0.03553507477045059, -0.032816141843795776, 0.028003521263599396, 0.06498659402132034, -0.013530750758945942, 0.0312667116522789, -0.015660811215639114, -0.00776742585003376, -0.004829467739909887, -0.015968922525644302, 0.04765664413571358, -0.0026502758264541626, 0.01891564577817917, 0.04119837284088135, 0.012158435769379139, 0.008338023908436298, -0.006039333995431662, 0.0630166307091713, -0.02758428454399109, 0.029347822070121765, -0.030129415914416313, 0.023165738210082054, 0.04064684361219406, 0.04446929693222046, -0.006133638322353363, -0.013095719739794731, -0.041152223944664, -0.01038535125553608, 0.01738007925450802, 0.0010595708154141903, -0.055003564804792404, 0.036829687654972076, -0.030270753428339958, -0.009607627056539059, 0.014103117398917675, 0.005140293389558792, 0.032931022346019745, 0.026972685009241104, -0.00039128100615926087, 0.00550195062533021, 0.062454141676425934, 0.02344602160155773, -0.01688288524746895, 0.011600837111473083, 0.009648085571825504, 0.012827200815081596, 0.02368510514497757, -0.044808436185121536, 0.006574536208063364, 0.03677171841263771, 0.021754244342446327, -0.0031720376573503017, -0.03498553857207298, -0.027119319885969162, 0.05196662247180939, 0.0063033513724803925, -0.002766692778095603, -0.03879206255078316, -0.005737128667533398, -0.02351462095975876, 0.04338989034295082, -0.03623301535844803, 0.003727369010448456, 0.044172726571559906, 0.06180792301893234, -0.025736358016729355, 0.01280374638736248, -0.01768171414732933, 0.0413120836019516, 0.036350950598716736, 0.020034022629261017, -0.00938474852591753, -0.04920303076505661, -0.1626604050397873, 0.0016566020203754306, -0.010797491297125816, 0.0037245014682412148, 0.039030417799949646, -0.009399985894560814, 0.016659803688526154, -0.047097429633140564, -0.00987484585493803, 0.020634479820728302, 0.005361238028854132, -0.05283225327730179, 0.002501025330275297, -0.004766151309013367, 0.00850654486566782, -0.0050267502665519714, -0.046555373817682266, 0.012670878320932388, 0.0018581973854452372, -0.010647253133356571, 0.01990092545747757, 0.02013244479894638, 0.04490885138511658, 0.029433563351631165, -0.01408607978373766, 0.029722925275564194, 0.04512600228190422, -0.04305345192551613, 0.0053901285864412785, -0.010685979388654232, 0.01516974437981844, 0.02340293675661087, -0.014181641861796379, -0.0013334851246327162, 0.020624764263629913, 0.06469231843948364, 0.016654038801789284, -0.043994754552841187, 0.025707466527819633, -0.004160136915743351, 0.021129926666617393, 0.041262850165367126, 0.006293899845331907, 0.056005991995334625, -0.006883381400257349, -0.07502268254756927, -0.02920101210474968, -0.019043054431676865, 0.00737513042986393, 0.013621360063552856, -0.02504715882241726, -0.01138006430119276, -0.010744514875113964, -0.02502342313528061, -0.03335903584957123, 0.012180354446172714, -0.03276645019650459, 0.05202409625053406, 0.03246080502867699, 0.03068908303976059, -0.029587913304567337, -0.04850265011191368, -0.006388102192431688, -0.03203853219747543, -0.050761956721544266, -0.021925227716565132, 0.036384399980306625, -0.011895880103111267, -0.007408954203128815, -0.012625153176486492, 0.0024322718381881714, -0.012196220457553864, -0.007011729292571545, -0.0337890200316906, -0.030034994706511497, 0.04638829082250595, -0.028362803161144257, -0.01176459901034832, 0.00956833828240633, -0.12054562568664551, -0.020540419965982437, 0.014624865725636482, -0.025515791028738022, -0.005027926992624998, -0.03586679324507713, -0.05585843697190285, -0.01700599677860737, -0.00044939795043319464, 0.029278729110956192, 0.25503888726234436, -0.024952411651611328, 0.005794796161353588, -0.007252118084579706, 0.03397773951292038, -0.0030146583449095488, -0.016645856201648712, -0.0008194005931727588, 0.02789629064500332, -0.039116114377975464, -0.035631854087114334, 0.04917449131608009, -0.006455820053815842, -0.011818122118711472, -0.00958359707146883, 0.013176187872886658, 0.037286531180143356, 0.022334400564432144, 0.05832865461707115, 0.010104321874678135, -0.04915979504585266, -0.022671189159154892, -0.016606582328677177, -0.007431587669998407, 0.0025214774068444967, -0.038979604840278625, 0.014895224012434483, 0.03583076596260071, 0.0006473385728895664, 0.04958082735538483, -0.017827684059739113, 0.015710417181253433, 0.062094446271657944, -0.014381879940629005, 0.0002880772517528385, 0.004948006477206945, -8.711735063116066e-06, -0.0029445397667586803, -0.044325683265924454, 0.047702621668577194, -0.03197811171412468, -0.02109563909471035, 0.03041824884712696, 0.021582895889878273, -0.004118872340768576, -0.025784745812416077, 0.06275995075702667, 0.006879465654492378, 0.04185185581445694, 0.02031264826655388, -0.02274201810359955, -0.009617358446121216, -0.04315454140305519, -0.033287111669778824, -0.025126483291387558, -0.003923895303159952, -0.041508499532938004, -0.0009355457150377333, -0.033565372228622437, 0.02229289337992668, -0.0026574484072625637, -0.0028596664778888226, -0.02223617024719715, -0.016868866980075836, 0.04172029718756676, 0.0014162511797621846, -0.037737537175416946, -0.010155809111893177, -0.010357595980167389, 0.04541466012597084, 0.03563382104039192, -0.019189776852726936, -0.012577632442116737, -0.013781189918518066, 0.026566311717033386, 0.020911909639835358, 0.02781282551586628, 0.053938526660203934, 0.0194545891135931, 0.0015139722963795066, -0.0357731431722641, -0.005088387057185173, 0.004257760010659695, 0.04332628846168518, -0.012149352580308914, -0.04734082147479057, 0.018029984086751938, -0.01322091929614544, -0.059820450842380524, -0.03677783161401749, -0.006745075341314077, -0.02209635078907013, -0.012663901783525944, -0.0059855030849576, 0.016270749270915985, -0.00725028058513999, 0.03019685670733452, 0.010252268984913826, -0.06314245611429214, -0.005512078758329153, -0.016377074643969536, -0.0014438428916037083, 0.029021194204688072, -0.015355946496129036, 0.02559172362089157, -0.04241044819355011, 0.010147088207304478, -0.016036594286561012, 0.023162752389907837, 0.047236304730176926, 0.0166736152023077, 0.01226564310491085, -0.015224735252559185, -0.01298521552234888, -0.008012642152607441, 0.028470756486058235, -0.013741613365709782, 0.019896863028407097, 0.01720179058611393, 0.01571199856698513, 0.030143165960907936, -0.02969514951109886, 0.014739652164280415, -0.01854291744530201, -0.045576371252536774, -0.04516203701496124, 0.02147211693227291, 0.007073952350765467, 0.008106761611998081, -0.01828523352742195, 0.002731812885031104, -0.04545339569449425, 0.019007619470357895, 0.03504781052470207, 0.037705861032009125, -0.0045634908601641655, 0.0070000626146793365, 0.0037205498665571213, 0.005224148277193308, -0.017060590907931328, -0.04246727377176285, -0.006265614647418261, -0.015374364331364632, -0.03380871191620827, -0.005029333755373955, 0.007065227720886469, 0.003886009333655238, 0.008613690733909607, -0.012133199721574783, 0.005556005053222179, -0.021959641948342323, 0.04834386706352234, 0.03787781298160553, -0.057815466076135635, 0.015909207984805107, -0.03855409845709801, 0.0018244135426357388, 0.04186264052987099, -0.054983459413051605, 0.006219237111508846, 0.03494301065802574, 0.023722950369119644, 0.0312604121863842, 0.05597991123795509, -0.030345493927598, 0.016615940257906914, -0.0207205917686224, 0.055960651487112045, -0.012713379226624966, -0.0261109359562397, 0.014332456514239311, -0.017245708033442497, -0.06636268645524979, 0.00592504907399416, 0.04649018123745918, -0.018362276256084442, 0.009620632976293564, -0.0044480785727500916, -0.0014729035319760442, 0.015621249563992023, 0.0367378331720829, -0.011857259087264538, -0.045088741928339005, 0.0006832792423665524, 0.02601524256169796, -0.02120809443295002, 0.018104318529367447, 0.008069046773016453, 0.013658273033797741, 0.004183551296591759, -0.04133244603872299, 0.05436890944838524, 0.009334285743534565, -0.014695074409246445, -0.011054124683141708, 0.009796642698347569, -0.008759389631450176, -0.06399217247962952, -0.0028859861195087433, -0.008736967109143734, -0.003506746841594577, 0.008123806677758694, 0.008794951252639294, -0.02940259501338005, 0.009597218595445156, -0.02197900228202343, -0.02082076109945774, 0.023915970697999, -0.059058744460344315, -0.010253551416099072, 0.024443935602903366, -0.029604850336909294, 0.008135135285556316, 0.03568771481513977, -0.017330091446638107, -0.003135789418593049, 0.035103678703308105, 0.0370408296585083, -0.01022601593285799, -0.045891791582107544, 0.01726667769253254, -0.008570673875510693, 0.015297998674213886, -0.015412220731377602, -0.01425748411566019, 0.031544867902994156, 0.013110813684761524, -0.057211123406887054, -0.0008968000765889883, 0.001981658162549138, -0.002101168967783451, -0.09516698867082596, -0.034693196415901184, 0.011157260276377201, 0.010063023306429386, -0.02550840750336647, 0.009959851391613483, 0.022281678393483162, -0.03908146917819977, 0.02196437120437622, 0.03520793840289116, -0.06856158375740051, -0.004901218693703413, 0.1122148334980011, -0.01498009730130434, 0.03165500983595848, -0.07618033140897751, -0.014297851361334324, 0.02150021120905876, 0.005999598652124405, -0.013493427075445652, 0.013868110254406929, 0.00079053093213588, 0.006475066766142845, 0.000955471652559936, -0.03403160721063614, -0.02295752801001072, 0.0041635241359472275, -0.03955964744091034, -0.04943346977233887, 0.00032474088948220015, 0.039174411445856094, -0.011974001303315163, 0.008057610131800175, 0.03809700161218643, -0.041719768196344376, 0.037615906447172165, -0.035932306200265884, 0.008293192833662033, -0.03261689469218254, -0.023902395740151405, -7.811257091816515e-05, -0.011328466236591339, -0.026476409286260605, 0.055370282381772995, 0.03128054738044739, -0.014991461299359798, 0.017835773527622223, 0.01642710715532303, 0.029273470863699913, -0.012139911763370037, 0.01371818222105503, -0.013113478198647499, -0.04071088507771492, 0.0233455840498209, -0.019497444853186607, -0.01747158169746399, 0.02493683062493801, 0.024074571207165718, -0.03614620864391327, -0.025289475917816162, -0.04030011221766472, -0.046772539615631104, 0.009969661012291908, 0.003724620910361409, 0.007474626414477825, -0.04855594411492348, 0.04697829484939575, 0.010695616714656353, 0.027944304049015045, -0.003937696572393179, -0.011591222137212753, -0.011533009819686413, 0.03215765953063965, -0.04699324443936348, -9.356102236779407e-05, -0.01535400003194809, -0.010238519869744778, 0.002703386126086116, 0.04759520664811134, 0.0074842446483671665, -0.04050430282950401, -0.028402622789144516, -0.03205197677016258, 0.011288953013718128, 0.006053865421563387, 0.04641448333859444, 0.005652922671288252, -0.018560705706477165, 0.02581481821835041, 0.00962467584758997, -0.017888177186250687, -0.026476262137293816, -0.005547264125198126, 0.012222226709127426, -0.004069746006280184, -0.020438821986317635, 0.01929863728582859, -0.0053736320696771145, 0.02221786603331566, -0.007175051141530275, 0.003961225971579552, -0.012380941770970821, -0.0040277824737131596, 0.009086307138204575, 0.012202796526253223, 0.018483169376850128, 0.017530532553792, 0.0422886498272419, 0.04987001419067383, 0.003722204128280282, 0.06421508640050888, -0.016258088871836662, -0.027659112587571144, 0.004458434879779816, -0.02898143045604229, -0.014475414529442787, 0.032039571553468704, -0.025734663009643555, -0.01585981249809265, 0.04900333285331726, -0.06422552466392517, -0.0007134959450922906, -0.04035528376698494, 0.03290264680981636, -0.0018848407780751586, 0.0068516512401402, 0.00032433189335279167, -0.002669606124982238, -0.017596688121557236, -0.026878179982304573, 0.014075388200581074, 0.020072080194950104, -0.00295435544103384, -0.01918656937777996, -0.007689833641052246, 0.039347097277641296, 0.0026605715975165367, 0.011779646389186382, 0.04189120978116989, -0.03846775367856026, -0.01993645168840885, 0.04546443000435829, 0.05682912468910217, -0.012384516187012196, -0.004507445730268955, 0.007476931903511286, -0.01160018052905798, 0.006559243891388178, 0.04354899004101753, 0.006185194011777639, 0.028355205431580544, -0.006518798414617777, -0.029528537765145302, 0.06740271300077438, -0.052158474922180176, 0.0025031850673258305, -0.005957300774753094, 0.00500349560752511, 0.022637680172920227, -0.0027129461523145437, -0.011677206493914127, -0.042732879519462585, -0.0021236639004200697, -0.1499215066432953, 0.02914350852370262, -0.031246500089764595, -0.027244996279478073, -0.006904688663780689, 0.01088196225464344, 0.01271661464124918, -0.0430884025990963, -0.020760131999850273, -0.006593034137040377, -0.0007962957606650889, -0.031729113310575485, 0.052976224571466446, -0.03149586543440819, 0.0392388291656971, 0.023318620398640633, -0.01383691094815731, 0.02858218550682068, 0.023135144263505936, 0.026421336457133293, 0.00027594034327194095, -0.03901490569114685, 0.008533132262527943, -0.03802476078271866, -0.011105065234005451, -0.028275510296225548, 0.04846742004156113, 0.021237077191472054, -0.027375172823667526, -0.02717825025320053, -0.031243441626429558, -0.021638689562678337, 0.024066096171736717, 0.05689090117812157, -0.04352620989084244, 0.03599394112825394, 0.05153508856892586, 0.002263782313093543, 0.047110624611377716, 0.006084555760025978, 0.003244618885219097, -0.0015037712873890996, 0.027960799634456635, -0.013650861568748951, 0.03281615301966667, 0.012363187968730927, 0.02162906341254711, -0.010951842181384563, -0.02786285988986492, 0.03754381462931633, 0.01957041770219803, -0.017010418698191643, -0.008339766412973404, 0.0755641758441925, 0.023412147536873817, -0.005748848430812359, -0.05465301498770714, -0.02190011739730835, 0.0054182386957108974, 0.032733004540205, -0.05342638120055199, 0.009907999075949192, -0.02370712347328663, -0.015652501955628395, -0.011254304088652134, -0.019827252253890038, -0.021032121032476425, -0.02607329562306404, -0.0008710312540642917, -0.06800976395606995, -0.017296750098466873, 0.015312970615923405, -0.015649013221263885, -0.016449443995952606, -0.012058117426931858, 0.002104945247992873, 0.020476385951042175, 0.014795565977692604, -0.02145536057651043, -0.028734024614095688, -0.041212357580661774, -0.008211270906031132, 0.033569078892469406, -0.0033273063600063324, -0.02339683100581169, 0.0421740785241127, -0.009677124209702015, -0.006869456730782986, -0.016001028940081596, 0.029614608734846115, -0.06062136963009834, -0.011824233457446098, 0.012096629478037357, -0.028248939663171768, -0.03703905642032623, 0.012119539082050323, -0.041021380573511124, 0.01975782960653305, -0.028443211689591408, 0.020459437742829323, 0.0073023103177547455, -0.06498327851295471, -0.004016770515590906, 0.06460512429475784, -0.053343966603279114, 0.03865537419915199, -5.4113028454594314e-05, -0.008642046712338924, -0.009384138509631157, -0.037736788392066956, -0.035090748220682144, 0.018596891313791275, -0.008763385005295277, 0.040228284895420074, 0.03811536356806755, -0.034618355333805084, -0.004665717948228121, 0.04813361540436745, -0.004303373862057924, 0.00795511994510889, -0.017838604748249054, 0.00563138909637928, -0.03171280398964882, -0.0259436946362257, 0.004301885142922401, -0.02739236131310463, 0.03270035237073898, 0.009064823389053345, -0.0363747663795948, 0.02325567975640297, 0.03453107923269272, -0.012906554155051708, 0.028347544372081757, 0.01234712265431881, 0.030589573085308075, 0.0024874424561858177, -0.0173872709274292, 0.0247347354888916, 0.004171399865299463, 0.02350561134517193, -0.05499064922332764, -0.023146219551563263, -0.012485259212553501, -0.0228674728423357, 0.013267520815134048, 0.021304689347743988, -0.018937893211841583, -0.0260267723351717, -0.022532619535923004, 0.0030378480441868305, -0.008528024889528751, -0.030528495088219643, -0.009305189363658428, -0.0074027362279593945, -0.020641637966036797, 0.006984233390539885, 0.04300186410546303, -0.033014994114637375, -0.006089311558753252, 0.04753036051988602, -0.036625705659389496, -0.04691743850708008, -0.007467558141797781, 0.0652017593383789, -0.03861508145928383, -0.00741452956572175, 0.003471594536677003, 0.016132064163684845, 0.01570185460150242, 0.018733495846390724, -0.019025148823857307, 0.003490244736894965, -0.017714614048600197, -0.003447450464591384, 0.015267218463122845, 0.015076974406838417, -0.002631498035043478, 0.005311752203851938, 0.014075293205678463, 0.0026123111601918936, 0.011874910444021225, 0.0714355856180191, 0.06941138952970505, 0.022251378744840622, 0.01972009800374508, 0.04719123989343643, 0.023544959723949432, 0.017852554097771645, 0.01843070052564144, -0.05294886603951454, -0.008682304993271828, 0.010625398717820644, 0.0428495928645134, 0.002173527143895626, 0.06291069090366364, 0.024296458810567856, 0.008714474737644196, 0.06520587205886841, 0.015627536922693253, 0.04247526824474335, 0.0009774811333045363, 0.00738496845588088, -0.024803027510643005, 0.013228596188127995, -0.037615202367305756, -0.028807995840907097, 0.012890785001218319, -0.01587829552590847, -0.01928863860666752, 0.0011809614952653646, -0.026926854625344276, -0.020252779126167297, -0.010968486778438091, -0.015348547138273716, 0.008559435606002808, -0.009286923334002495, 0.0014621232403442264, 0.03831499442458153, 0.016517579555511475, 0.037184324115514755, -0.041231196373701096, 0.03757374733686447, -0.039465345442295074, -0.04308579862117767, 0.0011091071646660566, -0.029794104397296906, 0.008459310978651047, -0.01713281124830246, -0.016625113785266876, -0.05582521855831146, -0.0415986105799675, 0.028725938871502876, 0.04966316372156143, 0.012718678452074528, -0.025533588603138924, 0.013822318986058235, -5.168768620933406e-05, 0.02616700902581215, -0.06113629788160324, -0.03175340220332146, 0.03593592345714569, -0.04014921560883522, -0.020605407655239105, 0.02186705358326435] +# Elasticsearch 文档 + +## 相关链接 +- 接口文档：http://rap.essa.top:88/workspace/myWorkspace.do?projectId=78#2187 +- Kibana 控制台：http://43.166.252.75:5601/app/dev_tools#/console/shell + +## 分词方面 + +Ansj 分词插件安装 +ES可以用的中文分词，效果最好的是hanLP和ansj，其次是jieba。 + +我们老的搜索 solr 已经在几年前替代掉了ik，使用的是mmseg。但是我没找到mmseg的ES插件。 + +为了分词方面不至于比老版本差，这里先安装了ansj + +### 1. 下载插件 +从 [elasticsearch-analysis-ansj releases](https://github.com/NLPchina/elasticsearch-analysis-ansj/releases) 选择对应版本下载： + +- ES 8.18 版本： +```bash +wget https://github.com/NLPchina/elasticsearch-analysis-ansj/archive/refs/tags/v8.18.0.zip +``` + +- ES 8.17 版本： +```bash +wget https://github.com/NLPchina/elasticsearch-analysis-ansj/archive/refs/tags/v8.17.6.zip +``` + +### 2. 编译 +执行 `mvn package` 命令，编译成功后将在 `target/releases/` 目录生成插件压缩包： +`elasticsearch-analysis-ansj-<版本号>-release.zip` + +### 3. 安装步骤 +1. 进入 ES 安装路径（默认：`/usr/share/elasticsearch/`） +2. 执行安装命令： +```bash +bin/elasticsearch-plugin install file:///xxx/绝对路径到/elasticsearch-analysis-ansj-8.18.0.0-release.zip +``` +3. 重启服务： +```bash +systemctl restart elasticsearch +``` + +其他分词插件安装方法： +《3.1_hanlp安装.md》 +《3.2_jieba插件安装.md》 +在ES8上面安装过，但是没试过具体的版本 8.17 8.18 + +### 4. 配置说明 +停用词、同义词配置位于 `/config/elasticsearch-analysis-ansj/ansj.cfg.yml`（暂未使用） + +## 字段说明 + +```bash +需要的字段： +id 商品skuId +goods_id 商品spuId +buyer_id 所属专属采购商id +trader_buyer_ids 所属贸易商名下平台客户的专属采购商id +goods_certification_types 商品证书类型 +supplier_code 供应商编码 +supplier_name 供应商名称 +supplier_certification_code 供应商企业证书编码（列表） +auth_buyer_level_list 商品可见采购商等级（集合） +show_price_level_list 价格可见采购商等级（集合） +goods_composition 成分列表（材质） +compositions_main_secondary 物料主副（主：1，副：2），格式：物料代码_主副类型 +goods_key_word_zh 商品关键词中文 +goods_key_word_en 商品关键词英文 +goods_key_word_ru 商品关键词俄文 +goods_copyright 版权（自有、第三方、无授权、A货） +goods_main_material 主材质（字典：材质） +is_in_new_protect 是否在新品保护期（0否，1是） +goods_new_protect_date_stamp 新品保护期日期时间戳 +goods_attribute_name_zh spu属性中文（列表） +goods_attribute_name_en spu属性英文（列表） +goods_attribute_name_ru spu属性俄文（列表） +purchase_moq 采购MOQ +ts 触发索引的时间 +deliver_day 货期 +factory_no 工厂货号 +factory_no_buyer 工厂货号(客户) +fir_on_sell_time 首次上架时间 +fir_on_sell_time_stamp 首次上架时间timestamp +no 商品编码 +hs_no 宏升编码 +package_type 包装类型值（来自商品属性编码：PKG） +package_type_id 包装类型ID（来自商品属性编码：PKG） +labelId_by_skuId_essaone_* essaone商品标签，国家编码标识 +sale_goods_certificate_* 商品证书ID，国家编码标识 +labelId_by_skuId_essa_* essa商品标签，区域ID标识 +``` + +## Mapping 配置 +参考文件 `create_index.sh`： + +## 快速入门 + +### Shell +参考 [索引和查询测试](../docs/3.3_索引和查询测试.md) 包含了在ES服务器进行本地进行一些常用的查询操作。 + +### Python +- [test_index_and_search.py](../tests/test_index_and_search.py) 是一个简单的例子，创建索引，导入数据，查询数据 +- [batch_bulk_goods.py](../batch_bulk_goods.py) 功能是 通过sql 读取最近3年的所有数据，按batch（1000）通过bulk接口进行逐批入库，入库到goods索引。 + +### Kibana + +#### 分词相关 +```bash +# 索引分词 +GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车,搪胶雪宝宝&type=index_ansj + +# 查询分词 +GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车,搪胶雪宝宝&type=query_ansj + +# 查看配置 +GET /_cat/ansj/config +``` +#### 查询相关 +GET /goods/_search +{ + "query": { + "match_all": {} + }, + "size": 5 +} + +#### 1. 查看字段分词结果 +```bash +# 查看中文名称分词结果 +GET /_cat/ansj?text=14寸第4代真眼珠实身冰雪公仔带手动大推车&type=index_ansj + +# 查看英文名称分词结果 +GET /_cat/ansj?text=14 inch 4th generation real eye snow doll with manual cart&type=standard +``` + +#### 2. 查看索引随机10条内容 +```bash +GET /goods/_search +{ + "size": 10, + "query": { + "function_score": { + "query": { "match_all": {} }, + "random_score": {} + } + } +} +``` + +#### 3. 关键词查询 +```bash +# 简单关键词匹配 +GET /goods/_search +{ + "query": { + "match": { + "name_zh": "冰雪公仔" + } + } +} + +# 多字段关键词匹配 +GET /goods/_search +{ + "query": { + "multi_match": { + "query": "冰雪公仔", + "fields": ["name_zh", "sub_name_zh", "category_name_zh"] + } + } +} +``` + +#### 4. 向量查询 +```bash +# 使用向量相似度查询 +GET /goods/_search +{ + "query": { + "script_score": { + "query": { "match_all": {} }, + "script": { + "source": "cosineSimilarity(params.query_vector, 'name_prefix') + 1.0", + "params": { + "query_vector": [0.1, 0.2, ...] # 1024维向量 + } + } + } + } +} +``` + +#### 5. SKUID查询 +```bash +# 精确匹配SKUID +GET /goods/_search +{ + "query": { + "term": { + "goods_id": "2817667" + } + } +} +``` + +#### 6. 名称查询测试 +```bash +# 中文名称模糊匹配 +GET /goods/_search +{ + "query": { + "match": { + "name_zh": { + "query": "冰雪公仔", + "fuzziness": "AUTO" + } + } + } +} + +# 英文名称匹配 +GET /goods/_search +{ + "query": { + "match": { + "name_en": "snow doll" + } + } +} + +# 俄语名称匹配 +GET /goods/_search +{ + "query": { + "match": { + "name_ru": "снежная кукла" + } + } +} + +# 使用 match_phrase 进行短语匹配 +GET /goods/_search +{ + "query": { + "match_phrase": { + "name_zh": "冰雪公仔" + } + } +} + +# 使用 match_phrase 进行多语言短语匹配 +GET /goods/_search +{ + "query": { + "bool": { + "should": [ + { + "match_phrase": { + "name_zh": "冰雪公仔" + } + }, + { + "match_phrase": { + "name_en": "snow doll" + } + }, + { + "match_phrase": { + "name_ru": "снежная кукла" + } + } + ], + "minimum_should_match": 1 + } + } +} + +# 使用 match_phrase 配合 slop 参数进行模糊短语匹配 +GET /goods/_search +{ + "query": { + "match_phrase": { + "name_zh": { + "query": "冰雪公仔", + "slop": 2 + } + } + } +} + +# 多语言 match_phrase 配合 slop 参数 +GET /goods/_search +{ + "query": { + "bool": { + "should": [ + { + "match_phrase": { + "name_zh": { + "query": "冰雪公仔", + "slop": 2 + } + } + }, + { + "match_phrase": { + "name_en": { + "query": "snow doll", + "slop": 1 + } + } + }, + { + "match_phrase": { + "name_ru": { + "query": "снежная кукла", + "slop": 2 + } + } + } + ], + "minimum_should_match": 1 + } + } +} +``` + +#### 7. 多语言查询测试 +```bash +# 同时查询中英文名称 +GET /goods/_search +{ + "query": { + "bool": { + "should": [ + { + "match": { + "name_zh": "冰雪公仔" + } + }, + { + "match": { + "name_en": "snow doll" + } + } + ], + "minimum_should_match": 1 + } + } +} +``` + +#### 8. 向量索引查询测试 +注意：向量查询中的向量维度必须与索引中定义的维度匹配（1024维） +```bash +# 使用向量相似度进行商品推荐 +GET /goods/_search +{ + "query": { + "script_score": { + "query": { "match_all": {} }, + "script": { + "source": "cosineSimilarity(params.query_vector, 'ru_name') + 1.0", + "params": { + "query_vector": [0.1, 0.2, ...] # 1024维向量 + } + } + } + }, + "size": 10 +} +``` + +#### 9. 关键词+向量索引组合查询测试 +```bash +# 关键词搜索+向量相似度提权 +GET /goods/_search +{ + "query": { + "function_score": { + "query": { + "match": { + "name_zh": "冰雪公仔", + "boost": 1.0 + } + }, + "functions": [ + { + "script_score": { + "script": { + "source": "cosineSimilarity(params.query_vector, 'name_prefix') + 1.0", + "params": { + "query_vector": [0.1, 0.2, ...] # 1024维向量 + } + } + } + } + ], + "boost_mode": "multiply" + } + } +} + +# source 可以支持embedding为空 ： "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", + +两者乘起来： +{ + "query": { + "function_score": { + "score_mode": "sum", + "boost_mode": "multiply", + "query": { + "match": { + "content": { + "query": keywords, + "boost": 1.0 + } + } + }, + "functions": [ + { + "script_score": { + "script": { + "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", + "params": {"query_vector": context.embeddings[0][1]} + } + } + } + ] + } + } + } + +#### 9. 向量搜索+关键词搜索 +GET /goods/_search +{ + "query": { + "match": { + "content": { + "query": "玩具", + "boost": 1.0 + } + } + }, + "knn": { + "field": "name_prefix", + "query_vector": [-0.05291186273097992, ...], + "k": 5, + "num_candidates": 10, + "boost": 1.0 + } + } + + + + +参考代码： +```python + def execute_search(self, context, search_type="match_phrase", search_type_attachment=0, size=10): + query = context.query + normalized_query = context.normalized_query + core_term = context.core_term + keywords = context.keywords + knn_boost_keywords = core_term if core_term else keywords + expand = context.expand + + seen_queries = set() + unique_queries = [] + for q, weight in [(query, 1.0), (normalized_query, 1.0), (keywords, 0.5)]: + if q and q not in seen_queries: + unique_queries.append((q, weight)) + seen_queries.add(q) + + # 关于混合检索： + # knn和文本查询同时作用： + # 8.12之前query里面不能包含knn，kNN搜索作为查询已在 8.12 版本中引入: https://www.elastic.co/search-labs/blog/knn-query-elasticsearch + # { + # "size": 3, + # "query": { + # "bool": { + # "should": [ + # { + # "knn": { + # "field": "embedding", + # "query_vector": [2,2,2,0], + # "num_candidates": 10, + # "_name": "knn_query" + # } + # }, + # { + # "match": { + # "description": { + # "query": "luxury", + # "_name": "bm25query" + # } + # } + # } + # ] + # } + # + # knn里面不能包含query。 + # knn和query并列（hybrid search 混合检索），是求或的关系。 + # knn里面可以加filter，比如： "filter": {"match": {"my_label": "red"}} + + if search_type == "match_phrase": + body = { + "query": { + "bool": { + "should": [ + { + "match_phrase": { + "content": { + "query": unique_query, + "boost": weight, + "slop": search_type_attachment + } + } + } for unique_query, weight in unique_queries + ], + "minimum_should_match": 1 + } + } + } + # 纯关键词检索 2 + elif search_type == "match_keywords": + body = { + "query": { + "bool": { + "must": [ + { + "match": { + "content": {"query": core_term, "boost": 1.0} + } + } + ], + "should": [ + { + "match": { + "content": {"query": q, "boost": boost} + } + } for (q, boost) in [(keywords, 1.0), (expand, 0.6), (normalized_query, 1.0)] if q + ], + "minimum_should_match": 1 + } + } + } + # 关键词搜索+向量排序 + elif search_type == "match&boost": + body = { + "query": { + "function_score": { + "score_mode": "sum", + "boost_mode": "multiply", + "query": { + "match": { + "content": { + "query": keywords, + "boost": 1.0 + } + } + }, + "functions": [ + { + "script_score": { + "script": { + "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", + "params": {"query_vector": context.embeddings[0][1]} + } + } + } + ] + } + } + } + # 这个太慢 + elif search_type == "match&boost2": + body = { + "query": { + "script_score": { + "query": { + "match": { + "content": { + "query": keywords, + "boost": 1.0 + } + } + }, + "script": { + "source": "doc['embedding'].isEmpty() ? 1.0 : dotProduct(params.query_vector, 'embedding') + 1.0", + "params": {"query_vector": context.embeddings[0][1]} + } + } + } + } + # 向量搜索+关键词搜索 + elif search_type == "match&knn": + body = { + "query": { + "match": { + "content": { + "query": knn_boost_keywords, + "boost": 1.0 + } + } + }, + "knn": { + "field": "embedding", + "query_vector": context.embeddings[search_type_attachment][1], + "k": 5, + "num_candidates": 10, + "boost": 1.0 + } + } + # 纯向量搜索 + elif search_type == "knn": + body = { + "knn": { + "field": "embedding", + "query_vector": context.embeddings[search_type_attachment][1], + "k": 5, + "num_candidates": 10 + } + } + + need_embedding = (search_type == "match&boost") + need_highlights = (search_type != "knn") + + body["_source"] = {"excludes": ["keywords", "quotes"]} + if not need_embedding: + body["_source"]["excludes"].append("embedding") + # 在填充highlight之前写入search_from + search_from = f'searchtype[{search_type}],param[{search_type_attachment}],body:{body}' + + if need_highlights: + body["_source"]["excludes"] = [] + body["highlight"] = { + "pre_tags": [settings.HIGHTLIGHT_PRE_TAG], + "post_tags": [settings.HIGHTLIGHT_POST_TAG], + "fields": {"chapter_name": {}, "content": {}} + } + + body["size"] = size + + se_debug_info = '' + start_time = time.time() + try: + es_response = context.es.search(index=context.index_name, body=body) + except Exception as e: + se_debug_info = f'Error in executing search: {e}. request: {body}' + return None, se_debug_info + end_time = time.time() + elapsed_time = end_time - start_time + total_hits = es_response.get("hits", {}).get("total", {}).get("value", 0) + returned_hits = len(es_response.get("hits", {}).get("hits", [])) + + if not '"' in search_from: + search_from = search_from.replace('\'', '"') + search_from = search_from if len(search_from) < 400 else search_from[:400] + '...' + + str_body = str(body) + if not '"' in str_body: + str_body = str_body.replace('\'', '"') + se_debug_info = f'({elapsed_time:.2f} seconds. Total: {total_hits}. Returned: {returned_hits}) : {search_from[:400]}' + + if not 'hits' in es_response or not 'hits' in es_response['hits']: + se_debug_info += f' InvalidResponce in executing search: {e}. request: {body}' + return None, se_debug_info + + for hit in es_response['hits']['hits']: + hit['search_from'] = search_from + + return es_response, se_debug_info + +``` + + +# 测试向量： +# [-0.05291186273097992, 0.0274342093616724, -0.016730275005102158, 0.010487289167940617, -0.022640341892838478, -0.048682719469070435, 0.04544096067547798, 0.023079438135027885, 0.007221410982310772, 0.023566091433167458, 0.026696473360061646, 0.08252757787704468, -0.042835772037506104, 0.0009668126585893333, -0.02860398218035698, -0.004426108207553625, -0.002644421299919486, -0.027699561789631844, 0.005749804899096489, -0.04468372091650963, -0.0296687763184309, -0.009487600065767765, 0.020041221752762794, 0.00778265530243516, 0.008522099815309048, 0.03497027978301048, -0.021573258563876152, -0.028293319046497345, -8.54598984005861e-05, -0.03164539486169815, -0.017121458426117897, -0.0006902766763232648, 0.04650883004069328, -0.030234992504119873, -0.010207684710621834, -0.035288386046886444, -0.0047269039787352085, -0.0006454040994867682, -0.056146346032619476, 0.008901881985366344, 0.010757357813417912, -0.013022932223975658, 0.04627145081758499, -0.020669423043727875, -0.02031278982758522, -0.052186835557222366, -0.0148158585652709, -0.018267231062054634, -0.059003304690122604, -0.011793344281613827, 0.027096575126051903, 0.019299808889627457, 0.04161312058568001, -0.019393721595406532, -0.02361445501446724, 0.07711422443389893, -0.02068573422729969, -0.004702702630311251, -0.011135494336485863, 0.0101374052464962, -0.020808257162570953, 0.011924360878765583, -0.020093027502298355, -0.007138500455766916, 0.014727798290550709, 0.05770261213183403, 0.017841406166553497, 0.044339124113321304, -0.01490224339067936, -0.008343652822077274, -0.04842463508248329, 0.0336640290915966, -0.004893577191978693, -0.021536342799663544, -0.032384153455495834, -0.009452177211642265, -0.027460120618343353, -0.009426826611161232, 0.006357531528919935, 0.019494572654366493, 0.009722599759697914, -0.00497430982068181, 0.023032115772366524, 0.05221958085894585, -0.01671120524406433, 0.061740316450595856, -0.06789620220661163, -0.023851843550801277, -0.02249223366379738, -0.01231105625629425, -0.0499565526843071, 0.004251780919730663, 0.05466651916503906, -0.024449756368994713, -0.034151963889598846, 0.037387508898973465, -0.0016276679234579206, -0.02609393745660782, 0.01800747588276863, -0.0028136332985013723, -0.06036405637860298, 0.028903907164931297, 0.006318055558949709, 0.012870929203927517, -0.0021476889960467815, -0.012034566141664982, -0.008372323587536812, 0.024942906573414803, 0.08258169889450073, 0.006757829803973436, 0.032017264515161514, -0.012414710596203804, 0.014826267957687378, -0.040858786553144455, -0.0060302577912807465, 0.00843990221619606, -0.031066348776221275, -0.06313654035329819, 0.0056659989058971405, -0.007768781390041113, 0.011673268862068653, 0.007261875085532665, 0.006112886127084494, -0.07374890148639679, 0.06602894514799118, -0.05385972931981087, -0.0010994652984663844, 0.05939924344420433, 0.015503636561334133, 0.034621711820364, 0.008040975779294968, -0.023962488397955894, -0.06270411610603333, 0.00027893096557818353, -0.0436306893825531, -0.006309020332992077, 0.02416943572461605, -0.015391307882964611, -0.012442439794540405, -0.003181715961545706, -0.0021985983476042747, 0.008671553805470467, 0.004063367377966642, -0.02560708485543728, 0.03469422832131386, -0.04249674826860428, -0.013552767224609852, -0.052823010832071304, 0.014670411124825478, -0.011493593454360962, 0.024076055735349655, 0.056352417916059494, -0.008510314859449863, 0.015936613082885742, 0.003935575485229492, 0.0037949192337691784, 0.015074086375534534, 0.016583971679210663, -0.0057802870869636536, 0.005751866847276688, -0.009386995807290077, -0.03710195794701576, -0.03144300729036331, -0.07106415182352066, -0.003882911056280136, -0.010697683319449425, -0.014338435605168343, 0.007036983501166105, -0.035716522485017776, 0.06593189388513565, 0.007752529811114073, -0.030261363834142685, -0.02513342909514904, -0.039278656244277954, 0.015320679172873497, -0.012659071013331413, 0.014207725413143635, 0.010264124721288681, 0.01617652177810669, -0.022644126787781715, -0.031033707782626152, 0.04160666465759277, -0.05329348146915436, 0.02423500455915928, -0.019389694556593895, 0.008645910769701004, -0.005958682857453823, -0.03648180514574051, 0.011972597800195217, 0.037404924631118774, -0.007001751102507114, -0.05138246342539787, 0.0013400549069046974, -0.03268183395266533, 0.07687076926231384, -0.02033335529267788, -0.020667986944317818, 0.0038236891850829124, 0.029960744082927704, 0.015430699102580547, 0.05047214776277542, 0.0052254535257816315, 0.013995353132486343, -0.031164521351456642, -0.014291719533503056, 0.015829795971512794, -0.0013409113744273782, -0.044300951063632965, 0.045415859669446945, -0.005037966184318066, -0.03883415088057518, 0.027200160548090935, 0.008182630874216557, -0.046456750482320786, -0.029778052121400833, 0.02067168429493904, -0.006381513085216284, -0.04693000763654709, 0.009974686428904533, 0.03109011799097061, -0.012696364894509315, 0.030124813318252563, 0.02372679114341736, 0.06566771119832993, 0.03553507477045059, -0.032816141843795776, 0.028003521263599396, 0.06498659402132034, -0.013530750758945942, 0.0312667116522789, -0.015660811215639114, -0.00776742585003376, -0.004829467739909887, -0.015968922525644302, 0.04765664413571358, -0.0026502758264541626, 0.01891564577817917, 0.04119837284088135, 0.012158435769379139, 0.008338023908436298, -0.006039333995431662, 0.0630166307091713, -0.02758428454399109, 0.029347822070121765, -0.030129415914416313, 0.023165738210082054, 0.04064684361219406, 0.04446929693222046, -0.006133638322353363, -0.013095719739794731, -0.041152223944664, -0.01038535125553608, 0.01738007925450802, 0.0010595708154141903, -0.055003564804792404, 0.036829687654972076, -0.030270753428339958, -0.009607627056539059, 0.014103117398917675, 0.005140293389558792, 0.032931022346019745, 0.026972685009241104, -0.00039128100615926087, 0.00550195062533021, 0.062454141676425934, 0.02344602160155773, -0.01688288524746895, 0.011600837111473083, 0.009648085571825504, 0.012827200815081596, 0.02368510514497757, -0.044808436185121536, 0.006574536208063364, 0.03677171841263771, 0.021754244342446327, -0.0031720376573503017, -0.03498553857207298, -0.027119319885969162, 0.05196662247180939, 0.0063033513724803925, -0.002766692778095603, -0.03879206255078316, -0.005737128667533398, -0.02351462095975876, 0.04338989034295082, -0.03623301535844803, 0.003727369010448456, 0.044172726571559906, 0.06180792301893234, -0.025736358016729355, 0.01280374638736248, -0.01768171414732933, 0.0413120836019516, 0.036350950598716736, 0.020034022629261017, -0.00938474852591753, -0.04920303076505661, -0.1626604050397873, 0.0016566020203754306, -0.010797491297125816, 0.0037245014682412148, 0.039030417799949646, -0.009399985894560814, 0.016659803688526154, -0.047097429633140564, -0.00987484585493803, 0.020634479820728302, 0.005361238028854132, -0.05283225327730179, 0.002501025330275297, -0.004766151309013367, 0.00850654486566782, -0.0050267502665519714, -0.046555373817682266, 0.012670878320932388, 0.0018581973854452372, -0.010647253133356571, 0.01990092545747757, 0.02013244479894638, 0.04490885138511658, 0.029433563351631165, -0.01408607978373766, 0.029722925275564194, 0.04512600228190422, -0.04305345192551613, 0.0053901285864412785, -0.010685979388654232, 0.01516974437981844, 0.02340293675661087, -0.014181641861796379, -0.0013334851246327162, 0.020624764263629913, 0.06469231843948364, 0.016654038801789284, -0.043994754552841187, 0.025707466527819633, -0.004160136915743351, 0.021129926666617393, 0.041262850165367126, 0.006293899845331907, 0.056005991995334625, -0.006883381400257349, -0.07502268254756927, -0.02920101210474968, -0.019043054431676865, 0.00737513042986393, 0.013621360063552856, -0.02504715882241726, -0.01138006430119276, -0.010744514875113964, -0.02502342313528061, -0.03335903584957123, 0.012180354446172714, -0.03276645019650459, 0.05202409625053406, 0.03246080502867699, 0.03068908303976059, -0.029587913304567337, -0.04850265011191368, -0.006388102192431688, -0.03203853219747543, -0.050761956721544266, -0.021925227716565132, 0.036384399980306625, -0.011895880103111267, -0.007408954203128815, -0.012625153176486492, 0.0024322718381881714, -0.012196220457553864, -0.007011729292571545, -0.0337890200316906, -0.030034994706511497, 0.04638829082250595, -0.028362803161144257, -0.01176459901034832, 0.00956833828240633, -0.12054562568664551, -0.020540419965982437, 0.014624865725636482, -0.025515791028738022, -0.005027926992624998, -0.03586679324507713, -0.05585843697190285, -0.01700599677860737, -0.00044939795043319464, 0.029278729110956192, 0.25503888726234436, -0.024952411651611328, 0.005794796161353588, -0.007252118084579706, 0.03397773951292038, -0.0030146583449095488, -0.016645856201648712, -0.0008194005931727588, 0.02789629064500332, -0.039116114377975464, -0.035631854087114334, 0.04917449131608009, -0.006455820053815842, -0.011818122118711472, -0.00958359707146883, 0.013176187872886658, 0.037286531180143356, 0.022334400564432144, 0.05832865461707115, 0.010104321874678135, -0.04915979504585266, -0.022671189159154892, -0.016606582328677177, -0.007431587669998407, 0.0025214774068444967, -0.038979604840278625, 0.014895224012434483, 0.03583076596260071, 0.0006473385728895664, 0.04958082735538483, -0.017827684059739113, 0.015710417181253433, 0.062094446271657944, -0.014381879940629005, 0.0002880772517528385, 0.004948006477206945, -8.711735063116066e-06, -0.0029445397667586803, -0.044325683265924454, 0.047702621668577194, -0.03197811171412468, -0.02109563909471035, 0.03041824884712696, 0.021582895889878273, -0.004118872340768576, -0.025784745812416077, 0.06275995075702667, 0.006879465654492378, 0.04185185581445694, 0.02031264826655388, -0.02274201810359955, -0.009617358446121216, -0.04315454140305519, -0.033287111669778824, -0.025126483291387558, -0.003923895303159952, -0.041508499532938004, -0.0009355457150377333, -0.033565372228622437, 0.02229289337992668, -0.0026574484072625637, -0.0028596664778888226, -0.02223617024719715, -0.016868866980075836, 0.04172029718756676, 0.0014162511797621846, -0.037737537175416946, -0.010155809111893177, -0.010357595980167389, 0.04541466012597084, 0.03563382104039192, -0.019189776852726936, -0.012577632442116737, -0.013781189918518066, 0.026566311717033386, 0.020911909639835358, 0.02781282551586628, 0.053938526660203934, 0.0194545891135931, 0.0015139722963795066, -0.0357731431722641, -0.005088387057185173, 0.004257760010659695, 0.04332628846168518, -0.012149352580308914, -0.04734082147479057, 0.018029984086751938, -0.01322091929614544, -0.059820450842380524, -0.03677783161401749, -0.006745075341314077, -0.02209635078907013, -0.012663901783525944, -0.0059855030849576, 0.016270749270915985, -0.00725028058513999, 0.03019685670733452, 0.010252268984913826, -0.06314245611429214, -0.005512078758329153, -0.016377074643969536, -0.0014438428916037083, 0.029021194204688072, -0.015355946496129036, 0.02559172362089157, -0.04241044819355011, 0.010147088207304478, -0.016036594286561012, 0.023162752389907837, 0.047236304730176926, 0.0166736152023077, 0.01226564310491085, -0.015224735252559185, -0.01298521552234888, -0.008012642152607441, 0.028470756486058235, -0.013741613365709782, 0.019896863028407097, 0.01720179058611393, 0.01571199856698513, 0.030143165960907936, -0.02969514951109886, 0.014739652164280415, -0.01854291744530201, -0.045576371252536774, -0.04516203701496124, 0.02147211693227291, 0.007073952350765467, 0.008106761611998081, -0.01828523352742195, 0.002731812885031104, -0.04545339569449425, 0.019007619470357895, 0.03504781052470207, 0.037705861032009125, -0.0045634908601641655, 0.0070000626146793365, 0.0037205498665571213, 0.005224148277193308, -0.017060590907931328, -0.04246727377176285, -0.006265614647418261, -0.015374364331364632, -0.03380871191620827, -0.005029333755373955, 0.007065227720886469, 0.003886009333655238, 0.008613690733909607, -0.012133199721574783, 0.005556005053222179, -0.021959641948342323, 0.04834386706352234, 0.03787781298160553, -0.057815466076135635, 0.015909207984805107, -0.03855409845709801, 0.0018244135426357388, 0.04186264052987099, -0.054983459413051605, 0.006219237111508846, 0.03494301065802574, 0.023722950369119644, 0.0312604121863842, 0.05597991123795509, -0.030345493927598, 0.016615940257906914, -0.0207205917686224, 0.055960651487112045, -0.012713379226624966, -0.0261109359562397, 0.014332456514239311, -0.017245708033442497, -0.06636268645524979, 0.00592504907399416, 0.04649018123745918, -0.018362276256084442, 0.009620632976293564, -0.0044480785727500916, -0.0014729035319760442, 0.015621249563992023, 0.0367378331720829, -0.011857259087264538, -0.045088741928339005, 0.0006832792423665524, 0.02601524256169796, -0.02120809443295002, 0.018104318529367447, 0.008069046773016453, 0.013658273033797741, 0.004183551296591759, -0.04133244603872299, 0.05436890944838524, 0.009334285743534565, -0.014695074409246445, -0.011054124683141708, 0.009796642698347569, -0.008759389631450176, -0.06399217247962952, -0.0028859861195087433, -0.008736967109143734, -0.003506746841594577, 0.008123806677758694, 0.008794951252639294, -0.02940259501338005, 0.009597218595445156, -0.02197900228202343, -0.02082076109945774, 0.023915970697999, -0.059058744460344315, -0.010253551416099072, 0.024443935602903366, -0.029604850336909294, 0.008135135285556316, 0.03568771481513977, -0.017330091446638107, -0.003135789418593049, 0.035103678703308105, 0.0370408296585083, -0.01022601593285799, -0.045891791582107544, 0.01726667769253254, -0.008570673875510693, 0.015297998674213886, -0.015412220731377602, -0.01425748411566019, 0.031544867902994156, 0.013110813684761524, -0.057211123406887054, -0.0008968000765889883, 0.001981658162549138, -0.002101168967783451, -0.09516698867082596, -0.034693196415901184, 0.011157260276377201, 0.010063023306429386, -0.02550840750336647, 0.009959851391613483, 0.022281678393483162, -0.03908146917819977, 0.02196437120437622, 0.03520793840289116, -0.06856158375740051, -0.004901218693703413, 0.1122148334980011, -0.01498009730130434, 0.03165500983595848, -0.07618033140897751, -0.014297851361334324, 0.02150021120905876, 0.005999598652124405, -0.013493427075445652, 0.013868110254406929, 0.00079053093213588, 0.006475066766142845, 0.000955471652559936, -0.03403160721063614, -0.02295752801001072, 0.0041635241359472275, -0.03955964744091034, -0.04943346977233887, 0.00032474088948220015, 0.039174411445856094, -0.011974001303315163, 0.008057610131800175, 0.03809700161218643, -0.041719768196344376, 0.037615906447172165, -0.035932306200265884, 0.008293192833662033, -0.03261689469218254, -0.023902395740151405, -7.811257091816515e-05, -0.011328466236591339, -0.026476409286260605, 0.055370282381772995, 0.03128054738044739, -0.014991461299359798, 0.017835773527622223, 0.01642710715532303, 0.029273470863699913, -0.012139911763370037, 0.01371818222105503, -0.013113478198647499, -0.04071088507771492, 0.0233455840498209, -0.019497444853186607, -0.01747158169746399, 0.02493683062493801, 0.024074571207165718, -0.03614620864391327, -0.025289475917816162, -0.04030011221766472, -0.046772539615631104, 0.009969661012291908, 0.003724620910361409, 0.007474626414477825, -0.04855594411492348, 0.04697829484939575, 0.010695616714656353, 0.027944304049015045, -0.003937696572393179, -0.011591222137212753, -0.011533009819686413, 0.03215765953063965, -0.04699324443936348, -9.356102236779407e-05, -0.01535400003194809, -0.010238519869744778, 0.002703386126086116, 0.04759520664811134, 0.0074842446483671665, -0.04050430282950401, -0.028402622789144516, -0.03205197677016258, 0.011288953013718128, 0.006053865421563387, 0.04641448333859444, 0.005652922671288252, -0.018560705706477165, 0.02581481821835041, 0.00962467584758997, -0.017888177186250687, -0.026476262137293816, -0.005547264125198126, 0.012222226709127426, -0.004069746006280184, -0.020438821986317635, 0.01929863728582859, -0.0053736320696771145, 0.02221786603331566, -0.007175051141530275, 0.003961225971579552, -0.012380941770970821, -0.0040277824737131596, 0.009086307138204575, 0.012202796526253223, 0.018483169376850128, 0.017530532553792, 0.0422886498272419, 0.04987001419067383, 0.003722204128280282, 0.06421508640050888, -0.016258088871836662, -0.027659112587571144, 0.004458434879779816, -0.02898143045604229, -0.014475414529442787, 0.032039571553468704, -0.025734663009643555, -0.01585981249809265, 0.04900333285331726, -0.06422552466392517, -0.0007134959450922906, -0.04035528376698494, 0.03290264680981636, -0.0018848407780751586, 0.0068516512401402, 0.00032433189335279167, -0.002669606124982238, -0.017596688121557236, -0.026878179982304573, 0.014075388200581074, 0.020072080194950104, -0.00295435544103384, -0.01918656937777996, -0.007689833641052246, 0.039347097277641296, 0.0026605715975165367, 0.011779646389186382, 0.04189120978116989, -0.03846775367856026, -0.01993645168840885, 0.04546443000435829, 0.05682912468910217, -0.012384516187012196, -0.004507445730268955, 0.007476931903511286, -0.01160018052905798, 0.006559243891388178, 0.04354899004101753, 0.006185194011777639, 0.028355205431580544, -0.006518798414617777, -0.029528537765145302, 0.06740271300077438, -0.052158474922180176, 0.0025031850673258305, -0.005957300774753094, 0.00500349560752511, 0.022637680172920227, -0.0027129461523145437, -0.011677206493914127, -0.042732879519462585, -0.0021236639004200697, -0.1499215066432953, 0.02914350852370262, -0.031246500089764595, -0.027244996279478073, -0.006904688663780689, 0.01088196225464344, 0.01271661464124918, -0.0430884025990963, -0.020760131999850273, -0.006593034137040377, -0.0007962957606650889, -0.031729113310575485, 0.052976224571466446, -0.03149586543440819, 0.0392388291656971, 0.023318620398640633, -0.01383691094815731, 0.02858218550682068, 0.023135144263505936, 0.026421336457133293, 0.00027594034327194095, -0.03901490569114685, 0.008533132262527943, -0.03802476078271866, -0.011105065234005451, -0.028275510296225548, 0.04846742004156113, 0.021237077191472054, -0.027375172823667526, -0.02717825025320053, -0.031243441626429558, -0.021638689562678337, 0.024066096171736717, 0.05689090117812157, -0.04352620989084244, 0.03599394112825394, 0.05153508856892586, 0.002263782313093543, 0.047110624611377716, 0.006084555760025978, 0.003244618885219097, -0.0015037712873890996, 0.027960799634456635, -0.013650861568748951, 0.03281615301966667, 0.012363187968730927, 0.02162906341254711, -0.010951842181384563, -0.02786285988986492, 0.03754381462931633, 0.01957041770219803, -0.017010418698191643, -0.008339766412973404, 0.0755641758441925, 0.023412147536873817, -0.005748848430812359, -0.05465301498770714, -0.02190011739730835, 0.0054182386957108974, 0.032733004540205, -0.05342638120055199, 0.009907999075949192, -0.02370712347328663, -0.015652501955628395, -0.011254304088652134, -0.019827252253890038, -0.021032121032476425, -0.02607329562306404, -0.0008710312540642917, -0.06800976395606995, -0.017296750098466873, 0.015312970615923405, -0.015649013221263885, -0.016449443995952606, -0.012058117426931858, 0.002104945247992873, 0.020476385951042175, 0.014795565977692604, -0.02145536057651043, -0.028734024614095688, -0.041212357580661774, -0.008211270906031132, 0.033569078892469406, -0.0033273063600063324, -0.02339683100581169, 0.0421740785241127, -0.009677124209702015, -0.006869456730782986, -0.016001028940081596, 0.029614608734846115, -0.06062136963009834, -0.011824233457446098, 0.012096629478037357, -0.028248939663171768, -0.03703905642032623, 0.012119539082050323, -0.041021380573511124, 0.01975782960653305, -0.028443211689591408, 0.020459437742829323, 0.0073023103177547455, -0.06498327851295471, -0.004016770515590906, 0.06460512429475784, -0.053343966603279114, 0.03865537419915199, -5.4113028454594314e-05, -0.008642046712338924, -0.009384138509631157, -0.037736788392066956, -0.035090748220682144, 0.018596891313791275, -0.008763385005295277, 0.040228284895420074, 0.03811536356806755, -0.034618355333805084, -0.004665717948228121, 0.04813361540436745, -0.004303373862057924, 0.00795511994510889, -0.017838604748249054, 0.00563138909637928, -0.03171280398964882, -0.0259436946362257, 0.004301885142922401, -0.02739236131310463, 0.03270035237073898, 0.009064823389053345, -0.0363747663795948, 0.02325567975640297, 0.03453107923269272, -0.012906554155051708, 0.028347544372081757, 0.01234712265431881, 0.030589573085308075, 0.0024874424561858177, -0.0173872709274292, 0.0247347354888916, 0.004171399865299463, 0.02350561134517193, -0.05499064922332764, -0.023146219551563263, -0.012485259212553501, -0.0228674728423357, 0.013267520815134048, 0.021304689347743988, -0.018937893211841583, -0.0260267723351717, -0.022532619535923004, 0.0030378480441868305, -0.008528024889528751, -0.030528495088219643, -0.009305189363658428, -0.0074027362279593945, -0.020641637966036797, 0.006984233390539885, 0.04300186410546303, -0.033014994114637375, -0.006089311558753252, 0.04753036051988602, -0.036625705659389496, -0.04691743850708008, -0.007467558141797781, 0.0652017593383789, -0.03861508145928383, -0.00741452956572175, 0.003471594536677003, 0.016132064163684845, 0.01570185460150242, 0.018733495846390724, -0.019025148823857307, 0.003490244736894965, -0.017714614048600197, -0.003447450464591384, 0.015267218463122845, 0.015076974406838417, -0.002631498035043478, 0.005311752203851938, 0.014075293205678463, 0.0026123111601918936, 0.011874910444021225, 0.0714355856180191, 0.06941138952970505, 0.022251378744840622, 0.01972009800374508, 0.04719123989343643, 0.023544959723949432, 0.017852554097771645, 0.01843070052564144, -0.05294886603951454, -0.008682304993271828, 0.010625398717820644, 0.0428495928645134, 0.002173527143895626, 0.06291069090366364, 0.024296458810567856, 0.008714474737644196, 0.06520587205886841, 0.015627536922693253, 0.04247526824474335, 0.0009774811333045363, 0.00738496845588088, -0.024803027510643005, 0.013228596188127995, -0.037615202367305756, -0.028807995840907097, 0.012890785001218319, -0.01587829552590847, -0.01928863860666752, 0.0011809614952653646, -0.026926854625344276, -0.020252779126167297, -0.010968486778438091, -0.015348547138273716, 0.008559435606002808, -0.009286923334002495, 0.0014621232403442264, 0.03831499442458153, 0.016517579555511475, 0.037184324115514755, -0.041231196373701096, 0.03757374733686447, -0.039465345442295074, -0.04308579862117767, 0.0011091071646660566, -0.029794104397296906, 0.008459310978651047, -0.01713281124830246, -0.016625113785266876, -0.05582521855831146, -0.0415986105799675, 0.028725938871502876, 0.04966316372156143, 0.012718678452074528, -0.025533588603138924, 0.013822318986058235, -5.168768620933406e-05, 0.02616700902581215, -0.06113629788160324, -0.03175340220332146, 0.03593592345714569, -0.04014921560883522, -0.020605407655239105, 0.02186705358326435] ``` \ No newline at end of file diff --git a/docs/ES/ES_8.18/2_kibana安装.md b/docs/ES/ES_8.18/2_kibana安装.md index cfef31b..923a1ee 100644 --- a/docs/ES/ES_8.18/2_kibana安装.md +++ b/docs/ES/ES_8.18/2_kibana安装.md @@ -1,51 +1,51 @@ - - - - - -## 1. yum安装 -添加yum仓库。 -Kibana通常不是yum默认仓库的一部分，因此需要添加 Elastic 仓库：： -```shell - -导入 Elastic 签名密钥： -sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch - -添加 Elastic 仓库： -sudo tee /etc/yum.repos.d/elastic.repo <