# Optional: HanLP query tokenization for the main backend venv (QueryParser). # # Install: # source activate.sh # pip install -r requirements_hanlp.txt # # Why pin transformers<5: # transformers 5.x no longer exposes `encode_plus` on `BertTokenizer`, but HanLP 2.1.x # still calls it → AttributeError during `hanlp.load(...)`. # Use transformers 4.44+ (4.x) which remains API-compatible with HanLP. hanlp>=2.1.0 transformers>=4.44,<5