# Local Translation Model Focused Benchmark

- Generated at: `2026-03-18T23:45:50`
- Python: `3.12.3`
- Torch: `2.10.0+cu128`
- Transformers: `5.3.0`
- CUDA: `True`
- GPU: `Tesla T4` (15.56 GiB)

## Scope

- Scenario 1: high batch size + low concurrency
- Scenario 2: high concurrency + low batch size
- Variants in this report:
  - `ct2_default`: `{}`
  - `ct2_tuned_t4`: `{'ct2_inter_threads': 2, 'ct2_max_queued_batches': 16, 'ct2_batch_type': 'examples'}`

## nllb-200-distilled-600m zh->en

- Direction: `zh -> en`
- Column: `title_cn`

### Variant `ct2_default`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 81.26 | 458.67 | 393.78 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 113.25 | 662.38 | 565.09 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 142.43 | 959.47 | 898.64 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 11.24 | 919.46 | 599.51 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 10.34 | 1904.9 | 1124.8 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 10.17 | 2876.88 | 1495.51 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 10.32 | 2837.64 | 1442.59 |

### Variant `ct2_tuned_t4`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 79.44 | 464.07 | 402.81 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 111.86 | 657.84 | 572.15 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 137.56 | 994.04 | 930.45 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 11.65 | 778.66 | 596.33 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 12.91 | 1368.92 | 902.12 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 12.8 | 2350.5 | 1237.45 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 11.06 | 2729.36 | 1413.83 |

## nllb-200-distilled-600m en->zh

- Direction: `en -> zh`
- Column: `title`

### Variant `ct2_default`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 70.7 | 481.89 | 452.61 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 96.27 | 701.75 | 664.81 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 114.27 | 1137.6 | 1120.15 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 5.54 | 1850.06 | 1287.52 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 5.51 | 4613.05 | 2252.26 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 5.46 | 5554.4 | 3022.86 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 5.47 | 5514.61 | 3035.08 |

### Variant `ct2_tuned_t4`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 67.87 | 499.47 | 471.45 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 93.36 | 721.79 | 685.53 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 115.0 | 1126.53 | 1113.05 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 7.9 | 1138.18 | 905.78 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 7.91 | 2039.17 | 1555.46 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 7.9 | 3912.75 | 2119.17 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 6.61 | 4565.12 | 2434.92 |

## opus-mt-zh-en zh->en

- Direction: `zh -> en`
- Column: `title_cn`

### Variant `ct2_default`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 130.2 | 544.94 | 245.76 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 164.1 | 597.69 | 389.99 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 196.91 | 768.55 | 650.03 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 29.4 | 324.88 | 230.83 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 28.26 | 693.67 | 415.98 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 27.5 | 1049.24 | 572.84 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 28.43 | 994.92 | 520.92 |

### Variant `ct2_tuned_t4`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 120.2 | 582.58 | 266.21 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 151.21 | 635.63 | 423.24 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 196.63 | 761.85 | 650.95 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 30.43 | 361.76 | 239.25 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 28.32 | 629.6 | 423.42 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 29.83 | 994.19 | 573.64 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 26.85 | 1092.87 | 612.05 |

## opus-mt-en-zh en->zh

- Direction: `en -> zh`
- Column: `title`

### Variant `ct2_default`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 78.22 | 626.96 | 409.09 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 114.34 | 699.88 | 559.7 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 162.29 | 796.29 | 788.66 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 11.25 | 798.77 | 489.06 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 11.22 | 1759.07 | 978.67 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 11.48 | 2453.59 | 1101.78 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 11.35 | 2447.81 | 1116.99 |

### Variant `ct2_tuned_t4`

| Scenario | Setting | Items/s | Req p95 ms | Avg req ms |
|---|---|---:|---:|---:|
| high-batch/low-concurrency | batch=32, concurrency=1 | 79.33 | 620.07 | 403.35 |
| high-batch/low-concurrency | batch=64, concurrency=1 | 121.87 | 634.29 | 525.15 |
| high-batch/low-concurrency | batch=128, concurrency=1 | 210.29 | 623.45 | 608.65 |
| high-concurrency/low-batch | batch=1, concurrency=8 | 15.94 | 1199.98 | 334.6 |
| high-concurrency/low-batch | batch=1, concurrency=16 | 12.65 | 1683.16 | 744.33 |
| high-concurrency/low-batch | batch=1, concurrency=32 | 12.6 | 2435.98 | 974.56 |
| high-concurrency/low-batch | batch=1, concurrency=64 | 12.09 | 2433.63 | 960.19 |
