应用场景需要处理一些较长的上下文,于是配置启动参数
```bash
--max-num-batched-tokens 8192
--enable-prefix-caching
```
实测输入较短提示词时,推理速度尚可,但多轮对话等场景,随着上下文逐渐增长,推理速度大幅下降,观察日志发现Prefix cache hit rate始终为0,做为测试,用完全一致的提示词多次调用也依然是0:
```bash
(APIServer pid=40) INFO 03-04 16:00:32 [loggers.py:257] Engine 000: Avg prompt throughput: 4228.4 tokens/s, Avg generation throughput: 1.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
...
(APIServer pid=40) INFO 03-04 16:00:52 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.1%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
...
```
请问这是由于Prefix缓存不生效导致的长上下文性能问题么?