(APIServer pid=1) INFO: 10.0.13.43:46412 - "POST /v1/chat/completions HTTP/1.1" 200 OK (APIServer pid=1) INFO: 10.0.13.43:46386 - "POST /v1/chat/completions HTTP/1.1" 200 OK (APIServer pid=1) INFO 03-27 10:49:04 [loggers.py:127] Engine 000: Avg prompt throughput: 975.5 tokens/s, Avg generation throughput: 466.9 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 8.8% (APIServer pid=1) INFO 03-27 10:49:14 [loggers.py:127] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 8.8% (APIServer pid=1) INFO 03-27 10:49:24 [loggers.py:127] Engine 000: Avg prompt throughput: 2298.1 tokens/s, Avg generation throughput: 1.2 tokens/s, Running: 13 reqs, Waiting: 14 reqs, GPU KV cache usage: 1.8%, Prefix cache hit rate: 8.4% loc("/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/_unifietritond_attention.py":215:66): error: operation scheduled before its operands loc("/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py":215:66): error: operation scheduled before its operands loc("/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py":215:66): error: operation scheduled before its operands loc("/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py":215:66): error: operation scheduled before its operands (APIServer pid=1) INFO 03-27 10:49:34 [loggers.py:127] Engine 000: Avg prompt throughput: 4552.6 tokens/s, Avg generation throughput: 2.8 tokens/s, Running: 43 reqs, Waiting: 7 reqs, GPU KV cache usage: 5.1%, Prefix cache hit rate: 10.1% (APIServer pid=1) INFO: 10.0.13.43:46496 - "POST /v1/chat/completions HTTP/1.1" 200 OK (APIServer pid=1) INFO: 10.0.13.43:46452 - "POST /v1/chat/completions HTTP/1.1" 200 OK