光合开发者社区-论坛

网

网友P2hf1T

3楼

你好，这个镜像那里下载的

2025/6/5

网

网友iPzjBx

回复 @网友P2hf1T

docker pull就行

2025/6/6

狂

狂风暴雨

2楼

#要完全禁用思考，您可以在启动模型时使用自定义聊天模板：

--chat-template ./qwen3_nonthinking.jinja

2025/5/22

网

网友V84ctl

回复 @狂风暴雨

qwen3_nonthinking.jinja的具体内容可以分享吗

2025/5/27

狂

狂风暴雨

回复 @狂风暴雨

{%- if tools %}

{%- if messages[0].role == 'system' %}

{%- endif %}

{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}

{%- for tool in tools %}

{%- endfor %}

{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}

{%- else %}

{%- if messages[0].role == 'system' %}

{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}

{%- endif %}

{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}

{%- for message in messages[::-1] %}

{%- set index = (messages|length - 1) - loop.index0 %}

{%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}

{%- set ns.multi_step_tool = false %}

{%- set ns.last_query_index = index %}

{%- endif %}

{%- endfor %}

{%- for message in messages %}

{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}

{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}

{%- elif message.role == "assistant" %}

{%- set content = message.content %}

{%- set reasoning_content = '' %}

{%- if message.reasoning_content is defined and message.reasoning_content is not none %}

{%- set reasoning_content = message.reasoning_content %}

{%- else %}

{%- if '</think>' in message.content %}

{%- set content = message.content.split('</think>')[-1].lstrip('\n') %}

{%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}

{%- endif %}

{%- if loop.index0 > ns.last_query_index %}

{%- if loop.last or (not loop.last and reasoning_content) %}

{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}

{%- else %}

{%- endif %}

{%- else %}

{%- endif %}

{%- if message.tool_calls %}

{%- for tool_call in message.tool_calls %}

{%- if (loop.first and content) or (not loop.first) %}

{%- endif %}

{%- if tool_call.function %}

{%- set tool_call = tool_call.function %}

{%- endif %}

{{- '<tool_call>\n{"name": "' }}

{%- if tool_call.arguments is string %}

{%- else %}

{%- endif %}

{{- '}\n</tool_call>' }}

{%- endfor %}

{%- endif %}

{%- elif message.role == "tool" %}

{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}

{%- endif %}

{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}

{%- endif %}

{%- endfor %}

{%- if add_generation_prompt %}

{%- endif %}

2025/5/29

Y

yangzhong

1楼

可以通过环境变量控制流式输出，

创建一个test.py脚本，内容如下

import os

import subprocess

# 设置环境变量禁用流式输出

os.environ["VLLM_DISABLE_STREAMING"] = "1"

# 启动vLLM服务器

subprocess.run([

"python3",

"-m",

"vllm.entrypoints.openai.api_server",

"--served-model-name", "Qwen3-32B",

"--model", "/models/Qwen/Qwen3-32B",

"--host", "0.0.0.0",

"--port", "8000",

"--tensor-parallel-size", "4",

"--max_model_len", "32768",

"--enforce-eager",

])

然后运行python3 test.py

2025/5/20