vLLM is an inference and serving engine for large language models (LLMs). From version 0.5.5 to before 0.11.1, the /v1/chat/completions and /tokenize endpoints allow a chat_template_kwargs request parameter that is used in the code before it is properly validated against the chat template. With the right chat_template_kwargs parameters, it is possible to block processing of the API server for long periods of time, delaying all other requests. This issue has been patched in version 0.11.1.

Affected: PyPI: vllm

### Summary The /v1/chat/completions and /tokenize endpoints allow a `chat_template_kwargs` request parameter that is used in the code before it is properly validated against the chat template. With the right `chat_template_kwargs` parameters, it is possible to block processing of the API server for long periods of time, delaying all other requests

### Details In serving_engine.py, the chat_template_kwargs are unpacked into kwargs passed to chat_utils.py `apply_hf_chat_template` with no validation on the keys or values in that chat_template_kwargs dict. This means they can be used to override optional parameters in the `apply_hf_chat_template` method, such as `tokenize`, changing its default from False to True.

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/openai/serving_engine.py#L809-L814

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/chat_utils.py#L1602-L1610

Both serving_chat.py and serving_tokenization.py call into this `_preprocess_chat` method of `serving_engine.py` and they both pass in `chat_template_kwargs`.

So, a `chat_template_kwargs` like `{"tokenize": True}` makes tokenization happen as part of applying the chat template, even though that is not expected. Tokenization is a blocking operation, and with sufficiently large input can block the API server's event loop, which blocks handling of all other requests until this tokenization is complete.

This…

Affected: PyPI: vllm

### Summary The /v1/chat/completions and /tokenize endpoints allow a `chat_template_kwargs` request parameter that is used in the code before it is properly validated against the chat template. With the right `chat_template_kwargs` parameters, it is possible to block processing of the API server for long periods of time, delaying all other requests

### Details In serving_engine.py, the chat_template_kwargs are unpacked into kwargs passed to chat_utils.py `apply_hf_chat_template` with no validation on the keys or values in that chat_template_kwargs dict. This means they can be used to override optional parameters in the `apply_hf_chat_template` method, such as `tokenize`, changing its default from False to True.

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/openai/serving_engine.py#L809-L814

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/chat_utils.py#L1602-L1610

Both serving_chat.py and serving_tokenization.py call into this `_preprocess_chat` method of `serving_engine.py` and they both pass in `chat_template_kwargs`.

So, a `chat_template_kwargs` like `{"tokenize": True}` makes tokenization happen as part of applying the chat template, even though that is not expected. Tokenization is a blocking operation, and with sufficiently large input can block the API server's event loop, which blocks handling of all other requests until this tokenization is complete.

This…

ray: Ray Job Submission Arbitrary Code Execution libtiff: TIFFRasterScanlineSize64 produce too-big size and could cause OOM libtiff: Segment fault in libtiff in TIFFReadRGBATileExt() leading to denial of service shadow-utils: Default subordinate ID configuration in /etc/login.defs could lead to compromise libssh: out-of-bounds read in sftp_handle() vllm: Server Side request forgery (SSRF) in MediaConnector sqlite: Integer Truncation in SQLite libtiff: LibTIFF Use-After-Free Vulnerability openssl: Out-of-bounds read & write in RFC 3211 KEK Unwrap libxslt: libxml2: Inifinite recursion at exsltDynMapFunction function in libexslt/dynamic.c libtiff: Libtiff Write-What-Where openssl: OpenSSL: Remote code execution or Denial of Service via oversized Initialization Vector in CMS parsing golang.org/x/oauth2/jws: Unexpected memory consumption during token parsing in golang.org/x/oauth2/jws golang.org/x/crypto/ssh: Denial of Service in the Key Exchange of golang.org/x/crypto/ssh os/exec: Unexpected paths returned from LookPath in os/exec runc: container escape with malicious config due to /dev/console mount and related races vim: Vim path traversial vim: Vim path traversal firefox: thunderbird: expat: libexpat in Expat allows attackers to trigger large dynamic memory allocations via a small document that is submitted for parsing vllm: Timing Attack in vLLM API Token Verification Leading to Authentication Bypass vllm: vLLM OpenAI-Compatible Server Resource Exhaustion via chat_template…