orkes.services.vLLMConnection#

class orkes.services.vLLMConnection(url, model_name=<class 'str'>, headers=None, api_key=None)[source]#

LEGACY: This class is maintained for backward compatibility only.

Deprecated since version 0.1.0: Use LLMFactory for prebuilt connections or create your own connection using UniversalLLMClient. vLLMConnection uses the OpenAI-compatible format which is being phased out.

Parameters:

  • url (str)

  • headers (Dict[str, str] | None)

__init__(url, model_name=<class 'str'>, headers=None, api_key=None)[source]#
Parameters:

  • url (str)

  • headers (Dict[str, str] | None)

Methods

__init__(url[, model_name, headers, api_key])

health_check([end_point])

Checks the health status of the LLM server.

send_message(message[, end_point, settings])

Sends a message to the LLM and receives the full response.

stream_message(message[, end_point, settings])

Streams the response from the LLM incrementally.