LibreChat Configuration Guide

docker-compose.override.yml

# For more details on the override file, see the Docker Override Guide:
# https://docs.librechat.ai/install/configuration/docker_override.html

version: '3.4'

services:
  api:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml # local/filepath:container/filepath

version

• Key : version

  • Type : String
  • Description : Specifies the version of the configuration file.
  • Example : version: 1.0.5
  • Required

cache

• Key : cache

  • Type : Boolean
  • Description : Toggles caching on or off. Set to true to enable caching (default).
  • Example : cache: true

fileStrategy

• Key : fileStrategy
  • Type : String
  • Options : "local" | "firebase"
  • Description : Determines where to save user uploaded/generated files. Defaults to "local" if omitted.
  • Example : fileStrategy: "firebase"

filteredTools

• Key : filteredTools
• Type: Array of Strings
• Example:

  filteredTools: ["scholarai", "calculator"]
  

• Description : Filters out specific tools from both Plugins and OpenAI Assistants endpoints
• Notes :
  • Affects both gptPlugins and assistants endpoints
  • You can find the names of the tools to filter in api/app/clients/tools/manifest.json
    • Use the pluginKey value
  • Also, any listed under the ".well-known" directory api/app/clients/tools/.well-known
    • Use the name_for_model value

secureImageLinks

• Key : secureImageLinks
  • Type : Boolean
  • Description : Whether or not to secure access to image links that are hosted locally by the app. Default: false.
  • Example : secureImageLinks: true

imageOutputType

• Key : imageOutputType
  • Type : String
  • Options : "png" | "webp" | "jpeg"
  • Description : The image output type for image responses. Defaults to "png" if omitted.
  • Note : Case-sensitive. Google endpoint only supports "jpeg" and "png" output types.
  • Example : imageOutputType: "webp"

fileConfig

• Key : fileConfig

  • Type : Object
  • Description : Configures file handling settings for the application, including size limits and MIME type restrictions.
  • Sub-keys:

  • endpoints

    • Type : Record/Object
    • Description : Specifies file handling configurations for individual endpoints, allowing customization per endpoint basis.

  • serverFileSizeLimit

    • Type : Number
    • Description : The maximum file size (in MB) that the server will accept. Applies globally across all endpoints unless overridden by endpoint-specific settings.

  • avatarSizeLimit

    • Type : Number
    • Description : Maximum size (in MB) for user avatar images.

• File Config Object Structure

rateLimits

• Key : rateLimits

  • Type : Object
  • Description : Defines rate limiting policies to prevent abuse by limiting the number of requests.
  • Sub-keys:
  • fileUploads
    • Type : Object
    • Description : Configures rate limits specifically for file upload operations.
    • Sub-keys:
    • ipMax
      • Type : Number
      • Description : Maximum number of uploads allowed per IP address per window.
    • ipWindowInMinutes
      • Type : Number
      • Description : Time window in minutes for the IP-based upload limit.
    • userMax
      • Type : Number
      • Description : Maximum number of uploads allowed per user per window.
    • userWindowInMinutes
      • Type : Number
      • Description : Time window in minutes for the user-based upload limit.
  • conversationsImport
    • Type : Object
    • Description : Configures rate limits specifically for conversation import operations.
    • Sub-keys:
    • ipMax
      • Type : Number
      • Description : Maximum number of imports allowed per IP address per window.
    • ipWindowInMinutes
      • Type : Number
      • Description : Time window in minutes for the IP-based imports limit.
    • userMax
      • Type : Number
      • Description : Maximum number of imports per user per window.
    • userWindowInMinutes
      • Type : Number
      • Description : Time window in minutes for the user-based imports limit.

• Example :

  rateLimits:
    fileUploads:
      ipMax: 100
      ipWindowInMinutes: 60
      userMax: 50
      userWindowInMinutes: 60
    conversationsImport:
      ipMax: 100
      ipWindowInMinutes: 60
      userMax: 50
      userWindowInMinutes: 60
  

registration

• Key : registration
  • Type : Object
  • Description : Configures registration-related settings for the application.
  • Sub-keys:
  • socialLogins : More info
  • allowedDomains : More info
• Registration Object Structure

interface

• Key : interface

  • Type : Object
  • Description : Configures user interface elements within the application, allowing for customization of visibility and behavior of various components.
  • Sub-keys:

  • privacyPolicy

    • Type : Object
    • Description : Contains settings related to the privacy policy link provided in the user interface.

  • termsOfService

    • Type : Object
    • Description : Contains settings related to the terms of service link provided in the user interface.

  • endpointsMenu

    • Type : Boolean
    • Description : Controls the visibility of the endpoints dropdown menu in the interface.

  • modelSelect

    • Type : Boolean
    • Description : Determines whether the model selection feature is available in the UI.
    • Note : Also disables the model and assistants selection dropdown from the right-most side panel.

  • parameters

    • Type : Boolean
    • Description : Toggles the visibility of parameter configuration options AKA conversation settings.

  • sidePanel

    • Type : Boolean
    • Description : Controls the visibility of the right-most side panel in the application's interface.

  • presets

    • Type : Boolean
    • Description : Enables or disables the presets menu in the application's UI.

• Interface Object Structure

modelSpecs

• Key : modelSpecs

  • Type : Object
  • Description : Configures model specifications, allowing for detailed setup and customization of AI models and their behaviors within the application.
  • Sub-keys:

  • enforce

    • Type : Boolean
    • Description : Determines whether the model specifications should strictly override other configuration settings.

  • prioritize

    • Type : Boolean
    • Description : Specifies if model specifications should take priority over the default configuration when both are applicable.

  • list

    • Type : Array of Objects
    • Description : Contains a list of individual model specifications detailing various configurations and behaviors.

• Model Specs Object Structure

endpoints

• Key : endpoints
  • Type : Object
  • Description : Defines custom API endpoints for the application.
  • Sub-keys:
  • custom
    • Type : Array of Objects
    • Description : Each object in the array represents a unique endpoint configuration.
    • Full Custom Endpoint Object Structure
  • azureOpenAI
    • Type : Object
    • Description : Azure OpenAI endpoint-specific configuration
    • Full Azure OpenAI Endpoint Object Structure
  • assistants
    • Type : Object
    • Description : Assistants endpoint-specific configuration.
    • Full Assistants Endpoint Object Structure

fileConfig / serverFileSizeLimit

The global maximum size for any file uploaded to the server, specified in megabytes (MB).

• Type: Integer
• Example:

  fileConfig:
    serverFileSizeLimit: 1000
  

• Note : Acts as an overarching limit for file uploads across all endpoints, ensuring that no file exceeds this size server-wide.

fileConfig / avatarSizeLimit

The maximum size allowed for avatar images, specified in megabytes (MB).

• Type: Integer
• Example:

  fileConfig:
    avatarSizeLimit: 2
  

• Note : Specifically tailored for user avatar uploads, allowing for control over image sizes to maintain consistent quality and loading times.

fileConfig / endpoints

Configures file handling settings for individual endpoints, allowing customization per endpoint basis.

• Type: Record/Object
• Description : Specifies file handling configurations for individual endpoints, allowing customization per endpoint basis.

fileConfig / endpoints / {endpoint_record} / disabled

Indicates whether file uploading is disabled for a specific endpoint.

• Type: Boolean
• Default: false (i.e., uploading is enabled by default)
• Example:

  openAI:
    disabled: true
  

• Note : Setting this to true prevents any file uploads to the specified endpoint, overriding any other file-related settings.

fileConfig / endpoints / {endpoint_record} / fileLimit

The maximum number of files allowed in a single upload request.

• Type: Integer
• Default: Varies by endpoint
• Example:

  assistants:
    fileLimit: 5
  

• Note : Helps control the volume of uploads and manage server load.

fileConfig / endpoints / {endpoint_record} / fileSizeLimit

The maximum size allowed for each individual file, specified in megabytes (MB).

• Type: Integer
• Default: Varies by endpoint
• Example:

  YourCustomEndpointName:
    fileSizeLimit: 1000
  

• Note : This limit ensures that no single file exceeds the specified size, allowing for better resource allocation and management.

fileConfig / endpoints / {endpoint_record} / totalSizeLimit

The total maximum size allowed for all files in a single request, specified in megabytes (MB).

• Type: Integer
• Default: Varies by endpoint
• Example:

  assistants:
    totalSizeLimit: 50
  

• Note : This setting is crucial for preventing excessive bandwidth and storage usage by any single upload request.

fileConfig / endpoints / {endpoint_record} / supportedMimeTypes

A list of regular expressions defining the MIME types permitted for upload.

• Type: Array of Strings
• Default: Varies by endpoint
• Example:

  assistants:
    supportedMimeTypes:
      - "image/.*"
      - "application/pdf"
  

• Note : This allows for precise control over the types of files that can be uploaded. Invalid regex is ignored.

interface / privacyPolicy

Contains settings related to the privacy policy link provided in the user interface.

• Type: Object
• Description : Allows for the specification of a custom URL and the option to open it in a new tab.
• Sub-keys :
  • externalUrl
    • Type: String (URL)
    • Description: The URL pointing to the privacy policy document.
  • openNewTab
    • Type: Boolean
    • Description: Specifies whether the link should open in a new tab.

interface / termsOfService

Contains settings related to the terms of service link provided in the user interface.

• Type: Object
• Description : Allows for the specification of a custom URL and the option to open it in a new tab.
• Sub-keys :
  • externalUrl
    • Type: String (URL)
    • Description: The URL pointing to the terms of service document.
  • openNewTab
    • Type: Boolean
    • Description: Specifies whether the link should open in a new tab.

interface / endpointsMenu

Controls the visibility of the endpoints menu in the interface.

• Type: Boolean
• Default: true
• Example:

  interface:
    endpointsMenu: false
  

• Note : Toggling this setting allows administrators to customize the availability of endpoint selections within the application.

interface / modelSelect

Determines whether the model selection feature is available in the UI.

• Type: Boolean
• Default: true
• Example:

  interface:
    modelSelect: true
  

• Note : Enabling this feature allows users to select different models directly from the interface.

interface / parameters

Toggles the visibility of parameter configuration options within the interface.

• Type: Boolean
• Default: true
• Example:

  interface:
    parameters: false
  

• Note : This setting is crucial for users who need to adjust parameters for specific functionalities within the application.

interface / sidePanel

Controls the visibility of the side panel in the application's interface.

• Type: Boolean
• Default: true
• Example:

  interface:
    sidePanel: true
  

• Note : The side panel typically contains additional navigation or information relevant to the application's context.

interface / presets

Enables or disables the use of presets in the application's UI.

• Type: Boolean
• Default: true
• Example:

  interface:
    presets: true
  

• Note : Presets can simplify user interactions by providing pre-configured settings or operations, enhancing user experience and efficiency.

modelSpecs / enforce

Determines whether the model specifications should strictly override other configuration settings.

• Type: Boolean
• Default: false
• Example:

  modelSpecs:
    enforce: true
  

• Note : Setting this to true can lead to conflicts with interface options if not managed carefully.

modelSpecs / prioritize

Specifies if model specifications should take priority over the default configuration when both are applicable.

• Type: Boolean
• Default: true
• Example:

  modelSpecs:
    prioritize: false
  

• Note : When set to true , it ensures that a modelSpec is always selected in the UI. Doing this may prevent users from selecting different endpoints for the selected spec.

modelSpecs / list

Contains a list of individual model specifications detailing various configurations and behaviors.

• Type: Array of Objects
• Description : Each object in the list details the configuration for a specific model, including its behaviors, appearance, and capabilities related to the application's functionality.

modelSpecs / list / {spec_item} / preset / modelLabel

The label used to identify the model in user interfaces or logs. It provides a human-readable name for the model, which is displayed in the UI, as well as made aware to the AI.

• Type: String (nullable, optional)
• Default: None
• Example:

  preset:
    modelLabel: "Customer Support Bot"
  

modelSpecs / list / {spec_item} / preset / endpoint

Specifies the endpoint the model communicates with to execute operations. This setting determines the external or internal service that the model interfaces with.

• Type: Enum ( EModelEndpoint ) or String (nullable)
• Example:

  preset:
    endpoint: "openAI"
  

modelSpecs / list / {spec_item} / preset / greeting

A predefined message that is visible in the UI before a new chat is started.

• Type: String (optional)
• Example:

  preset:
    greeting: "Hello! How can I assist you today?"
  

modelSpecs / list / {spec_item} / preset / promptPrefix

A static text prepended to every prompt sent to the model, setting a consistent context for responses.

• Type: String (nullable, optional)
• Example:

  preset:
    promptPrefix: "As a financial advisor, ..."
  

• Note : When using "assistants" as the endpoint, this becomes the OpenAI field additional_instructions

modelSpecs / list / {spec_item} / preset / {model_option}

These settings control the stochastic nature and behavior of model responses, affecting creativity, relevance, and variability.

• Types:
• temperature : Number (optional)
• top_p : Number (optional)
• top_k : Number (optional)
• frequency_penalty : Number (optional)
• presence_penalty : Number (optional)

• stop : Array of Strings (optional)

• Examples:

  preset:
    temperature: 0.7
    top_p: 0.9
  

modelSpecs / list / {spec_item} / preset / resendFiles

Indicates whether files should be resent in scenarios where persistent sessions are not maintained.

• Type: Boolean (optional)
• Example:

  preset:
    resendFiles: true
  

modelSpecs / list / {spec_item} / preset / imageDetail

Specifies the level of detail required in image analysis tasks, applicable to models with vision capabilities (OpenAI spec).

• Type: eImageDetailSchema (optional)
• Example:

  preset:
    imageDetail: "high"
  

modelSpecs / list / {spec_item} / preset / agentOptions

Specific to gptPlugins endpoint. Can be omitted either partially or completely for default settings

• Type: Record/Object (optional)
• Sub-fields include:
• agent : Type of agent (either "functions" or "classic"; default: "functions")
• skipCompletion : Whether to skip automatic completion suggestions (default: true)
• model : Model version or identifier (default: "gpt-4-turbo")

• temperature : Randomness in the model's responses (default: 0)

• Example:

  preset:
    agentOptions:
      agent: "functions"
      skipCompletion: false
      model: "gpt-4-turbo"
      temperature: 0.5
  

modelSpecs / list / {spec_item} / preset / tools

Specific to gptPlugins endpoint. List of tool/plugin names.

• Type: Array of Strings
• Optional
• Example:

  preset:
    tools: ["dalle", "tavily_search_results_json", "azure-ai-search", "traversaal_search"]
  

Notes :

• At the moment, only tools that have credentials provided for them via .env file can be used with modelSpecs, unless the user already had the tool installed.
• You can find the names of the tools to filter in api/app/clients/tools/manifest.json
  • Use the pluginKey value
• Also, any listed under the ".well-known" directory api/app/clients/tools/.well-known
  • Use the name_for_model value

modelSpecs / list / {spec_item} / preset / {assistant_option}

Configurations specific to assistants, such as identifying an assistant, overriding the assistant's instructions.

• Types:
• assistant_id : String (optional)

• instructions : String (optional)

• Examples:

  preset:
    assistant_id: "asst_98765"
    # Overrides the assistant's default instructions
    instructions: "Please handle customer queries regarding order status."
  

registration / socialLogins

Defines the available social login providers and their display order.

• Type: Array of Strings
• Example:

  socialLogins: ["google", "facebook", "github", "discord", "openid"]
  

• Note : The order of the providers in the list determines their appearance order on the login/registration page. Each provider listed must be properly configured within the system to be active and available for users. This configuration allows for a tailored authentication experience, emphasizing the most relevant or preferred social login options for your user base.

registration / allowedDomains

A list specifying allowed email domains for registration.

• Type: Array of Strings
• Example:

  allowedDomains:
    - "gmail.com"
    - "protonmail.com"
  

• Required
• Note : Users with email domains not listed will be restricted from registering.

endpoints / assistants / disableBuilder

Controls the visibility and use of the builder interface for assistants.

• Type : Boolean
• Example : disableBuilder: false
• Description : When set to true , disables the builder interface for the assistant, limiting direct manual interaction.
• Note : Defaults to false if omitted.

endpoints / assistants / pollIntervalMs

Specifies the polling interval in milliseconds for checking run updates or changes in assistant run states.

• Type : Integer
• Example : pollIntervalMs: 500
• Description : Specifies the polling interval in milliseconds for checking assistant run updates.
• Note : Defaults to 750 if omitted.

endpoints / assistants / timeoutMs

Defines the maximum time in milliseconds that an assistant can run before the request is cancelled.

• Type : Integer
• Example : timeoutMs: 10000
• Description : Sets a timeout in milliseconds for assistant runs. Helps manage system load by limiting total run operation time.
• Note : Defaults to 3 minutes (180,000 ms). Run operation times can range between 50 seconds to 2 minutes but also exceed this. If the timeoutMs value is exceeded, the run will be cancelled.

endpoints / assistants / supportedIds

List of supported assistant Ids

• Type: Array/List of Strings
• Description : List of supported assistant Ids. Use this or excludedIds but not both (the excludedIds field will be ignored if so).
• Example : supportedIds: ["asst_supportedAssistantId1", "asst_supportedAssistantId2"]

endpoints / assistants / excludedIds

List of excluded assistant Ids

• Type: Array/List of Strings
• Description : List of excluded assistant Ids. Use this or supportedIds but not both (the excludedIds field will be ignored if so).
• Example : excludedIds: ["asst_excludedAssistantId1", "asst_excludedAssistantId2"]

endpoints / assistants / retrievalModels

Specifies the models that support retrieval for the assistants endpoint.

• Type : Array/List of Strings
• Example : retrievalModels: ["gpt-4-turbo-preview"]
• Description : Defines the models that support retrieval capabilities for the assistants endpoint. By default, it uses the latest known OpenAI models that support the official Retrieval feature.
• Note : This field is optional. If omitted, the default behavior is to use the latest known OpenAI models that support retrieval.

endpoints / assistants / capabilities

Specifies the assistant capabilities available to all users for the assistants endpoint.

• Type : Array/List of Strings
• Example : capabilities: ["code_interpreter", "retrieval", "actions", "tools", "image_vision"]
• Description : Defines the assistant capabilities that are available to all users for the assistants endpoint. You can omit the capabilities you wish to exclude from the list. The available capabilities are:
• code_interpreter : Enables code interpretation capabilities for the assistant.
• image_vision : Enables unofficial vision support for uploaded images.
• retrieval : Enables retrieval capabilities for the assistant.
• actions : Enables action capabilities for the assistant.
• tools : Enables tool capabilities for the assistant.
• Note : This field is optional. If omitted, the default behavior is to include all the capabilities listed in the example.

endpoints / custom / name

A unique name for the endpoint.

• Type: String
• Example: name: "Mistral"
• Required
• Note : Will be used as the "title" in the Endpoints Selector

endpoints / custom / apiKey

Your API key for the service. Can reference an environment variable, or allow user to provide the value.

• Type: String (apiKey | "user_provided" )
• Example: apiKey: "${MISTRAL_API_KEY}" | apiKey: "your_api_key" | apiKey: "user_provided"
• Required
• Note : It's highly recommended to use the env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / custom / baseURL

Base URL for the API. Can reference an environment variable, or allow user to provide the value.

• Type: String (baseURL | "user_provided" )
• Example: baseURL: "https://api.mistral.ai/v1" | baseURL: "${MISTRAL_BASE_URL}" | baseURL: "user_provided"
• Required
• Note : It's highly recommended to use the env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / custom / iconURL

The URL to use as the Endpoint Icon.

• Type: Boolean
• Example: iconURL: https://github.com/danny-avila/LibreChat/raw/main/docs/assets/LibreChat.svg
• Notes :
  • If you want to use existing project icons, define the endpoint name as one of the main endpoints (case-sensitive):
    • "openAI" | "azureOpenAI" | "google" | "anthropic" | "assistants" | "gptPlugins"
  • There are also "known endpoints" (case-insensitive), which have icons provided. If your endpoint name matches the following names, you should omit this field:
    • "Mistral"
    • "OpenRouter"
    • "Groq"
    • APIpie
    • "Anyscale"
    • "Fireworks"
    • "Perplexity"
    • "together.ai"
    • "Ollama"
    • "MLX"

endpoints / custom / models

Configuration for models.

• Required
• default : An array of strings indicating the default models to use. At least one value is required.
• Type: Array of Strings
• Example: default: ["mistral-tiny", "mistral-small", "mistral-medium"]
• Note : If fetching models fails, these defaults are used as a fallback.
• fetch : When set to true , attempts to fetch a list of models from the API.
• Type: Boolean
• Example: fetch: true
• Note : May cause slowdowns during initial use of the app if the response is delayed. Defaults to false .
• userIdQuery : When set to true , adds the LibreChat user ID as a query parameter to the API models request.
• Type: Boolean
• Example: userIdQuery: true

endpoints / custom / titleConvo

Enables title conversation when set to true .

• Type: Boolean
• Example: titleConvo: true

endpoints / custom / titleMethod

Chooses between "completion" or "functions" for title method.

• Type: String ( "completion" | "functions" )
• Example: titleMethod: "completion"
• Note : Defaults to "completion" if omitted.

endpoints / custom / titleModel

Specifies the model to use for titles.

• Type: String
• Example: titleModel: "mistral-tiny"
• Note : Defaults to "gpt-3.5-turbo" if omitted. May cause issues if "gpt-3.5-turbo" is not available.
• Note : You can also dynamically use the current conversation model by setting it to "current_model".

endpoints / custom / summarize

Enables summarization when set to true .

• Type: Boolean
• Example: summarize: false
• Note : This feature requires an OpenAI Functions compatible API

endpoints / custom / summaryModel

Specifies the model to use if summarization is enabled.

• Type: String
• Example: summaryModel: "mistral-tiny"
• Note : Defaults to "gpt-3.5-turbo" if omitted. May cause issues if "gpt-3.5-turbo" is not available.

endpoints / custom / forcePrompt

If true , sends a prompt parameter instead of messages .

• Type: Boolean
• Example: forcePrompt: false
• Note : Combines all messages into a single text payload or "prompt", following OpenAI format , which uses the /completions endpoint of your baseURL rather than /chat/completions .

endpoints / custom / modelDisplayLabel

The label displayed in messages next to the Icon for the current AI model.

• Type: String
• Example: modelDisplayLabel: "Mistral"
• Note : The display order is:
• 1. Custom name set via preset (if available)
• 1. Label derived from the model name (if applicable)
• 1. This value, modelDisplayLabel , is used if the above are not specified. Defaults to "AI".

endpoints / custom / addParams

Adds additional parameters to requests.

• Type: Object/Dictionary
• Description : Adds/Overrides parameters. Useful for specifying API-specific options.
• Example :

      addParams:
        safe_prompt: true
  

endpoints / custom / dropParams

Removes default parameters from requests.

• Type: Array/List of Strings
• Description : Excludes specified default parameters . Useful for APIs that do not accept or recognize certain parameters.
• Example : dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]
• Note : For a list of default parameters sent with every request, see the "Default Parameters" Section below.

endpoints / custom / headers

Adds additional headers to requests. Can reference an environment variable

• Type: Object/Dictionary
• Description : The headers object specifies custom headers for requests. Useful for authentication and setting content types.
• Example :
• Note : Supports dynamic environment variable values, which use the format: "${VARIABLE_NAME}"

      headers:
        x-api-key: "${ENVIRONMENT_VARIABLE}"
        Content-Type: "application/json"
  

endpoints / azureOpenAI / plugins

Enables or disables plugins for the Azure OpenAI endpoint.

• Type: Boolean
• Example : plugins: true
• Description : When set to true , activates plugins associated with this endpoint.
• Note : You can only use either the official OpenAI API or Azure OpenAI API for plugins, not both.

endpoints / azureOpenAI / assistants

Enables or disables assistants for the Azure OpenAI endpoint.

• Type: Boolean
• Example : assistants: true
• Description : When set to true , activates assistants associated with this endpoint.
• Note : You can only use either the official OpenAI API or Azure OpenAI API for assistants, not both.

endpoints / azureOpenAI / groups

Configuration for groups of models by geographic location or purpose.

• Type: Array
• Description : Each item in the groups array configures a set of models under a certain grouping, often by geographic region or distinct configuration.
• Example : See example above.

endpoints / azureOpenAI / groups / {group_item} / group

Identifier for a group of models.

• Type: String
• Required
• Example : "my-westus"

endpoints / azureOpenAI / groups / {group_item} / apiKey

The API key for accessing the Azure OpenAI Service.

• Type: String
• Required
• Example : "${WESTUS_API_KEY}"
• Note : It's highly recommended to use a custom env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / azureOpenAI / groups / {group_item} / instanceName

Name of the Azure instance.

• Type: String
• Required
• Example : "my-westus"
• Note : It's recommended to use a custom env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / azureOpenAI / groups / {group_item} / version

API version.

• Type: String
• Optional
• Example : "2023-12-01-preview"
• Note : It's recommended to use a custom env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / azureOpenAI / groups / {group_item} / baseURL

The base URL for the Azure OpenAI Service.

• Type: String
• Optional
• Example : "https://prod.example.com"
• Note : It's recommended to use a custom env. variable reference for this field, i.e. ${YOUR_VARIABLE}

endpoints / azureOpenAI / groups / {group_item} / additionalHeaders

Additional headers for API requests.

• Type: Dictionary
• Optional
• Example :

  additionalHeaders:
    X-Custom-Header: ${YOUR_SECRET_CUSTOM_VARIABLE}
  

• Note : It's recommended to use a custom env. variable reference for the values of field, as shown in the example.
• Note : api-key header value is sent on every request

endpoints / azureOpenAI / groups / {group_item} / serverless

Indicates the use of a serverless inference endpoint for Azure OpenAI chat completions.

• Type: Boolean
• Optional
• Description : When set to true , specifies that the group is configured to use serverless inference endpoints as an Azure "Models as a Service" model.
• Example : serverless: true
• Note : More info here

endpoints / azureOpenAI / groups / {group_item} / addParams

Adds additional parameters to requests.

• Type: Object/Dictionary
• Description : Adds/Overrides parameters. Useful for specifying API-specific options.
• Example :

      addParams:
        safe_prompt: true
  

endpoints / azureOpenAI / groups / {group_item} / apiKey

Removes default parameters from requests.

• Type: Array/List of Strings
• Description : Excludes specified default parameters . Useful for APIs that do not accept or recognize certain parameters.
• Example : dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]
• Note : For a list of default parameters sent with every request, see the "Default Parameters" Section below.

endpoints / azureOpenAI / groups / {group_item} / forcePrompt

If true , sends a prompt parameter instead of messages .

• Type: Boolean
• Example: forcePrompt: false
• Note : This combines all messages into a single text payload, following OpenAI format , and uses the /completions endpoint of your baseURL rather than /chat/completions .

endpoints / azureOpenAI / groups / {group_item} / models

Configuration for individual models within a group.

• Description : Configures settings for each model, including deployment name and version. Model configurations can adopt the group's deployment name and/or version when configured as a boolean (set to true ) or an object for detailed settings of either of those fields.
• Example : See above example configuration.

Within each group, models are records, either set to true, or set with a specific deploymentName and/or version where the key MUST be the matching OpenAI model name; for example, if you intend to use gpt-4-vision, it must be configured like so:

models:
  gpt-4-vision-preview: # matching OpenAI Model name
    deploymentName: "arbitrary-deployment-name"
    version: "2024-02-15-preview" # version can be any that supports vision

endpoints / azureOpenAI / groups / {group_item} / models / {model_item=Object}

An object allows for detailed configuration of the model, including its deploymentName and/or version . This mode is used for more granular control over the models, especially when working with multiple versions or deployments under one instance or resource group.

Example :

models:
  gpt-4-vision-preview:
    deploymentName: "gpt-4-vision-preview"
    version: "2024-02-15-preview"

Notes:

• Deployment Names and Versions are critical for ensuring that the correct model is used.
  • Double-check these values for accuracy to prevent unexpected behavior.

endpoints / azureOpenAI / groups / {group_item} / models / {model_item=Object} / deploymentName

The name of the deployment for the model.

• Type: String
• Required
• Example : "gpt-4-vision-preview"
• Description : Identifies the deployment of the model within Azure.
• Note : This does not have to be the matching OpenAI model name as is convention, but must match the actual name of your deployment on Azure.

endpoints / azureOpenAI / groups / {group_item} / models / {model_item=Object} / version

Specifies the version of the model.

• Type: String
• Required
• Example : "2024-02-15-preview"
• Description : Defines the version of the model to be used.

endpoints / azureOpenAI / groups / {group_item} / models / {model_item=true}

When a model is enabled ( true ) without using an object, it uses the group's configuration values for deployment name and version.

Example :

models:
  gpt-4-turbo: true