> ## Documentation Index > Fetch the complete documentation index at: https://docs.openhands.dev/llms.txt > Use this file to discover all available pages before exploring further. # Azure LLM Gateway > Connect Azure OpenAI and Azure AI Foundry models through the OpenHands Enterprise LLM gateway. Connect OpenHands Enterprise to Azure OpenAI or Azure AI Foundry model deployments through its bundled LiteLLM gateway. These configurations apply whether OpenHands runs on AKS or another supported Kubernetes platform. Choose your installation method: * **Replicated**: configure the gateway in the Admin Console. * **Helm**: configure the gateway through your installation values. ## Choose the Provider Route Azure AI Foundry includes Azure OpenAI and other model deployments. Use the route that matches your deployment's inference API, rather than choosing by the portal name alone. | Deployment API | LiteLLM Provider | Configuration | | - | - | - | | Azure OpenAI | `azure/` | Azure resource endpoint, deployment name, API version and credentials | | Foundry model inference | `azure_ai/` | Deployment's inference endpoint, model/deployment identifier and credentials | An installation can expose both routes as separate model aliases in the same bundled gateway. Each route can use its own endpoint and Secret. ## Configure the Gateway > [tab] Replicated > > > [!NOTE] > > > > Confirm Azure OpenAI support for your installed Replicated release before using > > these settings. The API-version rendering issue tracked in > > [OpenHands-Cloud #1337](https://github.com/OpenHands/OpenHands-Cloud/pull/1337) > > remains unresolved. The Helm configuration below sets the API version directly. > > 1. Open the [Admin Console LLM configuration](https://docs.openhands.dev/enterprise/vm-install/admin-console-configuration.md#llm-configuration). > 2. Select `Azure` for Azure OpenAI. > 3. Under `Azure Authentication Method`, select `API Key` and enter your > resource key in `Azure OpenAI API Key`. To use a service principal, select > `Microsoft Entra ID — Service Principal` and complete the authentication > fields shown by the Admin Console. > 4. In `Azure OpenAI Endpoint`, enter your resource endpoint, such as > `https://.openai.azure.com/`. > 5. In `Azure OpenAI API Version`, enter a version that supports your model and > selected API mode. For the Responses example in this guide, enter > `2025-04-01-preview` instead of the displayed `2024-10-21` default. > 6. In `Azure OpenAI Deployments`, enter **one deployment name per line**. > Use the deployment names from your Azure resource, without an `azure/` > prefix. For example: > > ```text theme={null} > gpt-4.1-mini > gpt-4.1 > ``` > > All deployments must be reachable at the endpoint and API version above. > Deployment names can differ from the underlying model names. > 7. Save the configuration and deploy the updated version. > > Use the Admin Console to manage this configuration rather than applying the Helm > snippets in the Helm tab to a Replicated-managed installation. The Admin Console configures Azure OpenAI routes only. For Azure AI Foundry on a > Replicated installation, contact OpenHands Support. > [tab] Helm > > ### Prerequisites > > * A working [OpenHands Enterprise Helm installation](https://docs.openhands.dev/enterprise/k8s-install/installation.md). > * An Azure OpenAI or Azure AI Foundry model deployment that supports agent tool > use, with its inference endpoint and API key. > * Network access from the bundled LiteLLM pod to the Azure endpoint over HTTPS. > For a private endpoint, configure private DNS and routing from the cluster. > * An API version supported by your deployment and the request endpoints it uses. > > The following instructions use Azure resource API keys. > > ### Configure Azure OpenAI > > #### 1. Store the Azure Credential > > Use your secret manager to create `azure-openai-gateway` in the OpenHands > application namespace, with a key named `AZURE_API_KEY`. Alternatively, create it > from a private file containing only the Azure resource API key: > > ```bash theme={null} > kubectl -n openhands create secret generic azure-openai-gateway \ > --from-file=AZURE_API_KEY=/path/to/private/azure-api-key > ``` > > The Azure resource key is used by LiteLLM. It is separate from an OpenHands app > API key used to call conversation or automation APIs. > > #### 2. Add the Azure Model to Helm Values > > Merge this entry into your existing installation values. Keep all existing model > entries and environment Secrets: Helm replaces lists when applying overrides. > > ```yaml theme={null} > litellm-helm: > environmentSecrets: > - litellm-env-secrets > - azure-openai-gateway > proxy_config: > model_list: > # Retain your existing model entries here. > - model_name: azure-gpt-4.1-mini > litellm_params: > model: azure/ > api_base: https://.openai.azure.com/ > api_version: "2025-04-01-preview" > api_key: os.environ/AZURE_API_KEY > ``` > > Replace `` with the deployment name in your Azure resource. > It may differ from the underlying model name. `azure-gpt-4.1-mini` is the alias > that OpenHands uses to select this route. > > Set `api_version` directly on the route. Use a version that supports your > model and the API mode selected in OpenHands; this example uses Responses. > > #### 3. Apply the Updated Values > > Use the licensed chart URL and version from your installation. Apply your > complete installation values, including the Azure configuration: > > ```bash theme={null} > helm upgrade openhands "$OPENHANDS_CHART_URL" \ > --namespace openhands \ > --version "$OPENHANDS_CHART_VERSION" \ > --values values.yaml \ > --wait --timeout 10m > > kubectl -n openhands rollout status deployment/openhands-litellm > ``` > > Adjust the namespace and deployment name if you used a different release name. > > ### Configure Azure AI Foundry > > Obtain the inference endpoint, supported model/deployment identifier and key from > an existing Foundry deployment. Use the inference endpoint shown for that > deployment, rather than its project management URL. > > Create a separate Secret, `azure-ai-gateway`, containing `AZURE_AI_API_KEY` from > your secret manager or a private file: > > ```bash theme={null} > kubectl -n openhands create secret generic azure-ai-gateway \ > --from-file=AZURE_AI_API_KEY=/path/to/private/foundry-api-key > ``` > > Add the Secret to the gateway's existing `environmentSecrets` list and append a > model route to its existing `model_list`: > > ```yaml theme={null} > litellm-helm: > environmentSecrets: > - litellm-env-secrets > - azure-openai-gateway > - azure-ai-gateway > proxy_config: > model_list: > # Retain your Azure OpenAI and other existing model entries here. > - model_name: foundry-gpt-4.1-mini > litellm_params: > model: azure_ai/ > api_base: https://.services.ai.azure.com/models > api_version: "2024-05-01-preview" > api_key: os.environ/AZURE_AI_API_KEY > drop_params: true > ``` > > Use the endpoint and API version for your deployment's model-inference API. > Apply the complete updated values using the Helm upgrade command above. ## Select the Model in OpenHands For a Replicated installation, select the Azure model configured in the Admin Console. For a Helm installation, save a separate LLM profile for each gateway alias you want to use. For the GPT-4.1-mini Helm routes above, use these settings: | Field | Azure OpenAI Example | Foundry Example | | - | - | - | | Profile Name | `Azure-GPT-4.1-mini` | `Foundry-GPT-4.1-mini` | | Model | `openhands/azure-gpt-4.1-mini` | `openhands/foundry-gpt-4.1-mini` | | Base URL | `http://openhands-litellm.openhands.svc.cluster.local:4000` | `http://openhands-litellm.openhands.svc.cluster.local:4000` | | Canonical Model Name | `gpt-4.1-mini` | `gpt-4.1-mini` | | API Mode | `responses` | `chat` | | API Key | Leave unset; use the managed gateway credential | Leave unset; use the managed gateway credential | The Foundry example uses Chat Completions, while the Azure OpenAI example uses Responses. Choose the API mode supported by your configured inference route. Use your installation's internal LiteLLM Service DNS name. The profile base URL points to the bundled gateway; the Azure resource endpoint belongs in Helm `litellm_params.api_base`. ## Start Using the Model 1. Select the profile you configured and start a conversation. Ask the agent to run `pwd`, write a file containing a literal test string, and read it back. 2. Confirm the sandbox starts, tool actions succeed, and the agent completes. 3. If you use automations, follow [Enable Automations with Helm](https://docs.openhands.dev/enterprise/k8s-install/automations.md). Select the saved profile name, such as `Azure-GPT-4.1-mini` or `Foundry-GPT-4.1-mini`, dispatch one test run, and verify its status reaches `COMPLETED` and its output is correct. Disable any test schedule afterward. ## Troubleshooting > [accordion] Azure `404` > > Check the resource endpoint, deployment name and API version. Chat Completions support does not establish Responses support; use a version that supports your selected API mode. > [accordion] Managed gateway-token authentication error > > Set the profile base URL to the internal LiteLLM Service URL shown above if your release does not supply it automatically. > [accordion] Azure credential or permission error > > Check the resource key and Secret injection. For Entra authentication, grant the required inference permissions; subscription Contributor access alone does not grant model inference access. > [accordion] Azure `429`, repeated retries, or a conversation that keeps waiting > > Check deployment token quota, request limits and available capacity. Agent prompts and tool definitions can require more capacity than short direct completions. > [accordion] Unsupported model parameters > > Check provider support; `drop_params: true` can omit unsupported optional parameters, but does not add tool support to a model > [accordion] A different Foundry model fails > > Check its endpoint path, supported API and [LiteLLM provider configuration](https://docs.litellm.ai/docs/providers/azure_ai). Some model families require a different route, such as an Anthropic endpoint for Claude. > [accordion] Sandbox cannot start > > Check Kubernetes events and runtime registration. Inspect the configured sandbox runtime and node placement through your installation values. For Azure model routing details, see [LiteLLM's Azure OpenAI reference](https://docs.litellm.ai/docs/providers/azure). If your organization routes through an existing external gateway, use [External LLM Gateways](https://docs.openhands.dev/enterprise/integrations/external-llm-gateways.md). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.