ACA - Portkey Docs

Components and Sizing Recommendations

Component	Options	Sizing Recommendations
AI Gateway	Deploy in your ACA environment using Terraform	Use Container Apps with at least 1 vCPU and 2 GiB of memory per replica. For high availability, deploy with auto-scaling across multiple Availability Zones.
Logs Store (optional)	Azure Blob Storage or S3-compatible storage	Each log document is ~10kb in size (uncompressed)
Cache (Prompts, Configs & Providers)	Built-in Redis or Azure Managed Redis

Prerequisites

Ensure the following tools and resources are installed and available:

Azure Subscription with permissions to create Container Apps, Key Vault, Storage, VNet, Application Gateway, etc.
Azure CLI configured with credentials
Terraform v1.5 or later

Create a Strata Cloud Manager account

Go to the the AI Gateway website.
Sign up for a Strata Cloud Manager account.
Once logged in, locate and save your Organisation ID for future reference. It can be found in the browser URL: https://stratacloudmanager.paloaltonetworks.com/<organisation_id>/
Contact the Palo Alto Networks team and provide your Organisation ID and the email address used during signup.
The Palo Alto Networks team will share the following information with you:
- Docker credentials for the Gateway images (username and password).
- License: Client Auth Key.

Setup Project Environment

1. Prepare Azure Resources

# Login to Azure
az login
sub_id=<your-subscription-id>                    # Provide your azure subscription id
az account set --subscription ${sub_id}

rg=portkey-rg                                    # Provide name of resource group.
kv=portkey-kv                                    # Provide Key Vault name

# Change location as per requirement
# If Resource Group is not created yet create it with:
# az group create --name ${rg} --location eastus

# Create a Key Vault and store your secrets

az keyvault create --name ${kv} --resource-group ${rg} --location eastus --enable-rbac-authorization true

user_id=$(az ad signed-in-user show --query id -o tsv)

az role assignment create \
  --role "Key Vault Administrator" \
  --assignee ${user_id} \
  --scope "/subscriptions/${sub_id}/resourceGroups/${rg}/providers/Microsoft.KeyVault/vaults/${kv}"

# Store Docker credentials and Portkey secrets
az keyvault secret set --vault-name ${kv} --name docker-username --value "<docker-username>"         # Docker username shared by Portkey
az keyvault secret set --vault-name ${kv} --name docker-password --value "<docker-password>"         # Docker password shared by Portkey
az keyvault secret set --vault-name ${kv} --name portkey-client-auth --value "<portkey-client-auth>" # Shared by Portkey
az keyvault secret set --vault-name ${kv} --name organisations-to-sync --value "<organisation-id>"   # Provide your Portkey account organisation id

2. Create Terraform Configuration Files

Create a new directory for your deployment:

mkdir portkey-gateway
cd portkey-gateway

3. Create Module Configuration

Create a main.tf file:

terraform {
  required_version = ">= 1.5"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
}

provider "azurerm" {
  features {}
}

module "portkey_gateway" {
  source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/aca?ref=v1.1.3"                 # Change module version as per requirement

  # Project Configuration
  project_name        = "portkey-gateway"
  environment         = "dev"
  resource_group_name = "portkey-rg"

  # Docker Registry Configuration
  registry_type       = "dockerhub"
  docker_credentials  = {
    key_vault_name  = "portkey-kv"
    key_vault_rg    = "portkey-rg"
    username_secret = "docker-username"
    password_secret = "docker-password"
  }

  # Network Configuration
  network_mode   = "none"

  # Ingress Configuration
  ingress_type          = "aca"
  public_ingress        = true

  # Storage Configuration
  # Can provide existing Azure blob storage for storing logs otherwise a storage account will be create.
  # storage_config = {
  #   resource_group = "<storage-rg>"
  #   auth_mode = "managed"
  #   account_name = "<blob-storage-account-name>"
  #   container_name = "<blob-storage-container-name>"
  # }

  # Server Mode Configuration
  server_mode    = "gateway"                                        # Set to "all" for both AI Gateway and MCP deployment

  gateway_image = {
    image = "portkeyai/gateway_enterprise"
    tag   = "2.2.2"
  }
  gateway_config = {
    cpu   = 1
    memory = "2Gi"
    min_replicas = 1
    max_replicas = 2
    port = 8787
    cpu_scale_threshold = 70
  }

  redis_config = {
    redis_type = "redis"
    cpu        = 1
    memory     = "2Gi"
  }

  environment_variables = {
    gateway = {
      LOG_LEVEL             = "info"
      NODE_ENV              = "development"
      ANALYTICS_STORE       = "control_plane"
      AZURE_MANAGED_VERSION = "2019-08-01"
    }
  }

  secrets = {
    gateway = {
      PORTKEY_CLIENT_AUTH    = "portkey-client-auth"                # Name of Key Vault secret storing the client auth value
      ORGANISATIONS_TO_SYNC  = "organisations-to-sync"              # Name of Key Vault secret storing the organisation ID
    }
  }

  # Secrets Configuration
  secrets_key_vault = {
    name           = "portkey-kv"                                   # Key Vault name where the secrets are stored
    resource_group = "portkey-rg"
  }

  # NOTE: Values in the 'secrets' block should be the KEY VAULT SECRET NAMES,
  # not the actual secret values. The module will retrieve the actual values
  # from the specified Key Vault.
}
output "gateway_url" {
    value = module.portkey_gateway.gateway_url
}

Advanced Configuration

MCP Gateway (Optional)

By default, only the AI Gateway is enabled. To enable the MCP Gateway, update your terraform.tfvars: MCP Only:

server_mode = "mcp"

mcp_config = {
  cpu          = 1
  memory       = "2Gi"
  min_replicas = 2
  max_replicas = 10
  port         = 8788
}

Gateway + MCP (separate apps):

server_mode = "all"

mcp_gateway_base_url = "https://mcp.example.com"

# Gateway configuration
gateway_config = {
  cpu          = 1
  memory       = "2Gi"
  min_replicas = 2
  max_replicas = 10
  port         = 8787
}

# MCP configuration (independent scaling)
mcp_config = {
  cpu          = 0.5
  memory       = "1Gi"
  min_replicas = 1
  max_replicas = 5
  port         = 8788
}

Note: When server_mode = "all" with Application Gateway, you must configure either host-based.

Auto-Scaling Configuration

Control how replicas scale based on different metrics. CPU-based scaling (default):

gateway_config = {
  cpu                 = 1
  memory              = "2Gi"
  min_replicas        = 1
  max_replicas        = 10
  cpu_scale_threshold = 70  # Scale at 70% CPU
}

HTTP-based scaling:

gateway_config = {
  cpu                            = 0.5
  memory                         = "1Gi"
  min_replicas                   = 1
  max_replicas                   = 10
  cpu_scale_threshold            = null  # Disable CPU scaling
  http_scale_concurrent_requests = 100   # Scale at 100 concurrent requests per replica
}

Memory-based scaling:

gateway_config = {
  cpu                    = 1
  memory                 = "2Gi"
  min_replicas           = 2
  max_replicas           = 20
  cpu_scale_threshold    = null  # Disable CPU scaling
  memory_scale_threshold = 80    # Scale at 80% memory
}

Network Configuration with VNet

Deploy Gateway within a VNet: Create new VNet:

network_mode = "new"
vnet_cidr                = "10.0.0.0/16"

Use existing VNet and subnets:

network_mode = "existing"
vnet_id                   = "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.Network/virtualNetworks/<vnet-name>"
aca_subnet_id             = "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<aca-subnet-name>"
private_link_subnet_id    = "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<private-link-subnet-name>"
app_gateway_subnet_id     = "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<app-gateway-subnet-name>"

Application Gateway Ingress

Deploy Azure Application Gateway with WAF, SSL termination, and zone redundancy: Basic Configuration:

network_mode = "new"  # VNet is required
ingress_type = "application_gateway"

app_gateway_config = {
  sku_name     = "Standard_v2"  # or "WAF_v2"
  sku_tier     = "Standard_v2"  # or "WAF_v2"
  capacity     = 2
  enable_waf   = false
  public       = true
  routing_mode = "host"
  gateway_host = "gateway.example.com"
}

Host-based Routing:

app_gateway_config = {
  # ... other config ...
  routing_mode = "host"
  gateway_host = "gateway.example.com"
  mcp_host     = "mcp.example.com"  # if server_mode = "all"
}

Configure DNS:

gateway.example.com  A  <app-gateway-public-ip>
mcp.example.com      A  <app-gateway-public-ip>

SSL Certificate:

# Import certificate to Key Vault
az keyvault certificate import \
  --vault-name my-ssl-kv \
  --name my-ssl-cert \
  --file certificate.pfx \
  --password "<pfx-password>"

app_gateway_config = {
  # ... other config ...
  ssl_cert_key_vault_secret_id = "https://my-ssl-kv.vault.azure.net/secrets/my-ssl-cert"
  ssl_cert_key_vault_rg        = "my-ssl-kv-rg"
}

Private Application Gateway:

app_gateway_config = {
  # ... other config ...
  public = false
}

Azure Managed Redis

Use Azure Cache for Redis instead of the built-in container:

redis_config = {
  redis_type = "azure-redis"
  endpoint   = "rediss://<portkey-redis.redis.cache.windows.net>:6380"
  tls        = true
  mode       = "standalone"
}

Update secrets in main.tf:

secrets = {
  gateway = {
    PORTKEY_CLIENT_AUTH    = "portkey-client-auth"     # Key Vault secret name
    ORGANISATIONS_TO_SYNC  = "organisations-to-sync"   # Key Vault secret name
    REDIS_PASSWORD         = "redis-password"          # Key Vault secret name
  }
}

Note: The values above should be Key Vault secret names, not the actual secret values.

Storage Configuration

Using Auto-Created Storage (Default): No configuration needed. Terraform automatically creates a Storage Account and container. Optional: Customize container name:

storage_config = {
  container_name = "my-custom-container-name"
}

Using Existing Storage Account:

storage_config = {
  resource_group = "my-storage-account-rg"
  auth_mode      = "managed"
  account_name   = "my-storage-account-name"
  container_name = "my-container-name"
}

Integrating Gateway with Management Plane

Outbound Connectivity (Data Plane to Management Plane) The Data Plane integrates with the Management Plane over the internet.

Over the Internet

Ensure Gateway has access to the following endpoints over the internet:

https://aigw.portkey.ai
https://albus.portkey.ai

No additional configuration needed if your network allows outbound internet access.

Inbound Connectivity (Management Plane to Data Plane)

IP Whitelisting

Allows Management Plane to access the Data Plane over the internet by restricting inbound traffic to specific IP addresses. This method requires the Data Plane to have a publicly accessible endpoint. To whitelist, add an inbound rule to the Azure NSG or Firewall allowing connections from the AI Gateway Management Plane’s IPs on the required port. Contact the Palo Alto Networks team for the current address list. To integrate the Management Plane with the Data Plane, contact the Palo Alto Networks team and provide the Public Endpoint of the Data Plane.

Verifying Gateway Integration with the Management Plane

Send a test request to Gateway using curl.
Go to the AI Gateway website -> Logs.
Verify that the test request appears in the logs and that you can view its full details by selecting the log entry.

Uninstalling AI Gateway

terraform destroy

Example Configurations

Simple Deployment (No VNet)

This example shows a basic deployment with built-in Redis and auto-created storage: terraform.tfvars:

project_name        = "portkey-gateway"
environment         = "dev"
subscription_id     = "<your-subscription-id>"
resource_group_name = "portkey-rg"

registry_type = "dockerhub"
docker_credentials = {
  key_vault_name  = "portkey-kv"
  key_vault_rg    = "portkey-rg"
  username_secret = "docker-username"
  password_secret = "docker-password"
}

secrets_key_vault = {
  name           = "portkey-kv"
  resource_group = "portkey-rg"
}

network_mode   = "none"
ingress_type   = "aca"
public_ingress = true

redis_config = {
  redis_type = "redis"
}

storage_config = {
  container_name = "portkey-log-store"
}

server_mode = "gateway"

environment_variables = {
  gateway = {
    LOG_LEVEL          = "info"
    NODE_ENV           = "development"
    ANALYTICS_STORE    = "control_plane"
  }
}

secrets = {
  gateway = {
    PORTKEY_CLIENT_AUTH    = "portkey-client-auth"         # Key Vault secret name
    ORGANISATIONS_TO_SYNC  = "organisations-to-sync"       # Key Vault secret name
  }
}

Deployment with VNet and Application Gateway

This example shows a deployment with VNet, Application Gateway with WAF, and managed services: terraform.tfvars:

project_name        = "portkey-gateway"
environment         = "dev"
subscription_id     = "<your-subscription-id>"
resource_group_name = "portkey-rg"

registry_type = "dockerhub"
docker_credentials = {
  key_vault_name  = "portkey-kv"
  key_vault_rg    = "portkey-rg"
  username_secret = "docker-username"
  password_secret = "docker-password"
}

secrets_key_vault = {
  name           = "portkey-kv"
  resource_group = "portkey-rg"
}

# VNet Configuration
network_mode = "new"
vnet_cidr                = "10.0.0.0/16"

# Application Gateway with WAF
ingress_type = "application_gateway"
app_gateway_config = {
  sku_name                     = "WAF_v2"
  sku_tier                     = "WAF_v2"
  capacity                     = 2
  enable_waf                   = true
  public                       = true
  routing_mode                 = "host"
  gateway_host                 = "gateway.example.com"
  ssl_cert_key_vault_secret_id = "<https://my-ssl-kv.vault.azure.net/secrets/my-ssl-cert>"
}

# Azure Managed Redis
redis_config = {
  redis_type = "azure-redis"
  endpoint   = "rediss://portkey-redis.redis.cache.windows.net:6380"
  tls        = true
  mode       = "standalone"
}

# Existing Storage Account
storage_config = {
  resource_group = "my-storage-rg"
  auth_mode      = "managed"
  account_name   = "portkeysa"
  container_name = "portkey-log-store"
}

# Gateway Configuration
server_mode = "gateway"
gateway_config = {
  cpu                 = 2
  memory              = "4Gi"
  min_replicas        = 3
  max_replicas        = 30
  cpu_scale_threshold = 70
  port                = 8787
}

environment_variables = {
  gateway = {
    LOG_LEVEL          = "info"
    NODE_ENV           = "development"
    ANALYTICS_STORE    = "control_plane"
  }
}

secrets = {
  gateway = {
    PORTKEY_CLIENT_AUTH    = "portkey-client-auth"         # Key Vault secret name
    ORGANISATIONS_TO_SYNC  = "organisations-to-sync"       # Key Vault secret name
    REDIS_PASSWORD         = "redis-password"              # Key Vault secret name
  }
}

Gateway + MCP Deployment

This example shows how to deploy both AI Gateway and MCP Gateway: terraform.tfvars:

project_name        = "portkey-gateway"
environment         = "dev"
subscription_id     = "<your-subscription-id>"
resource_group_name = "portkey-rg"

registry_type = "dockerhub"
docker_credentials = {
  key_vault_name  = "portkey-kv"
  key_vault_rg    = "portkey-rg"
  username_secret = "docker-username"
  password_secret = "docker-password"
}

secrets_key_vault = {
  name           = "portkey-kv"
  resource_group = "portkey-rg"
}

network_mode = "new"
vnet_cidr                = "10.0.0.0/16"

ingress_type = "application_gateway"
app_gateway_config = {
  sku_name     = "Standard_v2"
  sku_tier     = "Standard_v2"
  capacity     = 2
  enable_waf   = false
  public       = true
  routing_mode = "host"
  gateway_host = "gateway.example.com"
  mcp_host     = "mcp.example.com"
}

redis_config = {
  redis_type = "redis"
}

storage_config = {
  container_name = "portkey-log-store"
}

# Deploy both Gateway and MCP
server_mode = "all"

mcp_gateway_base_url = "https://mcp.example.com"

gateway_config = {
  cpu          = 1
  memory       = "2Gi"
  min_replicas = 2
  max_replicas = 10
  port         = 8787
}

mcp_config = {
  cpu          = 0.5
  memory       = "1Gi"
  min_replicas = 1
  max_replicas = 5
  port         = 8788
}

environment_variables = {
  gateway = {
    LOG_LEVEL          = "info"
    NODE_ENV           = "development"
    ANALYTICS_STORE    = "control_plane"
  }
}

secrets = {
  gateway = {
    PORTKEY_CLIENT_AUTH    = "portkey-client-auth"         # Key Vault secret name
    ORGANISATIONS_TO_SYNC  = "organisations-to-sync"       # Key Vault secret name
  }
}

​Components and Sizing Recommendations

​Prerequisites

​Create a Strata Cloud Manager account

​Setup Project Environment

​1. Prepare Azure Resources

​2. Create Terraform Configuration Files

​3. Create Module Configuration

​Advanced Configuration

​MCP Gateway (Optional)

​Auto-Scaling Configuration

​Network Configuration with VNet

​Application Gateway Ingress

​Azure Managed Redis

​Storage Configuration

​Integrating Gateway with Management Plane

​Over the Internet

​Inbound Connectivity (Management Plane to Data Plane)

​IP Whitelisting

​Verifying Gateway Integration with the Management Plane

​Uninstalling AI Gateway

​Example Configurations

​Simple Deployment (No VNet)

​Deployment with VNet and Application Gateway

​Gateway + MCP Deployment