ECS - Portkey Docs

Components and Sizing Recommendations

Component	Options	Sizing Recommendations
AI Gateway	Deploy in your ECS cluster using Terraform.	Use Amazon ECS tasks with at least 1 vCPU (1024 CPU units) and 2 GiB of memory per task. For high availability, run tasks across multiple Availability Zones with auto-scaling enabled.
Logs Store (optional)	Amazon S3 or S3-compatible Storage	Each log document is ~10kb in size (uncompressed)
Cache (Prompts, Configs & Providers)	Built-in Redis or Amazon ElastiCache for Redis OSS or Valkey	Deployed within the same VPC as the AI Gateway.

Prerequisites

Ensure the following tools and resources are installed and available:

AWS Account with permissions to create ECS, EC2, VPC, ELB, IAM, S3, Secrets Manager, and CloudWatch resources.
AWS CLI configured with credentials.
Terraform v1.13 or later.

Create a Strata Cloud Manager account

Go to the the AI Gateway website.
Sign up for a Strata Cloud Manager account.
Once logged in, locate and save your Organisation ID for future reference. It can be found in the browser URL: https://stratacloudmanager.paloaltonetworks.com/<organisation_id>/
Contact the Palo Alto Networks team and provide your Organisation ID and the email address used during signup.
The Palo Alto Networks team will share the following information with you:
- Docker credentials for the Gateway images (username and password).
- License: Client Auth Key.

Setup Project Environment

1. Prepare AWS Secrets

Create the required secrets in AWS Secrets Manager. You can either use the CloudFormation template provided in the portkey-gateway-infrastructure repository, or create them manually using the AWS CLI. Option A: Using CloudFormation

Go to the AWS CloudFormation Console and create a stack.
Upload cloudformation/secrets.yaml from the portkey-gateway-infrastructure repository.
Provide the following parameters:
- Project Name — e.g., portkey-gateway
- Environment — e.g., dev
- Docker Username / Password — provided by the AI Gateway
- AI Gateway Client Auth — provided by the AI Gateway
- Organisations — your AI Gateway Organisation ID(s), comma-separated if multiple
After the stack completes, note the following outputs for use in the Terraform configuration:
- DockerCredentialsSecretArn
- ClientOrgSecretNameArn

Option B: Using AWS CLI

project_name=portkey-gateway                           # Provide a name for the project
environment=dev                                        # Provide the environment name
aws_region=us-east-1                                   # Provide the AWS region

# Store Docker credentials shared by Portkey
aws secretsmanager create-secret \
  --name ${project_name}/${environment}/docker-credentials \
  --region ${aws_region} \
  --secret-string '{"username":"<docker-username>","password":"<docker-password>"}'

# Store Portkey client auth and organisation ID
aws secretsmanager create-secret \
  --name ${project_name}/${environment}/client-org \
  --region ${aws_region} \
  --secret-string '{"PORTKEY_CLIENT_AUTH":"<client-auth>","ORGANISATIONS_TO_SYNC":"<organisation-id>"}'

Note the ARNs returned for both secrets — they will be used in the Terraform configuration.

2. Create Terraform Configuration Files

Create a new directory for your deployment:

mkdir portkey-gateway-deployment
cd portkey-gateway-deployment

(Optional) Create an S3 bucket to store the Terraform state remotely:

aws s3api create-bucket \
  --bucket portkey-tfstate-<account-id> \
  --region us-east-1

aws s3api put-bucket-versioning \
  --bucket portkey-tfstate-<account-id> \
  --versioning-configuration Status=Enabled

Create a backend.config file:

bucket = "portkey-tfstate-<account-id>"
key    = "portkey-gateway/dev.tfstate"
region = "us-east-1"

3. Create Module Configuration

Create a main.tf file:

terraform {
  required_version = ">= 1.13"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }

  backend "s3" {
    use_lockfile = true
  }
}

provider "aws" {
  region = "us-east-1"

  default_tags {
    tags = {
      Environment = "dev"
      ManagedBy   = "Terraform"
      Project     = "portkey-gateway"
    }
  }
}

module "portkey_gateway" {
  source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/ecs?ref=v2.0.0"      # Change module version as per requirement

  # Project Configuration
  project_name = "portkey-gateway"
  environment  = "dev"
  aws_region   = "us-east-1"

  # Docker Credentials (Secrets Manager ARN)
  docker_cred_secret_arn = "<DockerCredentialsSecretArn>"

  # Network Configuration
  create_new_vpc     = true
  vpc_cidr           = "10.0.0.0/16"
  num_az             = 2
  single_nat_gateway = true

  # ECS Cluster Configuration
  create_cluster   = true
  instance_type    = "t4g.medium"
  min_asg_size     = 1
  max_asg_size     = 2
  desired_asg_size = 1

  # Server Mode Configuration
  server_mode = "gateway"                                                   # Set to "all" to deploy both AI Gateway and MCP Gateway

  # Gateway Configuration
  gateway_config = {
    desired_task_count = 1
    cpu                = 1024
    memory             = 2048
    gateway_port       = 8787
    mcp_port           = 8788
  }

  # Redis Configuration (built-in)
  redis_configuration = {
    redis_type = "redis"
    cpu        = 256
    memory     = 512
    endpoint   = ""
    tls        = false
    mode       = "standalone"
  }

  # Object Storage (S3 Log Store)
  object_storage = {
    log_store_bucket = "<your-logs-bucket>"
    bucket_region    = "us-east-1"
  }

  # Load Balancer Configuration
  create_lb        = true
  internal_lb      = true                                                   # Set to false to create an internet-facing Load Balancer
  lb_type          = "network"                                              # "network" for NLB, "application" for ALB
  allowed_lb_cidrs = ["<X.X.X.X/Y>"]                                        # CIDR ranges allowed to reach the LB (e.g., the VPC CIDR for an internal LB)

  # Environment Variables
  environment_variables = {
    gateway = {
      SERVICE_NAME    = "gateway"
      ANALYTICS_STORE = "control_plane"
      LOG_STORE       = "s3_assume"
    }
  }

  # Secrets (Secrets Manager ARNs)
  secrets = {
    gateway = {
      PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
      ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
    }
  }
}

output "load_balancer_dns_name" {
  value = module.portkey_gateway.load_balancer_dns_name
}

output "vpc_id" {
  value = module.portkey_gateway.vpc_id
}

4. Deploy the Gateway

terraform init -backend-config=backend.config
terraform plan
terraform apply

Note: Values in the secrets block must be AWS Secrets Manager ARNs, not raw secret values. The ECS task definition references the secret ARN directly and AWS injects the secret value at runtime.

Advanced Configuration

MCP Gateway (Optional)

By default, only the AI Gateway is enabled. To enable the MCP Gateway, update your module configuration: MCP Only:

server_mode          = "mcp"
mcp_gateway_base_url = "https://mcp.example.com"             # MCP external domain clients use to reach MCP

gateway_config = {
  desired_task_count = 1
  cpu                = 256
  memory             = 1024
  gateway_port       = 8787
  mcp_port           = 8788
}

Gateway + MCP (single service, ALB required):

server_mode          = "all"
mcp_gateway_base_url = "https://mcp.example.com"

create_lb        = true
lb_type          = "application"
allowed_lb_cidrs = ["<X.X.X.X/Y>"]                                          # CIDR ranges allowed to reach the ALB

alb_routing_configuration = {
  enable_host_based_routing = true
  gateway_host              = "gateway.example.com"
  mcp_host                  = "mcp.example.com"
}

gateway_config = {
  desired_task_count = 2
  cpu                = 1024
  memory             = 2048
  gateway_port       = 8787
  mcp_port           = 8788
}

Notes:

mcp_gateway_base_url is required when server_mode is "mcp" or "all". It must be the MCP external domain (with https:// or http:// prefix) that clients use to reach the MCP service.
When server_mode = "all", an Application Load Balancer is required (lb_type = "application") and you must configure host-based routing via alb_routing_configuration (gateway_host and mcp_host).
For the initial deployment, you can set mcp_gateway_base_url to a placeholder, then update it after the Load Balancer is provisioned and DNS is mapped.

Server Modes

"gateway": Deploys only the AI Gateway. This is the default configuration.
"mcp": Deploys only the MCP Gateway. Requires mcp_gateway_base_url.
"all": Deploys both the AI Gateway and MCP Gateway. Requires mcp_gateway_base_url and an ALB with host-based routing.

Auto-Scaling Configuration

Control how ECS tasks scale based on CPU and memory utilisation:

gateway_autoscaling = {
  enable_autoscaling        = true
  autoscaling_min_capacity  = 3
  autoscaling_max_capacity  = 20
  target_cpu_utilization    = 70
  target_memory_utilization = 80
  scale_in_cooldown         = 120
  scale_out_cooldown        = 60
}

Deployment Strategies

ECS supports multiple deployment strategies via gateway_deployment_configuration: Blue/Green Deployment:

gateway_deployment_configuration = {
  enable_blue_green = true
}

Canary Deployment:

gateway_deployment_configuration = {
  enable_blue_green = false
  canary_configuration = {
    canary_bake_time_in_minutes = 5
    canary_percent              = 10
  }
}

Network Configuration with VPC

Deploy the Gateway within a VPC. Create a new VPC:

create_new_vpc     = true
vpc_cidr           = "10.0.0.0/16"
num_az             = 2
single_nat_gateway = true

Use an existing VPC and subnets:

create_new_vpc     = false
vpc_id             = "vpc-xxxxxxxxxxxxxxxxx"
public_subnet_ids  = ["subnet-xxxxxxxx", "subnet-yyyyyyyy"]
private_subnet_ids = ["subnet-aaaaaaaa", "subnet-bbbbbbbb"]

Load Balancer Ingress

The module can provision either an Application Load Balancer (ALB) or a Network Load Balancer (NLB) in front of the Gateway tasks. Pick based on what you need:

Requirement	Use
Host-based routing (required when `server_mode = "all"`)	ALB
HTTP/HTTPS-aware routing, WAF, ALB access logs	ALB
Layer-4 pass-through	NLB
Lowest-latency, simple TCP forwarding	NLB

Application Load Balancer (ALB)

Deploy a public ALB with TLS termination:

create_lb           = true
internal_lb         = false                                                 # true for an internal ALB
lb_type             = "application"
tls_certificate_arn = "arn:aws:acm:us-east-1:123456789012:certificate/xxxxxxxx"
allowed_lb_cidrs    = ["<X.X.X.X/Y>"]

Host-based Routing (required when server_mode = "all"):

alb_routing_configuration = {
  enable_host_based_routing = true
  gateway_host              = "gateway.example.com"
  mcp_host                  = "mcp.example.com"
}

Configure DNS:

gateway.example.com  A/CNAME  <alb-dns-name>
mcp.example.com      A/CNAME  <alb-dns-name>

Enable ALB Access Logs:

enable_lb_access_logs = true
lb_access_logs_bucket = "portkey-alb-access-logs"

Network Load Balancer (NLB)

create_lb        = true
internal_lb      = true                                                     # false for an internet-facing NLB
lb_type          = "network"
allowed_lb_cidrs = ["<X.X.X.X/Y>"]                                          # CIDR ranges allowed to reach the NLB (e.g., the VPC CIDR for an internal NLB)

Note: NLB does not support host-based routing, so it cannot be used when server_mode = "all".

Amazon ElastiCache for Redis

Use an existing Amazon ElastiCache cluster instead of the built-in Redis container:

redis_configuration = {
  redis_type = "aws-elastic-cache"
  cpu        = 256                                                            # Ignored for ElastiCache
  memory     = 512                                                            # Ignored for ElastiCache
  endpoint   = "master.portkey-redis.xxxxx.use1.cache.amazonaws.com:6379"     # Primary or Configuration endpoint
  tls        = true                                                           # Match the cluster's transit encryption setting
  mode       = "standalone"                                                   # or "cluster" for cluster-mode-enabled
}

If ElastiCache AUTH is enabled, store the AUTH token in AWS Secrets Manager (as JSON with a REDIS_PASSWORD key) and reference the secret ARN. Both the Gateway and Data Service connect to Redis, so the secret must be passed to both blocks if the Data Service is enabled:

secrets = {
  gateway = {
    PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
    ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
    REDIS_PASSWORD        = "<RedisAuthSecretArn>"
  }
  data-service = {
    PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
    ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
    REDIS_PASSWORD        = "<RedisAuthSecretArn>"
  }
}

Note: ElastiCache’s security groups must allow inbound TCP 6379 (or your configured port) from the Gateway and Data Service task security groups.

Object Storage (S3 Log Store)

Specify the S3 bucket for storing LLM access logs:

object_storage = {
  log_store_bucket   = "portkey-prod-logs"
  log_exports_bucket = "portkey-prod-exports"          # Optional, used by Data Service for log exports
  bucket_region      = "us-east-1"
}

The module attaches an IAM policy to the Gateway task role granting s3:PutObject and s3:GetObject permissions on the configured buckets.

Data Service (Optional)

The Data Service is responsible for batch processing, fine-tuning, and log exports. Enable it via:

dataservice_config = {
  enable_dataservice = true
  desired_task_count = 2
  cpu                = 512
  memory             = 1024
}

environment_variables = {
  data-service = {
    SERVICE_NAME      = "data-service"
    ANALYTICS_STORE   = "control_plane"
    LOG_STORE         = "s3_assume"
    HYBRID_DEPLOYMENT = "ON"
  }
}

secrets = {
  data-service = {
    PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
    ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
  }
}

Amazon Bedrock (Optional)

To allow the Gateway to invoke Amazon Bedrock models, attach an IAM policy with the required bedrock:InvokeModel (and related) permissions to the Gateway ECS task role via gateway_task_role_policy_arns:

gateway_task_role_policy_arns = {
  bedrock = "<IAM_POLICY_ARN>"
}

The module supports both same-account and cross-account Bedrock access (via sts:AssumeRole). For the full IAM policy documents, trust policy templates, and step-by-step setup for both modes, refer to following guide: Bedrock Access Configuration.

Integrating Gateway with Management Plane

Outbound Connectivity (Data Plane to Management Plane) The Data Plane integrates with the Management Plane over the internet.

Over the Internet

Ensure the Gateway has access to the following endpoints over the internet:

https://aigw.portkey.ai
https://albus.portkey.ai

No additional configuration is needed if your VPC allows outbound internet access via a NAT Gateway.

Inbound Connectivity (Management Plane to Data Plane)

IP Whitelisting

Allows the Management Plane to access the Data Plane over the internet by restricting inbound traffic to specific IP addresses of the Management Plane. This method requires the Data Plane to have a publicly accessible endpoint. To whitelist, add an inbound rule to the Load Balancer’s security group allowing connections from the AI Gateway Management Plane’s IPs on the listener port. Contact the Palo Alto Networks team for the current address list. Alternatively, set allowed_lb_cidrs in the module configuration:

# Replace with the Management Plane CIDRs provided by the Palo Alto Networks team
allowed_lb_cidrs = ["203.0.113.10/32", "203.0.113.11/32"]

To integrate the Management Plane with the Data Plane, contact the Palo Alto Networks team and provide the Public Endpoint of the Data Plane.

Verifying Gateway Integration with the Management Plane

Send a test request to the Gateway using curl:

# Replace <GATEWAY_ENDPOINT> with the Load Balancer DNS or your custom hostname
OPENAI_API_KEY=<OPENAI_API_KEY>
PORTKEY_API_KEY=<PORTKEY_API_KEY>

curl 'http://<GATEWAY_ENDPOINT>/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is a fractal?"}]
  }'

Go to the the AI Gateway website -> Logs.
Verify that the test request appears in the logs and that you can view its full details by selecting the log entry.

Uninstalling AI Gateway

terraform destroy

Example Configurations

Minimal Development Deployment

This example shows a basic deployment with built-in Redis, a new VPC, and no load balancer:

module "portkey_gateway" {
  source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/ecs?ref=v2.0.0"

  project_name = "portkey-gateway"
  environment  = "dev"
  aws_region   = "us-east-1"

  docker_cred_secret_arn = "<DockerCredentialsSecretArn>"

  environment_variables = {
    gateway = {
      SERVICE_NAME    = "gateway"
      ANALYTICS_STORE = "control_plane"
      LOG_STORE       = "s3_assume"
    }
  }

  secrets = {
    gateway = {
      PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
      ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
    }
  }

  create_new_vpc     = true
  vpc_cidr           = "10.0.0.0/16"
  num_az             = 2
  single_nat_gateway = true

  create_cluster   = true
  instance_type    = "t4g.medium"
  desired_asg_size = 1
  min_asg_size     = 1
  max_asg_size     = 2

  server_mode = "gateway"

  gateway_config = {
    desired_task_count = 1
    cpu                = 256
    memory             = 1024
    gateway_port       = 8787
    mcp_port           = 8788
  }

  redis_configuration = {
    redis_type = "redis"
    cpu        = 256
    memory     = 512
    endpoint   = ""
    tls        = false
    mode       = "standalone"
  }

  object_storage = {
    log_store_bucket = "<dev-logs-bucket>"
    bucket_region    = "us-east-1"
  }

  create_lb        = true
  internal_lb      = true
  lb_type          = "network"
  allowed_lb_cidrs = ["<X.X.X.X/Y>"]
}

Production Deployment with ALB, ElastiCache, and Data Service

This example shows a production-grade deployment with a public ALB, Amazon ElastiCache, the Data Service enabled, and auto-scaling:

module "portkey_gateway" {
  source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/ecs?ref=v2.0.0"

  project_name = "portkey-gateway"
  environment  = "prod"
  aws_region   = "us-east-1"

  docker_cred_secret_arn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/docker-credentials"

  environment_variables = {
    gateway = {
      SERVICE_NAME    = "gateway"
      ANALYTICS_STORE = "control_plane"
      LOG_STORE       = "s3_assume"
    }
    data-service = {
      SERVICE_NAME      = "data-service"
      ANALYTICS_STORE   = "control_plane"
      LOG_STORE         = "s3_assume"
      HYBRID_DEPLOYMENT = "ON"
    }
  }

  secrets = {
    gateway = {
      PORTKEY_CLIENT_AUTH   = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/client-org"
      ORGANISATIONS_TO_SYNC = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/client-org"
      REDIS_PASSWORD        = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/redis-auth"
    }
    data-service = {
      PORTKEY_CLIENT_AUTH   = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/client-org"
      ORGANISATIONS_TO_SYNC = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/client-org"
      REDIS_PASSWORD        = "arn:aws:secretsmanager:us-east-1:123456789012:secret:portkey-gateway/prod/redis-auth"
    }
  }

  # Network
  create_new_vpc     = true
  vpc_cidr           = "10.0.0.0/16"
  num_az             = 3
  single_nat_gateway = false

  # ECS Cluster
  create_cluster   = true
  instance_type    = "t4g.large"
  min_asg_size     = 2
  max_asg_size     = 10
  desired_asg_size = 3

  server_mode = "gateway"

  gateway_config = {
    desired_task_count = 3
    cpu                = 1024
    memory             = 2048
    gateway_port       = 8787
    mcp_port           = 8788
  }

  gateway_autoscaling = {
    enable_autoscaling        = true
    autoscaling_min_capacity  = 3
    autoscaling_max_capacity  = 20
    target_cpu_utilization    = 70
    target_memory_utilization = 80
    scale_in_cooldown         = 120
    scale_out_cooldown        = 60
  }

  gateway_deployment_configuration = {
    enable_blue_green = true
  }

  # Data Service
  dataservice_config = {
    enable_dataservice = true
    desired_task_count = 2
    cpu                = 512
    memory             = 1024
  }

  # Amazon ElastiCache
  redis_configuration = {
    redis_type = "aws-elastic-cache"
    cpu        = 256
    memory     = 512
    endpoint   = "prod-redis.xxxxx.cache.amazonaws.com:6379"
    tls        = true
    mode       = "cluster"
  }

  # S3 Log Store
  object_storage = {
    log_store_bucket   = "portkey-prod-logs"
    log_exports_bucket = "portkey-prod-exports"
    bucket_region      = "us-east-1"
  }

  # Public ALB with TLS
  create_lb           = true
  internal_lb         = false
  lb_type             = "application"
  tls_certificate_arn = "arn:aws:acm:us-east-1:123456789012:certificate/xxxxxxxx"
  allowed_lb_cidrs    = ["<X.X.X.X/Y>"]

  enable_lb_access_logs = true
  lb_access_logs_bucket = "portkey-alb-access-logs"
}

Gateway + MCP Deployment

This example shows how to deploy both the AI Gateway and the MCP Gateway behind an Application Load Balancer with host-based routing:

module "portkey_gateway" {
  source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/ecs?ref=v2.0.0"

  project_name = "portkey-gateway"
  environment  = "prod"
  aws_region   = "us-east-1"

  docker_cred_secret_arn = "<DockerCredentialsSecretArn>"

  environment_variables = {
    gateway = {
      SERVICE_NAME    = "gateway"
      ANALYTICS_STORE = "control_plane"
      LOG_STORE       = "s3_assume"
    }
  }

  secrets = {
    gateway = {
      PORTKEY_CLIENT_AUTH   = "<ClientOrgSecretNameArn>"
      ORGANISATIONS_TO_SYNC = "<ClientOrgSecretNameArn>"
    }
  }

  create_new_vpc     = true
  vpc_cidr           = "10.0.0.0/16"
  num_az             = 2
  single_nat_gateway = true

  create_cluster   = true
  instance_type    = "t4g.large"
  min_asg_size     = 2
  max_asg_size     = 6
  desired_asg_size = 2

  # Deploy both Gateway and MCP
  server_mode          = "all"
  mcp_gateway_base_url = "https://mcp.example.com"

  gateway_config = {
    desired_task_count = 2
    cpu                = 1024
    memory             = 2048
    gateway_port       = 8787
    mcp_port           = 8788
  }

  redis_configuration = {
    redis_type = "redis"
    cpu        = 256
    memory     = 512
    endpoint   = ""
    tls        = false
    mode       = "standalone"
  }

  object_storage = {
    log_store_bucket = "portkey-logs"
    bucket_region    = "us-east-1"
  }

  # ALB with host-based routing
  create_lb           = true
  internal_lb         = false
  lb_type             = "application"
  tls_certificate_arn = "arn:aws:acm:us-east-1:123456789012:certificate/xxxxxxxx"
  allowed_lb_cidrs    = ["<X.X.X.X/Y>"]

  alb_routing_configuration = {
    enable_host_based_routing = true
    gateway_host              = "gateway.example.com"
    mcp_host                  = "mcp.example.com"
  }
}

Multi-Environment Setup

To manage dev, staging, and prod from a single codebase, organise your project as follows:

my-infrastructure/
├── dev/
│   ├── main.tf
│   ├── backend.config
│   └── terraform.tfvars
├── staging/
│   ├── main.tf
│   ├── backend.config
│   └── terraform.tfvars
└── prod/
    ├── main.tf
    ├── backend.config
    └── terraform.tfvars

Each environment has its own remote state and variable values:

# dev/backend.config
bucket = "portkey-tfstate-<account-id>"
key    = "portkey-gateway/dev.tfstate"
region = "us-east-1"

# prod/backend.config
bucket = "portkey-tfstate-<account-id>"
key    = "portkey-gateway/prod.tfstate"
region = "us-east-1"

Deploy each environment independently:

# Dev
cd dev
terraform init -backend-config=backend.config
terraform apply -var-file=terraform.tfvars

# Prod
cd ../prod
terraform init -backend-config=backend.config
terraform apply -var-file=terraform.tfvars

Version Pinning and Upgrades

Always pin the module to a specific version in production:

# Recommended - pinned to v2.0.0
source = "github.com/Portkey-AI/portkey-gateway-infrastructure//terraform/ecs?ref=v2.0.0"

To upgrade, review the release notes, test the new version in a non-production environment first, then promote to production:

terraform init -upgrade
terraform plan       # Review changes carefully
terraform apply

To roll back, revert the ref to the previous version and re-run terraform init -upgrade && terraform apply.

​Components and Sizing Recommendations

​Prerequisites

​Create a Strata Cloud Manager account

​Setup Project Environment

​1. Prepare AWS Secrets

​2. Create Terraform Configuration Files

​3. Create Module Configuration

​4. Deploy the Gateway

​Advanced Configuration

​MCP Gateway (Optional)

​Auto-Scaling Configuration

​Deployment Strategies

​Network Configuration with VPC

​Load Balancer Ingress

​Application Load Balancer (ALB)

​Network Load Balancer (NLB)

​Amazon ElastiCache for Redis

​Object Storage (S3 Log Store)

​Data Service (Optional)

​Amazon Bedrock (Optional)

​Integrating Gateway with Management Plane

​Over the Internet

​Inbound Connectivity (Management Plane to Data Plane)

​IP Whitelisting

​Verifying Gateway Integration with the Management Plane

​Uninstalling AI Gateway

​Example Configurations

​Minimal Development Deployment

​Production Deployment with ALB, ElastiCache, and Data Service

​Gateway + MCP Deployment

​Multi-Environment Setup

​Version Pinning and Upgrades