Multi-Cloud Strategy: Avoiding Vendor Lock-in with Terraform and Crossplane

Vendor lock-in occurs when infrastructure provisioning depends on proprietary APIs, resource types, or management tools unique to a single cloud provider. This guide demonstrates how Terraform and Crossplane create abstraction layers that enable multi-cloud portability through standardized resource definitions and control plane orchestration.

The Abstraction Layer Strategy

Multi-cloud portability requires decoupling intent from implementation. Define infrastructure requirements in provider-agnostic terms, then map those requirements to cloud-specific resources. This approach prevents lock-in at the API level and allows migration between providers with minimal code changes.

Terraform excels at provisioning static infrastructure through modules that encapsulate provider-specific logic. Crossplane provides a continuous control plane that manages infrastructure lifecycle using Kubernetes-native APIs. Together, they create a comprehensive abstraction layer spanning initial provisioning and ongoing management.

Terraform: Infrastructure as Code Foundation

Terraform uses HCL (HashiCorp Configuration Language) to define desired infrastructure state. Providers translate these definitions into cloud-specific API calls. Modules create reusable abstractions that hide provider differences behind common interfaces.

Multi-Cloud Module Pattern

Create a module that accepts cloud provider selection as a variable and outputs standardized resource identifiers:

# modules/storage/main.tf
variable "cloud_provider" {
  type        = string
  description = "Cloud provider: aws, azure, or gcp"
  validation {
    condition     = contains(["aws", "azure", "gcp"], var.cloud_provider)
    error_message = "Provider must be aws, azure, or gcp"
  }
}

variable "storage_size_gb" {
  type        = number
  description = "Storage size in gigabytes (informational for object storage)"
}

variable "resource_group" {
  type        = string
  description = "Azure resource group name"
  default     = "default"
}

variable "location" {
  type        = string
  description = "Azure region"
  default     = "eastus"
}

variable "region" {
  type        = string
  description = "GCP region"
  default     = "us-central1"
}

resource "random_id" "bucket_suffix" {
  byte_length = 4
}

resource "aws_s3_bucket" "storage" {
  count = var.cloud_provider == "aws" ? 1 : 0
  bucket = "storage-${random_id.bucket_suffix.hex}"
}

resource "azurerm_storage_account" "storage" {
  count                     = var.cloud_provider == "azure" ? 1 : 0
  name                      = "storage${random_id.bucket_suffix.hex}"
  resource_group_name       = var.resource_group
  location                  = var.location
  account_tier              = "Standard"
  account_replication_type  = "LRS"
}

resource "google_storage_bucket" "storage" {
  count          = var.cloud_provider == "gcp" ? 1 : 0
  name          = "storage-${random_id.bucket_suffix.hex}"
  location      = var.region
  force_destroy = true
}

output "storage_id" {
  value = var.cloud_provider == "aws" ? aws_s3_bucket.storage[0].id :
          var.cloud_provider == "azure" ? azurerm_storage_account.storage[0].id :
          google_storage_bucket.storage[0].id
}

output "storage_endpoint" {
  value = var.cloud_provider == "aws" ? aws_s3_bucket.storage[0].bucket_domain_name :
          var.cloud_provider == "azure" ? azurerm_storage_account.storage[0].primary_blob_endpoint :
          google_storage_bucket.storage[0].url
}

The module uses conditional resources based on the provider selection. Outputs normalize identifiers and endpoints regardless of the underlying cloud, enabling downstream systems to consume resources without provider-specific logic. Note that object storage services (S3, Azure Blob, GCS) are elastic and do not enforce fixed size limits; the storage_size_gb variable serves informational purposes for capacity planning.

State Management Considerations

Terraform state files remain provider-specific. Use remote backends with workspace isolation for each cloud provider. Avoid mixing providers in a single state file to simplify migration and rollback scenarios.

Crossplane: Continuous Control Plane

Crossplane extends Kubernetes to manage external infrastructure as custom resources. It functions as a control plane that continuously reconciles actual state with desired state, automatically detecting and correcting drift without manual intervention.

Composite Resource Definitions (XRDs)

XRDs define provider-agnostic resource types that applications consume:

# xrd-storageclass.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: storageclasses.platform.example.com
spec:
  group: platform.example.com
  names:
    kind: StorageClass
    plural: storageclasses
  claimNames:
    kind: StorageClassClaim
    plural: storageclassclaims
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              sizeGB:
                type: integer
              provider:
                type: string
                enum: [aws, azure, gcp]
              region:
                type: string
            required:
            - sizeGB
            - provider

The XRD defines a custom resource type that abstracts storage across clouds. Applications request storage by creating a StorageClassClaim without knowing the underlying implementation.

Composition Mapping

Compositions map XRDs to provider-specific managed resources. When using mode: Pipeline, install the required function first:

kubectl apply -f https://raw.githubusercontent.com/crossplane-contrib/function-patch-and-transform/main/package/function-patch-and-transform.yaml

# composition-aws-storage.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: aws-storageclass
spec:
  compositeTypeRef:
    apiVersion: platform.example.com/v1alpha1
    kind: StorageClass
  mode: Pipeline
  pipeline:
  - step: patch-provider
    functionRef:
      name: crossplane-function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
      - name: s3-bucket
        base:
          apiVersion: s3.aws.crossplane.io/v1beta1
          kind: Bucket
          spec:
            forProvider:
              location: us-east-1
            providerConfigRef:
              name: aws-provider
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.region
          toFieldPath: spec.forProvider.location
        - type: FromCompositeFieldPath
          fromFieldPath: spec.sizeGB
          toFieldPath: metadata.annotations[platform.example.com/size-gb]
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.uid
          toFieldPath: metadata.annotations[crossplane.io/external-name]
          transforms:
          - type: string
            string:
              fmt: "storage-%s"
  matchLabels:
    storageclass.platform.example.com/provider: aws

The composition creates an AWS S3 bucket when the XRD specifies provider: aws. The pipeline transforms the abstract specification into provider-specific configuration. Note that S3 buckets do not have a configurable size attribute; the size value is stored as an annotation for reference. The region patch correctly overrides the default location with the value from the claim.

Continuous Reconciliation

Crossplane controllers continuously monitor managed resources. When drift occurs (manual changes outside the control plane), Crossplane automatically reconciles to restore the desired state. This capability differs from Terraform's on-demand reconciliation, which requires explicit terraform apply commands.

Hybrid Architecture: Combining Terraform and Crossplane

Terraform and Crossplane solve different problems. Use Terraform for initial infrastructure bootstrap and Crossplane for ongoing lifecycle management of cloud-native resources.

Terraform Bootstrap

# main.tf
variable "region" {
  type        = string
  description = "AWS region"
  default     = "us-east-1"
}

provider "aws" {
  region = var.region
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "platform-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["${var.region}a", "${var.region}b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]
}

provider "kubernetes" {
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
  }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.0"

  cluster_name    = "platform-cluster"
  cluster_version = "1.28"
  
  vpc_id          = module.vpc.vpc_id
  subnet_ids      = module.vpc.private_subnets

  eks_managed_node_groups = {
    default = {
      instance_types = ["m6i.xlarge"]
      min_size       = 2
      max_size       = 5
      desired_size   = 3
    }
  }
}

resource "helm_release" "crossplane" {
  name       = "crossplane"
  repository = "https://charts.crossplane.io/stable"
  chart      = "crossplane"
  namespace  = "crossplane-system"
  create_namespace = true

  set {
    name  = "args.security-pod-security-enforcement"
    value = "privileged"
  }
}

Terraform provisions the VPC, Kubernetes cluster, and installs Crossplane. The cluster becomes the control plane for subsequent resource management.

Crossplane Resource Consumption

Applications consume infrastructure through Kubernetes manifests:

# application-storage.yaml
apiVersion: platform.example.com/v1alpha1
kind: StorageClassClaim
metadata:
  name: app-storage
  namespace: production
  labels:
    storageclass.platform.example.com/provider: aws
spec:
  sizeGB: 100
  provider: aws
  region: us-west-2

Crossplane provisions the actual S3 bucket in us-west-2 and continuously manages its lifecycle. Changing provider to azure triggers automatic provisioning of equivalent Azure Storage without modifying application code.

Implementation Pitfalls

1. Over-Abstraction: Excessive abstraction creates lowest-common-denominator infrastructure that ignores cloud-specific optimizations. Allow provider-specific parameters to escape the abstraction layer when necessary.

2. State Fragmentation: Terraform state and Crossplane resources can become desynchronized. Establish clear ownership boundaries: Terraform owns bootstrap resources, Crossplane owns operational resources.

3. Feature Parity: Cloud providers offer different capabilities. Design abstractions around common denominators or implement feature flags that disable functionality on unsupported platforms.

Getting Started

1. Define Abstraction Boundaries: Identify resources suitable for multi-cloud abstraction (storage, compute, networking) versus those requiring cloud-specific implementation (managed services, AI/ML platforms).

2. Create Terraform Modules: Build provider-specific modules with standardized input/output interfaces. Test modules against all target clouds to ensure consistent behavior.

3. Install Crossplane: Deploy Crossplane to your Kubernetes cluster using the Helm chart. Install required composition functions like crossplane-function-patch-and-transform for pipeline mode. Configure provider credentials for each target cloud.

4. Define XRDs and Compositions: Create composite resource definitions that model your infrastructure requirements. Write compositions for each cloud provider mapping XRDs to managed resources. Ensure composition selectors match XRD labels or spec fields correctly.

5. Implement GitOps: Store Terraform configurations and Crossplane manifests in a Git repository. Use ArgoCD or Flux to synchronize changes, ensuring infrastructure and application definitions remain version-controlled and auditable.