Terraform Deployment Guide

Install Terraform

The easiest way to install Terraform on Windows is to use Chocolatey. Installation information for other platform can be found here

choco install terraform

Clone Brighterraform Repository

We use a Github repository to store our templates and modules that are used to deploy AWS resources. You can clone the repository using the command below.

git clone git@ghe.intra.nudatasecurity.com:ai-lifecycle/brighterraform.git

Configure AWS Credentials

You need to configure your AWS credentials before deploying AWS resources. You can copy the credentials from AWS SSO page ("Command line or programmatic access" link). To set the credentials, copy them to ~/.aws/credentials. Your file should have the following structure:

[default]
aws_access_key_id=...
aws_secret_access_key=...
aws_session_token=...

Configure Remote State for Terraform

It's important to configure remote state before deploying any AWS resources. Terraform stores the information about deployed resources in a local file by default. If you don't configure a remote state file, other people will likely mess up deployed resources when they apply your Terraform template. Note that our existing templates e.g. "cms-demo" already have remote state configured. DO NOT RE-USE EXISTING REMOTE STATE FOR YOUR RESOURCES!!! Re-using remote state to deploy a new set of resources might delete the resources that have already been deployed.

Here are the steps to configure a remote state:

  • Create an s3 bucket that will store the remote state.
  • Create a DynamoDB table that will be used as a lock to prevent multiple users from updating the state file. The table needs to have a primary key named LockID. There is already a table named terraform-state-lock-dynamo in our dev account, so you can re-use it.
  • Create backend.tf to store remote state configure. It should look like this:
terraform {
  backend "s3" {
    bucket         = "<s3-bucket-name>"
    key            = "<key-for-remote-state>"
    region         = "<aws-region>"
    dynamodb_table = "<dynamodb-table>"
  }
}

You can find more information about remote state configuration in the official Terraform documentation here.

You can also create a dummy template to create the resource for remote state:

resource "aws_s3_bucket" "terraform_state" {
  bucket = "<remote-state-bucket>"
  versioning {
    enabled = true
  }
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "<dynamo-db-table-name>"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

Create AWS Resources

You should create your own template for the project and reference the cms-demo template. If you use the cms-demo template directly, you won't be able to store your changes in the Github repository. You can find a sample template here. Basic idea is to reference the cms-demo template and pass in values for your own deployment. Keep in mind that S3 bucket names need to be globally unique. Also, you might have to manually approve SSL certificates created by Terraform depending on which account you use.

cms-demo variables

  • aws_region - AWS region where the application will be deployed.
  • common_tags - Tags that are added to created resources.
  • app_name - Deployment name. It is used to name some resources e.g. ECS cluster name.
  • env_type - Environment type e.g. dev, test, etc. It used to name some resources.
  • db_credentials - Credentials to access CMS database.
  • network - VPC network configuration.
  • whitelisted_ips - Array of IPs that have access to the VPC network.
  • subdomain_dns_name - Domain name where the application will be deployed. The UI will be available at ui.<subdomain>.
  • Remaining variables contain configurations for each microservice. Backend microservices use service-image-url and migrate-db-image-url that point to ECR repositories with service image and DB migration image. Some backend services also use environment variable configurations. Frontend microservices use s3-bucket that specifies which S3 bucket they are deployed to.

Sample Template

locals {
    app_name = "cms-release"
    env_type = "develop"
}

module "cms-demo" {
    source = "../cms-demo/aws"

    aws_region = "us-east-1"
    common_tags = {
        "Owner" : "Mo Xue Test"
        "Environment" : "develop"
        "Workstream" : "cms"
        "Customer" : "N/A"
        "Terraform" : "true"
    }
    app_name = local.app_name
    namespace = "cms-shared"
    env_type = local.env_type
    db_credentials = {
        username = "cmsuser"
        password = "cmspassword"
        name = "cms"
        port = 5432
    }
    network = {
        cidr = "10.17.0.0/16"
        private_subnets = [
            "10.17.0.0/24",
            "10.17.1.0/24",
            "10.17.2.0/24"
        ]
        public_subnets = [
            "10.17.3.0/24",
            "10.17.4.0/24"
        ]
        availability_zones = [
            "ca-central-1a",
            "ca-central-1b",
            "ca-central-1d"
        ]
    }
    whitelisted_ips = ["44.230.201.140/32"]
    subdomain_dns_name = "cms-demo.brighteriondev.com"

    user-admin-service = {
        service-image-url = "490470643515.dkr.ecr.us-west-2.amazonaws.com/user-admin-service"
        migrate-db-image-url = "490470643515.dkr.ecr.us-west-2.amazonaws.com/user-admin-service-db"
        database-name = "user-admin"
    }

    case-api-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-case-api"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-case-api-db"
        database-name = "case-api"
    }

    link-cases-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-link-cases-service"
    }

    cms-dashboard-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-dashboard-service"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-dashboard-service-db"
        database-name = "dashboards"
    }

    cms-audit-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-audit-service"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-audit-service-db"
        database-name = "audit"
    }

    cms-dispatcher-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-dispatcher"
    }

    cms-config-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-config-service"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-config-service-db"
        database-name = "config"
    }

    create-case-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-create-case-service"
    }

    event-route-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-event-routing"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-event-routing-db"
        database-name = "eventroute"
    }

    cms-route-validation-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-route-validation"
    }

    aml-trace-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/aml-trace-service"
    }

    aml-submit-trace-request = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/aml-submit-trace-request"
    }

    cms-attach-network-image = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-attach-network-image"
    }

    vocalink-proxy = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-vocalink-proxy"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-vocalink-proxy-db"
        database_name = "vocalink-proxy"
        route_prefix = "vocalink"
    }

    jobscheduler-service = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-jobscheduler-service"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-jobscheduler-service-db"
        db_name = "jobs"
    }

    mock-vlapi = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-mock-vlapi"
    }

    aml-update-cases = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/aml-update-cases"
        migrate-db-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/aml-update-cases-db"
        rabbitmq_exchange_name = "aml-update-cases"
        rabbitmq_queue_name = "aml-update-cases"
        db_name = "update_cases"
    }

    cms-metrics = {
        service-image-url = "226477880271.dkr.ecr.us-west-2.amazonaws.com/cms-metrics-server"
    }

    cms-navbar = {
       s3-bucket = "navbar-${local.app_name}-${local.env_type}" 
    }

    cms-dashboard = {
        s3-bucket = "dashboard-${local.app_name}-${local.env_type}"
    }

    aml-fraud-ui = {
        s3-bucket = "aml-ui-${local.app_name}-${local.env_type}"
    }

    cms-ui = {
        s3-bucket = "cms-ui-${local.app_name}-${local.env_type}"
    }

    single-spa = {
        s3-bucket = "single-spa-${local.app_name}-${local.env_type}"
    }

    cms-api-docs = {
        s3-bucket = "api-docs-${local.app_name}-${local.env_type}"
    }
}

Steps to deploy AWS resources

  • Create a folder that will contain your template e.g. cms-release
  • Add backend.tf to configure remote state - see the section Configure Remote State for Terraform
  • Add main.tf that contains configure resouces. It should just reference cms-demo - see terraform/cms-template-example.tf.
  • cd into the template folder
  • Initialize Terraform modules - terraform init
  • Create a Terraform plan - terraform plan -out terraform.plan
  • Make sure the proposed changes look good e.g. you shouldn't be destroying any existing resources since this is a new template.
  • Execute the plan - terraform apply terraform.plan

Configure Kong

Kong is used to route requests to corresponding microservices. The information on how to configure Kong is available here.

Configure Keycloak

The information on how to configure Keycloak is available here.

← IDP Integration Configuration