Managing Credentials and Secrets in Terraform

Imagine you're working on a project where you need to deploy resources to AWS using Terraform. In a rush to finish things, you hard-code your AWS credentials directly into your Terraform files. Everything works fine at first, and your resources are successfully deployed. But a few weeks later, you discover that your Terraform repository was accidentally made public. Suddenly, your AWS credentials are exposed to the entire internet. Exposing your credentials can lead to unauthorized access to your AWS account, leading to serious security problems.

This scenario highlights why it's essential to manage secrets and credentials securely. Hard-coding sensitive information in your codebase is risky. To avoid such issues, it's crucial to explore secure methods for managing Terraform's secrets and credentials.

In this blog, we'll explore several methods for securely managing secrets and credentials, including environment variables, GitHub Secrets, encrypted files with AWS KMS, and AWS Secrets Manager. We’ll also compare these methods to help you choose the best approach for your needs.

Method 1: Environment Variables

Using environment variables to manage secrets in Terraform is straightforward and commonly used. This approach keeps sensitive data like usernames and passwords out of your codebase and allows for easy integration with your CI/CD pipelines.

Imagine you need to create an AWS RDS instance, and you want to keep the database username and password secure.

1. Define Sensitive Variables in Terraform:

First, define your sensitive variables in your Terraform configuration file. Marking them as sensitive ensures that Terraform treats them securely.

variable "db_password" {
  type      = string
  sensitive = true
}

variable "db_username" {
  type      = string
  sensitive = true
}

2. Use Variables in Resource Definitions:

Use these variables in your resource definitions. Here’s an example of an AWS RDS instance where the database username and password are sourced from the defined variables.

resource "aws_db_instance" "example" {
  identifier           = "mydbinstance"
  engine               = "mysql"
  instance_class       = "db.t3.micro"
  allocated_storage    = 10
  publicly_accessible  = true
  skip_final_snapshot  = true
  username             = var.db_username
  password             = var.db_password
}

3. Set Environment Variables:

Before applying your Terraform configuration, set the environment variables for the database username and password. This can be done in your shell or CI/CD pipeline configuration.

export TF_VAR_db_username="my_db_username"
export TF_VAR_db_password="my_db_password"

4. Apply Terraform Configuration:

Finally, run your Terraform commands as usual. Terraform will read the environment variables and use them to configure your resources.

terraform init
terraform plan
terraform apply

Method 2: Encrypted Files (KMS)

Using encrypted files to manage secrets in Terraform is a robust approach that enhances security by leveraging AWS Key Management Service (KMS). This method ensures that sensitive information is stored in an encrypted format and decrypted only when needed by Terraform.

Imagine you need to create an AWS RDS instance, and you want to keep the database username and password secure by storing them in an encrypted file.

1. Create a YAML File with Credentials:

First, create a YAML file (db-creds.yml) that contains your database credentials:

db_username: my_db_username
db_password: my_db_password

2. Encrypt the YAML File Using AWS KMS:

Use the AWS CLI to manually encrypt your YAML file:

aws kms encrypt --key-id <your-kms-key-id> --region <your-region> --plaintext fileb://db-creds.yml --output text --query CiphertextBlob > db-creds.yml.encrypted

3. Define KMS Data Source in Terraform:

Use the aws_secretsmanager_secret_version data source in your Terraform configuration to retrieve and decrypt the encrypted file:

data "aws_secretsmanager_secret_version" "creds" {
  secret_id  = aws_secretsmanager_secret.db_creds.id
  depends_on = [aws_secretsmanager_secret.db_creds, aws_secretsmanager_secret_version.db_creds_version]
}

locals {
  db_cred = jsondecode(data.aws_secretsmanager_secret_version.creds.secret_string)
}

4. Apply Terraform Configuration:

Finally, run your Terraform commands as usual. Terraform will decrypt the file using AWS KMS and use the credentials to configure your resources.

Method 3: AWS Secrets Manager

AWS Secrets Manager provides a secure way to store and manage sensitive information such as database credentials, API keys, and other secrets. This method allows you to retrieve secrets dynamically within your Terraform configuration, ensuring that sensitive data is never hard-coded in your Terraform files.

Here’s how you can manage your database credentials using AWS Secrets Manager.

1. Store Your Secrets in AWS Secrets Manager using Terraform:

First, use Terraform to create a secret in AWS Secrets Manager.

variable "db_username" {
  type      = string
  sensitive = true
}

variable "db_password" {
  type      = string
  sensitive = true
}

resource "aws_secretsmanager_secret" "mysql_cred" {
  name = "mysql-cred"
}

resource "aws_secretsmanager_secret_version" "mysql_cred_version" {
  secret_id     = aws_secretsmanager_secret.mysql_cred.id
  secret_string = jsonencode({
    db_username = var.db_username
    db_password = var.db_password
  })
}

2. Retrieve Secrets in Terraform:

Use the aws_secretsmanager_secret_version data source to fetch the secret from AWS Secrets Manager:

data "aws_secretsmanager_secret_version" "creds" {
  secret_id  = aws_secretsmanager_secret.mysql_cred.id
}

locals {
  db_cred = jsondecode(data.aws_secretsmanager_secret_version.creds.secret_string)
}

3. Use Retrieved Secrets in Resource Definitions:

Use the retrieved credentials in your resource definitions:

resource "aws_db_instance" "example" {
  identifier           = "mydbinstance"
  engine               = "mysql"
  instance_class       = "db.t3.micro"
  allocated_storage    = 10
  publicly_accessible  = true
  skip_final_snapshot  = true
  username             = var.db_username
  password             = var.db_password
}

Method 4: GitHub Secrets

For projects managed with GitHub, using GitHub Secrets is a convenient way to store and manage secrets securely within GitHub Actions workflows. This method is particularly useful for CI/CD pipelines where you must keep sensitive data safe while automating deployments.

Here’s how you can manage your database credentials using GitHub Secrets.

1. Add Secrets to GitHub Repository:

Navigate to your GitHub repository, go to Settings > Secrets and variables > Actions, and add your secrets.

2. Access Secrets in GitHub Actions Workflow:

In your GitHub Actions workflow file (.github/workflows/your-workflow.yml), you can access these secrets as environment variables:

name: Deploy

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Set up Terraform
      uses: hashicorp/setup-terraform@v2
      with:
        terraform_version: <terraform_version>

    - name: Terraform Init
      run: terraform init

    - name: Terraform Apply
      env:
        TF_VAR_db_username: ${{ secrets.DB_USERNAME }}
        TF_VAR_db_password: ${{ secrets.DB_PASSWORD }}
      run: terraform apply -auto-approve

Comparison of Methods

Recommendations

• Development Environments: Environment variables or encrypted files (KMS) can be sufficient and are easier to set up.

• Production Environments: AWS Secrets Manager provides robust security features and is recommended for managing secrets in production.

Understanding and applying these methods ensures that your sensitive information remains secure and your Terraform configurations are well-managed.