Day 11: Working with Lists, Maps, and Sets

Welcome to Day 11! Today we’ll master Terraform’s collection types and manipulation techniques. You’ll learn powerful expressions that make your infrastructure configurations dynamic and flexible.

🎯 Today’s Goals

• Master collection manipulation (lists, maps, sets)

• Learn for expressions and splat expressions

• Use collection functions effectively

• Transform and filter data

• Build dynamic, data-driven configurations

📋 Collection Types Review

# List - ordered, indexed
variable "azs" {
  type = list(string)
  default = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

# Map - key-value pairs
variable "instance_types" {
  type = map(string)
  default = {
    dev  = "t2.micro"
    prod = "t3.large"
  }
}

# Set - unique, unordered
variable "security_groups" {
  type = set(string)
  default = ["sg-123", "sg-456"]
}

🔄 For Expressions

For expressions transform and filter collections.

Syntax

# List comprehension
[for item in list : transformation]

# Map comprehension
{for key, value in map : new_key => new_value}

# Conditional filtering
[for item in list : item if condition]

List to List

variable "names" {
  default = ["alice", "bob", "charlie"]
}

locals {
  # Convert to uppercase
  upper_names = [for name in var.names : upper(name)]
  # Result: ["ALICE", "BOB", "CHARLIE"]

  # Add prefix
  prefixed_names = [for name in var.names : "user-${name}"]
  # Result: ["user-alice", "user-bob", "user-charlie"]

  # Filter and transform
  long_names = [for name in var.names : upper(name) if length(name) > 3]
  # Result: ["ALICE", "CHARLIE"]
}

List to Map

variable "users" {
  default = ["alice", "bob", "charlie"]
}

locals {
  # Create map from list
  user_map = {
    for user in var.users :
    user => {
      email = "${user}@example.com"
      role  = "developer"
    }
  }
  # Result: {
  #   alice = { email = "alice@example.com", role = "developer" }
  #   bob = { email = "bob@example.com", role = "developer" }
  #   ...
  # }
}

Map to List

variable "servers" {
  default = {
    web = { ip = "10.0.1.10", port = 80 }
    api = { ip = "10.0.2.10", port = 8080 }
  }
}

locals {
  # Extract IPs
  server_ips = [for name, config in var.servers : config.ip]
  # Result: ["10.0.1.10", "10.0.2.10"]

  # Create connection strings
  connections = [
    for name, config in var.servers :
    "${config.ip}:${config.port}"
  ]
  # Result: ["10.0.1.10:80", "10.0.2.10:8080"]
}

This part:

• Uses a for-expression to loop through var.servers.

• For each server:

  • name is the key (web, api).

  • config is the value ({ ip = "10.0.1.10", port = 80 }).

• It extracts only the ip field from each server config.

Complex Transformations

🧱 Step 1: Define the variable

variable "subnets" {
  default = [
    { cidr = "10.0.1.0/24", az = "us-east-1a", type = "public" },
    { cidr = "10.0.2.0/24", az = "us-east-1b", type = "public" },
    { cidr = "10.0.10.0/24", az = "us-east-1a", type = "private" }
  ]
}

🧩 What this means:

• subnets is a list of maps (a list of objects).

• Each subnet has three attributes:

  • cidr → CIDR block of the subnet.

  • az → Availability Zone.

  • type → Whether it’s public or private.

📊 Visual structure:

---

🧮 Step 2: Locals (computed values)

Now you define local variables to transform or filter that data.

---

🟩 (1) Filter public subnets

public_subnets = [
  for subnet in var.subnets :
  subnet if subnet.type == "public"
]

🔍 Explanation:

• Terraform loops through every element in var.subnets.

• Each element is temporarily named subnet.

• The if condition filters only subnets where subnet.type == "public".

✅ Result:

[
  { cidr = "10.0.1.0/24", az = "us-east-1a", type = "public" },
  { cidr = "10.0.2.0/24", az = "us-east-1b", type = "public" }
]

So you now have a list that only contains the public subnets.

---

🟦 (2) Transform to a specific structure

subnet_configs = [
  for idx, subnet in var.subnets : {
    name = "subnet-${idx}"
    cidr = subnet.cidr
    az   = subnet.az
    tags = {
      Type  = subnet.type
      Index = idx
    }
  }
]

🔍 Explanation:

This is another for-expression, but this time:

• It uses both the index (idx) and the item (subnet).

• For every subnet, Terraform builds a new object with a custom structure.

Let’s break it down:

---

✅ Resulting local.subnet_configs:

[
  {
    name = "subnet-0"
    cidr = "10.0.1.0/24"
    az   = "us-east-1a"
    tags = {
      Type  = "public"
      Index = 0
    }
  },
  {
    name = "subnet-1"
    cidr = "10.0.2.0/24"
    az   = "us-east-1b"
    tags = {
      Type  = "public"
      Index = 1
    }
  },
  {
    name = "subnet-2"
    cidr = "10.0.10.0/24"
    az   = "us-east-1a"
    tags = {
      Type  = "private"
      Index = 2
    }
  }
]

🛠️ Essential Collection Functions

Length and Size

length(list)      # Number of elements
length(map)       # Number of keys
length(string)    # Number of characters

#Example
locals {
  az_count      = length(var.availability_zones)  # 3
  subnet_count  = length(var.subnets)             # 5
  name_length   = length("terraform")             # 9
}

Element and Index

element(list, index)  # Get element (wraps around)
index(list, value)    # Find index of value

locals {
  first_az  = element(var.availability_zones, 0)
  # Safe access - wraps around
  safe_az   = element(var.availability_zones, 10)  
  # Wraps to index 1 if list has 3 items
  # Find position
  az_index  = index(var.availability_zones, "us-east-1b")  # 1
}

Let’s go line by line.

---

1️⃣ first_az = element(var.availability_zones, 0)

• Picks the first element (index 0).

• Result:

    "us-east-1a"
  

---

2️⃣ safe_az = element(var.availability_zones, 10)

• The list has 3 items, but index 10 is out of range.

• Terraform wraps around using modulo math:

    10 % 3 = 1
    # Result 
    "us-east-1b"
  

• So it returns the element at index 1.

• This is why it’s called “safe access” — it won’t crash even if you go out of range.

---

3️⃣ az_index = index(var.availability_zones, "us-east-1b")

• Finds the position of "us-east-1b" in the list.

• "us-east-1b" is at position 1 (0-based).

✅ Result:

1

---

🧠 Summary Table

---

Contains and Keys/Values

contains(list, value)  # Check if list contains value
keys(map)              # Get all keys
values(map)            # Get all values

locals {
  has_prod = contains(["dev", "staging", "prod"], var.environment)

  # Map operations
  env_names = keys(var.instance_types)    # ["dev", "prod"]
  types     = values(var.instance_types)  # ["t2.micro", "t3.large"]
}

Distinct and Sort

distinct(list)  # Remove duplicates
sort(list)      # Sort alphabetically

locals {
  # Remove duplicates
  unique_azs = distinct(["us-east-1a", "us-east-1b", "us-east-1a"])
  # Result: ["us-east-1a", "us-east-1b"]

  # Sort
  sorted_names = sort(["charlie", "alice", "bob"])
  # Result: ["alice", "bob", "charlie"]

  # Reverse sort
  reverse_sorted = reverse(sort(var.names))
}

Lookup

Safely get a value from a map — with a default fallback if the key doesn’t exist.
Unlike direct access (var.map["key"]), lookup() won’t fail when the key is missing.

🔹 Syntax:

lookup(map, key, default)

---

💡 Example 1 — Basic lookup

variable "instance_types" {
  default = {
    dev  = "t2.micro"
    prod = "t3.large"
  }
}

variable "environment" {
  default = "staging"
}

locals {
  instance_type = lookup(
    var.instance_types,
    var.environment,
    "t2.micro"  # default
  )
}

✅ Since "staging" isn’t a key in var.instance_types,
local.instance_type = "t2.micro"

---

💡 Example 2 — Safe nested lookup

variable "database_config" {
  default = {
    dev = { port = 3306 }
    prod = { port = 5432 }
  }
}

variable "environment" {
  default = "qa"
}

locals {
  db_port = lookup(
    lookup(var.database_config, var.environment, {}),
    "port",
    5432
  )
}

Explanation:

• The first lookup() tries to get the qa configuration from var.database_config.
  If not found, returns {} (an empty map).

• The second lookup() then tries to get "port" from that map.
  If it’s missing, returns 5432 as the default.

✅ Result:
db_port = 5432

🧪 Hands-On Lab: Collection Mastery

Let’s build a complex infrastructure using advanced collection techniques!

Step 1: Create Project

mkdir terraform-collections-lab
cd terraform-collections-lab

Step 2: Create variables.tf

# variables.tf

variable "aws_region" {
  type    = string
  default = "us-east-1"
}

variable "project_name" {
  type    = string
  default = "collections-demo"
}

variable "availability_zones" {
  type    = list(string)
  default = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

variable "subnet_configurations" {
  description = "List of subnet configurations"
  type = list(object({
    name = string
    cidr = string
    type = string
    tier = string
  }))

  default = [
    { name = "web-1", cidr = "10.0.1.0/24", type = "public", tier = "web" },
    { name = "web-2", cidr = "10.0.2.0/24", type = "public", tier = "web" },
    { name = "app-1", cidr = "10.0.11.0/24", type = "private", tier = "app" },
    { name = "app-2", cidr = "10.0.12.0/24", type = "private", tier = "app" },
    { name = "db-1", cidr = "10.0.21.0/24", type = "private", tier = "db" },
    { name = "db-2", cidr = "10.0.22.0/24", type = "private", tier = "db" }
  ]
}

variable "instance_configs" {
  description = "Instance configurations per tier"
  type = map(object({
    instance_type = string
    count         = number
    user_data     = string
  }))

  default = {
    web = {
      instance_type = "t2.micro"
      count         = 2
      user_data     = "#!/bin/bash\\necho 'Web Server' > /var/www/html/index.html"
    }
    app = {
      instance_type = "t2.small"
      count         = 2
      user_data     = "#!/bin/bash\\necho 'App Server'"
    }
  }
}

variable "common_tags" {
  type = map(string)
  default = {
    Project   = "Collections Demo"
    ManagedBy = "Terraform"
  }
}

Step 3: Create locals.tf

# locals.tf

locals {
  # Filter subnets by type
  public_subnets = [
    for subnet in var.subnet_configurations :
    subnet if subnet.type == "public"
  ]

  private_subnets = [
    for subnet in var.subnet_configurations :
    subnet if subnet.type == "private"
  ]

  # Group subnets by tier
  subnets_by_tier = {
    for tier in distinct([for s in var.subnet_configurations : s.tier]) :
    tier => [
      for subnet in var.subnet_configurations :
      subnet if subnet.tier == tier
    ]
  }

  # Create subnet map for easy lookup
  subnet_map = {
    for subnet in var.subnet_configurations :
    subnet.name => subnet
  }

  # Get all tiers
  all_tiers = distinct([for s in var.subnet_configurations : s.tier])

  # Get all CIDR blocks
  all_cidrs = [for s in var.subnet_configurations : s.cidr]

  # Distribute subnets across AZs
  subnet_az_mapping = {
    for idx, subnet in var.subnet_configurations :
    subnet.name => var.availability_zones[idx % length(var.availability_zones)]
  }

  # Create security group rules matrix
  tier_connectivity = {
    web = ["app"]
    app = ["db"]
    db  = []
  }

  # Flatten instance deployments
  instance_deployments = flatten([
    for tier, config in var.instance_configs : [
      for i in range(config.count) : {
        tier          = tier
        index         = i
        instance_type = config.instance_type
        user_data     = config.user_data
        subnet        = local.subnets_by_tier[tier][i % length(local.subnets_by_tier[tier])].name
      }
    ]
  ])

  # Create deployment map
  deployment_map = {
    for deployment in local.instance_deployments :
    "${deployment.tier}-${deployment.index}" => deployment
  }

  # Calculate network statistics
  network_stats = {
    total_subnets    = length(var.subnet_configurations)
    public_subnets   = length(local.public_subnets)
    private_subnets  = length(local.private_subnets)
    tiers            = length(local.all_tiers)
    total_instances  = sum([for c in var.instance_configs : c.count])
    azs_used         = length(var.availability_zones)
  }

  # Merge tags for each tier
  tier_tags = {
    for tier in local.all_tiers :
    tier => merge(
      var.common_tags,
      { Tier = tier }
    )
  }
}

Step 4: Create main.tf

# main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

# VPC
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true

  tags = merge(var.common_tags, {
    Name = "${var.project_name}-vpc"
  })
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = merge(var.common_tags, {
    Name = "${var.project_name}-igw"
  })
}

# Subnets using for_each
resource "aws_subnet" "subnets" {
  for_each = local.subnet_map

  vpc_id                  = aws_vpc.main.id
  cidr_block              = each.value.cidr
  availability_zone       = local.subnet_az_mapping[each.key]
  map_public_ip_on_launch = each.value.type == "public"

  tags = merge(local.tier_tags[each.value.tier], {
    Name = "${var.project_name}-${each.key}"
    Type = each.value.type
  })
}

# Route Table for Public Subnets
resource "aws_route_table" "public" {
  count = length(local.public_subnets) > 0 ? 1 : 0

  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = merge(var.common_tags, {
    Name = "${var.project_name}-public-rt"
  })
}

# Route Table Associations
resource "aws_route_table_association" "public" {
  for_each = {
    for subnet in local.public_subnets :
    subnet.name => aws_subnet.subnets[subnet.name].id
  }

  subnet_id      = each.value
  route_table_id = aws_route_table.public[0].id
}

# Security Groups per tier
resource "aws_security_group" "tier_sgs" {
  for_each = toset(local.all_tiers)

  name        = "${var.project_name}-${each.key}-sg"
  description = "Security group for ${each.key} tier"
  vpc_id      = aws_vpc.main.id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = merge(local.tier_tags[each.key], {
    Name = "${var.project_name}-${each.key}-sg"
  })
}

# AMI Data Source
data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

# EC2 Instances
resource "aws_instance" "instances" {
  for_each = local.deployment_map

  ami                    = data.aws_ami.amazon_linux.id
  instance_type          = each.value.instance_type
  subnet_id              = aws_subnet.subnets[each.value.subnet].id
  vpc_security_group_ids = [aws_security_group.tier_sgs[each.value.tier].id]
  user_data              = each.value.user_data

  tags = merge(local.tier_tags[each.value.tier], {
    Name  = "${var.project_name}-${each.key}"
    Index = each.value.index
  })
}

Step 5: Create outputs.tf

# outputs.tf

output "network_statistics" {
  description = "Network statistics"
  value       = local.network_stats
}

output "subnets_by_tier" {
  description = "Subnets grouped by tier"
  value = {
    for tier, subnets in local.subnets_by_tier :
    tier => [for s in subnets : s.name]
  }
}

output "subnet_details" {
  description = "Detailed subnet information"
  value = {
    for name, subnet in aws_subnet.subnets :
    name => {
      id   = subnet.id
      cidr = subnet.cidr_block
      az   = subnet.availability_zone
    }
  }
}

output "instance_distribution" {
  description = "Instances per tier"
  value = {
    for tier in local.all_tiers :
    tier => length([
      for key, inst in local.deployment_map :
      inst if inst.tier == tier
    ])
  }
}

output "instance_ips_by_tier" {
  description = "Instance IPs grouped by tier"
  value = {
    for tier in local.all_tiers :
    tier => [
      for key, instance in aws_instance.instances :
      instance.private_ip if local.deployment_map[key].tier == tier
    ]
  }
}

output "all_instance_ips" {
  description = "All instance IPs"
  value       = values(aws_instance.instances)[*].private_ip
}

output "tier_security_groups" {
  description = "Security group IDs per tier"
  value = {
    for tier, sg in aws_security_group.tier_sgs :
    tier => sg.id
  }
}

Step 6: Deploy and Explore

# Initialize
terraform init
# Plan and review the collections magic!
terraform plan
# Apply
terraform apply -auto-approve
# View outputs
terraform output network_statistics
terraform output subnets_by_tier
terraform output instance_distribution
terraform output instance_ips_by_tier
# Clean up
terraform destroy -auto-approve

📝 Summary

Today you learned:

• ✅ For expressions (list, map transformations)

• ✅ Splat expressions for attribute extraction

• ✅ Essential collection functions

• ✅ Advanced filtering and grouping

• ✅ Complex data transformations

• ✅ Dynamic infrastructure patterns

🚀 Tomorrow’s Preview

Day 12: Count and For_Each - Creating Multiple Resources

Tomorrow we’ll:

• Master count and for_each

• Understand when to use each

• Create dynamic resources

• Manage resource indexes

• Build scalable configurations

---

← Day 10: Outputs & Locals | Day 12: Count & For_Each →

---

Remember: Collections and transformations are the key to dynamic infrastructure!