Capturing multiple outputs from Module -Terraform

So Terraform helps you create infrastructure, the way it works is we define resources that “need” to be created and TF helps you fulfill our requirements. What it does is, internally it maintains a state file and tries to reach the end state defined by us (as in our requirements for the infra) by taking a series of steps that it “figures” out itself and reaches that end/desired state.

How do we create resources in Tf, let's say we want to create an AWS IAM POLICY and attach some policy doc to it. so the TF code would look something like

resource “aws_iam_policy” “my_custom_policy” {
name = “custom_policy”
description = “custom_policy , tf code for medium”
policy = data.aws_iam_policy_document.custom_policy.json
}

#the policy json for above policy
data “aws_iam_policy_document” “custom_policy_doc” {
statement {
sid = “Alpha1”
effect = “Allow”
actions = [
“s3:ListBucket”,
“s3:ListAllMyBuckets”,
]
resources = [“arn:aws:s3:::bucket-85”]
}

What this does is creates an AWS I am policy named ‘my_custom_policy’ which has access to resources defined under the policy document “custom_policy_doc”. If you run tf-apply you can see on the AWS IAM console that indeed a policy is created.

Now if We want to create multiple policies based on some input, which can be a list of names like

buckets = [“bucket-1”, “bucket-2”, “bucket-99”], we will have to modify the above code based on variables

//main.tf
resource “aws_iam_policy” “my_custom_policy” {
name = “custom_policy_for_${var.bucket_name}”
description = “custom_policy for bucket — ${var.bucket_name}, tf code for medium”
policy = data.aws_iam_policy_document.custom_policy.json
}

#the policy json for above policy
data “aws_iam_policy_document” “custom_policy_doc” {
statement {
sid = “Alpha1”
effect = “Allow”
actions = [
“s3:ListBucket”,
“s3:ListAllMyBuckets”,
]
resources = [“arn:aws:s3:::${var.bucket_name}*”]
}

// variables.tf
variable “bucket_name” {
type = string
description = “bucket name”
}

What we have essentially written is a template for creating a policy based on a policy document which depends on an input variable. What this gives us is greater control such as

1. Can create as many policies depending on the input arguments, dynamic provisioning.we don't need to know the count.
2. Same code can be re used for N number of policies.Small and efficient code. No redudancy.
3. We have greater control over what an IAM role with this policy can access since we have passed the variable name as input to resources accessed.(resources = [“arn:aws:s3:::${var.bucket_name}*”])

Let’s create these policies now in a main.tf file outside the above module, keeping a proper directory structure helps as the root module accesses the child modules templates.

module  "my_custom_policies_based_on_input" {
  for_each            = var.buckets
  source              = "./modules/templated_policies"
  bucket_name         = each.value
}

Now if I run “terraform plan and terraform apply” , it will call the template code we wrote for each variable we pass as “bucket_name” and that variable will be replaced everywhere it is used/accessed. The difference is that earlier our Terraform resource was one object and now it is a collection of objects and will be represented in terraform state file as

“my_custom_policies_based_on_input[‘bucket1']”
“my_custom_policies_based_on_input[‘bucket2’]”
“my_custom_policies_based_on_input[‘bucket85’]”
“my_custom_policies_based_on_input[‘bucket99’]”
and so on…

what we have done so far is using the same piece of code , have created multiple AWS policies by templatising the code as a python function using input variables. We can add more to the above code and create IAM Roles and and IAM groups and attach these policies to the groups etc.
In earlier case where we were just creating single policy accessing the output is easy. We create an output.tf file where we name the output variables and then can use them in root modules or else as per demands of our code.

output "my_custom_policy_name" {
  value       = aws_iam_policy.my_custom_policy.name
  description = "the single policy we created"
}
output "my_custom_policy_arn" {
  value       = aws_iam_policy.my_custom_policy.arn
  description = "the single policy we created"
}

In a single policy scenario, its very easy to use. The relationships are clean.But it gets tricky where we have multiple policies under the same resource which is now a collection of policies, the names of which can be user driven or coming from a tvfar file at the root module.And how do we know which policy arn maps to which name since there are multiple objects now. What we can do essentially here is use zipmap

output "my_custom_policies_details" {
  value       = zipmap( values(module.my_custom_policies_based_on_input)[*].my_custom_policy_name,
  values(module.my_custom_policies_based_on_input)[*])
}

What we have done is created a kind of hasmap the key of which is policy name but how does terraform figures it out, It uses the “my_custom_policy_name” which is an output variable of the templatised code in the outputs.tf file . So lets say we need another info from the module , we simply add it to the output file and we can access that later on (while creating output “my_custom_policies_detail”)since we are putting the entire policy as value to the map based on the key which is policy name.

Now we can simply refer this output created as a map further as

resource ".." "..."{
  for_each = module.medium_module.my_custom_policies_details
   data = {
    POLICY_NAME               = each.key
    POLICY_ARN                = each.value["my_custom_policy_arn"]  }
}

and u can have ur directory structure as

The above is a sample directory structure (not written in stone), the idea is to just modularize your code so as to take advantage of reusing the code and the output from inner modules by root modules.

Hope it helps.