Terraform state management and best practices is key to becoming proficient with Terraform.
What is Terraform State File?
Terraform logs is created in a state file. This enables Terraform to have a control to update and destroy the resources. The terraform state file, by default, is named terraform.tfstate and is held in the same directory where Terraform runs. It is created after running terraform apply. The actual content of this file is a JSON formatted mapping of the resources defined in the configuration and those that exist in your infrastructure. When Terraform is run, it can then use this mapping to compare infrastructure to the code and make any adjustments as necessary.
State File Locking
If supported by your backend, Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state. State locking happens automatically on all operations that could write state.
Locking is natively supported
The state file will be automatically loaded, and encryption can be used to secure the state file on disk and in transit.
Remote backend storage such as Azure Storage or AWS S3 designed to be highly available. They also support versioning. You can roll back to a previous state file if it get corrupted.
Lastly, it is inexpensive to use. Usually, state files are very small and can normally fit into the free tier.
Force Unlock
It unlocks the state if unlocking failed. Be very careful with this command. If you unlock the state when someone else is holding the lock it could cause multiple writers. Force unlock should only be used to unlock your own lock in the situation where automatic unlocking failed.
To protect you, the force-unlock command requires a unique lock ID. Terraform will output this lock ID if unlocking fails. This lock ID acts as a nonce, ensuring that locks and unlocks target the correct lock.
Best Practices
Storing State Files
State files, by default, are stored in the local directory where Terraform is run. If you are using Terraform to test or for a personal project, this is fine (so long as your state file is secure and backed up!). However, when working on Terraform projects in a team, this becomes a problem as multiple people will need to access the state file.
Also, when using automation and CI/CD pipelines to run Terraform, the state file needs to be accessible, and permission given to the service principal running the pipeline to be able to access the storage account container that holds the state file. This makes shared storage a perfect candidate to hold the state file. Permissions can be granted as needed. Azure Storage accounts or aws s3 are an ideal choice.
You should store your state files remotely, not on your local machine!
The location of the remote state file can then be referenced using a backendblock in the terraform block (which is usually in the main.tf file).
The example below shows a configuration using a storage account in Azure:
terraform {
backend "azurerm" {
resource_group_name = "terraform-rg"
storage_account_name = "terraformsa"
container_name = "terraformstate"
key = "terraform.tfstate"
}
}
It is not a good idea to store the state file in source control.
This is because Terraform state files contain all data in plain text, which may contain secrets. Storing secrets anywhere other than a secure location is never a good idea and definitely should not be put into source control. To avoid this, for example, in Azure, Azure Key Vault can be referenced. Also, AWS S3 with permission controls works better!
Most version control systems do not allow the locking of files
It may cause issues when multiple people attempt to access the file at the same time.
Storing state files in version control can introduce human error, as the latest state file would need to be pulled each time Terraform is run.
This could result in an old state file being used and unexpected changes to the infrastructure.
Using a remote backend instead solves these challenges.
State files should be isolated to reduce the “blast radius”.
Projects are structured in a single folder and use a single state file for all resources. This immediately introduces risk, as a mistake in the configuration could change the state file and cause unwanted consequences to all your resources.
A better way is to use multiple state files for parts of your infrastructure. Logically separating resources from each other and giving them their own state file in the backend means that changes to one resource will not affect the other. Different state files for different environments are also a good idea.
Assuming we are holding all state files in a single storage account, the container folder structure for the remote backend could end up looking something like the example below:
terraformstate
--development
--webapp.tfstate
--sqldb.tfstate
--vnet.tfstate
--UAT
--webapp.tfstate
--sqldb.tfstate
--vnet.tfstate
--production
--webapp.tfstate
--sqldb.tfstate
--vnet.tfstate
The backend configuration for the Azure Web App in the development environment:
terraform {
backend "azurerm" {
resource_group_name = "terraform-rg"
storage_account_name = "terraformsa"
container_name = "terraformstate"
key = "development\webapp.tfstate"
}
}
Making a change to the Azure Web App in development will have no effect on the Azure Web App in production.
Isolation can also be achieved using Terraform Workspace. These are less useful in production environments but are good for a quick way to test isolated environments.
Using workspace to manage actual environments is discouraged
Terraform has one workspace by default (called default!). State files are isolated to each workspace. You can create a new workspace using the terraform workspace new command.
$ terraform workspace new
New development Created and switched to workspace "development"! You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration.
Useful command quick reference when working with workspaces:
$ terraform workspace list
— list your workspaces.
$ terraform workspace new [workspace name]
— create a new workspace.
$ terraform workspace select [workspace name]
— switch to the specified workspace.
Terraform Remote State
terraform_remote_state is used to fetch details from the remote state file directly. This is useful when you need to reference the outputs of configurations that are stored in different state files. When a block is defined in your configuration, the contents are included in the state file. These details can then be referenced elsewhere in your project.
terraform_remote_state Data Source Example
For example, again, consider we have an Azure SQL Database and Azure Web App. The configuration files for these should be set up to use different state files. When creating our database that the web app needs to connect to, we add an output block to expose the resulting ID of the database after it has been created:
output "sqldb_id" {
value = azurerm_sql_database.example.id
description = "Database ID"
}
We can now reference the resulting SQL database ID in the Web App configuration code by configuring a data block using terraform_remote_state:
data "terraform_remote_state" "dev_sqldb" {
backend = "azurerm"
config = {
storage_account_name = "terraformsa"
container_name = "terraformstate"
key = "development/sqldb.tfstate"
}
}
And then reference the name of the output where necessary in the configuration:
data.terraform_remote_state.dev_sqldb.outputs.sqldb_id
Manipulating the State File
It is sometimes necessary to directly interact with the state file, either to check its contents, remove items when they have been imported incorrectly or no longer exist in the real infrastructure, or import items that already exist to bring them under Terraform management.
The terraform state command can be used to perform advanced state management. All state management commands that modify the state create a timestamped backup of the state prior to making modifications.
Useful terraform state commands:
terraform state list — List the contents of the state file.
terraform state rm — Remove an item from the state file.
terraform state show — Show a resource in the state file.
To import existing items into the state file that have been created by other methods in the infrastructure to bring them under Terraform control, the terraform import command can be used. Each resource on the Terraform docs pages has an import section detailing how to use the command for a particular resource. For example, on azurerm docs page, it states that SQL Databases can be imported using the resource id and gives an example.
terraform import azurerm_sql_database.database1 /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myresourcegroup/providers/Microsoft.Sql/servers/myserver/databases/database1
The corresponding resource block azurerm_sql_database.database1 needs to be written in the configuration file prior to using this command, with the settings matching the real infrastructure. Importing resources can be very time-consuming, so avoiding any manual creation of resources from the get-go is normally a wise idea!
Conclusion
State files should be stored remotely as a general rule and isolated and organized in such a way that separate state files exist for logical groups of resources and environments in order to reduce the “blast radius” should any mistakes occur. The terraform_remote_state data source can be used to reference outputs from state files. Finally the terraform state and terraform import commands can be used to manipulate the contents of the state file.
Compiled by: Azizul maqsud
Reference: https://spacelift.io/blog/terraform-state
https://developer.hashicorp.com/terraform/language/state/locking#