What is Infrastructure as Code?

“Sourav, we need a new environment to perform integration testing of our application” -supervisor to me.
“Okay…” -me.
“Do you have that checklist that you created the last time we requested a new environment?”
“Yes, I was just thinking about that… But that was more than an year back and since then we’ve added a few more applications to our product.”
Supervisor thinking…
Me thinking as well, as I added “We should still be able to update the checklist with the changes made in the last one year and use the same”
“Ok good. Lets get started on that and let us know if you need something from the development team or the operations team”

My checklist was something like:
-request machines from data center operations teams with specific configurations (disk space, cpu, processor etc)
-request db connectionstrings for new environment from Database team
-request third party setup configuration values from data center operations teams
On the application machines:
-create environment (system) variables
-install windows features
-install third party software
-install windows certificate
-install and configure IIS sites and websites
-copy static apps and update config values
-create a new deployment group in Azure Devops and add these machines to the group

This checklist had all the components that had to be applied before application code was deployed to the agents. This particular environment had 4 machines so the same set of steps had to be applied on all the application machines.

I remember, working on setting up subsequent new environments wasn’t as exciting as was the first one. Moreover, there are a few limitations in my (manual) approach:
Not scalable and inefficient: time taken to work on 4 machines is 4 times to configure one machine
Error prone: since this process is manual, there is high reliability on the person setting up the environment
Non repeatable: someone else working on the checklist might attempt working at it differently -perform step 3 and 5 together etc or maybe miss step 2 altogether or maybe install a different version of a software (java or .net) and configure is differently

Moreover, once an environment had been commissioned and in use, any subsequent change (update) was done manually. Sometimes, in the development environment, more than necessary updates are made… until desired state is achieved. However, when a higher environment needs to be updated, the same set of changes are not always made.

Let me describe that through an example. Say to test a new feature in the product the developers realize that they need third party Installer A and B in development environment. After installing the same, the product does not appear to be behaving the way it should. They then realize that they need Installer A and C (and not B). With this information at hand, they install C and the product acts as expected. When the time to update the next higher environment comes (Environment: test), only Installer A and C are installed there (installer B is not; and rightfully so since that was not needed in the first place).
Development environment consist of Installer A, B and C.
Test environment consist of Installer A and C.

In this case development and test environments are no longer identical -environment drift. An environment drift occurs when no two environments (or servers) are 100% identical and thereby are no longer reliable.

A primary driver in my professional journey has been -how do I continuously delivery value to my employer and our customers and make my job intellectually stimulating such that I continue to add new skills and competencies which feed into my ability to deliver value.

I knew that an automated approach would address those factors such that all the steps mentioned could be converted into modules/scripts and an orchestrator could be used to execute these on the machines to bring them to desired state. I had a little experience working with Chef and I borrowed heavily on the ideas on which the tool is based. And that is when I learnt about infrastructure as code. I was fascinated by the fact that we could have a few (maybe a little more than that) lines of code (version controlled) and use them to provision environments on the fly.

Infrastructure as code is an approach to infrastructure automation based on practices from software development. It emphasizes consistent, repeatable routines for provisioning and changing systems and their configuration. Changes are made to definitions and then rolled out to systems through unattended processes that include thorough validation. -Kief Morris in Infrastructure as code (@kief)

Characteristics of a IaC module:
The idea to automate the process of provisioning an environment not only addresses challenges around productivity but also around quality. The author of an IaC module must be sensitive to the following:

• Create IaC modules as tiny, independent and decoupled as possible.

Although the first few iterations of IaC adoption in an organization might start from taking a pristine machine and then executing a series of automated steps to bring it to desired steps, that might not be the case always. As an organization’s IaC process matures, there can be instances when only a certain component of server provisioning might require an update. In that case, going through 1000s of lines of code and extracting a snippet to perform a specific task might distract the implementor. Creating components that are isolated and standalone ensures that the process of IaC is extendable and reliable.

• Automate provisioning, automate verification.

Each IaC module should encompass a verification step which effectively means that the system not only provisions but also checks the end result post provisioning. This kind of verification increases reliability in the process and ensures that there are no false positives (automation process completed without any error, but desired state of environment not achieved).

Benefits of IaC:
A few primary benefits that come to my mind immediately are as follows:
Speed: With an automation script, the process to commission an environment is going to be faster than a manual process.
Scale: Whether I need 4 machines or 400 machines, the setup time does not need to be very different than the time needed for one machine, unless otherwise desired.
Consistent: The same setup technique (scripts and configuration values) would ensure that the machines/environments that are being configured are identical.
Repeatable: The setup process is identical for all the machines and is being followed for each new provision.
Versionable: By storing IaC assets in a source control repository, it is possible to track how a module evolved over time with additional information about the author and related work items.

Challenges to IaC approach:
Occasionally there are instances when a product environment requires some update. This usually starts by someone making that change (manually) on one of the lower environments to align with the changing needs of the product being developed. Once that change has been tested, there is a tendency to proceed with the manual approach for all other environments.
This arises due to two reasons: (i) making manual change is quicker than writing code to automate the process and (ii) making manual change ensures that nothing else has changed; less faith in the entire automation technique -what if something else breaks?
However, in the long run, such a mindset is detrimental to the culture of an organization that is moving their infrastructure to code. The only way to have 100% faith in your IaC modules is to use it as many times as possible, improving with each iteration. And hence, from time to time, higher management must be firm in their resolve that infrastructure should also be treated as a product and follow an identical lifecycle.
The organization should understand and agree that adopting IaC is not a one-time process but a continuous one.
For an organization moving towards IaC, the first iteration is usually the longest one. Staying committed to the process is critical to early adoption.
Breakeven: How many machines and environments do we need to ensure that the spend towards IaC is cost effective? I think, if an organization is talking about IaC, they probably need IaC.
Conclusion:
Although servers and environments hosting the product are not what the customer sees directly, it greatly aids in customer experience. Hence any organization that is obsessed about customer experience, should take their infrastructure very seriously and adopting a IaC mindset makes that scalable, consistent and repeatable.

Further reading:  https://www.oreilly.com/library/view/infrastructure-as-code/9781491924334/ch01.html