Idempotency in Infrastructure as Code

(this thought process derives heavily from my experience as a scm, build and release engineer working on automation, scripting and trying to make my job exciting. Feel free to disagree)

In the earlier days (2000’s maybe?) most of the tasks around build and release involved manual steps in the follow sequence:
Step 1: development check-in code
Step 2: build solution
Step 3: create deployment package
Step 4: stop services (in case of windows -iis/windows services)
Step 5: copy package to deployment server
Step 6: start services
Step 7: start verification (testing)

Then the continuous integration wave automated Step 1 through 3 and continuous deployment automated step 4 through 6. After automating this, I think the next challenge was around automating infrastructure that supported the application.

While tacking infrastructure automation, the first round was around -do something.
But this approach had a limitation, in a way that it was not repeatable. Say for example the infra automation job was to copy a file into a shared location. The automation program would work the first time, but if we were to execute the program the second time (when the file already exists) the operation would fail. This could be true for any automation task -create an artifact, modify state of an artifact or delete an artifact.

This limitation introduced a wrapper around the automation program and that is to -check if the desired state already exists and modify only if it warrants a change. Simply put: check before you proceed. This is idempotency.

Do no change if system is already in desired state.

And, as an organization moves towards building their infrastructure as code, adhering to idempotency is a necessary condition for the code, else the turnaround cycle will be unnecessarily long.
For example imagine a IaC module that is not idempotent. This would imply that all processes/steps have to start from the very beginning; there is no support for those systems that are partially configured. Now imagine doing this when you have more than a few dozen servers to deal with. There would be a lot of wastage in the approach. Moreover, a script or program that is not idempotent is:
-not scalable: extending (or adding) functionality is challenging
-not repeatable: without bringing system back to initial state
-limiting: the purpose of automation (increased productivity) is defeated
-longer feedback loop: in a way that any subsequent change that is introduced would imply that all previously coded automation must execute before newer functionality can be implemented

On the other hand IaC module that is idempotent has all the opposite traits.
-scalable: build on existing state
-repeatable: do nothing is artifact exists in desired state
-not limiting: adheres with the objective of automation; increased productivity
-shorter feedback loop: since resources would not be spent on altering artifact state
-and also extremely reliable since it gets executed multiple times going across resources that are in various states of configuration.

Now the question that arises is -how do you build idempotent programs/scripts? And the solution is quite simple. There is only one condition block needed

-check if artifact exists in desired state:
If yes: do nothing
If no: do something

An alternate approach can be:
-check if artifact exists in desire state:
If yes: bring artifact back to initial state (delete/revert it) and then do something
If no: do something

Although, both approaches have their merits and demerits my preference is to use the former until that becomes a challenge (complex difficult to check logic) in which case using later would make sense.
Another feature (apart from idempotency) that I think is necessary in any IaC module is ease of understanding so that if someone else has to take over and maintain what you created, there is very less learning curve. It is important to note that an underlying objective of creating IaC modules is to make a system process dependent and not person dependent. At the same time, if someone has to modify the program (and if that someone is not you), you want them to be able to do that in as little time as possible. Remember, productivity is also an objective we are trying to achieve here. You do not want yourself or your IaC module to be a bottleneck. And in that regard, creating appropriate documentation (detailed comment in sripts/wiki) might be useful.
The easier your IaC modules are to understand and implement, the wider would be their acceptance and the longer will your legacy life -if that makes you feel good 😉

Oh, and do not forget to add your code into a source code repo.