Automation Weaknesses

Over the last few weeks I have been struggling with automating deployment and testing of virtual desktops for my own edification. This struggle has pointed out automation weaknesses which need to be addressed for automation and the software defined data center to succeed and to not only be deployed from software, but also to be self-healing and all the great things we associate with SDDC, SDN, etc. But current automation has some serious flaws and weaknesses. In essence, in order to automate something you must have a well known exact image from which to work.
Unfortunately, images today in the wild are not well known or exact, and once you start to automate those workloads, you run into issues. Automation works best when you can use a recipe that produces the same cookie from the same cookie cutter every time with the same dough. If there is an aberration in the dough, then the cookie will not turn out right. So we need to ensure that the dough is always the same. This is the goal of food packagers and the goal of the SDDC. But getting there requires some form of exact recipe to follow. When that recipe is not exact and does not account for differences, we run into huge issues with automation. In fact, it will fail.
This does not bode well for self healing. Automation scripts need to understand variances in order to automate (self-heal) systems back to a well-known state. If the state is so unknown to the automation suite, then it is crucial that the unknown pattern be learned so it can be corrected, and today that implies human intervention. If, for example, your automation requires that a particular GPO or registry entry be made, or a file be formatted in a very specific way, then the automation will fail spectacularly due to little variances between humans and machines.
For example, if you are to make an automated change to a configuration file you need to not only consider case within the configuration variable but the placement of the delimiter. Further, the deliiter could be zero or more spaces between each element. You also need to consider misspellings in the configuration variable and value. Another example: if you need a policy to be modified and the policy does not exist, then you will first need to add it before modifying it. These seem like simple items, but they catch many automation tools that do not think first about what exists before applying automation.
Once automation sets the patterns for a configuration or policy file, it can take over from there, but what happens if that file or policy is modified by an administrator by hand or even by another tool? Can the automation handle the change if it is not exactly what is expected?
This is in many ways one of the major automation weaknesses that exist today. First you need a perfect setup (which is hard to do, takes a huge amount of time) and that setup must be exact to what the automation tool requires. This is something we discovered while using Login VSI to perform virtual desktop testing in our own lab. Login VSI is a very powerful automation tool, but it is limited by how exact the environment in which it runs is. The advantage of something like Login VSI, however, is that there is very good documentation, as well as other resources, about the tool’s requirements and how to set things up to meet its exacting standard.
To combat this style of automation weaknesses, automation tools must also inject documentation about setups required, about configuration variable, etc. modification by which tool, and about the expected format and entries that need to be made to run the tool again. We should not only create recipes but create self-documenting recipes that document not only the changes within configurations but also document the per-machine run book or even ticket system so that humans can also understand what happened. Not only should automation provide documentation, but it should also understand the variances within a system and should both ask how automation should be applied (perhaps as part of the automation workflow) and have built into it understanding of common differences.
How do your automation tools deal with variances that occur? Do they account for variances in syntax? Do they require a well-known image from which to work? Can they self-heal based on user input? Can we have true SDDC if our automation scripts expect everything to be perfect beforehand? If so how do you get old workloads into a SDDC?