How to Measure Automation Deployment Success

Expert tips on understanding common risks when deploying automation and how to mitigate them

An example of how simulation and virtual commissioning leverages a digital twin to solve problems before they happen. (Image: ARM Institute)

An example of how simulation and virtual commissioning leverages a digital twin to solve problems before they happen. (Image: ARM Institute)

U.S. manufacturers of all sizes are interested in deploying automation to increase their throughput, augment their workforce gaps, and, as a whole, strengthen their operations. It can be overwhelming to even understand where to start with automation; concerns like the time spent on automation projects, the capital investment and expertise needed to get started, how automation will affect existing operations, and other issues can sputter manufacturers’ progress.

For these reasons and others, it’s critical to first understand how to measure the success of deploying automation. By keeping this at the forefront of your efforts, you can better understand your goals throughout the deployment process and adjust accordingly.

Lessons Learned

In my career, I’ve defined automation deployment success as launching the project on time and on budget with a rapid ramp-up to meet target production metrics without serious safety and cyber security incidents. To successfully deploy automation, it’s critical to identify common risks and make a plan to mitigate them.

What can limit your success in different settings? For new facility or production line (Greenfield) projects, late automation project launch is particularly serious if it delays launch of the entire building. Retrofit projects done during live production must be completed within allocated time windows with minimal disruption to continuing operations. 

Other factors that may limit your success are change orders. Change orders are a common source of delays on large, complex projects that have negative financial impact due to lost production and exceeded contingency budgets. If the project includes separate parallel engineering, construction, supplier, and integration teams, geometric clashes between equipment and building infrastructure are often not discovered until on-site. Early and continuous cross-functional coordination of internal and external teams is integral to minimizing change order risk. 

Differences in predicted construction also carry heavy risks. Building column locations may be two to six inches off from design drawing, due to standard concrete construction tolerances. Clashes can be detected in Building Information (BIM) Models by integrating 3D laser scanned “as built” reality capture of building, infrastructure, and existing equipment point clouds with CAD models for the new automation system. All installation teams should use common survey points and datums on the installation floor. 

There are also tools that you can use to better ensure deployment success. For example, virtual commissioning reduces the time and high cost of debugging control software and wiring on-site by running system simulations with robot controller and PLC production code. Most errors are found and addressed before shipment to the project site. Virtual commissioning that included the simulation of product mix and unplanned downtime events can be used to make the production code more robust. Concurrent introduction of Industry 4.0 data supports reduction of on-site adjustments and faster ramp time to target production OEE. 

Applying These Lessons in Real Time

To this end, the ARM Institute recently funded a technology project with Wichita State University’s National Institute for Aviation Research, Siemens Corporation, and Spirit AeroSystems on “Technology Assessment of Virtual Commissioning for Day One Manufacturing Readiness” ( This project is currently underway with the team creating a report that details the framework for the creation of a virtual twin for commissioning, and all the steps involved in its development. The framework package will contain the data and considerations needed to develop a full digital twin. Ultimately, this project will enable more successful installs and lead to better “day one” experiences after deploying automation.

Tablet-based standard work procedures and safety training also enable consistent installation quality and reduced lost-time accidents during deployment. Commissioning software application tools can confirm compliance with interface messaging and cyber security requirements.

Deploying automation can be risky, but when done correctly, it will reap major benefits for your operations. By understanding the risks, having a plan for mitigating those risks, and leveraging industry tools, you can ensure a smoother day one experience.