March 26th, 2018
Every engineering manager that I know has had trouble getting the funds to complete projects that creep into the Capital Expense (CapEx) request range. It is the nature of engineering in industrial manufacturing and many projects tend to be put off for years at a time or until a catastrophic failure occurs. This article suggests a path forward, a methodology of sorts, to progress that same high-cost project forward without CapEx approval from corporate. It should be noted that this is much easier said than done and is much more probable on small to medium-sized projects.
I worked in maintenance while paying my way through college for my first degree and, after that, I became a controls engineer. During that time, I saw a lot of projects get turned down and there was no choice but to get creative with implementation and costs. Since those days, I have participated as technical resources for sales and acted as a partner to many control system integrators and EPC companies. Needless to say, I have seen dozens of examples of solution provider quotes and their methodologies for project implementation; many of which were very good, especially for turn key CapEx solutions.
But very few offered flexibility to adjust timeline and costs to fit local budgets. Due to the nature of large project solution providers needing to schedule months to years out and delicately plan the resources, they just can not give the attention that many companies need for cost-effective, slower implementations.
Your facility needs to upgrade control systems for the following reasons:
- Reliability – Operators cannot meet production goals and schedules.
- Obsolescence – Replacements parts are harder to find than ever and can cost 1,000% higher than modern components. Not to mention, it is hard to trust eBay.
- Security – The legacy PC operating system (OS) is no longer supported by the manufacturer. Do you still have Windows XP in your plant?
- Skilled Labor – It is harder today to find experienced, skilled control engineers for legacy platforms than it has ever been in the past.
- Expansion Limitations – You cannot expand due to cost or lack of functionality for new production goals.
- Automation – Human error has caused reduced product quality and material re-work.
The time has arrived. You get quotes from manufacturers like Rockwell, Siemens, Emerson, and their partners. The budgetary numbers come in and you already know that a six-figure request is years from being approved. But based on your dwindling stockpile of eBay components, you have a year at best to get the system upgraded. Assuming no one realizes that it is cheaper to build a new plant than to replace all the legacy systems, you have work to do.
Analyze, Classify, & Mitigate
If you have ever completed a functional or process safety risk assessment, this should come easily. Log every legacy system in your plant and rate them based on the issues stated above in your scenario. Then create an implementation timeline to prioritize which systems need to be upgraded first. The idea is to reduce (mitigate) risk in the highest priority cases first.Hopefully, you do not have too many legacy systems holding you back.
Luckily, getting a quote from a qualified integrator or manufacturer is generally free. Ask specifically for quotes that itemize the tasks to be completed and discuss ways to phase the project. This will be different based on the supplier as methodologies change. It is critical to select a partner that has a written and proven methodology. Certifications such as CSIA or ISO 9001 can be a great clue that you are dealing with a qualified provider.
Step 1 of the project process should always be a functional design specification (FDS), especially if you do not have a well-documented user requirement specification (URS). The FDS defines the roadmap over the entire replacement period to plan your migration. Projects without this initial study are much more likely to come in over budget and blow the timeline, neither of which you can afford. Look specifically for a supplier that writes the study in plain language, no technical jargon, and outlines itemized downtime needs for startup.See the video below for an example that is far too technical.
These studies can range in pricing from just a few thousand to $60k. But I find that it tends to be manageable to most companies as the price of their system study tends to be somewhat proportional to size and revenue of the company. For instance, a large high-producing company may have an 8,000 I/O system (more money needed for study) while smaller ones may be 200 I/O and a smaller study is needed.
Congrats! Phase 1 of your project is done. You now have the study and once the local budget is replenished, you can start on the next phase. Most manufacturers now have software that can be upgraded with common support packages or other means so timeline shouldn’t affect versioning issues. Start by ordering the software and a block of engineering hours so that your supplier can get to work with configuration and programming using the earlier study.
As your budget allows, start ordering hardware and prepare to buy the panel design/build services. This is an area that you can save money if diligent. Make sure to request special pricing from manufacturers, and look for ways to save such as doing retrofits instead of new panel installs.
There are several itemized tasks that can be completed in this phase such as graphics development, interlock development, engineering station/client machine build and several others. Make sure to request that the programmer use the manufacturer’s standard library for graphics and other programming modules. This helps to lower the development costs and pushes the responsibility to the vendor to ensure migration paths for future versions. Customization = RISK! You should also be able to choose between different types of simulations that you need on the system. Many times, a simple simulation checking that the program works as outlined in the original study is all you need.
If using virtual machines, see if your supplier will allow your IT group to set up the servers or install software on local machines as that can take a significant amount of time while waiting for software to load or images to copy. Although you want the majority of tasks supervised by the supplier’s project manager, it is okay to ask if you can do something to save time, especially if on a Time and Material (T&M) contract.
This is similar to the functional design study. You must plan to do this in full without any caveats as the testing phase is the overwhelming factor that ensures project success. Most suppliers will do the FAT at their facility with you onsite and it is possible to remote in and accomplish the same tasks a little cheaper, but you absolutely do not want to skimp on this phase. There must be a testing phase prior to any onsite implementation and it must be executed via a proven strategy. This is critical to success!
Startup & Commissioning
You are now in the home stretch and can see the light at the end of the tunnel. It has probably been several months or possibly even a year since the original engineering study and your system is about to get installed. It is extremely necessary to plan more downtime than agreed upon just in case something goes wrong, but planning creatively can help.
Every facilities production schedule is different and many feel that they never stop running, which may be the case for you. Get creative and share production schedules with your supplier, as them to help create a plan that minimizes downtime. A few examples below are of creative startups.
- Simple Example 1: Facility closes on Saturday and Sunday, must start production again at 7 am Monday morning. Project manager requests 7 days (56 hours) of startup and commissioning.
- Solution 1: Integrator plans for 4 shifts with double engineering to take place on Saturday (3 shifts) and Sunday (1 shift) for a total of 64 hours. 8 hours more than needed. There is also leftover time on Sunday that can be used as a contingency if needed.
- Advanced Example 2: Facility runs 24/7 but has significant downtime for product changeover.
- Solution 2: Although you must be careful, and I recommend advanced simulation beforehand, the supplier can stage in the system on the product changeover downtime periods. This requires a separate panel, no retrofits. During production, the panel is set in place and power applied. During each additional downtime segment, a portion of I/O or communication is transferred to the new system. Obviously, the production will be running on both systems, new and old, until all the I/O and comms are transferred over. This must be done in a way that each transfer is tested before production startup.
There are a significant amount of exceptions for any example, but the point stands that with proper planning and communication, it can be done.
Throughout this process, the key to success is project timeline and that will be based on when your maintenance/local project budgets renew which could be monthly or quarterly. If you work with your supplier up front, you can coordinate the milestone payments to coincide with that schedule. If you request T&M quotations, be sure to add “Not to Exceed” so that the supplier can not run up the bill.
My favorite way to start a project is to separate it across two fiscal years. This way you can submit a purchase order in “this year’s” budget and submit separate purchase orders in “next year’s” budget. Generally, I like to at least complete the functional design specification and possibly even some programming or configuration in the first phase and then spread the rest of the tasks across the next year. Be sure to ask your supplier for assistance in planning that out and never ask them to do work before you have the PO. Be patient and honest.
Your supplier should be more than just a supplier, they should be your partner. If you succeed, they will succeed. Never settle for someone looking to up-sell and only add to their bottom line. Any great company knows that the best thing for them is to help you thrive. Communicate often and do not be afraid of asking for help. You will know instantly if they are looking to take advantage of you and if they are, politely move on to someone that truly cares about your goals.