Most large organizations have a system in place to handle changes, respond to incidents, and manage releases and configuration changes. Modern issue management tools provide a smart automated way to initiate, record, review, and track these changes. But what happens after an operations / network / infrastructure team receives the request? More often than not, network engineers take days to resolve a ticket – they have to write the script for the change, run it through several levels for approval, implement it, check if the device and the network are working fine after the change… basically manually perform all the tasks necessary to get the infrastructure components to the state that would support the needed change.
For a company that has a dozen applications and a small number of service requests, this process might work just fine. But what happens when you are dealing with over ten thousand change requests each month? A majority of these tickets may only require a server or router reboot, but they still take several days before an IT person can get to them.
This was exactly the situation that a large Financial Services company found themselves in at the start of 2015. With zero automation and multiple development teams requesting changes to applications or provisioning of services, it took an average of eight days for a ticket to get resolved. Multiply this by 10,000 tickets per month, and it’s clear why the company’s IT management was getting increasingly frustrated with the NetOps team’s inability to keep up with implementing important application changes and making required configuration changes on the load balancers.
AppViewX provided the Financial Services company with an automation platform that gave the Ops team much-needed workflows and self-servicing capabilities, allowing them to reduce typical ticket resolution time from over a week to just a few hours. In as little as two years, the team went from zero to over 90% automation, virtually eliminating manual errors and improving agility. The manual procedure that required three separate approvals was replaced with self-service templates, automating the entire process of deploying and configuring load balancers.
So, can anyone go from zero to hero like this company? Absolutely! Especially if you keep in mind a few commonsense principles:
- First, set achievable goals, especially if you are just starting out with automation. Many companies make the mistake of trying to automate too much too soon. You need to start with a well-defined project – like building workflows for configuration tasks for load balancers, for example. Once you get comfortable with how things are working and see good results, you can expand your solution to other network components.
- If something goes wrong, don’t immediately blame automation. Don’t let people who are resistant to automation use temporary setbacks as a reason to go back to the “tried and true” manual processes. Try to understand the nature of the problem and find a solution that would allow you to move forward.
- Choose a network automation software that is closed-loop and state-aware – a solution that is capable of verifying that desired outcomes have been achieved, rather than simply automate a collection of individual steps.
- Build self-service capabilities. Give your team the ability to launch automated workflows and open it up to other teams who can tap into your workflows via APIs.
Automated network infrastructure management will definitely save you labor costs. But even more importantly, it will help your Ops team increase agility, improve application performance, and achieve greater operational efficiency. Now, that’s what real heroes do!