Very recently, an IT infrastructure automation major released news of two critical vulnerabilities in its framework. Within 24 hours of its announcement, reports of exploitation started pouring in. Several elite organizations and open-source projects that use the solution were forced to take down entire server systems, even while others were reeling from malicious attacks and breaches.
This incident raises three questions:
How did the vulnerabilities come about?
According to news agencies, the vulnerabilities germinated from the solution’s master-slave communication channel, allowing attackers to bypass authentication/authorization controls and connect to the master server, access all its files, steal the encryption keys, and remotely execute commands as root on both the master and the minions. As researchers from F-secure, the cybersecurity firm that discovered the vulnerabilities, put it,
Why are the repercussions so devastating?
One possible candidate could be the architecture – in the master-slave architecture employed by the solution, one master controls all the minions in its purview. So when the master gets compromised, all the slaves under it automatically get compromised, too.
For organizations that use the solution to manage the whole or a significant part of their infrastructure, the master, when compromised, becomes the single point of failure. When attackers are in possession of the master’s fileserver and its encryption keys, they can publish any command on the master and subsequently make the slaves (devices) run them.
This opens up a world of opportunities to the attackers – they can install backdoors for siphoning data, steal passwords, inject ransomware, mine cryptocurrency – most of which the attackers already achieved by the time the world realized the gravity of the situation. The mitigation measures involved taking whole server systems down until the security patch could be deployed, causing disruptions in business continuity and loss of revenue.
Could it have been prevented?
A post-mortem on the vulnerabilities and subsequent attacks suggests that it could’ve been prevented if a more decentralized solution architecture were used to manage IT infrastructures. Running a master is a double-edged sword – it does have its advantages, primarily where centralized configuration management and governance are concerned, but when it fails, it takes the whole system down with it.
Can an agentless microservices-based architecture be the safer option?
Microservices-based architecture, where services are deployed as modules rather than as a single stack, eliminates the problem of a single point of failure. Even if one or two modules fail, they can be isolated and remediated without affecting the rest of the infrastructure. Solutions that employ microservices architecture have one other advantage – they’re also agentless. They communicate with the infrastructure devices over SSH or cloud providers’ APIs and remove the need for a superfluous communication channel that increases the surface area for attacks. Using SSH and APIs further reinforces authentication and authorization controls through built-in mechanisms, bringing down the possibility of attacks.
However, architectural superiority is highly subjective and debatable, so let’s turn our attention to the less drastic, more easily implementable ideas –
Better governance with improved data collection and analysis
Continuous scanning the network for vulnerabilities and data collection from various sources and endpoints helps predict anomalies before they occur. Solutions like AppViewX scan and inventory devices from multiple vendors, generate real-time data from the devices and present them in structured, easily readable reports. Administrators can analyze these reports to quickly detect outliers and investigate threats before they have an impact on the systems.
Automated patch management
Most of the organizations were laid prone to attacks discussed above because they didn’t install the patches that were released by the vendor in response to the vulnerabilities in a timely manner. Patch management is still a relatively manual process, where the patches are installed and tested on each device individually. AppViewX provides automated patch management, where it integrates with the vendor to receive patches, scans the network for devices that need to be patched, deploys and validates the patches on them, and also generates a status report with the updates.
Event-driven incident management with integrated alerting mechanism
If a threat has been detected, it’s crucial that the engineers/teams responsible for resolving it be informed without any delay. Far too often, the issue is detected, an attempt is made to diagnose it, and only after that is the engineer concerned called, making the remediation process painfully slow and stretching the window for further failures. AppViewX, with its integration partner PagerDuty, helps triage and streamline priorities and approvals. It sends out real-time notifications via phone and other media, expediting staff assignment and incident remediation, significantly bringing down the MTTR.
Context-aware remediation
Oftentimes, a fix applied on one side of the infrastructure can cause something to break on the other side. To avoid this, the entire context of the infrastructure and its operation should be taken into account while remediating an issue. AppViewX performs such context-aware remediation – it factors in the state, status, and performance of systems across environments and platforms before it executes the remediation workflow, thereby making sure the fix applied doesn’t impact the infrastructure negatively.
Improved secrets and authorization management with vault
In the attack covered in this document, all the authentication keys and sensitive device-specific data were stored on the master server. When the master server got compromised, all the keys and data were exposed, giving attackers unrestrained access to manipulate the system. Such unauthorized exposures can be avoided by storing the secrets and keys in a vault-like storage that’s removed from the core framework and integrated with it. AppViewX provides a secure vault where you can store device authentication credentials and identity tokens in an encrypted format, which can then be used to broker secure access to users and devices through a unified ACL.
To know about how AppViewX works, schedule a live demo with our experts.
Tags
About the Author
Nishevitha Ramamoorthy
Product Marketing Manager - AppViewX CERT+
Nishevitha is the product marketer at AppViewX. She writes, does research, and builds strategies to communicate the product's value to prospective buyers.
