Fortune 500 Global Insurance Provider Managed, Automated and Orchestrated their F5 ADC Infrastructure Using AppViewX ADC+

Client Information

The client is a leading global insurance organization, operating across more than 80 countries and jurisdictions. Also, a Fortune 500, providing insurance and other financial services to support their clients in business and in life.

Get a Copy of our Case Study Now!

Business Objectives

With a rapidly-growing customer base of over 100 million and thousands of internal and public-facing applications, the company’s IT infrastructure team was constantly challenged with endless service requests. Despite having one of the most powerful ADC infrastructures supporting their applications, the team found its core processes to be painfully manual, slow, and inefficient. They realized the immediate need for a more comprehensive, scalable and automated solution that could catapult them into the digital world.

Key Technical Objectives

  • Streamline F5 BIG IP configuration management
  • Automate LTM provisioning and VIP lifecycle management
  • Enable Application and Security teams with self-servicing capabilities
  • Obtain granular visibility into Network Infrastructure
  • Perform device backups on regular basis and restore when required

Business Challenges

With over 120 ADC devices and no management & automation platform, the network and application teams were unable to efficiently leverage the full potential of their F5 ADC infrastructure

  • Long queue of tickets and subsequent service delays – Every change request had to go through the network team. Application teams had to wait long hours even for a simple enable/disable operation, which again had to be done manually.
  • Outages due to implementation of faulty configurations – ADC configuration change requests were raised by multiple teams across organization without being vetted thoroughly at multiple levels (completely manual).The change records were manually opened, approved and executed.
  • 3 – 4 days to provision an LTM instance – Backup, rollback, migration, and provisioning of device configurations were all manual and error-prone.
  • Time-consuming software upgrades – The team was manually handling the high CVE upgrades, which was highly time-consuming thus were unable to run adequate validation checks, causing production outages.
  • No VIP clean-up process – Utilizing Orion and custom SNMP discovery to generate reports and were analyzed and validated manually. The change records were manually opened, approved and executed.

AppViewX ADC+ as a Solution

AppViewX’s ADC+, application delivery automation solution provided role-based management, automation, and orchestration of F5 BIG-IP services. It simplified version upgrades and enabled self-service capabilities to multiple lines of business for the client.

  • Self-Servicing with Controlled Network Dashboards
    Multiple teams could get real-time visibility into the state, status, health, and performance of devices and applications from the Controlled Network Dashboards. Network engineers could create automation workflows for application-centric tasks like application enable/disable for rerouting traffic or spinning up virtual instances for testing. These workflows could then be shared with application owners using role-based access controls to self-service application-centric tasks without relying on network teams, leading to a significant drop in tickets.
  • Configuration Management with Out-of-the-Box Automation Flows
    AppViewX ADC+ fully automated the migration of configurations across devices and reduced configuration errors with out-of-the-box automation flows, change control through ITSM, pre- and post-validations, and built-in approval management process. The Visual Workflow module of the ADC+ solution generated templates of existing device configurations that could be updated with the required variables and automatically pushed to the new devices after the automated mandatory checks. Visual Workflow also supports bulk migrations, eliminating the need to type out configurations from scratch. These templates could also be self-serviced by the application teams, further saving time and effort.
  • VIP/WIP Lifecycle Management
    It automated the lifecycle of VIP/WIP management on BIG-IP LTM and DNS – from creation, modification, deletion, and decommissioning. AppViewX ADC+ is integrated with BlueCat to reserve & fetch free IPs and map them to the virtual server(s). It enabled teams with configurable parameters to track the VIPs/WIPs that are up/down for ‘X’ time frame. It also automated the approval and validation processes involved in creating virtual IPs.
  • Software Version Upgrades
    Software version upgrades, too, could be easily accomplished with the APS templates. Configurations could be migrated to a new/unused instance where the upgrade could be applied and tested, and finally brought to production. AppViewX automated the whole gamut of pre- and post-validation checks, ensuring zero possibility of outages and other service disruptions.
  • Backup and Restore
    AppViewX ADC+ enabled engineers to take on-demand or scheduled backups of device configurations and attributes and store them in a centralized repository. It also facilitates easy rollbacks to the last working configuration in case of failure during migration.
  • End-to-End detailed Reporting
    The platform enabled F5 administrators with app-centric topology views through customized reports and dashboards. Leveraging REST APIs, it helped the client optimize application and ADC performance with real-time auto-generated reports on CPU utilization, application traffic statistics, and unused VIPs.

Business Benefits

  • 99% reduction in time taken to provision LTM/GTM configurations
  • 98% reduction in application service delays with self-serviceability
  • 75% reduction in time taken for software upgrades – 1 GB or bigger files copied to multiple devices via single workflow in minutes
  • 10X increase in application availability with almost zero outages
  • Total Cost of Ownership reduced by 92%

Nationwide Building Society Simplifies and Automates Certificate Lifecycle Management with AppViewX

Client Information

Nationwide Building Society is the seventh-largest cooperative financial institution and the largest building society in the world, with over 15 million members. It is headquartered in Swindon, England.

Get a Copy of our Case Study Now!

Business Challenges in Certificate Management

Prior to AppViewX, Nationwide was using a generic solution to manage certificates. The solution was manually-intensive – certificate request, issuance, renewal, etc. required multiple steps and several back-and-forth exchanges between stakeholders. Manual certificate management resulted in considerable delays and inefficiencies.

The AppViewX Advantage

AppViewX provides Nationwide with a unified, automated solution to manage its certificates end-to-end. The certificate team receives automated certificate alerts and can request and download certificates from a single point. Issuing and renewing certificates have also got a lot easier, as AppViewX collects all the data necessary for requesting a certificate, which means users no longer need to go back and forth to gather missing information. This has significantly reduced enquiries.

The solution sends notifications to the team to enable access whenever someone requests a certificate and allows them to quickly approve or decline requests after viewing, eliminating multiple steps from the previous process.

AppViewX comes with a made-to-measure workflow which integrates the customer’s Active Directory, simplifying access and creating more efficient processes for users. The interactive GUI makes navigation easy and intuitive for users, adding to efficiency.

Customer Testimonial

“The implementation of AppViewX has materially saved time and effort for users across the whole certificate management lifecycle, which is a great outcome.”

American Credit Card Giant Automates 400,000 LTM and DNS Service Requests a Year With ADC+

Client Information

The customer is a Fortune 500 financial services company in the US, specializing in credit cards. It has over 110 million customers worldwide.

Get a Copy of our Case Study Now!

Business Challenges

As a leading provider of financial services, the company handles tens of thousands of transactions every day, most of which happens online. The company has thousands of application instances running in distributed data centers across the globe to support the huge traffic. Application teams continuously develop new applications and feature updates to deliver seamless customer service.

Each new application instance or upgrade requires a configuration change on the LTM and DNS devices, which lands into the NetOps pipeline as a service request.

  • NetOps teams received over ten thousand service requests a month on average. The requests included creating/modifying/deleting VIPs, WIPs, and DNS records, enabling/disabling pools for server rotations, creating configuration items in ServiceNow, generating performance and resource utilization reports, etc.
  • NetOps engineers had to resolve each service request manually. Manually resolving a service request at any given time took up to 10 days, owing to long backlogs.
  • There were no standardized workflows for the above processes. NetOps engineers scripted different workflows for the same process, leading to inconsistencies and compliance issues

Solutions Delivered by ADC+

AppViewX ADC+ provided out-of-the-box solutions that helped NetOps teams standardize, automate, and orchestrate service requests end-to-end.

  • With ready-to-use workflows, NetOps teams could automate all repetitive service requests and resolve them in minutes as opposed to days, doing away with busywork.
  • App owners could self-service tasks such as enabling/disabling pool members for blue-green/canary deployments and server rotation within the change window by triggering pre-built workflows with RBAC.
  • Since workflows were templated, all NetOps engineers and app owners followed a standardized procedure to execute any task, such as creating a VIP or modifying a DNS record.

Business Benefits

The company began using ADC+ in 2017.

  • With automation and self-servicing, the number of service requests that the company could handle within the same period grew dramatically. It automated around 400,000 service requests a year.
  • Automation cleared backlogs and accelerated service delivery. The time required to process a service request fell from days to minutes.
  • Time saved with automation and resource optimization resulted in the company saving $20 million annually.

American Media Conglomerate Reduces TCO (Total Cost of Ownership) of F5 ADC Devices By 55% With ADC+

Client Information

The customer is a mass media and entertainment giant headquartered in the US, with offices worldwide. They generate annual revenue to the tune of $35 billion on average.

Get a Copy of our Case Study Now!

Business Objectives

In 2015, the company had around 1500 applications, 70 F5 ADC devices, and 9 network field technicians managing them. Year by year, the application count kept going up, warranting more ADC devices.

  • The company needed to employ more network technicians on contract to manage the growing ADC numbers, resulting in an exorbitantly high TCO (Total Cost of Ownership) projection.
  • Although the company had only 70 ADC devices initially, they lacked centralized management.
  • With no centralized management and strict access controls, the devices ran a high risk of getting compromised.
  • The team stored and exchanged certificates in spreadsheets and emails, worsening security issues.
  • ADC management was chiefly manual, resulting in significantly high TTM (Time To Market) for applications, which in turn impacted business.

The company realized that the only way to lower TCO and improve operational
efficiency was through automation, which led them to ADC+

Solutions Delivered by ADC+

ADC+ provided end-to-end management and automation of F5 ADC devices.

  • It provided centralized management of ADCs across data centers, allowing engineers to view and control ADCs from a single, GUI-based pane of glass.
  • Network engineers could gain real-time visibility into the health and performance of applications, as well as historical data, helping in optimization.
  • ADC+ allowed the company to schedule and automate both device and object-level
    backups of F5, improving efficiency.
  • External teams could use API for bi-weekly maintenance activity to bring applications up/down. This provides significant time saving while rolling out the maintenance patches.
  • Application owners could self-service the management of their respective applications with RBAC-powered dashboards, reducing the reliance on network engineers for repetitive tasks such as enable/disables.
  • RBAC also significantly brought down instances of unauthorized access,
    enhancing security.
  • ADC+ provided complete, application-centric visibility into the application’s underlying infrastructure, aiding lighting-quick detection and resolution of incidents.
  • ADC+ automated the Certificate Lifecycle Management of F5 devices
    end-to-end, from request to renewal. Spreadsheets out, security in.

Business Benefits

The company began using ADC+ in 2017.

  • The company’s ADC fleet grew from 70 devices in 2015 to 200 devices in 2020.
  • Their application count went up from 1500 in 2017 to 6500 in 2020.
  • Due to centralized management, automation, and self-servicing capabilities offered by ADC+, the number of contract network technicians required dropped from 9 for 70 ADC in 2015 to 4 for 200 devices in 2020.
  • As a result, the company was able to lower their TCO by 55% while increasing their F5 ADC footprint by almost 300%.
  • With automation, the TTM for applications reduced by over 80%
  • The MTTR (Mean Time To Resolution) of incidents reduced by 85% with visibility, monitoring, and context-aware troubleshooting.

US Biotech Giant Improves Security Posture for Remote Work

Client Information

A leading U.S.-based life sciences company that specializes in providing instrumentation, equipment, software, services, and consumables to the healthcare, pharmaceutical, and biotechnology sectors.

Get a Copy of our Case Study Now!

IT Background

This customer is a firm that has multiple internal public key infrastructures (PKIs) for issuing certificates. The organization also has both on- and off-network client computers that required regular system updates and patches. This is a critical infrastructure security, especially for supporting increased remote work scenarios used during the COVID-19 pandemic.

Microsoft System Center Configuration Manager (SCCM) performs the patch and software deployments. SCCM requires machine certificates to authenticate and establish connectivity with the hosts.

Machine-identity certificates used for application security purposes are issued to client computers, services, and servers. This PKI system required extensive management, particularly the acquisition, enrollment, and management of certificates.

Primary Business Challenges

The IT team sought an abstraction tool that could achieve the following objectives:

Uniform Certificate Auto-Enrollment: While computers connected to Active Directory can leverage the Group Policy-based Windows Auto-Enrollment feature, those without regular connectivity to Active Directory cannot; hence, there was no way to enroll certificates on these devices. SCCM could not function because it requires certificates to authenticate.

Furthermore, certificates had to be renewed, enrolled, and installed on their respective clients periodically, and new computers were continually onboarded onto the network. This necessitated use of a tool that could standardize enrollment and connect to computers primarily running off-network.

Inventory and Reporting: Frequent acquisitions, ad-hoc deployments, and use of multiple certificate authorities and vendors posed a challenge for obtaining a comprehensive overview of certificates and their respective endpoints. This leading life sciences company identified the need for a centralized inventory for certificates deployed across the network. The company also wanted a transparent view of the certificate infrastructure.

Delivering a solution with AppViewX

After careful analysis, the AppViewX team crafted a solution for each of the aformentioned challenges. By helping implement an EST-based enrollment agent and a full-cycle certificate management suite, the solution met all objectives and delivered rapid results, which are detailed below.

Standardized Auto-Enrollment: An agent leveraging the EST protocol for certificate enrollment was deployed. It enabled AppViewX to act as an EST server, thus automating the enrollment and provisioning process. This also established a standard means of enrolling certificates across all machines, where AppViewX acted as a single, uniform interface for auto-enrollment. This enabled smooth patch and software management on SCCM client computers. Furthermore, new certificates were configured automatically on the end devices without human intervention. This particular use case was a distribut- ed, multi-node deployment across multiple Amazon Web Services data centers across the U.S. and Europe.

Controlled Access to PKI: To preserve PKI confidentiality and integrity, a role-based access control system was enforced across the network. It restricted access to infrastructure components, and, when necessary, provisioned them on an ad hoc basis. AppViewX’s audit trail feature also helped in this regard.

Full-cycle visibility, management, and automation: AppViewX’s environment scanning and inventory consolidation tool helped IT Operations build comprehensive inventories of certificates on file, complete with endpoint maps, statuses, and cryptographic details. AppViewX’s workflow automation capabilities enabled automation of certificate request/renewal processes while its reporting capabilities provided clear visibility into critical details such as validity. This increased visibility and control help prevent outages and contributed toward upholding organization-wide business continuity.

Fortune 500 Health Insurance Provider Makes ADC Infrastructure Agile with AppViewX

Client Information

The customer is one of the largest providers of health insurance in the United States and is a Fortune 500 company. The customer uses F5’s ADCs to manage their applications.

Get a Copy of our Case Study Now!

Business Objectives

With a rapidly-growing customer base of well over 50 million and thousands of internal and public-facing applications, the company’s IT infrastructure team was a perpetually harrowed one. The demand on the infrastructure was rising faster than what the team could cope up with, and despite having one of the most powerful ADC infrastructures supporting their applications, the team found its core processes to be painfully manual, slow, and inefficient. Their existing ADC management software couldn’t scale up to meet the demand, and the team realized the immediate need for a more comprehensive, scalable, automated solution that could catapult them into the digital world.

Key Technical Objectives

  • To reduce troubleshooting time by minimizing human intervention
  • To streamline ADC configuration management
  • To automate LTM provisioning and Virtual IP lifecycle management
  • To empower application and security teams with self-servicing capabilities

Business Challenges

The network and application teams were plagued by siloed, manual processes that prevented them from harnessing the full potential of their F5 ADC infrastructure.

  • Every change request had to go through the network team, resulting in a long queue of tickets and subsequent service delays
  • Application teams had to wait long hours even for a simple enable/disable operation, which again had to be done manually and was therefore added to the queue.
  • Backup, rollback, migration, and provisioning of device configurations were all manual and error-prone – it took up to 2 days to provision an LTM instance.
  • Software upgrades were manual and risky – the team had neither the technology nor the bandwidth to run adequate validation checks, causing production outages.
  • Incident management was a highly specialized, subjective affair – only skilled network engineers could diagnose and remediate issues. They were therefore called upon at all hours in the night – 2 AM, 3AM, etc. – in the event of an incident.
  • Most company-wide IT processes such as auditing and policy management were siloed and disconnected, requiring repeated calls to several teams to get them done.

Solutions Delivered

The closed-loop, context-driven network orchestration solution by AppViewX helped automate and stitch together fragmented processes, make the ADC infrastructure agile, and enable application owners to self-service application-centric changes to the ADCs.

Self-Servicing

Network engineers could create workflows with low-code for application-centric tasks – such as application enable/disable for rerouting traffic or spinning up virtual instances for testing – and share them with application owners using role-based access controls. Application teams could self-service these tasks without relying on network teams, leading to a significant drop in tickets.

Configuration Migrations

AppViewX fully automated the migration of configurations across devices. The Application Provisioning System (APS) module of the ADC+ solution generated templates of existing device configurations that could be updated with the required variables and automatically pushed to the new devices after the necessary checks (also automated). APS also supports bulk migrations this way, eliminating the need to type out configurations from scratch. These templates could also be self-serviced by the application teams, further saving time and effort.

Version Upgrades

Software version upgrades, too, could be easily accomplished with the APS templates. Configurations could be migrated to a new/unused instance where the upgrade could be applied and tested, and finally brought to production. AppViewX automated the whole gamut of pre- and post-validation checks, ensuring zero possibility of outages and other service disruptions.

Backup and Restore

AppViewX enabled engineers to take on-demand or scheduled backups of device configurations and attributes and store them in a centralized repository. It also facilitated easy rollbacks to the last working configuration in case of failure during migration.

VIP/WIP Lifecycle Management

It automated the lifecycle of VIP/WIP management on BIG-IP LTM and DNS – from creation, modification, deletion, and decommissioning. AppViewX integrates with IPAM solutions like Infoblox to reserve and fetch free IPs and maps them to the virtual server(s). It also automated the approval and validation processes involved in creating virtual IPs.

Incident management

It provided high visibility into the ADC environment and enabled network engineers to quickly detect issues and apply appropriate troubleshooting workflows. Integrations with change management tools such as ServiceNow streamlined incident management, drastically reducing MTTR and bringing down application outages.

Cross-Team Collaboration

The platform’s integration with ChatOps solutions like Slack enabled stakeholders to get
notified along every step of the automation process and also collaborate effectively with one
another for cross-functional tasks.

Business Benefits

  • 99% reduction in provisioning times for new devices/instances – the same LTM instance could now be provisioned in under half an hour.
  • 90% reduction in ADC change requests, with application teams self-servicing most of them.
  • 70% reduction in time taken for software upgrades.
  • An infinitely agile F5 ADC Infrastructure, High application availability with almost zero outages

A Multinational BFSI Company Provisions More Than 600 Devices With AppViewX

Client Information

The customer is a multinational banking and financial services company headquartered in the UK.

Get a Copy of our Case Study Now!

Objective

To enable provisioning of over 600 switches into the infrastructure on an ad-hoc basis. The need for provisioning the switches is two-fold:

  • To replace the old switches that were nearing their EOL(End of Life)
  • To support the company’s expanding operations in new regions across the globe

They had to provision 4 types of Cisco Nexus switches – ISR, ASR, CAT9k, and Nx-OS.

Business Challenges

In the process of tightening their network-level security structure, the firm discovered the weak link in their PKI – the manual management of certificates via spreadsheets, which resulted in an onslaught of problems, including but not limited to:

  • Tedious, error-prone, iterative steps for implementing configurations on over 600 devices, with each of the 4 types having a different configuration
  • Need for expert technical guidance
  • Intensive collaboration between teams
  • Multiple-levels of approval
  • Need to ensure policy compliance for all 600 devices

Solutions Delivered

AppViewX enabled network engineers to automate the provisioning process for the 600+ switches and routers from start to finish.

  • AppViewX’s modularity enabled the devices’ golden configuration to be kept separate from the business logic. This removed the need for modifying the device configuration code every time a logical change had to be executed.
  • The bank’s mandatory AIG (Application Installation Guide) was made available on-demand to help create the provisioning workflow and guide the engineers through the process of on-boarding the devices into the bank’s network infrastructure.
  • The need for extensive technical support was removed with the user-friendly visual workflow builder. The visual workflows helped automate the manual processes involved in provisioning and dramatically brought down errors and implementation times.
  • The platform’s built-in integration with their existing ITSM solution enabled the usually time-consuming change management process to be fully automated
  • Robotic Process Automation in AppViewX automated device information template generation and change creation for all switches, which could then be updated in the various management platforms by the respective teams (self-service)
  • AppViewX provided both maker-checker approval and auto-approval models for device configuration validation
  • Engineers could reuse the workflows for configuring the remaining devices ad-hoc instead of spending hours starting from scratch.
  • Teams could get notified along every stage of the process through emails and chat channels. The change reports could also be downloaded as PDFs.
  • Engineers could perform pre- and post-implementation validations to make sure the devices configured were in compliance with company policies, and the overall health of the network was maintained after the change

American Mortgage Giant Eradicates Network Downtime

Client Information

A veteran in the home loan and consumer refinance product space, this customer has been in the business since 1995. Catering to consumers and institutional partners in the United States, the organization generates an annual revenue of over $800 Million.

Get a Copy of our Case Study Now!

Business Challenges

In the process of tightening their network-level security structure, the firm discovered the weak link in their PKI – the manual management of certificates via spreadsheets, which resulted in an onslaught of problems, including but not limited to:

Challenge #1:

The usage of spreadsheets to track TLS certificate expirations and renewals was an inefficient, manual method which led to frequent outages and downtime caused by the aforementioned expirations.
Consequence: Anomalous effects often follow system downtime – apart from internal inconveniences, outages of customer-facing systems are responsible for loss of business, customer trust, and more often than not, legal fines.

Challenge #2:

The establishment of a hybrid, diversified infrastructure with no real visibility into its workings.
Consequence: A lack of system transparency is synonymous with mismanagement – cluttered arrangements are layered upon each other, addling the system with unwanted complexity. For instance, tasks such as renewals of certificate chains become next to impossible, due to an inability to conveniently locate and document the associated keys.

Challenge #3:

The lack of a centralized system to manage certificates across multiple environments and endpoints.
Consequence: Considering the application used by an organization of this scale, it was bound to have thousands of certificates across environments. Manual management resulted in the credentials for each certificate being scattered across the various documentation methods, making certificate-related operations like renewals and revocations time-consuming, and in some cases, impossible, or at least error-prone.

Solutions Delivered

AppViewX CERT+ acted as a single point of control for the entire firm’s network-related processes – from certificate lifecycle automation to vendor integrations for network devices. Here are the highlights:

  • Our single-window capabilities for detection, renewal, and revocation helped centralize certificate lifecycle management, permitting documentation and grouping of certificates in a single, searchable repository
  • AppViewX’s holistic view functionality provided complete visibility into the PKI associated with every environment or endpoint, permitting effortless, error-free management.
  • Automated renewals, graphical reporting, and reminders eliminated certificate expirations, which, in turn, resulted in outages and application downtimes being nearly nullified.
  • The firm leveraged out-of-the-box vendor integration capabilities to automate network tasks across CAs (Entrust, GoDaddy etc.) and endpoints (Windows Server, F5, Citrix Netscalers, AWS etc.)

Granular role-based access allowed central PKI teams to delegate and self-service tasks such
as certificate creation, revocation, and deletion.

Global 2000 Bank Eliminates Certificate-related Outages

Client Information

A multinational bank with operations in 70+ countries across geographies, this customer is a seasoned player in the BFSI domain. Offering a diverse range of consumer, corporate, and institutional financial services to clients, the organization generates over $15 Billion of revenue annually.

Get a Copy of our Case Study Now!

Business Challenges

The network security teams across the firm’s global offices faced several PKI-related issues in their day-to-day operations:

Challenge #1:

The issue of paramount importance was the outages caused by expiring certificates, which, in turn, were triggered by less-than-ideal methods of certificate monitoring and management.
Consequence: System downtime often results in unprecedented ramifications – the internal inconvenience notwithstanding, outages of customer-facing systems are responsible for loss of business, customer trust, and more often than not, legal fines.

Challenge #2:

Rogue certificates were responsible for obvious compliance and security issues, resulting in unnecessary effort expended on remediation.
Consequence: The direct outcome of these issues is twofold: the possibility of regulatory
fines aside, exposure to risk of data theft is a huge cause of concern.

Challenge #3:

The PKI team was scattered across geographies, complicating cross-functional certificate management.
Consequence: Manually managing certificate infrastructures is a tedious exercise as is, and the existence of a geographical component only served to complicated it further, increasing the probability of human error plaguing the system.

Solutions Delivered

AppViewX CERT+ automated the entire certificate management process within the customer’s network. Here are the highlights:

  • Our single-window capabilities for detection, renewal, and revocation helped centralize certificate lifecycle management, permitting seamless collaboration between teams split across geographies
  • Implementing full-cycle certificate management and automation eliminated the possibility of human error across the board.
  • AppViewX’s integration with EJBCA (a widely-used CA) was a major draw, greatly simplifying the renewal/provisioning routine.
  • Our automation workflows and policy definition capabilities ensured that the customer remained compliant
  • The solution resulted in significantly lower network outages, security risks, and other miscellaneous vulnerabilities.