The State of Vulnerability Management and Patching in The Enterprise Environment

The Linux vulnerability landscape is becoming increasingly complex, in part due to a seemingly never-ending number of new vulnerabilities that are constantly surfacing. 

Even when Linux-based operating systems are used at a small scale, it is challenging to patch vulnerabilities consistently. At an enterprise scale, the task of managing hundreds of vulnerabilities over fleets of thousands of servers is not simple at all. 

Yes, there are a variety of tools that can help – but awareness of tools such as automated patching and live patching varies, and these tools are used inconsistently. With the management of vulnerability assessment and patching varying so much from one organization to another, TuxCare set out to investigate how enterprises approach this challenging task.

Our survey, State of Enterprise Vulnerability Detection and Patch Management, revealed several interesting insights into how organizations handle vulnerability and patch management at an enterprise scale. The survey explores how these tools are used and examines the restrictions faced by organizations in their ongoing fight against threat actors.

Vulnerability Management Is a Compliance Priority

One of the reasons that TuxCare initiated a survey into the enterprise vulnerability and patching environment is that, for large organizations, vulnerability management and patching is a compliance issue.

Over and above the obvious security concerns surrounding vulnerabilities, enterprise Linux users also need to meet compliance obligations. In other words, there are laws and regulations in place that demand that large organizations meet minimum requirements around the remediation of vulnerabilities. 

Where organizations covered by these regulations fail to meet minimum requirements it can lead to stiff penalties. The rules that apply to companies operating in a specific industry vary, with organizations that deal with personal data – finance and healthcare firms, for example – under much stricter supervision.

We mention compliance because it has a direct effect on how large organizations approach vulnerability management and patching. Some enterprise Linux users must respond much faster to emerging vulnerabilities than others. The results we gathered in our survey clearly highlight how compliance requirements affect day-to-day vulnerability operations.

The TuxCare Enterprise Vulnerability and Patch Management Survey

TuxCare started surveying key IT security personnel across enterprise organizations at the start of 2021. We wanted to take a close look at three key aspects of vulnerability and patch management: deployment practice, maintenance windows, as well as the broader level of security awareness in an organization. We published the initial results, but the survey is still actively running and you are welcome to contribute. 

Initial responses have already revealed several interesting observations. From the start, we noted that the geographic location of the respondent had a negligible effect on the response we received. In other words, there was no correlation between the location of a respondent, and the answers returned by that respondent. 

That indicates that vulnerability and patch management practices are roughly the same across the globe. However, our survey revealed significant differences between industries. The sector in which an organization operates clearly has an impact on the way that an organization manages vulnerabilities and patching.

Taking a First Look at the Results

A few points jumped out at us. For example, we noted that automated patching is commonly used by organizations around the globe, as 76% of our respondents said that they apply automated patching across their workloads.

We also noted that live patching, a step up from automated patching, is in use at many organizations, as about half of our respondents reported that they relied on live patching to fix vulnerabilities. It makes sense that, at the enterprise scale, teams would rely on automated and live patching because of the sheer number of vulnerabilities that require patching.

Given today’s pervasive cybersecurity threats, it is no surprise that automation is a commonly used tool, so we found it interesting to note that manually researching vulnerabilities via online resources is in fact the most commonly used tool in our respondents’ vulnerability management arsenal. Even though automation of vulnerability management is commonplace, comprehensive vulnerability management still requires a few manual steps.

Another interesting fact emerged: 73% of our respondents suggested that their server fleets rely on a single Linux OS. In other words, rather than utilizing a specific Linux distribution for each different server role, most respondents reported that they picked a single OS – in most cases, it was CentOS or a fork of CentOS. 

Organizations are probably choosing to do so because using a single distribution makes maintaining server fleets so much easier – whereas a mix of distributions increases the time spent on server maintenance and addressing vulnerabilities.

Vulnerability and Patch Management Practices Vary by Industry

Looking more closely at what our respondents said, we noticed that vulnerability and patch management procedures and practices varied significantly from one industry to another. 

For example, when compared to the banking and financial services sector, respondents in the tech sector reported spending three times as much time in any given week on vulnerability monitoring. It’s possible that tech sector respondents are simply much more aware of cybersecurity threats than those working in banking and finance.

Another observation we made is that the tolerance or indeed the need for patching-related downtime varied significantly from one industry to another. In transports and logistics, our respondents reported that their organizations experienced around 15 hours a week of patching-related downtime. In contrast, respondents working for healthcare enterprises reported downtime of only about an hour a week.

The staff resources dedicated to monitoring for vulnerabilities also appear to be allocated very differently depending on the industry the respondent works in. In public and social services, respondents suggested that a large proportion of staff hours are spent on monitoring tasks – whereas respondents in the industrial sector said that very little time is spent on monitoring for vulnerabilities.

Resources Remain a Restriction

In the last section, we pointed to the allocation of staff resources when it comes to vulnerability management. Staff hours are a limited resource, and we found a few interesting trends in the responses we received. First, when it comes to documenting patching efforts, our respondents reported that documentation takes up very little time when compared to the other efforts made around patching.

In fact, we found that respondents suggested that trying to settle on a maintenance window that keeps everyone happy takes up a significant amount of staff time. We suspect this may be because of the many stakeholders involved in settling on an acceptable maintenance window – after all, maintenance windows cause significant disruption.

Resourcing is without a doubt a restriction, as 38% of our respondents said that they wanted to increase their IT security headcount in an effort to improve how effective their patching regime is. In further supporting evidence, 29% of respondents suggested that on at least one occasion patch installation was delayed because of a lack of resources.

That’s probably why 54.5% of our respondents said that the staff resources at their disposal are not sufficient to meet the patching workload. A further 27.2% indicated they have active plans to hire more staff to cope with the growing vulnerability and patch management workload.

The Tools that Support IT Security Staff

We also asked our respondents to give us some insight into the tools used to support the human efforts behind vulnerability and patch management. We found that there were several key tools that respondents suggested would help them make better use of the resources at their disposal.

In response to our survey, respondents pointed to several features that they would like to see in a patch management tool. First, enterprise Linux users wanted quick responses to new CVEs to ensure new vulnerabilities are rapidly covered. Live patching was also top of the list, while respondents wanted to see more comprehensive automated reporting.

We left the question open-ended. One respondent suggested that vulnerability tools should offer better logging capabilities than they currently do. That may be because many tools simply do not offer a lot of transparency into the functionality of the tool, or how the tool modifies systems as it manages vulnerabilities.

Our respondents requested a few other features, including phased rollouts to manage patching in a more controlled manner in order to prevent disruption.

The Implications for Enterprise Linux Users

Just like any other major operating system, Linux-based operating systems are subject to new exploits on a weekly - if not daily - basis. The number of exploits keeps growing and one of the reasons for this is that threat actors rely on automation to find vulnerabilities.

Battling a cybersecurity threat that’s underpinned by automation won’t be easy and using automation in security efforts is really the only way forward. This includes patching automation, already used by many of our respondents. Similarly, automated vulnerability management tools that have just the right feature set will prove equally valuable.

It is heartening to see that so many of our respondents are engaging with automated and live patching, but neither of these tools has full penetration and there is little doubt that automation is the best way forward.

Win a Course for Kubernetes 

We stated earlier that the survey is still running. Even though we’ve collated some of the initial responses, we’re still eager to hear from respondents working in the enterprise Linux environment. For this reason, we’re offering ten free CKA (Certified Kubernetes Administrator) certification courses run by the Linux Foundation. 

You stand the chance of winning  one of ten courses from us simply by completing our survey on this link. By completing the survey, you also help us to gauge how vulnerability and patch management is handled by enterprise Linux users.

Don’t forget – you can download the full report covering the initial results of our survey, State of Enterprise Vulnerability Detection and Patch Management, state of enterprise vulnerability detection and patch management report.

 

Thank you to CloudLinux for contributing this article.