Retrospective: praise for the cybersecurity experts out there

2021 was a year full of breaches and cyber attacks. The Colonial pipeline, the Florida Water treatment plant, and the exchange server vulnerability, to mention only a few. Then again, the worst case scenario of broad outages or mass ransomware attacks just didn’t happen because of the work of cybersecurity experts. For example, let’s take a look at the most recent headline in cybersecurity.

The log4j vulnerability hit the information technology community hard a couple of weeks ago, but the broader public hardly took notice. Even mass media reporting about the vulnerability was terse and lacking urgency. The coverage was just a rough explanation of what was going on, at best. Meanwhile, in the “machine rooms” of literally every IT department in every company. An army of IT administrators were getting ahead of the problem and preventing disaster. And it wasn’t the first time this year.

2021 was more than log4j

In a quick summary of this past year, while log4j might have been the most prominent example, the Microsoft’s Exchange server vulnerability was an even bigger threat but was patched without incident. Even now there are new reports about a vulnerability in the Active Directory service (also by Microsoft) which will be patched before anyone can exploit it.

For those of us who do not maintain networks for a living, installing patches is just an annoying requirement involving clicking a few buttons and rebooting a system again. But it we have little understanding of what’s involved in hardening the system against criminals trying to exploit the vulnerability. And we have no idea what lurks in the darkest corners of operating systems and software. Nor do we kn if the patch was produced fast enough to stop a criminal from planting a backdoor for later use. Here are three examples.

During the log4j fiasco, a small business network was simply too small to lose sleep over. It would have few network components, a small server, a website and a few connected devices. Software patches were readily available for off-the-shelf products.

How software providers fix vulnerabilities

But the software providers publishing those patches are dealing with a highly complex problem. The average operating system has about 60-70 million lines of codes. An office software suite is about as large. A full-fledged enterprise solution such as SAP’s ERP system S/4HANA contains more than 300 million lines of codes. That level of complexity means developers must use off-the-shelf components in their software just to keep up.

An analogy on LinkedIn explained why that fact makes the log4j vulnerability so dangerous. To summiarize, Log4j is like a screw used in every car in the world because it is free. Every software provider selling software uses that “screw” in their product because it is free and everyone else uses it. But because it is so ubiquitous and because the software is so massive, they aren’t always sure where they are using it. That means thousands of software engineers and architects were working overtime to identify that component. Imagine a car mechanic searching for a specific screw in a modern car. They would literally need to de-assemble the entire car. And that is exactly what software providers had to do as well.

Software providers, however, only account for a fraction of organisations affected by the vulnerability. Just take the example of Minecraft, where you could exploit the vulnerability simply by putting in a specific message into a chat window. Since anyone can set up a Minecraft server pretty easily, there are probably hundreds of thousands servers affected. Of course, those are decentralized and needed to be patched by thousands of admins.

How to patch thousands of systems at once

One admin we spoke with mentioned that their organization hosts more than 20,000 systems. All of them need to be patched and hardened. It’s not difficult to imagine how many extra hours and stress that would mean for the IT department of that organization.

In cybersecurity, of course, there are more roles involved in operating the machine room. If you read about the log4j vulnerability, you might’ve heard that cyber criminals are taking advantage by probing systems, planting backdoors or initiating attacks right away. Cybersecurity analysts are handling those issues, identifying attack early on and doing forensic analysis after regardless of the attacks’ success or failure.

Cybersecurity consultants steering companies through those rough waters, are developing resilience so a vulnerability like log4j won’t hit as hard in the future. Researchers, are finding ways to prevent vulnerabilities altogether or help fight the next generation of cyber attacks.

Therefore, it’s time to praise these admins, analysts and consultants, who have done so much to prevent worse in 2021 and will surely do so in 2022 and the years to come.