If 2020 was the year that we became acutely aware of the consumer goods supply chain (toilet paper, anyone? Anyone?), then 2021 was the year that the software supply chain rose in our collective consciousness. In perhaps the most infamous attack of the year, thousands of customers, including several US government agencies, downloaded compromised SolarWinds updates.
Alas, SolarWinds was not alone. Indeed, the weaknesses in our software supply chain were all too evident with the recent Log4j vulnerability. Log4j is a widely used open source Java logging framework, so the vulnerability has put tens of thousands of applications (ranging from data storage services to online video games) at risk.
With so much lightly maintained code running in production, the software supply chain is ripe for exploits like the Log4j vulnerability. This is a hot topic in open source because a lot of people consume lightly maintained software libraries, put them into production, and never patch them again.
This is why I am declaring 2022 the year of [wait for it] software supply chain security. But I’m not just going to declare a year and leave it at that (a la Michael declaring bankruptcy in “The Office”).
Following are three practices I predict will (and should) rise in importance in 2022 as organizations work to strengthen their defenses against software supply chain attacks.
Diving deep into distroless
In the year and years ahead, companies should be thinking about standardizing and thoughtfully trimming down their container images, including distro elements. In fact, some would go so far as to say that organizations should go “distroless.”
In the distroless model, applications are still packaged in container images, but only the minimum vestiges of the operating system remain. The idea is that by stripping out as much of the operating system as possible (for example, removing package managers, libraries, and shells), the attack surface is decreased.
However, it’s important to understand that, just as there are servers in serverless computing, there are distros in distroless computing—there’s just less of a distro. And this may just be the real value in the distroless model, i.e. providing the framework for carefully picking and choosing what’s needed and what’s not, rather than focusing indiscriminately on reducing the size of a single container image, while ironically increasing attack surface because of a lack of standardization.
Scrutinizing container images and registries
Software has never been more complex than it is today, and if you don’t understand everything you are deploying, you’re going to have problems. As the use of containers increases, organizations need to be really thoughtful about how they are consuming and deploying container images. In other words, you need to download a trusted thing from a trusted place.
I can hear you now: “But that will slow me down!” Yeah, it will. But the “one bad apple” idiom applies. You can take your chances, cranking out products at lightning speed, and maybe everything will be alright. Or you can be super-careful, and slower, and pretty positive that you’re not going to be the next SolarWinds (or Kaseya, or... you get the idea).
In fact, some organizations have a very controlled, almost air-gapped environment when it comes to pulling in software from container registries. Other companies let developers pull from wherever they want.
As I’ve said before, this is kind of like letting every contractor manage its own supply chain contract. That’s scary enough when nothing malicious is intended, but downright terrifying with malice aforethought. When it comes to the container supply chain, it’s too easy to pull in an image that was hacked. Get your container images from a trusted supplier, and/or make sure you understand (and can rebuild, from scratch) every single container image in your supply chain. Every. Single. One.
Evaluating SLSA
I predict that companies will start exploring (and implementing) SLSA. Pronounced “salsa,” SLSA stands for supply chain levels for software artifacts. It is a framework for protecting the integrity of the software supply chain.
SLSA is based on Google’s internal Binary Authorization for Borg (BAB) platform, an internal deploy-time enforcement check designed to ensure that production software and configuration are properly reviewed and authorized. Google notes that adoption of BAB helps reduce insider risk, prevents attacks, and supports production system uniformity.
The goal of SLSA, according to Google, is to improve the state of the industry by protecting against threats, especially in an open source context. SLSA also gives software consumers peace of mind about the security posture of the software they consume.
And peace of mind is really hard to come by these days. If you aren’t already nervous about all this, check out this quote from Nick Weaver, a security researcher at UC Berkeley’s International Computer Science Institute, and prepare for chills down your spine: “Supply chain attacks are scary because they’re really hard to deal with, and because they make it clear you’re trusting a whole ecology,” Weaver told Wired. “You're trusting every vendor whose code is on your machine, and you’re trusting every vendor’s vendor.”
Google has released a SLSA proof of concept that allows users to create and upload provenance alongside their build artifacts, thereby achieving SLSA Level 1. I recommend that any company that produces software—which, these days, is pretty much every company—check out the proof of concept. In my view, SLSA also aligns well with the Biden Administration’s call for a software bill of materials as part of its executive order on improving the nation’s cybersecurity.
Never break the chain
Organizations have been through a lot the last couple of years, and the rise in software supply chain attacks adds to the challenges as a likely exploit of organizations’ COVID-distracted state. As companies plan for how they will move through and past the pandemic, securing the software supply chain should be at the top of the priority list.
Without that peace of mind, organizations and their customers will be in a continuous state of looking over their shoulders. Of course, the whole idea of a chain is that it is only as strong as its weakest link, which means that no one organization can secure the software supply chain on its own. I wrote about that issue in more depth here: “Deep container inspection: What the Docker Hub Minor virus and XcodeGhost breach can teach about containers”.
It will be important in the year ahead to consider strategies and techniques, such as the ones described here, that you can add to your existing best practices. This kind of continuous layering will help organizations stay up-to-date in the fight against software supply chain attacks, and to prevent the unwitting propagation of such attacks themselves.
We’re always looking out for the next “black swan” event—the next unseen threat that could rock the system and destroy all of the work we’ve put in to build it up. There’s really only one overall defense to this scenario: Pay close attention to your supply chain!
At Red Hat, Scott McCarty is senior principal product manager for RHEL Server, arguably the largest open source software business in the world. Focus areas include cloud, containers, workload expansion, and automation. Working closely with customers, partners, engineering teams, sales, marketing, other product teams, and even in the community, Scott combines personal experience with customer and partner feedback to enhance and tailor strategic capabilities in Red Hat Enterprise Linux.
Scott is a social media startup veteran, an e-commerce old timer, and a weathered government research technologist, with experience across a variety of companies and organizations, from seven person startups to 12,000 employee technology companies. This has culminated in a unique perspective on open source software development, delivery, and maintenance.
—
New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.