VulnCon, NVD, XZ, and More - March 25-31, 2024 - F5 SIRT - This Week in Security

Security Incident Response Team

Editor's introduction

Hello, it's MegaZone again. Seven weeks goes by fast - especially when you're organizing and then attending a conference! Yes, last week was VulnCon 2024. It was a great time - lots of incredible content, interesting conversations with great people from across the vulnerability ecosystem, and just an all around enjoyable, informative experience. Definitely the kind of event I'd love to see more of. If you missed it, see below for how you'll be able to see the content, and plan for next year.

Since last I wrote, back in January, I was elected to the position of CNA Board Liaison for CVE.org. That's voted on by the entire CNA community, so I'm honored to be elected and to represent CNAs on the CVE board for the next year (and hopefully beyond if you're all happy with me this year). While at VulnCon the three new CVE Board members, Tod Beardsley, Madison Oliver, and myself, recorded an episode of the 'We Speak CVE' podcast to introduce ourselves to the community - watch for that to hit the CVE Program YouTube channel soon.

VulnCon wasn't the only thing to happen last week though, so let's get into it.

VulnCon 2024 is In The Books - Onward to VulnCon 2025!

VulnCon 2024 was a rousing success, with around 600 total attendees - ~350 in-person, and ~250 remote (the official numbers should be out soon). That blew away our planning targets by a wide margin. The positive feedback we've received has been overwhelming. There were, of course, some issues, but for a first year event it went shockingly smoothly, IMHO.

All (but one) of the panels were streamed and recorded, and the recordings will be available on the CVE Program and FIRST.org YouTube channels in the next few weeks. There was so much great content and I highly encourage you to watch for those videos to be posted - and then watch those videos. There are several I'm looking forward to as there were three programming tracks and only one of me, so I have to wait for the recordings to see a few I wanted to attend but had to skip.

VulnCon 2025 will be back at the McKimmon Center in Raleigh, NC April 7-11, 2025. Instead of three days and three tracks, we're planning for four days and four tracks! That'll be even more great content to choose from! I suspect we're going to max out the capacity cap on that venue in 2025 for in-person attendees, but we don't want to grow too big, too fast. When the registration does open, I wouldn't wait too long is all I'm saying. There will still be the virtual attendance option, of course, for those who can't make it in person.

Thanks to all who attended, and special thanks to the fantastic organizing committee who pulled this off with only five months of planning! It was an honor and a pleasure to work with such an amazing, professional team.

Speaking of the organizing committee, this is most of us. Guess which one is me - reading name tags is cheating:

Photo of (most of the) VulnCon 2024 Organizing Committee

NVD News - from VulnCon!

In light of all the recent hubbub about NVD and the slowdown, indeed near cessation, of their publication of 'enriched' CVE data starting in mid-February, one of the most highly anticipated sessions at VulnCon was the NVD Symposium on Wednesday. In February, NVD had published a statement on their website indicating that there may be a temporary slowdown, and that statement has now been updated to provide more information. At the panel we received a somewhat vague explanation for the issues, which seem to boil down to budget cuts (NIST just had their budget cut by 12.5%) and the ever increasing number of CVEs being published. NVD just could no longer keep up, and now they are "prioritizing analysis of the most significant vulnerabilities." That is leaving a growing backlog of CVEs without 'enrichment', said enrichment primarily being the addition of NVD's CVSS, CWE, and CPE evaluations.

At VulnCon, NVD program manager Tanya Brewer announced the intention to launch a public-private consortium to take on the work NVD has been doing to date. Details of this consortium were sketchy, but the full announcement is expected within one-to-two weeks. What we do know is that those organizations looking to join the consortium will need to sign a Cooperative Research and Development Agreement (CRADA) and that a membership fee is being considered. The members will form a steering committee, and there will be working groups for individual issues. While we await the full details, those interested can contact nvd_consortium@nist.gov. I'll be interested in what the details are and what incentives there will be for commercial entities to join the consortium, especially if there is a fee involved.

As F5 is a CNA, and with myself being the primary point of contact with NVD, I certainly have thoughts and opinions. I've found NVD's CVMAP immensely frustrating. I think it is a waste of resources for NVD to recalculate CVSS and reevaluate CWE for CVEs where they have already been provided. Frankly, more often than not, I find that NVD is incorrect. They do not have all of the information that the CNA has, and deliberately so as it is common not to disclose all information about an issue to avoid giving malicious actors too much of a leg up. CNAs generally take great care to disclose enough information to illustrate the type of issue and its severity, while avoiding providing so much information that it becomes a roadmap for an exploit. CVMAP is deliberately designed to pressure CNAs into disclosing more information, potentially putting users are risk, as NVD will only consider public information in their evaluation. If a CNA disagrees with NVD's analysis, NVD's answer is invariably that the CNA should disclose more information in support of their argument, and NVD will then reanalyze the CVE.

That's bad enough, but, since humans are involved, there is also inconsistency. A CNA can create standardized language for types of exploits and publish multiple CVEs over time that share the same basics, using the same standardized language, only to have NVD analysts decide one should have a higher CVSS score, one lower, others are fine as is, and maybe a couple of different CWEs thrown in for good measure. This leads to CNAs writing Security Advisories and CVEs in a way tailored for NVD, and not for actual users. NVD runs with a small staff and simply can't be experts in all things covered by CVE, while CNAs work with a specific scope in which they are experts, especially vendor CNAs.

CWE has so many similar options that reasonable people can disagree on the best match - indeed more than one may be legitimate. A CNA could take the shotgun approach and provide a list of CWEs, hoping one matches with NVD, but that may not be the best approach for users looking for clarity. If a CNA were to deliberately downplay the severity of their issues, I believe the community would catch on and call them out. It is in a CNA's best interest to provide the best, most accurate information they can. I don't believe CVMAP has a positive impact on security, but it does result in frustration and more resources spent by CNAs trying to maintain status, and I've talked to a number of CNAs who feel this way. When I'm asked why NVD has a different CVSS score or CWE for an issue I tell people NVD's results need to be taken with a grain of salt as they do not have all of the information and will assume worst case.

Which brings me to a larger issue - the industry depends on NVD far too much. The fact that so many people are panicked about the slowdown at NVD just shows how dependent so many products and vendors are on one source. Monocultures are bad. If all a 'security vendor' is doing is bundling up NVDs data and putting a pretty interface on it, are they really adding value? Many CNAs already include CVSS and CWE in their CVEs, and we'd be better encouraging more CNAs to do so. There are other sources of enrichment as well - national agencies outside of the US, commercial vendors who do their own analysis, etc. This slowdown should be a wakeup call to the industry to start diversifying the ecosystem. It would be better in the end, even if NVD were to return to enriching all CVEs, to have diversity of opinions and analysis. NVD is hardly infallible.

Common Product Enumerators (CPEs) are not as commonly provided, and even NVD has indicated they're questioning CPE vs Package URLs (PURLs) or other systems. I think some of this will shake out of the world of SBOMs, VEX, and CSAF. Though today some CNAs do include CPE in their CVEs, but NVD is not set up to ingest this data and is providing their own. If I recall correctly from the presentation, CPE generation makes up ~60% of the workload for NVD. I think there would be far more value in NVD enriching CVEs that need it - CVEs that lack CVSS, CWE, or CPE. If you have limited resources, why waste them second-guessing the CNA and redoing their work? Work with the CVE program to encourage more CNAs to provide the data in CVEs and spend the resources filling the gaps.

Ideally, in my opinion, NVD would feed enriched data into the CVE program and could also stop spending resources on creating their own data feed, CVE website, etc. Everything would be in the CVE Record and the official CVE feeds and website. There is a way that could happen... but more on that in the future.

XZ: The Sleeper Awakens

Right at the end of my week, one of the biggest stories in years broke - the xz-utils backdoor into SSH. This is still an evolving situation, but this was a very near miss for the Internet community. This was a very sophisticated supply chain attack which would have, nigh-undetectably, compromised SSH on countless devices. The backdoor appears to allow an RCE on affected systems when specifically crafted packets are received by sshd. The vulnerability has been assigned CVE-2024-3094.

What's crazy is that the malicious code was caught purely by accident before it could make its way into major distributions, because of one person's attention to detail - Andres Freund. Andres was doing unrelated testing when he noticed a roughly 500ms delay in ssh.

With the backdoored liblzma installed, logins via ssh become a lot slower.

time ssh nonexistant@...alhost

before:
nonexistant@...alhost: Permission denied (publickey).

before:
real 0m0.299s
user 0m0.202s
sys 0m0.006s

after:
nonexistant@...alhost: Permission denied (publickey).

real 0m0.807s
user 0m0.202s
sys 0m0.006s

That's half a second, something very unlikely to be noticed during normal use of SSH. Something easily missed, or ignored if spotted as being inconsequential. But Andres pulled at that thread and unraveled everything. That's a very near miss.

My first thought when I heard the details, with the entity behind the code having spent a couple of years earning trust, is that this was likely a state actor. They'd have the resources to fund a long con, the patience to take things slowly to gain access and build trust, and sophistication to create the exploit. A sleeper agent that was finally called to action, only to have the plan stumble just before the finish line do to, really, dumb luck. And that seems to be the general consensus, though nothing has been proven at this point. It could possibly be a large criminal group, or even a truly dedicated solo actor. We may never know.

One thing this has done is made everyone a little more paranoid. If this was going on right out in the open without anyone noticing, how many other sleeper agents are active right now, diligently contributing to projects that underpin the Internet, just waiting for their call? Or how many have already received that call and delivered their payloads, which are running right now, undetected?

Rapid Roundup

This is getting long, so let's take a couple of items quickly. MOVEit came through my feed - again. I feel like almost every time I'm at the helm of TWIS some story on MOVEit crosses my feeds. CISA has posted a notice of proposed rulemaking for the Cyber Incident Reporting for Critical Infrastructure Act (CIRCIA). (That's a 447-page PDF, knock yourself out.) CIRCIA lays out reporting requirements for 'substantial cyber incidents' (72 hours) and ransomware payment (24 hours) for organizations considered critical infrastructure. Hardware attacks keep on coming. Back when Spectre and Meltdown hit I felt that was going to get more people interested, and it seems it has. The latest is GoFetch, a side-channel attack on Apple's M-series CPUs. The vulnerability exists in hardware and is being called unpatchable, though some mitigations may exist. But Apple wasn't the only chip-maker getting some bad news, AMD also got it on the fun. ZenHammer affects AMD Zen 2 & Zen 3 chips. As you may have guessed from the name, ZenHammer is a Rowhammer variant specific to this architecture.

Who Remembers OSI?

Something unrelated to security that popped up in my feeds this week is this old IEEE Spectrum article from 2013, OSI: The Internet That Wasn't. I'm guessing that a lot of people reading this don't really remember OSI, or maybe only know of it as 'The OSI Model' - that classic 'seven-layer' networking model into which TCP/IP doesn't really neatly fit. Which is, of course, because the model was for OSI, and not TCP/IP networks. (I don't know why we still reference the OSI model when the Internet has had its own, four layer, model since 1989 - RFC1122. It has always bugged me.)

When I first got online, in 1989, OSI was still a thing and there was still talk of OSI networks eventually replacing TCP/IP as a global standard. Of course, that never happened. This article was a bit of a trip down memory lane, but it is also a look into history for those who came of age after TCP/IP had conquered the world. It sent me down the rabbit hole reading about the protocol wars, remembering the active debates from my early years online.

As always, if this is your first TWIS, you can always read past editions. And there is a lot of other content from the F5 SIRT to check out as well. Until next time.

Published Apr 08, 2024

Version 1.0

f5 sirt

security

twis