The Vulnerability Management Warranty

We talk a great deal about vulnerability management and external attack surface management. These have become established categories with their own vendors and are now commonplace in the enterprise. But there is a question that almost no one asks directly. How do we know we are doing it right?

‍

Not how do we know we bought the right tool, checked the right box, or produced the right report? How do we know that the vulnerabilities that actually matter are being found? How do we know that the assets that actually exist are known? How do we know that the ordering of risk reflects reality rather than convenience or habit or the blind spots of the system itself?

‍

If you assume perfect execution, then perhaps this is all trivial, but even that mental exercise feels like fantasy. Imagine an organization that has fully enumerated every asset it owns, including the ones that are ephemeral and pop in and out of existence regularly. Okay, maybe there are a handful who have managed to do this well. Now imagine vulnerability scanners deployed everywhere they need to be with flawless coverage and zero misconfiguration. Getting harder to picture, right? The cost alone makes this impractical in many cases. Now, imagine a prioritization model that correctly understands exploitability, business impact, and the ephemeral nature of attacker behavior. This entire system remains only as accurate as these variables and the rate at which they change. Even if you believe such a thing is possible, it’s a stretch to believe it is common.

‍

More likely, it has never happened end-to-end in the way we like to imagine. Most organizations fail in at least one dimension and often in several at the same time. Assets are missed. Vulnerabilities are missed. Other vulnerabilities are found but ranked incorrectly. Teams respond rationally to the signals they are given, but the signals are often wrong.

‍

This creates a strange epistemic problem. We measure success by the absence of evidence of breach/loss rather than the presence of proof. Said another way, we believe nothing bad has happened yet, so we must be doing a good job. We imagine our coverage is complete because the system that is full of exceptions says so.

‍

But the system is grading its own homework.

‍

There is no external forcing function that compels VM or EASM to be correct.

‍

Now consider a different scenario. What if vulnerability management and external attack surface management were built around accountability rather than reporting? What if the vendor did not merely assert coverage but stood behind it with something that hurts when it fails? What if the product came with a warranty grounded in evidence?

‍

A warranty changes the geometry of incentives. Suddenly, unknown assets are not an abstract failure mode but a financial liability. Missed vulnerabilities are no longer theoretical gaps but measurable losses, and those losses end in a claim against the warranty. Mis-prioritization stops being a tuning issue and becomes an unacceptable outcome from an actuarial perspective. Payouts sting, and because they sting, they force rigor on behalf of the VM provider.

‍

To offer such a warranty, the vendor must collect real telemetry. You must know what you saw when you saw it and be honest with yourself about why you ranked it the way you did. You must be able to demonstrate that your picture of the environment corresponds to reality closely enough to bet money on it. This aligns the vendor with the customer in a way that beautiful dashboards never will.

‍

Warranties force a kind of intellectual honesty that security has long lacked. Warranties acknowledge that mistakes will happen, but by their nature, warranties refuse to let the vendor simply accept that those mistakes are the customer’s problem. Losses tied to assets we did not know about or vulnerabilities we did not find, or found and misunderstood, are not acts of nature; they are signals that the model is wrong. The model, therefore, must adjust and improve, getting closer to alignment with each passing day.

‍

If we want to know whether we are doing vulnerability management and external attack surface management correctly, we should stop asking the tools/vendors to tell us. We should ask them to prove it, with a warranty.

‍