External Vulnerability Scanning: Why It Isn't the Same as Continuous Assurance

‍

What is external vulnerability scanning?

External vulnerability scanning is an automated security check that examines an organisation's internet-facing assets, including servers, web applications, DNS records and email configuration, for exposures visible from outside the network.

It typically needs no credentials or internal access, so it approximates what an attacker can see without ever touching the organisation's internal systems.

A scan usually starts with discovery, mapping the domains, subdomains and IP addresses associated with an organisation before checking what each one exposes. From there it looks across several categories, covering open ports and the services listening on them, TLS certificate and configuration issues, HTTP security headers, DNS configuration, and email authentication controls such as SPF, DKIM and DMARC. Some tools also flag known vulnerabilities linked to software versions detected during the scan.

Scanners don’t have access inside the network, so they see what's publicly reachable, in the same way a search engine crawler or an attacker doing reconnaissance would. That's both the strength and the limit of the method, and it's the reason scanning needs to be read alongside other evidence rather than taken as a complete picture on its own.

The term gets used loosely alongside penetration testing, external attack surface management and security ratings. They overlap in what they look at, but they're built for different jobs. Scanning is automated and repeatable, a rating condenses signals into a score, and a penetration test involves a human trying to prove something can actually be exploited.

The NCSC treats scanning as one part of a wider vulnerability management practice that includes asset discovery, triage, remediation and verification, not a standalone activity.

‍

What external vulnerability scanning can tell you vs what it can't

External scanning shows what's technically observable at a point in time. It can't tell you whether that observation matters to your business, or whether the supplier has controls in place that make it a non-issue. Reading a scan result as a complete verdict on a supplier's security is the most common mistake buyers make with this data.

A scan can reliably surface things like exposed assets and open ports, misconfigured TLS or missing security headers, gaps in email authentication, and changes to the external attack surface over time. Each of these is a real, checkable fact.

What it can't establish is why a service is exposed, whether a compensating control sits behind it, how critical the affected system actually is to the supplier's operations, or what it means for your organisation specifically.

A scanner has no visibility into internal network controls, access management, incident response capability, or secure development practice, because none of that produces an external signal.

‍

Where scanning breaks down: attribution, false positives and severity

The most common reason a scan finding turns out to be wrong isn't a bug in the tool. It's attribution: the scanner has correctly identified a real technical condition, but on an asset that doesn't actually belong to, or isn't controlled by, the supplier you're assessing.

Shared hosting, content delivery networks, cloud infrastructure and formerly owned domains all make attribution harder than it looks. 

A supplier might have divested a business unit two years ago and still get flagged for a subdomain nobody at the company can log into. An agency might have registered a test environment that was never handed over. These things aren’t visible from outside, which is exactly why attribution needs a route for the supplier to confirm or dispute what's been discovered, not just a dashboard the customer reads alone.

Even where attribution is correct, two more issues open up:

• A scanner can misidentify the service running on a port, because version fingerprinting relies on banners that can be hidden, outdated or misleading, producing a false positive.
• And a finding's technical severity doesn't automatically tell you its business severity. A critical-rated exposure on an unused test system can matter less than a moderate misconfiguration on a service the supplier actually runs your data through.

The NCSC recommends assessing any scanning tool on both its false-positive and false-negative rates before relying on its output.

‍

External vulnerability scanning vs penetration testing

External vulnerability scanning and penetration testing answer different questions. Scanning checks a wide surface automatically and repeatedly, looking for known or observable weaknesses. Penetration testing puts a person in front of a narrower set of targets and asks whether those weaknesses, individually or chained together, can actually be exploited.

• A scan can run weekly or continuously and tell you a service is exposed, a certificate has expired, or a new subdomain has appeared, all without a human involved.
• A penetration test happens periodically, usually once or twice a year for a given scope, and it can show you that an exposed admin panel with a weak default password gives an attacker a path to internal systems, something a scanner alone would only flag as "admin interface detected."

Breadth versus depth is the trade-off: Scanning covers more ground more often, at the cost of not knowing whether a finding is actually reachable or exploitable in practice.

Testing proves exploitability on the systems it covers, at the cost of only covering a fraction of the estate and only at a point in time. Neither makes the other redundant. A supplier that passes an annual penetration test can still develop a new exposure the following month, which is exactly the gap scanning is suited to catching. A supplier with a clean scan history can still fail a penetration test if a chain of individually low-severity issues adds up to something exploitable, which is exactly what scanning alone won't show you.

For supplier assurance specifically, that means treating a clean scan and a passed penetration test as two different, non-substitutable pieces of evidence rather than either one standing in for the other.

‍

External vulnerability scanning vs attack surface management and security ratings

Two different comparisons tend to get bundled into this question, and so we’d recommend taking one at a time. Attack surface management is a broader process that scanning feeds into, whereas a security rating is a different output built on top of similar underlying data.

External Attack Surface Management (or EASM), typically covers discovering an organisation's externally exposed assets, attributing them correctly, classifying and monitoring them, identifying exposures, and tracking how the surface changes over time.

External vulnerability scanning is one component of that, the part that actually checks each discovered asset for weaknesses. A tool can do EASM without doing much scanning depth, and a scanner can run without doing full EASM-grade discovery. 

In practice the two overlap heavily, and vendors often bundle them, but the terms describe different scopes of work.

A security rating is a separate thing again. It takes selected external signals and compresses them into a single number, letter grade or ranking. Scanning produces the underlying observations; a rating is one way of summarising them. This can be useful for triage at scale, but it can obscure exactly what's driving the score, whether the attributed assets are correct, and whether the supplier has had a chance to add context. 

A rating isn't inherently wrong, but the risk is treating a summarised score as a complete, standalone judgement of supplier risk rather than a starting point for investigation.

‍

Building an effective scanning programme

A scanning programme works when every finding has an owner, a decision and a deadline, not when it produces the most findings. 

The programme needs a clear purpose, a defined scope, a way to validate what the scanner surfaces, and a triage method that turns a raw finding into an action rather than a permanent line on a dashboard.

Start with purpose and scope. Screening new suppliers, monitoring existing ones continuously, and validating specific assessment answers are different use cases, and conflating them tends to produce noise rather than signal. 

Decide which suppliers, domains and assets are actually in scope before turning the scanner on, and prioritise coverage by supplier criticality rather than treating every supplier the same way.

Triage comes next, which needs a small number of consistent categories as opposed to an open-ended backlog. The NCSC's approach uses three:

1. Fix, where the issue is remediated or mitigated
2. Acknowledge, where it's accepted for a documented reason and reviewed later
3. Investigate, which should be a temporary holding state while evidence is gathered, not a place findings go to be forgotten. 

Every finding should move through this quickly. A dashboard full of "investigating" six months on isn't a programme, it's a backlog.

You should also validate before you act. Confirm the asset actually belongs to the supplier, give them a route to dispute or add context, and treat a disputed finding as unresolved rather than silently correct or silently deleted.

Then close the loop: agree remediation with an owner and a date, and rescan to confirm the exposure is actually gone rather than just reported as fixed. CISA's guidance for federal systems, offered as a model for other organisations, recommends documenting mitigating actions and evidence-based review of any false-positive claim, rather than removing disputed findings without a record.

Track outcomes that reflect this, not vanity metrics. Time to validate, time to remediate, percentage of critical suppliers covered and recurrence rate tell you whether the programme is working. Total scans run or total findings created don't.

For regulated sectors, this isn't just good practice - it's increasingly a compliance expectation. DORA's Article 25 requires financial entities to test their ICT systems, including vulnerability scanning, as part of ongoing digital operational resilience. NCSC's CAF applies similar expectations to critical national infrastructure and public sector organisations. A scanning programme that can't show ownership, decisions and deadlines per finding isn't just weak practice - it's a harder story to tell an auditor or regulator.

‍

How we approach external monitoring at Risk Ledger

We built External Monitoring around a specific belief: a finding is only useful if it leads somewhere. That's why we don't present scan results as a standalone score, we treat them as a workflow, not a scoreboard.

‍

When something needs action, you can raise a discussion or a remediation directly from a finding, with the supplier in that conversation rather than finding out about an issue secondhand.

Our External Monitoring works this way because it isn't a standalone scanner with a dashboard. It's built into the Risk Ledger platform where suppliers already hold assessment evidence, answer questions, and connect with the other organisations assessing them, so a finding lands somewhere useful rather than in a separate tool nobody else can see.

A reviewer sees assessment answers and scan findings side by side, so checking a supplier's claim against what's externally visible doesn't mean exporting data between two systems. 

A supplier who has already verified their assets and added context for one customer starts the next customer's scan from evidence that's already been checked, not a blank slate, because that verification lives with the supplier's profile, not inside a single customer's account. 

And a remediation raised from a finding sits in the same discussion thread the supplier already uses for assessment queries, so nothing gets lost moving between tools.

The result isn't a bigger scanner - it's external evidence that gets more useful the more of the Risk Ledger network has already engaged with it, sitting inside a workflow that was built to get a finding fixed, not just recorded.

‍

_Sources

^{NCSC, Vulnerability scanning tools and services
NCSC, Carry out assessments - Vulnerability management guidance
CISA, CISA Insights: Remediate Vulnerabilities for Internet-Accessible Systems
NIST, SP 800-115: Technical Guide to Information Security Testing and Assessment
PCI Security Standards Council, PCI DSS v4.0.1 document library - Requirement 11.3.2.1}