Back to Blog

What Should an Advanced Cyber Security Assessment Actually Test?

By Jamie PopePublished 8 min readCyber Security
Documented. But tested? Cyber security assessment artwork contrasting a checked blueprint with a structure under security testing.

An advanced cyber security assessment should test the assumptions that matter to your business, using methods suited to the question. It should explain what was reviewed, what was challenged, what remains uncertain and which decisions or improvements follow.

“Advanced” is not a useful specification on its own. A long vulnerability report may reveal important weaknesses, but it does not automatically show whether a compromised account could disrupt a critical service, whether an alert would reach the right person or whether recovery would work.

Last week, we explored how to build useful ISO 27001 evidence across distributed teams. Evidence helps explain what a control is and how it operates. The next question is which conclusions need a more direct challenge.

Start with the business outcome you need to protect

Begin with a service, not a shopping list of testing tools.

That might be the ability to process orders, deliver care, access client records or pay suppliers. Identify the systems, identities, information and external dependencies that support it. Then agree the business consequences you want to understand.

A useful question is specific: “If one ordinary user account were compromised, what would prevent access to sensitive client information?”

A less useful brief is simply: “Test our security.”

The first gives the assessor an outcome to investigate. The second leaves room for a technically competent exercise that misses the question leadership thought it had commissioned.

Our recommended approach is to record the concern, the expected protection, the proposed assessment method and the decision the result should inform. This is a practical commissioning approach, not a mandatory template.

Distinguish assessment methods before buying them

Different methods provide different evidence. They can complement one another, but should not be sold or interpreted as interchangeable.

MethodUseful questionImportant limitation
Risk assessmentWhat could harm the business, and what should we prioritise?Analysis alone does not demonstrate a technical attack path
Configuration reviewDo selected settings and permissions match the intended design?A configuration snapshot does not establish every runtime outcome
Vulnerability scanningWhich detectable weaknesses appear within the scanned scope?Coverage, credentials and tool capabilities affect what can be found
Penetration testingCan specified security boundaries be breached under agreed conditions?Results are limited by scope, methods, time and the environment tested
Detection and response validationDoes an agreed test event produce the expected investigation and action?A successful scenario does not validate every possible attack
Recovery exerciseCan a selected service be restored to agreed operational expectations?Restoring a component does not necessarily restore the whole service

The NCSC’s penetration-testing guidance treats testing as an assurance activity that should improve ongoing vulnerability management. It is not a substitute for routine security work.

Its vulnerability-scanning guidance separately explains the range of scanning approaches. Ask what a proposed service actually includes, rather than relying on its label.

Turn control statements into testable questions

“We use multi-factor authentication” describes a control. It does not define the accounts covered, the exceptions or what happens to an already authenticated session.

“We have monitoring” describes a capability. It does not tell leadership whether an important alert will be understood and acted on.

For a scoped assessment, translate broad statements into questions such as:

  • Which relevant identities and privileged roles are included in the review?
  • Which routes into the selected service depend on suppliers or separate applications?
  • What should happen when a defined suspicious activity is simulated safely?
  • Who can authorise containment, and how is that person reached?
  • What evidence would demonstrate that a corrective change addressed the finding?

Not every question requires offensive testing. Some need a permissions review, an interview, a tabletop exercise or a controlled recovery test. Select the least disruptive method that can provide useful evidence for the question.

For Microsoft 365, for example, an assessment brief might include agreed identity settings, privileged access and selected sharing arrangements. The exact coverage depends on the tenant, licensing, service design and authorised access. A tenant review should not be described as a complete assessment of every connected application.

Scope the dependencies, not just the assets you own

A business service can cross several administrative boundaries.

Your team may manage devices while a provider manages identity, a software supplier operates the application and another organisation holds the recovery data.

Document which parties and environments are included. Separate exclusions from assumptions and identify who must approve access or testing.

A review of your own environment does not authorise testing a supplier’s infrastructure. It may instead identify questions to put to that supplier, evidence to request or a separately agreed exercise.

This is where co-managed IT responsibilities matter. Agree who supplies records, who supports the test window, who receives urgent findings and who owns changes afterwards.

A scope document should make missing coverage visible before anyone interprets the result.

Agree safe rules of engagement

Testing must have explicit written authorisation from the relevant owners, a defined scope and agreed boundaries.

Before work begins, document permitted activities, exclusions, dates, contacts, data-handling arrangements and stop conditions. Establish how urgent findings will be escalated and how the team will distinguish an authorised exercise from a real incident.

Production availability matters. Agree safeguards, relevant backup arrangements and any third-party permissions before potentially disruptive activity. Do not assume that a generic approval to assess security permits every technique.

For an exercise involving staff, agree the people, privacy and reporting arrangements in advance. The purpose should be to improve organisational resilience, not to publicly embarrass individuals.

A credible assessor should be willing to explain both what the exercise can show and what it will deliberately avoid.

Ask for findings your organisation can act on

A report should connect technical observations to the service and business concern in scope.

For each material finding, request:

  • the observation and supporting evidence;
  • affected systems and relevant dependencies;
  • demonstrated impact, separated from possible impact;
  • limitations and assumptions;
  • recommended treatment and an accountable owner;
  • the proposed way to verify completion.

A severity rating is useful input, not a complete business priority. Exposure, access, service importance, existing safeguards and the effort required to exploit a weakness may affect the response.

Avoid claiming that a plausible scenario was demonstrated when it was only discussed. Equally, do not interpret “not demonstrated” as “impossible”. Time limits or safety restrictions may have prevented a test.

Leadership needs that distinction to make a proportionate decision.

Include remediation and retesting in the plan

Do not spend the whole budget finding problems and leave no capacity to address them.

Before commissioning, agree how findings become work: who triages them, who approves service changes, who coordinates suppliers and how unresolved risks are escalated.

A completed change ticket records an action. Verification should establish whether the intended outcome now holds within the agreed scope.

Some fixes can be checked by a configuration review or a targeted rescan. Others need the original scenario repeated safely. If a change creates new dependencies, the verification scope may also need attention.

Record the retest date, method and result. Keep unresolved limitations visible.

Report assurance without promising certainty

A useful leadership summary answers five questions:

  1. What business concern did we assess?
  2. What did the assessment cover?
  3. What did it demonstrate?
  4. What remains unresolved or untested?
  5. What decision or action is required next?

A clean report is not a guarantee that the organisation is secure. Systems change, new weaknesses appear and any assessment has boundaries.

Likewise, technical testing does not replace an ISO 27001 management system, an internal audit or a certification decision. It can contribute evidence and inform risk treatment, but those activities have different purposes.

Commission a question, not just a report

Start with one important service and one assumption that leadership currently relies on. Define the scope and choose a suitable assessment method. Agree safe delivery, usable findings and a route to remediation.

Cloud Agile’s cyber security risk assessments connect business and technical scoping, control review and prioritised treatment. Where specialist testing is needed, agree its delivery arrangements and scope explicitly rather than assuming it is included in every assessment.

Find out which security assumptions deserve a closer look. Speak to Cloud Agile about scoping a cyber security assessment.

Frequently asked questions

What is an advanced cyber security assessment?

It is a description that needs defining in the proposal, not a universal package. Ask which risks, systems, methods and outcomes are included, and what remains outside scope.

Is a vulnerability scan the same as a penetration test?

No. Scanning automates checks for detectable weaknesses. Penetration testing uses an agreed approach to challenge security boundaries. Both have limitations and can support a wider assurance programme.

Can an assessment include Microsoft 365?

Yes, if the relevant tenant, identities, settings and connected services are explicitly scoped and authorised. Confirm the access needed and the limits of the review.

Does ISO 27001 certification remove the need for technical assessment?

Certification and technical assessment serve different purposes. Select testing according to relevant risks and requirements; do not assume a certificate demonstrates every technical control or attack scenario.

How often should we test?

Set a risk-based programme that reflects system changes, exposure and applicable obligations. Reconsider the scope after significant changes or incidents. There is no single frequency suitable for every organisation.

What should happen after the report?

Assign owners, agree treatment, manage changes and verify the result. Record accepted risks and unresolved limitations through the organisation’s authorised decision process.