Security Assessment and Testing Reference Guide
CISSP Domain 6 · 5 topics · Concepts, real-world examples, and controls
A security assessment strategy defines who tests what, how often, from what vantage point, and with what authorization. Without a deliberate strategy, testing is reactive and incomplete — covering only the systems someone thought to check, in the ways someone thought to check them. A mature strategy ensures coverage is systematic, findings are comparable over time, and the testing program reflects the organization’s actual risk profile.
Assessment perspectives
Internal assessment
Conducted by the organization’s own security team. Advantages: deep knowledge of the environment, continuous coverage, lower cost. Disadvantages: familiarity bias (testers know what controls exist and may unconsciously avoid finding problems), lack of independence reduces credibility with auditors, and resource constraints limit scope. Best for: continuous vulnerability scanning, log review, configuration audits.
External assessment
Conducted by an external party with no prior knowledge of the environment. Provides an attacker’s perspective — the tester discovers the environment the way a real adversary would. More credible for compliance and board reporting. Higher cost; limited time on target. Best for: annual penetration tests, pre-launch security assessments, and assessments that will be presented to regulators or the board.
Third-party assessment
Conducted by an independent party with no financial interest in the outcome and no relationship to either the organization or its vendors. Required for formal compliance certifications (PCI-DSS QSA, ISO 27001 certification body, SOC 2 CPA firm). Third-party independence is a legal and regulatory requirement in many frameworks — self-attestation is not sufficient.
Location dimensions
On-premises
Tests focus on physical infrastructure, network segmentation, on-premises Active Directory, server configurations, and local application security. Physical access testing (can a tester plug a device into an open ethernet port?) is possible. Full environment visibility with proper scoping.
Cloud environments
Cloud assessment requires explicit authorization from both the organization and the cloud provider (AWS, Azure, GCP all have penetration testing policies — unauthorized testing violates ToS and may trigger incident response). Focus: IAM misconfiguration, storage bucket permissions, network security group rules, serverless function security, logging coverage.
Hybrid environments
The most complex assessment scope. Must cover both on-premises and cloud components, plus the integration points between them — which are often where the highest-risk vulnerabilities exist. Requires testers with both on-premises infrastructure expertise and cloud-specific assessment skills. Hybrid identity synchronization paths are a critical test target.
Assessment strategy design — key decisions
Scope: What systems, applications, and environments are in scope? What is explicitly out of scope? Scope must be documented and agreed in writing — an out-of-scope system that a tester accidentally accesses creates legal and operational risk.
Frequency: Risk-based — higher-risk systems warrant more frequent testing. Penetration tests annually at minimum; continuous vulnerability scanning; real-time log monitoring. Frequency must also satisfy compliance requirements (PCI-DSS requires annual penetration testing).
Knowledge level: Black box (tester has no prior knowledge — simulates external attacker), grey box (tester has some knowledge — simulates insider or partially compromised environment), white box (tester has full knowledge of architecture, code, and credentials — most thorough coverage). Most enterprise tests are grey box.
Rules of engagement: Define what actions are permitted (can the tester attempt to exfiltrate data? Can they attempt to escalate to domain admin?), what systems are off-limits, what constitutes a stop condition, and how findings are reported and escalated during the test.
🌐 Real-world example — undefined scope
In 2019, a security researcher performing a contracted penetration test of a US retailer’s e-commerce platform discovered a server that appeared to belong to the retailer but was outside the defined scope. The server was hosting customer PII. The researcher faced a dilemma: report it (potentially exposing an unauthorized access to an out-of-scope system) or ignore it (leaving a genuine vulnerability undisclosed). The scope document had no process for handling out-of-scope discoveries. The organization had no process for receiving such reports. The researcher published a disclosure 90 days later after failing to reach the organization. A well-designed rules of engagement document would have defined a clear, safe process for reporting out-of-scope discoveries — protecting both the tester and the organization.
Conduct security control testing
Security control testing validates that controls are working as intended — not merely that they are configured. A firewall with a rule permitting all traffic between zones is technically present but functionally absent. Testing must be designed to detect both the absence of controls and the presence of controls that fail to achieve their security objective.
Red, blue, and purple team operations
Red team
Adversary simulation. The red team attempts to achieve specific objectives (exfiltrate data, compromise domain admin, access a specific system) using the full range of attacker techniques — phishing, exploitation, lateral movement, privilege escalation. Goal: find the paths a real attacker would take, not just identify vulnerabilities in isolation.
Best for: testing the organization’s overall security posture, detection capabilities, and incident response — not just finding individual vulnerabilities.
Blue team
Defense. The blue team operates, monitors, and defends the environment. During a red team exercise, the blue team’s goal is to detect, respond to, and contain the red team’s activities. Blue team performance metrics include: mean time to detect (MTTD), mean time to respond (MTTR), and whether specific attack techniques triggered alerts.
Best for: validating that detection and response capabilities work against realistic attack scenarios — not just synthetic alerts.
Purple team
Collaborative exercise where red and blue teams work together — red team shares TTPs (Tactics, Techniques, and Procedures) in real time while blue team tunes detection rules to catch them. Not adversarial — focused on maximizing learning and detection coverage. Requires psychological safety: blue team must be comfortable acknowledging detection failures.
Best for: rapidly improving detection coverage, building defender capabilities, and creating a library of validated detection rules based on real attack techniques.
🌐 Real-world example — red team value
In 2022, a financial services firm’s internal red team exercise revealed that 73% of their simulated attack scenarios went undetected by the blue team. The red team had achieved domain admin access within 4 hours of initial access and maintained persistence for 3 weeks without triggering a single alert. The vulnerability scan results and compliance reports had all been green. The red team exercise revealed what compliance checks cannot: the gap between “controls are present” and “controls are effective.” Post-exercise, the organization rebuilt its detection rule set, implemented EDR with behavioral detection, and re-ran the exercise 6 months later — achieving detection of 89% of the same scenarios.
Security control testing methods
Select a testing method to see its purpose, approach, and a real-world case where it mattered.
Vulnerability assessment
Identify and quantify weaknesses
Penetration testing
Exploit vulnerabilities to prove impact
Log reviews
Evidence of events in audit trails
Code review
Find vulnerabilities in source code
Breach attack simulation
Continuous automated adversary testing
Compliance checks
Validate against standards baselines
Interface testing
Test UI, API, and network interfaces
Misuse case testing
Test how the system handles abuse
Vulnerability assessment
A systematic process of identifying, quantifying, and prioritizing security vulnerabilities in systems, applications, and infrastructure. Vulnerability assessments use automated scanning tools (Nessus, Qualys, Rapid7 InsightVM) to identify known vulnerabilities, misconfigurations, and missing patches. Findings are categorized by severity (Critical, High, Medium, Low) using CVSS scores and contextualized by the environment’s specific risk profile.
Difference from penetration testing
Vulnerability assessment identifies what might be exploitable. Penetration testing proves it is exploitable and demonstrates the actual impact. Both are necessary — vulnerability assessments provide breadth and frequency; penetration tests provide depth and realistic impact assessment. A vulnerability assessment finding of “SQL injection possible” becomes a penetration test finding of “SQL injection exploited to extract 50,000 customer records.”
Real-world example
The 2017 Equifax breach: a public vulnerability scanner could have detected the unpatched Apache Struts vulnerability (CVE-2017-5638) on Equifax’s internet-facing application in minutes. The patch had been available for 2 months. Equifax’s vulnerability management process had failed to identify the affected application as in-scope for the patch. A continuous vulnerability scanning program with asset-complete coverage, tied to a patch management SLA, would have remediated the vulnerability before attackers found it.
Penetration testing
Authorized simulated attack on a system, network, or application to identify exploitable vulnerabilities and demonstrate real-world impact. Penetration testing goes beyond identifying vulnerabilities — it chains them together into realistic attack scenarios. A penetration tester who achieves remote code execution on a web server then pivots to demonstrate lateral movement, credential theft, and data exfiltration — simulating the full kill chain of a real attacker.
Testing approaches by knowledge level
Black box: Tester starts with no information — simulates external attacker. Finds most realistic external vulnerabilities but may miss internal issues. Grey box: Tester has some information (network diagrams, user credentials). Simulates insider threat or partially compromised environment. Most common enterprise approach. White box: Full access to source code, architecture, credentials. Most thorough — finds issues black/grey box would miss. Used for pre-launch application security reviews.
Real-world example
In 2020, a grey-box penetration test of a healthcare provider found that a developer’s test account — created for a demo and never deleted — had the same database permissions as a production admin account. The tester used the test account credentials (found in a public GitHub repository) to log into the production database and export 2 million patient records. The test account was technically in scope; the GitHub exposure was discovered during OSINT reconnaissance. The finding drove: decommissioning of all test accounts, mandatory secrets scanning on code repositories, and database permission reviews.
Log reviews
Systematic examination of log data from systems, applications, networks, and security tools to identify security events, policy violations, anomalous behavior, and indicators of compromise. Log reviews serve both detective and forensic functions — detecting incidents as they occur and reconstructing events after the fact. The quality of log reviews depends entirely on the quality and completeness of logging: logs that don’t capture relevant events cannot be reviewed for those events.
What to review
Authentication events (failed logins, after-hours logins, new device logins), privilege escalation events, configuration changes, data access patterns (bulk downloads, access to unusual data), network traffic anomalies, application errors, and security tool alerts. SIEM (Security Information and Event Management) tools automate correlation and alerting — but the rules must be maintained and the alerts must be triaged.
Real-world example
The 2023 Microsoft/Storm-0558 breach was discovered when a US government agency noticed anomalous access to Microsoft 365 email in their audit logs — specifically, a service account accessing email mailboxes it had never accessed before. The agency had purchased Microsoft’s highest-tier logging license, which included the audit events necessary to detect the anomaly. Agencies on lower-tier licenses lacked those log events entirely. After the incident, Microsoft expanded access to security logs at no additional cost following congressional pressure. The incident demonstrated that log coverage is a prerequisite for detection — you cannot detect what you cannot log.
Code review and testing
Examination of application source code, configuration, and build artifacts to identify security vulnerabilities, insecure coding patterns, and compliance issues. Code review can be manual (security engineer reviews code line by line), automated (SAST — Static Application Security Testing tools scan code for patterns), or a combination. DAST (Dynamic Application Security Testing) tests the running application rather than the source code — effective at finding issues that only manifest at runtime.
SAST vs DAST vs SCA
SAST — analyzes source code, bytecode, or binaries without executing. Finds issues early in the pipeline. High false positive rate — requires tuning. DAST — tests the running application from the outside. Finds runtime issues (authentication flaws, injection vulnerabilities, session management issues) that SAST misses. SCA (Software Composition Analysis) — identifies vulnerable third-party libraries and open-source components. Addresses supply chain risk at the code level. Essential given that 80%+ of modern application code is open-source dependencies.
Real-world example
The Log4Shell vulnerability (CVE-2021-44228) affected virtually every Java application that included the Log4j 2 logging library — estimated at billions of applications globally. Organizations that had SCA integrated into their build pipelines could generate a complete inventory of affected applications within hours of disclosure. Organizations without SCA spent weeks manually searching code repositories, deployment artifacts, and vendor software lists to determine their exposure. SCA converted a multi-week emergency into a hours-long inventory exercise.
Breach and attack simulation (BAS)
Automated, continuous simulation of attacker techniques against production controls — validating that security tools (EDR, SIEM, email security, firewall) detect and block specific attack techniques as expected. BAS platforms (Cymulate, AttackIQ, SafeBreach) run simulations aligned to the MITRE ATT&CK framework, generating findings like “T1059 PowerShell execution was not blocked by EDR on 3 of 15 endpoints” or “T1566 Phishing email with malicious attachment bypassed email security controls.”
Advantage over point-in-time testing
BAS runs continuously — detecting when a control configuration change, software update, or new deployment breaks a previously working detection. A penetration test validates security at a point in time; BAS validates it continuously. An EDR update that silently disables a detection rule will be caught by BAS within hours; it might not be discovered until the next annual penetration test without BAS.
Real-world example
A global bank implemented BAS and discovered in its first week that a recent Microsoft Defender for Endpoint update had accidentally disabled AMSI (Antimalware Scan Interface) integration — meaning PowerShell-based malware was no longer being inspected. The BAS platform had run a PowerShell-based simulation the day after the update and immediately flagged the detection gap. The organization remediated within 24 hours. Without BAS, the gap would have been invisible until the next penetration test — 11 months later.
Compliance checks
Automated or manual verification that systems and configurations meet the requirements of applicable security standards, regulatory frameworks, and organizational policies. Compliance checks validate that controls are configured correctly — they do not test whether those controls are effective. CIS Benchmarks, DISA STIGs (Security Technical Implementation Guides), and cloud security benchmarks (CIS AWS Foundations, CIS Azure Foundations) define specific configuration requirements that compliance checks validate.
Critical limitation
Compliance does not equal security. A system that passes all CIS Benchmark checks is properly configured by a defined standard — but may still be vulnerable to zero-day exploits, logic flaws in applications, or attacks that the benchmark does not address. Compliance checks provide a floor — a minimum baseline — not a ceiling. Target was PCI-DSS compliant at the time of their 2013 breach. Healthcare.gov was compliant at launch and still had security vulnerabilities.
Real-world example
After the 2016 Bangladesh Bank SWIFT heist ($81M stolen), SWIFT’s Customer Security Programme (CSP) introduced mandatory compliance attestation for all SWIFT users — requiring self-attestation against 16 mandatory security controls. By 2021, SWIFT made third-party verification mandatory for Tier 1 users after discovering that some organizations were self-attesting compliance without implementing the controls. The Bangladesh Bank had been using an unsecured SWIFT terminal — a compliance check against the CSP mandatory controls would have flagged this configuration before the attack.
Interface testing
Security testing of application interfaces — user interfaces (UI), application programming interfaces (APIs), and network interfaces. Each interface type presents distinct attack surfaces and requires distinct testing techniques. Interface testing validates that authentication, authorization, input validation, output encoding, and error handling are correctly implemented at every interface — not just the primary user-facing pathway.
API security testing focus
OWASP API Security Top 10 provides the testing checklist: Broken Object Level Authorization (BOLA), Broken Authentication, Broken Object Property Level Authorization, Unrestricted Resource Consumption, Broken Function Level Authorization, Unrestricted Access to Sensitive Business Flows, Server-Side Request Forgery (SSRF), Security Misconfiguration, Improper Inventory Management, Unsafe Consumption of APIs. BOLA (testing whether API returns objects the caller shouldn’t have access to) is the most critical and most commonly missed.
Real-world example
In 2022, Peloton’s API was found to expose private account data — including age, weight, workout history, city, and gender — for any Peloton user, regardless of whether their account was set to private. The API returned user profile data based solely on user ID, with no check on whether the requesting account was authorized to view that profile. A researcher discovered the issue during routine API testing and notified Peloton, who took 90 days to remediate. The endpoint had passed functional testing but had never been tested for authorization correctness — a gap that API security testing specifically addresses.
Misuse case testing
Testing that evaluates how a system behaves when users act in ways that violate intended use — whether accidentally or maliciously. While use cases describe intended behavior, misuse cases describe attacks and abuse scenarios. Misuse case testing asks: what happens when a user submits 10,000 requests per second? What happens when they enter SQL code in a form field? What happens when they manipulate a session token? What happens when they attempt to access resources they shouldn’t? What happens when they provide an input 100x larger than expected?
Relationship to threat modeling
Misuse cases are the testing implementation of threat modeling output. The threat model identifies “an attacker might attempt SQL injection through the login form”; the misuse case tests “what actually happens when SQL injection is attempted through the login form?” Threat modeling without misuse case testing produces untested security assumptions.
Real-world example
In 2011, Sony’s PlayStation Network was breached via SQL injection — an attacker submitted SQL commands in a form field that the application passed directly to the database. The attack was not sophisticated; the SQL injection vulnerability is one of the most well-known and easily tested web application vulnerabilities. A misuse case test that submitted SQL metacharacters in every input field would have discovered the vulnerability before the attackers did. The breach exposed 77 million accounts, cost Sony $171 million in direct costs, and resulted in a 24-day network outage. Misuse case testing of authentication and profile inputs would have cost an afternoon.
Collect security process data
Security process data provides the quantitative evidence that security controls are operating effectively — or reveals where they are degrading. Without systematic data collection, security management is opinion. With it, security performance becomes measurable, improvable, and demonstrable to leadership, auditors, and regulators.
Key performance and risk indicators
KPIs (Key Performance Indicators) measure how well security processes are executing. KRIs (Key Risk Indicators) measure the level of risk the organization is exposed to. Both are necessary — KPIs without KRIs tell you how fast you’re running without telling you whether you’re running in the right direction.
Mean time to detect
MTTD
Average time from incident occurrence to detection. Industry benchmark: under 24 hours for high-severity incidents. IBM/Ponemon 2023 report: global average 204 days.
Mean time to respond
MTTR
Average time from detection to containment. IBM/Ponemon 2023 report: global average 73 days to contain a breach after detection.
Patch compliance rate
PCR
Percentage of systems patched within SLA (e.g., critical CVEs within 72 hours). Target: 95%+ within SLA. Below 90% indicates systemic patch management failure.
Phishing click rate
PCR
Percentage of employees who click simulated phishing links. Baseline: 30–40% without training. Target: below 5% with mature awareness program. Tracks training effectiveness over time.
Vulnerability backlog
VBL
Count of open critical/high vulnerabilities by age. A growing backlog indicates remediation cannot keep pace with discovery. Tracked per system and per team.
Access review completion
ARC
Percentage of scheduled access reviews completed on time. Target: 100%. Below 90% indicates access governance is failing and access accumulation is occurring undetected.
Data categories and what they tell you
Account management data
Privileged account count, dormant account count (no login in 90+ days), orphaned account count (no associated active employee), accounts with excessive permissions. Tracking these over time reveals whether the IAM lifecycle process is functioning or accumulating debt.
Backup verification data
Backup completion rate, restore test results, recovery time achieved in last test, time since last successful restore test. A backup that has never been tested is an untested hypothesis. Organizations discover backup failures during incidents — not before.
Training and awareness data
Training completion rates by department and role, phishing simulation click and report rates, policy acknowledgment rates, repeat offender counts (employees who click phishing simulations multiple times). Measures behavior change, not just training consumption.
DR and BC data
Last test date and result for each critical process, RTO/RPO achieved vs. target in last test, gaps identified in last test and remediation status, tabletop vs. functional vs. full simulation exercise cadence. An untested DR plan is documentation, not preparedness.
Management review data
Exceptions approved by leadership (number, age, risk level), security decisions escalated but not yet resolved, open audit findings by age and severity. Management review data reveals whether leadership is making security decisions or simply deferring them.
🌐 Real-world example — backup data failure
In 2019, the City of Baltimore was hit by ransomware that encrypted government systems. Recovery took weeks because backup restoration testing had never been performed — when the city attempted to restore from backups, they discovered that several critical systems had never been successfully backed up and others had corrupted backup files. The city ultimately spent $18 million in recovery costs. A monthly backup verification process — running restore tests for critical systems and recording the results — is a KPI that would have surfaced the backup failures months before the ransomware attack made them catastrophic.
Analyze test output and generate report
A security test that produces findings but no actionable report has accomplished half its purpose. The output of every assessment must be analyzed, contextualized, prioritized, and communicated in a form that drives remediation. The report is not the end of the testing process — it is the beginning of the remediation process.
Finding severity and prioritization
| Severity | CVSS range | Definition | Remediation SLA | Example |
|---|---|---|---|---|
| Critical | 9.0–10.0 | Exploitable remotely with no authentication, full system compromise, or immediate, catastrophic business impact. | 24–72 hours | Unauthenticated remote code execution on an internet-facing server. Exploitable SQL injection that accesses the entire customer database. |
| High | 7.0–8.9 | Significant exposure — may require authentication or specific conditions but results in major data exposure or privilege escalation. | 7–14 days | Authenticated SQL injection. Privilege escalation from standard user to administrator. Sensitive data exposure via misconfigured API. |
| Medium | 4.0–6.9 | Moderate risk — limited impact or requires complex conditions for exploitation. Often requires chaining with other vulnerabilities. | 30 days | Reflected XSS requiring user interaction. Missing security headers. Verbose error messages disclosing system information. |
| Low | 0.1–3.9 | Minimal direct risk — informational findings, best-practice deviations, or vulnerabilities with very limited exploitability. | 90 days | Missing HSTS header. TLS version 1.1 supported in addition to 1.2/1.3. Non-sensitive information disclosure in responses. |
| Informational | 0.0 | Observations, best-practice recommendations, or positive findings that document existing effective controls. | Next planning cycle | Software version disclosure in HTTP headers. Security header present but set to a non-optimal value. Positive observation: MFA enforced on all admin accounts. |
Remediation, exception handling, and ethical disclosure
Remediation
The finding owner (system or application owner) receives findings with a remediation deadline. Remediation must be verified — the tester or a separate team confirms the fix is effective before the finding is closed. Remediation verification prevents “fixed” findings that only address the symptom while the root cause remains exploitable.
Exception handling
When a finding cannot be remediated within the SLA, a formal risk exception is submitted: the business justification, the compensating controls (if any), the risk owner’s acceptance signature, and an expiry date for review. An undocumented deviation from the remediation SLA is not an exception — it is an unmanaged risk.
Ethical disclosure
When a security researcher discovers a vulnerability in an organization’s systems, coordinated (responsible) disclosure gives the organization time to remediate before public release. Standard disclosure timeline: 90 days (Google Project Zero standard). The researcher notifies the vendor privately, allows time to fix, and publishes afterward — whether or not the vendor has remediated.
What makes a good security report
Executive summary: 1–2 pages. Business impact framing. No technical jargon. Risk posture compared to previous assessment. Three key findings and their business consequences. Trend line (improving/stable/deteriorating).
Technical findings: Each finding includes: title, severity, CVSS score, affected systems, description of the vulnerability, evidence (screenshot, request/response), business impact, and specific remediation steps with verification criteria.
Risk context: Findings are prioritized not just by CVSS score but by the organization’s actual risk profile. A medium CVSS finding on a payment processing server that handles customer card data may warrant higher priority than a high CVSS finding on an isolated internal development server.
Remediation roadmap: Prioritized list of findings with owner assignments, target remediation dates, and dependencies. Not a dump of raw data — an actionable work plan.
🌐 Real-world example — disclosure gone wrong
In 2016, a security researcher discovered a vulnerability in Uber’s systems that exposed driver data and submitted a report through Uber’s bug bounty program. Uber’s security team acknowledged the report — but rather than remediating it, Uber’s CSO paid the researcher $100,000 through the bug bounty program and demanded they sign an NDA and delete the data. Uber did not notify affected drivers or regulators for over a year. When the incident was discovered in 2017, Uber was fined $148 million by the FTC and state AGs. The CSO was later convicted of obstruction of justice. The incident established the legal and ethical standard: security vulnerabilities that result in data breaches must be disclosed to regulators and affected individuals, regardless of how the organization obtained the disclosure.
Conduct or facilitate security audits
Security audits provide an independent, systematic evaluation of an organization’s security controls, processes, and policies against a defined standard or requirement. Audits differ from assessments in their formality, independence, and purpose: audits produce formal findings with compliance determinations; assessments produce security recommendations. Both are necessary — assessments find problems; audits provide assurance that standards are met.
Audit types by perspective
Internal audit
Conducted by the organization’s internal audit function — independent from the security team but within the organization. Provides assurance to the board and audit committee that controls are operating as designed. Internal auditors follow IIA (Institute of Internal Auditors) standards. Cannot provide external credibility to regulators, customers, or partners.
Best for: ongoing compliance monitoring, control gap identification, pre-external-audit preparation, board reporting. Internal auditors know the business but may not have specialized security expertise — security consultants are often brought in to support.
External audit
Conducted by an independent external firm with no financial relationship to the organization beyond the audit engagement. Provides credibility to external stakeholders — regulators, customers, investors, and partners trust external audit findings because the auditor has no incentive to produce favorable results.
Best for: annual compliance audits required by regulation or contract, pre-IPO security reviews, customer security assessments, regulatory examinations. External auditors operate under professional standards (AICPA, ISACA) that carry legal and professional liability.
Third-party audit
An audit conducted by a party with no connection to either the organization or its primary business relationships — typically a certification body or standards organization auditor. Third-party audits produce certifications (ISO 27001, SOC 2 Type II) that represent independent assurance to the market as a whole, not just to a specific customer or regulator.
SOC 2 Type II (CPA firm attestation of cloud service providers’ security controls), ISO 27001 certification (accredited certification body), PCI-DSS QSA assessment (Qualified Security Assessor certified by the PCI SSC). Each produces a formal report or certificate with defined validity period.
Common audit frameworks and their outputs
| Framework | Who performs it | Output | Audience | Real-world context |
|---|---|---|---|---|
| SOC 2 Type I / II | Licensed CPA firm | Attestation report on Trust Services Criteria (security, availability, confidentiality, processing integrity, privacy) | Enterprise customers requiring assurance of cloud/SaaS vendor controls | Type I: controls are suitably designed (point in time). Type II: controls operated effectively over a period (6–12 months). Type II is the gold standard for enterprise SaaS procurement due diligence. |
| ISO 27001 | Accredited certification body (BSI, Bureau Veritas, etc.) | Certificate of compliance with ISMS standard, valid 3 years with annual surveillance audits | International enterprise customers, regulated industries, government procurement | Required for many EU government contracts and preferred by multinational enterprises. The certification audit includes document review, process interviews, and technical evidence review against all 93 Annex A controls. |
| PCI-DSS QSA | Qualified Security Assessor (certified by PCI SSC) | Report on Compliance (ROC) or Attestation of Compliance (AOC) | Card brands (Visa, Mastercard) and acquiring banks | Required for Level 1 merchants (6M+ card transactions/year) and service providers. QSA audits are the most prescriptive — each of the 12 requirements has defined testing procedures the QSA must execute. |
| FedRAMP | Third Party Assessment Organization (3PAO) | Authorization to Operate (ATO), reusable by all federal agencies | US federal agencies; required for cloud services to the US government | 3PAO assessment tests approximately 325 NIST SP 800-53 controls at FISMA Moderate baseline. ATO from the JAB (Joint Authorization Board) is reusable government-wide — eliminating duplicate assessments per agency. |
| HIPAA audit | HHS Office for Civil Rights (OCR) or qualified third party | Audit findings report; potential corrective action plan (CAP) and civil money penalty | HHS regulators; required for covered entities and business associates | OCR audits cover administrative, physical, and technical safeguards. Breach notification requirements trigger reactive OCR investigations. The investigation focuses on whether a compliant risk analysis was performed — not whether the breach was preventable in hindsight. |
Audit process — what to expect
🌐 Real-world example — audit finding ignored
In 2016, the US Office of Personnel Management (OPM) breach — which exposed the security clearance records of 22 million federal employees — was preceded by an OIG (Office of Inspector General) audit finding in 2014 that specifically identified inadequate two-factor authentication for remote access and inadequate system inventory as significant weaknesses. The finding was documented. Management acknowledged it. Remediation was planned. It was never completed. The 2015 breach exploited the exact attack path the 2014 audit had identified. Audit findings that are acknowledged but not remediated provide false assurance — the finding is “addressed” administratively while the vulnerability remains technically. Remediation tracking with evidence verification is not bureaucracy; it is the only thing that converts audit findings into security improvement.
The security assessment and testing chain
Every gap in this chain means a vulnerability exists, persists, or recurs without being detected — until an attacker finds it first.
Define strategy before testing begins
Scope, frequency, knowledge level, rules of engagement — all documented and approved before any test starts. Undefined scope creates legal risk for testers and incomplete coverage for the organization.
Test controls at multiple layers
Vulnerability assessment for breadth. Penetration testing for depth. BAS for continuity. Red team for realism. No single method provides complete coverage — they are complementary.
Test detection, not just prevention
Red/blue/purple team exercises validate whether security events are being detected and responded to — not just whether perimeter controls would have blocked the attack.
Collect process data continuously
MTTD, MTTR, patch compliance, phishing rates, backup test results, access review completion. Security posture is measurable — measure it.
Report findings with business context
Technical findings translated into business impact. Executive summary for leadership. Prioritized remediation roadmap. Risk context, not just CVSS scores.
Remediate, verify, and track to closure
Findings are not closed when the fix is deployed — they are closed when the fix is verified. Exceptions require documented approval and expiry dates.
Conduct formal audits for independent assurance
Internal audits for governance. External/third-party audits for compliance, customer trust, and regulatory requirements. Audit findings not remediated provide false assurance.
By profession, a CloudSecurity Consultant; by passion, a storyteller. Through SunExplains, I explain security in simple, relatable terms — connecting technology, trust, and everyday life.
Leave a Reply